Hello,
I believe I may have found a limitation of IoTaWatt’s InfluxDb code. I attempted to create a 62nd measurement for InfluxDB. As I am trying to access all the data (Watts, Amps, kwh, wh, VAR, etc) for each CT input I have. Which is a lot of measurements.
When I add the 62nd measurement I get this error in the IoTaWatt logs when trying to stop/start the InfluxDb Service on IoTaWatt:
8/09/20 22:00:01 influxDB: Json parse failed.
8/09/20 22:00:01 influxService: Invalid configuration.
I believe I tracked it down to this line of code (IoTaWatt/Firmware/IotaWatt/influxDB.cpp at aa3474c974f08ecdc3a85857a68a10819c2cba1b · boblemaire/IoTaWatt · GitHub), but without more verbose error logging I’m not really sure what is causing it.
I also double checked to make sure the config.txt JSON file was valid by running it through a JSON validator. So my only guess is a limitation in the code for more than 62 measurements in IoTaWatt?
There is a limit to resources in the ESP8266. How it handles each unique situation is a function of the demand placed on other resources. The goal is to have it handle a reasonable load and fail without crashing.
I wouldn’t classify 60 measurements as a reasonable load. There is no practical difference between Wh and KWh. They are the same measurement. While VAR and VARh are useful as parameters to billing on the mains measurement, I don’t see any value in that data for branch circuits. PF will give an indication of loads with high VAR.
As a practical matter, one frame of 60 measurements could be 4-6K of data, which pushes other limits.
I’d recommend you scale back your schema, and if you are going to have a lot of measurements, limit the length of any names used and the number of keys associated with each measurement.
If you really need all that data, an alternative is to use a second IoTaWatt and divide the load between them.
I was afraid of that answer! 
I was attempting to send all the units of each CT (and I only have 8 CT’s so far on my first IoTaWatt). I definitely plan to get more IoTaWatt’s but I need to use all the CT inputs to maximize on measurements/cost ratio (also physical space).
My thought was just send all the raw data to Influx DB. Then if I need it I can query on it per CT. But it looks like that wont work now. Also my key names are a bit long… (didn’t know there was a limitation with that as well)
Honestly i’m not sure I will need all of these units. Some I never heard of (like VAR & VARh) until I bought the IoTaWatt. But regardless figured I would just log the data into Influx, if I wanted to use it later. I heard of PF & VA but still not quite sure what they are either.
Are all of these values calculated just purely from the Amps and Volts measured by IoTaWatt? Maybe I can derive these values myself through mathematical queries in InfluxDB, if I needed them?
Heap memory is a scarce shared resource. Those names must be stored there. If average length is say 24 characters, there is an 8 byte overhead per heap allocation, so 32 bytes per x 60 = 1,920 for the names alone. That’s about 10% of typical available heap. Overall, each of those measurements is probably closer to 80 bytes so roughly 25% of available heap. Now you need large buffers to send that data, so not a good situation.
It’s not like this data isn’t available. You can get it direct from the IoTaWatt if you need to know.
IoTaWatt stores cumulative Wh and cumulative VAh for each input. Additionally, it uses the reference to the appropriate voltage channel and the cumulative measurement hours associated with each measurement interval. All of the metrics are derived from that.
As I said, kWh = Wh / 1000, so saving both is redundant. You can integrate Watts in influx Query to get Wh. Same goes for integrating VAR to get VARh. VAR is relatively new and is really only useful for billing in commercial tariffs.
I only upload Watts and Voltage.
Thanks @overeasy!
How does this look now?
I still have 6 unused CT inputs (they may have 3 measurements per input) but it comes down to roughly 46 measurements TOTAL vs. the ~120 I was going to have per IoTaWatt.
If it’s working, looks good. Couple of points though:
It looks as if you have a split-phase system with two mains. For purposes of monitoring, most folks are interested in the combined power in, generated, used, etc. The calculator/script system can be used to add the two phases together and upload as a single measurement. That saves you from adding them in influx Queries and also reduces the volume and complexity of upload and storage.
Util_Amps Amps = UtilityMain_PhaseA + UtilityMain_PhaseB
That cuts the whole thing in half. You will appreciate that if you try to upload a bunch of history or if there is a communication or influxDB server outage that causes a large backlog to upload. It will also simplify your queries immensely because you won’t need to query two values and add them together to get the total.
Update: Also, it looks as if you have solar generation. If you are interested in your consumption, as opposed to the net, influx is a good way to do that. It can’t be done in the IoTaWatt because it requires integrating the individual intervals. Assuming your mains go negative when exporting, your usage would be:
Usage Watts = UtilityMain_PhaseA + UtilityMain_PhaseB + SolarGeneration_Phase_A + SolarGeneration_PhaseB
@overeasy Thanks, yeah I did think of that. I was contemplating if I should combine at IoTaWatt or in Grafana InfluxDB queries. I ultimately decided to simply combine at the InfluxDB side (atleast what I thought was simple at the time). And just have IoTaWatt just send it’s raw data over. That way I had all the options available to me when trying to visualize the data, including per phase visualizations. Versus combining them at IoTaWatt and losing those details/options.
Well that being the case, rather than send the two individual sides, I’d recommend sending the total and one individual side. If you want the remaining individual side, you can subtract to get it. That way, for the same amount of data, you get the totals in a convenient form and if you find the individuals are not useful going forward, you can just drop them from the database.
1 Like
@overeasy Another question, If I go down the route you suggested, does it make sense to combine everything? For example when looking at say PF do you normally look at that per phase or combined (phase A+B)? Same goes for other types of units? I know with Watts you normally look at those combined unless your trying to monitor the unbalance per phase.
I suppose it depends on why you are interested in PF in the first place. I mostly use it to determine reasonability of measurements in diagnosing polyphase service issues, so discrete measures are more useful and the combined doesn’t really help me.
The formula for PF is Watts / VA. When you add inputs together in the calculator and select PF units out, the calculator will divide the sum of the Watts by the sum of the VA. If you are uploading Watts and VA, you can easily compute PF, so why bother uploading. BTW if you are not uploading VA, I would say it’s more useful than PF and enables you to compute PF anyway.
@overeasy I am uploading VA and PF (I may not upload PF now if the formula for that is that simple though).
I am primarily uploading more on my mains than on my other inputs. But I do have a relatively unique input, a Whole House Generator. Trying to do research on what data would be useful to graph to monitor the status of the generator more than just say Watts/Amps. And that’s hard as it normally only runs when we have outages 
Also regarding VA for the mains, Would that be useful in seeing how much load I am putting on the Utility’s Transformer? As I know they spec out transformers by the kVA. For example our transformer is a 25kVA. Does that mean if I see 6kVA on one of our 120v phase’s that I am using 6kVA of the 25kVA for one of the transformer’s 120v phase’s?
I’m going to get myself in trouble if I go too deeply into these topics. Providing the metrics is one thing, but pretending to be a power engineer is another. I’m a programmer by trade. That said…
If I had a whole-house generator with a transfer switch that fed through the mains, I would probably just be interested in VA through the mains when the generator is running. Plotting voltage with VA would probably make any generation period unique enough to identify.
25kVA is a little over 100A per leg. That’s a lot of power. Others over at the OEM site have written about how transformers are sized for the practical load and are typically less than the potential maximum load. Utilities don’t want overheated transformers, to be replacing burnt out transformers, or to service tripped fuses. Power engineers seem to do a pretty good job of sizing transformers for their expected loads. I don’t worry about my community transformer that services 6 households. If it starts to get overloaded and cause problems, the utility will replace it with a bigger one.
1 Like
