Strange behaviour since 02_06_02 (Wh vs kWh not matching anymore in InfluxDB) - explained

[jsimon]

Hi,

First of all, thanks for making this great product and giving continued support! It’s been working great and reliably. I also had a lot of fun setting up InfluxDB and Grafana to do all sorts of cool graphs.

Since the last update happened on my device around 21h on April 28th, I have errors in the data being sent to InfluxDB using the kWh outputs. For some reason, the kWh data is not matching the Wh data anymore.

This is the data for my Mains in Wh when using iotawatt.local/query?..

2021-04-24T00:00:00, 32462.31
2021-04-25T00:00:00, 46925.94
2021-04-26T00:00:00, 39972.35
2021-04-27T00:00:00, 46493.86
2021-04-28T00:00:00, 42064.07
2021-04-29T00:00:00, 36774.98
2021-04-30T00:00:00, 36390.33
2021-05-01T00:00:00, 43766.22

This is from Grafana querying InfluxDB for my Mains in kWh…

2021-04-24 00:00:00, 32.461644
2021-04-25 00:00:00, 46.924601
2021-04-26 00:00:00, 39.973394
2021-04-27 00:00:00, 46.496023
2021-04-28 00:00:00, 42.228679
2021-04-29 00:00:00, 37.515000
2021-04-30 00:00:00, 37.879000
2021-05-01 00:00:00, 44.236000

As you can see, starting on 2021-04-28 and even more obvious from 2021-04-29, the numbers have a pretty noticeable difference. Also, when adding my numbers in kWh in Grafana, it does it match the Mains since the update (the totals exceed the Mains values by a significant margin).

BTW, I also switched to InfluxDBv2 yesterday and that worked fine, but is unrelated to the problem above. Switching to the Flux syntax in Grafana seems to be a PITA, so for now I added user accounts in InfluxDBv2 to be able to keep using InfluxQL.

Keep up the good work!

[overeasy]

I’ll look into that, but can you more specific about the methodology. Can you post the IoTaWatt setup for influx and tell me how you derive kWh in influx/grafana - do you upload Watts and use integral or do you upload Wh and add them up?

[jsimon]

I have included screenshots of both Grafana and the InfluxDB v1 uploader settings. This was working fine before the update on April 28th. I noticed that if I switch to uploading “Wh” instead of “kWh”, the data seems good.

One workaround would be to switch everything to “Wh” and reupload from my installation date on 2020-12-01, but it takes a long time to reupload all that data keeping the 5s resolution (around 1 second to upload about 100 seconds of data).

Thanks.

[overeasy]

One Kw load uses 0.0013888 kWh in 5 seconds. You are seeing variation at three significant digits. When you upload Wh, it’s 1.3888, so 3 decimal places is more significant.

02_06 has significant reworking of the code that runs scripts. I’ll look at what changed, but uploading kWh to 4 or 5 decimal places is typically not appropriate. For higher resolution using Wh is the way to go.

Yeah uploading A year would take a long time. You could run a query to convert your kWh history to Wh, and resume with Wh.

[jsimon]

It does seem that it was previously sending kWh with more decimals before the update. Starting on 2021-04-29, we see only zeros after the 3rd decimal. When looking at data from my other inputs instead of the Mains, its the same thing. I can see a pretty big difference up to 30% between Wh and kWh data.

Unmetered data from Wh…

2021-05-05 00:00:00 15120.727642
2021-05-06 00:00:00 13008.832214
2021-05-07 00:00:00 10500.290597

Unmetered data from kWh…

2021-05-05 00:00:00 19.595000
2021-05-06 00:00:00 17.222000
2021-05-07 00:00:00 13.680000

Thanks for the suggestion to convert to using Wh without reuploading everything.