Influxdb update slower on one vs the other

@frogmore Are you able to share your influx config? I’m also looking to set that up?

Look here for a start:

Well, the plot thickens. I noticed that the uploading progress stalled again. Sure enough Grafana showed an annomoly at 11:10AM on 10/30. So I stopped the sending (which had restarted back at the beginning of the datalog and not at the point that I told it to start at). I set the start date a few days ahead and the upload is continuing. I ran some queries and this is what I see in the data:

> select * from iotawatt where Watts < 0
name: iotawatt
time                Amps Volts Watts                    channel   device
----                ---- ----- -----                    -------   ------
1592858730000000000            -2636                    Barn      IotaWatt
1592858730000000000            -1.6535469126360393e+51  Utility3  IotaWatt
1594426420000000000            -43.18                   HPWH_A    IotaWatt
1594426420000000000            -70208482881.11          Utility1  IotaWatt
1599354900000000000            -5.760780923401527e+164  Family    IotaWatB
1599354900000000000            -1.894185814057058e+280  k15       IotaWatB
1600551490000000000            -649.18                  Bed23     IotaWatt
1600551490000000000            -93103324898650900000    k13       IotaWatt
1600551490000000000            -29.48                   WH_Well_A IotaWatt
1600551490000000000            -63.59                   Utility3  IotaWatt
1600551490000000000            -29.95                   RangeVent IotaWatt
1600551490000000000            -23.32                   HPWH_A    IotaWatt
1600551490000000000            -639.95                  Dryer_A   IotaWatt
1602377000000000000            -5.439967369313442e+164  Family    IotaWatB
1602377000000000000            -1.7887000316273969e+280 k15       IotaWatB
1604081460000000000            -188951526.75            MB_Floor  IotaWatt
1604081470000000000            -1.3228952450799069e+51  Utility3  IotaWatt
1604081470000000000            -2093.83                 Barn      IotaWatt
> select * from iotawatt where Watts > 10000
name: iotawatt
time                Amps  Volts Watts                  channel  device
----                ----  ----- -----                  -------  ------
1592858730000000000             5.621438149643385e+227 Dryer_A  IotaWatt
1594426420000000000             779295162956.35        MB_Floor IotaWatt
1594426420000000000             1.025749877095174e+114 k13      IotaWatt
1594426420000000000             27401210191.42         Washer   IotaWatt
1599354900000000000             1.2710192031453963e+73 BarnB    IotaWatB
1602377000000000000             1.2002371002851973e+73 BarnB    IotaWatB
1604081460000000000             117286850979.92        Dryer_A  IotaWatt
1604081460000000000 4.603       23813.22               MainA    IotaWatt
1604081460000000000             8.991434343761321e+85  HPWH_A   IotaWatt
1604081470000000000             4.49734672892881e+227  Dryer_A  IotaWatt
1604081470000000000             1.6003708857643472e+79 HPWH_A   IotaWatt

IotaWatt is LegA and IotaWatB (8 char limit on the name in IotaWatt) is LegB. This time it shows some clearly incorrect values from IotaWatB. This one has never given me a problem. There are two timestamps that have an issue on Leg B:
1599354900000000000 Saturday, September 5, 2020 6:15:00 PM [GMT-07:00] DST
1602377000000000000 Saturday, October 10, 2020 5:43:20 PM [GMT-07:00] DST

So, it looks like there is something going on. The Sept 5 one is a little before the 6 week automatic restart, but that is probably just a coincidence. The Oct 10 one does have anything interesting around that time, but the next day it did update the FW.

I think I will be done uploading all the data to influxdb in the next day or so. After that, I will rearrange things and after that delete the data and history logs on both devices. I am seeing the following issues:

  1. Bad data or bad measurements are either being written to the datalog, or the datalog is getting corrupted.
  2. When uploading data to influxdb, if it hits bad data, it restarts the upload process
  3. When it restarts the upload process it restarts from the beginning of the datalog, and not the specified start date
  4. It doesn’t log anything in the message log when it restarts the upload process

It seems the error does not hit all channels at the same time. It is possible that there are more errors. I just searched for obviously wrong values. I don’t have solar, so I left all the channels at the default (allow negative power is unchecked). This means all the negative values are definitely wrong.

Many of the timestamps occurred with nothing interesting around them, so not clear what caused them. Could be a bug, but I doubt this one is going to be easy to figure out.

This is a little unsettling, but I don’t there is enough here to support your conclusions. I have some questions that I’d like to resolve.

What version of influx are you using? The reason I ask is that I don’t know how I could post -5.760780923401527e+164 if I wanted to. Maybe C++ is taking liberties with printf, but I’ll need to see that.

I’m starting to suspect there may be a bug in the datalog search algorithm that is returning the wrong data. It’s definitely a possibility, although this would be the first incident noticed. There is no index for the datalog. It is chronological and so can be searched pretty efficiently. But perhaps there is a boundary condition…

The restarting at the beginning is a separate issue. That’s why I’m asking about the influx version. I’m wondering if there is a translation between the V1 query and flux, and if that might be working improperly because a flux query is being created with too short a look-back window.

There’s a lot going on here, and with a newer version of influx, it may be we’re looking at compound issues. I definitely want to look at the IoTaWatt data from the other machine. I’ll post the date/times and what I’d like to see.

I agree that it is unsettling and I am committed to helping figure out what happened. So I will do what I can to get you the information that you need. I haven’t looked at the data structures used in the log so can’t really explain what might have happened. I also agree that there is more than one issue here, which makes it complicated to figure out what is going on.

So, first question. I am using Influx 1.8.3,

I am running this on Ubuntu 20.04 in an LXC (container)

The datalog is showing strange numbers in the graphs for the sample points I looked at. But, tell me what you want to see and I will check it out.

Good news, IotaWatB completed the catch up and got current yesterday. IotaWatt got current sometime in the evening and now I have lots of data in influxdb.