Doubling IotaWatt Data

OK, wasn’t expecting the unit to have been running uneventfully for the last 17 days. That does appear to be the end of the log. I’ll get back to you tomorrow. Thanks.
Can you post your config.txt in a private message please?

I’m at a complete loss to understand what has happened here. Some of my quandaries:

This isn’t just a switch to 5 second entries. The first such entry at 3:55:05 appears to have a 5 second value, but the subsequent 5 second entries have 10 second values. SO, I think there may be two different things happening here. The switch to uploading on 5, 15, 25, etc. with the correct value for the respective 10 second interval that overlays a separate previous (or later?) upload of true 10 second entries on even 10 second intervals.

When I look at the message log for that period I see that you were on the older MAJOR release 02_06_06 which predates the uploader rewrite that appears in 02_07_05. There was a WiFi disconnect on 12/07/21 00:20:42 (remember that) followed by a connect then another disconnect/connect. The log is then uneventful until 10 days later on 12/12/21 when a routine six-week restart happens. The influx uploader is started, but apparently cannot query influx for the ending time. It sits that way for 8 days until 12/25/21 when it seems to retry successfully and obtains a starting time of 12/07/21 00:19:40 - about a minute before that disconnect back on 12/07.

Now it gets interesting. The uploader is stopped 9 days later on 01/03/22 and the upload is current. A minute and a half later it is restarted and the upload date is 1/3/21 19:24:30. That would probably be the start date of your current log as it holds about 1 year. So either the database was wiped, or the measurements were changed and there were no matching previous measurements in the database. There are a few stops and restarts ending with a stop on 1/3/22 at 19:41 indicating the last post was 1/3/21 at 20:03:30.

Coincidentally, release 02_07_05 was made the MINOR release an hour later and it was installed. Upon restart, influx shows as stopped with entry on 1/3/21 at 20:45:20 as would be expected.

The next day (actually later that day after midnight, influx was started and it picked up from 1/3/22 at 19:22:10 - a year later.27 days later on 1/31/22 it was stopped, was current and then chugged until Valentines day when there was another six-week routine restart where influx started normally and up to date.

SD initialized.
12/17/21 13:28:35z Real Time Clock is running. Unix time 1639747715 
12/17/21 13:28:35z Reset reason: Software/System restart
12/17/21 13:28:35z Trace:  1:3, 1:3, 1:3, 1:3, 1:3, 1:3, 1:3, 1:1[3], 1:2[4], 9:0[4], 9:0, 9:1, 8:4, 8:6, 8:8, 8:9, 9:3, 9:5, 9:9, 1:2, 1:3, 1:3, 1:6[1], 1:6[2], 1:6[2], 1:6[2], 1:6[2], 1:6[3], 1:5[20], 1:6[4], 20:0, 20:1
12/17/21 13:28:35z ESP8266 ID: 6195317, RTC PCF8523
12/17/21 13:28:35z IoTaWatt 5.0, Firmware version 02_06_06
12/17/21 13:28:35z SPIFFS mounted.
12/17/21 08:28:35 Local time zone: -5:00, using DST/BST when in effect.
12/17/21 08:28:35 device name: IotaWatt
12/17/21 08:28:35 HTTP server started
12/17/21 08:28:35 influxDB_v1: Starting, interval:10, url:http://192.168.1.64:8086
12/17/21 08:28:35 timeSync: service started.
12/17/21 08:28:35 statService: started.
12/17/21 08:28:35 dataLog: service started.
12/17/21 08:28:37 dataLog: Last log entry 12/17/21 08:28:30
12/17/21 08:28:40 historyLog: service started.
12/17/21 08:28:40 historyLog: Last log entry 12/17/21 08:28:00
12/17/21 08:28:40 WiFi connected. SSID=ATT6AbiV3A, IP=192.168.1.100, channel=6, RSSI -58db
12/17/21 08:28:40 MDNS responder started for hostname IotaWatt
12/17/21 08:28:40 LLMNR responder started for hostname IotaWatt
12/17/21 08:28:40 Updater: service started. Auto-update class is MINOR
12/17/21 08:29:13 Updater: Auto-update is current for class MINOR.
12/21/21 19:55:34 Table file parse failed.
12/21/21 20:55:40 Updater: update tables from invalid to 0.0.4
12/25/21 20:45:29 influxDB_v1: Start posting at 12/07/21 00:19:40
1/03/22 19:22:44 influxDB_v1: stopped, Last post 01/03/22 19:22:00
1/03/22 19:24:20 influxDB_v1: Start posting at 01/03/21 19:24:30
1/03/22 19:32:58 influxDB_v1: stopped, Last post 01/04/21 19:40:20
1/03/22 19:40:57 influxDB_v1: Start posting at 01/03/21 19:41:10
1/03/22 19:41:07 influxDB_v1: stopped, Last post 01/03/21 20:03:30
1/03/22 20:44:44 Updater: Update from 02_06_06 to 02_07_05
1/03/22 20:44:44 Updater: download 02_07_05
1/03/22 20:44:51 Updater: Release downloaded 6934ms, size 951104
1/03/22 20:44:57 Updater: signature verified
1/03/22 20:45:05 Updater: firmware upgraded to version 02_07_05
1/03/22 20:45:05 Updater: Firmware updated, restarting.

** Restart **

SD initialized.
1/04/22 01:45:14z Real Time Clock is running. Unix time 1641260714 
1/04/22 01:45:14z Reset reason: Software/System restart
1/04/22 01:45:14z Trace:  18:2, 18:3, 18:2, 18:3, 18:2, 18:3, 18:2, 18:3, 18:4, 18:5, 1:6[6], 1:1[7], 1:2[8], 9:0[8], 9:0, 9:1, 8:4, 8:6, 8:8, 8:9, 9:3, 9:5, 9:9, 1:2, 1:3, 1:3, 1:6[1], 1:6[3], 1:5[5], 1:6[4], 5:0, 5:7
1/04/22 01:45:14z ESP8266 ID: 6195317, RTC PCF8523 (68)
1/04/22 01:45:14z IoTaWatt 5.0, Firmware version 02_07_05
1/04/22 01:45:14z Updater: Installing update files for version 02_07_05
1/04/22 01:45:14z Updater: Installing GRAPH.HTM
1/04/22 01:45:14z Updater: Installing GRAPH.JS
1/04/22 01:45:14z Updater: Installing GRAPH2.HTM
1/04/22 01:45:14z Updater: Installing GRAPH2.JS
1/04/22 01:45:14z Updater: Installing INDEX.HTM
1/04/22 01:45:16z Updater: Installing TABLES.TXT
1/04/22 01:45:16z Updater: Installing CNFSTYLE.CSS
1/04/22 01:45:16z Updater: Installing EDIT.HTM
1/04/22 01:45:16z Updater: Installing FAVICON.ICO
1/04/22 01:45:17z Updater: Installation complete.
1/04/22 01:45:17z SPIFFS mounted.
1/03/22 20:45:17 Local time zone: -5:00, using DST/BST when in effect.
1/03/22 20:45:17 device name: IotaWatt
1/03/22 20:45:17 HTTP server started
1/03/22 20:45:17 influxDB_v1: Starting, interval:10, url:http://192.168.1.169:8086
1/03/22 20:45:17 timeSync: service started.
1/03/22 20:45:17 statService: started.
1/03/22 20:45:17 dataLog: service started.
1/03/22 20:45:19 dataLog: Last log entry 01/03/22 20:44:50
1/03/22 20:45:22 historyLog: service started.
1/03/22 20:45:22 historyLog: Last log entry 01/03/22 20:44:00
1/03/22 20:45:22 WiFi connected. SSID=ATT6AbiV3A, IP=192.168.1.100, channel=6, RSSI -71db
1/03/22 20:45:22 MDNS responder started for hostname IotaWatt
1/03/22 20:45:22 LLMNR responder started for hostname IotaWatt
1/03/22 20:45:22 Updater: service started. Auto-update class is MINOR
1/03/22 20:45:24 Updater: Auto-update is current for class MINOR.
1/03/22 20:45:24 Updater: update tables from 0.0.4 to 0.0.5
1/03/22 20:45:24 influxDB_v1: stopped, Last post 01/03/21 20:45:20
1/03/22 21:45:29 Updater: update tables from 0.0.4 to 0.0.5
1/03/22 22:45:35 Updater: update tables from 0.0.4 to 0.0.5
1/03/22 23:45:41 Updater: update tables from 0.0.4 to 0.0.5
1/04/22 00:45:47 Updater: update tables from 0.0.4 to 0.0.5
1/04/22 00:49:34 influxDB_v1: Start posting at 01/03/22 19:22:10
1/04/22 01:45:53 Updater: update tables from 0.0.4 to 0.0.5
1/04/22 02:45:59 Updater: update tables from 0.0.4 to 0.0.5
1/04/22 03:46:05 Updater: update tables from 0.0.4 to 0.0.5
1/04/22 04:46:11 Updater: update tables from 0.0.4 to 0.0.5
1/04/22 05:46:17 Updater: update tables from 0.0.4 to 0.0.5
1/04/22 06:46:23 Updater: update tables from 0.0.4 to 0.0.5
1/04/22 07:46:29 Updater: update tables from 0.0.4 to 0.0.5
1/04/22 08:46:34 Updater: update tables from 0.0.4 to 0.0.5
1/04/22 09:46:40 Updater: update tables from 0.0.4 to 0.0.5
1/04/22 10:46:46 Updater: update tables from 0.0.4 to 0.0.5
1/31/22 18:56:53 influxDB_v1: stopped, Last post 01/31/22 18:56:10
1/31/22 19:01:06 influxDB_v1: Start posting at 01/31/22 18:56:20
2/12/22 16:58:47 influxDB_v1: stopped, Last post 02/12/22 16:58:20
2/12/22 17:38:39 influxDB_v1: Start posting at 02/12/22 16:58:30
2/14/22 20:47:25 timeSync: Six week routine restart.

** Restart **

SD initialized.
2/15/22 01:47:26z Real Time Clock is running. Unix time 1644889646 
2/15/22 01:47:26z Reset reason: Software/System restart
2/15/22 01:47:26z Trace:  1:3, 1:1[14], 1:2, 9:0, 9:0, 8:4, 8:6, 8:8, 8:9, 1:2, 1:3, 1:3, 1:6[1], 1:6[2], 1:6[2], 1:6[2], 1:6[2], 1:6[3], 1:5[19], 1:6[4], 1:6[6], 1:3, 1:3, 1:6[1], 1:6[2], 1:6[2], 1:6[2], 1:6[3], 1:5[20], 1:6[4], 20:0, 20:1
2/15/22 01:47:26z ESP8266 ID: 6195317, RTC PCF8523 (68)
2/15/22 01:47:26z IoTaWatt 5.0, Firmware version 02_07_05
2/15/22 01:47:26z SPIFFS mounted.
2/14/22 20:47:26 Local time zone: -5:00, using DST/BST when in effect.
2/14/22 20:47:26 device name: IotaWatt
2/14/22 20:47:26 HTTP server started
2/14/22 20:47:26 influxDB_v1: Starting, interval:10, url:http://192.168.1.169:8086
2/14/22 20:47:26 timeSync: service started.
2/14/22 20:47:26 statService: started.
2/14/22 20:47:26 dataLog: service started.
2/14/22 20:47:28 dataLog: Last log entry 02/14/22 20:47:25
2/14/22 20:47:31 historyLog: service started.
2/14/22 20:47:31 historyLog: Last log entry 02/14/22 20:47:00
2/14/22 20:47:31 WiFi connected. SSID=ATT6AbiV3A, IP=192.168.1.100, channel=6, RSSI -66db
2/14/22 20:47:31 MDNS responder started for hostname IotaWatt
2/14/22 20:47:31 LLMNR responder started for hostname IotaWatt
2/14/22 20:47:31 Updater: service started. Auto-update class is MINOR
2/14/22 20:47:32 influxDB_v1: Start posting at 02/14/22 20:46:40
2/14/22 20:47:34 Updater: Auto-update is current for class MINOR.
2/15/22 00:51:12 WiFi disconnected.
2/15/22 00:52:35 WiFi connected. SSID=ATT6AbiV3A, IP=192.168.1.100, channel=6, RSSI -67db
3/19/22 00:04:32 Updater: Invalid response from server. HTTPcode: -11
3/28/22 21:49:30 timeSync: Six week routine restart.

So that’s one story where part of it seems to missing like why the upload reverted back a year and then came back to reality. Was it that the database was changed or cleaned, or the upload measurements were changed (that would include changing tags) so that the existing data did not match the new uploads?

Regardless, there doesn’t appear to be any 5 second resolution data being uploaded at that time, or any other time. I’m at a loss to understand how this is happening. In your last post you had stopped the upload with a time of 12/29/22 23:03:05, then restarted at 12/29/22 at 23:03:10. Does it progress to 20, 30 etc or is it 15, 25 etc? When you look at the upload status display, is the time advancing as 15, 25… or 10, 20…?

I just did a query on the database for the last 10 records and they are showing 0 10 20 30 40 50. I just stopped and started the uploader to make sure the log was working. The only other thing I can think of that happened during this period (besides the other post I referenced above) was I had fiber installed at the house on Jul 28 2022. They put in a new router that screwed up a few IP addresses which I was able to fix, so I didn’t really think anything of it. As far as the status display, the last update on the uploaders is only changing every min and ends with :10. The Current data log time is changing every 5 sec.

select Wh from iotawatt order by time DESC limit 10
name: iotawatt
time . . . . . . . . . . . . . . . .Wh
----- . . . . . . . . . . . . . . . . ----
2022-12-30T21:00:10Z 7.047486
2022-12-30T21:00:00Z 7.0082503
2022-12-30T20:59:50Z 6.9645745
2022-12-30T20:59:40Z 6.9627852
2022-12-30T20:59:30Z 6.9774065
2022-12-30T20:59:20Z 6.9541957
2022-12-30T20:59:10Z 6.9655686
2022-12-30T20:59:00Z 6.9538705
2022-12-30T20:58:50Z 6.948476
2022-12-30T20:58:40Z 6.998385

I still question why the data seemed to be ok last month when I recorded the usage and it agreed with the electric company. However, when I noticed this month’s data was off and went back to last month, it was off as well. It would seem to me that whatever happened occurred sometime between (about) 17 Nov 2022, which is when last month’s bill was mailed and 24 Dec 2022 which is when I received this month’s bill and found the problem.

I’m convinced the basic problem is that the database was uploaded to twice. Once with time stamps that end in 0 and once with time stamps ending with 5. I have no idea how that happens, never saw it before with interval 10, yet your unit was doing it as recently as a few days ago.

It could be something that’s been there but most users have interval 5. I will set some of my test units to interval 10.

You might consider switching to interval 5 to preclude this happening again.

Can I do this:

Stop the updater
Delete all of the 2022 data in InfluxDB
Change the Upload History From date to 31 Dec 2021
Change the interval to 5
Restart the updater

Would that at least get all of the 2022 data back in sync?

Maybe. First, look at the start date of your current log. You should specify a date that is at least one day greater than that. The five minute intervals will probably take weeks to upload a year’s worth of data. You previously had problems with uploads stopping, suggesting 5hat there may be damage to your current log.

Might be better if you can figure a way to cleanup the data that you have. I