InfluxDb issues


#41

Sounds like you have your issues under control. Good luck. For the time clause you could make it relative with
where time > now() - 2d


#42

Hi Since last update 02_03_17 one of the 3 iotawatt has an issue with influxDb.

I tried multiple things:

  1. edit the webServer and set no date
  2. edit the server and set the 23.10.2018 (day where the update occurred)
  3. trigger reset…

None of those helped it looks like having problems to recover last entry.
SOmetimes it is hable to start but from july and it stops after a while…

Log:
** Restart **

SD initialized.
10/25/18 04:39:23z Real Time Clock is running. Unix time 1540442363
10/25/18 04:39:23z Version 02_03_17
10/25/18 04:39:23z Reset reason: Software/System restart
10/25/18 04:39:23z Trace: 18:3, 18:2, 18:3, 18:2, 18:3, 18:2, 18:3, 18:2, 18:3, 18:2, 18:3, 18:2, 18:3, 18:4, 18:5, 1:6, 1:1[10], 1:2[11], 9:0[11], 9:0, 9:1, 8:4, 8:6, 8:8, 8:9, 9:3, 9:5, 9:9, 1:2, 1:3, 1:4, 1:5[21]
10/25/18 04:39:23z ESP8266 ChipID: 6910350
10/25/18 04:39:23z SPIFFS mounted.
10/25/18 06:39:24 Local time zone: 2
10/25/18 06:39:24 device name: pinguIW1, version: 3
10/25/18 06:39:27 MDNS responder started for hostname pinguIW1
10/25/18 06:39:27 LLMNR responder started for hostname pinguIW1
10/25/18 06:39:27 HTTP server started
10/25/18 06:39:27 timeSync: service started.
10/25/18 06:39:27 statService: started.
10/25/18 06:39:27 WiFi connected. SSID=pingu, IP=192.168.18.73, channel=1, RSSI -74db
10/25/18 06:39:28 Updater: service started. Auto-update class is ALPHA
10/25/18 06:39:28 dataLog: service started.
10/25/18 06:39:28 dataLog: Last log entry 10/25/18 06:39:20
10/25/18 06:39:28 historyLog: service started.
10/25/18 06:39:29 historyLog: Last log entry 10/25/18 06:39:00
10/25/18 06:39:30 Updater: Auto-update is current for class ALPHA.
10/25/18 06:39:32 EmonService: started. url:80=emoncms.org, node=pinguITW, interval=10
10/25/18 06:39:32 influxDB: started, url=192.168.18.10:8086, db=itw02, interval=5
10/25/18 06:39:33 EmonService: Start posting at 10/24/18 16:13:20
10/25/18 06:39:33 EmonService: Stopped. Last post 10/24/18 16:13:10
10/25/18 06:39:39 influxDB: last entry query failed: -11
10/25/18 06:39:39 influxDB: Stopped. Last post 2/6/6 08:28:16


#43

Updated Log:
** Restart **

SD initialized.
10/25/18 06:22:27z Real Time Clock is running. Unix time 1540448547
10/25/18 06:22:27z Version 02_03_17
10/25/18 06:22:27z Reset reason: Software/System restart
10/25/18 06:22:27z Trace: 1:3, 1:4, 1:3, 1:4, 1:3, 1:4, 1:3, 1:4, 1:3, 1:4, 1:3, 1:4, 1:3, 1:4, 1:3, 1:4, 1:1, 1:2[1], 9:0[1], 9:0, 9:1, 8:4, 8:6, 8:8, 8:9, 9:3, 9:5, 9:9, 1:2, 1:3, 10:2, 10:3
10/25/18 06:22:27z ESP8266 ChipID: 6910350
10/25/18 06:22:27z SPIFFS mounted.
10/25/18 08:22:28 Local time zone: 2
10/25/18 08:22:28 device name: pinguIW1, version: 3
10/25/18 08:22:31 MDNS responder started for hostname pinguIW1
10/25/18 08:22:31 LLMNR responder started for hostname pinguIW1
10/25/18 08:22:31 HTTP server started
10/25/18 08:22:31 timeSync: service started.
10/25/18 08:22:31 statService: started.
10/25/18 08:22:31 WiFi connected. SSID=pingu, IP=192.168.18.73, channel=1, RSSI -74db
10/25/18 08:22:31 Updater: service started. Auto-update class is ALPHA
10/25/18 08:22:31 dataLog: service started.
10/25/18 08:22:32 dataLog: Last log entry 10/25/18 08:22:25
10/25/18 08:22:32 historyLog: service started.
10/25/18 08:22:32 historyLog: Last log entry 10/25/18 08:22:00
10/25/18 08:22:34 Updater: Auto-update is current for class ALPHA.
10/25/18 08:22:36 EmonService: started. url:80=emoncms.org, node=pinguITW, interval=10
10/25/18 08:22:36 influxDB: started, url=192.168.18.10:8086, db=itw02, interval=5
10/25/18 08:22:37 EmonService: Start posting at 10/24/18 16:13:20
10/25/18 08:22:37 EmonService: Stopped. Last post 10/24/18 16:13:10
10/25/18 08:22:42 influxDB: last entry query failed: -11
10/25/18 08:22:42 influxDB: Stopped. Last post 2/6/6 08:28:16
10/25/18 08:22:47 influxDB: started, url=192.168.18.10:8086, db=itw02, interval=5
10/25/18 08:22:54 influxDB: last entry query failed: -11
10/25/18 08:22:54 influxDB: Stopped. Last post 2/6/6 08:28:16
10/25/18 08:22:56 influxDB: started, url=192.168.18.10:8086, db=itw02, interval=5
10/25/18 08:23:02 influxDB: Start posting from 10/23/18 21:51:00
10/25/18 08:23:02 influxDB: Stopped. Last post 10/23/18 21:50:55
10/25/18 08:23:19 influxDB: started, url=192.168.18.10:8086, db=itw02, interval=5
10/25/18 08:23:21 influxDB: Start posting from 10/23/18 21:51:00
10/25/18 08:23:21 influxDB: Stopped. Last post 10/23/18 21:50:55
10/25/18 08:23:26 influxDB: started, url=192.168.18.10:8086, db=itw02, interval=5
10/25/18 08:23:27 influxDB: Start posting from 10/23/18 21:51:00
10/25/18 08:42:05 Incomplete HTTP request detected, id 7, restarting.

** Restart **

SD initialized.
10/25/18 06:42:07z Real Time Clock is running. Unix time 1540449727
10/25/18 06:42:07z Version 02_03_17
10/25/18 06:42:07z Reset reason: Software/System restart
10/25/18 06:42:07z Trace: 18:2, 18:3, 18:2, 18:3, 18:2, 18:3, 18:2, 18:3, 18:2, 18:3, 18:2, 18:3, 18:2, 18:3, 18:2, 18:3, 18:2, 18:3, 18:2, 18:3, 18:2, 18:3, 18:2, 18:3, 18:2, 18:3, 18:4, 18:5, 1:6, 1:3, 1:4, 1:5[21]
10/25/18 06:42:07z ESP8266 ChipID: 6910350
10/25/18 06:42:07z SPIFFS mounted.
10/25/18 08:42:08 Local time zone: 2
10/25/18 08:42:08 device name: pinguIW1, version: 3
10/25/18 08:42:11 Connecting with WiFiManager.
10/25/18 08:42:15 MDNS responder started for hostname pinguIW1
10/25/18 08:42:15 LLMNR responder started for hostname pinguIW1
10/25/18 08:42:15 HTTP server started
10/25/18 08:42:15 timeSync: service started.
10/25/18 08:42:16 statService: started.
10/25/18 08:42:16 WiFi connected. SSID=pingu, IP=192.168.18.73, channel=1, RSSI -74db
10/25/18 08:42:16 Updater: service started. Auto-update class is ALPHA
10/25/18 08:42:16 dataLog: service started.
10/25/18 08:42:17 dataLog: Last log entry 10/25/18 08:42:05
10/25/18 08:42:17 historyLog: service started.
10/25/18 08:42:17 historyLog: Last log entry 10/25/18 08:42:00
10/25/18 08:42:19 Updater: Auto-update is current for class ALPHA.
10/25/18 08:42:20 EmonService: started. url:80=emoncms.org, node=pinguITW, interval=10
10/25/18 08:42:20 influxDB: started, url=192.168.18.10:8086, db=itw02, interval=5
10/25/18 08:42:21 EmonService: Start posting at 10/24/18 16:13:20
10/25/18 08:42:21 EmonService: Stopped. Last post 10/24/18 16:13:10
10/25/18 08:42:27 influxDB: last entry query failed: -11
10/25/18 08:42:27 influxDB: Stopped. Last post 2/6/6 08:28:16


#44

There appear to be two issues here. Timeout of the last post query, and failure to post with the service stopping. I can’t see anything that changed in this release that would cause either problem, so I’d like more information and try something.

First, could you post the log portion from prior to the update to 17 until after the update. That would be at least three full restart cycles.

Next, could you change to auto-update MINOR (02_03_16) it should change right away, and post the log from the update and after.

If influx works fine on that, could you go back to ALPHA and post the logs again?


#45

With version 02_03_16 it is able to recover (even if I have to make some start and stop)
There the 2 logs
20181025_Log1.txt (68.3 KB)
20181025_Log2.txt (9.7 KB)


#46

It does look as though the posting stopped with the upgrade to 02_03_16, although nothing changed in that area, there are no other reports of influx failing after upgrade, and my systems took it in stride.

One thing unique about your usage is the 5 second interval, but that shouldn’t cause this. could you post your influx configuration for this system?

I’d recommend you stay on 02_03_16.

UPDATE: I found the problem. You should stay on 02_03_16 until a new release with a fix.


#47

@daniweb

I’ve fixed that problem, and also changed the query for last post time to use multiple individual queries, one for each measurement, using the last() function. Hopefully, this will solve your timeout issue.

This is a not a trivial change. I have installed it on all of my units without incident, but I’m reluctant to publish a release until I have a few days experience with it. I’m also interested in seeing if it solves your problem. The updates are merged into master in the git. If you have the ability to compile and load I’d be interested in the result. Although not critical, you should also download and install index.htm from the git.

Let me know if you need a binary.


#48

I’m not at home, and it will be difficult for me to find time before Tuesday.
Maybe better to get a binary. Do I have to upload it via the file browser of the iotawatt?


#49

I’ve put it into a development release. It may be ALPHA by Tuesday, if not, I can instruct you how to download a non-standard release.


#50

I have the 02_03_18 since 6h and ok for the moment, but I have to admint that I did not try to play to restart the webserver with a different date.

As impression I think that with the change from the 02_03_16 we have 3K or 4K of additional heep free.


#51

Just by virtue of installing the new release, you have restarted influx. I am hopeful that the log does not show timeouts in the last-post query process and that the restart did recognize the correct time. In your case, it may take a minute or more because of the high number of measurements that must be queried.

I expect that if you provide a begin-date that is greater than the last post date, it will sjip ahead to that. If you provide a begin-date for a new first key value it should begin uploading from 00:00:05 on that date.


#52

There a reboot occurred:

** Restart **

SD initialized.
10/30/18 08:18:06z Real Time Clock is running. Unix time 1540887486
10/30/18 08:18:06z Version 02_03_16
10/30/18 08:18:06z Reset reason: Software/System restart
10/30/18 08:18:06z Trace: 1:4, 1:3, 1:4, 1:3, 1:4, 1:3, 1:4, 1:3, 1:4, 1:3, 1:4, 1:3, 1:4, 1:3, 1:4, 1:3, 1:4, 1:3, 1:4, 1:3, 1:4, 1:3, 1:4, 1:3, 1:4, 1:3, 1:4, 1:3, 1:4, 1:3, 10:2, 10:3
10/30/18 08:18:06z ESP8266 ChipID: 6910350
10/30/18 08:18:06z SPIFFS mounted.
10/30/18 10:18:07 Local time zone: 2
10/30/18 10:18:07 device name: pinguIW1, version: 3
10/30/18 10:18:10 MDNS responder started
10/30/18 10:18:10 You can now connect to http://pinguIW1.local
10/30/18 10:18:10 HTTP server started
10/30/18 10:18:10 timeSync: service started.
10/30/18 10:18:10 statService: started.
10/30/18 10:18:10 WiFi connected. SSID pingu, IP 192.168.18.73, channel 1, RSSI -76db
10/30/18 10:18:10 Updater: service started. Auto-update class is ALPHA
10/30/18 10:18:10 dataLog: service started.
10/30/18 10:18:11 dataLog: Last log entry 10/30/18 10:18:00
10/30/18 10:18:11 historyLog: service started.
10/30/18 10:18:11 historyLog: Last log entry 10/30/18 10:18:00
10/30/18 10:18:13 Updater: Update from 02_03_16 to 02_03_18
10/30/18 10:18:13 Updater: download 02_03_18
10/30/18 10:18:40 Updater: Release downloaded 27068ms, size 638232
10/30/18 10:18:40 EmonService: started. url:emoncms.org:80,node:pinguITW,interval:10, unsecure GET
10/30/18 10:18:40 influxDB: started.
10/30/18 10:18:52 Updater: Update downloaded and signature verified
10/30/18 10:18:58 Updater: firmware upgraded to version 02_03_18
10/30/18 10:18:59 Firmware updated, restarting.

** Restart **

SD initialized.
10/30/18 08:19:06z Real Time Clock is running. Unix time 1540887546
10/30/18 08:19:06z Version 02_03_18
10/30/18 08:19:06z Updater: Installing update files for version 02_03_18
10/30/18 08:19:06z Updater: Installing GRAPH.HTM
10/30/18 08:19:06z Updater: Installing GRAPH.JS
10/30/18 08:19:06z Updater: Installing INDEX.HTM
10/30/18 08:19:07z Updater: Installing TABLES.TXT
10/30/18 08:19:07z Updater: Installing CNFSTYLE.CSS
10/30/18 08:19:07z Updater: Installing EDIT.HTM
10/30/18 08:19:08z Updater: Installation complete.
10/30/18 08:19:08z Reset reason: Software/System restart
10/30/18 08:19:08z Trace: 1:2[6], 9:0[6], 9:0, 9:1, 8:4, 8:6, 8:8, 8:9, 9:3, 9:5, 9:9, 1:2, 1:3, 1:4, 1:5[19], 1:6, 1:1[6], 1:2[7], 9:0[7], 9:0, 9:1, 8:4, 8:6, 8:8, 8:9, 9:3, 9:5, 9:9, 1:2, 1:3, 1:4, 1:5[5]
10/30/18 08:19:08z ESP8266 ChipID: 6910350
10/30/18 08:19:08z SPIFFS mounted.
10/30/18 10:19:09 Local time zone: 2
10/30/18 10:19:09 device name: pinguIW1, version: 3
10/30/18 10:19:10 MDNS responder started for hostname pinguIW1
10/30/18 10:19:10 LLMNR responder started for hostname pinguIW1
10/30/18 10:19:10 HTTP server started
10/30/18 10:19:10 timeSync: service started.
10/30/18 10:19:10 statService: started.
10/30/18 10:19:10 WiFi connected. SSID=pingu, IP=192.168.18.73, channel=1, RSSI -74db
10/30/18 10:19:10 Updater: service started. Auto-update class is ALPHA
10/30/18 10:19:10 dataLog: service started.
10/30/18 10:19:11 dataLog: Last log entry 10/30/18 10:18:30
10/30/18 10:19:11 historyLog: service started.
10/30/18 10:19:11 historyLog: Last log entry 10/30/18 10:18:00
10/30/18 10:19:13 Updater: Auto-update is current for class ALPHA.
10/30/18 10:19:15 EmonService: started. url:80=emoncms.org, node=pinguITW, interval=10
10/30/18 10:19:15 influxDB: started, url=192.168.18.10:8086, db=itw02, interval=5
10/30/18 10:19:16 EmonService: Start posting at 10/30/18 10:18:10
10/30/18 10:19:30 influxDB: Start posting at 10/30/18 10:18:05

** Restart **

SD initialized.
10/31/18 21:22:34z Real Time Clock is running. Unix time 1541020954
10/31/18 21:22:34z Version 02_03_18
10/31/18 21:22:34z Reset reason: Hardware Watchdog
10/31/18 21:22:34z Trace: 18:0, 18:2, 18:3, 18:2, 18:3, 18:2, 18:3, 18:2, 18:3, 18:2, 18:3, 18:2, 18:3, 18:2, 18:3, 18:2, 18:3, 18:2, 18:3, 18:2, 18:3, 18:2, 18:3, 18:2, 18:3, 18:2, 18:3, 18:2, 18:3, 18:2, 18:3, 18:4
10/31/18 21:22:34z ESP8266 ChipID: 6910350
10/31/18 21:22:34z SPIFFS mounted.
10/31/18 23:22:35 Local time zone: 2
10/31/18 23:22:35 device name: pinguIW1, version: 3
10/31/18 23:22:35 MDNS responder started for hostname pinguIW1
10/31/18 23:22:35 LLMNR responder started for hostname pinguIW1
10/31/18 23:22:35 HTTP server started
10/31/18 23:22:35 timeSync: service started.
10/31/18 23:22:36 statService: started.
10/31/18 23:22:36 WiFi connected. SSID=pingu, IP=192.168.18.73, channel=1, RSSI -74db
10/31/18 23:22:37 Updater: service started. Auto-update class is ALPHA
10/31/18 23:22:37 dataLog: service started.
10/31/18 23:22:37 dataLog: Last log entry 10/31/18 23:22:20
10/31/18 23:22:37 historyLog: service started.
10/31/18 23:22:38 historyLog: Last log entry 10/31/18 23:22:00
10/31/18 23:22:39 Updater: Auto-update is current for class ALPHA.
10/31/18 23:22:40 EmonService: started. url:80=emoncms.org, node=pinguITW, interval=10
10/31/18 23:22:40 influxDB: started, url=192.168.18.10:8086, db=itw02, interval=5
10/31/18 23:22:41 EmonService: Start posting at 10/31/18 23:22:30
10/31/18 23:22:53 influxDB: Start posting at 10/31/18 23:22:25


#53

Hardware watchdog is a tough one to sort out. The software watchdog has a shorter interval and usually is the one that catches runaway loops and such. Also, the piece of code that the trace reveals doesn’t have any explicit loop, although there may be an implicit one in a call to get the heap size.

More likely this has something to do with WiFi and TCP. I believe some of that stuff operates below the software watchdog purview and is more subject to hardware WDT. The recovery was quick and nothing seems to have been lost, which is what the multi-layer fail-safe design is supposed to do. So keep an eye on it. If it happens a lot, let me know.

PS: It looks like your influx last post issues are gone. It does take up to 15 seconds to query all those measurements, but there’s no real downside to that.