Crashing when enabling influxdb v2

Support
1

Crashing when enabling Influxdb V2 Cloud using reverse SSL proxy.

I have been using iotawatt for almost 3 years now with everything getting pushed into influx 1.x db. I just enabled the 2.x stuff using the influx cloud. I set it to upload just a couple of items from Oct 1, 2022. As soon as I enable it I can see it uploading throught he proxy logs, however it keeps crashing iotawatt If I stop the proxy server I can get logged back in. Here is a snippet of the logs:

** Restart **

SD initialized.
10/30/22 08:22:22z Real Time Clock is running. Unix time 1667118142 
10/30/22 08:22:22z Reset reason: Exception
10/30/22 08:22:22z Trace:  1:6[4], 31:0, 31:1, 31:2[6], 31:90, 21:110, 21:110, 31:91, 31:9, 31:1, 1:6[6], 1:3, 1:3, 1:6[1], 1:6[2], 1:6[3], 1:5[31], 1:6[4], 31:0, 31:1, 31:2[5], 31:120, 21:100[31], 21:101, 31:120, 31:122, 31:123, 31:124, 32:95, 32:95, 31:126, 31:1
10/30/22 08:22:22z ESP8266 ID: 2518078, RTC PCF8523 (68)
10/30/22 08:22:22z IoTaWatt 4.x, Firmware version 02_07_05
10/30/22 08:22:22z SPIFFS mounted.
10/30/22 01:22:22 Local time zone: -8:00, using DST/BST when in effect.
10/30/22 01:22:22 device name: IotaWatt
10/30/22 01:22:22 HTTP server started
10/30/22 01:22:22 influxDB_v1: Starting, interval:5, url:http://192.168.50.20:8086
10/30/22 01:22:22 influxDB_v2: Starting, interval:10, url:https://us-west-2-2.aws.cloud2.influxdata.com
10/30/22 01:22:22 timeSync: service started.
10/30/22 01:22:22 statService: started.
10/30/22 01:22:22 dataLog: service started.
10/30/22 01:22:24 dataLog: Last log entry 10/30/22 01:22:20
10/30/22 01:22:27 historyLog: service started.
10/30/22 01:22:28 historyLog: Last log entry 10/30/22 01:22:00
10/30/22 01:22:28 WiFi connected. SSID=CrzyWifi, IP=192.168.50.26, channel=6, RSSI -69db
10/30/22 01:22:28 MDNS responder started for hostname IotaWatt
10/30/22 01:22:28 LLMNR responder started for hostname IotaWatt
10/30/22 01:22:28 Updater: service started. Auto-update class is BETA
10/30/22 01:22:28 grid: Started
10/30/22 01:22:28 grid: Last log entry 10/30/22 01:22:20

** Restart **

SD initialized.
10/30/22 08:22:29z Real Time Clock is running. Unix time 1667118149 
10/30/22 08:22:29z Reset reason: Exception
10/30/22 08:22:29z Trace:  8:8, 8:9, 9:3, 9:5, 9:9, 1:2, 1:3, 1:3, 1:1[8], 1:2[9], 9:0[9], 9:0, 9:1, 8:4, 8:6, 8:8, 8:9, 9:3, 9:5, 9:9, 1:2, 1:3, 1:3, 1:1[9], 1:2, 9:0, 9:0, 8:4, 8:6, 8:8, 8:9, 1:2
10/30/22 08:22:29z ESP8266 ID: 2518078, RTC PCF8523 (68)
10/30/22 08:22:29z IoTaWatt 4.x, Firmware version 02_07_05
10/30/22 08:22:29z SPIFFS mounted.
10/30/22 01:22:29 Local time zone: -8:00, using DST/BST when in effect.
10/30/22 01:22:29 device name: IotaWatt
10/30/22 01:22:29 HTTP server started
10/30/22 01:22:29 influxDB_v1: Starting, interval:5, url:http://192.168.50.20:8086
10/30/22 01:22:29 influxDB_v2: Starting, interval:10, url:https://us-west-2-2.aws.cloud2.influxdata.com
10/30/22 01:22:29 timeSync: service started.
10/30/22 01:22:29 statService: started.
10/30/22 01:22:29 dataLog: service started.
10/30/22 01:22:31 dataLog: Last log entry 10/30/22 01:22:25
10/30/22 01:22:34 historyLog: service started.
10/30/22 01:22:35 historyLog: Last log entry 10/30/22 01:22:00
10/30/22 01:22:35 WiFi connected. SSID=CrzyWifi, IP=192.168.50.26, channel=6, RSSI -69db
10/30/22 01:22:35 MDNS responder started for hostname IotaWatt
10/30/22 01:22:35 LLMNR responder started for hostname IotaWatt
10/30/22 01:22:35 Updater: service started. Auto-update class is BETA
10/30/22 01:22:35 grid: Started
10/30/22 01:22:36 grid: Last log entry 10/30/22 01:22:25

** Restart **

SD initialized.
10/30/22 08:22:37z Real Time Clock is running. Unix time 1667118157 
10/30/22 08:22:37z Reset reason: Exception
10/30/22 08:22:37z Trace:  32:95, 31:126, 31:1, 1:6[6], 1:1, 1:2[1], 9:0[1], 9:0, 9:1, 8:4, 8:6, 8:8, 8:9, 9:3, 9:5, 9:9, 1:2, 1:3, 1:3, 1:1[1], 1:2[2], 9:0[2], 9:0, 9:1, 8:4, 8:6, 8:8, 8:9, 9:3, 9:5, 9:9, 1:2
10/30/22 08:22:37z ESP8266 ID: 2518078, RTC PCF8523 (68)
10/30/22 08:22:37z IoTaWatt 4.x, Firmware version 02_07_05
10/30/22 08:22:37z SPIFFS mounted.
10/30/22 01:22:37 Local time zone: -8:00, using DST/BST when in effect.
10/30/22 01:22:37 device name: IotaWatt
10/30/22 01:22:37 HTTP server started
10/30/22 01:22:37 influxDB_v1: Starting, interval:5, url:http://192.168.50.20:8086
10/30/22 01:22:37 influxDB_v2: Starting, interval:10, url:https://us-west-2-2.aws.cloud2.influxdata.com
10/30/22 01:22:37 timeSync: service started.
10/30/22 01:22:37 statService: started.
10/30/22 01:22:37 dataLog: service started.
10/30/22 01:22:39 dataLog: Last log entry 10/30/22 01:22:35
10/30/22 01:22:42 historyLog: service started.
10/30/22 01:22:43 historyLog: Last log entry 10/30/22 01:22:00
10/30/22 01:22:43 WiFi connected. SSID=CrzyWifi, IP=192.168.50.26, channel=6, RSSI -68db
10/30/22 01:22:43 MDNS responder started for hostname IotaWatt
10/30/22 01:22:43 LLMNR responder started for hostname IotaWatt
10/30/22 01:22:43 Updater: service started. Auto-update class is BETA
10/30/22 01:22:43 grid: Started
10/30/22 01:22:43 grid: Last log entry 10/30/22 01:22:35
10/30/22 01:22:46 Updater: Auto-update is current for class BETA.
10/30/22 01:22:46 influxDB_v1: Start posting at 10/30/22 01:11:35
2

Is any data being uploaded to influx2? Can I see the setup for influx2?

3

I don’t see any data showing up to influxdb. Here is the settings:

image
image702×825 43.6 KB

Very Basic. I can see the request hit the proxy server.

4

Can you show me the request and if you can see the response, I’d like to see that as well.

5

Sorry it was something with my nginx config. I am not sure what, but I tore it down and grabbed the default nginx container and configured from scratch and that seemed to have solved it.

6

I am still noticing a fairly often restart trend… if I am reading the logs correctly and working on the device it asks me to re login pretty often…

** Restart **

SD initialized.
10/31/22 04:50:48z Real Time Clock is running. Unix time 1667191848 
10/31/22 04:50:48z Reset reason: Software/System restart
10/31/22 04:50:48z Trace:  31:124, 32:95, 32:95, 31:126, 31:1, 1:6[6], 1:1, 1:2[1], 9:0[1], 9:0, 9:1, 8:4, 8:6, 8:8, 8:9, 9:3, 9:5, 9:9, 1:2, 1:3, 1:3, 1:6[1], 1:6[2], 1:6[2], 1:6[2], 1:6[3], 1:5[21], 1:6[4], 21:0, 21:1, 21:10, 21:10
10/31/22 04:50:48z ESP8266 ID: 2518078, RTC PCF8523 (68)
10/31/22 04:50:48z IoTaWatt 4.x, Firmware version 02_07_05
10/31/22 04:50:48z SPIFFS mounted.
10/30/22 21:50:48 Local time zone: -8:00, using DST/BST when in effect.
10/30/22 21:50:48 device name: IotaWatt
10/30/22 21:50:48 HTTP server started
10/30/22 21:50:48 influxDB_v1: Starting, interval:5, url:http://192.168.50.20:8086
10/30/22 21:50:48 influxDB_v2: Starting, interval:10, url:https://us-west-2-2.aws.cloud2.influxdata.com
10/30/22 21:50:48 timeSync: service started.
10/30/22 21:50:48 statService: started.
10/30/22 21:50:48 dataLog: service started.
10/30/22 21:50:50 dataLog: Last log entry 10/30/22 21:50:45
10/30/22 21:50:53 historyLog: service started.
10/30/22 21:50:54 historyLog: Last log entry 10/30/22 21:50:00
10/30/22 21:50:59 WiFi connected. SSID=CrzyWifi, IP=192.168.50.26, channel=11, RSSI -74db
10/30/22 21:50:59 MDNS responder started for hostname IotaWatt
10/30/22 21:50:59 LLMNR responder started for hostname IotaWatt
10/30/22 21:50:59 Updater: service started. Auto-update class is BETA
10/30/22 21:51:01 Updater: Auto-update is current for class BETA.
10/30/22 21:51:02 influxDB_v2: Resume posting 10/02/22 08:44:50
10/30/22 21:51:08 influxDB_v1: Start posting at 10/30/22 21:50:20
10/30/22 21:59:44 Heap memory has degraded below safe minimum, restarting.

** Restart **

SD initialized.
10/31/22 04:59:45z Real Time Clock is running. Unix time 1667192385 
10/31/22 04:59:45z Reset reason: Software/System restart
10/31/22 04:59:45z Trace:  18:3, 18:2, 18:3, 18:2, 18:3, 18:2, 18:3, 18:2, 18:3, 18:2, 18:3, 18:2, 18:3, 18:2, 18:3, 18:2, 18:3, 18:4, 18:5, 1:6[6], 1:3, 1:3, 1:6[1], 1:6[2], 1:6[2], 1:6[3], 1:5[21], 1:6[4], 21:0, 21:1, 21:10, 21:10
10/31/22 04:59:45z ESP8266 ID: 2518078, RTC PCF8523 (68)
10/31/22 04:59:45z IoTaWatt 4.x, Firmware version 02_07_05
10/31/22 04:59:45z SPIFFS mounted.
10/30/22 21:59:45 Local time zone: -8:00, using DST/BST when in effect.
10/30/22 21:59:45 device name: IotaWatt
10/30/22 21:59:45 HTTP server started
10/30/22 21:59:45 influxDB_v1: Starting, interval:5, url:http://192.168.50.20:8086
10/30/22 21:59:45 influxDB_v2: Starting, interval:10, url:https://us-west-2-2.aws.cloud2.influxdata.com
10/30/22 21:59:45 timeSync: service started.
10/30/22 21:59:45 statService: started.
10/30/22 21:59:45 dataLog: service started.
10/30/22 21:59:47 dataLog: Last log entry 10/30/22 21:59:40
10/30/22 21:59:50 historyLog: service started.
10/30/22 21:59:51 historyLog: Last log entry 10/30/22 21:59:00
10/30/22 21:59:56 WiFi connected. SSID=CrzyWifi, IP=192.168.50.26, channel=6, RSSI -71db
10/30/22 21:59:56 MDNS responder started for hostname IotaWatt
10/30/22 21:59:56 LLMNR responder started for hostname IotaWatt
10/30/22 21:59:56 Updater: service started. Auto-update class is BETA
10/30/22 21:59:58 Updater: Auto-update is current for class BETA.
10/30/22 22:00:00 influxDB_v2: Resume posting 10/02/22 17:20:50
10/30/22 22:00:04 influxDB_v1: Start posting at 10/30/22 21:59:05
10/30/22 22:06:51 Heap memory has degraded below safe minimum, restarting.  looks like very 8-10 min or so.   It is uploading date to influxdbv2 now though.

** Restart **

SD initialized.
10/31/22 05:06:53z Real Time Clock is running. Unix time 1667192813 
10/31/22 05:06:53z Reset reason: Software/System restart
10/31/22 05:06:53z Trace:  18:2, 18:3, 18:4, 18:5, 1:6[6], 1:3, 1:3, 1:1[10], 1:2[11], 9:0[11], 9:0, 9:1, 8:4, 8:6, 8:8, 8:9, 9:3, 9:5, 9:9, 1:2, 1:3, 1:3, 1:6[1], 1:6[2], 1:6[2], 1:6[3], 1:5[21], 1:6[4], 21:0, 21:1, 21:10, 21:10
10/31/22 05:06:53z ESP8266 ID: 2518078, RTC PCF8523 (68)
10/31/22 05:06:53z IoTaWatt 4.x, Firmware version 02_07_05
10/31/22 05:06:53z SPIFFS mounted.
10/30/22 22:06:53 Local time zone: -8:00, using DST/BST when in effect.
10/30/22 22:06:53 device name: IotaWatt
10/30/22 22:06:53 HTTP server started
10/30/22 22:06:53 influxDB_v1: Starting, interval:5, url:http://192.168.50.20:8086
10/30/22 22:06:53 influxDB_v2: Starting, interval:10, url:https://us-west-2-2.aws.cloud2.influxdata.com
10/30/22 22:06:53 timeSync: service started.
10/30/22 22:06:53 statService: started.
10/30/22 22:06:53 dataLog: service started.
10/30/22 22:06:55 dataLog: Last log entry 10/30/22 22:06:50
10/30/22 22:06:58 historyLog: service started.
10/30/22 22:06:59 historyLog: Last log entry 10/30/22 22:06:00
10/30/22 22:06:59 WiFi connected. SSID=CrzyWifi, IP=192.168.50.26, channel=6, RSSI -71db
10/30/22 22:06:59 MDNS responder started for hostname IotaWatt
10/30/22 22:06:59 LLMNR responder started for hostname IotaWatt
10/30/22 22:06:59 Updater: service started. Auto-update class is BETA
10/30/22 22:07:01 influxDB_v2: Resume posting 10/03/22 00:13:30
10/30/22 22:07:03 Updater: Auto-update is current for class BETA.
10/30/22 22:07:06 influxDB_v1: Start posting at 10/30/22 22:06:10
7

The influx2 history upload appears to be making headway, but it is restarting regularly due to degraded heap. This is always associated with WiFi problems. You RSSI could be better. I suspect that after the history upload completes, the situation will improve, but if not, try improving the WiFi signal.

8

Thanks ya I have been reading through the wifi issues, I am going to try an AP in the garage today and see if that helps.

9

I’ve been chasing the heap degradation problem for a long time and have come to believe that it is in lower-level code in the ESP8266 core. I have no proof, only my frustration of not being able to find anything in the IoTaWatt firmware.

The latest release, 02_08_00 includes a much-updated core, with substantial changes to the low-level TCP code. It was released to auto-update ALPHA on Oct 6 and has had zero reported problems. I will be advancing it to BETA soon, but you might try going ALPHA to see if it makes a difference. If so, I’d be interested in the result.

10

Ok so I have not changed my AP layout yet, but I did move my iotawatt from a wifi network that was dual band. So using both 2.4ghz and 5ghz. I moved to my dedicated iotnetwork that is locked to 2.4Ghz. This has now been running for 30 minutes with no crash doing the v2 db uploads. Free heap is running between 18000 and 13000

I thought this was an interesting piece. It is connected to the same AP just a different SSID.

11

For what it is worth, just moving SSID seemed to solve it. I am on Ubiquiti APs, but moving to just a 2.4 band SSID seemed to solve it.

image

1 Like
12

It’s always the WiFi.

1 Like