Hi there.
Yesterday I upgraded from 02_05_12 to 02_06_05 as it had been in MINOR for a while now and in the past 20 hours I have had 136 restarts where was previously almost all unattended restarts were the six week routine restart. I am have now switched back to MAJOR.
I am using the influxDB_v1 uploader to a local server, a script polling the status endpoint every 10 seconds, and in the last couple of works I have also set up the Home Assistant addon.
20 of the restarts have been logged as “Heap memory has degraded below safe minimum, restarting.” and the remaining 116 have a restart reason of “Exception” but these seem to be heap related too. Plotting the heap free from my custom logger shows it was sitting around 22k free before the update and post-update it is rapidly depleting. Heap starts around 25k free after the restart and drops linearly to ~10k over about 9 minutes. This then repeats.
There’s nothing standing out to me as unusual in the logs other than the restarts. Here’s logs for the 2 restarts before I downgraded:
** Restart **
SD initialized.
9/26/21 20:26:34z Real Time Clock is running. Unix time 1632687994
9/26/21 20:26:34z Reset reason: Exception
9/26/21 20:26:34z Trace: 1:3, 1:3, 1:3, 1:3, 1:3, 1:3, 1:3, 1:3, 1:3, 1:3, 1:1[2], 1:2[3], 9:0[3], 9:0, 9:1, 8:4, 8:6, 8:8, 8:9, 9:3, 9:5, 9:9, 1:2, 1:3, 10:13, 15:0[32], 15:3[32], 15:10, 15:14[32], 15:15[2], 15:16, 15:4
9/26/21 20:26:34z ESP8266 ChipID: 6920572
9/26/21 20:26:34z IoTaWatt 4.x, Firmware version 02_06_05
9/26/21 20:26:34z SPIFFS mounted.
9/26/21 20:26:34z Local time zone: +0:00
9/26/21 20:26:34z device name: IotaWatt
9/26/21 20:26:34z HTTP server started
9/26/21 20:26:34z influxDB_v1: Starting, interval:5, url:http://sentry.lan.backtick.org:8086
9/26/21 20:26:34z timeSync: service started.
9/26/21 20:26:34z statService: started.
9/26/21 20:26:34z dataLog: service started.
9/26/21 20:26:36z dataLog: Last log entry 09/26/21 20:26:30
9/26/21 20:26:39z historyLog: service started.
9/26/21 20:26:39z historyLog: Last log entry 09/26/21 20:26:00
9/26/21 20:26:39z WiFi connected. SSID=🍓, IP=192.168.14.229, channel=1, RSSI -50db
9/26/21 20:26:39z MDNS responder started for hostname IotaWatt
9/26/21 20:26:39z LLMNR responder started for hostname IotaWatt
9/26/21 20:26:39z Updater: service started. Auto-update class is NONE
9/26/21 20:27:19z influxDB_v1: Start posting at 09/26/21 20:26:30
9/26/21 20:35:18z Heap memory has degraded below safe minimum, restarting.
** Restart **
SD initialized.
9/26/21 20:35:19z Real Time Clock is running. Unix time 1632688519
9/26/21 20:35:19z Reset reason: Software/System restart
9/26/21 20:35:19z Trace: 1:2, 1:3, 10:13, 15:0[32], 15:3[32], 15:10, 15:14[32], 15:15[2], 15:16, 15:4, 1:3, 1:1[9], 1:2, 9:0, 9:0, 8:4, 8:6, 8:8, 8:9, 1:2, 1:3, 1:3, 1:6[1], 1:6[2], 1:6[2], 1:6[3], 1:5[21], 1:6[4], 21:0, 21:1, 21:10, 21:10
9/26/21 20:35:19z ESP8266 ChipID: 6920572
9/26/21 20:35:19z IoTaWatt 4.x, Firmware version 02_06_05
9/26/21 20:35:19z SPIFFS mounted.
9/26/21 20:35:19z Local time zone: +0:00
9/26/21 20:35:19z device name: IotaWatt
9/26/21 20:35:19z HTTP server started
9/26/21 20:35:19z influxDB_v1: Starting, interval:5, url:http://sentry.lan.backtick.org:8086
9/26/21 20:35:19z timeSync: service started.
9/26/21 20:35:19z statService: started.
9/26/21 20:35:19z dataLog: service started.
9/26/21 20:35:21z dataLog: Last log entry 09/26/21 20:35:15
9/26/21 20:35:24z historyLog: service started.
9/26/21 20:35:24z historyLog: Last log entry 09/26/21 20:35:00
9/26/21 20:35:24z WiFi connected. SSID=🍓, IP=192.168.14.229, channel=1, RSSI -45db
9/26/21 20:35:24z MDNS responder started for hostname IotaWatt
9/26/21 20:35:24z LLMNR responder started for hostname IotaWatt
9/26/21 20:35:24z Updater: service started. Auto-update class is NONE
9/26/21 20:36:44z influxDB_v1: Start posting at 09/26/21 20:35:15
9/26/21 20:43:48z Heap memory has degraded below safe minimum, restarting.
I’ll wait for your thoughts before I do anything further. Happy to do testing such as trying other versions, disabling influx v1 and/or Home Assistant integration, etc. to see if that stops the issue.
Cheers,
Jason