Saved graphs disappeared from Graph+

Support
1

This seems to happen intermittently. Sometimes when I navigate to Graph+, the dropdown with my saved graphs is empty. Looking at the Chrome network inspector in dev tools, /graph/getall does appear to have all (except maybe one) of my graphs, but the dropdown remains empty.

I don’t see any other JS or HTTP errors, and reloading the page (or just coming back later) sometimes fixes it. But sometimes I reload many times and the graphs never appear.

Is there anything I can do to troubleshoot?

Just took a look at the message log, and it seems device resets are happening with some frequency, with exceptions and hardware watchdog as the usual reasons.

11/17/19 23:24:21 Updater: service started. Auto-update class is BETA
11/17/19 23:24:22 EmonService: Start posting at 11/17/19 23:23:55
11/17/19 23:24:23 Updater: Auto-update is current for class BETA.
11/17/19 23:24:23 influxDB: Start posting at 11/17/19 23:23:20
11/17/19 23:25:14 historyLog: service started.
11/17/19 23:25:14 historyLog: Last log entry 11/17/19 23:23:00
11/17/19 23:25:14 PVoutput: started
11/17/19 23:25:14 PVoutput: System supermassive 7.25kW, interval 5, donator mode  
11/17/19 23:25:15 PVoutput: Start status beginning 11/17/19 23:20:00
11/18/19 09:13:05 dataLog: datalog WDT - restarting

** Restart **

SD initialized.
11/17/19 22:13:07z Real Time Clock is running. Unix time 1574028787 
11/17/19 22:13:07z Reset reason: Software/System restart
11/17/19 22:13:07z Trace:  25:64, 25:65, 25:61, 25:64, 25:65, 25:61, 25:64, 25:65, 25:61, 25:64, 25:65, 25:61, 25:64, 25:65, 25:61, 25:64, 25:65, 25:61, 25:64, 25:65, 25:61, 25:64, 25:65, 25:61, 25:64, 25:65, 25:55, 25:56, 25:41, 25:42, 25:43, 10:57
11/17/19 22:13:07z ESP8266 ChipID: 2518879
11/17/19 22:13:07z IoTaWatt 5.0, Firmware version 02_05_01
11/17/19 22:13:07z SPIFFS mounted.
11/18/19 09:13:08 Local time zone: +10:00
11/18/19 09:13:08 Using Daylight Saving Time (BST) when in effect.
11/18/19 09:13:08 device name: IotaWatt
11/18/19 09:13:08 MDNS responder started for hostname IotaWatt
11/18/19 09:13:08 LLMNR responder started for hostname IotaWatt
11/18/19 09:13:08 HTTP server started
11/18/19 09:13:08 timeSync: service started.
11/18/19 09:13:08 statService: started.
11/18/19 09:13:09 dataLog: service started.
11/18/19 09:13:09 dataLog: Last log entry 11/18/19 09:12:35
11/18/19 09:13:13 EmonService: started. url=emoncms.org:80, node=IotaWatt, interval=5, encrypted
11/18/19 09:13:13 influxDB: started, url=hassio.home:8086, db=iotawatt, interval=10
11/18/19 09:13:28 WiFi connected. SSID=supermassive 24, IP=192.168.50.244, channel=9, RSSI -59db
11/18/19 09:13:28 Updater: service started. Auto-update class is BETA
11/18/19 09:13:30 EmonService: Start posting at 11/18/19 09:12:15
11/18/19 09:13:31 Updater: Auto-update is current for class BETA.
11/18/19 09:13:32 influxDB: Start posting at 11/18/19 09:12:10
11/18/19 09:14:08 historyLog: service started.
11/18/19 09:14:08 historyLog: Last log entry 11/18/19 09:12:00
11/18/19 09:14:08 PVoutput: started
11/18/19 09:14:09 PVoutput: System supermassive 7.25kW, interval 5, donator mode  
11/18/19 09:14:09 PVoutput: Start status beginning 11/18/19 09:10:00

** Restart **

SD initialized.
11/18/19 05:18:17z Real Time Clock is running. Unix time 1574054297 
11/18/19 05:18:17z Reset reason: Hardware Watchdog
11/18/19 05:18:17z Trace:  1:6, 1:3, 1:4, 1:5[3], 3:0, 3:11, 1:6, 1:3, 1:4, 1:5[3], 3:0, 3:11, 1:6, 1:3, 1:4, 1:5[3], 3:0, 3:11, 1:6, 1:1[4], 1:2[5], 9:0[5], 9:0, 9:1, 8:4, 8:6, 8:8, 8:9, 9:3, 9:5, 9:9, 1:2
11/18/19 05:18:17z ESP8266 ChipID: 2518879
11/18/19 05:18:17z IoTaWatt 5.0, Firmware version 02_05_01
11/18/19 05:18:17z SPIFFS mounted.
11/18/19 16:18:18 Local time zone: +10:00
11/18/19 16:18:18 Using Daylight Saving Time (BST) when in effect.
11/18/19 16:18:18 device name: IotaWatt
11/18/19 16:18:18 MDNS responder started for hostname IotaWatt
11/18/19 16:18:18 LLMNR responder started for hostname IotaWatt
11/18/19 16:18:18 HTTP server started
11/18/19 16:18:18 timeSync: service started.
11/18/19 16:18:18 statService: started.
11/18/19 16:18:19 dataLog: service started.
11/18/19 16:18:19 dataLog: Last log entry 11/18/19 16:18:05
11/18/19 16:18:23 EmonService: started. url=emoncms.org:80, node=IotaWatt, interval=5, encrypted
11/18/19 16:18:23 influxDB: started, url=hassio.home:8086, db=iotawatt, interval=10
11/18/19 16:18:34 WiFi connected. SSID=supermassive 24, IP=192.168.50.244, channel=9, RSSI -52db
11/18/19 16:18:34 Updater: service started. Auto-update class is BETA
11/18/19 16:18:35 EmonService: Start posting at 11/18/19 16:17:40
11/18/19 16:18:36 Updater: Auto-update is current for class BETA.
11/18/19 16:18:36 influxDB: Start posting at 11/18/19 16:17:40
11/18/19 16:19:18 historyLog: service started.
11/18/19 16:19:18 historyLog: Last log entry 11/18/19 16:18:00
11/18/19 16:19:18 PVoutput: started
11/18/19 16:19:18 PVoutput: System supermassive 7.25kW, interval 5, donator mode  
11/18/19 16:19:19 PVoutput: Start status beginning 11/18/19 16:15:00

** Restart **

SD initialized.
11/18/19 20:15:20z Real Time Clock is running. Unix time 1574108120 
11/18/19 20:15:20z Reset reason: Hardware Watchdog
11/18/19 20:15:20z Trace:  1:4, 1:3, 1:4, 1:3, 1:4, 1:3, 1:4, 1:3, 1:4, 1:3, 1:4, 1:3, 1:4, 1:1[7], 1:2[8], 9:0[8], 9:0, 9:1, 8:4, 8:6, 8:8, 8:9, 9:3, 9:5, 9:9, 1:2, 1:3, 1:4, 1:5[21], 21:0, 21:1, 21:10
11/18/19 20:15:20z ESP8266 ChipID: 2518879
11/18/19 20:15:20z IoTaWatt 5.0, Firmware version 02_05_01
11/18/19 20:15:20z SPIFFS mounted.
11/19/19 07:15:21 Local time zone: +10:00
11/19/19 07:15:21 Using Daylight Saving Time (BST) when in effect.
11/19/19 07:15:21 device name: IotaWatt
11/19/19 07:15:21 MDNS responder started for hostname IotaWatt
11/19/19 07:15:21 LLMNR responder started for hostname IotaWatt
11/19/19 07:15:21 HTTP server started
11/19/19 07:15:21 timeSync: service started.
11/19/19 07:15:21 statService: started.
11/19/19 07:15:21 Updater: service started. Auto-update class is BETA
11/19/19 07:15:22 WiFi connected. SSID=supermassive 24, IP=192.168.50.244, channel=9, RSSI -54db
11/19/19 07:15:22 dataLog: service started.
11/19/19 07:15:22 dataLog: Last log entry 11/19/19 07:15:10
11/19/19 07:15:23 Updater: Auto-update is current for class BETA.
11/19/19 07:15:26 EmonService: started. url=emoncms.org:80, node=IotaWatt, interval=5, encrypted
11/19/19 07:15:26 influxDB: started, url=hassio.home:8086, db=iotawatt, interval=10
11/19/19 07:15:27 EmonService: Start posting at 11/19/19 07:14:55
11/19/19 07:15:28 influxDB: Start posting at 11/19/19 07:14:30
11/19/19 07:16:21 historyLog: service started.
11/19/19 07:16:21 historyLog: Last log entry 11/19/19 07:15:00
11/19/19 07:16:21 PVoutput: started
11/19/19 07:16:21 PVoutput: System supermassive 7.25kW, interval 5, donator mode  
11/19/19 07:16:22 PVoutput: Start status beginning 11/19/19 07:15:00

** Restart **

SD initialized.
11/19/19 21:24:57z Real Time Clock is running. Unix time 1574198697 
11/19/19 21:24:57z Reset reason: Exception
11/19/19 21:24:57z Trace:  1:4, 1:3, 1:4, 1:3, 1:4, 1:3, 1:4, 1:3, 1:4, 1:3, 1:4, 1:3, 1:4, 1:3, 1:4, 1:3, 1:4, 1:3, 1:4, 1:3, 1:4, 1:3, 1:4, 1:1[12], 1:2, 9:0, 9:0, 8:4, 8:6, 8:8, 8:9, 1:2
11/19/19 21:24:57z ESP8266 ChipID: 2518879
11/19/19 21:24:57z IoTaWatt 5.0, Firmware version 02_05_01
11/19/19 21:24:57z SPIFFS mounted.
11/20/19 08:24:58 Local time zone: +10:00
11/20/19 08:24:58 Using Daylight Saving Time (BST) when in effect.
11/20/19 08:24:58 device name: IotaWatt
11/20/19 08:24:58 MDNS responder started for hostname IotaWatt
11/20/19 08:24:58 LLMNR responder started for hostname IotaWatt
11/20/19 08:24:58 HTTP server started
11/20/19 08:24:58 timeSync: service started.
11/20/19 08:24:58 statService: started.
11/20/19 08:24:59 dataLog: service started.
11/20/19 08:24:59 dataLog: Last log entry 11/20/19 08:24:50
11/20/19 08:25:03 EmonService: started. url=emoncms.org:80, node=IotaWatt, interval=5, encrypted
11/20/19 08:25:03 influxDB: started, url=hassio.home:8086, db=iotawatt, interval=10
11/20/19 08:25:04 WiFi connected. SSID=supermassive 24, IP=192.168.50.244, channel=9, RSSI -54db
11/20/19 08:25:04 Updater: service started. Auto-update class is BETA
11/20/19 08:25:05 EmonService: Start posting at 11/20/19 08:24:45
11/20/19 08:25:06 influxDB: Start posting at 11/20/19 08:24:40
11/20/19 08:25:07 Updater: Auto-update is current for class BETA.
11/20/19 08:25:58 historyLog: service started.
11/20/19 08:25:58 historyLog: Last log entry 11/20/19 08:24:00
11/20/19 08:25:58 PVoutput: started
11/20/19 08:25:59 PVoutput: System supermassive 7.25kW, interval 5, donator mode  
11/20/19 08:25:59 PVoutput: Start status beginning 11/20/19 08:20:00

** Restart **

SD initialized.
11/19/19 21:44:08z Real Time Clock is running. Unix time 1574199848 
11/19/19 21:44:08z Reset reason: Hardware Watchdog
11/19/19 21:44:08z Trace:  1:4, 1:3, 1:4, 1:3, 1:4, 1:3, 1:4, 1:3, 1:4, 1:3, 1:4, 1:3, 1:4, 1:3, 1:4, 1:3, 1:4, 1:3, 1:4, 1:1[5], 1:2[6], 9:0[6], 9:0, 9:1, 8:4, 8:6, 8:8, 8:9, 9:3, 9:5, 9:9, 1:2
11/19/19 21:44:08z ESP8266 ChipID: 2518879
11/19/19 21:44:08z IoTaWatt 5.0, Firmware version 02_05_01
11/19/19 21:44:08z SPIFFS mounted.
11/20/19 08:44:09 Local time zone: +10:00
11/20/19 08:44:09 Using Daylight Saving Time (BST) when in effect.
11/20/19 08:44:09 device name: IotaWatt
11/20/19 08:44:09 MDNS responder started for hostname IotaWatt
11/20/19 08:44:09 LLMNR responder started for hostname IotaWatt
11/20/19 08:44:09 HTTP server started
11/20/19 08:44:09 timeSync: service started.
11/20/19 08:44:09 statService: started.
11/20/19 08:44:10 dataLog: service started.
11/20/19 08:44:10 dataLog: Last log entry 11/20/19 08:43:55
11/20/19 08:44:14 EmonService: started. url=emoncms.org:80, node=IotaWatt, interval=5, encrypted
11/20/19 08:44:14 influxDB: started, url=hassio.home:8086, db=iotawatt, interval=10
11/20/19 08:44:16 WiFi connected. SSID=supermassive 24, IP=192.168.50.244, channel=9, RSSI -54db
11/20/19 08:44:16 Updater: service started. Auto-update class is BETA
11/20/19 08:44:17 EmonService: Start posting at 11/20/19 08:43:55
11/20/19 08:44:17 influxDB: Start posting at 11/20/19 08:43:20
11/20/19 08:44:19 Updater: Auto-update is current for class BETA.
11/20/19 08:45:09 historyLog: service started.
11/20/19 08:45:09 historyLog: Last log entry 11/20/19 08:43:00
11/20/19 08:45:09 PVoutput: started
11/20/19 08:45:10 PVoutput: System supermassive 7.25kW, interval 5, donator mode  
11/20/19 08:45:11 PVoutput: Start status beginning 11/20/19 08:40:00

Perhaps when I “come back later” the device happens to have been reset and graphs start working again.

2

I just tried manually restarting the device (via the web interface), but graphs are still missing.

3

I’d be interested in the response from graph/getall in the Chrome debug. In particular the response content, which should be a bunch of Json.

That the graphs do reappear from time to time tells me that the saved files are OK, and it’s just a matter of returning them in getall.

How many files are there in the /graphs directory. The individual graphs are save with eight character hashed names. The old graph files are .txt and the Graph+ files are .jsn.

4

I just tried again, and it was working. Then I reloaded the page, and saved graphs were missing again.

Here’s the full response from /graph/getall:

[{"name":"Solar Analytics - Consumption","period":"today","group":"auto","unit":"Watts","endDate":"2019-10-31T10:47:05.000Z","beginDate":"2019-10-30T13:00:00.000Z","feedlist":[{"name":"air_conditioners","unit":"Watts","color":"#b22838","fill":true,"stack":false,"scale":1,"accrue":false,"dp":1,"plottype":"Line","dataindex":2,"yaxis":1},{"name":"hot_water_system","unit":"Watts","color":"#165fbb","fill":true,"stack":false,"scale":1,"accrue":false,"dp":1,"plottype":"Line","dataindex":3,"yaxis":1},{"name":"lights","unit":"Watts","color":"#31a670","fill":true,"stack":false,"scale":1,"accrue":false,"dp":1,"plottype":"Line","dataindex":4,"yaxis":1},{"name":"oven","unit":"Watts","color":"#ff338a","fill":true,"stack":false,"scale":1,"accrue":false,"dp":1,"plottype":"Line","dataindex":5,"yaxis":1},{"name":"power_points","unit":"Watts","color":"#8860be","fill":true,"stack":false,"scale":1,"accrue":false,"dp":1,"plottype":"Line","dataindex":6,"yaxis":1},{"name":"solar_analytics","unit":"Watts","color":"#ff6100","fill":true,"stack":false,"scale":1,"accrue":false,"dp":1,"plottype":"Line","dataindex":7,"yaxis":1},{"name":"production","unit":"Watts","color":"#fecb4e","fill":true,"stack":false,"scale":1,"accrue":false,"dp":1,"plottype":"Line","dataindex":1,"yaxis":1}],"yaxes":[],"id":"HWYRIBX2.TXT"},{"name":"Solar Analytics - Consumption (stacked)","period":"today","group":"auto","unit":"Watts","endDate":"2019-10-31T10:47:05.000Z","beginDate":"2019-10-30T13:00:00.000Z","feedlist":[{"name":"air_conditioners","unit":"Watts","color":"#b22838","fill":true,"stack":true,"scale":1,"accrue":false,"dp":1,"plottype":"Line","dataindex":2,"yaxis":1},{"name":"hot_water_system","unit":"Watts","color":"#165fbb","fill":true,"stack":true,"scale":1,"accrue":false,"dp":1,"plottype":"Line","dataindex":3,"yaxis":1},{"name":"lights","unit":"Watts","color":"#31a670","fill":true,"stack":true,"scale":1,"accrue":false,"dp":1,"plottype":"Line","dataindex":4,"yaxis":1},{"name":"oven","unit":"Watts","color":"#ff338a","fill":true,"stack":true,"scale":1,"accrue":false,"dp":1,"plottype":"Line","dataindex":5,"yaxis":1},{"name":"power_points","unit":"Watts","color":"#8860be","fill":true,"stack":true,"scale":1,"accrue":false,"dp":1,"plottype":"Line","dataindex":6,"yaxis":1},{"name":"solar_analytics","unit":"Watts","color":"#ff6100","fill":true,"stack":true,"scale":1,"accrue":false,"dp":1,"plottype":"Line","dataindex":7,"yaxis":1},{"name":"production","unit":"Watts","color":"#fecb4e","fill":true,"stack":false,"scale":1,"accrue":false,"dp":1,"plottype":"Line","dataindex":1,"yaxis":1}],"yaxes":[],"id":"L2B5S2T_.TXT"},{"name":"PVOutput - Show All","period":"today","group":"auto","unit":"Volts","endDate":"2019-10-31T12:21:45.000Z","beginDate":"2019-10-30T13:00:00.000Z","feedlist":[{"name":"voltage","unit":"Volts","color":"#df65b0","fill":false,"stack":false,"scale":"1","accrue":false,"dp":1,"plottype":"Line","dataindex":1,"yaxis":1},{"name":"production","unit":"Watts","color":"#339933","fill":false,"stack":false,"scale":1,"accrue":false,"dp":1,"plottype":"Line","dataindex":2,"yaxis":2},{"name":"consumption","unit":"Watts","color":"#ff5165","fill":false,"stack":false,"scale":1,"accrue":false,"dp":1,"plottype":"Line","dataindex":3,"yaxis":2},{"name":"production","unit":"Wh","color":"#ccff67","fill":true,"stack":false,"scale":1,"accrue":true,"dp":1,"plottype":"Line","dataindex":4,"yaxis":3},{"name":"consumption","unit":"Wh","color":"#ff94a1","fill":true,"stack":false,"scale":1,"accrue":true,"dp":1,"plottype":"Line","dataindex":5,"yaxis":3}],"yaxes":[{"unit":"Volts","min":"0","max":""}],"id":"UZW9AP_E.TXT"},{"name":"PVOutput - Net Generation","period":"today","group":"auto","unit":"Wh","endDate":"2019-10-31T12:28:00.000Z","beginDate":"2019-10-30T13:00:00.000Z","feedlist":[{"name":"grid_import","unit":"Wh","color":"#ff5065","fill":false,"stack":false,"scale":1,"accrue":true,"dp":1,"plottype":"Line","dataindex":1,"yaxis":1},{"name":"grid_export","unit":"Wh","color":"#fcd116","fill":false,"stack":false,"scale":1,"accrue":true,"dp":1,"plottype":"Line","dataindex":2,"yaxis":1},{"name":"mains_inverted","unit":"Watts","color":"#0274df","fill":false,"stack":false,"scale":1,"accrue":false,"dp":1,"plottype":"Line","dataindex":3,"yaxis":2},{"name":"mains_inverted","unit":"Wh","color":"#9bc1e2","fill":true,"stack":false,"scale":1,"accrue":true,"dp":1,"plottype":"Line","dataindex":4,"yaxis":1}],"yaxes":[{"unit":"Volts","min":"0","max":""},{"unit":"Watts","min":"-8000","max":"8000"},{"unit":"Wh","min":"-40000","max":"40000"}],"id":"YQCWTKJ6.TXT"},{"name":"PVOutput - Show Extended Data","period":"today","group":"auto","unit":"Watts","endDate":"2019-10-31T12:36:20.000Z","beginDate":"2019-10-30T13:00:00.000Z","feedlist":[{"name":"production","unit":"Watts","color":"#339934","fill":false,"stack":false,"scale":1,"accrue":false,"dp":1,"plottype":"Line","dataindex":7,"yaxis":1},{"name":"air_conditioners","unit":"Watts","color":"#b22838","fill":true,"stack":false,"scale":1,"accrue":false,"dp":1,"plottype":"Line","dataindex":1,"yaxis":1},{"name":"hot_water_system","unit":"Watts","color":"#165fbb","fill":true,"stack":false,"scale":1,"accrue":false,"dp":1,"plottype":"Line","dataindex":2,"yaxis":1},{"name":"lights","unit":"Watts","color":"#31a670","fill":true,"stack":false,"scale":1,"accrue":false,"dp":1,"plottype":"Line","dataindex":3,"yaxis":1},{"name":"oven","unit":"Watts","color":"#ff338a","fill":true,"stack":false,"scale":1,"accrue":false,"dp":1,"plottype":"Line","dataindex":4,"yaxis":1},{"name":"power_points","unit":"Watts","color":"#8860be","fill":true,"stack":false,"scale":1,"accrue":false,"dp":1,"plottype":"Line","dataindex":5,"yaxis":1},{"name":"solar_analytics","unit":"Watts","color":"#ff6100","fill":true,"stack":false,"scale":1,"accrue":false,"dp":1,"plottype":"Line","dataindex":6,"yaxis":1}],"yaxes":[],"id":"QKOIUHNG.TXT"},{"name":"Solar.web - Energy Balance","period":"today","group":"auto","unit":"Watts","endDate":"2019-10-31T12:43:20.000Z","beginDate":"2019-10-30T13:00:00.000Z","feedlist":[{"name":"grid_export","unit":"Watts","color":"#aed582","fill":true,"stack":false,"scale":1,"accrue":false,"dp":1,"plottype":"Line","dataindex":1,"yaxis":1},{"name":"grid_import_neg","unit":"Watts","color":"#ef5351","fill":true,"stack":false,"scale":1,"accrue":false,"dp":1,"plottype":"Line","dataindex":2,"yaxis":1}],"yaxes":[],"id":"VIRKRSX-.TXT"},,"id":"DTBM932W.TXT"},{"name":"Free","period":"today","group":"auto","unit":"Watts","endDate":"2019-11-06T22:40:40.000Z","beginDate":"2019-11-06T13:00:00.000Z","feedlist":[{"name":"free","unit":"Wh","color":"#3a87fe","fill":true,"stack":false,"scale":1,"accrue":true,"dp":1,"plottype":"Line","dataindex":1,"yaxis":1},{"name":"free","unit":"Watts","color":"#96d35f","fill":true,"stack":false,"scale":1,"accrue":false,"dp":1,"plottype":"Line","dataindex":2,"yaxis":2}],"yaxes":[],"id":"9BGH-2J1.TXT"}]

All the graphs appear to be listed except one named “Everything”, and where I assume that graph should appear, the data shows: "id":"VIRKRSX-.TXT"},,"id":"DTBM932W.TXT"} (double comma, invalid JSON?)

I tried reloading a few more times, with graphs still missing. Then I went back to http://iotawatt.home and navigated back into Graph+ and it worked. Here’s the complete output of /graph/getall which now does include “Everything”:

[{"name":"Solar Analytics - Consumption","period":"today","group":"auto","unit":"Watts","endDate":"2019-10-31T10:47:05.000Z","beginDate":"2019-10-30T13:00:00.000Z","feedlist":[{"name":"air_conditioners","unit":"Watts","color":"#b22838","fill":true,"stack":false,"scale":1,"accrue":false,"dp":1,"plottype":"Line","dataindex":2,"yaxis":1},{"name":"hot_water_system","unit":"Watts","color":"#165fbb","fill":true,"stack":false,"scale":1,"accrue":false,"dp":1,"plottype":"Line","dataindex":3,"yaxis":1},{"name":"lights","unit":"Watts","color":"#31a670","fill":true,"stack":false,"scale":1,"accrue":false,"dp":1,"plottype":"Line","dataindex":4,"yaxis":1},{"name":"oven","unit":"Watts","color":"#ff338a","fill":true,"stack":false,"scale":1,"accrue":false,"dp":1,"plottype":"Line","dataindex":5,"yaxis":1},{"name":"power_points","unit":"Watts","color":"#8860be","fill":true,"stack":false,"scale":1,"accrue":false,"dp":1,"plottype":"Line","dataindex":6,"yaxis":1},{"name":"solar_analytics","unit":"Watts","color":"#ff6100","fill":true,"stack":false,"scale":1,"accrue":false,"dp":1,"plottype":"Line","dataindex":7,"yaxis":1},{"name":"production","unit":"Watts","color":"#fecb4e","fill":true,"stack":false,"scale":1,"accrue":false,"dp":1,"plottype":"Line","dataindex":1,"yaxis":1}],"yaxes":[],"id":"HWYRIBX2.TXT"},{"name":"Solar Analytics - Consumption (stacked)","period":"today","group":"auto","unit":"Watts","endDate":"2019-10-31T10:47:05.000Z","beginDate":"2019-10-30T13:00:00.000Z","feedlist":[{"name":"air_conditioners","unit":"Watts","color":"#b22838","fill":true,"stack":true,"scale":1,"accrue":false,"dp":1,"plottype":"Line","dataindex":2,"yaxis":1},{"name":"hot_water_system","unit":"Watts","color":"#165fbb","fill":true,"stack":true,"scale":1,"accrue":false,"dp":1,"plottype":"Line","dataindex":3,"yaxis":1},{"name":"lights","unit":"Watts","color":"#31a670","fill":true,"stack":true,"scale":1,"accrue":false,"dp":1,"plottype":"Line","dataindex":4,"yaxis":1},{"name":"oven","unit":"Watts","color":"#ff338a","fill":true,"stack":true,"scale":1,"accrue":false,"dp":1,"plottype":"Line","dataindex":5,"yaxis":1},{"name":"power_points","unit":"Watts","color":"#8860be","fill":true,"stack":true,"scale":1,"accrue":false,"dp":1,"plottype":"Line","dataindex":6,"yaxis":1},{"name":"solar_analytics","unit":"Watts","color":"#ff6100","fill":true,"stack":true,"scale":1,"accrue":false,"dp":1,"plottype":"Line","dataindex":7,"yaxis":1},{"name":"production","unit":"Watts","color":"#fecb4e","fill":true,"stack":false,"scale":1,"accrue":false,"dp":1,"plottype":"Line","dataindex":1,"yaxis":1}],"yaxes":[],"id":"L2B5S2T_.TXT"},{"name":"PVOutput - Show All","period":"today","group":"auto","unit":"Volts","endDate":"2019-10-31T12:21:45.000Z","beginDate":"2019-10-30T13:00:00.000Z","feedlist":[{"name":"voltage","unit":"Volts","color":"#df65b0","fill":false,"stack":false,"scale":"1","accrue":false,"dp":1,"plottype":"Line","dataindex":1,"yaxis":1},{"name":"production","unit":"Watts","color":"#339933","fill":false,"stack":false,"scale":1,"accrue":false,"dp":1,"plottype":"Line","dataindex":2,"yaxis":2},{"name":"consumption","unit":"Watts","color":"#ff5165","fill":false,"stack":false,"scale":1,"accrue":false,"dp":1,"plottype":"Line","dataindex":3,"yaxis":2},{"name":"production","unit":"Wh","color":"#ccff67","fill":true,"stack":false,"scale":1,"accrue":true,"dp":1,"plottype":"Line","dataindex":4,"yaxis":3},{"name":"consumption","unit":"Wh","color":"#ff94a1","fill":true,"stack":false,"scale":1,"accrue":true,"dp":1,"plottype":"Line","dataindex":5,"yaxis":3}],"yaxes":[{"unit":"Volts","min":"0","max":""}],"id":"UZW9AP_E.TXT"},{"name":"PVOutput - Net Generation","period":"today","group":"auto","unit":"Wh","endDate":"2019-10-31T12:28:00.000Z","beginDate":"2019-10-30T13:00:00.000Z","feedlist":[{"name":"grid_import","unit":"Wh","color":"#ff5065","fill":false,"stack":false,"scale":1,"accrue":true,"dp":1,"plottype":"Line","dataindex":1,"yaxis":1},{"name":"grid_export","unit":"Wh","color":"#fcd116","fill":false,"stack":false,"scale":1,"accrue":true,"dp":1,"plottype":"Line","dataindex":2,"yaxis":1},{"name":"mains_inverted","unit":"Watts","color":"#0274df","fill":false,"stack":false,"scale":1,"accrue":false,"dp":1,"plottype":"Line","dataindex":3,"yaxis":2},{"name":"mains_inverted","unit":"Wh","color":"#9bc1e2","fill":true,"stack":false,"scale":1,"accrue":true,"dp":1,"plottype":"Line","dataindex":4,"yaxis":1}],"yaxes":[{"unit":"Volts","min":"0","max":""},{"unit":"Watts","min":"-8000","max":"8000"},{"unit":"Wh","min":"-40000","max":"40000"}],"id":"YQCWTKJ6.TXT"},{"name":"PVOutput - Show Extended Data","period":"today","group":"auto","unit":"Watts","endDate":"2019-10-31T12:36:20.000Z","beginDate":"2019-10-30T13:00:00.000Z","feedlist":[{"name":"production","unit":"Watts","color":"#339934","fill":false,"stack":false,"scale":1,"accrue":false,"dp":1,"plottype":"Line","dataindex":7,"yaxis":1},{"name":"air_conditioners","unit":"Watts","color":"#b22838","fill":true,"stack":false,"scale":1,"accrue":false,"dp":1,"plottype":"Line","dataindex":1,"yaxis":1},{"name":"hot_water_system","unit":"Watts","color":"#165fbb","fill":true,"stack":false,"scale":1,"accrue":false,"dp":1,"plottype":"Line","dataindex":2,"yaxis":1},{"name":"lights","unit":"Watts","color":"#31a670","fill":true,"stack":false,"scale":1,"accrue":false,"dp":1,"plottype":"Line","dataindex":3,"yaxis":1},{"name":"oven","unit":"Watts","color":"#ff338a","fill":true,"stack":false,"scale":1,"accrue":false,"dp":1,"plottype":"Line","dataindex":4,"yaxis":1},{"name":"power_points","unit":"Watts","color":"#8860be","fill":true,"stack":false,"scale":1,"accrue":false,"dp":1,"plottype":"Line","dataindex":5,"yaxis":1},{"name":"solar_analytics","unit":"Watts","color":"#ff6100","fill":true,"stack":false,"scale":1,"accrue":false,"dp":1,"plottype":"Line","dataindex":6,"yaxis":1}],"yaxes":[],"id":"QKOIUHNG.TXT"},{"name":"Solar.web - Energy Balance","period":"today","group":"auto","unit":"Watts","endDate":"2019-10-31T12:43:20.000Z","beginDate":"2019-10-30T13:00:00.000Z","feedlist":[{"name":"grid_export","unit":"Watts","color":"#aed582","fill":true,"stack":false,"scale":1,"accrue":false,"dp":1,"plottype":"Line","dataindex":1,"yaxis":1},{"name":"grid_import_neg","unit":"Watts","color":"#ef5351","fill":true,"stack":false,"scale":1,"accrue":false,"dp":1,"plottype":"Line","dataindex":2,"yaxis":1}],"yaxes":[],"id":"VIRKRSX-.TXT"},{"name":"Everything","period":"today","group":"auto","unit":"Watts","endDate":"2019-11-12T23:55:55.000Z","beginDate":"2019-11-12T13:00:00.000Z","feedlist":[{"name":"voltage","unit":"Volts","color":"#df65b0","fill":false,"stack":false,"scale":1,"accrue":false,"dp":1,"plottype":"Line","dataindex":1,"yaxis":1},{"name":"free","unit":"Watts","color":"#d6d6d6","fill":false,"stack":false,"scale":1,"accrue":false,"dp":1,"plottype":"Line","dataindex":2,"yaxis":2},{"name":"self_consumption","unit":"Watts","color":"#000000","fill":false,"stack":false,"scale":1,"accrue":false,"dp":1,"plottype":"Line","dataindex":3,"yaxis":2},{"name":"lights","unit":"Watts","color":"#31a670","fill":true,"stack":true,"scale":1,"accrue":false,"dp":1,"plottype":"Line","dataindex":4,"yaxis":2},{"name":"oven","unit":"Watts","color":"#ff338a","fill":true,"stack":true,"scale":1,"accrue":false,"dp":1,"plottype":"Line","dataindex":5,"yaxis":2},{"name":"power_points","unit":"Watts","color":"#8860be","fill":true,"stack":true,"scale":1,"accrue":false,"dp":1,"plottype":"Line","dataindex":6,"yaxis":2},{"name":"hot_water_system","unit":"Watts","color":"#165fbb","fill":true,"stack":true,"scale":1,"accrue":false,"dp":1,"plottype":"Line","dataindex":7,"yaxis":2},{"name":"air_conditioners","unit":"Watts","color":"#b22838","fill":true,"stack":true,"scale":1,"accrue":false,"dp":1,"plottype":"Line","dataindex":8,"yaxis":2},{"name":"solar_analytics","unit":"Watts","color":"#ff6100","fill":true,"stack":true,"scale":1,"accrue":false,"dp":1,"plottype":"Line","dataindex":9,"yaxis":2},{"name":"production","unit":"Watts","color":"#fecb4e","fill":true,"stack":false,"scale":1,"accrue":false,"dp":1,"plottype":"Line","dataindex":10,"yaxis":2},{"name":"consumption","unit":"Wh","color":"#ff94a1","fill":true,"stack":false,"scale":1,"accrue":true,"dp":1,"plottype":"Line","dataindex":11,"yaxis":3},{"name":"production","unit":"Wh","color":"#ccff68","fill":true,"stack":false,"scale":1,"accrue":true,"dp":1,"plottype":"Line","dataindex":12,"yaxis":3}],"yaxes":[{"unit":"Volts","min":"0","max":""}],"id":"DTBM932W.TXT"},{"name":"Free","period":"today","group":"auto","unit":"Watts","endDate":"2019-11-06T22:40:40.000Z","beginDate":"2019-11-06T13:00:00.000Z","feedlist":[{"name":"free","unit":"Wh","color":"#3a87fe","fill":true,"stack":false,"scale":1,"accrue":true,"dp":1,"plottype":"Line","dataindex":1,"yaxis":1},{"name":"free","unit":"Watts","color":"#96d35f","fill":true,"stack":false,"scale":1,"accrue":false,"dp":1,"plottype":"Line","dataindex":2,"yaxis":2}],"yaxes":[],"id":"9BGH-2J1.TXT"}]
5

Here is the contents of the graphs directory:

image

5x jsn files, but I have 8 saved graphs. I did have some old graphs (from before Graph+, and not this many) but these were “lost” and I recreated them from scratch. So not sure why there are only 5x jsn files, and why 8x txt files.

6

Also, I just navigated to http://iotawatt.home then through to graphs again, in a new tab, and graphs were missing right away. So I don’t think it matters if I reload the page directly or navigate to it from the home page.

7

Quick answer to that is that I appear to have had it backwards: old graphs .jsn, Graph+ .txt

8

Looking at the code, it would appear that there was a transient problem with the file. The contents should be between those two commas. So the question is why? Could use your help to narrow this down since you seem to be able to recreate it and know how to use the tools.

The files appear in the response Json array in the same order they appear in the file-manager directory list - the directory order. When the list will not load, you can uise a tool like Json lint to quickly find the missing file. It’s the one named

, , “id”: "DTBM932W.TX

So what I would like to know is if it’s always the same file. If it is, could you delete it and then recreate it with a subtly different name and see if the problem goes away. You don’t need to recreate the graph, it’s just a rename because the context of the graph remains after you delete and is subsequently saved again under the new name.

9

I believe it is always the “Everything” graph that is missing. At least, every time I have looked at the content of the /graph/getall response when graphs are missing (a few times already), that one has been missing.

I just compared the contents of all the txt files and that one graph is by far the longest at 2063 characters. The next closest in length is 1286 characters.

Although, I don’t see why the length would be changing and sometimes work and sometimes not, if the length was a factor.

I can’t see a way to delete or rename files. I can only see an option to upload or create files?

10

Nevermind. I Figured out how to delete. After I deleted the file, all my (other) graphs appeared to be working (but it’s an intermittent issue, so hard to say).

I renamed the first letter of the file to “a” and re-uploaded it. Then my graphs went missing again. Then they appeared, then after reloading a few more times they went missing again.

Here’s the /graph/getall response when graphs are missing:

[{"name":"Solar Analytics - Consumption","period":"today","group":"auto","unit":"Watts","endDate":"2019-10-31T10:47:05.000Z","beginDate":"2019-10-30T13:00:00.000Z","feedlist":[{"name":"air_conditioners","unit":"Watts","color":"#b22838","fill":true,"stack":false,"scale":1,"accrue":false,"dp":1,"plottype":"Line","dataindex":2,"yaxis":1},{"name":"hot_water_system","unit":"Watts","color":"#165fbb","fill":true,"stack":false,"scale":1,"accrue":false,"dp":1,"plottype":"Line","dataindex":3,"yaxis":1},{"name":"lights","unit":"Watts","color":"#31a670","fill":true,"stack":false,"scale":1,"accrue":false,"dp":1,"plottype":"Line","dataindex":4,"yaxis":1},{"name":"oven","unit":"Watts","color":"#ff338a","fill":true,"stack":false,"scale":1,"accrue":false,"dp":1,"plottype":"Line","dataindex":5,"yaxis":1},{"name":"power_points","unit":"Watts","color":"#8860be","fill":true,"stack":false,"scale":1,"accrue":false,"dp":1,"plottype":"Line","dataindex":6,"yaxis":1},{"name":"solar_analytics","unit":"Watts","color":"#ff6100","fill":true,"stack":false,"scale":1,"accrue":false,"dp":1,"plottype":"Line","dataindex":7,"yaxis":1},{"name":"production","unit":"Watts","color":"#fecb4e","fill":true,"stack":false,"scale":1,"accrue":false,"dp":1,"plottype":"Line","dataindex":1,"yaxis":1}],"yaxes":[],"id":"HWYRIBX2.TXT"},{"name":"Solar Analytics - Consumption (stacked)","period":"today","group":"auto","unit":"Watts","endDate":"2019-10-31T10:47:05.000Z","beginDate":"2019-10-30T13:00:00.000Z","feedlist":[{"name":"air_conditioners","unit":"Watts","color":"#b22838","fill":true,"stack":true,"scale":1,"accrue":false,"dp":1,"plottype":"Line","dataindex":2,"yaxis":1},{"name":"hot_water_system","unit":"Watts","color":"#165fbb","fill":true,"stack":true,"scale":1,"accrue":false,"dp":1,"plottype":"Line","dataindex":3,"yaxis":1},{"name":"lights","unit":"Watts","color":"#31a670","fill":true,"stack":true,"scale":1,"accrue":false,"dp":1,"plottype":"Line","dataindex":4,"yaxis":1},{"name":"oven","unit":"Watts","color":"#ff338a","fill":true,"stack":true,"scale":1,"accrue":false,"dp":1,"plottype":"Line","dataindex":5,"yaxis":1},{"name":"power_points","unit":"Watts","color":"#8860be","fill":true,"stack":true,"scale":1,"accrue":false,"dp":1,"plottype":"Line","dataindex":6,"yaxis":1},{"name":"solar_analytics","unit":"Watts","color":"#ff6100","fill":true,"stack":true,"scale":1,"accrue":false,"dp":1,"plottype":"Line","dataindex":7,"yaxis":1},{"name":"production","unit":"Watts","color":"#fecb4e","fill":true,"stack":false,"scale":1,"accrue":false,"dp":1,"plottype":"Line","dataindex":1,"yaxis":1}],"yaxes":[],"id":"L2B5S2T_.TXT"},{"name":"PVOutput - Show All","period":"today","group":"auto","unit":"Volts","endDate":"2019-10-31T12:21:45.000Z","beginDate":"2019-10-30T13:00:00.000Z","feedlist":[{"name":"voltage","unit":"Volts","color":"#df65b0","fill":false,"stack":false,"scale":"1","accrue":false,"dp":1,"plottype":"Line","dataindex":1,"yaxis":1},{"name":"production","unit":"Watts","color":"#339933","fill":false,"stack":false,"scale":1,"accrue":false,"dp":1,"plottype":"Line","dataindex":2,"yaxis":2},{"name":"consumption","unit":"Watts","color":"#ff5165","fill":false,"stack":false,"scale":1,"accrue":false,"dp":1,"plottype":"Line","dataindex":3,"yaxis":2},{"name":"production","unit":"Wh","color":"#ccff67","fill":true,"stack":false,"scale":1,"accrue":true,"dp":1,"plottype":"Line","dataindex":4,"yaxis":3},{"name":"consumption","unit":"Wh","color":"#ff94a1","fill":true,"stack":false,"scale":1,"accrue":true,"dp":1,"plottype":"Line","dataindex":5,"yaxis":3}],"yaxes":[{"unit":"Volts","min":"0","max":""}],"id":"UZW9AP_E.TXT"},{"name":"PVOutput - Net Generation","period":"today","group":"auto","unit":"Wh","endDate":"2019-10-31T12:28:00.000Z","beginDate":"2019-10-30T13:00:00.000Z","feedlist":[{"name":"grid_import","unit":"Wh","color":"#ff5065","fill":false,"stack":false,"scale":1,"accrue":true,"dp":1,"plottype":"Line","dataindex":1,"yaxis":1},{"name":"grid_export","unit":"Wh","color":"#fcd116","fill":false,"stack":false,"scale":1,"accrue":true,"dp":1,"plottype":"Line","dataindex":2,"yaxis":1},{"name":"mains_inverted","unit":"Watts","color":"#0274df","fill":false,"stack":false,"scale":1,"accrue":false,"dp":1,"plottype":"Line","dataindex":3,"yaxis":2},{"name":"mains_inverted","unit":"Wh","color":"#9bc1e2","fill":true,"stack":false,"scale":1,"accrue":true,"dp":1,"plottype":"Line","dataindex":4,"yaxis":1}],"yaxes":[{"unit":"Volts","min":"0","max":""},{"unit":"Watts","min":"-8000","max":"8000"},{"unit":"Wh","min":"-40000","max":"40000"}],"id":"YQCWTKJ6.TXT"},{"name":"PVOutput - Show Extended Data","period":"today","group":"auto","unit":"Watts","endDate":"2019-10-31T12:36:20.000Z","beginDate":"2019-10-30T13:00:00.000Z","feedlist":[{"name":"production","unit":"Watts","color":"#339934","fill":false,"stack":false,"scale":1,"accrue":false,"dp":1,"plottype":"Line","dataindex":7,"yaxis":1},{"name":"air_conditioners","unit":"Watts","color":"#b22838","fill":true,"stack":false,"scale":1,"accrue":false,"dp":1,"plottype":"Line","dataindex":1,"yaxis":1},{"name":"hot_water_system","unit":"Watts","color":"#165fbb","fill":true,"stack":false,"scale":1,"accrue":false,"dp":1,"plottype":"Line","dataindex":2,"yaxis":1},{"name":"lights","unit":"Watts","color":"#31a670","fill":true,"stack":false,"scale":1,"accrue":false,"dp":1,"plottype":"Line","dataindex":3,"yaxis":1},{"name":"oven","unit":"Watts","color":"#ff338a","fill":true,"stack":false,"scale":1,"accrue":false,"dp":1,"plottype":"Line","dataindex":4,"yaxis":1},{"name":"power_points","unit":"Watts","color":"#8860be","fill":true,"stack":false,"scale":1,"accrue":false,"dp":1,"plottype":"Line","dataindex":5,"yaxis":1},{"name":"solar_analytics","unit":"Watts","color":"#ff6100","fill":true,"stack":false,"scale":1,"accrue":false,"dp":1,"plottype":"Line","dataindex":6,"yaxis":1}],"yaxes":[],"id":"QKOIUHNG.TXT"},{"name":"Solar.web - Energy Balance","period":"today","group":"auto","unit":"Watts","endDate":"2019-10-31T12:43:20.000Z","beginDate":"2019-10-30T13:00:00.000Z","feedlist":[{"name":"grid_export","unit":"Watts","color":"#aed582","fill":true,"stack":false,"scale":1,"accrue":false,"dp":1,"plottype":"Line","dataindex":1,"yaxis":1},{"name":"grid_import_neg","unit":"Watts","color":"#ef5351","fill":true,"stack":false,"scale":1,"accrue":false,"dp":1,"plottype":"Line","dataindex":2,"yaxis":1}],"yaxes":[],"id":"VIRKRSX-.TXT"},,"id":"ATBM932W.TXT"},{"name":"Free","period":"today","group":"auto","unit":"Watts","endDate":"2019-11-06T22:40:40.000Z","beginDate":"2019-11-06T13:00:00.000Z","feedlist":[{"name":"free","unit":"Wh","color":"#3a87fe","fill":true,"stack":false,"scale":1,"accrue":true,"dp":1,"plottype":"Line","dataindex":1,"yaxis":1},{"name":"free","unit":"Watts","color":"#96d35f","fill":true,"stack":false,"scale":1,"accrue":false,"dp":1,"plottype":"Line","dataindex":2,"yaxis":2}],"yaxes":[],"id":"9BGH-2J1.TXT"}]

Same error at the same point, with the new filename.

11

Not in the file manager, in Graph+. When you load a saved graph, a delete button will appear under the name. Clicking that will delete the file, then type a new name and save to save it under a new name - viola! renamed.

But hold on. It’s altogether possible that this has something to do with the size of these files and the response to getall. Everybody thinks the poor little 8266 is an AWS server. You are probably choking it. The response to the getall request is built in a String class variable that needs contiguous heap. Everytime something is added, it reallocates and copies. Moreover, the file itself is copied into heap before adding to the Json in the String buffer. Is there any smoke coming out of the holes on the sides?

Seriously, there are a lot of heroics in there to manage things like parsing the config and producing the responses to queries. This code to return the graphs does not do anything fancy. There are some things that can be done to reduce the buffer duplication, but that only puts off the inevitable. I suspect some of thos WDT and other exceptions you are seeing may be collateral damage of this and similar overload issues.

The solution is to only return the names of the saved graphs and read the actual context file when you select it. It’s only a fraction of a second and solves the problem. I’ll do that in the next release. In the meantime, take it easy.

12

I have 8 graphs, with the biggest file being 2063 bytes and the total of all 8 files being 8942 bytes. Free heap is about 18880 (bytes?) at the moment, but it does vary slightly between 17-19k while watching for a few seconds just now.

Would stopping emoncms or influxdb services help?

UPDATE: I watched Free heap in the status page while reloading /graph/getall in a loop (manual) in another browser. I didn’t see any significant change to free heap, but it could be happening faster than the status page refresh. I saw most of the time the “Everything” graph data was in the response, but a few times it was not.

13

The web server handles one request at a time, so heap will dip while handling the graph/get all but no status requests would be serviced so it would go unobserved.

Stopping Emoncms or influx will release a little bit, but not a huge amount. The issue is that the String buffer needs to be contiguous, and when a new file is appended, it is reallocated and copied.

Just moving the buffer to an xbuf would allow using heap fragments and eliminate the copying to extend. But as I said, that’s a bandaid. The solution is to load the files as needed.

1 Like