Release 02_05_00 ALPHA auto-update

overeasy · October 5, 2019, 8:22pm

This release is now ALPHA auto-update. It contains the new Graph+ visualization application. The original graph module is still available, but is deprecated.

The new graph unlocks the full power of the IoTaWatt datalog and improves on the old capability in several key areas:

No longer need to define outputs to plot other units of measure like Amps, VA, Hz, PF.
Many predefined time windows that recognize true day, week, month and year boundaries, including daylight-time changes.
Data can be grouped by hour, week, month and year regardless of daylight-time change, number of days in month, or leap-year.
Sorting, stack and fill, and Wh accrual.

For more information see the documentation page.

For Alpha users, the default documentation at doc.iotawatt.com is for release 02_04_02. You can select 02_05_00 in the lower left, or just go directly to https://docs.iotawatt.com/en/02_05_00/

Here’s a busy sample of the a new Graph+ plot:

frogmore · October 6, 2019, 3:27pm

Looks really good in your picture. What has been your experience in its performance?

I checked and one of mine is set to alpha and it updated to 02_05_00, so I was excited to try it out. My experience was less than great. I got one graph of one channel. It looked good. I asked it to zoom out so I could see more. It looked like nothing happened, so I wasn’t sure I actually touched the control. Then several seconds later the graph updated, but the time resolution changed (since it was on auto, this wasn’t really unexpected). I then told it to move back and zoom in. That seemed to confuse it, so I reloaded the page. That didn’t help. Then I went to the status page and it was blank. So probably not quite ready for prime time, but that is why it is in alpha

frogmore · October 6, 2019, 4:00pm

Looks like the web server stopped responding. Will have to go down and look at the lights and possibly reboot it. It is responding to pings.

overeasy · October 6, 2019, 4:48pm

Performance is roughly the same as the original graph. Reading the datalog is what takes time and both require the same effort. Roughly one to two seconds for a plot with 720 groups. Querying less data as in a daily groups for a month is very fast with only 30 or so reads.

I’ve been using the query for several months now and it has been working fine, so if you have a failure case, I’d like details.

frogmore · October 6, 2019, 5:44pm

What details would you like?
It looks like the web server went down. It might be unrelated. I think a reboot is what is needed. I will take a look at the lights to see what they say. What should they be?

frogmore · October 6, 2019, 6:12pm

Went to the basement to take a look.
The light was steady green, so no error reported there, but also not good.

I rebooted it and it looked better, flashing green. I looked at the log, but it did not indicate any errors. I then tried the graph again and it stopped responding again.

I then tried to take out the SD card, but it would not come out. Not sure if it is a push, push or push, pull slot. It looks like it should be a push, push one. I decided to just push it back in and started it again. Now it is working fine.

So it looks like it might have had a flakey connection to the SD card. The graph is working fine and it shows missed data for the time after the first hang till my last reboot.

The new graph looks good. No feedback on it yet, need to spend more time with it. I do have some on the overall experience of using the UI.

I don’t like that I can’t use a URL to get to the view of the main page that I want, eg the status page view.
There should be a consistent way to get to the home page of the device. If I got to graph page from the home page I can usually use the back button. But if I got there by typing in the URL, back will take me back someplace else.

overeasy · October 6, 2019, 6:14pm

If you can’t communicate with it, then note the LED state, restart, then post the message log. If you can recreate the problem, then a some screenshots of the query that hangs would be helpful.

I have tried zooming out and in a variety of scenarios and works fine.

frogmore · October 6, 2019, 6:23pm

I’ll keep an eye out for issues, but I suspect the SD card was not totally accessable to the code, probably due to not being fully seated. It looks like this caused the code to hang, or perhaps it did try to write to the log about the failure which caused another failure, which caused it to write to the log about the failure …

The LED was solid green, so it was not updating that, since normal is flickering green. Not sure what causes the the LED to go red or other colors. Log indicates the WiFi signal is around -70. It is pretty close (next room in basement) to the router, but it did have a reported event a few days ago with disconnect and reconnect.

Anyway, it looks fine right now. The UI for the graph does not give great feedback on the button presses, but it now responds in a few seconds, so acceptable.

overeasy · October 6, 2019, 6:31pm

That’s not a scenario that I would consider plausible. Can you post the log such as it is?

frogmore · October 6, 2019, 10:25pm

Here is the file manager view (there are some found files)

Here is the log. Note that this is all of the log that shows up. My guess is the found files might contain more of the history of the log.:

8/09/19 22:32:03 HTTP server started
8/09/19 22:32:03 WiFi connected. SSID=FrogmoreN66, IP=192.168.0.38, channel=6, RSSI -77db
8/09/19 22:32:03 timeSync: service started.
8/09/19 22:32:06 statService: started.
8/09/19 22:32:06 Updater: service started. Auto-update class is ALPHA
8/09/19 22:32:06 dataLog: service started.
8/09/19 22:32:08 dataLog: Last log entry 08/09/19 22:32:00
8/09/19 22:32:08 historyLog: service started.
8/09/19 22:32:09 historyLog: Last log entry 08/09/19 22:32:00
8/09/19 22:32:10 Updater: Auto-update is current for class ALPHA.
8/20/19 01:56:27 Updater: Invalid response from server. HTTPcode: -4
8/20/19 06:56:41 Updater: Update from 02_04_00 to 02_04_02
8/20/19 06:56:41 Updater: download 02_04_02
8/20/19 06:56:50 Updater: Release downloaded 9323ms, size 734984
8/20/19 06:57:03 Updater: Update downloaded and signature verified
8/20/19 06:57:14 Updater: firmware upgraded to version 02_04_02
8/20/19 06:57:14 Firmware updated, restarting.

** Restart **

SD initialized.
8/20/19 13:57:25z Real Time Clock is running. Unix time 1566309445
8/20/19 13:57:25z Reset reason: Software/System restart
8/20/19 13:57:25z Trace: 1:2[4], 9:0[4], 9:0, 9:1, 8:4, 8:6, 8:8, 8:9, 9:3, 9:5, 9:9, 1:2, 1:3, 1:4, 1:5[19], 1:6, 1:1[4], 1:2[5], 9:0[5], 9:0, 9:1, 8:4, 8:6, 8:8, 8:9, 9:3, 9:5, 9:9, 1:2, 1:3, 1:4, 1:5[5]
8/20/19 13:57:25z ESP8266 ChipID: 2222239
8/20/19 13:57:25z IoTaWatt 4.x, Firmware version 02_04_02
8/20/19 13:57:25z Updater: Installing update files for version 02_04_02
8/20/19 13:57:25z Updater: Installing EDIT.HTM
8/20/19 13:57:25z Updater: Installing GRAPH.HTM
8/20/19 13:57:26z Updater: Installing GRAPH.JS
8/20/19 13:57:27z Updater: Installing GRAPH2.HTM
8/20/19 13:57:28z Updater: Installing GRAPH2.JS
8/20/19 13:57:29z Updater: Installing INDEX.HTM
8/20/19 13:57:30z Updater: Installing TABLES.TXT
8/20/19 13:57:31z Updater: Installing CNFSTYLE.CSS
8/20/19 13:57:31z Updater: Installation complete.
8/20/19 13:57:31z SPIFFS mounted.
8/20/19 06:57:33 Local time zone: -8:00
8/20/19 06:57:33 Using Daylight Saving Time (BST) when in effect.
8/20/19 06:57:33 device name: IotaWatt
8/20/19 06:57:33 MDNS responder started for hostname IotaWatt
8/20/19 06:57:33 LLMNR responder started for hostname IotaWatt
8/20/19 06:57:33 HTTP server started
8/20/19 06:57:33 WiFi connected. SSID=FrogmoreN66, IP=192.168.0.38, channel=6, RSSI -70db
8/20/19 06:57:33 timeSync: service started.
8/20/19 06:57:33 statService: started.
8/20/19 06:57:33 Updater: service started. Auto-update class is ALPHA
8/20/19 06:57:33 dataLog: service started.
8/20/19 06:57:35 dataLog: Last log entry 08/20/19 06:56:50
8/20/19 06:57:35 Updater: Auto-update is current for class ALPHA.
8/20/19 06:58:33 historyLog: service started.
8/20/19 06:58:34 historyLog: Last log entry 08/20/19 06:56:00
9/13/19 00:53:17 Updater: Invalid response from server. HTTPcode: -4
9/13/19 01:02:09 WiFi disconnected.
9/13/19 01:02:13 WiFi connected. SSID=FrogmoreN66, IP=192.168.0.38, channel=6, RSSI -71db
9/13/19 01:58:23 WiFi disconnected.
9/13/19 01:58:26 WiFi connected. SSID=FrogmoreN66, IP=192.168.0.38, channel=6, RSSI -70db
9/13/19 02:12:44 WiFi disconnected.
9/13/19 02:12:47 WiFi connected. SSID=FrogmoreN66, IP=192.168.0.38, channel=6, RSSI -70db
9/13/19 02:23:42 WiFi disconnected.
9/13/19 02:23:46 WiFi connected. SSID=FrogmoreN66, IP=192.168.0.38, channel=6, RSSI -71db
9/13/19 04:09:31 WiFi disconnected.
9/13/19 04:09:34 WiFi connected. SSID=FrogmoreN66, IP=192.168.0.38, channel=6, RSSI -70db
9/16/19 07:10:57 WiFi disconnected.
9/16/19 07:11:00 WiFi connected. SSID=FrogmoreN66, IP=192.168.0.38, channel=6, RSSI -66db
9/21/19 19:13:19 WiFi disconnected.
9/21/19 19:13:22 WiFi connected. SSID=FrogmoreN66, IP=192.168.0.38, channel=6, RSSI -70db
9/25/19 14:14:45 WiFi disconnected.
9/25/19 14:14:48 WiFi connected. SSID=FrogmoreN66, IP=192.168.0.38, channel=6, RSSI -68db
10/01/19 07:16:15 timeSync: Six week routine restart.

** Restart **

SD initialized.
10/01/19 14:16:16z Real Time Clock is running. Unix time 1569939376
10/01/19 14:16:16z Reset reason: Software/System restart
10/01/19 14:16:16z Trace: 9:0, 9:1, 8:4, 8:6, 8:8, 8:9, 9:3, 9:5, 9:9, 1:2, 1:3, 1:4, 1:5[19], 1:6, 1:1[8], 1:2[9], 9:0[9], 9:0, 9:1, 8:4, 8:6, 8:8, 8:9, 9:3, 9:5, 9:9, 1:2, 1:3, 1:4, 1:5[20], 20:0, 20:1
10/01/19 14:16:16z ESP8266 ChipID: 2222239
10/01/19 14:16:16z IoTaWatt 4.x, Firmware version 02_04_02
10/01/19 14:16:16z SPIFFS mounted.
10/01/19 07:16:17 Local time zone: -8:00
10/01/19 07:16:17 Using Daylight Saving Time (BST) when in effect.
10/01/19 07:16:17 device name: IotaWatt
10/01/19 07:16:17 MDNS responder started for hostname IotaWatt
10/01/19 07:16:17 LLMNR responder started for hostname IotaWatt
10/01/19 07:16:17 HTTP server started
10/01/19 07:16:17 WiFi connected. SSID=FrogmoreN66, IP=192.168.0.38, channel=6, RSSI -76db
10/01/19 07:16:17 timeSync: service started.
10/01/19 07:16:17 statService: started.
10/01/19 07:16:17 Updater: service started. Auto-update class is ALPHA
10/01/19 07:16:17 dataLog: service started.
10/01/19 07:16:19 dataLog: Last log entry 10/01/19 07:16:15
10/01/19 07:16:19 Updater: Auto-update is current for class ALPHA.
10/01/19 07:17:17 historyLog: service started.
10/01/19 07:17:18 historyLog: Last log entry 10/01/19 07:16:00
10/01/19 17:17:29 Updater: Invalid response from server. HTTPcode: -11
10/05/19 13:26:37 Updater: Update from 02_04_02 to 02_05_00
10/05/19 13:26:37 Updater: download 02_05_00
10/05/19 13:26:45 Updater: Release downloaded 7280ms, size 758448
10/05/19 13:26:57 Updater: signature verified
10/05/19 13:27:08 Updater: firmware upgraded to version 02_05_00
10/05/19 13:27:08 Updater: Firmware updated, restarting.

** Restart **

SD initialized.
10/05/19 20:27:19z Real Time Clock is running. Unix time 1570307239
10/05/19 20:27:19z Reset reason: Software/System restart
10/05/19 20:27:19z Trace: 9:0, 9:1, 8:4, 8:6, 8:8, 8:9, 9:3, 9:5, 9:9, 1:2, 1:3, 1:4, 1:5[19], 1:6, 1:1[7], 1:2[8], 9:0[8], 9:0, 9:1, 8:4, 8:6, 8:8, 8:9, 9:3, 9:5, 9:9, 1:2, 1:3, 1:4, 1:5[5], 5:0, 5:7
10/05/19 20:27:19z ESP8266 ChipID: 2222239
10/05/19 20:27:19z IoTaWatt 4.x, Firmware version 02_05_00
10/05/19 20:27:19z Updater: Installing update files for version 02_05_00
10/05/19 20:27:19z Updater: Installing GRAPH.JS
10/05/19 20:27:20z Updater: Installing GRAPH2.HTM
10/05/19 20:27:21z Updater: Installing GRAPH2.JS
10/05/19 20:27:22z Updater: Installing INDEX.HTM
10/05/19 20:27:24z Updater: Installing TABLES.TXT
10/05/19 20:27:24z Updater: Installing CNFSTYLE.CSS
10/05/19 20:27:25z Updater: Installing EDIT.HTM
10/05/19 20:27:25z Updater: Installing GRAPH.HTM
10/05/19 20:27:26z Updater: Installation complete.
10/05/19 20:27:26z SPIFFS mounted.
10/05/19 13:27:27 Local time zone: -8:00
10/05/19 13:27:27 Using Daylight Saving Time (BST) when in effect.
10/05/19 13:27:27 device name: IotaWatt
10/05/19 13:27:27 MDNS responder started for hostname IotaWatt
10/05/19 13:27:27 LLMNR responder started for hostname IotaWatt
10/05/19 13:27:27 HTTP server started
10/05/19 13:27:27 WiFi connected. SSID=FrogmoreN66, IP=192.168.0.38, channel=6, RSSI -74db
10/05/19 13:27:28 timeSync: service started.
10/05/19 13:27:28 statService: started.
10/05/19 13:27:28 Updater: service started. Auto-update class is ALPHA
10/05/19 13:27:28 dataLog: service started.
10/05/19 13:27:30 dataLog: Last log entry 10/05/19 13:26:45
10/05/19 13:27:30 Updater: Auto-update is current for class ALPHA.
10/05/19 13:28:28 historyLog: service started.
10/05/19 13:28:29 historyLog: Last log entry 10/05/19 13:26:00

** Restart **

SD initialized.
10/06/19 17:51:27z Real Time Clock is running. Unix time 1570384287
10/06/19 17:51:27z Power failure detected.
10/06/19 17:51:27z Reset reason: External System
10/06/19 17:51:27z ESP8266 ChipID: 2222239
10/06/19 17:51:27z IoTaWatt 4.x, Firmware version 02_05_00
10/06/19 17:51:27z SPIFFS mounted.
10/06/19 10:51:28 Local time zone: -8:00
10/06/19 10:51:28 Using Daylight Saving Time (BST) when in effect.
10/06/19 10:51:28 device name: IotaWatt
10/06/19 10:51:31 MDNS responder started for hostname IotaWatt
10/06/19 10:51:31 LLMNR responder started for hostname IotaWatt
10/06/19 10:51:31 HTTP server started
10/06/19 10:51:31 WiFi connected. SSID=FrogmoreN66, IP=192.168.0.38, channel=6, RSSI -73db
10/06/19 10:51:31 timeSync: service started.
10/06/19 10:51:31 statService: started.
10/06/19 10:51:31 Updater: service started. Auto-update class is ALPHA
10/06/19 10:51:31 dataLog: service started.
10/06/19 10:51:33 dataLog: Last log entry 10/06/19 08:18:15
10/06/19 10:51:33 Updater: Auto-update is current for class ALPHA.

** Restart **

SD initialized.
10/06/19 17:53:17z Real Time Clock is running. Unix time 1570384397
10/06/19 17:53:17z Power failure detected.
10/06/19 17:53:17z Reset reason: External System
10/06/19 17:53:17z ESP8266 ChipID: 2222239
10/06/19 17:53:17z IoTaWatt 4.x, Firmware version 02_05_00
10/06/19 17:53:17z SPIFFS mounted.
10/06/19 10:53:18 Local time zone: -8:00
10/06/19 10:53:18 Using Daylight Saving Time (BST) when in effect.
10/06/19 10:53:18 device name: IotaWatt
10/06/19 10:53:21 MDNS responder started for hostname IotaWatt
10/06/19 10:53:21 LLMNR responder started for hostname IotaWatt
10/06/19 10:53:21 HTTP server started
10/06/19 10:53:21 WiFi connected. SSID=FrogmoreN66, IP=192.168.0.38, channel=6, RSSI -70db
10/06/19 10:53:21 timeSync: service started.
10/06/19 10:53:21 statService: started.
10/06/19 10:53:21 Updater: service started. Auto-update class is ALPHA
10/06/19 10:53:22 dataLog: service started.
10/06/19 10:53:23 dataLog: Last log entry 10/06/19 10:51:50
10/06/19 10:53:23 Updater: Auto-update is current for class ALPHA.
10/06/19 10:54:21 historyLog: service started.
10/06/19 10:54:22 historyLog: Last log entry 10/06/19 08:18:00

Looking at my other one, it has a lot more stuff it the file manager UI. But, its log also does not start from the very beginning or even the start of a boot cycle. I believe what is happening is that the file system on the SD card gets corruption. This might have happened during one of the outages over the last several months. When a file name gets unusual characters in it, the file manager gets confused and does not display it. I am pretty sure it is still using the original SD card it came with and SD cards are known to not be that reliable, so that is probably part of the problem.

overeasy · October 6, 2019, 11:20pm

OK, lets unpack this.

The file system list is suspicious. The underlying ESP8266 does not have any code to access an SD card. All of the SD access is through the IoTaWatt firmware. There is no facility for creating that found.000 directory or it’s contents. I will need to look in the file-manager application to see if there is anything there. It looks to me as if that would have been created while the SDcard was mounted in another device. Is that possible? if so, what and when?

I know there is a school of thought that SDcards are unreliable. That’s not my experience with thousands of units out there. There was a particular release over a year ago that was susceptible to file corruption on restrart, but that was diagnosed and fixed. Less than a dozen users reported the problem, and there is now also code to detect and repair some of that damage, even though the original issue is now precluded by design.

By far, the most common problem with the SDcard is removal or insertion while the unit is powered up, and that includes adjusting it while the unit is powered up. You indicated that you reseated the SDcard. Was the unit powered up while you did that? If so, there could well be structural damage to the SDcard structures but they are not a fault of the SDcard.

When you display the message log through the configuration app, it only displays the last 100K of the file. You can see the &textpos=-100000 in the URL. You can remove that from the URL and get the whole file, or change it to -200000 or some other value. But the log you posted covers operation for about the last 2 months. There were no outages during that period.

All that said, your unit appears to have stopped logging from 08:18:15 to the restart at 10:51:31 today. Given what you reported, I suspect that the new Query was stuck in a loop during that period. It would be helpful to have a screenshot of Graph+ when something like this happens, either to you or anyone else.

frogmore · October 7, 2019, 1:54am

Okay, I remember now (had to search the history). This is the same device that had issues back in October 2018. I ended up taking out the SD and putting it in my PC. It detected corruption and I let it fix it. I also remember seeing it had a funny looking name on one of the files, which is what was causing the issue. It also looks like I never reported the issue before.

I pull the card tonight after shutting down the device and give you a proper look at what is there. It is most likely not related to 02_05_00, so I will start a new thread for the issue.