Updater: download 02_04_00 failed

brettbeeson · April 5, 2019, 6:32am

Hi! I’ve recently installed two units. They have run well until 2019-04-03T00:00:00+10:00 when samples were regularly lost. This occurred in the on-board logs and the influx database. I think the unit was trying and failing to download 02_04_00, which maybe took up too many resources?

I solved this by changing update class to “None” and restarting. This has fixed the problem. I have many other IotaWatts and only these two have a problem. The WiFi is crappy for these two.

I don’t need any “fix”, but thought I’d let you know.

See attached for the logs. I’ve removed repeated errors.

See attached for the before (intermittent data recorded) and after the restart (10s data recorded).

iotamsgs.txt (3.7 KB)

overeasy · April 5, 2019, 11:44am

Thanks for the heads up. The new release download usually takes about 5 seconds in North America where the server is located. The IoTaWatt blocks for the entire time. This is not supposed to be a frequent event.

I took a look at the code, and will change it to increase the time between attempts after several consecutive failures. Maybe go every hour, then once a day. Need to understand the failure mode better to decide that.

Since you have more IoTaWatt installed, would it be possible to take a look at how the others handled that download and how long it took? They should have all been around the same time on the same day. I’m curious if this is an Australia problem or a local internet connection problem. The WiFi signal strength for this unit doesn’t look that bad.

There is a bit of a catch-22 here when I make this change, as you will need to update the units to get the change.

brettbeeson · April 6, 2019, 11:06pm

All ten other units are reporting fine. I’ve checked one log - see below - showing the update.

Here is some more information on the units which failed to download to assist:

Two new units were installed on the same WiFi network
Simple setup, only a few circuits each.
WiFi network is known as “crappy” - this is just from the users. The signal strength is ok. I think they refer to dropout, reboots. I had no problems on site with a mobile (“cell”) phone and a laptop.
Since the units behaved the same, I assume it’s a WiFi - related problem.
Upstream of the WiFi, it’s a council (i.e. city government) site before the internet, which may have firewalls / filtering / etc. Maybe the download URL is blocked?

I’ll be on site again in the coming weeks and am happy to run any tests.

This is a successful log from a nearby IotaWatt on a different network:

3/12/19 11:40:59 influxDB: Start posting at 03/12/19 10:07:50
4/03/19 00:32:57 Updater: Update from 02_03_21 to 02_04_00
4/03/19 00:32:57 Updater: download 02_04_00
4/03/19 00:33:31 Updater: Release downloaded 33877ms, size 670392
4/03/19 00:33:39 Updater: Update downloaded and signature verified
4/03/19 00:33:45 Updater: firmware upgraded to version 02_04_00
4/03/19 00:33:45 Firmware updated, restarting.

** Restart **

SD initialized.
4/02/19 14:33:53z Real Time Clock is running. Unix time 1554215633 
4/02/19 14:33:53z Reset reason: Software/System restart
4/02/19 14:33:53z Trace:  1:2[7], 9:0[7], 9:0, 9:1, 8:4, 8:6, 8:8, 8:9, 9:3, 9:5, 9:9, 1:2, 1:3, 1:4, 1:5[19], 1:6, 1:1[7], 1:2[8], 9:0[8], 9:0, 9:1, 8:4, 8:6, 8:8, 8:9, 9:3, 9:5, 9:9, 1:2, 1:3, 1:4, 1:5[5]
4/02/19 14:33:53z ESP8266 ChipID: 6910648
4/02/19 14:33:53z IoTaWatt 4.x, Firmware version 02_04_00
4/02/19 14:33:53z Updater: Installing update files for version 02_04_00
4/02/19 14:33:53z Updater: Installing GRAPH.JS
4/02/19 14:33:53z Updater: Installing INDEX.HTM
4/02/19 14:33:54z Updater: Installing TABLES.TXT
4/02/19 14:33:54z Updater: Installing CNFSTYLE.CSS
4/02/19 14:33:54z Updater: Installing EDIT.HTM
4/02/19 14:33:55z Updater: Installing GRAPH.HTM
4/02/19 14:33:55z Updater: Installation complete.
4/02/19 14:33:55z SPIFFS mounted.
4/03/19 00:33:56 Local time zone: +10:00
4/03/19 00:33:56 device name: iotaa39
4/03/19 00:33:56 MDNS responder started for hostname iotaa39
4/03/19 00:33:56 LLMNR responder started for hostname iotaa39
4/03/19 00:33:56 HTTP server started
4/03/19 00:33:56 WiFi connected. SSID=NetComm 0405, IP=10.1.1.7, channel=11, RSSI -79db
4/03/19 00:33:56 timeSync: service started.
4/03/19 00:33:57 statService: started.
4/03/19 00:33:57 Updater: service started. Auto-update class is MINOR
4/03/19 00:33:57 dataLog: service started.
4/03/19 00:33:57 dataLog: Last log entry 04/03/19 00:33:30
4/03/19 00:33:57 historyLog: service started.
4/03/19 00:33:58 historyLog: Last log entry 04/03/19 00:33:00
4/03/19 00:33:58 Updater: Auto-update is current for class MINOR.
4/03/19 00:34:01 influxDB: started, url=live.phisaver.com:8086, db=iotaa39, interval=10
4/03/19 00:34:03 influxDB: Start posting at 04/03/19 00:33:00
4/04/19 10:48:48 WiFi disconnected.

overeasy · April 7, 2019, 2:14am

Even the successful download was 33 seconds, which is a constant problem with internet down there. I’m thinking the problem here is the internet link and not the WiFi. If cellular works ok there, it might work to do a hotspot with a phone to update the firmware. It’s a pita to switch networks and then back, but maybe the only way to get there. If you decide to do that, remember that the passwords for the AP mode will be the new device names.

brettbeeson · April 8, 2019, 6:06am

Fortunately this is a temporary install, so turning off the updates for the duration (a few months) is not a problem. If it’s helpful, I’ll try accessing http://iotawatt.com/firmware/iotaupdt.php from a laptop connected to that network next time I’m there, to simulate an update request.

overeasy · April 8, 2019, 1:29pm

That’s probably not going to provide much information. iotaupdt.php is just a script that responds to the iotawatt’s query about the latest release version for it’s auto-update class. Just accessing it is a single HTTP transaction, and without the HTTP headers that are part of that, it will respond with 403 anyway.

To attempt download of the actual release file use
iotawatt.com/firmware/bin/02_04_00.bin

brettbeeson · May 5, 2019, 11:30am

I was onsite yesterday and downloaded that file (02_04_00.bin) on a PC via the same WiFi network. It downloaded about in a second or two.

Since we’re in the middle of a energy efficiency trial, I didn’t want to test IotaWatt with an update.

So it’s unclear what the problem was, or even if it is resolved now. I’ll revisit (i.e. try to download update on the IotaWatts) in a few weeks when the trial complete.

overeasy · May 5, 2019, 2:52pm

Things have changed. 27 days ago the file was downloading from a server in theUSA. Over the past week, I’ve moved the server and also added Cloudflare, which should be caching that release file in multiple sites globally. Downloading in a “second or two” is encouraging, and I am optimistic that updates will work better in distant corners of the world.