Lockup after power failure?


#1

Hi @overeasy,

Hoping you could have a look at this. I expereienced my first supplier power failure since the iotawatt was installed and it did not seem to like it, From what i can tell the power was maybe 15 minutes? When I got home, i noticed when i opened my iPhone that it went to the Iotawatt captive portal, it scanned for a wifi point, and I connected it to my home wifi…

It takes about 3 minutes for my wifi to come back up after a failure.

It did not show back up so i thought a full re-config would be required. Went to the iotawatt and the LED was red, so i power cycled it and it started green flickering recording data.

Went to the computer and the portal was active, and all my settings were still there, however there is no data from the time of the power failure, to me power cycling it.

(In the log, 1057 would have been when i saw the captive portal, and 1130 is when i power cycled the iotawatt)

Below is the pasted log:

SD initialized.
12/05/18 21:43:52z Real Time Clock is running. Unix time 1544046232
12/05/18 21:43:52z Power failure detected.
12/05/18 21:43:52z Version 02_03_18
12/05/18 21:43:52z Reset reason: Power on
12/05/18 21:43:52z Trace: 195:68[204], 138:23[254], 187:131[70], 101:239[25], 242:96[168], 101:210[205], 221:210[246], 64:117[202], 118:68[95], 202:91[189], 189:92[225], 225:93[171], 209:111[16], 229:171[255], 90:152[121], 207:104[175], 93:116[128], 200:145[143], 226:164[38], 192:185[219], 210:32[72], 92:127[152], 180:183[44], 149:133[207], 253:151[254], 171:145[126], 171:67[73], 128:89[154], 93:129[26], 228:127[183], 124:201[100], 10:219[205]
12/05/18 21:43:52z ESP8266 ChipID: 2546248
12/05/18 21:43:52z SPIFFS mounted.
12/06/18 08:43:53 Local time zone: 11
12/06/18 08:43:53 device name: IotaWatt, version: 3
12/06/18 08:43:56 Connecting with WiFiManager.
12/06/18 10:57:34 No WiFi connection.
12/06/18 10:57:35 HTTP server started
12/06/18 10:57:35 timeSync: service started.
12/06/18 10:57:35 statService: started.
12/06/18 10:57:35 dataLog: service started.
12/06/18 10:57:35 dataLog: Last log entry 12/6/18 08:43:45
12/06/18 10:57:35 historyLog: service started.
12/06/18 10:57:35 historyLog: Last log entry 12/6/18 08:43:00
12/06/18 10:57:40 EmonService: started. url:80=emoncms.org, node=IotaWatt, interval=5, encrypted
12/06/18 10:57:40 influxDB: started, url=192.168.1.153:8086, db=grafana, interval=5

** Restart **

SD initialized.
12/06/18 00:31:14z Real Time Clock is running. Unix time 1544056274
12/06/18 00:31:14z Power failure detected.
12/06/18 00:31:14z Version 02_03_18
12/06/18 00:31:14z Reset reason: External System
12/06/18 00:31:14z Trace: 195:196[92], 8:23[222], 155:131[78], 101:230[25], 115:97[168], 103:210[205], 221:210[246], 80:117[203], 102:68[71], 74:123[191], 157:220[225], 225:253[233], 209:47[16], 101:179[255], 26:144[113], 199:232[175], 221:116[128], 200:153[171], 226:164[54], 193:185[219], 210:34[88], 92:127[208], 180:191[44], 145:165[207], 244:151[159], 170:149[254], 175:99[89], 162:89[155], 93:145[95], 164:126[183], 124:233[4], 10:219[205]
12/06/18 00:31:14z ESP8266 ChipID: 2546248
12/06/18 00:31:14z SPIFFS mounted.
12/06/18 11:31:15 Local time zone: 11
12/06/18 11:31:15 device name: IotaWatt, version: 3
12/06/18 11:31:18 Connecting with WiFiManager.
12/06/18 11:31:21 MDNS responder started for hostname IotaWatt
12/06/18 11:31:21 LLMNR responder started for hostname IotaWatt
12/06/18 11:31:21 HTTP server started
12/06/18 11:31:21 timeSync: service started.
12/06/18 11:31:21 statService: started.
12/06/18 11:31:21 WiFi connected. SSID=Home12874, IP=192.168.1.46, channel=11, RSSI -85db
12/06/18 11:31:21 Updater: service started. Auto-update class is MINOR
12/06/18 11:31:21 dataLog: service started.
12/06/18 11:31:21 dataLog: Last log entry 12/6/18 11:31:05
12/06/18 11:31:21 historyLog: service started.
12/06/18 11:31:21 historyLog: Last log entry 12/6/18 11:31:00
12/06/18 11:31:23 Updater: Auto-update is current for class MINOR.
12/06/18 11:31:26 EmonService: started. url:80=emoncms.org, node=IotaWatt, interval=5, encrypted
12/06/18 11:31:26 influxDB: started, url=192.168.1.153:8086, db=grafana, interval=5
12/06/18 11:31:27 EmonService: Start posting at 12/6/18 08:43:40
12/06/18 11:31:28 influxDB: Start posting at 12/6/18 08:43:40

My local time is Melbourne, Australia, which is GMT +11 hours at the moment,

also here is my missing data block:
26

He is also a shot from emoncms.org showing the data seems to have got stuck:

Thanks.


#2

Very good detailed description. Thanks.

This is the second time I’ve seen this. It seems to be a recent thing. On the surface, it appears that the WiFi Manager - a service by another contributor that I include to do the WiFi configuration - is blocking indefinitely until it gets an AP connection. It is supposed to time out and things move along without the WiFi which is not necessary after initial configuration and setting the RTC. I think there was a newer version recently. I’ll check that out and regress in the next release if so.

I’ll have to look deeper into this. These types of problems are very time consuming because you need to forst figure out the conditions to recreate them, fix it, then prove it’s fixed. I think I would prefer to just write a new AP connection routine that I can control.

But this is not very common, so I’m wondering if there are other factors that might be involved:

Were there any devices around at the time of the power failure that might have been used previously to connect to the AP, or that might just connect to any AP on repowering? Was your phone there?

I’ll take a look at this as soon as I get this PVoutput to ALPH, which I hope to do in the next day or two. Sorry for the loss of data. That should not happen.


#3

Thanks. No problem about the loss of data, just checking if it was something I did.

Phone was within 3 metres of the iotawatt at time of power failure, but I wasn’t home at the time. I have plenty of things connect to the same wifi point that all would have been connecting when the power came back.

Any more info always happy to help.

Thanks.
Michael.


#4

That may be significant, thanks. I’ll check with the other user and see if that might be a common thread. It might be that the after the power failure, the IoTaWatt came back on the air before the WiFi router, and your phone connected to it. Then they both just kind of sat there waiting for the other to do something.


#5

@overeasy and @mobile0408 I have also seen this type of behavior but thought it was more just my unique situation due to the time devices take to come back up after a power failure. Long story short, If our home loses power, my home firewall takes 10+ minutes to get up and operational. This security device is also my home DHCP and reserved IP server, so none of my devices (including my AP) receive an IP until after the 10 minute boot. Understanding that this is such a long time, I assumed the IoTaWatt just timed out and I would go to the basement and pull the power and plug it back in. After that, all good. I know this boot time is excessive, so I never brought it up as power is rather reliable where I live. Just letting you know that is never seemed to recover itself, but required manual intervention. In my case, not an issue as I can often just go downstair and correct it. I can see this being an issue for people with remote deployments.


#6

Same question for you, is your phone or some other d3vice picking up and connecting to the IoTaWatt AP before the WiFi comes up?


#7

No sir. No wireless devices can get a connection to anything until my Firewall comes up and hands out the DHCP (reserved IP) to my AP. I have a few IP Cameras that have static IPs assigned and a few Raspberry Pis also set to static, but most devices are set to receive a dynamic IP from the firewall/DHCP server. No other active APs or devices that would hand out such on my network.

Just for clarity, when I reboot the IoTaWatt after the firewall is up and operational all is good and keeps working. So, it does seem to be an issue with timeout waiting for an IP which 10 minutes is nearly an eternity to an ESP device.


#8

Let’s try that again. When there is a power failure, the firewall goes down as does the IoTaWatt. Things like phones, laptops etc, do not. When the power comes back on, it takes awhile for the firewall to come back and broadcast it’s SSID (or not) and open up for business. I get that.

Meanwhile, the IoTaWatt comes back after only a few seconds, and seeing that it’s configured SSID is not available, activates AP mode and itself becomes an access point.

Now if any of those phones or laptops were previously connected to the IoTaWatt during the initial WiFi setup, where they were connected to that AP SSID, they could connect to the IoTaWatt AP because their ordinary SSID isn’t available.

That’s the situation I’m looking into as the WiFi Manager, which is a black box to me, may now be blocking until it can consummate it’s relationship with the phone or laptop, which it never does.

The question: Is that possible?


#9

Sorry to complicate things or not being clear in my post. I was attempting to just interject some info that may be helpful in troubleshooting. I did not intend to confuse even more or take from this thread.

When we have a power failure, all devices apart from laptops (running on their own battery) go down until the power is restored. This could be any length of time from minutes to days; you know snow in the NE. The hardware firewall I have is also my DHCP server so my AP (Ubiquity) does not get its assigned RESERVED IP until the firewall is fully operational; which takes about 10 minutes once power is restored. So, when there is a power outage, we simply wait for the firewall to come before attempting to reach the internet or other internal device. All my wireless devices seem to be able to recover (laptops, Alexa, phones, pads, etc.) Wired systems with a hard-coded static IP can communicate on the LAN in an outage, but anything that depends on DHCP has to wait for the firewall to come alive first. I have my DHCP provide assigned reserved IPs for devices based on MAC address for security policy and auditing.

I do not attempt to connect to the IotAWatt during an outage knowing it will not be available until the firewall is operational and can hand out IPs via my wireless to the various devices (such as the IoTaWatt). So, I assume the IoTaWatt times out waiting for the AP and DHCP service. I often do not connect directly to the device, but I can see that in my EmonCMS that the device is no longer feeding data to my local RBP+ with Emon. When I go down and pull the two plugs from the IoTaWatt, the system starts back up once the firewall/DHCP is working. Things just continue where they left off once again.

So, in my situation, I’m not saying that the unit stops recording energy info or goes into the initial AP setup mode, but it simply does not recover from the timeout and will not send data to my local EMonCMS until I physically reboot it; not a reset but a simple reboot. Again, not an issue but wanted you to be aware if this is a simular issue a few others have been stating.


#10

What I’m saying is that while your firewall is out, the IoTaWatt IS an AP, and if you used a phone or laptop to initially connect to the IoTaWatt during installation, it may still have that network configured and connect to it.

If I drop my regular WiFi and power up just about any IoTaWatt that I have ever configured to WiFi with my iPad, My iPad will connect to it’s AP - not through the main WiFi, but directly to the IoTaWatt which is in AP mode. I have to “forget” the SSID of a particular IoTaWatt to stop that from happening.

No matter, I’ll explore this with my equipment.


#11

@overeasy, now I understand what you are saying here in your previous post. I setup the IoTaWatt using my phone, but I have never look at it or the IoTaWatt during an outage to see if the two try to connect. The device is in the basement and my phone us often with me on the third floor so it may be hard to pair at that distance. Not sure how much the ESP provides here as an AP and range. If it is an extended outage, I simply turn off wireless on the phone and work via cell. I can make sure that I “Forget” the device’s AP from when I initially set it up and get back to you with any results. Again, not here to take over the OPs original post or question. Just wanted to provide some context that I though may be helpful or give some insight.

Loving the product and thanks for all the hard work and support. Looking forward to PVOutput when it is GA and the visibility I can glean from the device is amazing.