I think I may have a bad IoTaWatt - RESOLVED

I received my IoTaWatt last Monday and the next day I connected it to wifi and set it up. I’m thrilled with the features and functionality of the IoTaWatt. My issue is that the unit keeps resetting at random times, sometimes losing all data. I’ve since configured influxdb and grafana, so I’m not losing history other than the gaps during which it seems to be offline.

I thought it might be a power issue because I realized that I used the power supply from a Raspberry PI instead of the one that came with it. Yesterday I swapped it out with the one in the box but it didn’t make a difference. The one I used at the beginning was a 5v 2A supply vs the 600mA supplied one so that shouldn’t have been it. It has reset since then, so I’m ruling out the power supply as the issue.

This is connected in my basement approximately 30 feet from the wifi access point. I noticed on some of the messages that the RSSI was as low as -79db, so I moved the unit a bit higher and also increased the power output on my access point. After that, the log reset messages indicated that the RSSI was -69db to -67db so I don’t think wifi connectivity is the issue.

Today, it went offline and was pingable but would not accept connections on port 80 at all – it would just respond with a TCP RST. It stayed that way for hours. I got home this evening and power cycled it and it has started working again for now. Before power cycling it, I noticed that the LED was blinking green, red, red.

My questions are:

Is there a console/uart port that I might be able to get more details from?

Is there a tool to help debug what the reset trace is telling me?

It seems like this is using the ESP8266 and I have several NodeMCU boards from other projects. I haven’t disassembled the IoTaWatt, but if there is a NodeMCU board inside, would it be possible to flash firmware onto one and swap it in as a troubleshooting step?

I’m very excited about the IoTaWatt and am willing to try anything to get this to be stable. Any help would be appreciated.

Thanks,

-David

** Restart **

SD initialized.
10/06/19 05:58:44z Real Time Clock is running. Unix time 1570341524
10/06/19 05:58:44z Reset reason: Software Watchdog
10/06/19 05:58:44z Trace:  9:9, 1:2, 1:3, 1:4, 1:5[7], 7:0, 7:7, 7:7, 7:7, 7:7, 7:7, 7:7, 1:6, 1:1[1], 1:2[2], 9:0[2], 9:0, 9:1, 8:4, 8:6, 8:8, 8:9, 9:3, 9:5, 9:9, 1:2, 1:3, 1:4, 1:5[7], 7:0, 7:7, 7:7
10/06/19 05:58:44z ESP8266 ChipID: 2526429
10/06/19 05:58:44z IoTaWatt 5.0, Firmware version 02_04_02
10/06/19 05:58:44z SPIFFS mounted.
10/06/19 05:58:45z Local time zone: +0:00
10/06/19 05:58:45z device name: IotaWatt
10/06/19 05:58:45z MDNS responder started for hostname IotaWatt
10/06/19 05:58:45z LLMNR responder started for hostname IotaWatt
10/06/19 05:58:45z HTTP server started
10/06/19 05:58:45z timeSync: service started.
10/06/19 05:58:45z statService: started.
10/06/19 05:58:45z Updater: service started. Auto-update class is MINOR
10/06/19 05:58:45z dataLog: service started.
10/06/19 05:58:45z IotaLog: file damaged iotawatt/iotalog.log

10/06/19 05:58:45z IotaLog: Creating diagnostic file.
10/06/19 05:58:49z IotaLog: Deleting iotawatt/iotalog.log and restarting.


** Restart **

SD initialized.
10/06/19 05:58:49z Real Time Clock is running. Unix time 1570341529
10/06/19 05:58:49z Reset reason: Software/System restart
10/06/19 05:58:49z Trace:  5:0, 5:4, 1:6, 1:1[6], 1:2[7], 9:0[7], 9:0, 9:1, 8:4, 8:6, 8:8, 8:9, 9:3, 9:5, 9:9, 1:2, 1:3, 1:4, 1:5[19], 1:6, 1:1[7], 1:2, 9:0, 9:0, 8:4, 8:6, 8:8, 8:9, 1:2, 1:3, 1:4, 1:5[19]
10/06/19 05:58:49z ESP8266 ChipID: 2526429
10/06/19 05:58:49z IoTaWatt 5.0, Firmware version 02_04_02
10/06/19 05:58:49z SPIFFS mounted.
10/06/19 05:58:50z Local time zone: +0:00
10/06/19 05:58:50z device name: IotaWatt
10/06/19 05:58:50z MDNS responder started for hostname IotaWatt
10/06/19 05:58:50z LLMNR responder started for hostname IotaWatt
10/06/19 05:58:50z HTTP server started
10/06/19 05:58:50z timeSync: service started.
10/06/19 05:58:50z statService: started.
10/06/19 05:58:50z dataLog: service started.
10/06/19 05:58:50z dataLog: New current log created.
10/06/19 05:58:50z dataLog: Last history entry: 10/06/19 05:57:00
10/06/19 05:58:54z WiFi connected. SSID=TheDivas, IP=10.0.1.83, channel=11, RSSI -67db
10/06/19 05:58:54z Updater: service started. Auto-update class is MINOR
10/06/19 05:58:55z influxDB: started, url=10.0.1.10:8086, db=iotawatt, interval=10
10/06/19 05:58:55z Updater: Auto-update is current for class MINOR.
10/06/19 05:58:57z influxDB: Start posting at 10/06/19 05:57:50
10/06/19 05:59:50z historyLog: service started.
10/06/19 05:59:50z historyLog: Last log entry 10/06/19 05:57:00

Your IotaWatt is set to UTC time, so the times in the message log may not be your local time. In any event,
today at 5:58z, your system restarted because of a Software Watchdog while in the influxDB service. A second later a problem with the datalog was found. A diagnostic file was produced and the current log was deleted.

The unit restarted and a new current log was created. That was at 5:59z today. Looking at thetrouble shooting guide at docs.iotawatt.com the GRR led code means the SDcard was not working.

There were some issues with bad cards during this last manufacturing run, and it may be that your card is defective. If you are running now, it may be best to see what happens before acting on that.

If the unit continues to reset, there may be a problem associated with your influxDB configuration, but I think this is all SD related. Let’s see what happens now that the datalog has been rebuilt. (The history log should still have your previous data at 60 second intervals).

I appreciate your willingness to apply heroic measures to diagnosing and fixing these problems, but the IoTaWatt is typically very stable and has plenty of diagnostic and remedial features built in.

The software is completely open on Github, so if you want to understand the trace, you can familiarize yourself with the code and it will become obvious. Right now, these problems are straightforward and I think worst case will only require a new SDcard. What was your order number?

Thank you for the prompt response. That is great news that it may simply be the SD card. If this continues to reset I will replace the card and see if that fixes the issue.

I haven’t had a chance to dig into the software, but I will take a look at that when I get some time.

I don’t think it is the influxdb configuration because this started happening before I had influxdb configured. In fact, it was the loss of history that pushed me to set up influxdb so I could have an external record of the power/energy data.

Also, my order number was 66790.

Thanks again!

-David

I just wanted to circle back to say that replacing the micro SD card 100% fixed the issue. My IoTaWatt has been rock solid for four days now, so I’m calling it fixed.

Thanks,

-David

2 Likes

Got my IotaWatt today ( order number is 104323 ) and eagerly started setting it up.
Connected to my network and assigned a static IP no problems.
Configure the voltage transformer, and 4 CT’s
After about 14 minutes the web GUI (firefox) became unresponsive and eventually the page timed out.
Very strange.
Check and my router says that the IotaWatt is connected okay but I cannot get a GUI response.
Power cycled the IotaWatt, and after it boots up, it is again accessible.
Couple of minutes later, no dice.
Reboot, okay for a few minutes and then nothing.

Went searching this forum and found this thread.

Had a spare sd card, so copied the files over from the supplied card to my card, inserted, rebooted and so far… 1 hour later, all seems good.
Looks like I got a faulty card as well :thinking:

The troubleshooting section details how to interpret the LED status when your IoTaWatt has problems. That information, along with the message log, is what I would need to diagnose this problem.

If you continue to have issues. Please note the LED state and post the message log contents @fter a restart.

As I said,
I replaced the SD card and it has been running solidly since then.

The green light was dimly lit and solid

Trace: 93:181[238], 150:121[203], 149:186[248], 169:112[222], 4:200[174], 251:112[189], 237:111[72], 106:63[163], 255:188[250], 175:63[70], 23:101[110], 139:171[248], 147:217[64], 96:62[166], 44:20[136], 206:139[138], 220:30[175], 145:133[14], 146:254[9], 188:16[42], 250:214[252], 244:134[65], 226:63[215], 48:156[121], 13:21[230], 188:153[210], 161:247[164], 98:144[225], 141:187[248], 7:1[156], 230:3[35], 119:176[209]
ESP8266 ChipID: 6145634
IoTaWatt 5.0, Firmware version 02_04_02
SPIFFS mounted.
Local time zone: +0:00
device name: IotaWatt
Connecting with WiFiManager.

** Restart **

SD initialized.
Real Time Clock not initialized.
Reset reason: External System
Trace: 92:181[238], 150:57[201], 151:186[120], 173:112[222], 4:204[174], 123:112[189], 237:109[72], 106:59[160], 255:252[250], 143:63[70], 23:101[238], 139:171[248], 51:217[64], 96:62[166], 44:20[140], 202:139[10], 220:30[175], 209:133[14], 146:255[73], 172:48[40], 250:214[252], 244:132[65], 242:63[151], 48:157[121], 29:21[198], 188:145[210], 161:213[164], 98:128[224], 141:187[249], 39:65[156], 230:3[35], 119:48[209]
ESP8266 ChipID: 6145634
IoTaWatt 5.0, Firmware version 02_04_02
SPIFFS mounted.
Local time zone: +0:00
device name: IotaWatt
Connecting with WiFiManager.
MDNS responder started for hostname IotaWatt
LLMNR responder started for hostname IotaWatt
HTTP server started
WiFi connected. SSID=SpeedyDave, IP=192.168.1.106, channel=1, RSSI -65db
timeSync: service started.
statService: started.
Updater: service started. Auto-update class is MINOR
11/26/19 06:08:44z timeSync: RTC initalized to NTP time
11/26/19 06:08:44z dataLog: service started.
11/26/19 06:08:44z dataLog: New current log created.
11/26/19 06:08:46z Updater: Auto-update is current for class MINOR.
11/26/19 06:10:44z historyLog: service started.
11/26/19 06:10:44z historyLog: first entry 11/26/19 06:09:00

** Restart **

SD initialized.
11/26/19 06:32:33z Real Time Clock is running. Unix time 1574749953
11/26/19 06:32:33z Power failure detected.
11/26/19 06:32:33z Reset reason: External System
11/26/19 06:32:33z ESP8266 ChipID: 6145634
11/26/19 06:32:33z IoTaWatt 5.0, Firmware version 02_04_02
11/26/19 06:32:33z SPIFFS mounted.

** Restart **

SD initialized.
11/26/19 06:32:36z Real Time Clock is running. Unix time 1574749956
11/26/19 06:32:36z Power failure detected.
11/26/19 06:32:36z Reset reason: External System
11/26/19 06:32:36z ESP8266 ChipID: 6145634
11/26/19 06:32:36z IoTaWatt 5.0, Firmware version 02_04_02
11/26/19 06:32:36z SPIFFS mounted.
11/26/19 14:32:37 Local time zone: +8:00
11/26/19 14:32:37 device name: IotaWatt
11/26/19 14:32:40 Connecting with WiFiManager.
11/26/19 14:32:43 MDNS responder started for hostname IotaWatt
11/26/19 14:32:43 LLMNR responder started for hostname IotaWatt
11/26/19 14:32:43 HTTP server started
11/26/19 14:32:43 WiFi connected. SSID=SpeedyDave, IP=192.168.1.106, channel=1, RSSI -56db
11/26/19 14:32:43 timeSync: service started.
11/26/19 14:32:43 statService: started.
11/26/19 14:32:44 Updater: service started. Auto-update class is MINOR
11/26/19 14:32:44 dataLog: service started.
11/26/19 14:32:44 dataLog: Last log entry 11/26/19 14:32:20
11/26/19 14:32:44 historyLog: service started.
11/26/19 14:32:44 historyLog: Last log entry 11/26/19 14:32:00
11/26/19 14:32:45 Updater: Auto-update is current for class MINOR.
11/26/19 14:36:10 WiFi disconnected.
11/26/19 14:36:14 WiFi connected. SSID=SpeedyDave, IP=192.168.1.106, channel=8, RSSI -57db

** Restart **

SD initialized.
11/26/19 07:01:03z Real Time Clock is running. Unix time 1574751663
11/26/19 07:01:03z Power failure detected.
11/26/19 07:01:03z Reset reason: External System
11/26/19 07:01:03z ESP8266 ChipID: 6145634
11/26/19 07:01:03z IoTaWatt 5.0, Firmware version 02_04_02
11/26/19 07:01:03z SPIFFS mounted.
11/26/19 15:01:04 Local time zone: +8:00
11/26/19 15:01:04 device name: IotaWatt
11/26/19 15:01:07 MDNS responder started for hostname IotaWatt
11/26/19 15:01:07 LLMNR responder started for hostname IotaWatt
11/26/19 15:01:07 HTTP server started
11/26/19 15:01:07 WiFi connected. SSID=SpeedyDave, IP=192.168.1.106, channel=8, RSSI -54db
11/26/19 15:01:07 timeSync: service started.
11/26/19 15:01:07 statService: started.
11/26/19 15:01:07 Updater: service started. Auto-update class is MINOR
11/26/19 15:01:07 dataLog: service started.
11/26/19 15:01:07 dataLog: Last log entry 11/26/19 15:00:00
11/26/19 15:01:07 historyLog: service started.
11/26/19 15:01:07 historyLog: Last log entry 11/26/19 15:00:00
11/26/19 15:01:09 Updater: Auto-update is current for class MINOR.

** Restart **

SD initialized.
11/26/19 07:34:13z Real Time Clock is running. Unix time 1574753653
11/26/19 07:34:13z Power failure detected.
11/26/19 07:34:13z Reset reason: External System
11/26/19 07:34:13z ESP8266 ChipID: 6145634
11/26/19 07:34:13z IoTaWatt 5.0, Firmware version 02_04_02
11/26/19 07:34:13z SPIFFS mounted.
11/26/19 15:34:14 Local time zone: +8:00
11/26/19 15:34:14 device name: IotaWatt
11/26/19 15:34:17 MDNS responder started for hostname IotaWatt
11/26/19 15:34:17 LLMNR responder started for hostname IotaWatt
11/26/19 15:34:17 HTTP server started
11/26/19 15:34:17 WiFi connected. SSID=SpeedyDave, IP=192.168.1.106, channel=6, RSSI -55db
11/26/19 15:34:17 timeSync: service started.
11/26/19 15:34:17 statService: started.
11/26/19 15:34:17 Updater: service started. Auto-update class is MINOR
11/26/19 15:34:17 dataLog: service started.
11/26/19 15:34:17 dataLog: Last log entry 11/26/19 15:11:30
11/26/19 15:34:17 historyLog: service started.
11/26/19 15:34:17 historyLog: Last log entry 11/26/19 15:11:00
11/26/19 15:34:19 Updater: Auto-update is current for class MINOR.

From what I can see in the log, there is nothing of note.

When my browser lost the ability to display the IotaWatt GUI, I checked my router and the iotawatt was definitely still connected.
I could not connect either with iotawatt.local or the IP address 192.168.1.106

power down , wait, and then power up the iotawatt allowed me to reload the gui, but it froze out again after only a few minutes.
I went thru this a few times before deciding to try another SD card.

I removed the original SD card, and put it in my PC.
Copied all the files onto a desktop folder.
Instered the replacement SD card and copied across the original files.
Put the replacement card back into the iotawatt and rebooted.
all good and solid since.

Thanks for the diagnostic info. This is clearly not the problem reported in the original post where the LED was blinking GRR.

What I see as significant in the logs is that in every power cycle restart except the last, both the current and history logs were up to date at the time of the restart. At the last restart they appear to be have been up to date until you presumably shut it down and copied the SD card.

So the SD card was working all the time, and the GRR LED indication was not present. Does the voltage input show a credible value during the times when the device was incommunicado?

Not saying it wasn’t the SD, just that this isn’t the same straightforward SD issue in the original post. You have been running OK for eight hours, so that’s good, but the problem had occurred twice since the RTC was successfully set.

So fingers crossed it’s good now, but I would like to caution others reading this post that taking your new device apart and removing the SD card is not recommend until the external symptoms are diagnosed to warrant that action. In this case, while the short-term results indicate success, the real cause remains elusive. My goal isn’t to just get you running, it’s to figure out the problem and preclude recurrence for everyone.

Thanks for the reply.
Yes my light was never GRR, I just jumped onto this thread as it seemed to be pretty close to my issue.

The reason I restarted my unit was because I had lost communication with the configuration app.
When the issue occurred ( first time about 14 minutes up time ) I could “see” it connected in my router GUI, but could not connect with either firefox or EI.

When powered back up, after about a minute, I was able to access the configuration app again in firefox.

I noticed that my previous CT and VT setup was missing (I thought maybe i didn’t hit the save button ) and so I reconfigured the VT and 4 CT’s, making sure that I hit the save button.

That session lasted about 3 minutes before again I lost access to the app.

I think I rebooted again, but that session did not last very long at all until I lost comms to the app.

It was at that point i remembered reading (here) about the possibly faulty SD card.


Not sure where you can see that it was running for 8 hrs ? as when i reached out for support, it had been up for only 1 hr.
I left it running last night till bed time, which was about 4 hrs, and all good.

Just fired it back up this morning and all good, with only 1 hiccup.

I have noticed that I cannot access the app if I enter http://iotawatt.local in the browser, but if i enter just the actual IP address I can ?

I have not changed the “name” of the device and so I am really puzzled by this.

any explanation ?

You posted that log 11 hours ago. It’s 0100UTC now so the log was posted at 1400 UTC today. I posted the response an hour later at about 1500 UTC. The last restart in the log was 0734 UTC, OR 7 1/2 hours before my post. That’s where the 8 hours came from.

As long as you’re running, could you tape the old SD to something rigid and drop it into an envelope to:

IoTaWatt inc
PO Box 1679
Wolfeboro, NH 03894

I’ll try to recreate the problem. If you provide a return address, I’ll send you a new SD card

Happy Thanksgiving,
Bob

Ok, I see. (UTC thing )

This mornings uptime is now 1hr 30mins with no problems. fingers crossed

It is pretty easy to see if/when comms with the app stop.
On the main app “status” page, the “IoTaWatt Statistics” shows the uptime clock, ticking away.
When I noticed the issue, that clock had stopped.
Once, when the issue was happening, I managed to open another area (tools or setup i think) then go back to the “status” area and the clock jumped up to the correct reading.
That didn’t last long until comms dropped completely.

What about the iotawatt.local not being available, but access via the actual IP address is ok ??
Any ideas ??
I did do a DNS flush on my PC, but that made no difference.

I will certainly send you the SD card in the post.

Hopefully you can recreate the issue.

That’s a red herring with this issue. Try installing Bonjour to make the .local work, otherwise, just bookmark the IP address.

My local PC is a Windows machine ?

yes, there is a version of bonjour for Windows.

Interesting.
Just tried the iotawatt.local on another PC ( wifi laptop) and it connected no worries.

Looks like it is something on my PC.:thinking:

Will look into it further , thanks

And now http://iotawatt.local resolves from my PC Okay !
Very weird.

I have been playing with some of the network adapter settings and install bonjour, … so whatever it was is now fixed.

Thanks so much for your time overeasy, I really do appreciate it.

The .local is a protocol called Mdns or Multicast Dns. The way it works is when the local host sees the .local top level domain, rather than go to the DSN server, it broadcasts the name to everyone on the LAN essentially asking “Anybody out there calls themselves iotawatt?”. The IoTaWatt listens for that and responds to the requestor with it’s IP address. The requestor then knows how to reach the IoTaWatt and saves the IP away. Not all browser/OS combinations work the same. Apple’s version is Bonjour, originally developed to make printers on the LAN addressable without configuring on each desktop. There was a subsequent effort to standardize called “zeroconf” which didn’t work out, but Microsoft built their own, called LLMNR. That’s what these two messages at startup are about:

11/20/19 22:11:07 MDNS responder started for hostname IotaTest
11/20/19 22:11:07 LLMNR responder started for hostname IotaTest

1 Like