Export data after power failure

theju · August 30, 2018, 8:23pm

Hi,

Today I had a power failure on the AC/AC adaptator (not the main power source)

All seems to restart correctly except the 2 exports. Emoncms and influxdb are not updating anymore.

I’ve try to restart the iotawatt, influxdb server and emoncms server but no changes.

On the status tab if I hit stop / start, the export fot influx db is moving 1 minute forward and did not change after that.
If I do that to emoncms, time is still the same :

In the log, I’ve only an error in the emoncms export :

8/30/18 22:10:47 EmonService: Stopped.  Last post 8/30/18 19:01:50 
8/30/18 22:10:51 EmonService: started. url:192.168.10.181:80/emoncms/,node:IotaWatt,interval:10, unsecure GET
8/30/18 22:10:52 EmonService: Start posting at 8/30/18 19:02:00 
8/30/18 22:10:59 EmonService: Invalid response, retrying.
8/30/18 22:10:59 influxDB: Stopped. Last post 8/30/18 19:16:10 
8/30/18 22:11:00 influxDB: started.
8/30/18 22:11:01 influxDB: Start posting from 8/30/18 19:16:50

Thank you for your help

Juju

overeasy · August 30, 2018, 8:28pm

Hi Juju,
Could you expand the Data Logs tab and show the entire status display please?

theju · August 30, 2018, 8:34pm

Hello

Here is the log after reboot. You can see my tests to start an stop export services services

Nothing more in the past logs (or maybe I’m not looking at the right page ? )

** Restart **

SD initialized.
8/30/18 19:55:06z Real Time Clock is running. Unix time 1535658906 
8/30/18 19:55:06z Version 02_03_13
8/30/18 19:55:06z Reset reason: Software/System restart
8/30/18 19:55:06z Trace:  9:3, 9:4, 9:5, 9:9, 1:2, 1:3, 1:4, 1:5[7], 7:0, 7:7, 7:7, 7:7, 7:7, 7:7, 7:7, 1:6, 1:1[4], 1:2[5], 9:0[5], 9:0, 9:1, 8:4, 8:6, 8:8, 9:3, 9:4, 9:5, 9:9, 1:2, 1:3, 10:2, 10:3
8/30/18 19:55:06z ESP8266 ChipID: 6911180
8/30/18 21:55:06 Local time zone: 2
8/30/18 21:55:06 device name: IotaWatt, version: 3
8/30/18 21:55:09 MDNS responder started
8/30/18 21:55:09 You can now connect to http://IotaWatt.local
8/30/18 21:55:09 HTTP server started
8/30/18 21:55:09 timeSync: service started.
8/30/18 21:55:09 statService: started.
8/30/18 21:55:09 WiFi connected. SSID Juju, IP 192.168.10.185, channel 1, RSSI -66db
8/30/18 21:55:09 Updater: service started. Auto-update class is BETA
8/30/18 21:55:09 dataLog: service started.
8/30/18 21:55:09 dataLog: Last log entry 8/30/18 21:55:00 
8/30/18 21:55:09 historyLog: service started.
8/30/18 21:55:10 historyLog: Last log entry 8/30/18 21:55:00 
8/30/18 21:55:11 Updater: Auto-update is current for class BETA.
8/30/18 21:55:14 EmonService: started. url:192.168.10.181:80/emoncms/,node:IotaWatt,interval:10, encrypted POST
8/30/18 21:55:14 influxDB: started.
8/30/18 21:55:15 EmonService: Start posting at 8/30/18 19:02:00 
8/30/18 21:55:15 influxDB: Start posting from 8/30/18 19:12:20 
8/30/18 21:55:22 EmonService: Invalid response, retrying.
8/30/18 21:55:26 influxDB: Post Failed: 400
8/30/18 21:56:15 EmonService: Stopped.  Last post 8/30/18 19:01:50 
8/30/18 21:56:19 EmonService: started. url:192.168.10.181:80/emoncms/,node:IotaWatt,interval:10, encrypted POST
8/30/18 21:56:20 EmonService: Start posting at 8/30/18 19:02:00 
8/30/18 21:56:27 EmonService: Invalid response, retrying.
8/30/18 22:01:50 influxDB: Stopped. Last post 8/30/18 19:12:10 
8/30/18 22:01:53 influxDB: started.
8/30/18 22:01:55 influxDB: Start posting from 8/30/18 19:13:20 
8/30/18 22:04:31 EmonService: started. url:192.168.10.181:80/emoncms/,node:IotaWatt,interval:10, encrypted POST
8/30/18 22:04:32 EmonService: Start posting at 8/30/18 19:02:00 
8/30/18 22:04:38 EmonService: Invalid response, retrying.
8/30/18 22:04:42 influxDB: started.
8/30/18 22:04:42 influxDB: Start posting from 8/30/18 19:14:20 
8/30/18 22:05:58 EmonService: Stopped.  Last post 8/30/18 19:01:50 
8/30/18 22:06:04 EmonService: started. url:192.168.10.181:80/emoncms/,node:IotaWatt,interval:10, encrypted POST
8/30/18 22:06:05 EmonService: Start posting at 8/30/18 19:02:00 
8/30/18 22:06:12 EmonService: Invalid response, retrying.
8/30/18 22:06:30 influxDB: Stopped. Last post 8/30/18 19:14:10 
8/30/18 22:06:33 influxDB: started.
8/30/18 22:06:34 influxDB: Start posting from 8/30/18 19:15:20 
8/30/18 22:09:21 EmonService: started. url:192.168.10.181:80/emoncms/,node:IotaWatt,interval:10, unsecure GET
8/30/18 22:09:22 EmonService: Start posting at 8/30/18 19:02:00 
8/30/18 22:09:29 EmonService: Invalid response, retrying.
8/30/18 22:10:38 influxDB: started.
8/30/18 22:10:39 influxDB: Start posting from 8/30/18 19:16:20 
8/30/18 22:10:47 EmonService: Stopped.  Last post 8/30/18 19:01:50 
8/30/18 22:10:51 EmonService: started. url:192.168.10.181:80/emoncms/,node:IotaWatt,interval:10, unsecure GET
8/30/18 22:10:52 EmonService: Start posting at 8/30/18 19:02:00 
8/30/18 22:10:59 EmonService: Invalid response, retrying.
8/30/18 22:10:59 influxDB: Stopped. Last post 8/30/18 19:16:10 
8/30/18 22:11:00 influxDB: started.
8/30/18 22:11:01 influxDB: Start posting from 8/30/18 19:16:50

overeasy · August 30, 2018, 9:49pm

Not much to go on here. When the Emoncms and Influx services start, they are able to query their respective servers to determine the time of last posting. Then they begin to try to upload the history.

Both appear to be running into errors posting the data, which I suspect has been corrupted. I’ve looked at the code, and neither is insulated from the “NaN” (not-a-number) result, which possibly has resulted from running without the VT. I’ll put that on the list to fix, but you will need to get your IoTaWatt to skip over that section and resume posting later on, after the VT was reconnected.

First, the IoTaWatt is currently working OK taking measurements and putting them in the log. When this is resolved, that data should be OK to upload. There are three ways to do this:

You can delete the current log, which will be recreated immediately starting with the current time. Your Emoncms and influx would resume posting from that time and off you go, missing the data from when the VT was unplugged until now. You will still be able to local graph that period from the history file (minus the period when the VT was unplugged).

You can wait until my next release in a week or so where I’ll have that fixed and it should upload all the history.

You can manually insert a single entry into both the Emoncms and influx databases to trigger IoTaWatt to leapfrog over the bad VT data. If you choose this option, I will need to see a graph of your voltage from the IoTaWatt graph to determine when the event ended, and I will need to see your influx and Emoncms configurations to make HTTP transactions for you to issue to make the entries. Be sure to obfuscate the Emoncms write-key if you post it.

theju · August 31, 2018, 6:54am

Last results.

This morning I change the start date of export from to today.

Emoncms is currently uploading datas and Influxdb is still blocked at yesterday even if I choose today as starting date.

For the influxdb, each time I hit start/stop, the stop date is increasing for 1 minute. I’ve hit it until the time at which the VT as been plugged again as been reached.

All is ok now.

Thank you

Juju

theju · October 1, 2018, 7:43pm

Hello,

Once more data upload is blocked, this time after the update to 2_3_16. Yhis time, Influxdb is OK, only the emoncms is blocked.

I’ve tried to delete the log file but this time, it doesn’t move and stay at “Stop Running. Last update 01/10/2018 21:07:30”

10/01/18 21:36:11 Updater: service started. Auto-update class is ALPHA
10/01/18 21:36:11 dataLog: service started.
10/01/18 21:36:11 dataLog: Last log entry 10/1/18 21:35:40
10/01/18 21:36:11 historyLog: service started.
10/01/18 21:36:12 historyLog: Last log entry 9/3/18 09:44:00
10/01/18 21:36:14 Updater: Auto-update is current for class ALPHA.
10/01/18 21:36:16 EmonService: started. url:192.168.10.181:80/emoncms/,node:IotaWatt,interval:10, encrypted POST
10/01/18 21:36:16 influxDB: started.
10/01/18 21:36:18 EmonService: Start posting at 10/1/18 20:59:30
10/01/18 21:36:18 influxDB: Start posting from 10/1/18 21:34:40
10/01/18 21:36:18 EmonService: Stopped. Last post 10/1/18 20:59:20
10/01/18 21:36:57 EmonService: started. url:192.168.10.181:80/emoncms/,node:IotaWatt,interval:10, encrypted POST
10/01/18 21:36:58 EmonService: Start posting at 10/1/18 20:59:30
10/01/18 21:37:08 EmonService: Invalid response, retrying.
10/01/18 21:37:12 influxDB: Post Failed: 400
10/01/18 21:38:43 EmonService: Retry successful after 24 attempts.
10/01/18 21:38:58 EmonService: Invalid response, retrying.
10/01/18 21:41:17 EmonService: Stopped. Last post 10/1/18 21:07:30
10/01/18 21:41:26 EmonService: started. url:192.168.10.181:80/emoncms/,node:IotaWatt,interval:10, encrypted POST
10/01/18 21:41:27 EmonService: Start posting at 10/1/18 21:07:40
10/01/18 21:41:38 EmonService: Invalid response, retrying.

Thank you

overeasy · October 1, 2018, 9:21pm

Can I see the log from before you updated to 02_03_16 to present please? The log above shows that there was some advance in the Emoncms posting time from 20:59:30 to 21:07:30. Without the log for that period, it’s not possible to know if that’s significant.

In general, it seems that folks are jumping to the conclusion that deleting the current log is a cure all for posting problems. It is not, and the situation is probably quite different with 02_03_16 as there has been a lot of work done to address current log integrity.