Occasionally I still need to remove the Emoncms userid detail on the IoTaWatt Web Server setup window to restart communications with emoncms.org. I’m very tempted to find out if the asynchronous update will cure the issue. That said, will I risk losing any data on IoTaWatt if I change the Auto-update Class: to ALPHA in the ‘Configure IoTaWatt Device’ window? I would also appreciate guidance whether it would it be best to restart IoTaWatt before or after changing to the ALPHA Class.
Short answer is that there is no evidence of increased risk of losing data. That said, the ground rules for ALPHA are that you should be capable of flashing new firmware and formatting a new SDcard if something goes terribly wrong. That hasn’t been necessary to date, but that’s doesn’t mean it won’t happen.
The first production system to get any ALPHA update is my own home system:
As you can see, I’ve got continuous data for 14 months. My current log has wrapped, which means that unlike all of the newer systems out there, when my current log ending time advances by 5 seconds, the starting time also advances, because it is a fixed size circular file now. The history log, although capable of wrapping, will not do so for about 10 years. So if you were to lose your history log, it would be automatically recreated within the range of the current log. If you were to lose your current log, a new one would be created beginning at the present time. However, the history log would still be available and would support queries at 1 minute resolution.
The IoTaWatt will auto-update spontaneously within an hour of changing the auto-update class. You can force an immediate update by restarting it.
This is an ongoing problem that I am working, although I don’t have it with about a half-dozen local systems that I monitor on ALPHA. It’s not clear if the problem is actually caused by the encrypted protocol that is triggered by specifying userid. A characteristic of the way the Emoncms service works is that when you change protocol, the current data packet that has been rejected is discarded as if it were successfully posted and the IoTaWatt begins posting with the next sequential data points. So my suspicion is that it’s a data issue, with something being rejected by eMoncms, rather than a protocol/encryption problem. There is also evidence to suggest a possible memory leak, so I continue to look into this. If you could post the message log from one of those events, I could add that to the body of evidence on the problem. I’m in the process of adding some enhanced diagnostics to specifically address this.
ALPHA will soon be a new release with redesigned async influxDB support, so the clock will start again with that.
Unlike the previous two occasions, I have a problem providing a copy of the message log for the period when communications were lost, which is believed to be around mid day on 15/03/18. The emoncms userid data was removed at 23:00:58 GMT on Sat 17 Mar 2018 and this is a recent copy of the message log. It was noticed that it starts from 3/16/18 20:15:10 and not 23 Oct 2017 as the earlier copies!
IoTaWatt SD Card Messages 19_03_18.txt (66.2 KB)
That said, the copy of the IoTaWatt ‘Status Screen’ shows the current and history logs being available from 23/10/2017.
So why the earlier records in this latest message log are missing is beyond my understanding.
If you know of a way to get the message log to start from the beginning once more and include information on the 15/03/18, please let me know.
The message log display only delivers the last 10000 bytes of the message log (rounded to a line boundary). That’s usually enough to see what has been happening. When you view the display in your browser, the url should look something like:
http://iotademo.local/iotawatt/iotamsgs.txt?textpos=-10000
Note the query ?textpos=-10000. You can change that to say -40000 and refresh to get a bigger payload.
If that’s not adequate, another approach is to download the whole /iotawatt/iotamsgs.txt file. You can look at it in your browser if it’s not too big by just removing the query
http://iotademo.local/iotawatt/iotamsgs.txt
or you can run the file manager, right click on it in the left column file list and select download to download to your client’s download folder. Then you can upload the file to the forum.
Sorry, I do not appear to be having much success.
The uploaded text file I presented in my previous message was created by choosing the File Manager and Editor, left click to expand iotawatt and then left click on iotamsgs.txt. The resultant display was copied and uploaded. N.B. this is the method used for all previous uploads when all the files started on 23 Oct 2017.
As the above is different to the suggested routes you have just provided, I have tried each option to see if it makes a difference. The results are:
Option 1 -
iotawatt.local/iotawatt/iotamsgs.txt?textpos=-40000
The file starts at 3/17/18 08:05:39 EmonService: Resending EmonCMS data:8
by increasing to -3000000
The file starts at 3/16/18 20:15:10 Real Time Clock is running. Unix time: 1521231310
by increasing to -30000000
The file starts at 3/16/18 20:15:10 Real Time Clock is running. Unix time: 1521231310
No further back than -3000000.
Option 2 -
iotawatt.local/iotawatt/iotamsgs.txt
The file starts at 3/16/18 20:15:10 Real Time Clock is running. Unix time: 1521231310
Same output as the -300000 option above.
Option 3 -
run the file manager, right click on it in the left column file list and select download
The file starts at 3/16/18 20:15:10 Real Time Clock is running. Unix time: 1521231310
Same output as the -300000 option above.
In summary, it would appear that the earliest available information currently retrievable from iotamsgs.txt is 3/16/18 20:15:10, unless I have misunderstood what you wish me to do.
You have it right. The log gets deleted at restart when it grows beyond 1mb. It’s somewhat arbitrary, but is usually adequate. Thanks for the effort.
I’ll continue to pursue this with the resources that I have. If you are interested in trying something else, the asynchronous HTTP release was advanced to BETA today after two weeks in ALPHA.
As much as I am looking forward to using the asynchronous HTTP release, I think it would probably be best to wait for the MAJOR upgrade. As a user with a minimal understanding of computing, I don’t want to cause you any more problems than I have already.
OK, I understand. If the problem continues to recur, it would be helpful to capture the message log at the time of the next failure.
Thanks for your efforts.
@OddJob, I’ve isolated and fixed a problem with Emoncms encrypted post. The fix is in a new release that will take awhile to mature, but I think this is the problem you are having. In the meantime, could you run on the unsecure protocol and advise me if you have any problems with that?
Thanks for the update.
The last two entries in the iotamsgs.txt file after removing the Emoncms userid are:
3/21/18 14:36:00 EmonService: started.url: emoncms.org:80, node: IotaWatt, post interval: 10, unsecure GET
3/21/18 14:36:00 EmonService: Start posting at 1521642950
Based on several previous periods operating on the unsecure protocol I do not anticipate having problems but will let you know how I get on.
Still successfully recording using the unsecure protocol and the iotamsgs.txt file is not indicating any abnormal problems.