Upload "History Log" to EmonCMS

Support
1

I recently have had a mare with my EmonCMS install, I originally installed it with Docker, and they’ve since moved away from that. I’ve since installed a new EmonCMS instance, but I’m unable to do a clean backup from the original one. My understanding is that IoTaWatt will upload the history as long as I set the history date correctly, however the history I want is in the History_Log, not Current_Log:

image

Is there a way to get this to upload as well? I did set the “Upload history from” date to my original IoTaWatt install date (ish):

image

SD initialized.
8/07/21 18:54:44z Real Time Clock is running. Unix time 1628362484 
8/07/21 18:54:44z Reset reason: Software/System restart
8/07/21 18:54:44z Trace:  15:10, 15:14[32], 15:15[2], 15:16, 15:4, 3:71, 15:10, 15:14[144], 15:15, 15:16, 3:71, 31:1, 1:6[6], 1:1[14], 1:2, 9:0, 9:0, 8:4, 8:6, 8:8, 8:9, 1:2, 1:3, 1:3, 1:6[1], 1:6[2], 1:6[2], 1:6[3], 1:5[5], 1:6[4], 5:0, 5:7
8/07/21 18:54:44z ESP8266 ChipID: 2516990
8/07/21 18:54:44z IoTaWatt 4.x, Firmware version 02_06_05
8/07/21 18:54:44z Updater: Installing update files for version 02_06_05
8/07/21 18:54:44z Updater: Installing GRAPH.HTM
8/07/21 18:54:44z Updater: Installing GRAPH.JS
8/07/21 18:54:44z Updater: Installing GRAPH2.HTM
8/07/21 18:54:44z Updater: Installing GRAPH2.JS
8/07/21 18:54:45z Updater: Installing INDEX.HTM
8/07/21 18:54:46z Updater: Installing TABLES.TXT
8/07/21 18:54:46z Updater: Installing CNFSTYLE.CSS
8/07/21 18:54:46z Updater: Installing EDIT.HTM
8/07/21 18:54:46z Updater: Installation complete.
8/07/21 18:54:46z SPIFFS mounted.
8/08/21 06:54:46 Local time zone: +12:00
8/08/21 06:54:47 Using Daylight Saving Time (BST) when in effect.
8/08/21 06:54:47 device name: IotaWatt
8/08/21 06:54:47 HTTP server started
8/08/21 06:54:47 emoncms: Starting, interval:10, url:http://10.111.111.36:8080
8/08/21 06:54:47 influxDB_v1: Starting, interval:10, url:http://10.111.111.35:8086
8/08/21 06:54:47 timeSync: service started.
8/08/21 06:54:47 statService: started.
8/08/21 06:54:47 dataLog: service started.
8/08/21 06:54:47 dataLog: Last log entry 08/08/21 06:54:20
8/08/21 06:54:52 historyLog: service started.
8/08/21 06:54:52 historyLog: Last log entry 08/08/21 06:54:00
8/08/21 06:54:55 WiFi connected. SSID=DCDC, IP=10.111.111.80, channel=1, RSSI -66db
8/08/21 06:54:55 MDNS responder started for hostname IotaWatt
8/08/21 06:54:55 LLMNR responder started for hostname IotaWatt
8/08/21 06:54:55 Updater: service started. Auto-update class is MINOR
8/08/21 06:54:55 emoncms: Start posting at 08/08/21 06:54:10
8/08/21 06:55:01 Updater: Auto-update is current for class MINOR.
8/08/21 06:56:14 influxDB_v1: Start posting at 08/08/21 06:53:50
8/08/21 14:07:45 WiFi disconnected.
8/08/21 14:12:02 WiFi connected. SSID=DCDC, IP=10.111.111.80, channel=1, RSSI -78db
8/08/21 14:16:46 WiFi disconnected.
8/08/21 14:17:44 WiFi connected. SSID=DCDC, IP=10.111.111.80, channel=1, RSSI -69db
8/15/21 14:51:52 WiFi disconnected.
8/15/21 14:51:53 WiFi connected. SSID=DCDC, IP=10.111.111.80, channel=1, RSSI -66db
8/15/21 15:02:59 WiFi disconnected.
8/15/21 15:03:00 WiFi connected. SSID=DCDC, IP=10.111.111.80, channel=1, RSSI -64db
8/27/21 12:41:35 emoncms: stopped, Last post 08/27/21 12:41:25
8/27/21 12:42:29 emoncms: Start posting at 08/27/21 12:41:30
2

The uploaders will only upload from the current log. You should set your start date to the beginning of the current log. If your current log has a year of data, the start date is a moving target as the file wraps and begins to overwrite the oldest entry with the new entries. So in that case it’s best to set the upload from date to the current log day + 1.

3

Damn. That’s working perfectly fine, I was hoping I could also upload from the history as my current EmonCMS instance is tanked, unable to do backups from.

4

@overeasy I am planning to just write a script to import the history.log and current.log into EmonCMS before turning on the data uploader again, but I can’t actually download the history.log? It fails every time at about 20% / ~100M.

image
image915×171 6.41 KB

Doesn’t matter if I use Chrome / Windows / whatever.

5

The webserver isn’t setup to move a big file like that, and I think there’s probably an issue with the file being open in the Histlog service for read/write and writes happening once per minute.

The quickest and best way to get the datalogs is to:

Power down the unit.
Open it up.
Lift out the PCB and carefully pull the SD card straight out. Don’t crack it!

Mount it in a PC or alternative and copy.

Reinsert, reassemble, restart.

The actual file copy should only take a few seconds in a PC.