Today, I had my new RaspPI B- up and running with a working emoncms server. It started collecting data at 9.44pm on Jan 26 2019. I have set the IotaWatt to upload from Jan 15 2019, just to test the upload historic data functionality. But it does not upload any data before 9.44pm. At least, nothing is shown after about an hour. I tried different ‘upload from’ dates (the iotawatt log file goes back to Jan 5), but nothing happens. I purchased the iotawatt late December 2018.
Not clear the order of things her. If you configured the IoTaWatt without setting the Jan 15 date first, and it started posting from the current time, then you cannot upload history. That is a limitation of EmonCMS. You would need to stop the Iotawatt posting (with the stop button in the status tab, then delete all of the inputs and associated feeds in EmonCMS, then start the IoTaWatt EmonCMS service again.
I think you nailed it. When switching API key and server from emoncms.org to my local setup yesterday, I never stopped the iotawatt from posting. I’ll try that, delete the feeds&inputs and restart posting. Mainly to learn how/if it works.
A second question though: yesterday the RPI had two hickups, one from just freezing (had to pull the power) and at least one reboot. The emoncms data have holes during two periods from the same time, although I cannot verify exact timing. What is the expected behaviour of the iotawatt + emoncms where the iotawatt logs but the emoncms have is out of service for a shorter or longer period?
The IoTaWatt will retry the failed write until it receives an OK from Emoncms. Is it possible that a EmonCMS on the RPi is acknowledging the write before it is actually committed to the database? I know that Emoncms accepts new writes and then puts the data in a queue to be processed to memory. If you want to try something, unplug theRpi for Say five minutes, and see how big the gap is after yourestart it and IoTaWatt shows iris caught up.
Deleted all feeds and inputs on the emoncms server.
Started the posting again, setting the first posting date to Jan 5 2019 (my first data point)
Inputs reappeared, showing physical live values in the value column.
Reconfigured all inputs and feeds.
While doing this, the emoncms inputs all are listed as “inactive” in the “updated” column. While writing this post, the iotawatt has posted about two days worth of data, and needs to catch up with another 20 days before it is ajour.
At what time will the inputs be “active” again? Or am I doing something wrong here?
EDIT: Just checked, and there are new data filling up into the emoncms database, so it seems to be working. Just curious about the “inactive” status.
That’s all Emoncms functionality. Not sure the exact cutoff they use to call something inactive, but I think it’s around two-three days. So when you get close, it should start the countdown. As long as the input values appear to be changing, it’s uploading.
It takes awhile. That’s a lot of data and the IoTaWatt is still spending the majority of it’s time sampling your current usage. You can time a day’s worth of upload and extrapolate the time to completion.
Had the exact same problem, but i have two iotawatts number one appears to have loaded historical data but number two did not. Followed the instructions and all is well
New setup: The Iotawatt is now posting to an emoncms db that was put into operation today (May 1 2019). The old emoncms db was running on a RPI3B+ that I configured myself, and it worked from January until today. The new db is running on a pre-installed EmonPI (also a RPI3B+), and posting of new data works fine. However, after an hour or so, not one single historic data sample has been uploaded from the Iotawatt (I have set it to post all data from jan 5 2019).
Emoncms will not accept data that is older than the most recent post. So if you posted recent data and are now trying to load older data, that won’t work. IoTaWatt knows this and will always resume from the most recent post, regardless of the upload from date.
To load the history, you must delete all of the inputs and feeds then start the IoTaWatt update with your historic date.
I switched back to my old emoncms database today, after about 16.5 hours. Then I obviously got a gap from about 9pm yesterday until 1:30pm today. Will this gap be closed?
I don’t understand how to force the iotawatt to post all historic data. The emoncms db on the new EmonPI has some temperature data from a few weeks back. Is that the reason why the iotawatt did not upload from January? These can be deleted. If I delete all feeds and inputs, then add the iotawatt inputs once more, will that do the trick?
BTW: The reason I switch back and forth is that the EmonPI does not behave, I have not been able to get a stable wifi connected yet.
No. It cannot be closed. Emoncms will not allow it with the bulk update protocol. Data must be sent in chronological order.
Could be. When I say data must be chronological, I mean within a particular node. If the temp data was posted with the same node name, that could cause problems.
When the Emoncms service starts, it looks at the time of last input on the node, the time set for history upload in IoTaWatt, and the start time in your current log. It then begins posting at the greater of the three.
The node names are not the same. The temp nodes are emonTHs and are detected as separate nodes on the emonpi. However, there is something strange going on with the iotwatt node. It seems to be detected three times, as node “1”, “15” and “somethingelseIrenamed to iotawatt”. All these three nodes have 20 input tags, which is what my Iotawatt is sending.
When I configured the Iotawatt node last night, I took all the inputs in chronological order within 5 minutes or so.
I never could understand what Emonpi is doing there. I have one and itsquirts out all kinds of node data like that as well. It can be configured as I understand it but I just don’t have the patience to try and figure it out. The thing is a Swiss Army knife, trying to do all things for all people. My opinion is that it should be shipped with all that stuff turned off and then I might not mind trying to figure out what I need rather than how to disable what I don’t need.
At the end of the day, you can always wipe a node and reload from IoTaWatt as far back as it has data in the current log, which would be up to a year. There may be ways to delete selective recent data, but I’m just not keeping up with what they are doing.
The good news is that IotaWatt still has everything. Have you looked at influxDB? Runs great on a Pi.
I will try ask for some help at the openenergy community as well.
I am a real linux n00b. Tried installing influxDB, but ran into trouble at once, as two things happened: close to 100% cpu load on the pi, and not able to actually have the iotawatt uploading any data…
The binaries for arm plattforms are in the indluxdata repo now. Just use the instructions for debian to install influxDB on the pi. You will have it up and running in under 10 minutes, without high cpu consumption: Installing InfluxDB OSS | InfluxDB OSS 1.7 Documentation