I setup an EmonCMS instance in Docker and told IotaWatt to upload the total watts combined from my two Mains inputs I had it upload history from 9/1/2019 as that is when I setup my IotaWatt. It took about 16 hours but finally shows current in EmonCMS. The problem is EmonCMS show the data starting on 9/4/2019 and the total kWh in EmonCMS is about 100 less than what shows on the IotaWatt graph. Looking at the IotaWatt graph confirms it has data starting on 9/1/2019 and the usage between 9/1 and 9/4 is roughly the difference between the EmonCMS reading and the IotaWatt total. Should I try deleting the feeds out of EmonCMS and Iota and set it up again to see if it all goes through this time?
If having the correct history for the first three days of September is important, you could try that. You would wipe the Emoncms data, but not the IoTaWatt.
The story of what happened is probably in your message log on the IoTaWatt. You might also check if the Emoncms shows “null” for that period or zero. If it had been zero, it would not have accepted the updates from IoTaWatt.
The message log shows it started sending data from 09/01
1/30/20 00:05:25 EmonService: started. url=192.168.20.140:8080, node=Mudridge, interval=5, encrypted
1/30/20 00:05:27 EmonService: Node doesn’t yet exist.
1/30/20 00:05:27 EmonService: Start posting at 09/01/19 12:29:45
I removed the webserver from Iota then removed the inputs and feeds from Emon, Started it again and recreated the feed after it created the input and now it’s showing feed starting on 09/01 so not really sure what happened the first time. Thank you for the help. Unfortunately I cleared the feed out of Emon before digging into it but I believe it was “null” there was literally no input prior to 09/04 according to Emon, not an input of 0.
Not worth trying to figure it out. The protocol is synchronous, so IoTaWatt waits for acknowledgement from Emoncms before sending more data. If it says “ok” I have to assume it got it. What happens after that is outside the IoTaWatt domain.
Understood, glad deleting and re-adding it has added everything from the beginning. Will make sure the Iota number and the Emon number line up tomorrow once it’s done uploading, or at least be within a few kWh.
1 Like
I figured out why the feed was missing the first few days. As it was my first time generating anything in Emon it took me a few minutes to create the feed. I did not realize it would not use all the data it had received once the feed was created so it only started from the point the feed was created. Since I couldn’t create the feed until Iota started sending data to create the input it took me a few tries to get the feed created within seconds of the input being generated by Iota to have the feed started at the exact time the Iota started logging data from the point I created the outputs.
Is there some way to generate the input from Iota without it automatically starting to send data in order to create the feed that I’m missing? If I decide to start sending a new output to Emon is there a way to get it to send all the data that Iota has stored without starting all over by deleting the Emon server from Iota and removing all feeds from Emon?
Yes. You can send a GET /input/bulk transaction from your browser to Emoncms with zeroes for all of the inputs and timestamped with a date, UTC in UNIXtime of your desired start date/time-interval. The format for Emoncms inputs is described here.
Another approach is to startup the IoTaWatt Emoncms service and as soon as status indicates it is uploading, stop the upload. Note the time of last update indicated after it stops. If that’s close enough for you, create the feeds in Emoncms then resume the upload.
The first method is absolute, the second may still miss a few minutes.
The GET transaction makes sense. I’m still rather new to API oriented instructions having to manually be done. I will try that next time. I was trying the second approach but when I would click stop it wouldn’t stop the service, I think I was trying to stop it too soon and it hadn’t fully started so it was not registering the command. I ended up just hitting save to create the Emon connection, switching tabs refreshing a couple of times and clicking through the feed as fast as possible. Thank you for the help.
The stop button was an afterthought, not intended as a primary control. It’s not connected directly to the service. When you click it, a new config is saved with a stop indicator for the Emoncms service. At appropriate times, (there are inappropriate times, like when a transaction is pending) the service checks for that and stops. The next status request by the app picks up the stopped status and updates your status display to show stopped.
Long story short, it can take a few seconds to actually stop the service and show stopped in the display. If you get impatient and click stop again, you nullify the first click and start another series of events that cancel the first. I’ve learned to be patient and it usually responds to a single click.
This incident has also got my wheels turning about the basic problem of setting up a fresh Emoncms node. The folks at OEM have some templating facilities, and I’m thinking of offering an option to setup a basic “log to feed” and/or “power to kWh” feed for new nodes. Have to investigate the Emoncms APIs available for that.
That explains a lot. It was 3am and I was being impatient and re-clicking it multiple times. At one point it even forced a reboot of the Iota.
If the API would support setting up those feeds upon creation that would probably be the best way to go about it but sounds like it could be a lot of work for you. My problem would have been easily mitigated by running the API call to create the input before creating the Emon connection in Iota and then creating the feed and then creating the Emon connection. Maybe an update to the Iota documentation on creating that API call before hand in the Emon section would be easier?