IotaWatt and PV Output

Support
1

Guys, trying to get my IotaWatt outputting to PV Output.
The unit is not in its final location yet, as I am trying to configure it and sort out the inputs and data push to PVO.

My PV Output (donor) account has been up and running for about 7 years, with no issues.
Inverter generation get sucked out via PV Beancounter and pushed up to PV Output (API)

Current setup I have my IotaWatt running with the VT connected and calibrated.
I have one CT connected and configured as “Mains”, and hooked onto the PV Inverter mains cable.

Looking at the graphs page on the iotawatt app, the inverter output (from the Mains CT) is damn near identical to the graph on PV Output. As you would expect.

All good so far.

So I setup the PV Output server in the iotawatt, enter API and ID etc and as a test, I set the output to “mains Voltage”

Wait a few minutes and refresh the PV Output , Yep ! Voltage appears on the PV Graph !

So I now add the “Mains” input as “consumption” in the PV Output server.

Wait a few minutes and refresh the PV graph… bugger !

PVout graph is ADDING the consumption onto the generation ! and there is a 5 minute difference between generation and consumption.!

Screenshot
Screenshot.png1881×431 75.2 KB

Really odd because the IotaWatt graph (data) doesn’t correspond.

Screenshot(1)
Screenshot(1).png1460×896 103 KB

I have also noticed that if I try and edit the PV Output service,

Screenshot(2)
Screenshot(2).png978×890 51.7 KB

and by that I mean to DELETE an output,

Screenshot(3)
Screenshot(3).png1006×853 47.1 KB

the “save” button disappears and I cannot remove an output.

Some guidance would be very appreciated

2

5 minute is the live data interval. It probably has to do with your inverter generation being interpreted as the preceding five minutes and the IotaWatt data is for the 5 minutes after the time stamp. As far as I know there is no standard for what period a time stamp represents, but IoTaWatt tries to be consistent throughout with the time stamp being the time of the start of the reporting period.

If the IoTaWatt matches the inverter so well, why not use only the IoTaWatt for all of the livedata?

That is because you must specify either a generation or consumption metric. The PV output specifications require one of those metrics in a status update.

3

Thanks overeasy,
The reason why I want to use PVBC for my generation data is that the info comes directly from the inverter and not from a CT, so I figured it would be more accurate.
PVBC also archives a daily CSV file of the inverter generation and historic generation totals, and I would like to continue to save that data.

At some point I will probably use the IotaWatt to upload all data, but just want to get everything reporting to PVO correctly first.

I still do not know why the IotaWatt consumption data is getting added onto the generation data in PVO ?

4

Just noticed that my IotaWatt “restarted” when I setup the PVO service ?

log of event

SD initialized.
12/19/19 08:38:00z Real Time Clock is running. Unix time 1576744680
12/19/19 08:38:00z Reset reason: Exception
12/19/19 08:38:00z Trace: 15:1, 15:1, 15:0, 15:1, 15:1, 15:0, 15:1, 15:1, 15:0, 15:1, 15:1, 15:0, 15:1, 15:1, 15:0, 15:1, 15:1, 15:0, 15:1, 15:1, 15:0, 15:1, 15:1, 15:0, 15:1, 15:1, 15:0, 15:1, 15:1, 15:0, 15:1, 15:1
12/19/19 08:38:00z ESP8266 ChipID: 6145634
12/19/19 08:38:00z IoTaWatt 5.0, Firmware version 02_04_02
12/19/19 08:38:00z SPIFFS mounted.
12/19/19 16:38:01 Local time zone: +8:00
12/19/19 16:38:01 device name: IotaWatt
12/19/19 16:38:01 MDNS responder started for hostname IotaWatt
12/19/19 16:38:01 LLMNR responder started for hostname IotaWatt
12/19/19 16:38:01 HTTP server started
12/19/19 16:38:01 WiFi connected. SSID=SpeedyDave, IP=192.168.1.106, channel=11, RSSI -76db
12/19/19 16:38:01 timeSync: service started.
12/19/19 16:38:01 statService: started.
12/19/19 16:38:01 Updater: service started. Auto-update class is MINOR
12/19/19 16:38:01 dataLog: service started.
12/19/19 16:38:01 dataLog: Last log entry 12/19/19 16:37:55
12/19/19 16:38:01 historyLog: service started.
12/19/19 16:38:01 historyLog: Last log entry 12/19/19 16:37:00
12/19/19 16:38:04 Updater: Auto-update is current for class MINOR.
12/19/19 16:38:11 PVoutput: started
12/19/19 16:38:11 PVoutput: System Speedy Dave’s, interval 5, donator mode
12/19/19 16:38:12 PVoutput: Start status beginning 12/19/19 16:35:00
12/19/19 16:54:17 PVoutput: Stopped.
12/19/19 16:57:38 PVoutput: started
12/19/19 16:57:39 PVoutput: System Speedy Dave’s, interval 5, donator mode
12/19/19 16:57:39 PVoutput: Start status beginning 12/19/19 16:55:00
12/19/19 20:04:12 PVoutput: System Speedy Dave’s, interval 5, donator mode
12/19/19 20:04:13 PVoutput: Reload status beginning 12/19/19 00:05:00
12/19/19 20:49:02 PVoutput: System Speedy Dave’s, interval 5, donator mode
12/19/19 20:49:02 PVoutput: Reload status beginning 12/19/19 00:05:00
12/20/19 08:11:59 PVoutput: Stopped.
12/20/19 10:06:48 PVoutput: started
12/20/19 10:06:49 PVoutput: System Speedy Dave’s, interval 5, donator mode
12/20/19 10:06:50 PVoutput: Start status beginning 12/20/19 09:55:00

SD initialized.
12/20/19 02:07:03z Real Time Clock is running. Unix time 1576807623
12/20/19 02:07:03z Reset reason: Exception
12/20/19 02:07:03z Trace: 15:1, 15:1, 15:0, 15:1, 15:1, 15:0, 15:1, 15:1, 15:0, 15:1, 15:1, 15:0, 15:1, 15:1, 15:0, 15:1, 15:1, 15:0, 15:1, 15:1, 15:0, 15:1, 15:1, 15:0, 15:1, 15:1, 15:0, 15:1, 15:1, 15:0, 15:1, 15:1
12/20/19 02:07:03z ESP8266 ChipID: 6145634
12/20/19 02:07:03z IoTaWatt 5.0, Firmware version 02_04_02
12/20/19 02:07:03z SPIFFS mounted.
12/20/19 10:07:04 Local time zone: +8:00
12/20/19 10:07:04 device name: IotaWatt
12/20/19 10:07:04 MDNS responder started for hostname IotaWatt
12/20/19 10:07:04 LLMNR responder started for hostname IotaWatt
12/20/19 10:07:04 HTTP server started
12/20/19 10:07:04 WiFi connected. SSID=SpeedyDave, IP=192.168.1.106, channel=11, RSSI -77db
12/20/19 10:07:04 timeSync: service started.
12/20/19 10:07:04 statService: started.
12/20/19 10:07:04 Updater: service started. Auto-update class is MINOR
12/20/19 10:07:04 dataLog: service started.
12/20/19 10:07:04 dataLog: Last log entry 12/20/19 10:07:00
12/20/19 10:07:04 historyLog: service started.
12/20/19 10:07:04 historyLog: Last log entry 12/20/19 10:07:00
12/20/19 10:07:06 Updater: Auto-update is current for class MINOR.
12/20/19 10:07:14 PVoutput: started
12/20/19 10:07:14 PVoutput: System Speedy Dave’s, interval 5, donator mode
12/20/19 10:07:15 PVoutput: Start status beginning 12/20/19 10:05:00

any ideas ?

5

What can I say? Why not setup another PVO account fed by the IoTaWatt and see if it’s “correct”?

Maybe I’m just a little thick, but I have a hard time, looking at the documentation, figuring out exactly how it works when two different sources are updating the same status data. The IoTaWatt PVO service is intended and tested to update all of the data including generation and consumption. It does it well, with no lapses regardless of communications or PVO server maintenance or other outage.

Users that want to try to make two datasources work will need to figure out the nuances of that approach.

That’s a transient bug that I’m presently hunting with additional tracing in newer releases. While an issue, a restart literally takes two seconds and does not materially impact the basic mission.

6

Okay thanks, good to know that glitch is not something that I should have known about.

Not exactly the response that I thought I would get, but oh well…

My understanding was that the various uploaders (PVBC , iota , ? ) just pushed data to the relevant “slot” on PV Output database

I assumed, wrongly, that if there was no generation variable specified, then nothing would be uploaded to the PVO “generation” slot.

Looks like IotaWatt pushes generation data, regardless of whether there a “generation” output specified or not.
If there is NO generation field specified in the Web Service, it pushes a “null” value.
This presents in the PVO data set as a “-” or no data entry.

If there is a specified “generation” field in the iota service, then that value gets written to the PVO data and is then overwritten when PVBC uploads its data.

If the generation field is specified as just a number, then that number is pushed as the generation.
If the generation field is specified as one of the CT’s, then that data is pushed to the PVO generation field.

Now the good bit (for me) is that PVBC data is 5 minutes behind the iota data, so it then overwrites the “-” (or iota specified) generation value and inserts its own generation data, bringing the PVO data set back into reality.

I could not verify the reason that the 2 data sets were getting ADDED together, but whatever I have tweaked (in PVO), that glitch is no longer present.

Screenshot(4)
Screenshot(4)1881×492 43.7 KB

So now it is clear to see that the consumption data and the generation data are now separate (as they should be ) (note that the “consumption data” is via a CT on the AC output of the inverter. This is ONLY FOR TESTING PURPOSES and lets me verify the data sets easily)

You can also see that there is a very slight difference between the gen and cons data, because the gen data is derived directly from the data stream of the inverter, and the cons data is via CT.

Because it should have been easier to just be able to upload a specific piece of data, eg consumption, and have that one piece of data displayed on the PVO graph. I really did not expect the iota to push a null value to an unspecified data slot, although I do understand the reason why it has been coded that way…

7

Where are you getting this information from? Have you looked at the PVO ADD-STATUS API? What you are calling slots, are just positional values in a CSV string. If you don’t specify a source for a value like generation, the “slot” is left empty, I.e. “,”. The CSV protocol is positional, so the comma placeholders are required. IoTaWatt does not upload “,null,”, it uploads “,”.

It may have something to do with whether the consumption data is interpreted as net or gross. If it’s net, then adding the generation would be appropriate to arrive at gross consumption. The difference is whether the inverter output is connected before (gross consumption) or after (net consumption) the CTs on the main(s).

Again, PVoutput seems to be very nuanced as it appears to have grown organically as different data upload situations were added. I’m confident that when used as directed, IoTaWatt will maintain a complete and very accurate record of generation and consumption.

Slight variation from the inverter is meaningless without another reference. Think about it, what is the incentive for an inverter to err on the high side? What is the inverter reporting? Is it peak Watts during the period? IoTaWatt is reporting time-weighted average Watts and cumulative Watt-Hours during the five minute interval.

I question this. As previously explained, IoTaWatt timestamps the five-minute intervals with the time of the start of the interval. That is to say that if the data is for 0820-0825, the timestamp will be 0820. There is no explicit guidance that I recall in the PVoutput specifications, but that convention is implied. The intervals that make up a day are timestamped 00:00 to 23:55.

This is the difference evidenced in your plots. The data from the inverter appears to be timestamped 5 minutes later, so the timestamp is the end of the 5 minute period. I can’t see how your inverter does this, but I can see how IoTaWatt does it, and it is as I describe, so the inverter must be the other way. I would maintain that given the 00:00-23:55 “day” in PVoutput, the inverter is incorrectly time-stamping the data.

This has implications in your analysis of the problem, because the IoTaWatt data for a particular time interval would be sent 5 minutes after, not before, the inverter data for the same “period”, and any interaction would be caused by the IoTaWatt data “overwriting” any PVBC data, if that’s what’s happening.

8

Thanks for your input Bob.
I haven’t looked at the PVO Add status (?), just reporting what I have observed with my system.

With no “generation” data selected for upload, iota definitely writes or PVO interprets a zero value for that particular placeholder on PVO.

As I have said before, I am currently in a “learning how to configure” stage and as such, I am using 1 ( one) CT only. ( I have the VT of course )'It is plugged into “input 1” and I have named that input as “mains” ( as that is where it will eventually be installed with the final installation )

I have experimented with associating this “mains” CT to different “status outputs” in the web server area, to see what effect that has on the PVO graph.

It is very likely that at some point I had it assigned as “generation” and hence that data was ADDED onto the generation data that was already uploaded by PVBC. That makes sense.

I now have the “mains” CT assigned to “consumption” and the data appears to be valid.

There is no “net” or “Gross”. there is no “inverter output” before or after the mains CT. as i said, I have 1 (one) CT ONLY plugged in to the iota, which is just providing a “test” data stream.
It just so happens that the data stream is ~= inverter generation, but that is only because the cable was easy to get to, and I could validate the data really easily.

I like your point regarding the inverter output “What is the inverter reporting? Is it peak Watts during the period?” a very valid point. I have no idea exactly what the 5 minute inverter data really is. I just figured that it “should” be “more” accurate than a CT.

Once I am comfortable with the iota data and figure out what formulas to use etc, I will use it to monitor both generation and consumption.
I like the way that PVBC queries the inverter and saves the 5 minute data CSV’s, which also generate a weekly report to my email.
I’ll try and turn off the PVBC uploads and see if it still saves the CSV’s and pushes weekly data emails

Just a technical question; How much historical data does the iotawatt store ? and where does it store it ?
If I needed to re-upload a specific day from say, 3 months ago, how can that be done?

9

I’m not talking about the physical setup or what is going on in IoTaWatt, just what PVO perceives about the two data streams and their meaning.

You can get a 5 minute CSV file from IoTaWatt with the new query API

If you simply plot a day’s consumption and generation with Graph+, there is a button to download the CSV. A day will be at 2 minute resolution. A week will be at 20 minute resolution. Using the API, you can get a week at 5 minute resolution.

10

Tried the query api but all I get is a “bad request” , sounds good but !

11

My graph options are “original” and “enhanced”, no "save CSV’ button found.

Where do I find the graph+ ??

12

It’s in release 02_05_02. 02_05 has been in ALPHA and BETA auto update for several months now. I’ve just set auto-up[date class MINOR to that as well and will do MAJOR soon so as to get everyone on this newer platform. “bad request” would be normal on prior releases as the query API isn’t there.

This release also has the diagnostic trace that was added for the restart you saw, as well as a fix for what I suspect is the problem.

1 Like
13

Thanks Bob, my system updated to 02_05_02 @16:06 UTC.

Just had a play with the Graph+ and it looks really good.
Great that I can now download data CSV’s as well.

Tried out the query API and managed to get a couple of basic replies out of it.

Would be great if there was a “sticky” post that simply listed peoples query strings that they have used to retrieve “XYZ” data

Thanks for all your work Bob, you really are doing a fantastic job !

14

I’m thinking of adding away to view and copy the queries generated by Graph+

15

Nice. The more I get my head around the IotaWatt platform, the more I like it :grinning: