@overeasy It is close enough. I think HA has a rounding issue.
Comparison to PVOutput which is 100% correct
That’s helpful.
HA is integrating 30 second averages where PVoutput is getting 5 minute averages, so I would expect PVoutput to miss a little and be lower.
How do you know that?
@overeasy I thought that the PVOutput implementation takes ALL values and sends the average over the 5 mins.
The PVOutput results are always very very close to my meter reads.
The HA implementation only get a value every 30 seconds and averages that. Just assumed that this was inherently less accurate.
Sorry, No data behind such claim.
I confirm that the above approach works. Let me try to make it as clear as possible, by describing a step-by-step approach.
This is my home setup:
I have a 3-phase system without neutral. Total use is measured with 2 CT’s, connected to inputs 1 & 2 of my iotawatt. Solar is measured using CT on input 3.
Inputs configured in Iotawatt:
Looking at my home setup, you’ll see that the power exchanged with the grid is:
Grid = (Bord_Mid + Bord_Rechts) + Solar
(in this configuration, Solar has negative values when my solar panels produce)
I configured following outputs in my Iotawatt:
With these formulas I calculated not only my grid exchanged power, but I splitted this into two separate components: import and export. This split is needed in Home Assistant energy management.
The following graph visually shows how the split is made:
Now let’s go to Home Assistant:
Go to Developer Tools.
In the ‘States’ page, search for all entities which have the attribute “type: Output”:
This will display all outputs you defined in Iotawatt.
You’ll notice that each output is listed twice.
For example, output “export” has entities:
- sensor.export : contains the Watts value
- sensor.export_wh : contains the Wh value
We will need the Watts values.
Write down the entities names for the import, export and solar measurements.
In my case:
- sensor.export
- sensor.import
- sensor.solar_2
Now open your configuration.yaml file, where we will add some configuration. For more information on how to open this file: Configuration.yaml - Home Assistant .
Add the following lines to the configuration.yaml file:
sensor: - platform: integration source: sensor.import name: import_wh_calculated unit_prefix: k round: 2 - platform: integration source: sensor.export name: export_wh_calculated unit_prefix: k round: 2 - platform: integration source: sensor.solar_2 name: solar_wh_2_calculated unit_prefix: k round: 2
This applies the mathematical Rieman Integral to the Watts values (Integration - Riemann sum integral - Home Assistant).
It converts power (Watts) values into energy (Wh) values. The Home Assistant energy management works with energy values.
Now we go to the Energy configuration screen: Configuration > Energy dashboard.
There, assign the newly created energy entities to your dashboard:
Now, your Energy dashboard in HA will be populated:
The profile matches well with the graph I generate in iotawatt Graph+:
Hope this helps
Kind regards,
Michaël
2 Likes
Thanks @dotnetdan,
The 5 minute average is mathematically an average of all the values in the interval - typically 60. But so is the 30 second average to HA. If both methods give the same import during an import only period as at night, or during the middle of a very sunny day when there is no possibility for import, then I would lean toward the smaller 30 second increment giving more accurate results. That’s all good information.
The good news is that both are very close and reasonably accurate.
The bad news is that because PVoutput uses the difference between generation and consumption to calculate net import/export, no amount of integration of the 5 second data can improve it.
The HA Energy model can be made dead accurate to 5 second integration the way it’s setup now. They are talking about tapping into a net-energy value to get import and export. If they do that and remove the current method, they too will have limited accuracy with larger intervals.
A new solution is coming together nicely. It may require an external integration to get around the inappropriate rules for HA acceptance, but we’ll get it done.
Thanks Michael,
I was not aware that you are a three-wire system. I don’t have that documented. Did you get the setup from the forum or did I get involved? Is it derived reference and how well does it match your meter?
I always like a line drawing. Takes out all the guesswork. You should send this to our mutual friend. I have a few comments/questions:
Why integrate the Solar? Isn’t it always negative or nearly so with standby power?
Have you upgraded to the new HA release and latest integration?
I can see that the 30 second integration in HA is a little more accurate than the one hour integration in Graph+, especially in the transition hours. Once it gets a head of steam in the midday, they appear identical. Again as expected.
If you click on statistics with Graph+, you should see slightly different Wh numbers for import and export. As an exercise, click group=auto and I suspect they will get much closer as the interval will come down from 1 hour to 2 minutes.
So proof of concept from both you and @dotnetdan. I’m pretty confident in proceeding according to plan.
I’ve been at a disadvantage for years without solar, relying on others to do this. I’ve had the idea to insert a software pseudosolar input that could be used like an input in scripts and queries. The idea would be to produce the power and energy over any period assuming a sine profile from say 8-4 multiplied by a system power constant.
@michaelpiron - wow great wrote up. I just got my iotawatt last week (good timing with hass energy dashboard) but im having an issue i cant see my head thru it. Seems you have totally nailed it so thought id ask you. two problems actually
1/ Basically hass / iotawatt assumes my house is using the ENTIRETY of my solar produced. Thats not the case. It should be more like something like the red hand drawn line below.
2/ second issue is ‘Return to grid’ is showing 0 (yellow hand drawn line). Also i think it actually winds back my ‘consumed from grid’ numbers.
Your solar producing negative confuses me. Should mine be? Its currently positive. Have i got my CT clamp the wrong way around? Ive created 2x outputs (mainsconsumed, mainsexport) as per the hass docs. Ive not created your solar output logic. Any help really appreciated, thank you!
you need to follow @michaelpiron post to the letter - including adding the integration step in configuration.yaml file. Prior to going through the steps - my results looked like yours. Now - they’re showing up correctly with return to grid and consumption graphed correctly - this is my graph from the other day -
Ok thanks @dheatherly - good to know im in the right place for help! Seems i need the following new output (i only have one phase)
GRID = PHASE1 + SOLAR
(And yes i already have setup entries in my iota + hass for mainsconsumption and mainsexport as per docs here IoTaWatt - Home Assistant so i think im good here for the other two)
My question stands RE getting negative values for solar though. I thought i read you should put the CT clamps over the wires with the arrows pointing in the way the electricity flows. Ie solar system toward switchbox. ?
Just noticed the original poster (@Sciro) has this output config. Does this assume every circuit in the switchboard is CT clamped? Im not currently doing this (i did purchase enough CT clamps to cover all, but im walking before i can run!)
I am falling down with electricity concepts here. I am technically minded but this hurts my head. I guess my question here is in order to get HASS dashboard working, do i need to clamp ALL my house usage circuits as a prerequisite before i can sort out the return to grid problem (my screenshot three posts above).
There are a lot of different ways to setup the IoTaWatt, but most users, and most examples in this forum, adopt the convention of solar being positive, mains being positive when importing power, and branch circuits being positive. If your solar connects to the grid through your load-center, as most in North America, then the mains will go negative when exporting power to the grid.
While @michaelpiron example is fine for his setup, I cringed at the idea of having the example be a three-phase system with the solar opposite to convention, and what is by convention called a type-1 installation where the solar feed is upstream (on the grid side) of the mains. Both are atypical.
Even with the solar observing this convention, the solution is not cookie-cutter. It varies depending on:
-
Do you have single, split or three-phase service?
-
Does your solar connect to the grid before or after your mains CTs?
Perhaps if you posted your inputs and outputs configuration, I can take a pass at it. The Hass people don’t seem to appreciate that IoTaWatt provides an intelligent layer between the CTs and HA, and using that adapts to any power system worldwide.
Thanks @overeasy - i really do appreciate your time. i am single phase here is sydney australia and here is my very simple understanding of the setup i have here this far. Including the extent by which CT clamps are currently deployed (red C) and their directionality.
As mentioned i did purchase enough CT clamps to cover all circuits (the remaining black Xs) so if me deploying them all is part of the optimal solution im of course open to that)
Heres my inputs/outputs at the minute:
In terms of your other questions
-
single phase
-
RE ‘Does your solar connect to the grid before or after your mains CTs’ - im not entirely sure, BUT my setup also shows negative values on IoTaWatt when solar is producing more than house is consuming and its flowing back to grid, so does that confirm one way or another?
Thanks again.
I think this is about as straightforward as it gets, but given that I have no direct experience with this yet, I hope I can get it right.
HA Energy wants to know three things:
- How much energy was imported from the grid.
- How much energy was exported to the grid.
- How much energy was generated locally.
You have said that your solar has the ability to drive your mains negative, so that indicates you have what is by OEM convention a type-2 solar service, meaning the solar inverter feed downstream (on the load side) of your mains. So your single CT on the mains provides all the data you need for items 1 and 2. Import is the positive component and export is the negative component.
I would change MainsConsumption as you have it to Consumption = GRID + SOLARPRODUCTION
Then define a new output MainsImport as GRID max 0 that satisfies Item 1 above. MainsExport satisfies item 2. SOLARPRODUCTION satisfies item 3.
Now change the YAML configuration entry to:
sensor:
- platform: integration
source: sensor.MainsImport
name: import_wh_calculated
unit_prefix: k
round: 2
- platform: integration
source: sensor.MainsExport
name: export_wh_calculated
unit_prefix: k
round: 2
.
The Energy configuration stays the same, except Solar production is changed to SOLARPRODUCTION_wh
Give that a try. Unlike influxDB or PVoutput, there is no way that I’m aware of to update the history in Hass, so unfortunately this will only take effect going forward.
Thanks for posting this, please post your results in a day or two when it shows results. This is exactly the simpler example I had in mind as it’s more of a typical case. Once (if) this proves out. I’ll elaborate on the simple changes needed for split-phase systems and type 1 (solar feed upstream of the mains) systems.
amazing. Thanks so much for your time helping me through this. Ill try and get to after work today (after 4 hours or so). Ill loop back soon.
Ok so i think ive made that changes ok in IoTaWatt:
And HASS config yaml (although question - the hass docs specify iotawatt: keyword, shaded, let me know if thats a drama)
But might be coming unstuck on the energy dash config. Heres how i have it now (a guess).
….
…. But a bit confused by the other entities that were created by the yaml and if you wanted me to pick them instead
Thanks let me know how im doing!
Keiran
I think you’re close. The Energy configuration uses the outputs of the Riemann sum integrations that you created in the YAML file, so Grid consumption should be import_wh_calculated and Return to grid should be export_wh_calculated.
Ok awesome- i just flipped them. Ready for a clean run tomorrow on these settings. Will screenshot the dashboard tomorrow after we start getting solar in.
1 Like
Just one more thing. I don’t know if Hass sensor names are case-sensitive, your YAML integrations are all lower case source: names. The outputs from IoTaWatt have some upper case. Might be prudent to make those the same.
It works! (i am now seeing my below 0 vales plotting as purple, and increasing numbers going back to the grid).
As for the lowercase thing, yeah no dramas there, HASS normalises all entities to lowercase names, so my source: is fine.
And while we are at it, heres the value of the export_wh_caculated you had me define in the YAML:
im a bit confused about why we need YAML config for 2 out of my 3 IoTaWatt outputs (we are missing the ‘Consumption’ one… actually checking my HASS config i dont believe ii even reference this anywhere? do i even need that output?)
Now that i know this works, ill go through and rename stuff to make it a bit more user-friendly to me 
(so i can remember what the hell we did here a year down the track).
thank you so much again! You have done your small part for the environment today my friend
Keiran.