Interfacing with Home Assistant

Support
1

I’ve configured a REST sensor in Home Assistant to pull data from http://iotawatt.local/status?inputs=yes&outputs=yes&stats=yes. One input is my hot water system.

When on standby the IoTaWatt graph shows it is consistently using ~1.5W (+/- 0.25W) and so I have configured another template sensor in Home Assistant to report when the HWS is “on” (>100, because it jumps to 75W momentarily on startup) else “standby” (>1) else “off”. But the data reported by the status page linked above consistently shows that input as “0”, with very occasional jumps to “2” or “3” (~10 seconds apart when refreshing the IoTaWatt status page).

So Home Assistant thinks my HWS is always “off” instead of “standby”, but every half hour or so (when its poll happens to return to value of 2 or 3) it reports it as standby then immediately off again.

Is there a better way to get data from IoTaWatt and into Home Assistant? Why does IoTaWatt “damp” a consistent 1.25-1.75W value down to 0?

UPDATE: To be clear, I think a damped value should be OK for Home Assistant, but why is the damped value being reported as “0” 90% of the time when it’s actually a stable ~1.25-1.75W?

1 Like
2

Over time, the algorithms for the status page have been refined so as to give a good snapshot of what’s going on. For most folks, 1W should be zero, especially if it’s going from zero to one randomly. It just looks bad, never mind that it’s amazing IoTaWatt can detect 1W at all with a 50A CT.

I know there are folks that are using the status API to get data, and until now I’ve not offered any alternative, so I guess I should expect folks want to redefine what it is. Wait a minute, did he just say “until now”?

Yes I did. With version 02_05_01 now in ALPHA and BETA, there is a new query API that you can use to get data directly from the datalog. You could have done that before with the API used by the old Graph, but it wasn’t documented and doesn’t have a lot of the features of the new query API. With this API, you can ask for the current value, or the last n values. You can get the response in Json or as a CSV table.

The documentation is here https://docs.iotawatt.com/en/staged/query.html.

This query drives the new Graph+ that is also documented in that “staged” version of the documentation. I’ll be publishing a 02_05_01 version when that goes MINOR, probably next week.

2 Likes
3

Awesome. I’ll take a look at those docs and give that a try instead. I understand rounding 0-1 down to 0, but I still think a consistent value between 1 and 2 should be rounded to 1 or 2, not 0 :smiley:

4

It’s open source. Have at it and take the cure.

1 Like
5

@overeasy I am experimenting with the new show/select API now. I am having a bit of trouble figuring out how to get the current value for each input/output, and get it into Home Assistant in an easily usable format.

The closest I can figure out is to get the most recent 5 second average with http://iotawatt.home/query?select=[series1,series2,...]&begin=s-5s&end=s&group=5s&header=yes

But even then, the format of the JSON returned by IoTaWatt is not easy to access in Home Assistant via REST sensor. I’ve had to resort to rendering a custom JSON string via Jinja in an intermediate template sensor and then converting that string back into an object in a third template sensor:

Could we possibly add a new ha endpoint to the query API that returns the unit type and latest value (or average across the specified time period) for each input/output, as a top level map?

For example:

{'latest': {'voltage': {'unit_type': 'Volts', 'Volts': 249.5}, 'mains': {'unit_type': 'Watts', 'Watts': 123.5, 'Wh': 12345.6}}}

It should then be super easy to add this JSON data directly as an attribute on a simple REST sensor in Home Assistant, that could be used throughout Home Assistant as-is or subsequently used as the basis for further template sensors. Something like this:

sensor:
  - platform: rest
    name: IoTaWatt
    json_attributes:
      - latest
    resource: http://iotawatt.home/query?ha
  - platform: template
    sensors:
      hot_water_system:
        value_template: '{{ state_attr('sensor.iotawatt', 'latest').hot_water_system.watts }}'

Alternatively, an additional option to the existing select API to return a single/average value and input/output name in a map as an extra formatting option?

1 Like
6

Have you tried to use the CSV output format?

7

I’m open to developing a generic HTTP output service provided it could satisfy a larger audience.

As an old warhorse trying desperately to keep up with modern tools, I am constantly blown away by new tools that I discover. So I’m open to suggestion on this. But my current thinking is a basic service that sends a user formatted HTTP request at a specified regular interval to a some given URL.

This would be for current data only. The average of some specified interval relative to the sending time.

The payload could be a string defined with a C++ printf format, and the data fields could be strings and values defined as a set of calculator outputs.

I have not had time to explore this, but I think there are MQTT agents that will accept HTTP, and so this might be able to serve the purpose of directly producing MQTT from IoTaWatt.

If I had confidence that this would satisfy more of the feature needs of the user base, it’s really a pretty simple service.

1 Like
8

I have not tried CSV, but I don’t see how it could solve the problem I’m facing.

I want to access current input and output data by name so that changes to input names don’t require that I adjust a tonne of numerical indexes in home assistant configuration.

CSV would still have one row of sequential input and output names and one row of sequential values that I can’t zip together into a map in home assistant.

Having IoTaWatt push data via mqtt or having home assistant poll for data is neither here nor there for me.

I just need a URL to poll that gives me a json map of the data as a top level key (so I can save the whole map as a sensor attribute) with input and output names as the key.

Reformatting the existing output to provide each series of data in a separate list keyed by input name, instead of as a list of lists, could also work. But would require changing your existing consumers.

Adding a new formatting option that returns the data like that, keyed by input and output name, could support other consumers that work differently than the internal graph library.

It would also be nice if I could request “all” series without explicitly naming them all. I haven’t found a way to dynamically build a rest sensor resource url in home assistant that includes these, so I have to hard code them all.

9

FYI in case anyone is interested in Home Assistant integration, and to show @overeasy the kind of hoops I currently need to jump through, here’s a working (with some caveats) Home Assistant configuration that uses the new IoTaWatt select API:

1 Like
10

Would it be useful if the current value were included in the show=series query? My thinking is that it might default to the most recent measurement (s-5s, s) or allow specifying begin and end as in the select= query with group=all implied.

11

I think you’d need to add the current value(s) and display the series as a nested map (with the input name as keys) instead of a list of maps (with the input name as values).

With data like {"series": [{"name": "mains", ...}, ...]} I’ll still have to do some template gymnastics in additional “sensors” in Home Assistant that exist purely to reformat the data.

Might as well just keep show=series as-is and add show=current or show=ha that outputs more (all current) data and in a different format?

For example:

{
    "voltage_series": {
        "voltage": {
            "V": 245.6,
            "Hz": 50.1
        }
    },
    "power_series": {
        "mains": {
            "Wh": 567.8,
            "W": 123.4,
            "A": 5,
            "kVA": 6.7,
            "PF": "0.988"
        },
        "oven": {
            "Wh": 567.8,
            "W": 123.4,
            "A": 5,
            "kVA": 6.7,
            "PF": "0.988"
        },
        "air_conditioners_1": {
            "Wh": 567.8,
            "W": 123.4,
            "A": 5,
            "kVA": 6.7,
            "PF": "0.988"
        },
        "hot_water_system": {
            "Wh": 567.8,
            "W": 123.4,
            "A": 5,
            "kVA": 6.7,
            "PF": "0.988"
        },
        "solar_analytics": {
            "Wh": 567.8,
            "W": 123.4,
            "A": 5,
            "kVA": 6.7,
            "PF": "0.988"
        },
        "power_points_1": {
            "Wh": 567.8,
            "W": 123.4,
            "A": 5,
            "kVA": 6.7,
            "PF": "0.988"
        },
        "lights_1": {
            "Wh": 567.8,
            "W": 123.4,
            "A": 5,
            "kVA": 6.7,
            "PF": "0.988"
        },
        "power_points_2": {
            "Wh": 567.8,
            "W": 123.4,
            "A": 5,
            "kVA": 6.7,
            "PF": "0.988"
        },
        "lights_2": {
            "Wh": 567.8,
            "W": 123.4,
            "A": 5,
            "kVA": 6.7,
            "PF": "0.988"
        },
        "air_conditioners_2": {
            "Wh": 567.8,
            "W": 123.4,
            "A": 5,
            "kVA": 6.7,
            "PF": "0.988"
        },
        "fronius_inverter": {
            "Wh": 567.8,
            "W": 123.4,
            "A": 5,
            "kVA": 6.7,
            "PF": "0.988"
        },
        "latronics_inverter": {
            "Wh": 567.8,
            "W": 123.4,
            "A": 5,
            "kVA": 6.7,
            "PF": "0.988"
        },
        "air_conditioners": {
            "Wh": 567.8,
            "W": 123.4,
            "A": 5,
            "kVA": 6.7,
            "PF": "0.988"
        },
        "consumption": {
            "Wh": 567.8,
            "W": 123.4,
            "A": 5,
            "kVA": 6.7,
            "PF": "0.988"
        },
        "grid_export": {
            "Wh": 567.8,
            "W": 123.4,
            "A": 5,
            "kVA": 6.7,
            "PF": "0.988"
        },
        "grid_import": {
            "Wh": 567.8,
            "W": 123.4,
            "A": 5,
            "kVA": 6.7,
            "PF": "0.988"
        },
        "grid_import_neg": {
            "Wh": 567.8,
            "W": 123.4,
            "A": 5,
            "kVA": 6.7,
            "PF": "0.988"
        },
        "lights": {
            "Wh": 567.8,
            "W": 123.4,
            "A": 5,
            "kVA": 6.7,
            "PF": "0.988"
        },
        "mains_inverted": {
            "Wh": 567.8,
            "W": 123.4,
            "A": 5,
            "kVA": 6.7,
            "PF": "0.988"
        },
        "power_points": {
            "Wh": 567.8,
            "W": 123.4,
            "A": 5,
            "kVA": 6.7,
            "PF": "0.988"
        },
        "production": {
            "Wh": 567.8,
            "W": 123.4,
            "A": 5,
            "kVA": 6.7,
            "PF": "0.988"
        },
        "self_consumption": {
            "Wh": 567.8,
            "W": 123.4,
            "A": 5,
            "kVA": 6.7,
            "PF": "0.988"
        },
        "unmeasured": {
            "Wh": 567.8,
            "W": 123.4,
            "A": 5,
            "kVA": 6.7,
            "PF": "0.988"
        }
    }
}

Then my Home Assistant integration could be as simple as:

sensor:
  - platform: rest
    name: IoTaWatt
    resource: http://iotawatt.home/query?show=current
    json_attributes:
      - voltage_series
      - power_series

And I could access those values by input/output name in my HA automations, or create additional template sensors for the data I need to expose directly:

sensor:
  - platform: template
    sensors:
      iotawatt_air_conditioners_w:
        friendly_name: Air Conditioners
        unit_of_measurement: Watts
        value_template: '{{ state_attr("sensor.iotawatt", "power_series").air_conditioners.W }}'

It could select from begin=d to end=s by default (and allow you to input your own begin/end) and report the last recorded value in the range as W, and Wh for the whole period.

1 Like
12

Let e mull that over. I have a few questions:

Is one second data of interest?

If IoTaWatt could be ganged such that one could speak for all, how do you see that working in this scenario? Would it be a flat namespace where all of the ganged units would use unique names, or a hierarchical system where names are qualified by “node”? What would be the Json representation of that to HA?

Does HA only pull, or can you push to it?

13

On one hand, I’d love to see 1 second data, but I imagine that is overkill for Home Assistant integration. HA’s rest sensor polls at 30s by default, but it is configurable via scan_interval.

I don’t know if there’s a minimum value allowed, but I think 1s would probably produce too much data (HA logs it all to produce graphs of its own) for little actual noticable benefit. For example if an automation to automatically turn off a switch when all its devices enter standby mode triggers in 1s or 30s is of little consequence.

In terms of aggregating the data from multiple units, I don’t think it matters much. HA could be configured to fetch data from each IoTaWatt separately and the input/output names would not need to be unique. Users would access the data first via the rest sensor namespace.

If IoTaWatt were to aggregate the data, the input/output names could either be unique and flattened, or non-unique and grouped under a node. As long as the node itself had a unique and consistent name.

I don’t know if HA supports having data pushed to it. I assume so, but the rest sensor is so simple and easy to integrate with anything that can provide JSON data. I had a quick look at the MQTT docs but it wasn’t immediately obvious to me how it would work, and looks to require more configuration and a bit more learning than a rest sensor.

Plus anything that involves the IoTaWatt pushing data to HA would mean configuring two things, HA receiver and IoTaWatt sender. So I’m in favour of keeping things simple. The rest sensor is simple. It just needs a slightly different JSON format to overcome some limitations in HA and make the data more easily and consistently accessible.

UPDATE: Making the average period configurable so users can make it match the HA scan interval, and perhaps 30s by default, but allowing users to set it as low as 1s if they want to poll that frequently, would be good (if 1s polling is not too heavy for the IoTaWatt).

14

Might be a bit late to the discussion here - but I just set up my IotaWatt -to- home assistant link (Hass.io on pi4 if it matters).

My config/sensors.yaml has the following for each item I’m after –

- platform: rest
  name: iotawatt_Water_Heater_Power
  resource: http://xx.xx.xx.xx/query?select=[Water_Heater.watts.d1]&begin=s-10s&end=s&group=10s
  value_template: '{{value_json[0][0]}}'
  unit_of_measurement: "W"
  scan_interval: 10

I figure a 10 second average once every 10 seconds is perfectly fine for my general use. You could obviously change the interval, but I imagine too many sensors at a faster refresh might be a bit of a burden on both devices. The smallest interval I was able to request was for 5 seconds with “…&begin=s-5s&end=s&group=5s”

It did not work using “iotawatt.local” in the HA config, though that address does return results directly via browser or command line. I had to set the IotaWatt to a static IP (via DHCP reservation in the router) and use that IP in my hass.io config file.

Where I have [Water_Heater.watts] you can use the name of any input or output you have configured. After the dot, you can request any of the standard parameters (.volts, .hz, .watts, .wh, .amps, .va, .pf).

The ‘.d1’ at the end of the [Water_Heater.watts.d1] chunk solves OP’s problem directly by telling IotaWatt I want one digit after the decimal. This seems to give meaningful (though maybe not accurate) data up to ‘.d3’ .

I’ve also noticed that you can access undefined parameters for your outputs. For example I have an output called “Total_Power” which sums my A and B phase into a single Wattage value. Through the Rest request, I can get [Total_Power.wh] (the other values wouldn’t be properly calculated, so I don’t use them in this instance, but I could).

2 Likes
15

I’ve tried to automate the generation of template sensors for Home-Assistant.

Basically what I do is to ask my iotawatt which outputs that exists and generate template sensors from that.

So, go to http://iotawatt.local/query?show=series

And paste after set show = in the Template editor of Home-Assistant.

In my case I get the following:

{% set show = {"series":[{"name":"Voltage","unit":"Volts"},{"name":"TM","unit":"Watts"},{"name":"VVB1","unit":"Watts"},{"name":"VVB2","unit":"Watts"},{"name":"Dishwasher","unit":"Watts"},{"name":"Inverter","unit":"Watts"},{"name":"Living_room","unit":"Watts"},{"name":"Bedroom_2","unit":"Watts"},{"name":"Bedroom_1","unit":"Watts"},{"name":"Fridge","unit":"Watts"}]} %}

 - platform: rest
     resource: http://192.168.1.22/query?header=yes&select=[
{%- for s in show['series'] %}
{{- s['name']+"."+s['unit']+".d1" }}{% if not loop.last %},{% endif -%}
{% endfor -%}
]&begin=s-1m&end=s&group=all
    name: IoTaWatt
    value_template: "{{"{{value_json['data'][0][0]}}" }}"
    json_attributes: 'data'
    device_class: power
    unit_of_measurement: "{{show['series'][0]['unit'] }}"
    scan_interval: 60
  - platform: template
    sensors:
{%- for s in show['series'] %}
      iotawatt_{{ s['name'].lower() }}:
        value_template: {{ '{{{{ state_attr("sensor.iotawatt", "data")[0][{}] }}}}'.format(loop.index-1) }}
        unit_of_measurement: {{s['unit'] }}
        device_class: power
        friendly_name: {{ s['name'] }}
{%- endfor %}

Which renders this config:

- platform: rest
     resource: http://192.168.1.22/query?header=yes&select=[Voltage.Volts.d1,TM.Watts.d1,VVB1.Watts.d1,VVB2.Watts.d1,Dishwasher.Watts.d1,Inverter.Watts.d1,Living_room.Watts.d1,Bedroom_2.Watts.d1,Bedroom_1.Watts.d1,Fridge.Watts.d1]&begin=s-1m&end=s&group=all
    name: IoTaWatt
    value_template: "{{value_json['data'][0][0]}}"
    json_attributes: 'data'
    device_class: power
    unit_of_measurement: "Volts"
    scan_interval: 60
  - platform: template
    sensors:
      iotawatt_voltage:
        value_template: {{ state_attr("sensor.iotawatt", "data")[0][0] }}
        unit_of_measurement: Volts
        device_class: power
        friendly_name: Voltage
      iotawatt_tm:
        value_template: {{ state_attr("sensor.iotawatt", "data")[0][1] }}
        unit_of_measurement: Watts
        device_class: power
        friendly_name: TM
      iotawatt_vvb1:
        value_template: {{ state_attr("sensor.iotawatt", "data")[0][2] }}
        unit_of_measurement: Watts
        device_class: power
        friendly_name: VVB1
      iotawatt_vvb2:
        value_template: {{ state_attr("sensor.iotawatt", "data")[0][3] }}
        unit_of_measurement: Watts
        device_class: power
        friendly_name: VVB2
      iotawatt_dishwasher:
        value_template: {{ state_attr("sensor.iotawatt", "data")[0][4] }}
        unit_of_measurement: Watts
        device_class: power
        friendly_name: Dishwasher
      iotawatt_inverter:
        value_template: {{ state_attr("sensor.iotawatt", "data")[0][5] }}
        unit_of_measurement: Watts
        device_class: power
        friendly_name: Inverter
      iotawatt_living_room:
        value_template: {{ state_attr("sensor.iotawatt", "data")[0][6] }}
        unit_of_measurement: Watts
        device_class: power
        friendly_name: Living_room
      iotawatt_bedroom_2:
        value_template: {{ state_attr("sensor.iotawatt", "data")[0][7] }}
        unit_of_measurement: Watts
        device_class: power
        friendly_name: Bedroom_2
      iotawatt_bedroom_1:
        value_template: {{ state_attr("sensor.iotawatt", "data")[0][8] }}
        unit_of_measurement: Watts
        device_class: power
        friendly_name: Bedroom_1
      iotawatt_fridge:
        value_template: {{ state_attr("sensor.iotawatt", "data")[0][9] }}
        unit_of_measurement: Watts
        device_class: power
1 Like
16

I just installed two IotaWatts, and am looking at what people have done so far.

This topic looks like progress has been made but it begs a more basic question: Should HA be pulling data directly, or should we have Iotawatt push into Influxdb, and then HA pulls what it needs (including graphs) from it? Or maybe Emoncms though i have yet to look at it.

The above very clever (seriously) approach looks nice, but if I’m understanding it, it pulls a 1 second view every 60 seconds, and if HA is ever down there’s a gap. Iotawatt (again, if I understand) will buffer data and keep it continuous in Influxdb or emoncms. Haven’t done it, just from reading.

Bottom line question (as I try to pick a direction): What are you choosing to pull directly instead of via a data store?

17

I’ll let the HA experts explain the way it works, but the query pulls a 60 second average every 60 seconds.

18

So, with the upgrade to 0.118 this now fails to work.

so I my sensors.yaml I had

- platform: rest
  name: IoTaWatt (Select)
  resource: http://192.168.10.3/query?select=[AC,Main,Tesla,Total,Export,Import,Phase3_VT,Total_Solar]&begin=s-5s&end=s&group=5s&header=yes
  json_attributes:
    - labels
    - data
  value_template: 'on'
- platform: template
  sensors:
    iotawatt:
      friendly_name: IoTaWatt
      attribute_templates:
        data: >-
          {
          {%- for i in range(0, state_attr('sensor.iotawatt_select', 'labels')|length|default(0)) -%}
          "{{ state_attr('sensor.iotawatt_select', 'labels')[i] }}": {{ state_attr('sensor.iotawatt_select', 'data')[0][i] }}{% if not loop.last %}, {% endif %}
          {%- endfor -%}
          }
      value_template: 'on'
    iotawatt_ac:
      friendly_name: AC
      unit_of_measurement: Watts
      value_template: '{{ (state_attr("sensor.iotawatt", "data")|from_json).AC }}'
    iotawatt_main:
      friendly_name: Main
      unit_of_measurement: Watts
      value_template: '{{ (state_attr("sensor.iotawatt", "data")|from_json).Main }}'
    iotawatt_total:
      friendly_name: Total
      unit_of_measurement: Watts
      value_template: '{{ (state_attr("sensor.iotawatt", "data")|from_json).Total }}'
    iotawatt_tesla:
      friendly_name: Tesla
      unit_of_measurement: Watts
      value_template: '{{ (state_attr("sensor.iotawatt", "data")|from_json).Tesla }}'
    iotawatt_solar:
      friendly_name: Solar
      unit_of_measurement: Watts
      value_template: '{{ (state_attr("sensor.iotawatt", "data")|from_json).Total_Solar }}'
    iotawatt_import:
      friendly_name: Import
      unit_of_measurement: Watts
      value_template: '{{ (state_attr("sensor.iotawatt", "data")|from_json).Import }}'
    iotawatt_export:
      friendly_name: Export
      unit_of_measurement: Watts
      value_template: '{{ (state_attr("sensor.iotawatt", "data")|from_json).Export }}'
    iotawatt_voltage:
      friendly_name: Voltage
      unit_of_measurement: Volts
      value_template: '{{ (state_attr("sensor.iotawatt", "data")|from_json).Phase3_VT }}'

this no longer works. in the ‘IoTaWatt (Select)’ the content is:

Labels
AC, Main, Tesla, Total, Export, Import, Phase3_VT, Total_Solar
Data

    • 26.8
    • 1276.5
    • 3.2
    • 1306.5
    • -1070.3
    • 0
    • 246.1
    • -2376.8

Any ideas how to update that with the new “native” type template?

19

I’m working on a new integration in Home Assistant that creates IoTaWatt entities without using templates.

Right now it’s a work in progress and has to be added manually. Hopefully I can get it added to Home Assistant in the new few releases. But if you’re feeling adventurous the details are here at the Home Assistant forum.

2 Likes
20

Hello @gregtd,

This is fantastic. Over the past year this thread has had 1.3K views, so there’s a lot of interest in a native integration. I see where you had one successful user in the HA forum. If there’s anything I can do to help get this merged into HA, let me know.

Bob