I’m working on a project and would like to gauge community interest.
Briefly, it’s a Home Assistant App (add-on) that fixes many the common HA pain points:
- Missing history during HA reboots or communication dropouts.
- Inability to have use outputs as HA sensors.
- Inability to see 5-second granularity data.
- Inability to automatically import history and long-term statistics into HA.
In a nutshell, the app automatically configures the influx uploader to send the necessary buckets from all inputs (and outputs) at a 5-second post interval (or whatever post interval you like) starting at any available historical date you choose, then records that data for use within HA. It essentially reconstructs the IoTaWatt log fields in a local database. (This is trickier than it sounds.)
Once the uploader catches up, the data also populates the same W/WH and V/hZ sensors as the existing HA integration, as well as any outputs (including ones that use integrators) configured on the IoTaWatt and keeps those updated as sensors as well.
Any time the app restarts, recovers from a lapse in the uploader posts, or any time HA restarts, the app will attempt to fill in any missing history in HA.
The app supports multiple IoTaWatts. (I have 400A service, so I have 2 panels and 2 IoTaWatts.)
For the first version, the app acts as an IoTaWatt to MQTT bridge, which means you don’t need a separate integration if you already have MQTT. In the future I might develop a companion app or create a PR to allow the core integration to pull data from the app rather than directly from the IoTaWatt(s).
There’s more, but this is already a long post - so the rest is some background on how the project came to be.
The history of the IoTaWatt/HA integration is somewhat long and complex. There are fundamental incompatibilities between the HA and IoTaWatt approaches to handling data that can’t be resolved by an integration alone. HA sensors are always “immediate” - so you lose history during the time HA reboots or if there is a connectivity issue with the IoTaWatt. Coming the other way, there is no “unique ID” for IoTaWatt outputs, so they are not eligible to be a device entity in HA. Both of these have been fiercely argued in numerous forums, and the compromise is what we have today. Fundamentally, HA doesn’t consider itself a historian. It’s a recorder. The HA devs have added APIs to allow for modifying history and long-term statistics, but nothing in the core IoTaWatt integration that brings it full circle.
It’s frustrating because all of the granular data is faithfully recorded in the IoTaWatt log for a whole year, and less granular long-term statistics for an additional year - but the standard integration is useless for dealing with this.
There are other problems. It’s common to encounter HTTP exceptions in the HA logs whenever the integration times out on a query, which in my case is relatively common for one of my IoTaWatts that doesn’t have a perfect signal. The responses can be slow, creating a choppy sampling set. It’s also not possible to get 5-second granular data for inputs/outputs using the core integration. I’ve played with changing the integration to support this, and it’s a disaster with timeouts and seems to even cause spurious IoTaWatt reboots.
You can “fix” some of this using a combination of Ingress or EMonCMS and various tools and tricks to import the data into HA - nothing fully automatic and self-contained exists.
So, with some free time and an abundance of frustration with the limitations of the existing system, I started working on this project. I have been through several re-designs as I learned more about how both IoTaWatt and HA do things internally.
I’ve mostly wrangled it down to a well-behaved HA app (or docker container if you prefer) that uses very few resources, as little storage space as possible, and handles most of the corner cases I see with my system.
I still have some wrinkles to iron out before I would consider publishing, but if there’s enough interest, I will continue ironing and do the extra work to push to a public repository and publish as a Home Assistant app.
I’m also open to criticism and to answer questions if anyone is curious about the details.