New Install, two IotaWatt, US Split phase panel - photos

Your solution is only as good as the lookback window, and while the influx folks may think it’s a great solution on an AWS server, it might not scale down to a RPi very well.

I’m running on a HyperV on an old windows PC, on SSD, so I’m OK with it so long as it doesn’t degrade in some fashion due to internal InfluxDB data structures. I haven’t looked into what kind of maintenance/reorg is needed there (if any – I’m guessing with their Shard structure there is none).

I’m currently considering writing an external chron task that would submit downsample queries.

That’s likely the simplest, though I had in the back of my head to look if there’s something in HA where I can do it also (or Node Red).

One problem I find with cron is I tend to forget about them, especially once they broke crontab up into user specific files, there’s now WAY to many nooks and crannies for me to write a cron job then forget I wrote it.

Now one might call that a flaw in my memory, but I’m sticking with blaming Linux.

I really don’t get why a time series database can’t handle the concept of late arriving data and scheduling queries. I mean they CALL Them continuous, but they aren’t. If they were – if all data subject to them ran through them on arrival – this wouldn’t be an issue!

Take a look at Kapacitor. I didn’t get deep enough to see if it can do that. There is a stream handler that would work if it can operate on the input stream to influxDB. The way I read it, it had to operate on a stream out of a query. If you have time see if you can figure it out. It’s a pretty modular concept.

1 Like

This is exactly what I’m looking to do… Maybe I’ll setup an Influx / Grafana. I was trying to get away from my day-job with this, but this is too spot on. Nice work.

In the ongoing saga…

A few days ago I stupidly left a refrigerator door ajar, and made a bit of a mess, and it occurred to me that IoTaWatt and HA gives me the tools to deal with that. What I really want is a temperature alarm inside the fridge, but batteries do not work well there, and I’m not sure about running a USB cord in through the door insulation. Maybe.

But this is easy. I’m taking minute by minute watt measurements, and I see the fridge is only a few watts at idle, and 100+ separately, and tends normally to run in 30-45 minute periods. So…

So I fired up Node Red and built a flow that is centered on this influxdb query:

select time, ct, Watts 
from Watts 
where Watts < 40 and ct =~ /Fridge/ 
group by * 
order by time desc 
limit 1 
tz('America/New_York')  

(Line breaks added for clarity). I have two fridges, and the “ct” distinguishes them. This returns a message payload that has the time (sadly not TZ converted, so that aspect is probably moot) of the last time that fridge was using less than 40 watts.

Then in the next node I pull it apart, calculate whether I’ve complained recently or need to complain. The core math looks like this in a function node:

for (loopcnt=0; loopcnt<msg.payload.length; loopcnt++)
{ 
    var last = new Date(msg.payload[loopcnt].time); 
    var diff = now - last; 
    var min = Math.round(diff / 1000 / 60); 
    if(diff > FridgeMaxRuntimeMS)
    {
      speakstring += (speakstring===''?'':', '); 
      speakstring += msg.payload[loopcnt].ct + " has been running " + min + " minutes "; 
    }
} 

A nice aspect of Node Red is the loop – I could handle any number of devices that fit this sort of model.

FridgeMaxRuntimeMS is a context variable that gives the limit of how long I let it run before complaining (I use 90 minutes, the time is in milliseconds). I then construct an announcement string and let Google broadcast it on all the house speakers. There’s some logic also in there so it will only repeat the announcement every 30 minutes not even loop.

I did this in Node Red inside of HA, instead of HA itself, as to do it in automation in HA you still need that sort of query (probably in a value template that turns it to “alarm” or “not alarm”), but dealing with the “when and how do I alert” is much easier in Node-Red. I’m doing almost all automation in Node-Red now.

I also threw in a separate calculation into a rollup field for duty cycle that looks like this:

SELECT count(running) / count(total) AS DutyCycle INTO HA.hourly.DutyCycle 
FROM 
   (SELECT Watts AS total FROM HA.autogen.Watts GROUP BY *), 
   (SELECT Watts AS running FROM HA.autogen.Watts WHERE Watts > 40 GROUP BY *) 
GROUP BY *, time(1h) fill(0) 

This looks at the minute by minute periods to see when a device is running over 40 watts (a rather arbitrary “idle” limit) and saves as a fraction of total intervals. So this is a decent estimate of the duty cycle in use.

I’m not sure it is hugely better than just kWh used over time, but I figure if it starts to show more a more erratic nature, or starts to grow significantly, it might indicate some kind of issue. I didn’t do this just for the fridge but for anything – the A/C duty cycle should somewhat track outside temperature, if over months I start seeing a significant change I can look into its efficiency or refrigerant levels.

Kind of cool the stuff you can get.

Oh… I left the door open on a fridge that just had sodas and such in it (i.e. nothing went bad); 90 minutes later Google starts complaining. I was surprised, I closed the door and it took almost 2 hours to stop running, so four complaints at least. Good thing google’s got a polite voice.

Here’s what some of the data looked like. One thing I found odd was that my fridge apparently has two speeds for the compressor – after failing to cool adequately for a while, it doubled its current draw. You can see how the yellow envelop, which is duty cycle, pretty much parallels kWh, so it’s not clear how useful that is going to be over time, but who knows.

The peak at about 4pm on 3/27 is when I loaded it up after a trip to the store. What’s mysterious is the 5am peak on 3/29 – some kind of defrost cycle? It jumped up to 500w (from about 100 normally) for about 20 minutes.

Anyway, just sharing stuff in case it helps anyone with ideas.

2 Likes

Great details and use cases for how the IoTaWatt can help monitoring and alerting on various details. Thanks for the hard work and posting the info.

I would additionally suggest one pick up a freezer temperature sensor (Accurite sells some) that operate on the 433mhz frequency. Use this to hook into whatever tool using an SDR (software defined radio) and NodeRed as well. I use this type of setup to being in Temp from many sensors around my property along with my energy production/consumption.

Good work on this method as well.

Hopefully @overeasy will yell if I am filling up his server too much…

So today I stopped using Continuous Queries and moved them into Node Red. By putting them there, I can run them any time with any parameters. I set up something simple so they all run at 6 minutes after every hour, and resample for 2 days.

The resample is all just done inside a simple function node like this:

var now  = new Date(); 
from = new Date(now - (2 * 24 * 60 * 60 * 1000));   // 2 days 
var hourlyFrom = new Date(from.getFullYear(), from.getMonth(), from.getDate(), from.getHours(), 0, 0); 
var dailyFrom  = new Date(from.getFullYear(), from.getMonth(), from.getDate(), 0              , 0, 0); 
if(msg.payload === "DOALL")
{
    hourlyFrom = new Date("2000-01-01T00:00:00Z");
    dailyFrom  = new Date("2000-01-01T00:00:00Z");
}
var hourlyTo = new Date();
var dailyTo  = new Date(); 
var newMsg = { dailyFrom  : dailyFrom, dailyTo:    dailyTo, 
               hourlyFrom : hourlyFrom,  hourlyTo: hourlyTo 
             };  
return newMsg;

Essentially this just picks a time period from an even hour and day boundary about 2 days ago, up to the current time, and passes it out where the next function nodes weave it into a query string, e.g. like this:

var queryStr =  "SELECT integral(Watts, 1000h) AS kWh INTO HA.hourly.kWh FROM HA.autogen.Watts where time >= '" + msg.hourlyFrom.toISOString() + "' AND time <= '" + msg.hourlyTo.toISOString()  + "' GROUP BY *, time(1h)"; 
var newMsg = { query: queryStr };  
return newMsg;

The former function allows an override of “DOALL” which I inject from an inject node if I want to wipe all data and recalculate from the basic minute by minute data (obviously not possible beyond the retention period).

It would be easy to have two inject nodes, one runningly daily and one hourly, so it recalculates daily only once a day, but I kind of like day-to-date data.

Maybe I’m old school, but I like that I can drive the queries any way and any time I like; the continuous queries default intervals as well as exactly when they ran were … annoying. Node Red is a pretty cool automation tool.

Looks like this:

Can you share a info link to the freezer/refrigerator sensor you are using? Thanks.

You could go with the more expensive route such as this Acurite package (still on 433mhz), but no need for the display unless you want to have a display and be able to collect them via a SDR.
Brushed Stainless Steel Digital Refrigerator and Freezer Thermometer

However, I use 8 of these devices around the outside (US Northeast so it can get cold), and inside of the home. Indoor / Outdoor Temperature / Humidity Sensor – Weather Sensors & Parts | AcuRite Weather

You could put one of these in both the fridge and freezer and then collect the data from the air via an inexpensive SDR (Software Defined Radio). I use this (Amazon.com) from Amazon for SDR and an Open Source tool called RTL_433 which reads and decodes items from the air over 433mhz. I then use NodeRed to take that inputs and push them to EmonCMS; or other tool.

Hope that helps.

Thanks, Just what I’m looking for!

I use similar on a Rpi zero and have RTL_433 automatically start & scanning 433 and 915 mhz and send the data to influxdb, then on to Grafana for alerts. Thanks again.

Sounds like you are on track for one or two of the tower sensors then. With that, you should be good to go with your existing setup. Let me know know it goes.

So I might make this work by putting it on a separate rPi, but my HA runs in a VM that can’t use a USB.

I tried using a WIRED esphome based device for this, and it worked beautifully, except the gaskets around fridge doors are not designed to let you run a wire inside, they leak air. The fridge wouldn’t be so bad, but I had a snow storm in the freezer. Using an esphome device on batteries is not going to work well unless you like replacing batteries.

Have you run across any of the 433mhz devices whose base (I don’t mind having the display if needed) are then easily integrated via wifi? Or zwave?

PS. The wire-into-fridge depends a lot on model. If I had a pure side-by-side, I would put a hole in the gasket and all would be well. But I have two fridges with french doors so the freezer is a pull out drawer. This means you can’t go through the gasket (too much wire slack needed) but the gasket has to snug up to the wire on being closed, and this just doesn’t work. Maybe cut the gasket up and lots of soft silicon. Maybe. Send photos if you get it to work.

Most units have internal electrical components like lights, sensors and fans. They usually exit through a harness in the back of the unit. Typically all that is accessible by removing some plastic covers. Might be worth looking into running wires out through the same hole in the back.

I have not looked for devices that use WiFi or Zigbee as a means of transmitting a signal. I know the 433mhz is the cheaper solution and this is how you can get a unit for ~12.00USD or cheaper. You could even build your own if you wanted using an ESP8266 and some sensors, etc. I think it may be more work and more expensive than the off the shelf option. I have a dedicated RPI running the software which has all the decoding and formatting options like JSON, XML, TXT, etc. which can be consumed by most tools like NodeRed.

A wired solution would be good if you can find a way to get the wired into the device easily without much loss of cold air. A drill and some silicon sealer?

I should probably take a run at doing that. I looked in the service manual for one of the fridges, and it looked like these were all sealed up in something that looked like epoxy blocks (from a picture, remember I did not disassemble). Maybe I should take one apart and look. But they are full of food (you know, trying to limit grocery trips to once every 10 days or so) and I have this vision of breaking something and can’t get parts/repair and meat spoiling …

But I really should look.

My approach would be to start by removing the back to see if I could locate where and how the harness goes through. Shouldn’t need to disturb the innards. If it looks possible, I’d probably see if there is a YouTube of someone servicing an interior component. I love YouTube DIY videos.

Another approach might be to look at a schematic to see how the units own temperature sensors work. If they are 10k sensors, it’s pretty easy to sense that directly from the external PCB using an arduino or ESP.

1 Like