Connected Services API Output Question

Support
1

New IotaWatt user here. Loving it so far and greatly simplified and sped up a project that I was looking to do on my own. One outstanding opportunity I’m looking to setup is to write my own web service to receive the data. Once I have the data I have a number of future projects planned. Seeing as how C++ is NOT my native tongue it seems easier to just mock one of the existing APIs and continue running stock firmware.

Right, wrong, or indifferent I decided to mock the EmonCMS API. I think I was able to reverse engineer the majority of the info to understand how this works. However, I have one remaining question… In each array of data I think the first number is a time offset. Do I just take the time parameter that is sent to the web service and add that many seconds to it? Also, I’m receiving 26 arrays (up to 250). Can you explain the meaning of that, why 26?

[[0,“IotaWatt”,122.52,1330.44,139.34,85.88,0.00,0.00,0.00,0.00,0.00,0.00,0.00,0.00,0.00,0.00,0.00],
[10,“IotaWatt”,122.51,1327.10,139.32,85.90,0.00,0.00,0.00,0.00,0.00,0.00,0.00,0.00,0.00,0.00,0.00],
[20,“IotaWatt”,122.48,1327.79,138.66,85.89,0.00,0.00,0.00,0.00,0.00,0.00,0.00,0.00,0.00,0.00,0.00],
[30,“IotaWatt”,122.52,1326.01,139.00,85.95,0.00,0.00,0.00,0.00,0.00,0.00,0.00,0.00,0.00,0.00,0.00],…]

2

Reverse engineering is a challenge. The specs for the API are here. IoTaWatt uses the bulk protocol. The first entry in the array is the time offset from the provided time= parameter. The number of frames in one write is basically what IoTaWatt can fit in the buffer when uploading history.

You also need to respond appropriately to the initial /input/get?node= query. At a minimum you need to send a response "Node does not exist".

If you respond with a Json response in emoncms format, you can provide the time of the last successful post and IoTaWatt will resume posting from that time.

3

Thanks. That is helpful. Keep up the good work! I couldn’t be more thrilled with your offering and especially commend you for understanding what your purpose is and staying true to it. I see a number of people (like myself) coming up with new ideas and customizations and I see you respectfully repeating what your focus is and then providing additional info to steer us in the right direction. In my years of business it’s always been businesses such as yours that survive the course of time.

1 Like
4

You didn’t say much about what you are trying to do, so I just answered the question. You approach is probably the best if you want IoTaWatt to push the data. If you want to pull it, there is a query API that the graph app uses, and there is a real-time query API others have used that feeds the status display and is pretty simple.

5

So, my goal is to feed this data into Microsoft Azure. That way I can setup SMS alerts, take automated actions in response to events, and more generally fit it into my own custom built home automation system. I’m a .net developer who enjoys creating learning opportunities for myself and so things like using PowerBI for custom reporting and writing my own Xamarin mobile app for controlling circuits and reporting usage are of interest to me. The reality is that there’s easier ways, but I’m focused on learning and growing above practicality.

As you mentioned, I’m concerned with the graph api, from a cloud integration perspective, because it requires open ports on my firewall, dealing with static IPs and/or dynamic DNS, and security. Versus if I can get the IotaWatt to feed the data to me then all I have to do is build a web service to capture the data the IotaWatt sends.

If I was to make a product recommendation, it would be to add a 4th web server type… “Custom”. This would offer great opportunities for us to then install the IotaWatt and write our own customizations. Also, I’ve come across situations on the forum where people are interested in sending to either other cloud services and/or sending to multiple EmonCMS instances. For something like that I’ve come across a number of API Gateway type solutions where we could send one stream out (duplicate and/or transform the data as needed) and then redirect the request to one or more cloud endpoints. For the small price of a bit of additional latency we can do a number of things that are seemingly not easily possible today.

6

Ok, I get your push vs pull logic. Not saying I would do anything with it, but do you have examples of what general purpose REST service looks like? I’m not that worldly, is there some script or specification that is common for this? Also, how do you view the utility of MQTT?

7

I would actually vote for MQTT support over custom REST, despite the fact that I haven’t worked with MQTT to date (would like to someday). My reasoning is that MQTT is an ISO Standard for IoT and as such has wide existing support for various IoT platforms. FWIW, I think that direction would be better for your business than custom REST.

The biggest benefit of you developing a custom service (either REST or MQTT) to feed data would be that you are largely in control of the data format. I’ve done a significant amount of integrations between various platforms over the years and the biggest risk in these scenarios has always been what happens when one party changes their standards. Depending on the maturity of the vendor and their integrations community you may or may not receive advanced warning. If I mock the EmonCMS API I’d label it a “hack” at best. At any point and time they can change their standard, likely without my notice anyhow, and all of a sudden my integration will break. I’m not thrilled about that on a personal level, but generally a non-issue. However, risks of this sort in the business world equal all sorts of problems from instability to increased costs.

Should you pursue MQTT support, I’m comfortable with building an MQTT endpoint to receive and process the data feed. Once I receive the data I can basically do anything I want to with it… :slight_smile:

I would also be willing to help with designing the REST service if that would ever be of interest to you. However, I’ll warn you I’ve been to many tech conferences and there’s often a session or two labeled “How To Design A REST Service” and as you might imagine there’s a lot of varying opinions…

1 Like
8

Interesting discussion. I see two major use cases for data from iotawatt. First is the desire/need for historical data that is complete. IotaWatt appears to cater to that, with emoncms and influxdb support.

The other use case is what is going on right now. I might save the right now data for viewing later, but mostly I an interested in what is happening right now, not 10min or 5 years ago. For this use case MQTT is great. But be aware, while MQTT is a standard, the contents of a message is left as an exercise to the developer.

1 Like
9

Exactly right. MQTT is a transport system. The payload is undefined. An analogy would be to say that the Emoncms, influxDB and PVoutput feeds use HTTP, but the content of those messages is a higher level protocol. So it is with MQTT. We are left with the same problem as HTTP that the higher level protocol needs to be defined.

At a basic level, the topic with payload as a value is often the way I see this done, and that’s not a stretch for IoTaWatt. You define an output with a name, and send off a [sub]topic with that name and the scripted value.

I can see it working real-time only, and probably with the basic QOS. Either you get it or you don’t, there’s another one right behind it, and it is only synchronized to the extent that the broker delay is
minimal.