Posting to emoncms via Dataplicity

Support
1

Sorry, another request for help getting IotaWatt data into an emoncms server …

I helped a friend get his IotaWatt setup today and it works fine in his house. We want to send his data to my emoncms server running at my house but I’m not seeing it show up as an Input. I use Dataplicity make my emonserver accessible outside my local network, so I set up the emoncms service as shown here:
Als%20Iota%20setup
Note I use http not https, and I do not specify a port. My emoncms server is working fine - it’s currently receiving data from a bunch of other devices. The Emoncms log does not show any indication of an IotaWatt input stream (i.e., no authentification errors). I’m quite certain I’m using the correct emoncms API Read/Write key.

He restarted the IotaWatt after saving the web server edits. Still no Inputs showing up in my emoncms server. His IotaWatt log shows this - note the :80 port is specified here:

Als%20Iota%20log
Als Iota log.JPG1009×96 22.2 KB

Any suggestions? Thank you!

2

I can’t say what’s going on here because I don’t know how this network pass through is supposed to work. You are getting a 301 permanent redirect, so it looks as if the gateway is dying to tell us to go someplace else.

I’d suggest you look at the Emoncms input documentation and try to talk to it from outside using a browser. Once you establish contactwith intelligent life inside the gateway, you can use that URL in the IoTaWatt setup.

3

Thanks, so I had my friend post the two URLs below using a web browser (one is http: and the other is https:). Both worked - each time the browser returned “ok” and the data were received in my emoncms server. So this suggests to me that his network or gateway does not have trouble reaching my dataplicity link. I’ve got “http://subhuman-lark-xxxx.dataplicity.io” entered as the server URL in the IoTaWatt emoncms setup - does this seem right? (I’m happy to share the real values obscured by xxxx if that helps you troubleshoot). I can also get logs from his IoTaWatt if helpful. Thanks

http://subhuman-lark-xxxx.dataplicity.io/input/post.json?node=AlTest&apikey=xxxxxxxx&json=Power1:1023.7,Power2:2034.8

https://subhuman-lark-xxxx.dataplicity.io/input/post.json?node=AlTest&apikey=xxxxxxxxx&json=Power1:1123.7,Power2:2234.8

4

I see a ‘/‘ (slash) at the end of the URL in the message:

image
image.png1009×96 52.7 KB

I don’t see it in your configuration, but it may just be too large to show in the box. In any event, if you are specifying the URL ending with a slash, try removing it.

5

No, we did not include a slash after the .io of the URL (and there’s no :80 or white space at the end either). Seems the IoTaWatt is adding the :80/ seen in the log.
Is there a way to see the full message that the IoTaWatt is trying to post?

6

Sure wire-shark. (Not baby shark)

7

It’s true that the IoTaWatt is adding the port number and then what it sees as the uri, which would seem to be the slash. I looked at the parser, and if you didn’t put it there, I’m baffled how it got there.

Could you download the config.txt file and send it to me in a PM?

8

I think I know what’s happening. Your dataplicity is requiring https and is sending a 301 permanent redirect to the IoTaWatt specifying a new URL with port 443 (HTTPS). When you were issuing that request from the browser using http, the browser was receiving the redirect and resending the request to port 443. If you look at the URL in the ok response, it should be https whether you send the request using http or https.

You need to open up port 80 to Emoncms.

9

You are right - when I send the request as http, the ok response is returned as https. So I think I understand the problem. But I’m not clear on the solution. Can you explain a little more what “… open up port 80 to Emoncms” means?

Thank you so much. I want to state that I found the IoTaWatt very easy to install and get running - much much easier than the Brultech 1240 that I installed in my own house a couple of years ago. And my friend is have great fun analyzing his power use using Graph+ on the IoTaWatt

10

Still no expert on HTTP, but here goes.

You have a URL that maps to the IP of your dataplicity box. So when a client wants to connect to it for some service it goes to a DNS server and looks up that IP using the domain name. Now it can send a TCP/IP packet to your device using that IP. But what does the dataplicity box do when that packet when it comes in?

The device could be running any number of services. For instance, Unix based boxes usually run a SSH service for remote administration. Emoncms runs on a web server, usually Apache.

So when a request comes in, it also has a port number associated with it. Most popular services have staked out a port number that they use. SSH uses 22, HTTP web services are serviced on port 80, and secure HTTPS web services are usually serviced on port 443. As each service is started, they declare the port(s) that they want to listen on to the lower level TCP/IP stack.

Somewhere in the configuration of your dataplicity, there’s a web server configuration, as I said, probably Apache. I don’t know Apache, but somewhere in that configuration there’s a way to say direct it to reject port 80 requests and ask the client to permanently redirect requests to port 443. In effect saying that all communications must be secure.

That’s not a bad thing. It’s just that IoTaWatt doesn’t have the memory needed to handshake and use the encrypted protocols. There are solutions to that using proxies, but that just gets you in deeper. Better to just open up the dataplicity to HTTP traffic on port 80.

Maybe someone with more in depth knowledge of your equipment will weigh in with more specific advice.

UPDATE: I did an http request and the 301 redirect was issued from a server that identified itself as “awselb/2.0”

The https redirected request was serviced by Nginx, which must be the server that is hosting your Emoncms.

I knew nothing about dataplicity, but after a quick google it starts to make sense. Apparently dataplicity is a VPN like service hosted on AWS. You probably won’t be able to open up port 80 to aws. They’re pretty strict about that.

Your best bet is to open up a port on your WiFi router. That usually involves assigning a static local IP to the server, and setting port forwarding to forward an arbitrary unused port to port 80 on the server. Then you can address the server using your WAN IP address and the arbitrary port that you chose.

If your WAN IP is not fixed, you will need to use some kind of dynamic dns service to map the volatile IP.

Such is the life of running your own server.

11

Thanks very much for your troubleshooting and the explanation. I see a few other options, like my friend sets up an account at emoncms.org, or he opens up a port so he can access his IoTaWatt and use Graph+ remotely. Choices! I’ve got to sleep on it!

1 Like