There are a couple of users here that are interested in using your service. IoTaWatt supports direct uploading to influxDB, but the differences between influx and your product are restricting use of that support.
I don’t know yet if the native influx POST upload is going to work because we have not been able to get past the initial query to determine where to begin. IoTaWatt saves all of it’s measurements, and when beginning to upload to influx, checks the time of the last uploaded measurement. It then resumes uploads from that time so that the data is continuous even if there was a communication or server interruption. That query is currently a POST query. It’s possible to change to a GET, but there are a couple of other issues:
influxDB accepts an authorization header with the user and password. This works fine with influx. Corlysis does not seem to recognize the base64 encoded basic authorization header and seems to require that the password is sent plaintext in the very exposed GET URL.
Corlysis seems to require specific syntax in the key value field. Specifically, IoTaWatt always sends key values as single quoted strings. They do not appear to be acceptable to Corlysis.
These query issues need to be resolved in order to be able to move on to the measurement write POSTS, which will probably work (maybe problems with quoted key values).
Bottom line is that this is a hybrid protocol that isn’t very well documented, so there is a need to figure out what’s different. That users are successfully uploading with other software isn’t the issue. IoTaWatt users are successfully uploading to influx. Reconciling the differences is what’s needed and supporting features like POST query and authorization headers would go a long way to making influx support more compatible with your product.
The rookie restrictions on your new account have been removed.
we SUPPORT base64 encoded basic authorization header! We do not send password by plaintext!!! Our recommendation is to use HTTPS and “basic authorization” as I wrote: curl -G 'CORLYSIS_URL/query?db=emon' -u token:AABBCC --data-urlencode "q=select last(value) from Volt"
This works with influx, so if you could take a look and see if you can spot something wrong with the “q” parameter. Note: when it works with influx, it’s the payload of a POST transaction and not URL encoded. In order to append the query to the GET URL, it was necessary to convert spaces to %20. That seems to be working OK.
there is a mistake. The correct query is: curl -G 'https://corlysis.com:8086/query?db=DB_NAME&q=select%20last%28value%29%20from%20meas_test' -u token:AAAA
IoTaWatt cannot do HTTPS. That’s just not going to happen. So port 8087 seems to be the correct port for HTTP.
IoTaWatt doesn’t use curl. It builds the URL in c++.
These are the differences that I see between what I posted and the above
IoTaWatt also sends &epoch=s
The IoTaWatt uses capitalized keywords
The IoTaWatt is using a where clause to discriminate by key value
Curl urlencodes the parenthesis in LAST(value)
If you see anything else, please point it out.
It’s important to note that the IoTaWatt query does work fine with influx, but I’m open to someone pointing out where it nevertheless violates the influx query specification.
I’ll try urlencoding the parenthesis.
The where clause is an important part of how IoTaWatt maintains its upload context, so removing that is not on the table.
I have tried going lower case on the keywords with no success.
I don’t believe the epoch is an issue, but I will see what happens if I remove it. Potentially the returned time of the last measurement requested will be in milliseconds, and that would be a compatibility issue for bona-ride influxDB users.
The urlencoded parenthesis doesn’t seem to matter, and the epoch doesn’t seem to be a problem, nor the capitalized keywords.
Revisiting the documentation is always a good idea. I found this:
You may also use a POST request by sending the same parameters either as URL parameters or as part of the body with application/x-www-form-urlencoded .
Which is what IoTaWatt does successfully using the bona-fide influx interface.
While we seem to be zeroing in on the incompatibilities, I want to caution the participants that I would be in no hurry to change the query over to use GET, for two reasons:
The current POST works fine and exposes less data through the URL.
There are hundreds of satisfied users relying on the current support and using many varied installations of influxDB. A change like this requires a lot of testing and field trials to avoid disruption.
That said, simply adding support for POST in Corlysis should be relatively straightforward, and IMO a better solution.
I have not reverted back to post and tried without the where clause, so I’ll reserve final word until I try that. It could be that the issue is only the where clause.
UPDATE: Tried using POST without the where clause. Get the missing q message.
IoTaWatt does use asyncHTTPrequest to send data to influx, and it was easy to change to GET (although there was a sequencing problem with the authorization header). What in particular do you think needs to be changed here?
I understand. What you are looking at is the released code. In the tests that I did, I added the query to the URL. The problem seems to be the where clause. Without that, my expanded GET URL worked fine.
Not too clear on what kind of change would facilitate marrying the URL and Query in asyncHTTPrequest.
I looked over your PR for absolute URL, and will have to look further at both the change and the implementation. My world is getting more complicated with so many dependent users. I’m afraid to break something.
Not a part of corlysis