that looks amazing, such progress!
At this rate, I’ll be able to hook dad up with one next Christmas! The dream lives 
You’re a gorram wizard, Bob. I wish I had half your smarts!
Is HTTPS that big a deal for how this is used ?
If it is going to give issues (futher down the track with heap again) putting it in there as compared to adding other more useful features (like MQTT for instance ?)
Craig
Commercial users are very interested in HTTPS. There are workarounds with proxies, but having it native will open up things like influDB 2. It remains to be seen how the PSRAM will contribute. If the certs can be stored and processed in PSRAM, the only heap requirement as I understand it is a larger buffer during the handshake.
Like HTTP, MQTT can also use TLS. With TLS you can get assurance that the broker you are connecting to is authentic and you can insure the data is not altered or viewed. While most folks might not care about that, I think the trend will be toward TLS MQTT as it becomes easier to accomplish with more capable IOT devices.
For users that want MQTT, I think it would be easy to translate IoTaWatt query output to MQTT using NodeRed.
Yeah thats what i am doing with NR now and it is working well.
Have you seen the new iteration of the ESP32 that has come out recently ? Same form factor, but processors up to 240mhz and a few other enhancements that may be worth looking at
Craig
1 Like
I’m running the ESP32-WROVER-IE with 8Mb PSRAM at 240MHz. Very fast.
2 Likes
Great to see you progress. Just wondering , is 14 extra inputs the maximum, or could one stack 2 or more units together? (by adding 2 ZIF connectors instead of one?)
Short answer is that I don’t think it’s practical.
The issues would be:
-
The cycle sample rate has only increased 50% from 40 to 60 (33.3 to 50 at 50Hz) yet the number of inputs is doubled. So the the frequency of samples is down to about every 500ms/600ms (60Hz/50Hz) where it was about 375ms/450ms with the current ESP8266. Note that a 14 channel ESP32 is 250ms in a 15 inputs system (60Hz).
-
One of the resources that doesn’t scale well is GPIO for ADC chip select. Initially, I designed this with a decoder chip to output the 4 chip selects using three GPIO. Two pins are bank and ADC where the third is the actual chip select. There is only one remaining free output GPIO pin so it would require adding an 8 output decoder and using the additional pin to input a two bit bank designation. With that, the number of ADCs could be 8 subject to these other limitations.
-
I don’t believe there is a FFC connector that can go in and back out. Moreover the 12 conductor that I am using would need to go to 14 or 16 to accommodate more CS lines. That said, there will be plenty of real-estate on the expansion boards to accommodate two connectors.
IMO the better approach would be to use the higher communications bandwidth to have a second unit communicate it’s data to a master with an expanded datalog. I’m pretty confident I will not be doing that, but that would be my approach.
Is there any timeline for the ESP32?
Should I start saving my pennies, or perhaps I should be saving dollars 
I’m working on it pretty much daily, but although the core energy monitoring is done and working great, there is a lot of other functionality that has not been replicated and I need to fully explore the options for HTTP and HTTPS in this new environment.
With tooling and regulatory and safety testing it cost tens of thousands of dollars to bring the ESP8266 version to market and involved a lot of effort that I am frankly becoming less than enthusiastic about. I enjoy doing the hardware and software development and am currently thinking of just pursuing it as a hobby project.
I may make some form available as a kit with no certifications, but realistically a product as packaged and solid as the ESP8266 is unlikely anytime soon. Best case is that an entity comes forward to invest in the technology and handles the regulatory and business end of it.
2 Likes
I would be interested in a ESP32 kit if that’s how it would have to come to market. Even if its a bare PCB and a bag of components that need to be soldered on.
It’s actually a lot easier to have a limited run of PCBs at the fab plant than to source and bag up components. Besides, I have a template and reflow oven and it’s still a PIA to assemble by hand.
Put me down for one in whatever way you make it available.
2 Likes
Just wondering, is there anything we as a community can do that makes any of the tooling, safety or regulatory stuff less of a pain? Thinking if there’s people here who have the sorts of skills in these things that are interested in contributing it would be amazing if there was a way to share the burden around to help bring this to market.
Looking through the community threads there are people left right and centre getting so much out of these both from a energy savings point of view but also learning about their own homes so much, I’m sure there would be some that would be happy to help!
3 Likes
PROGRESS REPORT:
Have managed to get HTTPS working for the client activities like posting data to cloud services PVoutput, influxDB and Emoncms. Also will work for the OTA update system when I port that.
With the ESP8266, I had built a client facility called asyncHTTPrequest because the ESP8266 could not afford to block while posting data. That has worked very well. I published the class as open-source and has had modest interest with quite adapters. With the ESP32 I had made it threadsafe and use the ESP32 vesion of asyncTCP, but it has the same limitation of no HTTPS because not supported in asyncTCP.
I have reworked that class to use the ESD-IDF native esp-http-client facility which supports HTTPS. Once converted, I simply changed the method in the URL to https and voila, it worked fine.
Heap is an issue. I’ve currently got about 140K available, but doing TLS requires about 40K while a TLS session is connected. So that could add up fast. There are a few ways to manage that:
I’m using an ESP-WROVER with PSRAM available. Will eventually work to move whatever heap usage I can to that address space. There are some IDF options to cause lwip and wifi to use ity more, and it may be possible to force the TLS buffer there, it’s too early to tell.
The 40K seems to be released immediately upon disconnect, so I can create a resource semaphore for open TLS sessions and limit the number active at any one time.
I’m still studying my options for making the web-server secure. There is a similar ESP-IDF facility for that, but like the client, it is a series of “C” calls, and not implemented as a C++ class. I’m currently using me-no-dev’s asyncHTTPserver, and it’s fast and easy, but it doesn’t do HTTPS. There is another one available that looks good on Github. I’m thinking about how much of the functionality I really need and whether it might be better to just use the native ESP-IDF web server with a few helper classes for the tricky stuff. If I roll my own, I may be able to control the total number of TLS client and server sessions.
3 Likes
@overeasy Just out of curiosity, beyond prototyping, what advantage does using an off-the-shelf ESP32 dev kit have over just soldering the ESP directly if you’re already doing SMD parts? The number of components required to run the ESP32 is very small. Here’s a schematic I did as an example: ESPixelClock/hardware at master · mattncsu/ESPixelClock · GitHub. You’d save space, cost, and improve supply chain should it become a mass-market hit since you wouldn’t have to rely on the continued existence of the dev boards.
I’d stumbled across the Emporia Vue 2 on Amazon which renewed my interest in the power monitoring segment. Had to rule it out since it doesn’t measure real power. I’m pleased to come back and see all the progress on your ESP32 version!
If I were manufacturing this in China, that might be a win, but the components are too expensive here. The CP2102 is very expensive here, who knows how the Chinese do it. If you look close, a lot of them don’t have any markings. The micro USB connector is expensive in my quantities and is an expensive part to assemble because of the thru-hole tabs needed for strength. When you add up the cost of the parts it’s not a big savings.
With the nodeMCU I get a tested functioning unit. There is also an issue with the antenna being mounted directly to the main PCB. The recommendation is to have it hang off the PCB and I don’t have any space to do that. The nodeMCU raises it up off the main PCB and it works pretty well.
Then there is the enclosure. Moving the location of the usb plug may seem trivial, but it’s a big deal. That part of the enclosure mold is an interchangeable cam assembly, and would require a whole new assembly. The ESP32 nodeMCU itself has necessitated moving the slot and the AC plug, and I’ve already started the process of having a new cam assembly built.
You are right. If I were anticipating selling a higher volume, the economies of scale would prevail, but at this point I don’t see a clear advantage. I haven’t tried to buy the ESP32 devkits in quantity yet, but I do have the ESP8266 devkits custom made with ESP12S modules for FCC compliance. They are cheaper than you might think and much better quality than the usual fare.
For my boards, I don’t bother with having USB on-board. With two transistors and two resistors, I can have an auto-reset circuit that’ll let me easily program my board using one of these: https://www.amazon.com/gp/product/B08246HWJ4. That adapter can supply 3.3V and 5V. I use 2x3 header holes on my PCB to break out 3.3V, GND, RX, TX, RTS, and CTS. For prototype units, I’ll solder in a male or female header but for bulk programming boards, I just push one of these onto the holes and hit go: https://www.amazon.com/pin-AVR-ICSP-Pogo-Adapter/dp/B075Q25BK3. It programs exactly the same in Arduino IDE as a dev board with onboard USB.
Once my code progresses a little, I usually add an OTA update feature to avoid having to open the case after initial programming.
For the PCB antenna, I use a keepout area in Eagle to prevent any copper under the antenna area and I haven’t noticed any issues. Modules are available from digikey for under $3 and some allow for external antennas (< $4) if signal range is an issue.
I used OshPNP for USA fab and they were great to work with and very supportive of small run just-above-hobby-level jobs.
Sounds like a plan. As an open software/hardware project, I think the homebrew folks appreciate the convenience of the nodeMCU. The USB also serves as a field debug as the IoTaWatt pushes out additional diagnostic information via Serial. With USB anybody can connect and use a serial monitor like Putty and a standard cable.
The field upgrade in IoTaWatt is custom and uses a signed release blob for security, but if for some reason a unit gets bricked, it can be reflashed without opening the case using a common USB cable.
Good to know about the success with a keepout on the PCB. Maybe someday.
1 Like