Latest on ESP32

[overeasy]

Been working on the ESP32 a lot lately, and the latest prototype is coming together with progress on the firmware. First the visuals. Here is the main board coupled with an extension board that can be used to double the number of inputs:

This prototype used headers to allow the ESP32 DEVKIT-C to be plugged in. The production unit will directly solder the DEVkit to the board. This arrangement allows me to swap in different processors. It can currently run with either a WROOM-32 or WROVER ESP32.

Probably not easily noticed is that the SDcard is now a push-push holder instead of the push-pull used on the ESP8266 IoTaWatt. Thjis allows much easier insertion and removal, and also should solve a problem in manufacturing with the old holder.

This board has a separate 3 Volt power supply running off the 5V input to supply lower noise ADC reference and bias. It also affords the ability to change the ADC reference voltage by simply using a different LDO regulator. I’ll be trying out a 1.5V version with 10 Ohm burden in the near future.

The big improvement is the ability to connect an extension board using Flat Flexible Cable (FFC). This cable is not only physically flexible, but also introduces a lot of flexibility in packaging.

As you can see, with the long cable shown, Each of the boards can be packaged in a standard IoTaWatt enclosure and the cables run out and in through a slot in the base.

Another way to do it is to use a short cable and stack the boards:

With this approach, a riser could be fabricated to accomodate the extension within the existing enclosure. Either way, by including the inexpensive connector on the base unit, it can now be field upgraded. The connector is a ZIF (zero insertion force) design with a simple locking lever.

This extension board is the same PCB as the main board with fewer components and a few jumper SMT “resistors”. It has it’s own ADC reference power supply so it can be run at a different voltage from the main board. This board will be much cheaper to manufacture not only because it uses fewer parts, but because it doesn’t have thru-hole components.

A final version will be a unique PCB and will probably have an I/O expansion chip as well as some circuitry to allow using other sensors. Maybe 10K temperature and/or 4-20mA. There is one MCP3208 ADC port available and one ESP32 ADC port available.

I’ll post another entry to talk about the changes to the firmware and what I hope can be added going forward.

The thing is rock solid. Here’s a preview. Channel 15 is on the extension board.

[craigcurtin]

Wow - look at all the free heap space too !!

Great Progress Bob - wish we could clone you to get it out the door faster !

Craig

[overeasy]

@craigcurtin

Wow - look at all the free heap space too !!

I hope it’s enough to support HTTPS for both the web server and the client sessions. More of that memory will be needed for buffers as I port the remaining services. The ESP-WROVER has 4Mb of PSRAM that may be the key to SSL and a better datalog.

[Tony]

that looks amazing, such progress!
At this rate, I’ll be able to hook dad up with one next Christmas! The dream lives
You’re a gorram wizard, Bob. I wish I had half your smarts!

[craigcurtin]

Is HTTPS that big a deal for how this is used ?

If it is going to give issues (futher down the track with heap again) putting it in there as compared to adding other more useful features (like MQTT for instance ?)

Craig

[overeasy]

Commercial users are very interested in HTTPS. There are workarounds with proxies, but having it native will open up things like influDB 2. It remains to be seen how the PSRAM will contribute. If the certs can be stored and processed in PSRAM, the only heap requirement as I understand it is a larger buffer during the handshake.

Like HTTP, MQTT can also use TLS. With TLS you can get assurance that the broker you are connecting to is authentic and you can insure the data is not altered or viewed. While most folks might not care about that, I think the trend will be toward TLS MQTT as it becomes easier to accomplish with more capable IOT devices.

For users that want MQTT, I think it would be easy to translate IoTaWatt query output to MQTT using NodeRed.

[craigcurtin]

Yeah thats what i am doing with NR now and it is working well.

Have you seen the new iteration of the ESP32 that has come out recently ? Same form factor, but processors up to 240mhz and a few other enhancements that may be worth looking at

Craig

[overeasy]

I’m running the ESP32-WROVER-IE with 8Mb PSRAM at 240MHz. Very fast.

[promy]

Great to see you progress. Just wondering , is 14 extra inputs the maximum, or could one stack 2 or more units together? (by adding 2 ZIF connectors instead of one?)

[overeasy]

Short answer is that I don’t think it’s practical.

The issues would be:

• The cycle sample rate has only increased 50% from 40 to 60 (33.3 to 50 at 50Hz) yet the number of inputs is doubled. So the the frequency of samples is down to about every 500ms/600ms (60Hz/50Hz) where it was about 375ms/450ms with the current ESP8266. Note that a 14 channel ESP32 is 250ms in a 15 inputs system (60Hz).

• One of the resources that doesn’t scale well is GPIO for ADC chip select. Initially, I designed this with a decoder chip to output the 4 chip selects using three GPIO. Two pins are bank and ADC where the third is the actual chip select. There is only one remaining free output GPIO pin so it would require adding an 8 output decoder and using the additional pin to input a two bit bank designation. With that, the number of ADCs could be 8 subject to these other limitations.

• I don’t believe there is a FFC connector that can go in and back out. Moreover the 12 conductor that I am using would need to go to 14 or 16 to accommodate more CS lines. That said, there will be plenty of real-estate on the expansion boards to accommodate two connectors.

IMO the better approach would be to use the higher communications bandwidth to have a second unit communicate it’s data to a master with an expanded datalog. I’m pretty confident I will not be doing that, but that would be my approach.

[frogmore]

Is there any timeline for the ESP32?
Should I start saving my pennies, or perhaps I should be saving dollars

[overeasy]

I’m working on it pretty much daily, but although the core energy monitoring is done and working great, there is a lot of other functionality that has not been replicated and I need to fully explore the options for HTTP and HTTPS in this new environment.

With tooling and regulatory and safety testing it cost tens of thousands of dollars to bring the ESP8266 version to market and involved a lot of effort that I am frankly becoming less than enthusiastic about. I enjoy doing the hardware and software development and am currently thinking of just pursuing it as a hobby project.

I may make some form available as a kit with no certifications, but realistically a product as packaged and solid as the ESP8266 is unlikely anytime soon. Best case is that an entity comes forward to invest in the technology and handles the regulatory and business end of it.

[PimpMyPC]

I would be interested in a ESP32 kit if that’s how it would have to come to market. Even if its a bare PCB and a bag of components that need to be soldered on.

[overeasy]

It’s actually a lot easier to have a limited run of PCBs at the fab plant than to source and bag up components. Besides, I have a template and reflow oven and it’s still a PIA to assemble by hand.

[LosinIt]

Put me down for one in whatever way you make it available.

[Tony]

Just wondering, is there anything we as a community can do that makes any of the tooling, safety or regulatory stuff less of a pain? Thinking if there’s people here who have the sorts of skills in these things that are interested in contributing it would be amazing if there was a way to share the burden around to help bring this to market.
Looking through the community threads there are people left right and centre getting so much out of these both from a energy savings point of view but also learning about their own homes so much, I’m sure there would be some that would be happy to help!

[overeasy]

Thanks @Tony I’ll keep that in mind. That phase is still on the horizon.

[overeasy]

PROGRESS REPORT:

Have managed to get HTTPS working for the client activities like posting data to cloud services PVoutput, influxDB and Emoncms. Also will work for the OTA update system when I port that.

With the ESP8266, I had built a client facility called asyncHTTPrequest because the ESP8266 could not afford to block while posting data. That has worked very well. I published the class as open-source and has had modest interest with quite adapters. With the ESP32 I had made it threadsafe and use the ESP32 vesion of asyncTCP, but it has the same limitation of no HTTPS because not supported in asyncTCP.

I have reworked that class to use the ESD-IDF native esp-http-client facility which supports HTTPS. Once converted, I simply changed the method in the URL to https and voila, it worked fine.

Heap is an issue. I’ve currently got about 140K available, but doing TLS requires about 40K while a TLS session is connected. So that could add up fast. There are a few ways to manage that:

I’m using an ESP-WROVER with PSRAM available. Will eventually work to move whatever heap usage I can to that address space. There are some IDF options to cause lwip and wifi to use ity more, and it may be possible to force the TLS buffer there, it’s too early to tell.

The 40K seems to be released immediately upon disconnect, so I can create a resource semaphore for open TLS sessions and limit the number active at any one time.

I’m still studying my options for making the web-server secure. There is a similar ESP-IDF facility for that, but like the client, it is a series of “C” calls, and not implemented as a C++ class. I’m currently using me-no-dev’s asyncHTTPserver, and it’s fast and easy, but it doesn’t do HTTPS. There is another one available that looks good on Github. I’m thinking about how much of the functionality I really need and whether it might be better to just use the native ESP-IDF web server with a few helper classes for the tricky stuff. If I roll my own, I may be able to control the total number of TLS client and server sessions.

[matt1]

@overeasy Just out of curiosity, beyond prototyping, what advantage does using an off-the-shelf ESP32 dev kit have over just soldering the ESP directly if you’re already doing SMD parts? The number of components required to run the ESP32 is very small. Here’s a schematic I did as an example: ESPixelClock/hardware at master · mattncsu/ESPixelClock · GitHub. You’d save space, cost, and improve supply chain should it become a mass-market hit since you wouldn’t have to rely on the continued existence of the dev boards.

I’d stumbled across the Emporia Vue 2 on Amazon which renewed my interest in the power monitoring segment. Had to rule it out since it doesn’t measure real power. I’m pleased to come back and see all the progress on your ESP32 version!