HI,
is there a way to have the file download as non blocking?
Thanks
First, understand that this is only the download of a new release blob from the IoTaWatt.com server. It only happens when you upgrade to a new release. Since introduction of async HTTP until now, it has been non-blocking. There were various random errors that reflected poor file integrity after download. The file is signed, so the errors were detected and recovery was initiated, but there were other issues…
The problem is that the file is written to SD in an asynchronous onData callback. The underlying SD code was not written to operate in this environment. So if the onData callback is iniated while another service is writing to the SD (dalalog, history, Emoncms, influxDB, messageLog) the results are unpredictable and potentially catastrophic. It took me awhile to figure this out, but it explains a whole lot of weirdness that was in the question mark file.
It takes about 5-10 seconds to download the release blob with all resources seized. Even ALPHA users will only see this every few weeks at most, so not a big deal.
IoTaWatt still uses the ESP8266WebServer as opposed to the AsyncWebServer by me-no-dev. So all other downloads are synchronous, although not necessarily blocked. The data is received and buffered by the web server, and the handler that writes to SD is not an asynchronous handler.
When I get a chance to further develop flow-control in asyncHTTPrequest, there is a way to go back to non-blocking, but there is no real downside to the status-quo.
It is when I download the iotalog or the histolog as backup.
I understand that it is not your first priority 
PS: With those I just made my first Python test to extract data and inject to mysql database and visualize with grafana…
OK, download and upload are all relative terms. When I talk about downloading the release blob, IoTaWatt is the client downloading from the IoTaWatt.com server. So IoTaWatt requests the file as a client and receives it as the response.
When you “download” a file from IoTaWatt, IoTaWatt is the server, and your browser (or application) is the client. That’s a whole different area of the firmware. As a client, IoTaWatt does everything asynchronously. As I said before, the webserver portion is synchronous - always has been.
There is an asyncWebServer, and I’ve used it for other purposes. The problem is that IoTaWatt spends up to 67% of it’s time sampling power. The firmware is structured to do everything else in 8-10ms windows between sampling. With a synchronous web server, I can control the scheduling of the request handling, and the sampling isn’t significantly affected.
The long term solution is using the ESP32, some other multi-core MCU, or another processor to do the sampling (STM32?) - but right now, with a single core, scheduling drives the architecture.
I have never tried to download the iotalog file. It’s potentially 2GB. The ESP8266 does not have MMC, so SDaccess is slow. It could take days. You would be better off reading the contents of the file with the API and adding to your backup file, or shutting down the IoTaWatt periodically and copying the file.
I have a project planned to provide another layer of protection for the log file as is, and I believe the bug fixed by this recent change to block during release download was responsible for most of the iotalog damage reported in the past. Iotalog updates every 5 seconds, so it has the highest probability of being affected by an asynchronous SD write.
1 Like
There you may use as example a STM32L0 with 16x12Bits ADC, STM32L0 with 24x12Bits ADC or an STM32F3 with 21x16Bits ADC (just indicative I did not look about alternate functions would reduce in case of SPI or I2C).
Then communicate the measured values via I2C or SPI.
But with this approach you would also have to integrate as example the serial protocol to download the firmware to the STM32…
But ok this is another story 
Or you may use some of the cirrus logic CS5484 that is doing all for you (temperature compensation…), but it may be to expensive… I used the 2 channel version once for a project…
In a far off land. To a hammer, everything is a nail.