Finding a robust solution for general problems related to SD cards

1

For the past three years, I’ve been building several units based on the open-source version of IoTaWatt and installing them in various locations. Fortunately, the devices generally work well, but most of the issues we’ve encountered are related to the SD cards. In fact, about 90% of the problems with these devices have been caused by SD card failures—not at the beginning, but after several weeks or even months of operation.

For example, in one project where five devices were installed, one of them started to hang or crash. Replacing the SD card resolved the issue. Interestingly, we were using the same brand and model of SD card in all units, yet only one of them had this issue.

It seems that some SD cards—regardless of brand, advertised speed, or class—have underlying reliability issues that only surface after extended use.

So I have a few questions:

  1. Has anyone with IoTaWatt experience encountered similar SD card issues? If so, I’d appreciate hearing your experiences or suggestions.
  2. Is there a recommended SD card brand, type, or read/write speed that works best with IoTaWatt? Are there any specs that should be avoided?

Given that most of the problems we’re facing are SD card-related, I believe we need a more robust solution to eliminate or minimize this issue altogether.

For example:

  • Is there any software tool that can be used to test SD card health, integrity or endurance before deployment?
  • Are there any specific brands that are known to be more reliable, even beyond those with advertised 80–100 MB/s read and 20 MB/s write speeds?
  • Or maybe this issue isn’t related to speed at all, and is more about endurance or manufacturing quality?

For more info. I used some SD brands like sumsung, vicco, toshiba, trancend, …
about 2 out of 10 have probelms. somethimes they cause the device to crash. somethimes the device hangs or etc.

Any guidance would be greatly appreciated!

2

Short answer is that 16GB Sandisk Ultra speed class 10 have proven to be ROCK SOLID over several years of deployment.

I’m not saying there are no other equal quality SDcards out there, but this brand/model has survived the test of time and pretty much does not fail in a production IoTaWatt with recent firmware.

In particular, you might notice that there are few problems posted in this forum these days with many thousands of IoTaWatt running worldwide. That said, when there are problems, it is always either WiFi or SDcard.

WiFi is a separate issue and is almost always environmental or a problem with the router/AP. Some of the field units have been running for five or six years on the original SDcard. SDcard issues can be caused by a number of factors not all related to the SDcard:

Power supply. This is a broad category in that it can be attributable to using a low-quality USB charger rather than a low ripple power supply with steady voltage and adequate current.

Poor power from the nodeMCU. The public version of IoTaWatt hardware simply specifies using an ESP12S nodeMCU. These are hard to find, and I’ve been getting them custom made in China for years. That said, ESP8266 nodeMCUs are a dime a dozen from any number of sources but are not ESP12S (FCC part B certified) and will usually work ok but can sport a variety of different LDO voltage regulators, diodes and related tantalum capacitors. Let’s just say that some are better than others.

The SDcards themselves have a number of variables:

Format is a big issue. While most SDcards come reliably formatted, I’ve had some that are factory supplied with an obsolete format. While they might work in a lot of situations, they are not reliably supported by the SD drivers that are linked with the IoTaWatt firmware. I had a batch of cards branded by a US supplier and touted to be equivalent to Sandisk that turned out to be cheap China units that started failing after a few weeks. The supplier replaced them with Sandisk industrial, but the damage was done. Although only about 400 made it to the field, the majority of SDcard issues reported for several years were related to those cards.

Another card issue is wear-leveling. SDcards can only reliably write a sector so any times before the sector becomes unreliable. I believe better cards can tolerate more writes, but more importantly each manufacturer has their own proprietary wear leveling algorithms built into the card and logical sectors are mapped to physical sectors with high write sectors remapped to fresh physical sectors. There are three considerations here:

  • Does the card even have wear leveling capability?

  • Is the wear leveling effective?

  • Although IoTaWatt needs less than 4GB of SD storage, larger cards provide more physical sectors for remapping and thus extend life. (max supported is 32GB).

In the early days, there were also firmware related problems with the datalogs. Some of these were actual bugs in the modules that manage the datalog format, some to inconsistent state after power failure and some related to excessive writes. All of those have been identified and remedied for several years now and are a non-issue for up-to-date firmware.

Fortunately, IoTaWatt has an auto-update feature that keeps the vast majority of units running the latest major release of firmware automatically. That said, there are still a few stalwarts out there who refuse to allow automatic update (configurable) and continue to be exposed to these potential problems.

For those with older (>2 years) units, consider changing your SDcard to a 16GB Sandisk Ultra 10 and you will most likely never have SDcard problems.

4 Likes
3

A few years ago, I replaced my two microSD cards with Samsung Pro Endurance. I had not had any problems with the microSD cards that had originally come with my pair of IoTaWatt devices, but I thought it would be a good preventative maintenance activity. I went with 32GB cards as that should provide a ton of flash memory cells to allow the wear leveling algorithm to spread writes across.

https://www.amazon.com/dp/B09WB35BXS

I have not had any issues whatsoever. YMMV, of course!

I also use these same SD cards in Raspberry Pi boards and Wyze Cams. They’ve been pretty darn solid.

1 Like
4

Hi.
Thank you for your prompt response.

5

I bought different brands with different capacities and tested them with softwares like CrystalDiskMark.
Finally I ended up with a similar result.
32GB Sumsung Endurance is my current choice. I also, tested Sandisk and it was good too.