Verify the integrity of a flash/SD card on a Mac

Last minute camera shopping

The week before going on a holiday, I realised my old Canon Ixus 40 camera would no longer live up to my expectations. So I quickly checked dpreview.com for the latest and greatest compact camera’s and decided I wanted the Canon PowerShot SX240 HS. 15 Minutes later I found myself in the Media Markt with the camera in my hand (unfortunately only the silver colour was available, I had hoped to buy the black one).

I need a bigger memory card

I then realised that my existing 512MB and 1GB SD cards would no longer be large enough for the 12Mpixel photos of this new camera. A few meters down the corridor was a huge wall covered in memory cards. I decided to buy a Samsung 8GB SDHC card, which would be able to hold up to 1500 photos or an hour of HD video. When I returned home, I formatted the card in-camera, and played around with the photo and video functions. Three days later my holiday would start and I would have a good (enough) camera to capture some fine moments!

Corrupted photos

Fast-forward about two weeks, half-way into my holiday. I’ve happily been taking photos and videos with my new camera. While I’m reviewing the last five photos, the camera suddenly displays a “broken picture” image on the screen. I take another photo, review it and it’s immediately “broken” again. I’m approximately on two-thirds of the card’s capacity and it is corrupting my photos. I do what I’ve learned from experience with corrupt memory cards: turn off the camera immediately, remove the SD card, and to be sure, “lock” the card (making it read-only) using the physical switch on the SD card. At this point, I just hope for the best that only those last two photos were corrupted.

Back at home a week-and-a-half later, it turns out I’m extremely lucky! Indeed only those last two pictures were corrupted!

Preventive detection of bad memory cards

How could I have detected on beforehand the memory card was corrupt? And how would I be able to determine the corrupt block(s) now? I just started google-ing for “verify integrity sd card”, expecting to find hundreds of results. Not so much really, in fact, I couldn’t find any fully automated test programs at all!

Windows based “h2testw”

Until I stumbled upon “h2testw“, written by Harald Bögeholz. It’s Windows only, but fortunately I have a VMWare Fusion hosted Windows 7 virtual machine. This software will write to all available free blocks on a volume and will then verify (read back) whether the files are still in the right shape. It does this byte-by-byte, so not a single bit will go untested (well, actually, the very last Megabyte might go untested, see the Readme).

Testing SD card using h2testw on Windows on VMWare Fusion

In VMWare Fusion, I connected my MacBook Pro’s “Apple Internal Memory Card Reader” to the virtual machine (this will temporarily disconnect the device from Mac OS X, so eject any card you have in the reader before attaching it to the virtual machine). Next, I inserted the suspicious memory card. Since h2testw will only test empty (free) space on a selected volume, I “quick” formatted the sd card before starting the verification. To my surprise, the software found no errors! I formatted again, reran the verification and again, no errors! Was my camera faulty?

I read a bit more about SD cards and it seems they can re-assign bad blocks (just like a regular hard drive), but only after you’ve tried to write to them. Had I fully written to the SD card immediately after buying it, the bad blocks would probably have been re-assigned.

Testing SD card using F3 on Mac OS X

So, I’d found a solution for Windows, but what about the Mac? Google-ing for “h2testw mac” pointed me (eventually) to the software “F3” (either meaning “Fight Flash Fraud” or “Fight Fake Flash” according to its author Michel Machado). A page with a manual for F3 can be found here, while the source of F3 is on Github. F3 is an open-source implementation of the same algorithm h2testw employs to determine the integrity of a memory card (or any volume you mount on your Mac for that matter).

Compiling the source of F3

[Edit 30 March 2018]
Save yourself the trouble of downloading and compiling source code, use brew downloaded from https://brew.sh, after which you can simply brew install f3. After you’ve done that, f3read and f3write will be symlinked in /usr/local/bin (and probably in your PATH) and readily available from the Terminal.
[/Edit]

I first downloaded the source of the latest version of F3 from Github as a Zip file. Next, I double-clicked the Zip file in the Finder to extract the files. A folder “AltraMayor-f3-c24c098” was created.
I opened Terminal and entered:

cd ~/Downloads/AltraMayor-f3-c24c098

The source code still needs to be compiled into a program. I think you’ll need XCode (since you need gcc to compile), but I’m not sure since I’ve had XCode installed since I don’t know when… Following the instructions on Github, the only thing I need to enter now is:

make mac

Zero-ing out the memory card

Before we run the tests, let’s completely erase the memory card. Open “Disk Utility” on your Mac with the SD card inserted. Select the disk (not the underlying volume), mine is called “8,07 APPLE SD Card Reader Media”. Go to the second tab “Erase”. Make sure the “Format” is “MS-DOS (FAT)”, the name doesn’t really matter (I left it at “UNTITLED”). Click on the “Security Options…” button, select “Zero Out Data” and click on “OK”. It now says “Erase option: Zero Out Data selected.” above the “Security Options…” and “Erase…” buttons. Next, click on the “Erase…” button and click “Erase” to confirm. Using Activity Monitor, I could see Disk Utility was writing zeroes to my memory card at 15MB/s. It took about 8 minutes to totally fill my 8GB memory card.
This is a good test to start with. If zero-ing the card yields errors, it certainly is corrupt.

Testing SD card using F3 on Mac OS X

There are a couple of new files in the folder now, most notably f3write and f3read, which are the actual programs we’ll need to continue testing the SD card.
First, let’s find out what the actual “volume” of the SD card is. In Terminal, enter:

cd /Volumes/
ls

Those are two separate commands, each line followed by hitting “enter/return”! This is what I got as a result:

$ cd /Volumes/
$ ls
Macintosh HD	UNTITLED

So I’ve got two mounted volumes “Macintosh HD” and “UNTITLED”. My best guess is I’m interested in the latter 🙂
Let’s go back to the F3 program folder:

cd ~/Downloads/AltraMayor-f3-c24c098/

And execute the f3write program first:

./f3write /Volumes/UNTITLED/

The F3 write program now first starts to write to all free/empty space and will report its progress. This is also a good moment to check the write speed of the card. Mine delivered a minimum rate of about 10MB/s. According to Wikipedia, this is “Class 10”. That’s also what is printed on my SD card, so lucky me!

Next, we’ll have to check whether the data is OK. To this avail, we need to run the “f3read” program. In Terminal, I entered:

./f3read /Volumes/UNTITLED/

You can monitor its progress for each xxxx.fff file the program finds (these .fff files were written by f3write). The manual page linked above explains the different columns (ok/corrupted/changed/overwritten).

Finishing up

So, that’s it! I finally have a program for Mac OS X to check the integrity of a SD card! From now on, the first thing I will do after buying a memory card is validating whether it isn’t corrupt, if only to re-assign potential bad blocks…