Verify the integrity of a flash/SD card on a Mac

Last minute camera shopping

The week before going on a holiday, I realised my old Canon Ixus 40 camera would no longer live up to my expectations. So I quickly checked dpreview.com for the latest and greatest compact camera’s and decided I wanted the Canon PowerShot SX240 HS. 15 Minutes later I found myself in the Media Markt with the camera in my hand (unfortunately only the silver colour was available, I had hoped to buy the black one).

I need a bigger memory card

I then realised that my existing 512MB and 1GB SD cards would no longer be large enough for the 12Mpixel photos of this new camera. A few meters down the corridor was a huge wall covered in memory cards. I decided to buy a Samsung 8GB SDHC card, which would be able to hold up to 1500 photos or an hour of HD video. When I returned home, I formatted the card in-camera, and played around with the photo and video functions. Three days later my holiday would start and I would have a good (enough) camera to capture some fine moments!

Corrupted photos

Fast-forward about two weeks, half-way into my holiday. I’ve happily been taking photos and videos with my new camera. While I’m reviewing the last five photos, the camera suddenly displays a “broken picture” image on the screen. I take another photo, review it and it’s immediately “broken” again. I’m approximately on two-thirds of the card’s capacity and it is corrupting my photos. I do what I’ve learned from experience with corrupt memory cards: turn off the camera immediately, remove the SD card, and to be sure, “lock” the card (making it read-only) using the physical switch on the SD card. At this point, I just hope for the best that only those last two photos were corrupted.

Back at home a week-and-a-half later, it turns out I’m extremely lucky! Indeed only those last two pictures were corrupted!

Preventive detection of bad memory cards

How could I have detected on beforehand the memory card was corrupt? And how would I be able to determine the corrupt block(s) now? I just started google-ing for “verify integrity sd card”, expecting to find hundreds of results. Not so much really, in fact, I couldn’t find any fully automated test programs at all!

Windows based “h2testw”

Until I stumbled upon “h2testw“, written by Harald Bögeholz. It’s Windows only, but fortunately I have a VMWare Fusion hosted Windows 7 virtual machine. This software will write to all available free blocks on a volume and will then verify (read back) whether the files are still in the right shape. It does this byte-by-byte, so not a single bit will go untested (well, actually, the very last Megabyte might go untested, see the Readme).

Testing SD card using h2testw on Windows on VMWare Fusion

In VMWare Fusion, I connected my MacBook Pro’s “Apple Internal Memory Card Reader” to the virtual machine (this will temporarily disconnect the device from Mac OS X, so eject any card you have in the reader before attaching it to the virtual machine). Next, I inserted the suspicious memory card. Since h2testw will only test empty (free) space on a selected volume, I “quick” formatted the sd card before starting the verification. To my surprise, the software found no errors! I formatted again, reran the verification and again, no errors! Was my camera faulty?

I read a bit more about SD cards and it seems they can re-assign bad blocks (just like a regular hard drive), but only after you’ve tried to write to them. Had I fully written to the SD card immediately after buying it, the bad blocks would probably have been re-assigned.

Testing SD card using F3 on Mac OS X

So, I’d found a solution for Windows, but what about the Mac? Google-ing for “h2testw mac” pointed me (eventually) to the software “F3” (either meaning “Fight Flash Fraud” or “Fight Fake Flash” according to its author Michel Machado). A page with a manual for F3 can be found here, while the source of F3 is on Github. F3 is an open-source implementation of the same algorithm h2testw employs to determine the integrity of a memory card (or any volume you mount on your Mac for that matter).

Compiling the source of F3

[Edit 30 March 2018]
Save yourself the trouble of downloading and compiling source code, use brew downloaded from https://brew.sh, after which you can simply brew install f3. After you’ve done that, f3read and f3write will be symlinked in /usr/local/bin (and probably in your PATH) and readily available from the Terminal.
[/Edit]

I first downloaded the source of the latest version of F3 from Github as a Zip file. Next, I double-clicked the Zip file in the Finder to extract the files. A folder “AltraMayor-f3-c24c098” was created.
I opened Terminal and entered:

cd ~/Downloads/AltraMayor-f3-c24c098

The source code still needs to be compiled into a program. I think you’ll need XCode (since you need gcc to compile), but I’m not sure since I’ve had XCode installed since I don’t know when… Following the instructions on Github, the only thing I need to enter now is:

make mac

Zero-ing out the memory card

Before we run the tests, let’s completely erase the memory card. Open “Disk Utility” on your Mac with the SD card inserted. Select the disk (not the underlying volume), mine is called “8,07 APPLE SD Card Reader Media”. Go to the second tab “Erase”. Make sure the “Format” is “MS-DOS (FAT)”, the name doesn’t really matter (I left it at “UNTITLED”). Click on the “Security Options…” button, select “Zero Out Data” and click on “OK”. It now says “Erase option: Zero Out Data selected.” above the “Security Options…” and “Erase…” buttons. Next, click on the “Erase…” button and click “Erase” to confirm. Using Activity Monitor, I could see Disk Utility was writing zeroes to my memory card at 15MB/s. It took about 8 minutes to totally fill my 8GB memory card.
This is a good test to start with. If zero-ing the card yields errors, it certainly is corrupt.

Testing SD card using F3 on Mac OS X

There are a couple of new files in the folder now, most notably f3write and f3read, which are the actual programs we’ll need to continue testing the SD card.
First, let’s find out what the actual “volume” of the SD card is. In Terminal, enter:

cd /Volumes/
ls

Those are two separate commands, each line followed by hitting “enter/return”! This is what I got as a result:

$ cd /Volumes/
$ ls
Macintosh HD	UNTITLED

So I’ve got two mounted volumes “Macintosh HD” and “UNTITLED”. My best guess is I’m interested in the latter 🙂
Let’s go back to the F3 program folder:

cd ~/Downloads/AltraMayor-f3-c24c098/

And execute the f3write program first:

./f3write /Volumes/UNTITLED/

The F3 write program now first starts to write to all free/empty space and will report its progress. This is also a good moment to check the write speed of the card. Mine delivered a minimum rate of about 10MB/s. According to Wikipedia, this is “Class 10”. That’s also what is printed on my SD card, so lucky me!

Next, we’ll have to check whether the data is OK. To this avail, we need to run the “f3read” program. In Terminal, I entered:

./f3read /Volumes/UNTITLED/

You can monitor its progress for each xxxx.fff file the program finds (these .fff files were written by f3write). The manual page linked above explains the different columns (ok/corrupted/changed/overwritten).

Finishing up

So, that’s it! I finally have a program for Mac OS X to check the integrity of a SD card! From now on, the first thing I will do after buying a memory card is validating whether it isn’t corrupt, if only to re-assign potential bad blocks…

61 thoughts on “Verify the integrity of a flash/SD card on a Mac”

  1. I was able to change directories but I couldn't compile the app and it wouldn't run either. I get the following

    I checked and I also have X11 installed

    Ideas

  2. Well I'll try again

    robert-c-fishers-macbook-pro-7:f3v2_1 rcfisher$ make mac
    -bash: make: command not found

  3. X11 (Unix windowing app stuff) is not the same as Xcode (Apple's compiler and development environment), although they're both optional installations. Xcode is included with the install discs for older releases of OSX, and can be downloaded from https://developer.apple.com/xcode/ for newer ones.

    (Note: the headline version on the Apple site right now requires OSX 10.8. Older OS releases will require older Xcode versions.)

    F3 is now covered by the Homebrew project ( mxcl.github.com/homebrew/ ), which will make things a lot easier. You'll still need to install Xcode as well as Homebrew itself, but then you just have to say "brew install f3" to have it download and build the programs automagically.

  4. It turns out that the "gcc" compiler required is no longer included in the Xcode install and also it's a huge download. To avoid that, you can go to:

    https://developer.apple.com/downloads

    You'll need to establish a free developer account. (This can be attached to your exisiting Apple ID, I believe).

    Then in the downloads searchbox, enter "command line tools." Select the version of the command line tools for your Mac OS version (either Lion or Mountain Lion), download it, and then install the package. Then you will be able to execute the "make mac" command.

  5. Pingback: GEARTALK: THE STORY OF THE NOT SO FAKE SD-CARD | ISMONO Blog

  6. Does f3 hold any advantage over the standard *nix "badblocks" , which can be found in "e2fsprogs" on Mac Homebrew for example? That tool can perform read-write checks as well.

  10. Thanks, Jan van Haarst. I have installed the latest sdspeed 1.1 beta, yet it does not see the broken USB pendrive.

    To cut a long story short, the pendrive is
    LaCie XtremKey USB 3.0
    http://www.lacie.com/us/products/product.htm?id=10612

    It broke when automatically unmounted while copying a VMware Fusion disk image.

    Since I cannot repair it (Disk Utility, DiskWarrior, TechTool Pro, DiskTools Pro & SpeedTools Utilities Pro, and Drive Genius on Mac with OS X 10.9.1 Mavericks), I just want to format it, yet I cannot with any tool tested so far.

    Any idea how to repair or format it?
    Thanks.

  11. Thank you for the detailed information. My PNY StoreEDGE seems to be built pretty bad. Got my Parallels File corrupted. I ordered a second one, hopefully this one will work, we´ll see.

    Is it really necessary to be Fat? Why would this not work with ExFat or OSX Journaled?

  12. Hello there,

    I am testing my 64GB microSDXC card with this program. After filling my card with files for about three days, this is what happened next:

    Removing old file 1.h2w ...
    Removing old file 2.h2w ...
    Removing old file 3.h2w ...
    Removing old file 4.h2w ...
    Removing old file 5.h2w ...
    Removing old file 6.h2w ...
    Removing old file 7.h2w ...
    Removing old file 8.h2w ...
    Removing old file 9.h2w ...
    Removing old file 10.h2w ...
    Removing old file 11.h2w ...
    Removing old file 12.h2w ...
    Removing old file 13.h2w ...
    Removing old file 14.h2w ...
    Removing old file 15.h2w ...
    Removing old file 16.h2w ...
    Removing old file 17.h2w ...
    Removing old file 18.h2w ...
    Removing old file 19.h2w ...
    Removing old file 20.h2w ...
    Removing old file 21.h2w ...
    Removing old file 22.h2w ...
    Removing old file 23.h2w ...
    Removing old file 24.h2w ...
    Removing old file 25.h2w ...
    Removing old file 26.h2w ...
    Removing old file 27.h2w ...
    Removing old file 28.h2w ...
    Removing old file 29.h2w ...
    Removing old file 30.h2w ...
    Removing old file 31.h2w ...
    Removing old file 32.h2w ...
    Removing old file 33.h2w ...
    Removing old file 34.h2w ...
    Removing old file 35.h2w ...
    Removing old file 36.h2w ...
    Removing old file 37.h2w ...
    Removing old file 38.h2w ...
    Removing old file 39.h2w ...
    Removing old file 40.h2w ...
    Removing old file 41.h2w ...
    Removing old file 42.h2w ...
    Removing old file 43.h2w ...
    Removing old file 44.h2w ...
    Removing old file 45.h2w ...
    Removing old file 46.h2w ...
    Removing old file 47.h2w ...
    Removing old file 48.h2w ...
    Removing old file 49.h2w ...
    Removing old file 50.h2w ...
    Removing old file 51.h2w ...
    Removing old file 52.h2w ...
    Removing old file 53.h2w ...
    Removing old file 54.h2w ...
    Removing old file 55.h2w ...
    Removing old file 56.h2w ...
    Removing old file 57.h2w ...
    Removing old file 58.h2w ...
    Removing old file 59.h2w ...
    Removing old file 60.h2w ...
    Removing old file 61.h2w ...
    Removing old file 62.h2w ...
    Removing old file 63.h2w ...
    Free space: 62.00 GB
    Creating file 1.h2w ... OK!
    Creating file 2.h2w ... OK!
    Creating file 3.h2w ... OK!
    Creating file 4.h2w ... OK!
    Creating file 5.h2w ... OK!
    Creating file 6.h2w ... OK!
    Creating file 7.h2w ... OK!
    Creating file 8.h2w ... OK!
    Creating file 9.h2w ... OK!
    Creating file 10.h2w ... OK!
    Creating file 11.h2w ... OK!
    Creating file 12.h2w ... OK!
    Creating file 13.h2w ... OK!
    Creating file 14.h2w ... OK!
    Creating file 15.h2w ... OK!
    Creating file 16.h2w ... 24.93% -- 2.78 MB/s -- 22:00:24

    and then it automatically began the whole process again. Is this suppose to happen?

  13. there is other possiblity for this, for endusers? if i have to choose to do everithing you did or throwing away the card, i shurely throw away the card...
    simple as that: download - install - have a damn test - repairing the card id necessary - end

  14. Has anyone compiled a usable binary yet, or is there another Mac utility out there that works GUI?
    Or is this the only thing that is working at this time?
    If there's nothing else, anyone know of a GUI program that will run under Ubuntu?
    Thanks

  15. I was able to get a usable binary on OS X using the homebrew package manager and running "brew install f3". It installed the binary under /usr/local/Cellar/f3/2.0/bin. The homebrew package manager is available at http://brew.sh/. f3 2.0 might be older but it still works nicely.

  16. Small correction - I just updated my copy of homebrew and it now builds f3 3.0!

  17. Hi. Thank you for the info. But could you please post a compiled version for the mac. It will be very appreciated. Thank you.

  18. Code does not compile on recent OSX versions (issues with flags available only in non Posix mode). However, a new project has adapted the code for OSX only (and changed it to avoid the compile problems) and it also has a nice GUI (but OSX only).

    It is called sdspeed and should be available on https://github.com/flagsoft/sdspeed

    Hope this helps.

  19. Hello,

    Maybe i'm doing something wrong but it says -bash: ./f3write: No such file or directory
    i'f done this before and it worked, but not any more.

  20. I would like to check for errors on SD cards, can this be used to look for single event upsets on an SD . use the f3write then expose the sd card to some error inducing environments and use f3read to check for single bit errors.

    was not clear if your algorithm was checking every bit.

    Any comments.

  21. Hi is there any chance of a step by step walkthrough? For example how do you and where do you enter 'make mac'?

  22. This is the response I get at present - 'Anitas-MacBook-Air:~ AnitayAdam$ f3write /Volumes/DISK_IMG
    -bash: f3write: command not found'

    Any ideas on what I need to do please?

    Thanks

  24. @Adam: use preceding dot slash to run binary in current directory. so:

    ./f3write /Volumes/DISK_IMG

  25. How to compile the source for mac? ‘make mac’ doesnt work. There are no instructions.

  26. It returns the statement below after running this command "./f3write /Volumes/NO NAME"

    "F3 write 5.0
    Copyright (C) 2010 Digirati Internet LTDA.
    This is free software; see the source for copying conditions.

    Usage: f3write [--start-at=NUM] [--end-at=NUM] "

  28. I am trying to compile f3write.c and I can't get it to work. I installed xCode 6.1.1 and the Command Line Tools for OSX 10.10. I opened Terminal and typed this:

    MyComputer:~ username$ cd f3-5.0
    MyComputer:f3-5.0 username$ gcc f3write.c

    Here is (I presume) my error message.

    Undefined symbols for architecture x86_64:
    "_adjust_unit", referenced from:
    _fill_fs in f3write-27521c.o
    _pr_freespace in f3write-27521c.o
    _measure in f3write-27521c.o
    "_full_fn_from_number", referenced from:
    _unlink_old_files in f3write-27521c.o
    _create_and_fill_file in f3write-27521c.o
    "_ls_my_files", referenced from:
    _unlink_old_files in f3write-27521c.o
    "_parse_args", referenced from:
    _main in f3write-27521c.o
    "_posix_fadvise", referenced from:
    _measure in f3write-27521c.o
    _end_measurement in f3write-27521c.o
    ld: symbol(s) not found for architecture x86_64
    clang: error: linker command failed with exit code 1 (use -v to see invocation)

    What am I doing wrong? Am I missing a file?

  29. @Mike: Just execute "make" inside the folder with the extracted files from the .zip - instead of calling gcc, this should do the trick.

  31. On Mavericks:

    1) Browse to: https://github.com/AltraMayor/f3
    2) Click Download ZIP button on side
    3) Open Terminal (Application->Utilities->Terminal)
    4) Type in (or copy and paste): cd ~/Downloads/f3-master
    5) Hit [Enter]
    6) Type in (or copy and paste): make
    7) Hit [Enter]
    You should see:
    <<<<<<<<<<<<<<<
    cc -std=c99 -Wall -Wextra -pedantic -MMD -ggdb -c -o utils.o utils.c
    cc -std=c99 -Wall -Wextra -pedantic -MMD -ggdb -c -o f3write.o f3write.c
    cc -o f3write utils.o f3write.o -lm
    cc -std=c99 -Wall -Wextra -pedantic -MMD -ggdb -c -o f3read.o f3read.c
    cc -o f3read utils.o f3read.o
    <<<<<<<<<<<<<< Or whatever the name is, from step 8.
    10) Hit [Enter]
    You should see something like:
    <<<<<<<<<<<<<<<
    Free space: 59.54 GB
    Creating file 1.h2w ... OK!
    Creating file 2.h2w ... OK!
    Creating file 3.h2w ... OK!
    Creating file 4.h2w ... OK!
    Creating file 5.h2w ... 7.88% -- 5.17 MB/s -- 2:40:51
    <<<<<<<<<<<<<<<
    And so on.

  32. Steps 8 and 9 seems to have been bungled. Here they are:

    8) Check the name of the sdcard in finder. It is under Devices, probably is UNTITLED, and should have the eject symbol to it's right.
    9) Type in (or copy and paste): ./f3write /Volumes/UNTITLED/
    ----> Or whatever the name is, from step 8.

  33. @Brian Tkatch
    Thank you! Your info helped me run f3-5.0 on my mac running Mavericks

  35. For whatever reason this post has come up a few times in my searches for this topic.

    It may be easier to use "badblocks" from the ext2 programs. I did so via Mac Ports "e2fsprogs" while running El Capitan. I've done similar with badblocks on a Linux box itself... but this is a Mac Centric thread, so here goes:

    with Mac Ports installed, install e2fsprogs. It's super simple:
    "port install e2fsprogs"

    There's a few ways to run badblocks. The one we're after is the destructive write. Here's two recipes - one to check just a partition and one to check the whole disk:

    First, for both of these, you can find the device name(s) that you need to work with with this method:
    # plug the storage in, wait a few seconds for it to dance with the host # busses, then:
    diskutil list

    # Recipe 1: check a partition
    # my drive has a single mounted fs as /dev/disk4s2:
    diskutil unmount /dev/disk4s2
    badblocks -wsv /dev/disk4s2

    # Recipe 2: check the whole disk
    #for each mounted partition in diskutil list, unmount them.
    # lets say I have disk4s1 and s2 mounted:
    diskutil unmount /dev/disk4s1
    diskutil unmount /dev/disk4s2
    #With those unmounted, I can do badblocks on the whole disk:
    badblocks -wsv /dev/disk4
    # if it passes, I make a new partition table, etc. with "Disk Utility"
    # but have been known to fiddle with diskutil to automate the whole
    # shebang.

  36. The simplest solution without need to compile anything is just to run`brew install f3`?

  37. Brew will install ver 5.0 of F3 and it goes on without error BUT....
    it will come up with the same problem that @zak had back in Jan 2015.
    "It returns the statement below after running this command “./f3write /Volumes/NO NAME”

    “F3 write 5.0
    Copyright (C) 2010 Digirati Internet LTDA.
    This is free software; see the source for copying conditions.

    Usage: f3write [–start-at=NUM] [–end-at=NUM] “"

    I am in the process of installing ver 6.0 of F3 but I have to wait till the XCODE gets install (which can take hours to install) I am running this on a fully loaded 15in MacBook Pro late 2013 with 16gig of mem running OS 10.11.5 will update when I get the ver 6.0 complied.

  38. @Richard you're sure the problem isn't caused by the space in "NO NAME"?
    You could try ./f3write /Volumes/NO\ NAME or wrap the path in quotes ./f3write "/Volumes/NO NAME"

  39. after installing xcode (over 4.5 hours) complied and ran f3write again and got the same output as with version 5.0

    F3 write 6.0
    Copyright (C) 2010 Digirati Internet LTDA.
    This is free software; see the source for copying conditions.

    Usage: f3write [--start-at=NUM] [--end-at=NUM]

    tried a number of ways and all I get is the same output
    Has anyone got this to work ?

  40. Thijs, that's the problem... I should have know that thanks for the fix... it works now

  41. Error creating file 'G:\44.h2w'.
    (The file or directory is corrupted and unreadable. Code 1392)
    Warning: Only 62286 of 63997 MByte tested.
    Writing speed: 6.37 MByte/s
    H2testw v1.4

  43. Good tip! I was able to test my sd card installing f3 with homebrew: "brew install f3"

  44. What is the EASIEST way to Install this SD card checker????
    I MEAN EASIEST

    I'm 87 years old , an have no idea of any programming kinda stuff. I would like to Click on a ICON thing, and the app opens and does its job.

    So what do I need to install ? I have Apple Macbook they say it's

    OX 10.9.5

    Like to verify my SD cards.... Thank you

  45. Pingback: Установка UniFi Controller 5 на pine64 – Ish West Script WAU!(lt)
  46. Pingback: Installing UniFi Controller 5 on pine64 – Ish West Script WAU!(lt)

  47. would not be easier if there were an installer, so that people that do not have a clue of what is github, what is even a compile, dude, make it easier for the people..

  48. xxxx:AltraMayor-f3-53772d3 mp6c$ make mac
    make: *** No rule to make target `mac'. Stop.xxxx:AltraMayor-f3-53772d3 mp6c$ make
    cc -std=c99 -Wall -Wextra -pedantic -MMD -ggdb -c -o utils.o utils.c
    In file included from utils.c:22:
    ./utils.h:7:10: fatal error: 'argp.h' file not found
    #include /* For struct argp_state. */
    ^
    1 error generated.
    make: *** [utils.o] Error 1

  50. Does this program give accurate results of the maximum speed of a micro sdxc card? I ran the test on a micro sd card with 128gb of capacity and read speeds up to 80mb/s. The capacity was accurate but the speed at which the data was read came out to an average of 18-20 mb/s.

Leave a Reply

Your email address will not be published. Required fields are marked *