How can I securely delete the contents of a thumb drive so data can't be recovered.
TL/DR: Make sure you get the right device name, ensure it's not mounted, and do as many random overwrites as you can afford. You can follow it by an erase command designed for flash hardware, if you are on a recent enough distribution. In these checks, always use the drive (like /dev/sdh) and not the partition name (which would be /dev/sdh1)
# dmesg|grep sdXX
[3600.000001] sd 6:0:0:0: [sdXX] 125106176 512-byte logical blocks: (64.0 GB/59.6 GiB)
# blkid|grep sdXX
/dev/sdXX1: PARTUUID="88a03bb2-ced8-4bb2-9883-0a51b4d460a8"
# df|grep /dev/sdXX
# shred -vzn8 /dev/sdXX
shred: /dev/sdXX: pass 1/9 (random)...
shred: /dev/sdXX: pass 1/9 (random)...46MiB/3.8GiB 1%
...
shred: /dev/sdXX: pass 9/9 (000000)...3.8GiB/3.8GiB 100%
# blkdiscard -s /dev/sdXX
blkdiscard: /dev/sdXX: BLKSECDISCARD ioctl failed: Operation not supported
# blkdiscard /dev/sdXX
blkdiscard: /dev/sdXX: BLKDISCARD ioctl failed: Operation not supported
#
In theory, overwriting with zero with dd is just fine. However, due to how the internals of a flash drive are built, if you use a single overwrite pass, there may be several layers of data hidden behind the actual blocks that are still storing leftover information.
Typically a part of flash storage is faulty, and is marked so during manufacturing. There are also other bits that can go wrong (becoming unchangeable, unsettable, or unclearable), these parts must be marked faulty as well during the lifetime. This information is stored in a reserved space, on the same chips as your data. This is one of the several reasons a 4GB thumb drive is not showing 2^32 bytes capacity.
Flash storage is also internally organised in larger blocks, sometimes much larger than the filesystems working on the drive. A typical filesystem block size is 4KB, and the flash segments that can be erased in one go may range from 64KB to even several megabytes. These large blocks can only be erased in whole, which resets all of the block to a known state (all 1s or all 0s). Afterwards a data write can alter any of the bits (change the default 1s into 0s where needed, or change the default 0s into 1s), but only once. To change any of the bits back into the default, all of the segment needs to be erased again!
So, when you want to change a 4KB block (the filesystem is asked to change a single character in the middle of a file), the flash controller would need to read and buffer all 64KB of the old data, erase all of it, and write back the new contents. This would be very slow, erasing segments is the slowest operation. Also, a segment can only erased by a limited times (tens of thousands is typical), so if you make too many changes to a single file, that can quickly deteriorate the drive.
But this is not how it's done. Intelligent flash controllers simply write the 4KB new data elsewhere, and make a note to redirect reads to this 4KB of data in the middle of the old block. They need some more space, that we can't see to store this information about redirects. They also try to make sure that they go through all the accessible segments to store data, this is called wear levelling.
This means that typically old data is still on the drive somewhere! If you just cleared all accessible blocks, all the hidden blocks still keep a quite recent version of the data. Whether this is accessible to an attacker you want your data to be protected from, is a different question.
If you have a recent enough distribution, and the USB drive is programmed to reveal that it is a flash drive, blkdiscard can use the underlying TRIM operation, which is the segment erase that we talked about above. It also has an additional flag to make sure that even the invisible hidden data is fully erased by the hardware:
# blkdiscard -s /dev/myusbdevice
-s, --secure Perform a secure discard. A secure discard is the same as a regular discard except that all copies of the discarded blocks that were possibly created by garbage collection must also be erased. This requires support from the device.
It won't necessarily work, as I demonstrated above. If you get Operation not supported, either your kernel, your utilities, or the USB gateway chip (which allows the flash controller to look like a drive via USB) does not support passing TRIM command. (The flash controller must still be able to erase segments on its own). If it is supported by the vendor of your drive, this is the safest way.
Another, less safe way to make sure you're allowing less of the old data to linger around somewhere is to overwrite it several times, with random values, if possible.
Why random, you ask? Just imagine if the USB drive were made too intelligent, and detected that you wanted to clear a sector, and just made a change in a bitmap that this sector is now free, and will need clearing later. As this means it can speed up writes of zeros, so it makes for a pendrive that appears more efficient, right? Whether your drive is doing it, hard to tell.
At the most extreme, the drive could just remember how much from the start you have cleared, and all it needs to store is about 4 bytes of information to do this, and not clear anything from the data you want to disappear. All so that it could look very fast.
If you are overwriting the data with random, unpredictable values, these optimizations are impossible. So the drive has to make sure the data ends up stored inside the flash chips. But you still won't be able to rule out that some of the previously used sectors are still there with some old data of yours, but the drive just didn't consider important to erase it just yet, since it's not accessible normally. Only the actual TRIM command can guarantee that.
To automate overwriting with random values, you may want to look into using shred, like:
# shred -vzn88 /dev/myusbdrive
The options used:
- -v for making it show the progress
- -z to zero it at as a final phase
- -n8 is to do 8 random passes of overwrites
If possible, use both blkdiscard and shred, if blkdiscard -s is supported by your drive, it's the optimal solution, but can't hurt to do a shred beforehand to rule out firmware mistakes.
Oh, and always double-triple-check the device that you are trying to clear! dmesg can help to see what was the most recently inserted device, and also it's worth to check the device name you intend to clear with ls -al, even for the devices node numbers, and the blkid output to see what partitions may be available that you DON'T want to clear.
Never use these commands on an internal drive that you want to keep using - blkdiscard will only work on solid state drives, but it's not worth to try to lose data!
There may be other ways to clear data securely as technology progresses.
One other way mentioned is the ATA SECURITY ERASE command that can be issued via hdparm commands. In my experience, it is not really supported on flash drives. It was designed for enterprise hard drives, and the feature is not always implemented in lowest cost storage devices.
The TRIM/DISCARD operation is much newer than the SECURITY ERASE command, and was created in response to the flash features, so it has a much higher chance of being implemented, even in cheap USB drives, but it's still not ubiquitous. If you want to erase an SD/micro SD card in an USB dongle, and blkdiscard reports it is not supported, you may want to try a different dongle/card reader, and/or do it in a machine with a direct SD/MMC slot.
I realize this isn't really an answer to your question, but the simplest way is to physically destroy the drive (smashing it repeatedly with a sledgehammer generally does the trick, but industrial shredding or incineration are also options). If you're worried enough about security to want to make sure the data is unrecoverable, the value of that data is likely many times more than the cost of the drive itself, and the physical method is cheap and reliable.
Almost all of the available secure-delete functions were designed for spinning magnetic media, where the computer can usually determine the physical location[1] of the blocks used for the file and write over those same blocks. Solid-state storage controller chips abstract the physical location of the data from the system and likes to move it around.
Your question really depends on who is going after your data. If you are concerned about random loss or casual theft then a regular format will be fine. If you are concerned about being targeted by a highly motivated, technically competent user with substantial resources at their disposal[2] then just incinerate the drive and spend $10 on a new one.
yes, there is logical block mapping. But logical block A will always map to physical block B.
this person would open the drive up, pull the memory chip off the card and read the flash memory directly, bypassing the controller