I want to remove my files securely without worrying about anyone restoring them, I know I can use shred but it takes too long even with -n 1 so I thought maybe if I remove files and write on the disk using dd, filling up all available space I won't have to use shred right? I believe dd will be faster than shred especially that my files almost fill up most of the available space in the disk
So does filling up the disk using dd guarantee that my files will be securely removed?
This will go against most conventional wisdom on the Internet, but here we go...
If this is a modern rotating disk, a simple pass of dd with /dev/zero is enough to foil almost any attempt at data recovery, even from a professional data recovery house. It might be possible to extract some data with extremely expensive specialized equipment (e.g. a government lab), but that is out of reach of pretty much anyone that isn't willing to spend $millions on you. (Note this will not comply with any official-sounding government standards for data disposal, but it works.)
The problem with most of the wisdom you read on the Internet about this topic, is that it is more urban legend than actual fact. If you look for an actual source on this topic, most people refer back to a paper that was published in 1996, and was referring to MFM/RLL drives (pre-IDE). Additionally, most of the government standards for data destruction that people refer to are decades old.
The logic behind multiple passes to erase data boils down to the idea that residual information can linger in the space between sectors on a platter. On older drives, the density of sectors was relatively low, and there was lots of empty space on the platters where this residual data could linger. Since 1996, hard drive capacities have increased by orders of magnitude, while platter size has remained the same. There simply is not that much empty space in a platter for data to linger anymore. If there was usable extra space in the platters, drive manufactures would be using it and selling you a higher-capacity disk.
The wisdom of these secure erase standards has been picked apart, and papers have been published that say a single pass is enough.
A few years ago, someone issued the Great Zero Challenge, where someone overwrote a drive with dd and /dev/zero, and issued an open challenge for someone to extract the data. There were no takers as I recall. (Disclaimer: The original web site for this challenge is gone now.)
But what about Solid State Drives? Because of the flash wear leveling, bad sector remapping, and garbage collection, and additional "hidden capacity", traditional overwrite methods may not actually overwrite the data (although it will appear overwritten to the host PC). A single pass of dd with /dev/zero will stop a casual user from reading back any data from the SSD. However, a dedicated attacker with a logic analyzer can crack open the drive and extract data from the flash chips inside.
This problem was identified a while ago. So, a command called Secure Erase was added to the ATA standard. The firmware in the drive will securely erase all of the flash cells. Most modern SSDs will support this command. I beleive it also works with rotating drives. Note that this command can sometimes be tricky for an end user to access. You typically need a special utility to use it, some BIOSes implement a "security freeze" that can get in the way. Check with the SSD manufacturer for a utility. If they do not have one you there are 3rd party ones that may work.
Note that some people have raised concerns about the reliability of the secure erase functionality built into the drive firmware. There was a paper published in 2011 that showed some drives will leave data behind after a secure erase. Note that SSD firmware has advanced quite a bit since then. If secure erase is an important function to you, I would recommend purchasing drives from a top-tier manufacturer, preferably something in their server/datacenter line (where buggy firmware is less likely to be tolerated).
If the above make you nervous about data remaining on the drive, your next best option is to fill the drives with random data multiple times, as this will hopefully take care of overwriting the excess hidden capacity in the SSD, but you cannot be absolutely sure without knowledge of the internal workings of the firmware. This will also shorten the lifespan of the SSD.
What you should take away from this:
Define "securely". How badly do you want this data to be gone, and how painful would it be to lose the storage space? If the data absolutely positively must never be seen ever again by anybody, the proper tool isn't dd or shred: it's a sledgehammer.
If you don't quite need to go that far, then you can use software, but the question still remains: who do you think might want the data, and how important is it that they not get it? A pass of zeroes will stop a script kiddie, but if anyone is actually being paid to recover your data, they won't have much trouble getting it. A pass of zeroes, a pass of ones, and a pass of /dev/random will make a determined attacker's life pretty miserable, but if they've got the resources and want your data badly enough, there are still ways to get it, and running that many passes takes long enough that you might as well use shred anyway.
The bottom line is that if you really want to be secure, there's no reason to use dd for erasing things. It won't stop any kind of sophisticated attack. If shred takes longer than a sledgehammer would, then use the sledgehammer instead. If you can't afford to lose the disk space, then take the time to use shred. Using dd for this purpose just isn't secure enough.
dd isn't less secure than shred. In fact with the right arguments they will do the same thing.
If you are overwriting an entire disk, then dd should the job fine AFAIAA. shred allows the secure deletion of individual files, and even if you shred every file on the disk, it preserves the filesystem. So the difference is that they operate at different levels. Shred's need to figure out where a particular file is stored and only overwrite that bit is presumably why it takes longer.
No, they won't be securely removed. If you overwrite the disk with zero's then it still will be possible to recover the data. If you do it with random data (/dev/random) once, it still will be possible to recover the data. You need to overwrite the disk with dd at least couple of times. Which will take exactly the same time (or even longer) as with shred.
So to sum up, it always takes a lot of time, there is no quick method.
/dev/random as the input file might do the trick, especially if you loop if several times. I'm sure dd can be configured to to wipe a disk according to whatever DoD standard you like, as it is just a low level tool that copies data, the user defines the data. +1 for shred though.
Jan 30, 2014 at 14:39
As others have stated DD is not going to do what you need. DBAN is a very good tool for this. DD will not make the files un-recoverable. Even DBAN or other tools do not have a 100% guarantee. If you are concerned with data security the best option is to destroy the physical media.
DBAN is a self-contained boot disk that automatically deletes The contents of any hard
disk that it can detect. This method can help prevent identity theft before recycling a
computer. It is also a solution commonly used to remove viruses and spyware from
Microsoft Windows installations. DBAN prevents all known techniques of hard disk forensic
analysis. It does not provide users with a proof of erasure, such as an audit-ready
erasure report.
