Take the 2-minute tour ×
Server Fault is a question and answer site for professional system and network administrators. It's 100% free, no registration required.

Is it safe to backup data to a hard drive and then leave it for a number of years?

Assuming the file system format can still be read, is this a safe thing to do. Or is it better to continually rewrite the data (every 6 months or so) to make sure it remains valid?

Or is this a stupid question?

share|improve this question
add comment

14 Answers

up vote 37 down vote accepted

I wouldn't trust important backups to any single device for any significant length of time.

I've had plenty of CDs that couldn't be read after a while. (Cheap ones, admittedly, but I'm leary of the longevity claims made.)

I've had hard disks silently corrupt data.

I seem to remember I've even had SSD failures, although with a low number of writes I'd expect them to be pretty reliable.

Aside from all of these things, using a single copy means you've got no protection against physical disasters: fire etc. If you have multiple copies, you can separate them physically. Ideally I'd take some number (e.g. 3) of copies and run a checksum (I usually use MD5) periodically over everything. If one of the copies becomes corrupt in some way, if you've got multiple other copies you should be able to trust the majority, and create a new backup to replace the corrupted one. (Of course, if you keep the correct checksums in a separate place, you could trust even a single backup which still gives the right checksums, as the canonical source for replacements.)

Of course, how much trouble you go to depends on the value of the data. My personal home data is only backed up on a RAIDed NAS. My work data is in Google datacenters, which I trust fairly strongly :)

share|improve this answer
add comment

Given your other options for backup, HDD is the safest way to go. Other options include Magnetic Tape, SSD and optical media.

Let's examine the pitfalls of each:

MT: More prone to erasure when exposed to a magnetic field than a HDD. Readers are also becoming harder and harder to find. You don't want to come back in 5 years and find that there's no way to remove the data from your medium.

SSD: Reliable in that there are no moving parts. They are prone to electrical degradation after several read/write cycles which is troublesome and potentially dangerous. The likelihood of losing data while the drive is not in use is slim, however.

Optical Media: The least reliable of the bunch. They're extremely prone to physical degradation (bending/warping) and it requires very little to throw them out of their deflection spec. Further, the encoding scheme used to write data to most optical media is rather complex, creating a greater likelihood of single element failure leading to unreadability.

HDDs: Solid, sealed devices. Can be damaged by physical shock more easily than most of the above devices. Has precise mechanical parts that can lead to failed read/writes if damaged.

The benefit of HDDs, however, is that they ARE sealed. All of the moving parts are stored in an air-filtered enclosure. The magnetic stability of the bits on the disk is quite high and unlikely to change.

Further, if the mechanical parts fail, it is possible to have the platters removed and the data recovered from them directly.

There's no perfect option, but of the imperfect ones, HDD would probably be your best bet.

share|improve this answer
1  
Wrong. Tape is more safe - tape is MADE for long termarchival and has no logic in it. HD electronic fail?the little motor failing to spin it up - you are dead. Tape engine fail - use another drive. Tapes + some optimcal media are specifically made for long term archival. –  TomTom Jan 12 '12 at 5:04
add comment

I would say you should recycle the media every other year or so - that is, replacing the drive, disc or tape with whatever there is to replace it with and keep more than one copy.

Few things lasts forever, optical media can degrade rapidly depending on quality, method of writing to it and environment where it's stored. Mechanical parts can always fail or there could be bugs in the firmware that is related to time or to wear and tear.

I've pondered over your question often, it would be convenient with something that is guaranteed to stay working for say 5 years. There's tapes and other form sof backup media rated for 10 or more years but I'd never trust that, at least not without a decent amount of redundancy (several copies on different batches).

Keeping the data fresh and continually recycled seems to be the reliable way to go - that way you get to test it regularly as well.

share|improve this answer
add comment

From the article

What advice do you have for long-term storage of disk drives and other media?

Keep your hard drives in a climate controlled environment within an acceptable temperature and humidity range. Also, protect the drives from electrostatic discharge (ESD) and vibration -- this is normally done in their packaging, but it's important to prevent ESD, physical shock and excessive vibration when the drives are removed for storage.

All magnetic storage media has a finite life because magnetic fields start to decay as soon as they are written. This means a tape or drive will not retain its data forever. In a proper storage environment, it's reasonable to expect that the drive should remain readable for up to 10 years.

The concern is more about the drive's mechanical reliability; will it physically spin up? After a very long period of disuse, the spindle bearings or head actuator may be stiff, resulting in read/write errors. These considerations are particularly important for long-term archival storage systems, as well as the new class of removable hard-disk drives that are now appearing from ProStor Systems Inc., Imation Corp., Quantum Corp. and Iomega Corp.

share|improve this answer
add comment

HDDs have actually quite high life-expectations, at least from the magnetic side (setting external magnetic fields asside). The main problem with them is, they could eventually suffer mechanically, i.e. not spin up if they are not used regularely, as some oils and coplings could become a problem.

The safest approaches to really long-time storage in my opinion are:

  • stream to one or more magnetic tapes
  • print to paper and/or micro-film
  • keep copies on operating (running) HDDs distributed over several physical machines and locations
  • use an additional external backup space like Amazon S3

Optical media, especially the ones available for consumer use have unexpected low life-expectancy. You should at least check the quality of the raw data read every two years. You could have lost data in the meantime, though.

EDIT: An important aspect in this case is also that you should add checksums to the stored files (MD5, SHA1, etc.), so you'd be able to realize that some corruption occured (or not).

share|improve this answer
add comment

Hard drives are fine, but to keep the best file integrity you need to recopy the information every now and then. if you just leave the information sitting there and expect it to be perfect in 5 years, think again. you should recopy all information on HDD's every 6 months or so if they are not in regular use. its fine to use the same drive, you just need to recopy all of the information on it and replace the contents fresh.

To avoid some of the issues above, i would use a fairly new hard drive that hasn't had too many read/writes to start with. i've had great performance from lacie hd's but i also have some western digital mybooks that have held up well even though they were "budget" hard drives (the mybooks have had slightly more corruption over time than the lacie's but i'm guessing this is relative and can just be the luck of the draw from any manufacturer.

But don't quote me on this.). i'm in the process of backing up all of my digital photography onto a couple of extra externals to store in two separate places besides the two backups on my desk. One is going to my parents house 2000 miles away and the other is going to a safety deposit box. I plan to update/recopy the safety deposit box drive every few months and the drive at my parents house every 6 months to a year.

The 2 backups on my desk get updated every week or so.

share|improve this answer
add comment

I've had hdds fail while "in storage", i.e. sitting in a climate-controlled room for a few years that, when called into duty again, refused to spin up or be booted from.

So no, I wouldn't say that this is a particularly good idea. As others have said, as part of a blunderbuss strategy it is one way of keeping a copy of your data, but it probably shouldn't be your only one.

share|improve this answer
add comment

Do not store your hard drives for any length of time. They are designed to be on. If you don't let the HD spin up every now and then, they will go bad. I'm talking months or a year here.

They will break if not used. MTBF is "guaranteed" for drives in use, not in storage.

share|improve this answer
1  
Note that spinning a hard drive up "every now and then" is also not great for it -- Startup time for drives is a high-wear period as all the moving parts have to get moving again, so if you take the drive out for a 5 minute spin-up every month or so you'll be shortening its life considerably. MTBF figures for server-class hard drives are usually based on continuous or near-continuous operation, with a cycle limit (spin up/spin down) also noted. –  voretaq7 Jan 12 '12 at 4:28
    
@voretaq7: yes, spinning up a hard drive puts lots of stress in it, OTOH we're talking about 120 spin-ups in a 10 years time. Regular users go through this much in a single year if not faster... –  Hubert Kario May 26 '12 at 10:44
add comment

It sounds like you're not so much concerned about hardware failures, but file corruption and bit rot. In this case, ZFS is your best ally. If data preservation is your goal, consider using RAIDZ2 if you can afford it, or at least RAIDZ1. RAIDZ is comparable to RAID5, except it uses a variable stripe width to eliminate the infamous RAID5 write hole. This is especially useful with a cheap NAS, because power failures likely won't corrupt the array. The file corruption and bit rot are taken care of by real-time disk scrubbing, in which the data is constantly being checksummed to verify it's accurate. Those are just the tip of the iceberg with how ZFS is THE filesystem of choice.

If you want an easy NAS setup at home with ZFS built in, check out http://freenas.org. The latest release candidate includes ZFS, and it's not that hard to set up.

It will be interesting to see the long term results of switching to ZFS simply for data preservation... it's too new at the moment. However, the facts are all there, and it's a no-brainer: the best file system for data integrity is ZFS.

share|improve this answer
add comment

I want to say probably (as long as you keep it away from magnets ;-), but I'm not sure. For long-term storage I would transfer the data to some archival format like a DVD - I think that like CDs, they're supposed to last 100 years. You can still keep the HD around for easy access, of course, but the DVD gives you a reliable backup.

share|improve this answer
    
Just curious, but can you point to studies that demonstrate the long term reliability of optical media? Are there certain materials/brands that last longer than others? I have heard (but not experienced) that regular consumer-writable optical disks can become unreadable in a matter of years. –  Jeremy Huiskamp May 7 '09 at 21:08
    
DVDs (especially the ones you use for burning yourself) have lower life-expecations as HDDs. –  Martin C. May 7 '09 at 21:10
    
Most CD/DVD lifetime estimates I've seen point to more around 10-15 years, and that's in optimal conditions. Non-optimal storage of optical storage media tends to decrease their lifetime dramatically. And I've seen this reflected in my observations. –  patjbs May 7 '09 at 21:13
    
Backing up a 500GB drive to DVD is pretty painful. Optical media may or may not last as long as a hard drive, depending on many factors. –  Eddie May 7 '09 at 21:13
1  
There are "archival grade" writable optical media, but the actual lifespan is questionable AND require very specific environmental conditions. –  duffbeer703 May 7 '09 at 23:10
add comment

One additional suggestion is that you should also always move data to the current format technology as soon as the previous technology is nearing its end of life. For example, currently I'd suggest moving away from IDE drives, as computers are starting to ship without IDE connectors and controllers.

Similarly in the audio/video archiving we have moved from Cassette (VHS) to CD (or LaserDisc) to DVD, and now to flash storage.

You might keep a USB-to-IDE adapter around, but along with regularly recycling your data between storage devices and locations, you should also keep in mind moving the data to current technology so as not to wait 10 years to find out you can no longer access the data on that 5.25" floppy disk.

share|improve this answer
add comment

If you want your data to survive or any period of time:

  • Use tape if access is infrequent. Follow environmental guidelines and do the homework to determine how often you need to rotate the media.
  • Use disk if you need access to the data. The disks should be "active". A disk in a closet is likely to either fail or get thrown out a few years down the road.

Using a third party provider is another alternative. Something like Amazon S3, Mozzy or a similar service gives you an ultra-low cost way to store stuff.

share|improve this answer
add comment

Blu-ray seems to be nice solution for this problem.

share|improve this answer
add comment

They will store data safely for a few years but you would be better to copy them every two years or so yes.

share|improve this answer
add comment

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.