Take the 2-minute tour ×
Server Fault is a question and answer site for professional system and network administrators. It's 100% free, no registration required.

Apologies for the horribly vague question...

What are the error rates for reading and writing to storage devices, and do our file systems and OS's account for this?

I've had bad sectors on my laptop's 320gb hdd for years now, and it seems to keep chugging along. I keep backups of everything because I'm anticipating a major failure anytime now.

What happens when a HDD has a bad sector? Are bad sectors common? Will operating systems work around bad sectors?.

What about the occasional HDD read write errors - seemingly these errors could cause catastrophic damage - do file systems have a mechanism for detecting and correcting these problems (eg. check sum)?

share|improve this question
    
Voting to close. Better suited for SuperUser. –  TomTom Oct 16 '10 at 12:53
add comment

2 Answers

Now guess what RAID was invented for ;) Seriously.

What happens when a HDD has a bad sector?

Read error.

Are bad sectors common?

There are tons around.

I have about a dozen on my hard disk, is this acceptable, and will operating systems work around these problems.

Well, the disc will not use the sectory anymore and map them to spare sectors it has in reserve for that. The OS will see nothing. The data on those is gone.

Read up on RAID which is done exactly to handle issues with storage device reliability. Never trust normal storage. What you do if tomorrow the disk just dies? Yes, happens. Quite frequently.

share|improve this answer
1  
RAID is definitely not the end of it... there's much more to securing to data than most of us can handle at times... read up on silent data corruption for example, which even most of the more expensive raid systems don't notice, you'll need a FS with checksum support to account for things like that... –  Marcel G Oct 16 '10 at 13:30
    
Ok given raid - so now you have multiple storage devices .... does this push the read/write error rate beyond acceptable levels? How does one acquire a sense of trust with storage. I assume there are diminishing returns on replicated storage, where is the "sweetspot"? –  chickeninabiscuit Oct 16 '10 at 13:37
    
I'm fairly comfortable with a Raid solution - what happens when you are transferring several TB of data - for example say 12TB - what level of replication would you consider error proof? The reason I ask this question is that I don't know the relationship between the datastore size and error rates? –  chickeninabiscuit Oct 16 '10 at 13:43
add comment

What are the error rates for reading and writing to storage devices, and do our file systems and OS's account for this?

It varies a lot, and depends on the manufacturing processes surrounding the drive in question. There are a couple of different types of storage errors that can impact data and data access:

  • Non-recoverable read errors. These are errors that occur as data is attempting to be read from the drive. They're transient, but can have major impacts. If one arrives during a rebuild of a RAID5 array, chances are good that the entire array will be lost. Bad mojo.
  • Bad sectors Actually bad sectors on a drive, like what you have.

The non-recoverable read error rate is something that shows up on hard-drive spec sheets. It'll be listed as something like, "1 sector in 10^14 sectors read". This is what will limit how large of a RAID5 array you can create with drives of that type.

Bad sector error rate can't be predicted, at least with rotational media. They just come. This is why drives have a small amount of sector-relocation space. When an OS attempts to write to a sector that has suddenly gone bad, the drive will write that sector instead to the relocation area and increment the Bad Blocks list in the SMART data. Problems start arriving when the drive runs out of relocation-blocks, at which point it is really seen by the OS. Handling methods vary by OS and file-system.

The newest filesystems on Linux (btrfs, and probably ext4) support checksumming of journal and data writes. This allows for catching when a bad sector has been encountered and not reported by the media (such as when the reallocated-sectors are fully consumed). This is a somewhat expensive operation, as things go, but things are fast enough these days that it's not much of a problem for most workloads.

Windows NTFS does not support journal or data checksumming, and probably won't for several years. Bad-sector detection is done after the fact when reading data fails for some sectors or clusters. Checksumming is a very new idea (relative to file-system development) that hasn't fully percolated out to every corner of the market yet. The most nimble operating systems (Linux, probably BSDs, until recently OpenSolaris ZFS builds) get it first, the rest will get it eventually.

share|improve this answer
add comment

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.