What are the error rates for reading and writing to storage devices, and do our file systems and OS's account for this?
It varies a lot, and depends on the manufacturing processes surrounding the drive in question. There are a couple of different types of storage errors that can impact data and data access:
- Non-recoverable read errors. These are errors that occur as data is attempting to be read from the drive. They're transient, but can have major impacts. If one arrives during a rebuild of a RAID5 array, chances are good that the entire array will be lost. Bad mojo.
- Bad sectors Actually bad sectors on a drive, like what you have.
The non-recoverable read error rate is something that shows up on hard-drive spec sheets. It'll be listed as something like, "1 sector in 10^14 sectors read". This is what will limit how large of a RAID5 array you can create with drives of that type.
Bad sector error rate can't be predicted, at least with rotational media. They just come. This is why drives have a small amount of sector-relocation space. When an OS attempts to write to a sector that has suddenly gone bad, the drive will write that sector instead to the relocation area and increment the Bad Blocks list in the SMART data. Problems start arriving when the drive runs out of relocation-blocks, at which point it is really seen by the OS. Handling methods vary by OS and file-system.
The newest filesystems on Linux (btrfs, and probably ext4) support checksumming of journal and data writes. This allows for catching when a bad sector has been encountered and not reported by the media (such as when the reallocated-sectors are fully consumed). This is a somewhat expensive operation, as things go, but things are fast enough these days that it's not much of a problem for most workloads.
Windows NTFS does not support journal or data checksumming, and probably won't for several years. Bad-sector detection is done after the fact when reading data fails for some sectors or clusters. Checksumming is a very new idea (relative to file-system development) that hasn't fully percolated out to every corner of the market yet. The most nimble operating systems (Linux, probably BSDs, until recently OpenSolaris ZFS builds) get it first, the rest will get it eventually.