As we all know, modern hard disk drives do internal bad sector management inside their firmware, that is, when the drive detects a physically damanged or unreliable physical sector, it replaces the bad one with a good one from the reserved sector store. This mechanism eliminates the need for OS to do bad sector managment itself. Users can know the development of sector replacement count from S.M.A.R.T item called "reallocated sector count".

A sample HDD drive with too many reallocated sectors(the index 133 drops below threshold 140):

HDD SMART err1 (CrystalDiskInfo-6.5.2) HDD SMART err2

Now my question is: When does the HDD firmware determine a sector should be replaced. I imagine such ideal situation:

When HDD writes a sector, it does read verification immediately(verify CRC checksum and employ ECC to recover the soft error bits). In case the verification fails or the sector is deemed unreliable by the firmware, the firmware can immediately replace the sector so no user data is lost(at least not lost at the moment it is written to disk surface). This is a nice moment to do sector replacement because the to-be-written data now surely remains in hard drive's internal cache. The drive can try as many "hot spare" sectors as he will until he finds a good one.

If my idea is not the fact, then the HDD can only verify the written sector the next time user fetches it; but in case the next fetch fails CRC and ECC, and its corresponding cache has been discarded, we can do nothing but to know the previously written data has been permanently lost.

Which behavior is the truth? Is there any web articles clarifying this?

SSD disk should face the same situation, then what about SSD? SSD's case seems easier to understand, because I can easily imagine that SSD can do a quick read verification after write, very quickly; however, is such scheme practical for a spindle hard drive? It seems that the HHD head can do either read or write at one given time, so in order to verify the write, it has to wait for another spinning round until the written sector is positioned again under the head -- which would drastically slow down the data throughput. -- I'm so baffled.

Drives to perform a form of write-verification, and have been doing so for quite some time. Back in the olden days of loud drives, you could audibly tell when a drive was grinding on a bad block from the sound of it. I can't explain the exact mechanism, and I'm fairly sure the exact way has shifted a bit as we've moved through the various magnetic technologies. But the failure can be detected on write, and the sector will be pulled out of the reallocation pool.

It can also be detected on read, with the failure case you've already identified. Some of the higher-end RAID arrays (and of course ZFS) have background scanning routines to read storage during idle times specifically to locate these errors. In parity or mirrored RAID, you theoretically have a good copy elsewhere which makes recovery easy.

An SSD cell that fails to program will return a similar fail-state, and a fresh block will be pulled out of the reallocation pool. Worn-out SSD cells tend to go read-only, so the data is recoverable. Actual-broken cells is another story, same as a bit of grit landing on a drive-platter.

  • Could you give some citation about some working detail of disk head write-verifying behavior? That Wikipedia link (en.wikipedia.org/wiki/Disk_read-and-write_head) does not say anything about this. – Jimm Chen Sep 2 '15 at 6:39
  • @JimmChen I'm afraid I'll have to wimp out of that (citation needed) and claim personal experience as the basis of my claim. – sysadmin1138 Sep 2 '15 at 16:08
  • "ZFS have background scanning routines to read storage during idle times" no. zfs does not have any background scanning routines. can you point to the relevant code that does what you claim to do? here is the source: github.com/openzfs/openzfs. you can still do initiate MANUAL scrub, you can create OS-scheduled scans, but no automatic scanning happens by default INSIDE the zfs code. – Costin Gușă Feb 4 at 10:00

Your Answer

By clicking "Post Your Answer", you acknowledge that you have read our updated terms of service, privacy policy and cookie policy, and that your continued use of the website is subject to these policies.

Not the answer you're looking for? Browse other questions tagged or ask your own question.