I received this rather nice email today suggesting one of the drives in a RAID1 array has failed.

A Fail event had been detected on md device /dev/md4.

It could be related to component device /dev/sdc2.

Faithfully yours, etc.

P.S. The /proc/mdstat file currently contains the following:

Personalities : [raid1]  md4 : active raid1 sdd2[1] sdc2[2](F)
      87667136 blocks [2/1] [_U]
       md3 : active raid1 sdd1[1] sdc1[0]
      250304 blocks [2/2] [UU]

The strange thing is that sdc2 is just one partition and the other partition has not failed.

Since the server in another country, I can't physically inspect it. Any suggestions as to how to test whether this is really a failure or a glitch?

  • 1
    You can get a failure if there's a bad sector in /dev/sdc2. The drive itself doesn't have to die for mdadm to detect an error. – cjc Mar 20 '12 at 20:14

If your hard is smart-enabled: smartctl -t long /dev/sdc

and after one or two hours: smartctl -a /dev/sdc

and have a look if it reports errors.

  • 1
    thanks. smartctl confirmed the drive is failed. the other drive in the array was also flagged as failing. luckily there were 2 other drives in there and i've backed up the drives and mirrored onto the 2nd RAID1 mirror. incidentally, the other RAID1 are old hard disks that have been running for years, the failed/failing array were SSDs that were put in less than 6 months ago... – Q789Bart Mar 20 '12 at 20:34
  • @Q789Bart Lifetimes of electronic devices have been said to follow a Bathtub curve, with a relatively high infant mortality, followed by a typical geometric distribution of failures through old age. This is partly confirmed by a Google study of disk failures several years ago. – ephemient Mar 21 '12 at 0:52

Re-Add the partition and look if the error reappears. If it does your drive is starting to fail.

  • +1 - we've had "green" drives (the things we didn't know when we originally installed them...) that would drop out and successfully re-add over a period of a few months before dying entirely (both members of the mirror within a week or so of each other, too). – Andrew Mar 21 '12 at 2:21

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.