0

I've got 4x 500GB drives in software RAID. /dev/md0 is RAID 1 and mounted to /boot /dev/md1 is RAID 10 and is swap /dev/md2 is RAID 10 and is the main system and data device

I looked at mdadm this evening and noticed on md2... 

State : clean, degraded
Number   Major   Minor   RaidDevice State
0       8        3        0      active sync   /dev/sda3
1       0        0        1      removed
2       8       35        2      active sync   /dev/sdc3
3       8       51        3      active sync   /dev/sdd3

Checking md0 and md1 all drives are shown as active sync and the device state as clean.

Here's the full outputs from mdadm for each device and also the output from /proc/mdstat http://pastebin.com/VL0uYdU9

So it looks like /dev/sdb1 and /dev/sdb2 are functioning in /dev/md0 and /dev/md1 respectively. But /dev/sdb3 has dropped out (apparently it's been removed) from /dev/md2

With RAID 10 I believe the data is ok unless I lose the other drive on the opposite side of the mirror. I am of course backing up to an external device and have verified that these are stable.

I've done some log grepping and noticed this pair of log lines...

Dec  9 04:25:37 hostname smartd[3199]: Device: /dev/sdb, 1 Currently unreadable (pending) sectors
Dec  9 04:25:37 hostname smartd[3199]: Device: /dev/sdb, 1 Offline uncorrectable sectors

Repeating every 30 minutes. It appears this has been the case for a while and it looks like the drive has failed a SMART data check.

On Jan 7th an idiot user rebooted the server, thinking it would solve a mail relay problem.
Here's the the boot from /var/log/messages... http://pastebin.com/jGVsDD54

Why do /dev/sdb1 and /dev/sdb2 appear to be functioning ok and just /dev/sdb3 failed?
Just a particular failed sector that happens to be on sdb3?

Is it worth attempting to re-add this partition to the md2 array?
Or should I bin the drive and replace with a fresh drive?

Improve this question

1 Answer 1

Reset to default
3

A SMART failure indicates that an overall drive failure is imminent (the timeframe is impossible to predict, however); replace this drive ASAP.

Improve this answer
4
  • Cool, I thought that might be the case and was planning to do that just to be sure. I didn't know if the SMART data alert might be a "false positive".
    – batfastad
    Jan 14, 2013 at 12:35
  • There are various kinds of SMART failure; "uncorrectable errors" are the bad kind ;)
    – adaptr
    Jan 14, 2013 at 12:48
  • What's wierd is that only the one partition on that drive has failed the SMART check (although it is the biggest partition by a long way (~498GB)). I thought SMART data applied on a per-drive basis and surely this should affect md0 and md1 as well, although md0 is a 4 drive RAID 1.
    – batfastad
    Jan 14, 2013 at 15:13
  • SMART data is global, yes. The drive reports SMART events as a whole. "Uncorrectable sectors" means spare sectors had to be allocated to replace faulty ones. There is a finite number of spares; once they're used up, the drive will start exhibiting bad blocks and should be replaced immediately.
    – adaptr
    Jan 14, 2013 at 15:17

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .