up vote 0 down vote favorite
share [g+] share [fb]

One the drives in a 2-disk software RAID-1 array on a dedicated server running CentOS failed suddenly this weekend and took the entire server down. While I haven't had time to review the logs and monitoring completely there appears to have been no warning (although I was not specifically monitoring raid). Since I'm new to RAID and have a few questions about the failure:

  1. Should a disk failure in a RAID normally bring down the server? My understanding was that RAID was specifically to prevent this.
  2. When looking at the logs is there any log or thing in particular I should look for as a cause or precursor?
  3. How should I monitor my RAID in the future? Is looking at /proc/mdstat sufficient?
link|improve this question
100% accept rate
Well, I guess you're using the linux software RAID (md). What level of RAID did you deploy ? (because RAID 10 is not available for a 2-disks array) – Fravadona Nov 7 '11 at 14:15
What level RAID? What's your configuration? – Bart Silverstrim Nov 7 '11 at 14:22
If you're using software RAID, it's possible for it to bring down the server, depending on what the nature of the failure is. Hardware RAID controllers help abstract more issues with drives from the system, but it's not perfect. – Bart Silverstrim Nov 7 '11 at 14:23
I ask about RAID level because you say you have RAID 10 on a 2-disk system...RAID 10 requires 4 disks, last I knew... – Bart Silverstrim Nov 7 '11 at 14:24
1  
Fixed: It was indeed RAID 1 not RAID 10 and it is a software RAID. – uesp Nov 7 '11 at 14:34
feedback

1 Answer

up vote 1 down vote accepted

Usually a disk failure doesn't bring the system down. I've had a few disk failures and I have just recieved an email that the disk has failed, the array is marked as degraded and I should change the drive.

I would look into /var/log/messages, as that is where usually this kind of stuff goes. If you want to be notified in advance you should have smart tools installed. They will send you and email if something is probably going to fail.

I would look at /proc/mdstat, set up email alerting and use smart to monitor the drives. Then you can also set up weekly drive tests using smart.

link|improve this answer
for md email alerts edit /etc/mdadm.conf and add:MAILADDR jdoe@somemail.com – Jure1873 Nov 7 '11 at 15:42
and smart's config is in /etc/smartd.conf (the -m parameter) – Jure1873 Nov 7 '11 at 15:44
feedback

Your Answer

 
or
required, but never shown
discard

Not the answer you're looking for? Browse other questions tagged or ask your own question.