Take the 2-minute tour ×
Server Fault is a question and answer site for system and network administrators. It's 100% free, no registration required.

I have a software RAID 1 setup in my CentOS 6.2 and set to be bootable in any of the HDDs in case one of them fails.

Questions:

  1. How can I recognize if one of the HDDs fail? or early signs of failure on CentOS? (preventive maintenance)
  2. If one of the disks fails, what are the things to be done? Like any DATA RECOVERY methods. (corrective maintenance) or (how to copy the data in the remaining HDD and copy it to the new HDD)

I would gladly appreciate if you could give me any references.

Update:

I tried to boot in only one of the disks. I removed sdb first and the system successfully booted in sda. Then I removed sda and booted in sdb and still successfully booted. But when I put them back together and executed cat /proc/mdstat & mdadm -D /dev/md0, it shows that one of the disks is still removed.

share|improve this question
    
What sort of machine is the OS running on? Dedicated HP/Dell server, or desktop machine with two hard disks in? –  tombull89 Oct 18 '12 at 7:54
    
Only a desktop with two hard disks Mr. tombull89 –  reena saur Oct 18 '12 at 7:59
    
Okay, I guess in that case they are plugged straight into the motherboard and you're not using a RAID card. If a disk fails you'll have to power off the machine to replace it instead of using a servers hot-swap ability. If you were using a dedicated server or RAID card there would like be fuctions to monitor disk status. –  tombull89 Oct 18 '12 at 8:13

2 Answers 2

  1. If you are lucky (and have enabled the daemon) you will get SMART warnings in the logs before the disk fails. This is not guaranteed, however. In my experience I see SMART errors before disks blow up in less than 50% of the cases. Make sure you have something monitoring the logs.
  2. After a disk failure you replace the disk and rebuild. The RAID system should recover from this. Just hope that you don't have another disk error while rebuilding...

I would highly recommend having a good backup strategy instead of planning for data recovery. Raid is perfect for improving uptime of a server, but all it takes is one little software bug and all your data is gone.

share|improve this answer
    
hello pehrs, thanks for answering! I'm really kinda new with this RAID thing and I really need all the help I can get. If one of the hard disks fail, do i just have to replace it with a new one? and how to rebuild? And what are some good backup strategies you can suggest? –  reena saur Oct 18 '12 at 8:14
    
You replace the physical disk, and use the mdadm tool to rebuild the array. –  pehrs Oct 18 '12 at 8:21

1 Software raid in Centos? So, this must be a mdraid. Your can see raid status like this: cat /proc/mdstat or: mdadm --detail /dev/md[0-9] , which md is using you can found by fstab, mount, pvs or elsewhere. And toy can also setup MAILADDR in /etc/mdadm.conf for email notification, if some event occurs.

2 just remove faulty disk by mdadm --manage /dev/md[0-9] --remove /dev/sd[a-z], replace it and add new disk to raid array (mdadm --re-add, add)

additionally, you may need to recover mbr of failed disk to make it bootable too.

share|improve this answer
    
I tried to boot in only one of the disks. I removed sdb first and the system successfully booted in sda. Then I removed sda and booted in sdb and still successfully booted. But when I put them back together and executed cat /proc/mdstat & mdadm -D /dev/md0, it shows that one of the disks is still removed. I can't locate which drive it is. –  reena saur Oct 18 '12 at 8:43
    
Then you connect both 2 disks, its only bios boot order play with you. So, if in bios you set boot for example disk with serial number X, then it must be in your md, and disk with serial Y - beeing failed in array. The same story, if Y is been booted - X will be failed. You need to find mostly actual disk and make it bootable, and recover secound from it. Actual disk in normal situation have last metadata event count (from --examine), but you already booted from both disks, so disks have writes on it and metadata event count not the true story for you... –  unlo Oct 18 '12 at 10:00

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.