Take the 2-minute tour ×
Server Fault is a question and answer site for professional system and network administrators. It's 100% free, no registration required.

I have 4 disks, multiple partitions and from some partitions raid10 arrays. Something went wrong and three disks "failed" at the same time (i think it was a software problem, disks are ok). Now I have not working array. Is there a way to recreate the array with all data on these partitions? Data should be ok, I only need to rebuild and sync the array but I don't know how (I don't want to loose data althought I have backups but the entire restore process will take ~20 hours).

Thanks for your help!

share|improve this question
1  
Please provide cat /proc/mdstat output –  Alex Aug 14 '12 at 22:02
    
Slightly related see this. serverfault.com/questions/369891/linux-software-raid-recovery –  Zoredache Aug 14 '12 at 22:45

2 Answers 2

up vote 3 down vote accepted

Assuming that your drives are still in good condition, then you probably need to spend some time determining exactly how your array was configured. By using mdadm --examine and --detail against the individual partitions and volumes.

Then you might be able to re-create the array using the --assume-clean option, which will create a new array without doing anything to the data. If you re-create the array using the correct parameters, then you might be able to mount your volume depending on exactly how it failed, and the level of corruption from the failure.

If you are able to get it up, you will want to force a resync, and it would probably be a good idea to verify that your data hasn't been corrupted.

If you are not familiar with the process though, it might be easier to just rebuild your array now and then restore. Spending time on trying to rebuild the volume is a bit of a gamble. If you are unlucky then your volume could be in a state where it simply cannot be fixed and you have to restore anyway. It might be better to start the restore now.

In an ideal world you would have a second set of disks that you could be restoring to while you try and recover the original. If you can be restoring and trying to recover the volume in parallel, I suggest you do that.

share|improve this answer
    
Marking as accepted, it leaded me to solving the problem. Firstly, I disabled sda (this one didn't failed) and rebooted from some linux cd. Without sda, the raid array was automatically detected and started as degraded. I made full data backup, rebooted, destroyed mdX device, zeroed superblocks on sd[abcd]X, created new array with first disk missing (sda was on first position) and then added the sdaX to mdX. I made it with each array and now, server is up and running without data loss :) Thanks –  Bruce Aug 15 '12 at 10:23

Normally, if 3 disks fail in a RAID10 array, your recoverability is basically nil (you may possibly be able to force the filesystem to give you something, but half of your data is going to be gone).

I doubt, if the controller thinks the drives failed, that they are entirely OK; at the very least, the RAID information is corrupt. You may be able to access the drives using LVM on linux, if they are OK. Otherwise, it is a data recovery job and your backups would be a way better choice.

share|improve this answer
    
I think there must be a way to restore the array. All three disks went down at once so I can try to restore the array from these three disks and the fourt I will add later and sync. Disks are working, there was a software shutdown (perhaps some bug in kernel) no hardware failure. –  Bruce Aug 14 '12 at 22:19
    
Not all software issues are trivially recoverable. People lose data to filesystem bugs that is insanely difficult to get back, too. If LVM recognizes that the disks are part of an array and can read them, you're in luck; otherwise, I can't think of anything that would help at all. –  Falcon Momot Aug 14 '12 at 22:29

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.