up vote 0 down vote favorite
share [g+] share [fb]

someone has asked me to rebuild a faulty raid 5 on a 3ware raid controller... i have 3 drives all 160 GB
ive gone through the steps
ie selected the degraded array unit and then gone over to "Rebuild Array" and inplace of "Degraded" it now says "REBUILDING (after F8)"

i hit F8 and it attempts to load the OS but then it fails a disk check at 57.6% :S any ideas?

OS is debian by the way.

link|improve this question
47% accept rate
feedback

4 Answers

up vote 4 down vote

Doesn't sound good...You can try ripping the "rebuilding" drive out, on the chance that you just got a DOA drive, but chances are it was one of the others that failed. Rebuilding a failed drive is a common source of failure with raids...It's an intensive process.

I once had a 20 drive array fail one drive after another as the array was being rebuilt; when the smoke cleared I'd replaced 18 drives because they warned of failure, and 2 others just because I didn't trust 'em (This is why you never buy 20 drives from the same production group).

link|improve this answer
1  
Ouch. This is (one reason) why having one 20 drive array is a bad idea! At least stripe between a couple of arrays. It's just not worth the extra space savings. – Kamil Kisiel Nov 10 '09 at 17:48
1  
Actually, the 20 drive raid was 5 sets of 4 drives, striped raid 5...The problem was that the drive array served this big database, so before I started the rebuild, I'd export the tables that were on the failing array, and copy them into the slack space of a neighboring disk set, and it was the copying process that (as often as not) started the other drives freaking out. – Satanicpuppy Nov 10 '09 at 20:10
For fancier systems like ZFS, you actually do gain a substantial benefit from having bigger arrays rather than multiple smaller ones. Look at the end of this article for the reasoning. dtrace.org/blogs/ahl/2006/06/18/double-parity-raid-z – Paul McMillan Oct 26 '10 at 7:52
@paul mcmillan: They were running UFS, and the system was older so it didn't actually support RAID 10. That being the case, a bunch of small RAID 5 arrays was the optimal setup for redundancy/performance – Satanicpuppy Oct 26 '10 at 20:27
feedback
up vote 0 down vote

Did the RAID controller run into any errors on the rebuild or is it now reporting a healthy volume? By disk check do you mean fsck? Is the error on the file system or is the RAID Volume disappearing to the OS and triggering IO errors?

As we all know, or find out the hard way, RAID is not backup and in most cases like this the fastest way to get a working system back up is to restore the data from backups.

link|improve this answer
the rebuild (as far as i could tell) didnt happen it just attempted to boot debian and then while booting it fsck was forced and gets to 57.6% every time. – Aceth Nov 11 '09 at 16:48
If you go into the raid bios what does it say the status is on your volume? Another option would be to boot up on a live cd and install the tw_cli tool from 3ware's support site, or run it off a thumbdrive. Then use that to look at the status of the raid volume. Do you have good backups? Because it seems to me that you should be able to restore your files from a backup in just a few hours since you're only dealing with 320GB of data, if it's 100% full. – 3dinfluence Nov 11 '09 at 17:30
feedback
up vote 0 down vote

do you know which disk failed first? If you do, you can try to recreate the array, and pull the first failing disk offline, so you will get a working, degraded raid array. enough to get the data off of it

link|improve this answer
yeah i replaced it with another hdd but then when i turn the machine back online it says that the 2 remaining drivers are not installed or something similar (cant remember off top of my head) – Aceth Nov 11 '09 at 16:49
feedback
up vote 0 down vote

Boot from a bootable CD, copy out any critical or recently-modified data, replace all three disks, and recover the rest from a backup. It's not worth trying to solve problems like this, and there's always a risk that some corrupted data will sneak through and be ready to bite you in the future. This is why you make backups.

You can attempt to requalify the three drives once this machine is back up and running if you want to.

link|improve this answer
feedback

Your Answer

 
or
required, but never shown
discard

Not the answer you're looking for? Browse other questions tagged or ask your own question.