Take the 2-minute tour ×
Server Fault is a question and answer site for professional system and network administrators. It's 100% free, no registration required.

But I don't believe it.

The machine is a Dell PowerEdge 2600 server running Windows Server 2008 trial 32bit (yah, its not supposed to...but it works! [well, it used to]).

For the sake of confusion: the drives are numbered 0, 1 and 2.

I was coding away as usual when I noticed the Dell logo on the front of the case was orange. So I opened the case door and saw that the HD vents were completely covered with dust (I know its not related to the orange light...but I hate dust). Since the drives are hot-swappable I yanked drive 2 out and cleaned off the dust and put it back in. I then yanked drive 1 out and cleaned the dust off that one and put it back in. Someone asked be to help set up a printer on their machine so I got up and 20 minutes later came back to see 'No boot device available - strike F1 to retry boot, F2 for setup utility` displayed on the server's monitor. I look down at the drives and drives 1 and 2 have orange lights instead of the green ones!

Since then here is what I have tried:

  • Installed drives into a Dell PowerEdge 2500. Drives were detected fine. Got a message stating Missing operating system.
  • Reset the bios on the original PowerEdge 2600 (pulled the bios battery out). All drives appear fine. Get the Missing operating system message when booting. A drive lights are green.
  • Booted Ubuntu from a CD to inspect the drives. 2 of the drives are displayed in Computer. Since the data is striped the files/folders in the drives are gibberish.
  • Booted Ubuntu and opened Terminal and executed sudo fdisk -l which listed the 3 drives. On the 3rd drive listed it states Disk identifier: 0x00000000 Disk /dev/sdb doesn't contain a valid partition table

Do you think the drives ARE actually toast?
Could it be SCSI or other hardware failure?
Could it be incorrect System Settings? Is there any way to create a virtual RAID in Ubuntu on the 2 drives that are "valid" so I can copy the data to a network share?
Should I try reinstalling the Windows Server OS(eek!)? Do you have any suggestions that I can try?


UPDATE

After doing lots of googling I came across Raid Reconstructor. I tried this program using my Dell PowerEdge 2600 using a bootable windows XP CD but it did not work (no drives detected). I then installed two of the drives into the PowerEdge 2500 alongside the 2500's existing single-drive RAID 0 running Microsoft Server 2003. I then installed and activated Raid Reconstructor which created a virtual image of the RAID-5 array, opened the image with Captain Nemo, and backed up my C:/Websites directory to another computer...with ALL files intact (so far)!!!

I will hopefully be able to restore the drives 100%

Lessons learned:

  • I don't care if the server can "hot-swap" drives. DON'T FREAKING DO IT!
  • Back up your data, dummy!

Thanks for all your help, answers, and comments (and for being wrong about the data loss. haha)!

share|improve this question
10  
Of course you can hot-swap drives with no problems, assuming you do it right. You can't just yank a drive, put it back, and immediately yank another drive. Arrays need to rebuild after a failure, and removing a drive is treated as a failure. Just because you didn't know this doesn't mean that there is a problem with hot-swapping drives. –  MDMarra Jun 4 '10 at 22:49
    
I've spoken to others who had bad experiences with hot-swapping drives in Dell x9xx machines. On Dell 2950 and 1950 machines, we've had a number of problems when hot-swapping a drive (both SAS and SATA drives). It should work, but I've seen some pretty bad failures, but for us most of the failures were recoverable. –  Stefan Lasiewski Jun 4 '10 at 22:57
4  
@Stefan: The issue isn't with the hot swapping at all. The problem is when drive 2 was pulled, the array would have gone to Degraded state, then when drive 1 was pulled, the array would have gone Failed, and as Chopper3 aptly puts it - is toast. –  Ben Pilbrow Jun 4 '10 at 23:02
    
@Stefan - We are almost exclusively a Dell shop and have dozens of 19xx and 29xx series servers in production and haven't had a single issue with hot-swap. We also keep the RAID controller firmware up to date when the release notes indicate a worthwhile fix. But many failed drives have been hot-swapped by me personally in those systems with no ill effects. –  MDMarra Jun 4 '10 at 23:08
3  
A better clarification of, "I don't care if the server can "hot-swap" drives. DON'T FREAKING DO IT!" would be to make sure you thoroughly understand RAID, what it means, what the different RAID levels are, and how each RAID level handles disk loss/removal. If this has been a RAID10 array, you probably would have been fine (assuming you didn't pull two disks from the same mirrored pair). It all comes down to understanding your system before you touch things like that. Hot Swap was never the problem here. Ignorance was. –  Christopher Cashell Jun 23 '10 at 16:21
show 2 more comments

3 Answers

up vote 2 down vote accepted

After retrieving my data with Raid Reconstructor I went to reconfigure my raid and reinstall the OS.

When I got the OS install prompt I decided one last time to try to repair the OS boot files manually from CMD prompt....

It Worked.

Computer is back up and running (limping). I still need to do a full repair install since some system files are being reported as corrupt.

share|improve this answer
    
Great news, lucky though eh ;), thanks for introducing us to those recovery tools though –  Chopper3 Jun 18 '10 at 18:22
    
I would not trust that file system at all. It's likely corrupt in many subtle ways. –  Chris S Sep 23 '11 at 19:46
add comment

a lot of times its the backplane or scsi controller thats bad, if it was a backplane issue then in my case 9 times out of 10 it was a firmware issue.

on the 2500 did you have it rebuild the array or just put the drives in and it found the array?

edit:
should have read your question better! Chopper3 is right.

share|improve this answer
add comment

Biggest 'Doh!' of the week I reckon - sorry dude.

The drives themselves won't be physically broken, this is simply that you've killed the array by removing a second disk before the first one had rebuilt - I'm >90% sure your array is toast. Basically you shouldn't have removed them at all while live, if you absolutely had to you should have waited for the array to rebuild before doing the second disk.

It's reinstall/restore time I'm afraid - your data is gone.

share|improve this answer
    
Sounds like this is exactly what happened. –  egorgry Jun 4 '10 at 14:20
    
Yah. Thats what I was afraid of too. –  David Murdoch Jun 4 '10 at 14:26
1  
Never, ever pull a working drive in a live server unless you absolutely have no choice in the matter (like someone has a gun to your head). Shutdown the server (or heck, yank the power cords) before ever messing with working hardware. –  Chris S Jun 4 '10 at 14:26
    
Hope you didn't have anything planned for the weekend :( - I think we've all found out the hard way that you should never remove a disk just for the hell of it - I did just the same kind of thing back when you were in kindergarden ;) –  Chopper3 Jun 4 '10 at 14:27
    
I still get nervous swapping drives on our netapp with raid-dp. :) Funny little story I once worked with a gentleman that tried to hot swap a scsi cable on a HPUX system, I think a K580? That didn't work out in his favor. –  egorgry Jun 4 '10 at 14:35
show 7 more comments

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.