9

I've read the story about the manager taking out a disk from a RAID 5 array, and then a second one, but I’d just like to try out for myself what happens when I simply disconnect a disk from a live system. It’s an HP ProLiant DL585 G7 series server, so it must be hot-swappable.

But before I just go for it I thought it might be better getting some input from more experienced folks before doing anything really, really silly.

3
  • Wow, thanks all for the quick answers.. I'm going for it ^-^
    – Berzemus
    Jun 25, 2009 at 19:46
  • Been there, done that ! It really is liberating, I'm glad I did it, thanks all. (Now I just have to find someone to show it to..)
    – Berzemus
    Jun 25, 2009 at 19:58
  • 2
    Just make sure that the array has completely re-mirrored itself before you show someone (or make sure that you pull the same drive you did this time). It is nice to confirm that the kits works and protects you the way it claims to, isn't it?! Jun 25, 2009 at 20:30

7 Answers 7

22

It depends on your controller.

If it supports hot-swap, then yes. If not, then you might blow the controller and kill the whole array.

If you do take a drive out of the array (either while running or powered off) you will have a full rebuild to do once you put it back in which will take a while and degrade performance while it happens.

Testing your RAID setup like this is not a bad idea. Just make sure your backups are correct and up-to-date first just in case something goes wrong and the array doesn't survive the test.

3
  • 1
    +1 for the comment about having an up to date backup. ;-)
    – KPWINC
    Jun 25, 2009 at 21:10
  • 2
    And make sure you can restore from the backup as well.
    – jay_dubya
    Jun 26, 2009 at 5:19
  • Backups are to be considered invalid until they have successfully restored once to a test machine. And even then they should all be viewed with much suspicion.
    – Mark Allen
    Mar 31, 2010 at 23:27
8

Well, according to a simple search that server should have "Hot-plug 2.5" SAS" drives, so if that's the case, yes you should be able to pull drives out and put drives in while the machine is live. However, you need to double check your actual setup.

This is something that you should do not only to impress your boss, but more importantly, to demonstrate to yourself how it works. You'll want to pull a drive to make sure that the system stays alive, and to make sure that your hot spare kicks in immediately. It's also a good idea to watch the rebuild process and notice just how long it takes. Replace the "missing" drive with a different drive, and check that you can create and assign it as a hot spare. Then pull the other drive that was part of the original array and see if the machine stays alive and the new hot spare kicks in.

Another fun test is to remove the drives with the power off, and put them back in in a different order, then fire up the machine and see what the controller says.

It's important to do this stuff before the machine is in production so that you a) know that it actually works, and b) can write up a procedure. You might be surprised at just how often a particular RAID setup just doesn't work as expected.

Since you're new at this, you might also see what a pulled drive looks like if you plug it in to some other machine and try to get at the data. You might also experiment with different sized drives to get a handle on how the system deals with them.

Call me paranoid, but ideally you might also consider buying a few extra replicates of whatever drives you're going to use, and test each one before the system is in production. I've had lots of stupid issues with minor drive size differences between manufacturers, defective drives that took out entire SCSI channels when inserted, etc. Life is good if you've got a few spare drives that are identical to the drives you're using.

2
  • +1 for re-arranging the disks. Some controllers have a heart attack at this, but the better ones will just re-shuffle their order in its setup. Jun 25, 2009 at 21:41
  • 1
    I wish I could +2 this actually, for your 2nd comment about buying identical disks. I had a disk that was a whole 1.2mb smaller than the disk it was replacing (they were 500gb disks) and as a result it was useless. Jun 25, 2009 at 21:42
4

Yes, if they are hot swappable you can just pull one of the disk and things should keep working.That is how you would replace a failed disk.

1

Depending on the controller and/or software, it should work, and in many cases, the redundancy would be begin to recreated a few moments after putting the disk in.

I would never do it on a production system, because it's a heavy load on the disks, it may take hours to mirror the disks again (during this time you have no redundancy) and and and, but in theory it's supposed to work.

In testing systems, I always do it before going live, and it always has restored...

1

We got a few old dell servers from clearing out a aquired data center. One of them happen to have a raid5 set up, at the time we didn't have this in our office yet. So I did the only logical thing.. Booted it up, started a few processes, and started pulling hard drives out for science. 1 was fine, 2 was fine, 3 not so much. So the short answer is yes. It is liberating!

2
  • If you got two disks out before it failed, that sounds like RAID-6 to me. But yes, the first thing I did when I got my hands on my first 14-disk array was pull one out! Jun 25, 2009 at 21:40
  • Very nice!!!!!!!!!
    – Kyle
    Jul 1, 2009 at 16:02
1

I pulled a drive out of a RAID5 array one time as a live proof of concept / hardware test while I was actually in the facility installing the machine. I had done this several times back at the office, which had always been fine, so I assumed it would be this time also. It went back in, but took two weeks worth of me fiddling around with it to be recognized again... so, if you're thinking "it'll be fine!", well, it should be yes. But listen to all cautionary tales.

0

Yeah, I've done it (purely by accident, I hasten to add).

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .