I'm looking to replace my hard-drives with bigger-ones and I would like to know what are the best practices in doing this. I have a HP ProLiant with its 256MB version of RAID controller:

03:00.0 RAID bus controller: Hewlett-Packard Company Smart Array G6 controllers (rev 01)

I also have a pair of 10.000 rpm 146GB SAS drives in RAID 1. The OS is Ubuntu 10.04, 64-bit. I'm not using LVM.

zduric@pbksrv002:~$ df -h
Filesystem            Size  Used Avail Use% Mounted on
/dev/cciss/c0d0p1     130G  119G  9.7G  93% /
none                  3.0G  192K  3.0G   1% /dev
none                  3.0G     0  3.0G   0% /dev/shm
none                  3.0G   64K  3.0G   1% /var/run
none                  3.0G     0  3.0G   0% /var/lock
none                  3.0G     0  3.0G   0% /lib/init/rw
zduric@pbksrv002:~$ mount
/dev/cciss/c0d0p1 on / type ext4 (rw,errors=remount-ro)

We experienced a huge growth in content stored there and now I need to replace this array with another pair, 300GB disks. Ubuntu 10.04 is up there with a tomcat and a document management application (which is filling up most of the drive). I'm looking for the best way to do it.

I was thinking that I could do one of the following:

Option I: replacing a drive at a time. Basically, I'd do something like this:

  1. Replace drive 1 with the new drive.
  2. Rebuild the array to put the data on the new drive.
  3. Replace drive 2 with the new drive.
  4. Rebuild the array.

I don't like the two-times rebuilding part about it. I also don't know, if that would work at all and even worse, how much downtime is that. (around 115GB data, OS included.)

Option II: Create a new array. The idea here is that I would add the new disks and create a second array. I could then migrate my tomcat application to the new array - this would mean a simple cp operation.

I like this option much better, because it would also leave me the first array to use solely for the OS and I could do OS upgrades more easily in the future.

Is the option II doable at all? Where could I find some good resources on doing this whole procedure?

Are there other, better, smarter options I should try?

Thanks in advance.

  • If it were me,I'd go for #2 – SpacemanSpiff Mar 19 '12 at 12:20
  • Can you provide the OS distribution detail information and the server model? I know it's a G6-era ProLiant on Ubuntu. I'm more curious if you have the HP management drivers installed. – ewwhite Mar 19 '12 at 12:33
  • @ewwhite how can I check the HP management drivers? I use the /dev/cciss thing, that's for sure. I'm on 64-bit Ubuntu 10.04, like I said.. What else can I provide? – Zlatko Mar 19 '12 at 13:19

You have options...

You specific case is covered on page 81 of the HP Smart Array Controllers User Guide.

Follow that with the information on page 20 of the HP Smart Array Reference Guide for an overview of what to do once the new capacity is available...

  • Do you have a battery-backed or flash-backed cache unit on your RAID controller?
  • Can you post the output of df -h?
  • Are you using LVM?

Expansion of a logical drive using an HP Smart Array RAID controller can be accomplished by replacing each disk with a larger drive, one at a time, allowing time to rebuild in between. That will result in a larger logical drive (in HP terms) that will provide unused space that can be allocated to the existing drive or carved into a new logical drive.

Doing the drive replacement will not require much downtime, other than a reboot to recognize the new array size. An online expansion (using the HP Array Configuration Utility) or an offline expansion (using the bootable SmartStart CD) is needed to expand the logical drive. And whatever needs to be done at the filesystem level to resize your partitions. I can do this in one or two reboots, depending on how the server OS is configured.

It may also make sense to install the HP management agents and drivers for your OS. Also updating the firmware of the server components may also be useful. There's a bootable firmware DVD available to accomplish that.

  • I guess this is what you would need: zduric@pbksrv002:~$ df -h Filesystem Size Used Avail Use% Mounted on /dev/cciss/c0d0p1 130G 119G 9.7G 93% / No LVM here. How can I check about which cache I have on my RAID controller? – Zlatko Mar 19 '12 at 13:20
  • All in all, I could go with the option II too, right? Add another array and just copy over the data? – Zlatko Mar 19 '12 at 13:35
  • Yes, you could. – ewwhite Mar 19 '12 at 13:44

If you just replace the drives one at a time you might have problems as the new array build will be the same size as the previous; the filesystem won't know about the expansion.

Then you would have to try resizing the partition and filesystem on the fly. Assuming you RAID controller wouldn't have a problem with this. And the filesystem. Some systems are better suited for these acrobatics than others (using LVM?)

What I personally would do is look at building another server and running it in parallel with the original, then when everything is in the clear, cut it over to the new production system.

If you couldn't do that, I'd schedule a time to shut down the server, create a brand-spankin' new array entirely from scratch, then restore from backup. This would also have the benefit of having the old drives available if there was a problem and you had to abort the rebuild.

Those two approaches would save the "taking chances" part, test your backups, and reduce the chances of having to cut over to "plan B" (or C or D) at three in the morning, when you're already cranky, tired, and more apt to make stupid mistakes.

Otherwise you're going to have to look at your filesystem and volume management software (LVM and filesystem resize utilities) and hope your RAID controller doesn't do something goofy to cause issues which might make you end up having to restore from backup anyway.

That's my take on it...perhaps someone with a model controller and server more like yours can give personal experiences. My approach tends to be more generic and I try to cut out the "just in case" edge problems.

  • It would be nice to have the extra server for this, but I don't. So I'll have to "create a brand-spankin'new array entirely from scratch" or similar. Thanks anyway. – Zlatko Mar 19 '12 at 13:27
  • 1
    It probably would be nice to have extra servers. I added this as a potential approach for people researching potential solutions to issues like yours, as this is supposed to be a site for more than just individual use cases. Otherwise your question could be deleted from the site once you have an accepted answer. – Bart Silverstrim Mar 19 '12 at 13:30
  • Yeah, you're probably right. Although, I'm in a specific situation: 1 server with 2 old and 2 new disks, so a generic answer to such situation should be devised, right? – Zlatko Mar 19 '12 at 13:32
  • The generic just means it should work regardless of goofy controllers or proprietary approaches (can't go wrong with "nuke and start over" if the backups are good). But there are ways that are clever or proprietary tools that may allow online resizing, as long as you have in mind that you might need to resize the filesystem or volume container afterwards. Every proprietary approach as a factor of "hope this doesn't wipe my data" on a production server if untested. – Bart Silverstrim Mar 19 '12 at 13:46

Option 1 is not really an option, since copying an existing RAID array's contents to a new disk does not increase the array size.

Unless the HP RAID controller offers online array resizing it's not even physically possible - and even then this only enlarges the array, not the partitions on them.

Go for option 2: add the drives to the controller, make a new RAID-1 array, create a new partition using the full size of the array, and move your content over.

There will be some downtime while you are copying the data, but you can minimize this by using rsync after the first full copy action, and while the applications are stopped.

  • The HP controllers allow online resizing. – ewwhite Mar 19 '12 at 13:04
  • Thanks, this is useful too. I just need a first full copy action (with the app down), after that I don't care. In theory, I could probably even test my backup system - pull the files down from rackspace and then just rsync the remainder... – Zlatko Mar 19 '12 at 13:24

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.