1

I have a Dell T300 with a Perc 6i, to which 3 500Gb drives are connected. The are assembled into a raid 5 ~1000Gb VD. This setup is working fine, however system is filling up and I would like to increase capacity.

My idea was something along the following lines:

  1. Add a new 2Tb drive as a hot spare to the array
  2. Remove one of the 500Gb drives, so the 2Tb drive will replace it
  3. Once reconstruction is finished, repeat the process until there only are 2Tb drives in the array
  4. Expand the array size to fill the 2Tb drives, making the VD about 4Tb.

However, looking in the docs, I cannot find any information about point 4. So is the perc 6i able to expand a raid5 array? How?

Or should I proceed completely differently to achieve my goal?

2 Answers 2

1

I realize this question is old now, but it was never actually answered, and I could have used a guide when I needed to do something similar. So, in the interest of answering the OP's question with facts rather than opinion, here is how I upgraded an existing 3-drive RAID5 array on a Dell PE R430 with a PERC H730P (aka MegaRAID SAS-3 3108 rev2) using perccli64 v7.1327 running on Linux kernel 4.15.

For reference, my topology:

  • Controller 0
  • Enclosure 32
  • Slots 1, 2, 3 (slot 0 holds an SSD)
  • Old HDD size 4TB (3.637TiB) x3
  • Vol 0 size 7.276TiB (before expansion)
  • New HDD size: 12TB (10.913TiB)

Please pay attention to the difference between Terabyte (TB or 10^12) and Tebibyte (TiB or 2^40) sizes. perccli64 displays sizes in Tebibytes, but uses "TB" as their suffix. I know.. I don't like it either.

Background: with a ~8TB RAID5 volume, we were running out of space. I calculated that a 20TiB volume was needed, so ordered three 12TB HGST SATA drives. The plan:

  • Swap out 1 drive, let the volume rebuild.
  • Repeat for drive 2
  • Repeat for drive 3
  • Expand the array to make use of the larger drives
  • Resize the existing filesystem(s) to use the larger volume
  • Do it all while the system is running and without access to iDRAC

On the PERCs, it is possible to just pull an active drive and pop a replacement in, but I was directing this through smarthands, and wanted to be crystal clear on which drive to pull, so I first brought a single drive offline, starting with slot 1, by running

perccli64 /c0/e32/s1 set offline

IMPORTANT NOTICE: this puts the RAID5 in a degraded state! I recommend looking at SMART statistics first, and if one drive is showing any hints of errors, replace that one first. If two or more drives show any prefailure indications, don't degrade the array without a DR plan in place and a full backup validated and within easy reach. In fact, a smart person would do that in any case. You can check SMART stats (after installing smartmontools) with:

smartctl -a /dev/sdb -d megaraid,n

(replace the final n for the slot you wish to examine).

Once offline, to make it easy for smarthands to identify the drive, flash the locator LED with

perccli64 /c0/e32/s1 start locate

Then, whomever is at the server can pull the blinking drive and put a new one in. Running:

perccli64 /c0/e32/s1 show

should show the drive first with a state of "Ofln" (Ofline), then unavailable while none are present, then a state of "Rbld" (Rebuilding) after the new drive is inserted, and finally "Onln" (Online) when rebuild is complete.

perccli64 /c0/e32/s1 show rebuild

will show a progress percentage during the rebuild (don't trust the time estimates), and

perccli64 /c0/v0 show

will show the volume as "Dgrd" (Degraded) while rebuilding, then "Optl" (Optimal) once complete. My drives are SATA, and the system load is moderately heavy, but they still only took about 15 hours each to rebuild the 4TB portion of the volume on the new drives.

Wait for the array to become optimal, then repeat the same steps for the remaining drives, in my example, slots /c0/e32/s2 and /c0/e32/s3.

Once all drives are replaced, your topology should show an optimal array volume made of larger HDDs but still with the original smaller array size, in my case 7.276TiB (shown incorrectly as TB, not TiB). The next step is to expand the array itself, but you will need to do some math first...

The perccli64 expand command is pretty poorly documented. The only help given is:

SYNTAX: perccli /cx/vx expand Size=<xx> [expandarray]

  DESCRIPTION: Expands VD to the specified size in the array

  OPTIONS:
    expandarray - expand array if possible to accommodate the given size
    Size - If the size format is not mentioned, given size will be processed in MB

What is not clear from the help output is that the Size parameter, which is required, is the size to ADD to the array, not the desired size after expansion. We want to give it the size of the new array in MiB, less the size of the old array in MiB. In my case, 12TB (actually 12,000,138,625,024 bytes, as shown by smartctl) less 4TB (actually 4,000,781,611,008) gives 7,999,357,014,016. A RAID5 of three disks is double that at 15,998,714,028,032 bytes, and divided by 2^20 is 15,257,562 MiB. If you specify too much, it will fail with the message saying there is insufficient space. To avoid that, knock off a few gigs, since it rounds up to the nearest one percent (!!!) of the total space anyway... Like I said, the expand command is poorly documented, which is why I'm sharing what I've learned here.

The "expandarray" option appears to be needed only when the base disk size has changed, as it has here, as attempting to run it without that option returned an error showing insufficient space, but with an additional note to "expand array to get the additional space." Finally, a useful diagnostic message! So I ran it as

perccli64 /c0/v0 expand Size=15250000 expandarray

which quickly returned success as it kicked off a background initialization (BGI) task. The BGI means the new space is available immediately, and you can view the initialization progress with

perccli64 /c0/v0 show bgi

That output is horrendously bad in estimating time remaining, as it shows a percentage that starts with what portion was already existing as already complete. I think it takes the first portion as having initialized quickly, and includes that when estimating the time remaining for the part that actually does need to be written to disk. Where 4TB took 15 hours before, and this is another 8TB per disk, I expected it to take about 30 hours, but it could take much longer since it's running at a lower background priority. I don't know how long it actually ended up taking in my case.

While the controller magically makes the space available immediately by running the iitialization task in the background, we still need to get the OS to recognize the new volume size. Start with a rescan of the scsi device by echoing '1' to /sys/class/block/sdb/device/rescan (specifying correct device name if not sdb), after which it shows the new larger size when running

blockdev --getsize64 /dev/sdb

That completes the hard part. The remaining steps use standard Linux commands. There are plenty of other tutorials on these steps, and each system is different, so I won't go into minute details. This is what worked for me. I used parted to fix the GPT (it offers the first time it tries to print the table), then add a new partition starting after the last sector in use and ending at the end of the expanded disk (parted command "mkpart pv2 15625877504s 46873438207s"). Again, you'll have to do some math to come up with the correct sector values. Then I put the new partition to use with

pvcreate /dev/sdb5

vgextend volgrp /dev/sdb5

lvextend -l +95%FREE /dev/volgrp/logvol

resize2fs /dev/volgrp/logvol

After this, the df output showed about 14TB of additional space, and my capacity problem was solved. Note that I reserved 5% of the free space in case I needed to expand one of the other LVs too.

0

RAID5 is dead. Don't use it if you like your data, especially not with 2TB drives. Also, while your method might work (I don't know about the Perc6i and your point 4 either), it will take an awful lot of time and during this time your drives will be constantly active.

Depending on your production requirements, I would recommend to put your data away on a temp drive, remove the old disks, create a new RAID (6 or 10) on the new disks and put it back there from the temp drive.

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .