Take the 2-minute tour ×
Server Fault is a question and answer site for professional system and network administrators. It's 100% free, no registration required.

I have created 3 RAID0 arrays using mdadm on 3 Rackspace CloudServers using Rackspace's CloudBlockStorage as their devices. Each array has their own set of CBS volumes. RS-CBS has an upper limit of 1TB for their drives and I needed larger storage. One of the servers is for data storage, the second is a live mirror of that server, and the third is a nightly snapshot. I created the array on each of the servers using this command:

mdadm --create /dev/md0 --level=0 --raid-devices=3 /dev/xvdb1 /dev/xvdd1 /dev/xvde1

When it came time to grow the array, and since this was my first time growing one, I started on the least critical server: the snapshot server. I added a new volume and, with a little fiddling, came up with this command:

mdadm --grow /dev/md0 --raid-devices=4 --add /dev/xvdf1

According to the mdadm man page it said you are not able to grow a RAID0 so mdadm converts it to RAID4, grows it, and then converts back. (It never converts back, but that is another issue). The array reshapes, but the size never changes. My assumption is that the new drive just became the parity drive for the array. So to get the space I need, I added another volume:

mdadm --grow /dev/md0 --raid-devices=5 --add /dev/xvdg1

This reshapes and gives me the extra 500GB I was hoping for so I do the resize2fs /dev/md0 and am good to go; 2TB of space.

So this is where the weirdness happens. I switch to grow the array on the mirror server. Knowing from the previous steps that I will need to add two volumes to get the extra space I need, I use this command:

mdadm --grow /dev/md0 --raid-devices=5 --add /dev/xvdf1 /dev/xvdg1

It reshapes and things look fine, but when I do the resize2fs /dev/md0 followed by a df -hal I see that it did grow and is reporting 2.5TB of space, not the 2.0TB I was expecting. It's as if it didn't create the parity drive I assumed it did from the first run and used all available space for the array.

One thing that looked different was the output of mdadm -D /dev/md0 after the reshape on the mirror server. I saw this for the RAID devices:

    Number   Major   Minor   RaidDevice State
   0     202       17        0      active sync   /dev/xvdb1
   1     202       49        1      active sync   /dev/xvdd1
   2     202       65        2      active sync   /dev/xvde1
   5     202       97        3      active sync   /dev/xvdg1
   4     202       81        4      active sync   /dev/xvdf1
   5       0        0        5      removed

The output of cat /proc/mdstat shows that it did convert to RAID4:

    Personalities : [raid0] [raid6] [raid5] [raid4]
md0 : active raid4 xvdg1[5] xvdf1[4] xvdd1[1] xvdb1[0] xvde1[2]
      2621432320 blocks super 1.2 level 4, 512k chunk, algorithm 5 [6/5] [UUUUU_]

unused devices: <none>

So my concerns are ... did I grow the first array (snapshot server) incorrectly and it should've gotten 2.5TB, but my error only gave it 2.0TB? Or did I grow the second array (mirror server) incorrectly and don't have the parity drive that I thought I was going to get? What can I do, or what should have I done, to grow these arrays consistently?

share|improve this question
    
really interesting question, I always have a problem in growing RAID 0 on mdadm. I use EBS volumes on EC2 and every time I use all the commands on the internet and end up removing everything and build the array with new volumes all over again and sync with other replica sets. Really painful, hope someone gives us an straight forward way to grow RAID 0. –  Maziyar Oct 10 '13 at 1:19
add comment

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Browse other questions tagged or ask your own question.