6

I have a server with three 500GB drives, with most of my data in a RAID5 configuration spanning the three of them.

I just purchased and installed four 1TB drives, and the intention is to move off of the old drives and onto the new ones. I have enough SATA ports and power connectors to power all seven of my drives at once, so I've kept the old RAID running while I figure out what to do with the new drives.

My question is: Should I create a whole new array on the 1TB drives, then move everything over and reconfigure linux to boot from the new md arrays? Or should I just expand the array, swapping out each of the three 500GBs with the 1TB, then adding the final drive?

I've read up on the mdadm extending drive setup, and it makes sense, but I imagine I would use one of the drives as a full backup while I move things over, then add that drive back into the array once things are up and running on three of the 1TB drives, so there's some complication in going that route as well... I'm just not sure which is safer/recommended.

4 Answers 4

5

Create a whole new array with the new drives, it's not worth the risk or hassle to mess with old drives. Consider carefully the type of raid you are setting up. RAID5 on drives that large leads to very long rebuild times during which you are vulnerable to a second drive failure and complete data loss. There are strategies to reduce this risk, such as using RAID6 where you can survive losing two of the drives.

Also consider running LVM on top of your new raid array as this will give you options for growing or migrating filesystems in the future.

5
  • My primary concern with a RAID6 configuration is the amount of time it would take for me to need to expand once again. Going from the RAID5 of three drives to the RAID6 of four would push me up to 2TB, rather than 3TB. Granted, I don't have a lot of experience with RAID setups, so I don't know how common a drive failing on rebuild might be, but having to repeat this process more often may be a bit on the expensive side for a home media server.
    – hellfur
    Jan 17, 2011 at 4:54
  • I should add that as a home server, I do have the ability to shut down access to writes on the drive while I'm re-building an array... I'm not sure if that impacts a recommendation for RAID5 or RAID6, but with less I/O, I imagine the loss of a drive during rebuild is minimized. Again, I'm not terribly experiences with RAID configurations, so I'm not sure how true that statement is.
    – hellfur
    Jan 17, 2011 at 5:12
  • No, raid5 rebuild time is mostly determined by drive size, not load. Jan 17, 2011 at 5:22
  • Sorry, I should clarify. I am aware that the time rebuilding would be the same, but are the odds of a drive failing not lower with a smaller amount of I/O? ... Also: With 4 drives, if the consideration is to go to RAID6, wouldn't it make more sense to do a RAID 10? That way even if two drives failed, I'd have a 66% chance that my data would still be okay, since it would depend on which drives failed.
    – hellfur
    Jan 17, 2011 at 5:26
  • RAID10 will give you better performance, but at the expense of 'wasting' more space. Your call on that. raid10 is certainly simpler. Jan 17, 2011 at 5:53
3

If your goal is to convert a 3-device mdadm RAID5 array into a 4-device array, I believe your only choice right now is your first option: Create a second array with the four 1TB drives, then copy all your data over from the first array.

As far as I know, the mdadm "grow" mode does not currently support adding more devices to an existing RAID5, so, unfortunately, you can't use your second option. Or rather, if you do go with the second option, you will end up with a 3-device RAID5 on 1TB drives, with a 1TB drive left over as a spare. (I believe you could configure that extra drive as a hot spare.)

2
  • According to this, it is possible to add disks to the array and then increase the number of devices in said array, but I could be reading it wrong. raid.wiki.kernel.org/index.php/Growing
    – hellfur
    Jan 17, 2011 at 3:02
  • @hellfur: According to your reference, it should indeed be possible to increase the number of devices in a RAID5 with a --grow operation. This is a relatively recent improvement to mdadm. I guess I should a prefaced my answer with the caveat that I hadn't worked with mdadm RAID5 in quite a long time. Jan 17, 2011 at 3:53
2

Changing an existing RAID is always asking for trouble, also how old are your 3 500GB drives? If they are more than 3 years old it would be a good idea to remove them from production anyways as they are getting to the end of there lives and you might have more issues with them than you want anyways, re-purpose them in to a low priority dive somewhere else.

1
  • The intention is the remove the 3 drives entirely. The primary issue is getting the data onto the new drives in a relatively safe way.
    – hellfur
    Jan 17, 2011 at 4:47
1

While there are options to move the data over without creating a new array, there is not much point in doing so, the file systems will be offline during the migration either way.

In agreement with Phil Hollenback, I think you should move to a RAID6 setup with LVM (the only reason I post this as an answer is that I may not yet comment on this site), but I think he understressed how RAID5 is a bad idea.

As the drives in your set are likely going to be somewhat similar (although ideally from different production batches), I would expect them to fail close to each other. With a RAID5 with one failed drive, you have no redundancy left if a single drive fails, and the first thing you do at this point is put all your drives through the extra stress of a rebuild.

Using LVM allows you to keep your downtime small in case you ever want or need to migrate again. I think a logical "next step" for your server, in case you do not want to use the opportunity now, would be a dedicated RAID controller card, which would then appear as a single device to the kernel, so you would need to move the data again at that point, and LVM can do that while the file systems are mounted (with some caveats for the root filesystem).

If you have the budget for it, you might as well take the plunge and invest in a hardware RAID solution; sensible ones should start in the $500 price range and include RAID6 support and a battery back up unit. Be aware that the products from a pretty large company in this segment do not allow shrinking units, so you either stay clear of them, or lose some flexibility compared to Linux software RAID, but the additional performance of a battery-buffered write cache is well worth it, especially if you use a journalling file system.

2
  • Thankfully, I do have the entire system on a UPS that has a far greater capacity than I need. I got it cheaply from a friend after he decided to move to a simpler solution (it's rack-mounted). At the moment, I don't really have the budget for a dedicated RAID controller, so the plan is definitely to stick with mdadm for the foreseeable future... As for LVM, I don't really have any familiarity with it. Would you be able to point me to some resources where I could get up to speed on what it does and how it's used properly? Thanks!!
    – hellfur
    Jan 18, 2011 at 7:51
  • The UPS does not give you the same benefit that a battery back up in the RAID controller does -- your system will still have to write the journal to disk, then perform the metadata operation, then update the journal. A battery back up in the RAID controller allows delaying the journal update to idle time, because even if the host system is turned off or rebooted, the write request for the journal is still somewhat safe in the RAID controller. This is a significant speed boost, as you save four head movements per metadata update. Jan 27, 2011 at 13:05

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .