1

I am currently building a storage unit for our office. It is rather low budget at the moment, but it needs to be extendable.

basically we have a huge database that will grow over the next few months quite heavily. Therefore, ideally we would just like to throw hard discs at our new server.

We have not purchased the server yet, but going through some details. However, I would like to get an answer to a question first.

How easy is it to expand existing RAID systems?

We will start with two HDD 4TBs WD black. But after about 1 month we will need to add another 2 4TB disks. The server we are going to get has 12 bays.

Mirroring is important, However RAID 1 only works with 2 disks. Raid 10, would already allow us to mirror a RAID 0. And from what I have seen even the raid 10 can be installed with two disks. However, what happens after that ? Is there any recommendation to achieve a flexible RAID system ?

On the OS layer I would just like to build a LVM, that recognises once there is space added to the "disk" so that it can be expanded. But in fact, it lies on several disks which are managed by the RAID controller.

  • What database system are you using? Microsoft SQL allows you to add more data files as you add more logical and physical volumes. It will then slowly redistribute the data to balance it across all the volumes. – James Feb 8 '15 at 4:56
2

There are quite a few options with varying degrees of resilience, disk efficiency and ease of operation. Here are a few:

RAID 0 & 1 RAID 0 and 1 are immediately out of the question, RAID 0 offers no redundancy (in fact it increases risk) and RAID 1 is limited - as you mentioned - to the maximum size of 1 disk.

RAID 5 Is an option and you only lose 1 disk for parity; though this is bitter-sweet as the more disks you have the chances of finding an error on 2 (or more) disks rises (and you're screwed if that happens). Write speeds are often lacking. Minimum 3 disks to start with. Expanding is time consuming and has a high risk of total failure.

RAID 6 Is a more resilient option, same as RAID 5 except it uses 2 x disks for parity, thus you can lose 2 x disks to failure as still be able to rebuild. Write speeds are often lacking. Minimum 4 disks to start with. Expanding is very time consuming and has a lower-than-RAID5 risk of total failure, but not negligible risk.

RAID 10 Is the most resilient option of all the RAIDs and also the lease efficient as it uses half of all disks present for mirroring. One major benefit over RAID 5 and 6 is write speeds are often significantly improved with every disk you add (as opposed to declining in performance) - this can be essential depending on what type of database you're implementing. Minimum 4 disks to start with, adding 2 at a time after that. Expanding is probably the fastest with the minimum risk associated.

RAID 50/60 Is the middle ground between RAID 5/6 and RAID 10 - has better disk usage efficiency than RAID 10 but has the requirement of having lots of disks to start with (Minimum 6), also performs better than the basic RAIDs. Expansion is very time consuming, risk depends on how many disks are in the array but is somewhere between RAID 10 and 5/6 (weighted towards 5/6).

LVM I don't use this much, so I'll leave that avenue for someone else to comment on.

Filesystem-based RAID BTRFS and ZFS can both perform RAID 0,1,5 transparently across disks without the need for Linux RAID managment. Adding/removing/altering array sets are easy (though time consuming, as RAID is). ZFS has the benefit of being tried and tested for many years, whereas BTRFS is still an emerging filesystem.

Conclusion:

Linux RAID is somewhat more forgiving than hardware RAID, where RAID 5/6/50/60 are involved Linux RAID can make your life a bit easier if things go pear shaped (like losing 2 disks on a RAID 5 array, you can still assemble the array and try to recover whereas most HBAs will outright refuse). RAID 10 with hardware RAID is usually the safest bet both in regards to resiliency, I/O throughput and expanding times. So put it down to my top 2:

If I/O throughput is not a high priority: * Linux RAID 5, but routinely back your data up elsewhere to offset the risk. Expansion is as simple as a 1-line command, though it'll take a while to complete.

If I/O throughput is a priority: * Hardware RAID 10, the schedule for backing your data up can be relaxed somewhat. Expansion will depend on the hardware RAID type, but won't take as long to initialise.

  • This doesn't appear to answer the question. – Michael Hampton Feb 8 '15 at 2:55
  • Added comments that address the question more directly. – Litch Feb 8 '15 at 3:02
2

There are a couple of points I would like to stress: first, I would try to use a dedicated storage device if possible. Hardware built for an specific purpose tends to be more fit for it that generic servers. That being said, I understand that if the budget is low, there may not be a chance of getting a small disk array.

Therefore, if you need to use a normal server, you can usually get advanced hardware RAID controllers that allow for logical disk expansion. If you use one of them, you just need to re-scan the bus and expand the filesystem once the controller has finished to rebuild the array (provided there was no partition table). If This is a non-disruptive operation even though a backup strategy is strongly recommended. Please, do not forget that the array rebuild will impact the performance. However, even though this is possible, I would not recommend depending solely on the server controller. I will elaborate on this later on.

Second, it seems we are looking at one capacity factor here, the disk space. How about performance? Using 4TB drives in a RAID5 or RAID6 configuration is going to give you extremely poor random/write access. In the range of 10 or 20 IOPS/TB. If your database is not frequently accessed that would be ok but if you have a need on that sense, this may be a problem.

Taking both points into consideration, my advice would be:

  • Separate volumes (logical-disks) according to performance needs
  • Use a advanced disk array controller that offers multiple arrays in RAID5, RAID6 mode; battery backed cache and spare disks.
  • For data sets with high performance requirements I would use a large ratio of spindles per TB and fast drives (SAS?) and a RAID configuration with small RAID penalty (RAID10)
  • For data sets with no performance requirements I would use low spindles per TB ratio. Capacity drives would be ideal here (2, 3, 4TB NLSAS) in a RAID5 or RAID6 configuration. I would focus on creating the complete arrays from day 0.
  • For expanding your logical drives, I would create new arrays at the controller level and them manage them using LVM.

As an example. If you need, 300 GB of performing storage and 2TB of capacity storage, I would create two logical disks using the array controller:

  • Performance: RAID1, 2x600GB SAS 10k disks
  • Capacity: RAID5, 3+1 NLSAS 2TB disks

Both managed via LVM (separate Volume Groups)

If I need more "Capacity" disk space, I would acquire 4 more NLSAS disks and create a new 3+1 array and its capacity added to the Capacity Volume group. If you get out of disk bays, some vendors have JBOD shelfs you can connect to the same disk controller and they will be managed by it.

The exact RAID5 and RAID6 widths will be determined by your needs and your controller capabilities. Please, have a look at the manual to see the vendor recommendations in that sense. Frequently recommended configurations are RAID5 in 3+1 and RAID6 in 6+2.

Last but not least, please, monitor your disk failures and have a look at your spare disk count.

I hope this helps!

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.