8

Is it possible to get linux server to share several hard drives as one storage pool through Samba? So that when I use the share I don't have to worry about saving to a certain drive that has free space, but rather I save to the pool that takes care of all that.

I guess I could do this through some kind of (software) raid, but a lot of my files don't really need to be raided. My aim is to have a setup where I have ~500GB raided and a few TB unraided space.

I currently have a light W2008 server and would like to move to linux and this would be really nice feature to have.

3 Answers 3

19

On Linux, you could use LVM to gather several hard drives (PV) into one Volume Group (VG) and partition it with the Logical Volumes (LV) you wish to share through samba.

See this link for more info.

LVM Schema

1
  • 1
    Great choice of picture to illustrate the concept.
    – andol
    Dec 18, 2009 at 10:00
7

I would second Raphink's suggestion of LVM (and upvote it, in fact) - this is pretty much exactly what LVM is designed for and it works well in my experience.

One thing to note is that using LVM over a bunch of drives is only a little safer than RAID0 - if one drive does down you may lose more than one drive worth of filesystems due to logical volumes straddling drive boundaries. So while "a lot of my files don't really need to be raided", make sure you have a good backup plan for those files that are not so easy to replace.

If you the storage server is likely to see a lot of activity, you might consider using RAID0 as well as LVM. This will give you a significant performance boost for many I/O patterns, and will not reduce total storage space available (as RAID0 offers no redundancy, so uses no space for mirrored data or parity blocks). Once you have tied some drives together in a RAID0 array, you can make the array an LVM PV just like any other drive/partition and use logical volumes to partition out the space as needed.

Of course with RAID0 you pretty much definitely lose everything if a drive dies, but if you have a backup plan that is sufficient for JBOD (which is what you are doing with just LVM, it is equivalent to what some RAID references call "linear mode") you have one that is sufficient for data on RAID0 (and anyone who says not isn't paranoid enough about the data they have over multiple drive using LVM!).

You can mix and match RAID levels on the same drives. On one of the machines under my control that runs as a VM host for development and testing, the drives in its array have some parts as RAID0 (for the VMs themselves) and some as RAID1 (for backups of the VMs). Each of the drives is split into ~100Gb partitions and each set or partitions can be a RAID array. The first two on each drive are currently RAID0 (linked as one logical volume by LVM), the last three RAID1 (again linked as one LV), and the ones left in the middle are free to be either when more space is needed. If you do this you of course need to be wary of I/O contention (it isn't a problem in my case as the two RAID arrays rarely see noticeable activity at the same time because the RAID1 set is only ever acces when making a new backup or restoring an old one) and excess head movements when both arrays are in active use (as they are at opposite ends of the drives), but you should be aware of those for just using LVM without any RAID anyway. This arrangement meant that I didn't have to accurately guess, at install time, what balance of how much high-speed (R0) space and high safety (R1) space the machine would need X months down the line without compromising and just going for a single RAID10 array.

7
  • "if one drive does down you may lose more than one drive worth of filesystems due to logical volumes straddling drive boundaries." Thanks for this nugget, it seems I'll have to revise my plan somewhat.
    – Illotus
    Dec 18, 2009 at 10:00
  • Note that this is the same problem that you'll have with a RAID0 system: if one drive goes down, you lose data. As suggested in David's answer, you can combine RAID and LVM. Combining a RAID0 system with LVM is redundant and useless, but combining a RAID1, 10, 4, 5 or 6 with LVM ensures your data won't be lost if you lose one or more hard drives. It will also cost you more in hardware, since you need more disk space that actual usable data space. The kind of RAID depends mostly on the kind usage you place to make of it. If you do not have a hardware RAID, you can use LVM on top of mdadm.
    – raphink
    Dec 18, 2009 at 10:34
  • 3
    Combining RAID0 with LVM is not useless - depending on your I/O load (say, many concurrent file operations over Gbit network) you could see quite a performance boost from the striping performed by RAID0. Even over a 100Mbit link you might see a difference for several concurrent and/or very non-sequential accesses. For less arduous I/O load the difference may be insignificant though. Dec 18, 2009 at 10:46
  • Combining RAID-0 with LVM isn't useless. LVM by default allocates space sequentially from the beginning. This can impact performance if one part of the LVM set gets all the I/O attention, focusing that activity on one disk. RAID-0 explicitly stripes data across all disks in the RAID-0 array. This can speed read and write activity. The difference is, if one disks in a LVM set fails, you will probably lose at least some data. If one disk in a RAID-0 set fails, you WILL lose ALL data. The RAID-5 idea posited by Raphink gives you at least the illusion of some protection against disk failure Dec 18, 2009 at 16:27
  • Thanks for these clarifications David & David, that's very interesting to consider. Do you consider that LVM on top of RAID0 is worth it even if you use a software RAID system such as mdadm, or only with a hardware RAID system?
    – raphink
    Dec 18, 2009 at 16:34
2

I would also recommend taking a look at ZFS which also contains drive pooling and RAID mirroring built in.

There are some issues with licensing making it a little awkward to implement but I have been running it on my home system with 3 x 1TB drives with no issues.

1
  • 2
    I'm a big ZFS fan but I wouldn't recommend running it under Linux as a FUSE file system in a production environment. For Linux you're better off waiting for BTRFS to mature. BTRFS has a better design than ZFS but I think it's still at least a year, perhaps a couple, from being production ready. Dec 18, 2009 at 18:14

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .