Take the 2-minute tour ×
Server Fault is a question and answer site for professional system and network administrators. It's 100% free, no registration required.

I run several Linux machines at home and at work. Most of my computing is I/O-limited, e.g., large compiler regression suites and the like. At home I also have one machine that captures high-definition video from a terrestrial antenna. One sporting event goes into a single file of about 30GB; I have over 4TB of storage devoted to HD video.

My question is under what circumstances should I stripe? My current setup for my home directory and root filesystem is to stripe XFS over two drives using LVM. Sample output from lvdisplay -m:

Logical extent 0 to 117247:
  Type                striped
  Stripes             2
  Stripe size         64 KB
  Stripe 0:
    Physical volume   /dev/sdb2
    Physical extents  0 to 58623
  Stripe 1:
    Physical volume   /dev/sda2
    Physical extents  0 to 58623

However, I've just read articles with titles like RAID: Not Such a Clever Idea for Your Home PC , Are two drives better than one?, and a piece from storagereview.com—all of which say that striping with RAID-0 is a waste of money and effort for a single-user desktop machine. But

  • These studies use Windows, not Linux

  • They use hardware RAID, not LVM

  • They're not dealing with 30GB video files

I'd like other people's experience on the question for what sorts of configurations, if any, does LVM striping across two disks improve the single-user Linux desktop experience?

share|improve this question
    
+1 for the links to the articles. I didn't realize that RAID 0 didn't give a good performance boost in a desktop setting, but the articles make a good case that it doesn't. –  Adam Brand Jul 26 '09 at 17:41
add comment

9 Answers

up vote 3 down vote accepted

My question is under what circumstances should I stripe?

If you're talking about a "mirror" stripe, you would use it to migrate data from one drive to another. You would not use it as a form of RAID-1, as it is no where as efficient as the regular md driver.

If you're talking about a stripe that extends drive space, well, that's what it's used for: when you run out of space.

Otherwise, I wouldn't bother with the LVM stripe options.

I'd like other people's experience on the question for what sorts of configurations, if any, does LVM striping across two disks improve the single-user Linux desktop experience?

Well, I've turned on LVM mirroring for a busy server, with two drives in a eSATA enclosure. The behavior I observed was that it would "burst" the mirroring writes to the other drive, and often lagged behind the primary drive by a second or two. This made me very nervous, as there could be potential consistency issues during a primary drive failure. The performance was also abysmal under heavy write loads, and read performance was not improved because it did not use the other drive for reading. I scrapped that setup and switched it to a RAID-1 via the md driver and the speed increase was immediately visible, with no "bursty" behavior.

Some recommendations:

  • If you have four drives of the same size, and all of your data would fit onto two drives, I would easily recommend a RAID-10 over a RAID-5 any day. Yes, you lose 2 drives, but you gain slightly more redundancy than RAID-5 and your access times "even out".
  • If you have just two drives and they can accommodate all of your data, I would recommend a mirror (RAID-1).
  • If you have drives of very different size and are trying to economize, you could get away with using LVM to extend the volume onto the second drive, provided you're aware of the risks involved. A single drive failure means you would lose whatever you had.

However, I've just read articles with titles like RAID: Not Such a Clever Idea for Your Home PC, Are two drives better than one?, and a piece from storagereview.com — all of which say that striping with RAID-0 is a waste of money and effort for a single-user desktop machine.

RAID-0 striping exists for a reason: you're out of storage capacity and you temporarily need to increase it. The key word here is "temporarily", the idea being that you'll soon remedy the problem with a permanent fix.

If you're talking about some gaming box that you can wipe and reinstall, then yeah, it's a waste of time.

If you're talking about a box that handles your livelihood, and the only way to make things work is to extend the volume onto another drive, then that's a different story. You need to do what you need to do.

share|improve this answer
    
I think you're confusing concatenating drives with RAID-0 striping. –  pgs Jul 27 '09 at 10:52
    
It was late writing this. I suspect it sounds confusing. True, RAID-0 stripes and LVM stripes are not the same thing - RAID-0 can interleave the blocks, but LVM just "tacks on storage". –  Avery Payne Jul 27 '09 at 15:31
add comment

Considering that HD broadcast uses a max bit rate of 19.4Mbit/sec. That's less than 2.5MB/s. You could record 4-5 HD broadcasts at once and a single hard drive would be able to keep up. So I would say that the risks outweigh any benefits in this case.

The only real situations that you may notice a difference is if you're copying these 30GB files to another system that also had a raid of some sort over Gigabit ethernet or possibly editing these videos. But my guess is that we are talking about a mythtv box in this case which is probably flagging commercials as well. So the commercial flagging may be slightly faster but mythtv does this as a background process once the stream has been captured. I don't see any improvements there from an end user point of view though.

share|improve this answer
    
I've never been able to get mythtv to work. I flag commercials manually using xtscut, and extracting the content is definitely I/O bound and time-consuming. For capture and viewing, however, you're right: any old disk will do. –  Norman Ramsey Jul 27 '09 at 2:42
add comment

Striping will probably not help your everyday desktop activities. Striping is best used when you have large and (relatively) long IO operations. On a regular desktop, you usually have a lot of small, random IO operations, which will not benefit from striping.

IMHO, striping is best used:

  1. When you have large files you need to read/write. Several disks will most likely out perform a single disk.
  2. You have good RAID support under your physical volumes. As previously noted, striping doubles your chance to lose data.
  3. Having separate controllers will help you get even better performance as the work will be truly divided into several channels
share|improve this answer
1  
FYI, I do a lot of work with HD video; file sizes range from 1-30GB. –  Norman Ramsey Nov 25 '09 at 20:39
add comment

I say - always use LVM. You never know when you might want to extend the LV you're using, so why not have the option to do that just by adding another drive, or by dd-ing the entire thing to a larger drive and adding a PV on the empty space.

Not to mention the possibility of snapshots, and the rest of the features LVM provides

And as for those articles, raid and LVM are different things, and are there to achieve different purposes. Raid is mainly to increate the robustness of the storage, or to improve it's speed. LVM is about flexible space management, and some extra goodies like snapshots, which is only possible on high end storage systems otherwise.

share|improve this answer
2  
I do always use LVM. The question is, when should I stripe acroess two disks? –  Norman Ramsey Jul 26 '09 at 17:33
add comment

Striping is only useful for where you can risk losing the data. Your block device will be half as reliable as a single drive. If you can live with that in return for extra performance, then striping might work out well for you.

share|improve this answer
    
There's always a risk of losing data, and striping doubles it. I can live with the in return for extra performance. The question is, under what circumstances will I actually get the extra performance I am paying for (in increased risk)? –  Norman Ramsey Jul 26 '09 at 17:34
    
Well, your drives need to be on different buses (which under SATA is fairly easy) Large reads should be quicker from striped partitions. Small reads will probably be no quicker. Again, large writes will be quicker, but I can't see small writes being that much quicker. Large depends on the size of your striping. You need to be reading or writing across more than one stripe be improve your performance. –  David Pashley Jul 26 '09 at 22:55
add comment

Whether you get the extra performance will depend on the rest of your system. Are there bottlenecks between the SATA chips and the rest of the system. Are you really I/O bound etc.

The only easy way to see if you will gain anything is by doing the experiment with the job load you want to use the system with.

As others have said always use LVM because of the flexibility it gives you. I would not use un-mirrored stripes at home unless it was for tmp space.

share|improve this answer
add comment

In general, the "hardware RAID" adapters on workstation-class devices are junk. There is a wealth of StackOverflow questions on that topic, so I'll leave it at that.

Whatever route you'll take, striping (aka RAID-0) is a good technique to improve the performance of IO bound applications, which you have. The price of that performance is a higher, unknown additional risk of catastrophic failure.

Is that a big deal? For me, it isn't -- I'm using a Mac Pro with 4 striped drives, all backed up with Time Machine. I can live with being out of commission for a few hours while I restore from backup. For you, I would recommend employing a backup/restore strategy that you can live with before proceeding with striping.

share|improve this answer
add comment

IMO, you should use striping by itself only when you don't care at all about your data.

when you stripe data across multiple drives, you increase the risk of losing everything on all drives because the failure of any one drive means complete loss of the entire volume...and the more drives you stripe over, the greater the risk.

accordingly, IMO, the answer to "when should i stripe?" is either "never" or "when you're striping mirrored volumes, as in RAID-10".

if both the data itself and IO performance is important to you, then get a good hardware SAS RAID card (e.g. an adaptec 3805 or 5805 or similar) with a large battery-backed write cache, and make a RAID-6 volume. to get 4TB with RAID-6, you'll need 6 x 1TB drives. plus one more as a hot or cold spare.

SAS controllers support both SAS drives and SATA drives. the models mentioned above support up to 8 drives directly but can support more through the use of SAS expanders (at the cost of performance - more drives mean less IO bandwidth per drive, but you could probably expand up to 16 drives or so without noticing any real performance hit. 3Gbps per SATA channel gives you maybe 250MB/s of IO, and current good non-SSD drives can use about 100-120MB/s or so each).

alternatively, use software RAID-10 (a striped array of mirrored volumes). a 4TB array would require 8x1TB drives. e.g. 4 x RAID-1 arrays striped together with RAID-0 (or LVM) for a single 4TB volume.

you can use LVM on top of these RAID arrays to manage the space. If you're going the RAID-10 route, then the striping can be done with LVM rather than RAID-0.

one other thing to consider is to separate the IO-consuming applications so that they don't compete for IO. e.g. keep your OS on one smallish drive, say 80GB (or a RAID-1 mirrored pair), your source code for compiler regression on another drive or RAID-1 pair, and your video data on either sw RAID-10 or hw RAID-6.

and install as much memory as you possibly can into the machine as linux will use it all for disk buffering. most common motherboards support up to 4 DDR-2 or 6 DDR-3 memory sticks, so with 2GB sticks being far cheaper than 4GB sticks you can install a maximum of 8GB or 12GB at a reasonable price. if you need more than that, it's more cost-effective to replace the motherboard with a server MB (from Tyan or SuperMicro etc) with more RAM sockets than it is to use 4GB sticks.

oh, and hot swap bays are a good idea - when (not if, when) a drive fails you need to be able to replace it as quickly as possible. RAID-6 can cope with any two drives failing simultaneously, so when 1 drive fails it will only take one more drive dying to take everything with it. RAID-10 can cope with more drives failing (up to half of the drives can fail as long as 1 of each mirrored pair survives).

and, finally, backup. RAID, as has been mentioned many times before by many people, is NOT a substitute for backup. The only tape medium currently capable of backing up the quantity of data you have in a reasonable time without spending days swapping cartridges is LTO-4. The drives for this are expensive and the tape cartridges seem expensive (but are actually cheaper than hard drives when you calculate the cost per gigabyte). if your budget doesn't stretch to that, then you could use multiple extra drives (connected via e-sata, firewire, a spare hot-swap bay...or even USB) to backup to - insert drive, run backup, remove drive, store on shelf or off-site...current drive capacities are up to 2TB, and will get larger and cheaper over time. BTW, at current prices (approx $1500-$2000 for a bare LTO drive vs approx $100 for a 1TB hard disk - approx. current australian dollar prices), the cost of an LTO drive would buy you 15 to 20 hard drives for backup...and you could buy them as you need them rather than all at once, with prices dropping noticably each time.

share|improve this answer
add comment

It might be because the drive all my games were installed on is a bit older than the drive my main system/work partitions are on, but I noticed a definite improvement in loading and inter-level times for some games when I added a second drive (of the same lineage) and set then up as as RAID0 array. Not close to the theoretical twice-the-performance, but enough to be quite noticeable. This is of course quite subjective: I have not perform any statistically sound benchmarks.

Those articles are fairly old, and the storagereview one specifically talks about the limitations of the PCI bus (meaning that it would be the bottleneck in terms of bulk throughput, not modern drives, limiting the bulk performance of RAID0). But if your controller is PCI-E based then you will have greater throughput to play with.

Having said that, I only setup RAID0 on my desktop box as an experiment to see what difference it would actually make. Were I to put together a brand new desktop PC at home with nce new shiney fast drives I'm not sure if I would do the same again or not. For desktop use generally I would recommend using a pair of drives as a RAID1 array to get the redundancy instead of using them in a RAID0 configuration for performance reasons.

share|improve this answer
add comment

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.