3

I'm measuring using hdparm -t on the PVs (sda3 and sdb2) and on the LV (work). It's actually faster on the PVs.

Is this really just the way lvm is, or am i doing something wrong?

# lvs -a -o +devices
LV              VG     Attr   LSize   Origin Snap%  Move Log       Copy%  Convert Devices                          
root            foobar -wi-ao 165.00G                                             /dev/sda3(10752)                 
swap            foobar -wi-ao   2.00G                                             /dev/sda3(10240)                 
work            foobar mwi-ao 148.52G                    work_mlog 100.00         work_mimage_0(0),work_mimage_1(0)
[work_mimage_0] foobar iwi-ao 148.52G                                             /dev/sda3(52992)                 
[work_mimage_1] foobar iwi-ao 148.52G                                             /dev/sdb2(0)                     
[work_mlog]     foobar lwi-ao   4.00M                                             /dev/sdb1(0)                     

3 Answers 3

6

There is a small extra overhead to passing the data through the extra virtual block device interfaces (in the kernel). Usually it's on the order of line noise (no more than about 3%).

If you want improved performance from multiple drives then you need to look at the md (metadisk) drivers ... RAID 1 (mirroring) or RAID 10 (mirroring + striping). In those cases your system may be able to interleave reads and writes from multiple drives (spindles) concurrently. (Note that there are some hardware configurations from which you won't benefit from the mirroring; for example two drives hanging off a plain old IDE/PATA cable; the one controller/cable is a bottleneck).

The two major factors that affect overall drive performance are seek time and throughput ... how long it takes to position the drive heads over the requested data and how quickly the data can be transferred over the cable. Mirroring over two I/O channels obviously improves the read throughput (potentially almost twice the data transferred over any given interval). The affect on seek time is dramatic and more probabilistic ... the heads on the two drives are likely to be over different parts of their respective platters. The read request can come from either drive, thus request can be routed to the drive which happens to have its heads closer to the desired area. (As far as I know this is really just done by a rough heuristic in the Linux kernel ... the drivers don't know details of drive geometry but simply treat the requests as offsets in a "linear block array" table).

More importantly a mirroring RAID configuration can service multiple read requests concurrently.

Note that RAID mirroring offers no benefit to writes. Every write must be done on multiple drives ... thus it much be transferred over every I/O channel in the set, and you suffer from the worst seek time in the set (rather than gaining from the optimally positioned head in the case of reads).

Note that it is possible to use LVM and md RAID features together in Linux. Typically you'd use the md* drivers as a lower virtual block layer (aggregating the physical disks into RAID sets) and then use LVM over those (making each of your top level md* (1, 5 or 6) devices into LVM PVs --- physical volumes). For the most part I would say that RAID 0 (striping) makes no sense with LVM. (LVM can aggregate multiple drives into larger virtual volumes, just as RAID 0 does. However it does so far more flexibly).

5
  • 3
    I'm not going to vote your answer down as most of it is good information. But LVM can mirror volumes. See linux.die.net/man/8/lvcreate and have a look at the -m option. Btw it uses this feature in the background when you use pvmove. Mar 15, 2010 at 19:45
  • @3dinfluence: Ooops. My bad. They snuck that in at some point.
    – Jim Dennis
    Mar 16, 2010 at 4:34
  • 1
    I'm not sure when that feature was added but I do know it's been there for as long as I've been using LVM which is over 7 years. Mar 16, 2010 at 13:51
  • 1
    Please update your answer with the correct information now that everyone's on the same page regarding mirroring. That will eliminate any confusion for others.
    – MT.
    Mar 17, 2010 at 14:33
  • I removed the part about the mirroring, so it'll draw no more negative attention. Jim, please review your post in case you find it lacks consistency now.
    – wzzrd
    Mar 17, 2010 at 15:47
1

One problem I had at some point was that with LVM volumes the default values for the max readahead was set to something stupidly low. IIRC that has been fixed since, but I suppose it won't hurt to check. E.g. the

blockdev --getra /dev/foo/bar

to see the readahead setting of the device, and --setra N to set it to N sectors. And you probably want to set a decent readahead on either the underlying device or on the LVM volume, not both.

Also, for "proper" benchmarking, use something like iozone, fio, bonnie++ instead of hdparm. hdparm accesses the drives directly, and I'm not sure it does anything relevant when you run it on a virtual device such as a LVM LV.

2
  • 1
    --getra is 256 on the PVs as well as the LV Mar 18, 2010 at 15:44
  • 2
    what is "stupidly low". Mine says 256 as well.
    – Stu
    Dec 9, 2015 at 16:31
0

This could be to any number of reasons. I know that certain file systems have "copy-on-write", including Ext3, ZFS, WAFL, VERITAS (NetBackup), and btrfs. However, while COW (copy-on-write) COULD slow your system down, it should not be something easily noticable.

Also, if you have snapshots enabled (with whatever controller you are using to create the LV), that would also slow them down.

Depending on the size of the physical extents in the volume, it could slow it down (say, perhaps, that each physical drive was of a different total volume). Or perhaps something is going on with one of the individual physical disks that make up your LVM.

Try to do a few diagnostics on the disks themselves and see if any of them return bad sectors/slices. Are you growing this logical volume automatically, or is this a fixed-size volume?

It could just very well be the way that your LVM mirror is set up. The controller, or physical disks, may have something going on. You also have to remember the speed on the controller, since the path that data takes to hit the drives goes through the controller - having an older controller with newer HDDs doesn't speed things up (depending on the older controller, i've got one that is ages old and almost always outperforms some of the newer ones).

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .