36

I would be thankfull if someone who understands how LVM works, could tell me a rough estimate, how much slower using LVM (with a Software RAID1) will be.

(What I do not want to know how much slower LVM will be if the LVM Volume is currently in snapshot mode doing Copy on Write). I only need some rough estmiate how much LVM will slow down reads and writes in a normal operation scenario.

Any links are also very much appreciated I was not able to find any good performance benachmarks about this question.

1

6 Answers 6

25

LVM is fairly lightweight for just normal volumes (without snapshots, for example). It's really just a table lookup in a fairly small table that block X is actually block Y on device Z. I've never done any benchmarking, but I've never noticed any performance differences between LVM and just using the raw device. It's some small extra CPU overhead on the disc I/O, so I really wouldn't expect much difference.

My gut reaction is that the reason there are no benchmarks is that there just isn't that much overhead in LVM.

The convenience of LVM, and being able to slice and dice and add more drives, IMHO, far outweighs what little (if any) performance difference there may be.

4
  • hello sean, thanks for this helpfull answer! now I understand how lvm works and it seems reasonalbe that this table is held in memory and will incure only a very small overhead for the "in memory lookup". thanks for describing it that clearly. thanks!
    – jens
    Dec 7, 2010 at 4:46
  • No problem, glad to help. Welcome to Server Fault. Dec 7, 2010 at 12:58
  • 1
    I dunno, my experience with lvm isn't that impressive. I ran a little test, I put a 70 gig image on a raw ext4 volume and the same 70 gig image on an lvm 2 disk mirror formatted to ext4. Same machine same exact disks. I then dd'd the image to /dev/null, it took 6 minutes 9 seconds to read from ext4 and 16 minutes 1 second to read from the mirror. That's READing.
    – Stu
    Dec 7, 2015 at 22:02
  • Did I mention reading, from a mirror? Should have gone faster than the plain ext4 disk.
    – Stu
    Dec 7, 2015 at 22:02
17

I am installing a 48T Dell MD-1200 and I was curious about this question. The MD1200 is connected to a hardware RAID card set up as RAID-6, so it looks to Linux like just a (big) drive. I tested an XFS filesystem on an LVM physical volume vs. an XFS filesystem on a straight disk partition. I used a Dell R630 machine with two E5-2699 CPUs in it. The system was set for Performance; whatever energy saving features I could find in the BIOS were turned off.

I installed CentOS 6.7 on it. Kernel is 2.6.32-573.el6.x86_64 (sorry for the oldie kernel but that's what I need for production). LVM is version 2.02.118.

I let CentOS create an XFS partition during the build. It is 1T in size. Then I created another 1T partition on the disk and created a logical volume:

vgcreate vol_grp1 /dev/sdb1
lvcreate -l 100%FREE -n lv_vol1 vol_grp1
mkfs.xfs /dev/vol_grp1/lv_vol1

My XFS-only filesystem was called /data_xfs. The LVM-backed XFS filesystem was called /data_lvm. I tested using bonnie++ v 1.03e.

The commands were: bonnie++ -u 0:0 -d /FILESYSTEM -s 400G -n 0 -m xfsspeedtest -f -b where FILESYSTEM was either /data_xfs or /data_lvm . Results are summarized as follows:

Test                        XFS on Partition        XFS on LVM
Sequential Output, Block    1467995 K/S, 94% CPU    1459880 K/s, 95% CPU
Sequential Output, Rewrite   457527 K/S, 33% CPU     443076 K/S, 33% CPU

Sequential Input, Block      899382 K/s, 35% CPU     922884 K/S, 32% CPU

Random Seeks                 415.0 /sec.              411.9 /sec.

Results seemed comparable in my view. In the Sequential Input test, LVM actually seemed to perform a little better.

12

There is a short paper published 2015 by Borislav Djordjevic and Valentina Timcenko which used a few 7200RPM 80GB Western Digital drives using EXT3, tested using PostMark software that 'simulates loading an internet mail server' with Linux kernel 2.6.27. They found that past research that had looked at just bonnie or dd tests alone had varied results.

The tests seem to suggest the performance drop can be from 15% to 45% with LVM, compared to when not using it. They found an even bigger drop when two physical partitions are used within one LVM setup. They concluded that the biggest performance impacts were the use of LVM, as well as the complexity of it's use.

https://www.researchgate.net/publication/284897601_LVM_in_the_Linux_environment_Performance_examination http://hrcak.srce.hr/index.php?show=clanak&id_clanak_jezik=216661

7

with snapshot active lvm performs ... badly.

take a look here to see in-depth benchmark

6

This is an old question, but it deserves a current answer.

Short answer: classical/linear LVM commands minimal - almost zero - overhead and can be used with no performance concern, unless snapshots are used (as they destroy write speed). Thin volumes are somewhat less quick by default but have much faster snapshots, providing modern CoW (and thin provisioning) to classical filesystems. Other LVM segment types have different overhead depending on the data transformation they do.

Long answer: the core tech remains device mapper - a framework to map (possibly transformed) blocks on different devices. Different device mapper targets provide different features with various performance overhead. LVM is itself built on device mapper: one can see LVM as a CLI/wrapper around device mapper and other related tools.

The most common targets are:

  • linear: the basic LVM target. It is simple yet quite useful to extend a single volume over different block devices without using/involving RAID (the old mirror LVM segment type was deprecated and removed). It has minimal overhead, and RHEL use it by default for the root filesystem. Snapshots taken via this target (ie: classical, "fat" snapshots) are very slow for writes and become even worse if multiple concurrent snapshots are "alive", so they should be used only for backing up data and promptly destroyed when finished;
  • thin: it is a modern CoW target enabling fast rolling snapshots and volume cloning. The drawback is somewhat-reduced base performance (ie: when no snapshots exist) versus fat/linear volumes. Snapshots write performance hit is constant and generally limited to ~50% max (often much smaller). I use thin volumes for production data, leaving / (root) on classical fat volumes;
  • vdo: it is a new target providing data compression and deduplication. Write performance take a significant hit, which can be limited by using a BBU-enabled HW RAID card;
  • cache: a performance enhancing target enabling both read and write caching of slower devices (ie: SSD caching for an HDD-based main storage pool);
  • raid: another performance/reliability enhancing target, bridging over linux mdraid
  • integrity: it provides a software-only implementation of SAS T-10 standard for detecting data corruption. journal mode is the safest but it has high write overheard.

As you can see, there is not a single "LVM overhead". Rather, based on the targets and the expected usage, the overhead can range from zero to >90%. So be sure to model your workload accurately before LVM setup.

1
  • Excellent answer! Thanks for the "current" update. I'm pretty sure this question will continue to be asked over the decades. Feb 13 at 19:32
4

There is an excellent (be it old) whitepaper, written by a SUSE guy, about LVM and it's overhead here. It shows some (simple) benchmarks and explains the tech behind LVM. Good read.

2

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .