Take the 2-minute tour ×
Server Fault is a question and answer site for professional system and network administrators. It's 100% free, no registration required.

performance of my setup is quite good (geekbench, how it 'feels', ...) even disc throughput (libvirt on raw lvm-partition) is very close to the raw performance on the server) but IOP/s are as low as 100-200 guestside (compared to ~1000 hostside) on both linux and windows-guests.
Is this a thing to live with (kvm just can't do it better) or am i doing something completely wrong ?

The interesting thing is that i was able to impact the throughput by changing the setup (qcow2 vs raw-image vs raw-partition) or the configuration (caching or io-scheduling) and variations but IOPs stayed at the same low point over all those combinations.

hardware#

• supermicro dual xeon E5520 with 24GB RAM
• 2x seagate constellation 1TB (RAID1 on Adaptec 3405)
• 2x seagate cheetah (RAID1 on Adaptec 6405).

software

• ubuntu 11.10 3.0.0-13-server
• kvm/QEMU emulator version 0.14.1 (qemu-kvm-0.14.1)
• benchmarking the disks (bonnie++, hdparm) from the hosts and guests (bonnie++, hdparm, hdtune on windows)

config

i tested several disc-configurations, the current setup is:

linux hosts

(They just don't "need" hight IO-Performance so i keep the more comfortable discfiles)
• qcow2 disc files on lvm on my constellations
• qemu/ide

<disk type='file' device='disk'>
  <driver name='qemu' type='qcow2'/>
  <source file='/media/vm/images/mex/mex_root.qcow2'/>
  <target dev='hda' bus='ide'/>
  <address type='drive' controller='0' bus='0' unit='0'/>
</disk>

windows hosts###

(Running SQL-Server and Remote-Desktop-Services, so here I definitely need a good IO-Performance)
• raw lvm partitions on my cheetahs
• virtio

<emulator>/usr/bin/kvm</emulator>
<disk type='block' device='disk'>
  <driver name='qemu' type='raw' cache='none'/>
  <source dev='/dev/Cheetah/mts'/>
  <target dev='vda' bus='virtio'/>
  <address type='pci' domain='0x0000' bus='0x00' slot='0x06' function='0x0'/>
</disk>
share|improve this question
add comment

2 Answers

The optimal configuration is (usually) as follows:

  1. On the host, set elevator=deadline
  2. Use virtio and only virtio
  3. use raw LVs whenever possible. Qcow2 gives overhead. Files on a FS also have overhead
  4. in the VM use the elevator=noop
  5. both in host and VM, use noatime,nodiratime in fstab wherever possible
  6. Make sure the virtio drivers are up to date, especially the windows ones.
  7. Debian based distros are (arguably) not as good as Fedora and RHEL for QEMU/KVM. Not to start a flamewar, but most of the development and testing is done on Fedora and RHEL, and in my own experience, there have been lots of issues on Ubuntu and Debian that I couldn't reproduce on Fedora and RHEL. You can ignore this particular bullet if you want, but if you're looking for a solution, a quick benchmark on another distro is usually worth a try
share|improve this answer
    
thank you very much! i'll try out these points this weekend and keep you informed about my progress. do you know how much overhead on iops is 'normal' ? my throughput is currently at about 10% with virtio on lvm-partitions... is this a realistic value for iops too ? –  phhe Feb 16 '12 at 16:15
    
I've seen closer to 95% baremetal speeds with direct IO (using dd from /dev/zero to /dev/vda) –  dyasny Feb 16 '12 at 16:22
add comment

Try setting "deadline" as the I/O scheduler for your host's disks before starting KVM:

 for f in /sys/block/sd*/queue/scheduler; do echo "deadline" > $f; done

If you have I/O bound load, it might be your best choice as this IBM paper suggests.

share|improve this answer
    
elevator=deadline in grub is usually good enough for hosts that are only hypervisors –  dyasny Feb 16 '12 at 10:50
    
@dyasny I always thought it would have very much the same effect - but require a reboot. –  the-wabbit Feb 16 '12 at 11:21
    
It would be persistent over reboots though. And if you want to avoid rebooting, you can combine the two –  dyasny Feb 16 '12 at 11:33
add comment

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.