I've a DOM0 packed with two 3TB hard drives configured as a software RAID1, also sporting 16GB of memory. The DOM0's Linux distro is a CentOS 6.3 and the hypervisor is a libvirt-managed KVM.

I'm experiencing performances degradations between the DOM0 and the DOMU, with DOMU write throughput around 40% of the DOM0's. I tested the performances using the following simple dd command:

dd bs=1M count=1024 oflag=direct if=/dev/zero of=testb conv=fdatasync

I configured the DEADLINE elevator for DOM0 and the NOOP one for DOMU, which in turn is a Debian Unstable:

Linux vm1 2.6.32-5-amd64 #1 SMP Sun Sep 23 10:07:46 UTC 2012 x86_64 GNU/Linux

root@vm1:/home/paris# cat /etc/debian_version

As for DOMU configuration in libvirt, here is the XML fragment for one of the virtual drives which are actually plain raw fallocated files:

<disk type='file' device='disk'>
  <driver name='qemu' type='raw' cache='none' io='native'/>
  <source file='/mnt/vmstore2/disks/vm1/boot.img'/>
  <target dev='vda' bus='virtio'/>
  <alias name='virtio-disk0'/>
  <address type='pci' domain='0x0000' bus='0x00' slot='0x05' function='0x0'/>

I also noticed that in the command line generated by libvirt, the "if" parameter of the -drive command line switch is set to "none" instead of "virtio":

-drive file=/mnt/vmstore2/disks/vm1/root.img,if=none,id=drive-virtio-disk2,format=raw,cache=none,aio=native
-device virtio-blk-pci,scsi=off,bus=pci.0,addr=0x7,drive=drive-virtio-disk2,id=virtio-disk2

I don't know whether this performace degradation is normal, but I read that VIRTIO should be able to guarantee an almost identical throughput between DOM0 and DOMU.

Thank you for any help.

  • UPDATE. I tried to mount the raw image file on DOM0 and, repeating the test, now the performance degradation raised to 80%. – Cristiano Paris Jan 11 '13 at 17:09
  • 1
    What is all this about dom0 and domU? You said you were using KVM, not Xen? – Michael Hampton Jan 11 '13 at 17:57
  • YEp! I'm coming from Xen and testing KVM. Sorry for that: s/domU/guest/g s/dom0/host/g – Cristiano Paris Jan 11 '13 at 18:29

Even with virtio, using an image file for a guest disk is going to have a certain amount of associated overhead. This can vary significantly depending on other I/O activity on the same physical device that the image file resides on.

If you were to pass a physical disk as a block device to KVM guest with the virtio driver, you would see much closer to native throughput. Same goes for a partition or LVM volume, with the same I/O caveats as above (though these will usually both still peform better than a disk image file).

We do use image files as disk for some of our KVM guests, but we keep the images on a separate physical drive or array than the host whenever possible.

  • If you do need disk images, try preallocating them first. It makes a huge difference. – n8whnp Jan 11 '13 at 20:30

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.