Take the 2-minute tour ×
Server Fault is a question and answer site for system and network administrators. It's 100% free, no registration required.

I've been using a QEMU system to build software. The problem I've been having is that the system seems to suffer from extremely slow disk access. This is not necessarily a problem as it's not really a time-intensive operation, but I'd like to have it work as fast as possible.

Here's all I've done to attempt to speed up I/O:

  • KVM is enabled. This is the difference between night and day as far as emulated processor speed is concerned.

  • Qemu is run with -display none mode through a forwarded SSH port, so there's no emulated SDL display eating CPU cycles.

  • The virtual hard drives are mounted with -o noatime,nodiratime to cut down on unnecessary writes.

  • The hard disk images (8 and 12 gigs respectively) are in qcow2 format and were created with -o cluster_size=2M (this is the maximum–default is 65,536 bytes). Quoth the man page: "Smaller cluster sizes can improve the image file size whereas larger cluster sizes generally provide better performance."

  • The filesystem is EXT4 with the -o extents option, which the man page claims "is a much more efficient encoding which speeds up filesystem access, especially for large files."

  • The virtual disk images do not have any snapshots stored in them. I am unsure whether this would slow them down, but it seems like it would; the actual file image size does not grow when a snapshot is recorded, so my guess is that the data is stored in some convoluted way that makes it so the snapshot and the "working" drive will share a cluster until it is changed, then the entire 2 MB cluster is copied into new space, then the data is changed. Ingenious but totally inefficient.

My question is whether there is anything I'm missing to increase hard disk efficiency, or what has worked for people in the past in order to achieve near-native I/O speeds.


Here are a couple of other things that I have observed:

  • When I mount the hard drive in the virtual host with -o sync the system slows way down. This seems counter-intuitive to me since the writes to the virtual hard drive, which is just a file on my actual hard drive, are already buffered in RAM before written. This is essentially double buffering to the disk, using twice as much RAM as it has to. The dramatic speedup is especially strange since doing a grep ^Dirty /proc/meminfo never says anything over 1 MB.

  • The virtual machine seems to refuse to use its swap unless I set the "swappiness" (/proc/sys/vm/swappiness) to 90 or higher. While I agree this is probably a good thing in terms of disk I/O, it makes the build processes "fight" with the fs buffering for use of the RAM. Yes, I'm sure it has enough RAM.

share|improve this question

migrated from unix.stackexchange.com Aug 27 '11 at 15:15

This question came from our site for users of Linux, FreeBSD and other Un*x-like operating systems..

2  
Expected behavior for -o sync. The option commands the kernel to block writing applications and write to the disk immediately, rather than just stuff the data in the cache, and let it be flushed to the disk in the background. –  psusi Aug 24 '11 at 19:42
    
Also asked here. –  amphetamachine Aug 27 '11 at 10:28
    
What do you mean by running the VM with KVM enabled? –  Belmin Fernandez Sep 4 '11 at 1:29
    
@Beaming Mel-Bin: Just that. I already have KVM enabled in my kernel, so it was a matter of doing a modprobe kvm-amd ; chown root:users /dev/kvm ; chmod 0660 /dev/kvm as root, then using -enable-kvm when starting qemu. Yes, I know I should probably write a udev rule. –  amphetamachine Sep 4 '11 at 1:50

1 Answer 1

up vote 3 down vote accepted

Some performance tips are here, you should definitely try to use raw format instead of qcow2 and raw devices instead of image files. You should also install paravirtualized disk drivers on guests and switch disk type from IDE to virtio.

share|improve this answer
    
A tutorial on how to install paravirtual disk drivers is here: linux-kvm.com/content/… –  Alex Aug 27 '11 at 10:47
    
Would this be faster or slower than using NFS? –  amphetamachine Aug 27 '11 at 11:04
    
@amphetamachine Yes, it would be faster for sure. –  Alex Aug 27 '11 at 11:08

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.