I've been using a QEMU system to build software. The problem I've been having is that the system seems to suffer from extremely slow disk access. This is not necessarily a problem as it's not really a time-intensive operation, but I'd like to have it work as fast as possible.
Here's all I've done to attempt to speed up I/O:
KVM is enabled. This is the difference between night and day as far as emulated processor speed is concerned.
Qemu is run with
-display nonemode through a forwarded SSH port, so there's no emulated SDL display eating CPU cycles.
The virtual hard drives are mounted with
-o noatime,nodiratimeto cut down on unnecessary writes.
The hard disk images (8 and 12 gigs respectively) are in qcow2 format and were created with
-o cluster_size=2M(this is the maximum–default is 65,536 bytes). Quoth the man page: "Smaller cluster sizes can improve the image file size whereas larger cluster sizes generally provide better performance."
The filesystem is EXT4 with the
-o extentsoption, which the man page claims "is a much more efficient encoding which speeds up filesystem access, especially for large files."
The virtual disk images do not have any snapshots stored in them. I am unsure whether this would slow them down, but it seems like it would; the actual file image size does not grow when a snapshot is recorded, so my guess is that the data is stored in some convoluted way that makes it so the snapshot and the "working" drive will share a cluster until it is changed, then the entire 2 MB cluster is copied into new space, then the data is changed. Ingenious but totally inefficient.
My question is whether there is anything I'm missing to increase hard disk efficiency, or what has worked for people in the past in order to achieve near-native I/O speeds.
Here are a couple of other things that I have observed:
When I mount the hard drive in the virtual host with
-o syncthe system slows way down. This seems counter-intuitive to me since the writes to the virtual hard drive, which is just a file on my actual hard drive, are already buffered in RAM before written. This is essentially double buffering to the disk, using twice as much RAM as it has to. The dramatic speedup is especially strange since doing a
grep ^Dirty /proc/meminfonever says anything over 1 MB.
The virtual machine seems to refuse to use its swap unless I set the "swappiness" (
/proc/sys/vm/swappiness) to 90 or higher. While I agree this is probably a good thing in terms of disk I/O, it makes the build processes "fight" with the fs buffering for use of the RAM. Yes, I'm sure it has enough RAM.
-o sync. The option commands the kernel to block writing applications and write to the disk immediately, rather than just stuff the data in the cache, and let it be flushed to the disk in the background.
modprobe kvm-amd ; chown root:users /dev/kvm ; chmod 0660 /dev/kvmas root, then using
-enable-kvmwhen starting qemu. Yes, I know I should probably write a udev rule.