qemu don't start quest with disk image on tmpfs

Question

I have 32bit windows 2k3r3 guest (terminal server) with 4GB guest ram and swapping.

I created separate disk image for guest swapping and user's temp dirs.

I have enouth RAM in host system and want to save disk IO by moving this image to tmpfs, but guest doesn't start with this error message:

qemu-kvm: -drive file=/mnt/tmpfs/vh1-tmp.qcow2,if=none,id=drive-ide0-1-1,format=qcow2,cache=none: could not open disk imag│ 4098 qemu 20 0 4949M 4146M 5496 S 28.5 17.2 1h00:31 /usr/bin/qemu-kvm -name vh1 -S -M pc-1.3 -cpu kvm64 -enable- e /mnt/tmpfs/vh1-tmp.qcow2: Invalid argument

Host system:

#uname -a
Linux srv-vh1.su.local 3.7.10-1.16-default #1 SMP Fri May 31 20:21:23 UTC 2013 x86_64 x86_64 x86_64 GNU/Linux

srv-vh1:/mnt/tmpfs # virsh version
Compiled against library: libvirt 1.0.2
Using library: libvirt 1.0.2
Using API: QEMU 1.0.2
Running hypervisor: QEMU 1.3.0

srv-vh1:/mnt/tmpfs # free
             total       used       free     shared    buffers     cached
Mem:      24627548    5084724   19542824          0      60640     138792
-/+ buffers/cache:    4885292   19742256
Swap:      8384444          0    8384444

srv-vh1:/mnt/tmpfs # cat /etc/mtab | grep tmpfs
devtmpfs /dev devtmpfs rw,relatime,size=12296608k,nr_inodes=3074152,mode=755 0 0
tmpfs /dev/shm tmpfs rw,relatime 0 0
tmpfs /run tmpfs rw,nosuid,nodev,relatime,mode=755 0 0
tmpfs /sys/fs/cgroup tmpfs rw,nosuid,nodev,noexec,mode=755 0 0
tmpfs /mnt/tmpfs tmpfs rw,relatime 0 0
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
tmpfs /tmp tmpfs rw,relatime 0 0
tmpfs /var/lock tmpfs rw,nosuid,nodev,relatime,mode=755 0 0
tmpfs /var/run tmpfs rw,nosuid,nodev,relatime,mode=755 0 0

srv-vh1:/mnt/tmpfs # df
Filesystem  1K-blocks used      Avaible     Used%          Mountpoint
devtmpfs          12296608           68  12296540            1% /dev
tmpfs             12313772            0  12313772            0% /dev/shm
tmpfs             12313772         6772  12307000            1% /run
/dev/md1         454131992    218835836 212227596           51% /
tmpfs             12313772            0  12313772            0% /sys/fs/cgroup
tmpfs             12313772          192  12313580            1% /mnt/tmpfs
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
tmpfs             12313772           20  12313752            1% /tmp
tmpfs             12313772         6772  12307000            1% /var/lock
tmpfs             12313772         6772  12307000            1% /var/run

srv-vh1:/mnt/tmpfs # virsh pool-info tmpfs
Name:         tmpfs
UUID:           6287028a-9faf-f762-20de-d36d63657be3
Status:   working
Persistent:     yes
Autostart: yes
Capacity: 11,74 GiB
Выделение: 0,00 
Avaible: 11,74 GiB

srv-vh1:/mnt/tmpfs # ls -la
total 196
drwxrwxrwt 2 root root     60 сен  9 11:42 .
drwxrwxr-x 4 qemu qemu   4096 сен  8 19:39 ..
-rw-rw-rw- 1 root root 197120 сен  9 11:42 tserver-tmp.qcow2

What am i doing wrong ?

Answer 1

Apparently, if you set cache=NONE for a disk image file on any host filesystem that does not support Direct IO, Virt-Manager will currently give a not very helpful error message saying "Something something... Invalid Argument" and refuse to start the Guest VM.

One example of such a filesystem --that does not support Direct IO-- is the tmpfs. Another such filesystem where maybe you can optionally disable Direct IO is GlusterFS (which I had not heard of before I also hit the same problem as Sergei did and was researching the error.) For tmpfs, not supporting Direct IO seems to be a technical limitation or design conflict at present and I don't know whether it will/can be rectified in future.

In my case, I had put a 3.6 GB Ramdisk for a Win7Pro Guest VM running on CentOS7 and had set cache=NONE for the ramdisk in Virt-Manager. The other options that the tmpfs img used were virtio and raw. The VM would refuse to start with the same/similar error saying "... Invalid Argument".

For technical details and notes directly from the Redhat Engineer and Developer that coded the patch for the feature cache=NONE & maintains (?) Virt-Manager (Daniel Berrange), see the discussion at the following link:

https://bugs.launchpad.net/nova/+bug/959637

To quote Daniel from the URL above: " > could not open disk image /mnt/vmstore/instances/instance-0000001a/disk: Invalid argument

probably means that the filesystem does not support Direct IO. AFAIK, all filesystems except tmpfs should support this.."

Also, Daniel continues: "- So what we'll want todo then, is to [....] do a check whether the storage volume supports Direct IO. If it does, then use cache=none, otherwise fallback to cache=writethrough which does not use Direct I/O, but is still crash safe."

In my case, I was able to verify that setting cache=NONE for the tmpfs img file was NOT starting the VM and showed the error as discussed. It WAS able to successfully start the VM if cache was set to either "Default" or explicitly "Write-through". There is no sense in going with "Write-back" since this filesystem both expendable and was anyways entirely in RAM, so obviously I did not use Write-back.

Hope that helps!

Answer 2

Why not just give the guest more ram so it doesn't need to swap?

srv-vh1:/mnt/tmpfs # ls -la
total 196
drwxrwxrwt 2 root root     60 сен  9 11:42 .
drwxrwxr-x 4 qemu qemu   4096 сен  8 19:39 ..
-rw-rw-rw- 1 root root 197120 сен  9 11:42 tserver-tmp.qcow2

I notice you have the folder owner set to qemu:qemu, if you are running qemu as a different user then you might want to change the owner of the image file from root to qemu

Answer 3

It's a bug. tmpfs doesn't support cache=none.