37

we are using kvm/qemu with qcow2-images for our virtual machines.

qcow2 has this nice feature where the image file only allocates the actually needed space by the virtual-machine. but how do i shrink back the image file, if the virtual machine's allocated space gets smaller?

example:

1.) i create a new image with qcow2 format, size 100GB

2.) i use this image to install ubuntu. installation needs about 10 gb, the image-file grows up to about 10GB. nothing unexpected so far.

3.) i fill up the image with about 40 GB of additional data. the image-file grows up to 50GB. i am ok with that :-)

4.) this is where it gets strange: i delete all of the 40GB data on the image, but the image-size still eats up 50GB.

question: how do i free up that 40GB of data and shrink the image to the only needed 10 GB?

thanks in advance, berni

6 Answers 6

22

virt-sparsify can do all this with less hassle on your part: http://libguestfs.org/virt-sparsify.1.html

2
  • 5
    @dyasny If the existing answers to old questions have become obsolete over time, it is worth writing a new answer. This particular new answer might however not be up to standards for this site.
    – kasperd
    Aug 21, 2014 at 19:08
  • 6
    @dyasny If somebody want to spend the effort on writing a new quality answer, each time the old one is obsolete, they should be encouraged to do so. A more thorough discussion on how to deal with outdated answers is better suited for meta.
    – kasperd
    Aug 21, 2014 at 19:52
33

The image will not shrink automatically, since when you delete files, you don't actually delete data (this is why undelete works). Qemu has a facility to shrink qcow2 images back, but what the utility does is really deduplicate the zeroes from the disk, leaving all other information intact. So the idea would be to:

  1. Zero-fill the drive (dd if=/dev/zero of=/some/file until you run out of space)
  2. delete /some/file
  3. shut down the VM
  4. cd to where the images for the VM are kept and run qemu-img convert -O qcow2 original_image.qcow2 deduplicated_image.qcow2
  5. change the VM settings to use the new deduplicated_image.qcow2, test the VM is working, and remove the old image

This, afaik, will only work with qcow2 images, I haven't tested other formats.

1
  • 5
    If you have a windows guest run SDelete -z to zero up all free space (steps 1,2 above)
    – ndemou
    Jul 1, 2016 at 6:57
9

You can use virt-sparsify for this. It automatically zeros and sparsifies free space on most filesystems it finds within the source disk image (supports ext2/3/4, btrfs, NTFS, etc) .

Step-by-step instructions for in-place conversion

  1. Shut down the VM

  2. Keep a backup

  3. Convert: virt-sparsify --in-place disk.img

Step-by-step instructions for conversion to another file (this is safer but requires more free space):

  1. Shut down the VM

  2. Convert: virt-sparsify disk.img new-file.img

  3. Swap old with new image file: mv disk.img disk.img.bak && mv new-file.img disk.img

  4. If VM starts normally you can remove the backup: rm disk.img.bak

P.S.: You can also convert between formats when you use virt-sparsify:

virt-sparsify disk.raw --convert qcow2 disk.qcow2
2
  • dyasny's answer is complete, Blake's answer is up-to-date but I want the best of them :-)
    – ndemou
    Jun 1, 2020 at 18:48
  • virt-sparsify --in-place disk.qcow2 didn't reduce the file size for me. virt-sparsify disk.qcow2 disk2.qcow2 did.
    – Leon V
    Jan 13 at 1:26
9

In addition to the accepted answer (which describe the more common, general method to shrink a qcow2 file), modern versions (ie: RHEL7+) of the libvirt/qemu/qcow2 stack provide the virtio-scsi driver which supports the discard='unmap' option.

If the virtual machine uses virtio-scsi and the libvirt definition include discard='unmap', a simple fstrim on the guest filesystem will release assigned-but-unused free space on the host. This can be checked with du -hs <guestdisk> on host side (note: ls -al <guestdisk> will not show the real allocated size, just the logical one).

Note: qemu 4.x added discard support for virtio-blk too, removing the needing for virtio-scsi for trim to work.

On newer libvirt/qemu instances (ie: RHEL8+), when using virtio-scsi, guest file size can be reduced even if the guest OS does not support trim: by enabling both discard='unmap' and discard_zeroes='unmap' writing a sequential stream of zero on guest side (ie: via dd if=/dev/zero of=/zero.img bs=1M count=1024) will trigger host-size trimming of the affected LBA ranges. However, please note that this can be CPU-intensive for the host (which had to "parse" any guest write searching for repeating zeroes), so it should be only enabled in specific cases.

Both methods will reduce the physical allocated size (what du -hs shows). For reducing the logical size (what a simple ls -al gives) you need to use virt-spasify or qemu-img (as described in the other answers).

3
  • Correct syntax in libvirt is <driver name='qemu' type='qcow2' discard='unmap' detect_zeroes='unmap'/>
    – NiKiZe
    Jun 25, 2020 at 3:02
  • 1
    virtio_blk now supports discard as of host qemu 4.0 and more recent guest linux kernels.
    – gps
    Jan 19, 2021 at 7:35
  • @gps true, I'll update my answer.
    – shodanshok
    Jan 19, 2021 at 7:38
1

virt-sparsify --compress fileA fileB will reduce the size of the image, however it creates the qcow2 image as another file, so you will have to rename and move things around. You cannot use --compress and --in-place at the same time. See the following link to learn more. http://libguestfs.org/virt-sparsify.1.html

1

In addition to other answers about virt-sparsify:

Temporary Folder

virt-sparsify uses your /tmp folder to create/manipulate the new disk image before place it in destination folder. So, your / partition should have free space even if your volume file is in other partition.

If you don't have enouth space, you can set a new temp folder with --tmp my_tmp_folder option. Example:

mkdir my_tmp_folder
virt-sparsify \
  --tmp my_tmp_folder \
  ubuntu.qcow2 new_ubuntu.qcow2
  • This will not work with --in-place;
  • You can remove the tmp folder after the process is complete.

Verbose

There is an verbose option to help you see what's going on. Example:

virt-sparsify \
  -v \
  ubuntu.qcow2 new_ubuntu.qcow2

Check before delete

As said before, always boot your vm and check if everything is working before delete the original disk image.

1
  • after shrinking compressed qcow2 --in-place gives read error when used qemu-img convert. Be mindful of this. Aug 11, 2023 at 11:47

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .