Is there a good way to shrink disk usage of sparse file containing luks-encrypted btrfs file system image?

Question

I've created a sparse file filesystem.img, formatted in with cryptsetup luksFormat, created a btrfs filesystem on it. The image file disk usage expanding fine while adding files to the btrfs filesystem. However deleting a file on it of course do not reduce sparse file disk usage, so I need a solution to do it manually.

Unfortunately fstrim does not work, saying the discard operation is not supported.

I can't just write zeros by 'dd' or 'freezero' to a filesystem's file since encrypted zeros are not zeros and this will result into enlarging, not reducing image size.

I probably could resize the filesystem to it's minimal size and then truncate the image file to the filesystem size + luks offset size, but I found that btrfs is very shrink-unfriendly, currently btrfs filesystem usagereports ~23G free and ~81G used but I can't reduce it further, so I have ~28% overusage.

'btrfs balance' probably would help, but looks like it could runs even longer than recopying of all data to new image.

The last of course a solution but not a good one. And it is not always possible to create a new disk image of required space.

I tried to find how 'decoded zeros' looks like by creating same passphrase-encrypted zero-empty image, but each of 512 byte block (the size reported by cryptosetup status) is different. Looks like luks do not crypt each block with the same key.

Is there any other ideas?

UPD. What I've also tried to fill btrfs with zero file, find it offsets:

# filefrag -b4K -ves zero
Filesystem type is: 9123683e
File size of zero is 34811904 (8499 blocks of 4096 bytes)
ext:     logical_offset:        physical_offset: length:   expected: flags:
 0:        0..    3477:      17664..     21141:   3478:            
 1:     3478..    4732:       3399..      4653:   1255:      21142:
 2:     4733..    5673:      11673..     12613:    941:       4654:
 3:     5674..    6379:      16400..     17105:    706:      12614:
 4:     6380..    6908:       4654..      5182:    529:      17106:
 5:     6909..    7305:      12614..     13010:    397:       5183:
 6:     7306..    7823:      15770..     16287:    518:      13011:
 7:     7824..    8220:      17106..     17502:    397:      16288:
 8:     8221..    8338:       5183..      5300:    118:      17503:
 9:     8339..    8418:      13011..     13090:     80:       5301:
10:     8419..    8477:      17503..     17561:     59:      13091:
11:     8478..    8489:      13091..     13102:     12:      17562:
12:     8490..    8493:      13564..     13567:      4:      13103:
13:     8494..    8496:       3328..      3330:      3:      13568:
14:     8497..    8498:      13103..     13104:      2:       3331: last,eof

save it into another filesystem file zero.frag and try to fill image file `physical' blocks with zeros:

# offset=4096 
# cat zero.frag | tail -n +4 | head -n -1 | while read rec
  do seek=${rec#*:*:}; seek=${seek%%.*}; seek=$((seek+offset))
     count=${rec#*:*:*:  }; count=${count%%:*}; count="${count#"${count%%[![:space:]]*}"}"
     dd if=/dev/zero bs=4096 seek=$seek count=$count of=filesystem.img
  done

but this destroyed the filesystem. It was still mountable, but existing files was incorrect. Also 'filesystem.img''s disk usage became even less than btrfs filesystem used space. So still unsolved.

Answer 1

It can be done manually in 3 steps:

1. Find the locations and lengths of all areas that are not in use by the encrypted filesystem. I haven't quickly found an easy way to get these. A possibility is to extract it by inverting the block map in the output of partclone.btrfs -D (+ marks used blocks).
2. Add the LUKS data offset to all locations to get the locations in filesystem.img. If you partitioned the virtual disk you further need to add the partition offset. For LVM, you need to check the LVM metadata and apply its mapping to each LVM extent.
3. Run fallocate --punch-hole with --offset and --length as determined in step 2. Repeat for each area.

Do not include all-zero blocks that are in use as reading from a sparse block through the LUKS layer does not produce a block all all zeros (or is extremely unlikely to do so).

Note that cryptsetup luksDump reports the payload offset in units of 512 bytes.

Warning: partclone issue "domain mapfile is missing essential block(s) for btrfs"

Resources:

• Convert partclone domain mapfile to fallocate --punch-hole shell script: https://gist.github.com/jowagner/4fa63bfedf0b3d9389550995505871c9
• partclone -D to list used blocks: https://github.com/Thomas-Tsai/partclone/issues/174

Answer 2

The problem here is that you've used a raw disk image file to store your filesystem. This severely limits the means by which you could make this file sparse again.

The tool virt-sparsify can make the file sparse, but requires that it is not in use, so you would have to shut down the VM that uses the image before running virt-sparsify on it.

In future, if you have a need to discard blocks online, use a backing store which supports this, such as qcow2 images, or ZFS, btrfs, or LVM thin provisioning block devices.