I have a 2TB USB hard drive that I use as a backup disk. The disk contains a GPT partition table with one partition, type bf00. On that partition, I created a ZFS pool with encryption and compression enabled, and one single dataset.
While I rsyncing my files to the disk, I noticed that the total size of the mounted dataset got smaller and smaller (please note: this is the weird part, it really is the total size, not the available size). How can that be? And how can I use the full capacity?
This is the output of
df -h, the total size is already down to 1.2T (rsync is still copying at the moment):
backup/DATA 1,2T 380G 834G 32% /backup
# zpool list NAME SIZE ALLOC FREE CKPOINT EXPANDSZ FRAG CAP DEDUP HEALTH ALTROOT backup 1,81T 964G 892G - - 3% 51% 1.01x ONLINE -
And this is
# zfs list NAME USED AVAIL REFER MOUNTPOINT backup 973G 832G 98K none backup/DATA 381G 832G 381G /backup
So it seems there is about one third of the capacity missing, how can that be? Can I reclaim the space somehow? And where did it go? I'm using Arch Linux (5.3.8-arch1-1) with zfs-dkms 0.8.2-1.
BTW: I'm not talking about the 2 TB vs 1.8 TebiByte issue, this is something else.
Here's the output of zpool status:
zpool status pool: backup state: ONLINE scan: none requested config: NAME STATE READ WRITE CKSUM backup ONLINE 0 0 0 BackupDisk1 ONLINE 0 0 0 errors: No known data errors
zfs list -o space NAME AVAIL USED USEDSNAP USEDDS USEDREFRESERV USEDCHILD backup 793G 1011G 0B 98K 0B 1011G backup/DATA 793G 422G 0B 422G 0B 0B
Ok, I left the system to itself over night just to see what would happen. When I last looked, the figures were like above, the total space of the dataset backup/DATA was shrinking while copying some hundred GB onto it. And even when rsync finished, the drive was busy (as indicated by the LED). There was also a large background CPU usage.
When I took a look this morning, the total size of backup/DATA was back at 1.8TB and all background work has obiously finished. Tadaa! :-)
I think what might have happened is this: rsync was throwing a large amount of files at the dataset. ZFS seems to receive and kind of buffer the files that need to be written. This buffer probably shrinks the total usable size while it exists. As I have compression and encryption enabled on the pool resp. dataset, this may have taken a while (long after rsync finished), even on my quite decent workstation (12 cores, 32 GB RAM), perhaps because the USB drive really isn't fast.
Can somebody confirm that this (or something in that direction) is what happens? I think it would be good to know for everyone who runs into a similar issue.