I'm really confused by this. To do some experiments with GlusterFS I created two virtual machines with VirtualBox, running Ubuntu 12.04, each with 10GB of storage. I wrote a script that created lots of little files in lots of folders. Each file was 100k of random data generated by:
dd if=/dev/zero of=#{name} bs=1 count=0 seek=100K
When copying the files from one machine to the other, the recipient run out of space. Which should have been impossible, since they are both the same size and I didn't store anything in them. Trying to figure out what was wrong I found a really confusing situation. This is all on the source machine.
According to df, I'm only using 26% of the space:
root@revisionist:~# df -h
Filesystem Size Used Avail Use% Mounted on
/dev/mapper/revisionist-root 9.2G 2.3G 6.5G 26% /
udev 237M 4.0K 237M 1% /dev
tmpfs 50M 292K 49M 1% /run
none 5.0M 0 5.0M 0% /run/lock
none 246M 0 246M 0% /run/shm
/dev/sda1 228M 48M 169M 22% /boot
according to du, the whole structure of files I created takes 772M:
root@revisionist:~# du -h files/
...
4.0K files/3x/ey/hs
8.0K files/3x/ey
508K files/3x
772M files/
which is clearly wrong, as just that folder contains a single 100K file:
root@revisionist:~# ls -lh files/3x/ey/hs
total 0
-rw-r--r-- 1 root root 100K May 19 11:30 3xeyhst4lnkzg5abzeasiw.txt
If I target that directory and file with du, the data is clearly wrong:
root@revisionist:~# du -h files/3x/ey/hs/
4.0K files/3x/ey/hs/
root@revisionist:~# du -h files/3x/ey/hs/3xeyhst4lnkzg5abzeasiw.txt
0 files/3x/ey/hs/3xeyhst4lnkzg5abzeasiw.txt
This led me to find the option --apparent-size which is documented as:
--apparent-size
print apparent sizes, rather than disk usage; although the apparent size is usually smaller, it may be larger due to holes in (`sparse') files, internal fragmentation, indirect blocks, and the like
and indeed this looks better:
root@revisionist:~# du -h --apparent-size files/3x/ey/hs/3xeyhst4lnkzg5abzeasiw.txt
100K files/3x/ey/hs/3xeyhst4lnkzg5abzeasiw.txt
root@revisionist:~# du -h --apparent-size files/3x/ey/hs
104K files/3x/ey/hs
but when I run it for the whole structure I end up with a surprising result:
root@revisionist:~# du -h --apparent-size files
...
104K files/3x/ey/hs
108K files/3x/ey
6.8M files/3x
11G files
How can it be 11G? What's going on? Just rounding up? but then why does it fail to fit on the other machine?