3

today, I noticed during backup that docker 17.05 creates two very large files in /var/lib/docker/devicemapper/devicemapper/ on Debian Stretch.

-rw------- 1 root root 100G Dez  8 22:50 data
-rw------- 2 root root 2,0G Dez  7 01:22 metadata

My problem: My server has a 120 GB SSD and the / partition is

root@server:/# df -h /
Dateisystem                 Größe Benutzt Verf. Verw% Eingehängt auf
/dev/mapper/server--vg-root   71G     32G   37G   47% /

So how is that possible? And why? This files consumes huge amounts of my backup and I deleted it. Docker daemon won't start without it.

I have one single app running in docker: PlexMediaServer with this script: https://hub.docker.com/r/linuxserver/plex/

And yes there is a volume containing all my music and video files. But thats on the RAID, not on the system drive.

When restoring the file from backup, is takes hours and I'm pretty sure it will fill up my SSD completely and then stop.

2 Answers 2

7

That file is known as a sparse file. Here is an excerpt from the wikipedia definition:

In computer science, a sparse file is a type of computer file that attempts to use file system space more efficiently when the file itself is mostly empty. This is achieved by writing brief information (metadata) representing the empty blocks to disk instead of the actual "empty" space which makes up the block, using less disk space. The full block size is written to disk as the actual size only when the block contains "real" (non-empty) data.

It is not actually consuming the full 100GB of disk space. What is being displayed by ls is the "apparent size". To view the amount of disk space it is actually consuming, you can use the du command.

# ls -lh /var/lib/docker/devicemapper/devicemapper/data 
-rw------- 1 root root 100G Dec 13 15:00 /var/lib/docker/devicemapper/devicemapper/data
# du -sh /var/lib/docker/devicemapper/devicemapper/data 
43G     /var/lib/docker/devicemapper/devicemapper/data

The docker documentation contains information on best practices for backing up volumes.

3
  • du -shsays for my restored rsnapshot backup that it's size is 101G. Maybe rsnaphot can't handle sparse files? I actually don't need to backup my volumes as the data inside the olumes is already backed up twice. Dec 16, 2017 at 8:55
  • @Corni rsync, which rsnapshot uses, requires the --sparse option to efficiently backup sparse files.
    – jordanm
    Dec 16, 2017 at 18:57
  • Thanks. Is there a way to let docker 'recreate' these files? As I restored them from backup, they are no real sparse files anymore. I already deleted any old image and container but it did not change the data sparse files size. Dec 18, 2017 at 8:19
-1

I've solved the problem:

mv /var/lib/docker /raid/docker
rm -r /var/lib/docker
ln -s /raid/docker /var/lib/docker

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .