1

I have a Linux machine (Ubuntu with Btrfs) which runs Docker and 4 cointainers.

Docker images

# docker images
REPOSITORY                      TAG                 IMAGE ID            CREATED             SIZE
alpine                          3.8                 11cd0b38bc3c        9 months ago        4.41MB
grafana/grafana                 5.1.5               6216a7d47b26        10 months ago       238MB
quay.io/prometheus/prometheus   v2.3.1              b82ef1f3aa07        10 months ago       119MB

Docker containers

# docker ps -a
CONTAINER ID        IMAGE                                  COMMAND                  CREATED             STATUS              PORTS                              NAMES
e7fdc76b0437        alpine:3.8                             "/alertmanager/alert�~@�"   8 days ago          Up 12 minutes       0.0.0.0:9093-9094->9093-9094/tcp   monitoring_alert_manager_1
732ce2eaace7        grafana/grafana:5.1.5                  "/run.sh"                8 days ago          Up 12 minutes       0.0.0.0:3000->3000/tcp             monitoring_grafana_1
75587d6787e4        quay.io/prometheus/prometheus:v2.3.1   "/bin/prometheus --l�~@�"   8 days ago          Up 12 minutes       0.0.0.0:9090->9090/tcp             monitoring_prometheus_1
a1d57121c8ce        alpine:3.8                             "/blackbox_exporter �~@�"   6 months ago        Up 12 minutes       0.0.0.0:9115->9115/tcp             monitoring_blackbox_exporter_1

Volumes

# du -h -d 1 "/var/lib/docker/volumes/"
451M    /var/lib/docker/volumes/monitoring_prometheus
2.3M    /var/lib/docker/volumes/monitoring_grafana
4.0K    /var/lib/docker/volumes/monitoring_alertmanager
453M    /var/lib/docker/volumes/

The problem is that I cannot explain what is using much of the disk space.

# btrfs fi df /
Data, single: total=13.01GiB, used=10.82GiB
System, DUP: total=32.00MiB, used=16.00KiB
Metadata, DUP: total=639.75MiB, used=101.23MiB
GlobalReserve, single: total=16.00MiB, used=0.00B

So, around 11GB is used (also reported by df -h), but I would expect that much less should be used. There are 3 images, which take around 350MB in total and around 450MB is used for actual data, so less than 1GB + OS itself. If I try to dig where are the files that are taking up disk space - du reports that only 4.8G of the disk is used (ignoring btrfs' copy on write and 2.6G of this is /var/lib/docker).

# du -sh /
du: cannot access '/proc/1209/task/1209/fd/4': No such file or directory
du: cannot access '/proc/1209/task/1209/fdinfo/4': No such file or directory
du: cannot access '/proc/1209/fd/3': No such file or directory
du: cannot access '/proc/1209/fdinfo/3': No such file or directory
4.8G    /

I found some hints that this can be in case that some large files are open by application, but deleted. I looked in lsof and rebooted the machine, but the result is the same. Second thing could be subvolumes, but it's apparently the root one that is taking majority of disk space.

# btrfs-du /
Subvolume                                                         Total  Exclusive  ID
var/lib/docker/btrfs/subvolumes/2fddeb2724f7b7ebd816c313e67580b71f36a07386bdb126e3b269cd25d40081    1.20MiB   64.00KiB  259
var/lib/docker/btrfs/subvolumes/5652e05094bbf4b171053cc6be6626d922405f699d79be265f8487b8ce3fd643    3.93MiB  784.00KiB  260
var/lib/docker/btrfs/subvolumes/c9b5313b82c7aa1e562e681e5c3344ac770e2e69820ee6bdbe9b14bf654e472f   71.81MiB  784.00KiB  261
var/lib/docker/btrfs/subvolumes/cf656f2dd19233625e66974ca89fd295583526daf4614e3f738458f629aaf8b6  114.72MiB  784.00KiB  262
var/lib/docker/btrfs/subvolumes/93a4c2f43dc245d01715a9b3c46f268f319e2d6dba2c56a7ad6078c1a98775d9  114.74MiB  784.00KiB  263
var/lib/docker/btrfs/subvolumes/5237ab02149b30f6b45de51cee89a9bec76cdc26b2e40d3cd2e37a88b331fbae  114.76MiB  800.00KiB  264
var/lib/docker/btrfs/subvolumes/6726bf1b2712fb2ed11db8629c339fd0f0afad55a9655a5e3d6ae7ca809e4c62  114.79MiB  800.00KiB  265
var/lib/docker/btrfs/subvolumes/b4eed058bca3d31d9f118b5db82515a3aa6dabcff0fb7d564e2bda1462459ce9  114.79MiB  800.00KiB  266
var/lib/docker/btrfs/subvolumes/76a59d8518448c251436254b0f7c206f65d3ccc56e3e45d1ed571c0d9377618d  114.79MiB  800.00KiB  267
var/lib/docker/btrfs/subvolumes/c740a1bfe14870a61e7fe26da24f0279a1df1d658bcfa4cf537d355f3e7c0c83   57.26MiB    3.64MiB  268
var/lib/docker/btrfs/subvolumes/23c7fccc98ddc4aa5fa1c14f54ae26efcd667e3f7a57fdf4b0870369c85861f4  232.76MiB    3.14MiB  269
var/lib/docker/btrfs/subvolumes/fe33b8bde93cb411035756fa901091261201b86260501035084db889f3e353f4  232.78MiB    3.14MiB  270
var/lib/docker/btrfs/subvolumes/300d2b8c2dfcbbccd09d3cae14d0d80618f5ccd2b53ddacab110057bc4cf9c4f    4.47MiB  160.00KiB  322
var/lib/docker/btrfs/subvolumes/7b5e78985035cc9b7e9c5acdda898cfa661dc0aa85809aa9a2fa437df43a38f9-init    4.47MiB  160.00KiB  509
var/lib/docker/btrfs/subvolumes/7b5e78985035cc9b7e9c5acdda898cfa661dc0aa85809aa9a2fa437df43a38f9    4.47MiB  160.00KiB  510
var/lib/docker/btrfs/subvolumes/2f34bf0d5958d5d156970742d13859f82ec5990fb4e610e1155b95fb1483f3b5-init  114.79MiB  800.00KiB  699
var/lib/docker/btrfs/subvolumes/2f34bf0d5958d5d156970742d13859f82ec5990fb4e610e1155b95fb1483f3b5  114.79MiB  800.00KiB  700
var/lib/docker/btrfs/subvolumes/e202293240ca4a4c1f6b996be9ba2d044d4b3007df0c7e5a29a32a949f49ae57-init  232.78MiB    3.14MiB  701
var/lib/docker/btrfs/subvolumes/e202293240ca4a4c1f6b996be9ba2d044d4b3007df0c7e5a29a32a949f49ae57  232.78MiB    3.76MiB  702
var/lib/docker/btrfs/subvolumes/ffb1461bb86799876e58e4213483e3da2b1ffee401e975087e296015d6591972-init    4.47MiB  160.00KiB  703
var/lib/docker/btrfs/subvolumes/ffb1461bb86799876e58e4213483e3da2b1ffee401e975087e296015d6591972    4.47MiB  160.00KiB  704
Total exclusive data                                                           25.42MiB

Quotas

# btrfs quota rescan /
quota rescan started
# btrfs qgroup show /
qgroupid         rfer         excl
--------         ----         ----
0/5          10.80GiB     10.80GiB
0/259         1.21MiB     64.00KiB
0/260         3.94MiB    784.00KiB
0/261        71.81MiB    784.00KiB
0/262       114.73MiB    784.00KiB
0/263       114.74MiB    784.00KiB
0/264       114.77MiB    800.00KiB
0/265       114.79MiB    800.00KiB
0/266       114.79MiB    800.00KiB
0/267       114.79MiB    800.00KiB
0/268        57.27MiB      3.64MiB
0/269       232.77MiB      3.14MiB
0/270       232.79MiB      3.14MiB
0/322         4.47MiB    160.00KiB
0/509         4.47MiB    160.00KiB
0/510         4.47MiB    160.00KiB
0/699       114.79MiB    800.00KiB
0/700       114.79MiB    800.00KiB
0/701       232.79MiB      3.14MiB
0/702       232.79MiB      3.77MiB
0/703         4.47MiB    160.00KiB
0/704         4.47MiB    160.00KiB

Does anyone have any clue what is using disk space?

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.