I have a server which is running Ubuntu 18.04 and it is also a worker node for K8s. Sometimes I see that K8s is killing pods on this machine because of disk presseur and when I get
df -h --total I can see that 85% (1.5T) of the disk is in use at
~$ df -h --total Filesystem Size Used Avail Use% Mounted on udev 126G 0 126G 0% /dev tmpfs 26G 5.3M 26G 1% /run /dev/sda2 1.8T 1.5T 276G 85% / tmpfs 126G 0 126G 0% /dev/shm tmpfs 5.0M 0 5.0M 0% /run/lock tmpfs 126G 0 126G 0% /sys/fs/cgroup /dev/loop0 90M 90M 0 100% /snap/core/7917 /dev/loop1 90M 90M 0 100% /snap/core/8039 /dev/sdb1 9.8G 203M 9.1G 3% /boot /dev/sdb2 511M 6.1M 505M 2% /boot/efi /dev/sdb3 1.8T 100M 1.7T 1% /home /dev/loop2 128K 128K 0 100% /snap/austin/42 /dev/loop3 3.0M 3.0M 0 100% /snap/micro/648 tmpfs 26G 0 26G 0% /run/user/1001 total 4.0T 1.5T 2.4T 38% -
The problem is when I go to
/ and get
sudo du -BG -s * I can just find 313G of that used data and nothing more:
/$ sudo du -BG -s * 1G bin 1G boot 0G dev 1G etc 1G home 0G initrd.img 0G initrd.img.old 1G lib 1G lib64 1G lost+found 1G media 1G mnt 1G opt du: cannot access 'proc/22512/task/22580/fdinfo/20': No such file or directory du: cannot access 'proc/45752/task/45752/fd/4': No such file or directory du: cannot access 'proc/45752/task/45752/fdinfo/4': No such file or directory du: cannot access 'proc/45752/fd/3': No such file or directory du: cannot access 'proc/45752/fdinfo/3': No such file or directory 0G proc 1G root 1G run 1G sbin 1G snap 1G srv 9G swap.img 0G sys 1G tmp 3G usr 313G var 0G vmlinuz 0G vmlinuz.old
How I can find the rest of the data and solve the disk pressure problem?
My problem/question was different from the suggested solution. In that case the problem was deleted files but my problem is docker. I post an answer so I can close this question.