Take the 2-minute tour ×
Server Fault is a question and answer site for professional system and network administrators. It's 100% free, no registration required.

I am on a large instance on Amazon's EC2 servers. I run the df command and get:

root@db:~# df -h
Filesystem            Size  Used Avail Use% Mounted on
/dev/sda1             9.9G  9.1G  284M  98% /
tmpfs                 3.8G     0  3.8G   0% /lib/init/rw
varrun                3.8G  116K  3.8G   1% /var/run
varlock               3.8G     0  3.8G   0% /var/lock
udev                  3.8G   80K  3.8G   1% /dev
tmpfs                 3.8G     0  3.8G   0% /dev/shm
/dev/sdb              414G  957M  392G   1% /mnt
/dev/sdf               50G   12G   35G  26% /byp
/dev/sdk               99G   31G   63G  33% /backups

I then run the du command and get:

root@db:/# du -s -h /*
31G     /backups
5.5M    /bin
136K    /boot
12G     /byp
80K     /dev
5.8M    /etc
12K     /home
70M     /lib
11M     /lib32
0       /lib64
16K     /lost+found
759M    /mnt
4.0K    /opt
du: cannot access `/proc/6917/task/6917/fd/4': No such file or directory
du: cannot access `/proc/6917/fd/4': No such file or directory
0       /proc
31M     /root
7.7M    /sbin
4.0K    /selinux
4.0K    /srv
0       /sys
11M     /tmp
1.1G    /usr
114M    /var

If you notice, when you add up all the sizes on the du command output of non-mounted directories, you will not get anywhere close to 9.1G as seen in the df command.

Does this mean I have a bad disk? If so, how can I fix it?

share|improve this question

3 Answers 3

up vote 11 down vote accepted

It's entirely possible that you have a very large deleted file (or lots of little ones) that a process still has an open file handle on. The way to find them is to run

# lsof | grep "deleted"

If you see lots of lines that end with "(deleted)" then you can find the process Id that has them open and restart it. Once that happens, your disk space should return.

If this does not fix it, then I'd recommend a fsck.

share|improve this answer
1  
Awesome! That was it. I had a postgresql log that was still getting logged to. –  sheats Sep 3 '09 at 18:39
    
If you want to free up some space without restarting a daemon, rather than deleting the file, use "echo > file" instead. This will truncate the file, but because the handle is still open, it will become a sparse file. This means it will still be the same size, but will take up much less disk space. –  David Pashley Sep 3 '09 at 19:57
    
lsof +L1 can sometimes work better than that grep... –  derobert Sep 5 '09 at 6:23

There are a bunch of reasons du does not equal df. See the answers to this question.

Some are overlay mounts, lots of small files and a larger block size, and deleted files that are still in use. Overlay mounts are when you mounted a filesystem on a mount point that had files in it, so du doesn't see them.

The main difference between the two is that df just checks the superblock and trusts it, where as du scans all the files that it is able to see, and adds them up. See this IBM link for information on the superblock.

share|improve this answer

Always use the -x option with du when you are chasing problems like this. It keeps du from crossing filesystems.

share|improve this answer
    
This wouldn't have made any difference, as the OP explicitly asked for /*, so asked for the other partitions. It would have helped if something was mounted under, say, "/mnt/backups" –  David Pashley Sep 3 '09 at 19:56

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.