Take the 2-minute tour ×
Server Fault is a question and answer site for professional system and network administrators. It's 100% free, no registration required.

df -h reports the '/' partition to be 100% full. While running du -hs * on each directory of this partition shows there is still lot of space.

tune2efs which reports just one block to be free. Ran fsck as well, which also shows all blocks being used.

ionode use is 14% on the '/' partition.

Please note that /var, /usr, /dev, /tmp, /home are mounted on different partitions and have space available in them.

Can you please let me know the possible causes for all blocks to be occupied and disk reporting to be full even tough there is a lot of space.

share|improve this question
1  
what log files did you delete recently? –  Mike Mar 7 '13 at 6:22
    
Try du -shx / and see if that still says there's plenty of room. That will evaluate all of / without relying on you catching all of the possible directories. –  Insyte Mar 7 '13 at 6:53
    
du -shx / 188M The '/' partition is of 2GB. –  Ankit Mar 7 '13 at 7:04

4 Answers 4

up vote 13 down vote accepted

As well as the open files problem that commonly causes otherwise free space to be held unavailable, a not uncommon problem is files shielded by mount points. For instance if you have /tmp as a separate logical volume but still have files in the directory /tmp in the root filesystem, those files will be consuming space but will be hidden by the mount.

Try dropping into single-user mode at boot so nothing is running that might hold mounts open, unmount everything, and make sure there are no files hiding in the directories that are normally used as mount points.

share|improve this answer
1  
Thank You David for the Idea. I went working on this direction. Found out that a USB HDD was attached to the machine. The machine was rebooted in the morning. It seems that the HDD was not detected at boot up and some its data went to the '/' partition. I figured this when i unmounted /tmp and the external drive mount point. –  Ankit Mar 7 '13 at 12:21

This is often caused by having a file open for writing that has been deleted but the process writing to the file hasn't been restarted thus relinquishing the file. You can use lsof to find files that are open but unlinked (deleted)

lsof +L1

should do the trick. As the man page states:

A specification of the form +L1 will select open files that have been unlinked. A specification of the form +L1 <file_system> will select unlinked open files on the specified file system.

share|improve this answer
    
lsof +L1 gives nothing. i.e. no open unlinked file. Anything else I can check. –  Ankit Mar 7 '13 at 6:50

If a file is deleted while another process holds it open, that process can continue to write and, eventually, invisibly fill the disk. As soon as the process holding open the file exits, the blocks are made available.

Try evaluating each of the running daemons. If possible, restart them. If you can't figure it out, rebooting the box should clear it up.

share|improve this answer
    
Rebooted the system, ran fsck, nothing helped. –  Ankit Mar 7 '13 at 6:50

It's probably because some percentage of the available space is reserved for the root user.

See Reserved space for root on a filesystem - why? or Disk full on linux server, blocks used is much less then blocks avalable

share|improve this answer

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.