0

OS: RHEL 6.9

Problem
My file system /dev/mapper/vg_rayruhsso-lv_root that is mounted on / has no available size.

# df -h
Filesystem            Size  Used Avail Use% Mounted on
/dev/mapper/vg_rayruhsso-lv_root
                      148G  145G     0 100% /
tmpfs                  20G     0   20G   0% /dev/shm
/dev/sda1             477M   41M  411M   9% /boot
/dev/mapper/vg_rayruhsso-lv_home
                      270G   54G  202G  22% /home

What I want to solve
Figuring out what consumed the whole space (I don't want to extend it's size, since it was extended last week from 52G to 148G). The problem is that while tracking down, I found that /root is consuming much space (128G).

# du -sh *
984K
9.3M    bin
39M     boot
4.0K    CdbsMutex
4.0K    cgroup
180K    dev
41M     etc
54G     home
215M    lib
28M     lib64
16K     lost+found
4.0K    media
0       misc
4.0K    mnt
0       net
70M     opt
du: cannot access `proc/4754/task/4754/fd/4': No such file or directory
du: cannot access `proc/4754/task/4754/fdinfo/4': No such file or directory
du: cannot access `proc/4754/fd/4': No such file or directory
du: cannot access `proc/4754/fdinfo/4': No such file or directory
0       proc
128G    root
17M     sbin
0       selinux
4.0K    srv
0       sys
5.5M    tmp
12G     u01
3.2G    usr
2.0G    var

I have checked its sub-directories and found out that the largest file/directory in there is 330M even though du -sh displays 128G.

#du -sh /root/*
4.0K    /root/anaconda-ks.cfg
4.0K    /root/Desktop
4.0K    /root/Documents
4.0K    /root/Downloads
64K     /root/install.log
12K     /root/install.log.syslog
4.0K    /root/Music
4.0K    /root/Pictures
4.0K    /root/Public
4.0K    /root/Templates
330M    /root/veritas
4.0K    /root/Videos

# cd /root/
# du -sh
128G    

What's wrong? How to investigate what file is consuming all this space?

I tried to find out the open files but I didn't return any output:

lsof | grep deleted

Any idea?

Edit:

# du -h --max-depth=1 /root | sort -h
4.0K    /root/Desktop
4.0K    /root/Documents
4.0K    /root/Downloads
4.0K    /root/.gvfs
4.0K    /root/Music
4.0K    /root/.nautilus
4.0K    /root/Pictures
4.0K    /root/Public
4.0K    /root/Templates
4.0K    /root/Videos
8.0K    /root/.abrt
8.0K    /root/.ssh
12K     /root/.dbus
16K     /root/.gnote
16K     /root/.gnupg
16K     /root/.thumbnails
20K     /root/.cache
28K     /root/.gnome2
64K     /root/.config
144K    /root/.pulse
276K    /root/.gconf
364K    /root/.local
396K    /root/.kde
330M    /root/veritas
128G    /root
  • Use du -h --max-depth=1 /root | sort -h to search for your space consuming stuff. At the moment you are skipping the dot-files and dot-directories, due to glob expansion of du -sh /root/*. Please add the modified command and output to your question. – Thomas Mar 4 '18 at 12:42
  • had a typo in the sort command. Use sort -h` instead of sort -u. – Thomas Mar 4 '18 at 12:43
  • As I said before, the largest file/directory is 330M. – Mussa Mar 4 '18 at 12:45
  • I want to send you the output but didn't due to format issue in the comment section. Anyway, all the files are under 330M – Mussa Mar 4 '18 at 12:46
  • Can't you edit your question and put the new output there including the command? – Thomas Mar 4 '18 at 12:50
1

After a bit forth and back, and a private discussion...

Use du -h --max-depth=1 /root | sort -h to search for your space consuming stuff.

At the moment you are skipping the dot-files and dot-directories, due to glob expansion of du -sh /root/*.

Now with the results including the dot-files and dot-folders from the edit, it seems the files consuming the space are directly in /root.
To get an overview of which files take most space in /root use the command as follows.

ls -alhSr /root
  • Hi Thomas! Thanks for giving the time to answer. I have run that command and the largest file is 396M. – Mussa Mar 4 '18 at 12:38
  • It doesn't have to be an actual file, it can be a bound file descriptor, that has the storage allocated and never released. Check what processes are using /root and try to restart them one by one, until you release the used storage. – Gothrek Mar 5 '18 at 11:35
  • @Gothrek: If it would have been that filedescriptor, the space would not have been seen with du and would only have been reflected by df. Also the OP ruled out open filedescriptors. – Thomas Mar 5 '18 at 18:13
  • Oh, you’re right – Gothrek Mar 6 '18 at 7:15
0

Is it possible that you've issued a delete command and it never ended because the files you were trying to delete were in use by some application?

Try running:

/usr/bin/lsof | grep deleted

The above command will display all the files which are waiting to be deleted and are being used by a process.

If such files exist, kill their process and check the capacity again.

Explanation:

On Linux or Unix systems, deleting a file via rm or through a file manager application will unlink the file from the file system's directory structure; however, if the file is still open (in use by a running process) it will still be accessible to this process and will continue to occupy space on disk. Therefore such processes may need to be restarted before that file's space will be cleared up on the filesystem.

More information on this subject can be found here.

0

You might also have a look at ncdu. It's a lot like treesize and helped me many times to locate large files and directory structures.

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.