Take the 2-minute tour ×
Server Fault is a question and answer site for professional system and network administrators. It's 100% free, no registration required.

When big files are deleted on a server, the files might still be referenced by processes, so the file system doesn't have more free space.

I tried to use lsof, but it seems it didn't list the deleted files. Fuser -c did better work, but the list of processes is just too long to check it out for each process, especially since each process is an Oracle process.

bash-3.2# fuser -c /var
/var:      105o   29999o   20444c    3528c   27258o    7715o    3864o    3862o    2494o   18205o   17450co   17445co   14912co   14824co   14818co   14816o   14814o    8532c    8530c    7633com    7118o    6958o    6790c    6784co    6734o    6693o    6689o    6684o    6675o    6635o    6594c    6548o    6547o    6546o    6545o    6544o    6543o    6542o    6541o    6540o    6537o    6535o    6456o    6128co    6113o     335o     245co     229o     161o       8o
bash-3.2# du -hs /proc
 139T   /proc

It happens sometimes that a file gets deleted by an application or a user, e.g. a logfile and that this file is still being referenced by a process that cannot be restarted.

Are there goods methods to reclaim disk space on deleted files without restarting the process that has a reference to this deleted file?

share|improve this question
    
for reference.. a better way to delete a opened file is to copy /dev/null into the file cp /dev/null file –  Mike Apr 23 '13 at 19:43
    
@Mike cp /dev/null is a null command as cp has nothing to copy, a simple redirection is strictly equivalent :>file or even >file –  jlliagre Apr 23 '13 at 21:47

3 Answers 3

Here is a simple exemple with less:

let's assume we have a file my10MBfile

$ dd if=/dev/zero of=/tmp/my10MBfile bs=1M count=10
10+0 enregistrements lus
10+0 enregistrements écrits
10485760 octets (10 MB) copiés, 0,0454491 s, 231 MB/s

$ ls -l /tmp/my10MBfile
-rw-r--r-- 1 max max 10485760 avril 23 22:49 /tmp/my10MBfile

$ df -m /tmp
/dev/disk/by-uuid/6835b2fd-971d-420c-ba18-3c729ec2e8a0     14637  9225       4662  67% /

Now i open that file with less (yes it is a binary file... nevermind)

$ less /tmp/my10MBfile &

$ lsof -p $(pidof less) | grep 10MBfile
less    29351  max    4r   REG    8,3 10485760 521464 /tmp/my10MBfile

Then I remove that file

$ rm /tmp/my10MBfile

$ lsof -p $(pidof less) | grep 10MBfile
less    29351  max    4r   REG    8,3 10485760 521464 /tmp/my10MBfile (deleted)

$ df -m /tmp
/dev/disk/by-uuid/6835b2fd-971d-420c-ba18-3c729ec2e8a0     14637  9225       4662  67% /

Still there but deleted, look at the 4th column of the lsof output: File Descriptor number 4 open for Reading (4r)

let's running GDB!

$ gdb -p $(pidof less)

GNU gdb (GDB) 7.4.1-debian
....
Attaching to process 29351
....

(gdb) p close(4)
$1 = 0
(gdb) q

That's it!

$ df -m /tmp
/dev/disk/by-uuid/6835b2fd-971d-420c-ba18-3c729ec2e8a0     14637  9215       4672  67% /

Our 10MB are welcome back :)

$ ls /proc/29351/fd
0  1  2  3

$ ps 29351
29351 pts/0    S+     0:00 less /tmp/my10MBfile

Process is still running.

share|improve this answer
1  
Ok, but for how long? many processes simply exit if they can't write to their log file. –  longneck Apr 23 '13 at 21:48
    
Logrotate can't do this for you? –  maxxvw Apr 24 '13 at 6:16
    
logrotate sends the process a signal to close the log file and open a new one. –  longneck Apr 24 '13 at 11:29

This command will show all deleted files still open on a Solaris system:

find /proc/*/fd -type f -links 0

You can truncate the ones you are sure you want with this command:

:> /proc/p/fd/x

with p being the process id and x the file descriptor returned by the first command.

Don't worry if with some programs the size reported by ls is restored to the size before truncation after a while, the actual size used on disk will be much smaller as the file is now a sparse one.

share|improve this answer

You can try to go to /proc//fd directory and the truncate corresponding file descriptor. Let say fd=3 points to deleted file of pid == 123:

# echo "" >! /proc/123/fd/3
share|improve this answer
    
do you have an exemple where this method works? Cannot find a way to alter FD from there –  maxxvw Apr 23 '13 at 21:15
    
Yes, this works, but how to find the pid of the process? –  ujjain Apr 23 '13 at 21:25

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.