Take the 2-minute tour ×
Server Fault is a question and answer site for professional system and network administrators. It's 100% free, no registration required.

My Ubuntu server has stopped due to a lack of disk space. I deleted some log files which has grown huge very quickly. But df -h still shows I have no space left. When I run du -sh /* I can see that I should have plenty of disk space left after deleting the logs.

I ran lsof +L1 and it brought up two files: /var/log/mail.log and /var/log/mail.err. These are two logs I had deleted. I restarted apache, postfix and mysql (mysql wont restart because of lack of disk space, it think) but still df -h shows no space.

share|improve this question

3 Answers 3

up vote 13 down vote accepted

In this case the files are probably managed by syslog. You may have rsyslog, sysklogd or syslog-ng. From memory I think rsyslog is the default, so try:

sudo service rsyslog restart

This should release the files, and allow them to actually get cleaned up. Until the last filehandle closes, the filesystem can't free up space.

(or you could do like a Windows admin and just reboot of course...)

share|improve this answer
    
thank you, the worked perfectly:) –  Mark Jun 30 '12 at 18:28
    
glad to hear :) –  Bron Gondwana Jun 30 '12 at 19:40
    
Ouch, low blow. Bad Windows admins reboot. Good ones know how to troubleshoot and correct issues well enough to keep the system up... until the monthly flood of reboot-requiring Windows Updates, anyway. :) –  HopelessN00b Jul 1 '12 at 6:57
    
@HopelessN00b - yeah, I know. Bad Windows admins or Windows admins who learned their trade 10 years ago when that was the recommended fix. Mind you, I would have wanted to be a Windows admin yesterday - no Linux leapsecond bug! –  Bron Gondwana Jul 1 '12 at 7:59

You need to restart the service that created these logs, as they have a lock on these logs. eg. If it's apache, just restart apache and usage should go down.

share|improve this answer
    
ok, I get it now. I restarted rsyslog and its working now. Thanks for the help –  Mark Jun 30 '12 at 18:28
    
You should be able to release the space by causing rsyslog to reopen its files. sudo killall -HUP rsyslogd or sudo reload rsyslog should do it. Try installing logrotate, which should periodically rotate your logs for you. –  BillThor Jul 1 '12 at 0:14
    
@BillThor just installed logrotate, works a charm, thanks for help –  Mark Jul 1 '12 at 15:47

Next time this happens you should check with fuser if the file is in active use by a process. If so either stop that process, then delete, or use cp /dev/null TARGETFILE to reduce the file-size to zero (this will not change the file-handle, and you do not have to restart anything).

share|improve this answer

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.