Take the 2-minute tour ×
Server Fault is a question and answer site for professional system and network administrators. It's 100% free, no registration required.

I have a Nagios alert about disk space on our /var/log mountpoint on a production system.

We have a catalina.out (Tomcat) file which has grown to 768 megs, and there are 171 megs left on the mount point.

Obviously logrotate should have been running on this machine, but wasn't. The standard practice for catalina.out is to use copytruncate to be able to process catalina.out without bringing down Tomcat in order to do it.

If I simply add our standard Tomcat logrotate configuration, which uses copytruncate, I assume the copy happens before the truncate, and therefore I'll run out of disk space.

The log stretches back to December last year, so it's not essential to keep all of the log's contents.

What's the safest way to get out of this situation? Purely logically speaking, I think I need to get rid of the beginning 500 megs of this file before I install the logrotate configuration, but I don't even know if that's possible.

I'm running RHEL 5.3.

share|improve this question

1 Answer 1

up vote 1 down vote accepted

The safest way is to copytruncate to another mount point.

BTW, 171 megs (and even 800M if you'll completely delete file) is not enough. Ideally, you should have about 20% of free space in case of something would need it immediately.

Also, I'd suggest to to deploy some monitoring solution (zabbix/nagios/zenoss, at least monit) and monitor disk space to prevent such issues.

share|improve this answer
    
We're using Nagios - that's what alerted us to the problem in the first place :) –  Rich Apr 27 '11 at 8:28
    
Than, threshold is too low. It should be altered earlier. –  rvs Apr 27 '11 at 8:37

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.