Take the 2-minute tour ×
Server Fault is a question and answer site for professional system and network administrators. It's 100% free, no registration required.

I have a huge log file I need to delete on a production web server. I'm worried it'll bring the system to a crawl if I rm it on Linux. Any brilliant ideas?

Update:

Filesystem: ext3

Partition: /var (mostly logs and MySQL data)

Log file is no longer being written to. (No additional data is being appended)

Web Server is LAMP (lots of IO)

share|improve this question
1  
Duplicate of serverfault.com/questions/128012/… ? What is your underlying file system? –  Ben DeMott Dec 2 '11 at 5:55
    
ext3 - hoping for something more inspiring that ionice might work... I read the man page (linux.die.net/man/1/ionice) and it sounds like the right tool... I'd like a someone with experience using it to chime in. I don't feel very adventurous when it comes to production servers. –  objectoriented Dec 2 '11 at 6:06
    
Did any of these solve the problem? –  ewwhite Mar 5 '12 at 2:14
    
No, I've delayed dealing with the problem so far. I'll update this topic after I try one of the suggested methods. I'm leaning towards zeroing the file. –  objectoriented Mar 6 '12 at 18:01
add comment

2 Answers 2

up vote 8 down vote accepted

It may be faster to zero/truncate the file than remove it. I also mention this because that's a really large log file, so there must be a tremendous amount of process activity writing to it. Try : > /path/to/logfile.log if you're not in a position to stop and start the production services.

share|improve this answer
    
Thanks! I was able to stop the process writing to the file so no additional data is being appended to the file. –  objectoriented Dec 2 '11 at 15:58
    
Go ahead and zero it. –  ewwhite Dec 2 '11 at 16:25
add comment

ionice -c3 rm yourfile.log is your best shot, then rm will belong to idle I/O class and only uses I/O when any other process does not need it. ext3 is not stellar when deleting huge files and there's not very much you can do about it. Yes, the rm command will slow down your system. The amount of slowness and the duration of the deletion is something one can only guess, it depends so much on hardware, kernel version and ext3 file system creation settings.

For log servers and other servers with large files I tend to use XFS, as it is very fast with them.

share|improve this answer
1  
Thanks Janne. I worry there won't much idle time? I think ionice -c2 -n7 rm big.log is more appropriate. –  objectoriented Dec 2 '11 at 6:17
    
I have no idea if your server has lots of free disk I/O time or not. But which one of you is more important to you: 1) time it takes to remove the file or 2) your server stays ~smooth during deletion? –  Janne Pikkarainen Dec 2 '11 at 7:06
    
Method with least disruption is the goal. My point here is that thing is a web server (LAMP to be specific) so it's business is IO. I just don't see the disks sitting idle very much. I worry a giant file removal would take days or longer if the system is waiting for periods of zero IO. –  objectoriented Dec 2 '11 at 15:53
    
IONICE is the kicker here. I spent hours trying to figure out what was slowing down my server while deleting a substantial amount of files. "find . -type f -delete -print" was using 99.9% of my I/O and other processes were stacking up because they could not complete their I/O. –  NickWoodhams Aug 15 '12 at 11:53
add comment

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.