Take the 2-minute tour ×
Server Fault is a question and answer site for professional system and network administrators. It's 100% free, no registration required.

We have a custom Windows Service in our production environment that for auditing and diagnostic reasons generates log files in the file system. The log files are quite verbose (by necessity) and produce about 100 Mb of output daily. The log contents must be retained for a 12-month period, but are highly compressible.

We are considering enabling O/S compression on this folder to save on space - which is already at a premium on this server.

Are there any potential pitfalls or problems that we may encounter by doing this? Are there any performance, compatibility, or stability issues to be consider? If there are, how can we determine whether this is a good idea for us?

share|improve this question
add comment

3 Answers

Personally I wouldn't configure any "live" logging to log to a compressed or encrypted folder. I would create an archive folder and move the log files from the application's log folder to this compressed archive folder after the application is done writing to them. I'm assuming that the log files "roll" on a daily basis, so today's log file is active and being written uncrompressed to the application's log folder and when the application starts a new log file, the previous log file is moved to the compressed folder.

share|improve this answer
    
Is there a specific reason you don't like logging to a compressed folder? –  LBushkin Dec 6 '10 at 21:20
    
I think that you're risking putting an unneccessary load on the application and the server by having it write "live" files to a compressed folder. –  joeqwerty Dec 6 '10 at 22:11
add comment

I've personally done this on the one Windows server I was in charge of. I think it's a good idea. As you mentioned, the repetitive plain text in log files compresses VERY well. The overhead involved with compression seemed pretty small. I used it for firewall logs, and easily dumped out 100MB+ per day. (although I had it broken into 20MB files) Unless your CPU is pegged to begin with, I don't think this would be a problem. You'll essentially trade a little bit of CPU power, for a great deal of disk space. Sometimes, that's an extremely good tradeoff. Other times, notsomuch.

Obviously, testing is a good idea. But I didn't run into too many troubles. Just be advised the compression isn't transferrable, like an old fashioned zip file. So if you move this via FTP/CIFS/etc, you're moving the uncompressed amount. If you copy them outside of the compressed folder, you're moving the uncompressed amount. If you use backup software, you're backing up the uncompressed amount. Etc.

Might be worth noting there is the initial compression when you actually flip the flag for this folder. So you might want to compress a subset at a time, so you don't bring your server to it's knees while it compresses your logs for an hour. This may be significant depending on how much log info you have, and how important your application is. But all that being said, you can always reverse the process, and uncompress the directory just as easily. So if you find performance is too crappy, then you can always* flip it back.

As others have mentioned, compressing just the long term archived logs isn't a bad idea. If you wanted to put a bit more effort into the solution, you could always automate a zipping process on those logs, and spare yourself doing it through the OS.

*Nothing "always" works in IT. ;)


--Christopher Karel

share|improve this answer
add comment

If you have no heavy CPU load already, you will probably benefit from compressing your log files.

share|improve this answer
add comment

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.