3

We have a custom Windows Service in our production environment that for auditing and diagnostic reasons generates log files in the file system. The log files are quite verbose (by necessity) and produce about 100 Mb of output daily. The log contents must be retained for a 12-month period, but are highly compressible.

We are considering enabling O/S compression on this folder to save on space - which is already at a premium on this server.

Are there any potential pitfalls or problems that we may encounter by doing this? Are there any performance, compatibility, or stability issues to be consider? If there are, how can we determine whether this is a good idea for us?

4 Answers 4

3

I've personally done this on the one Windows server I was in charge of. I think it's a good idea. As you mentioned, the repetitive plain text in log files compresses VERY well. The overhead involved with compression seemed pretty small. I used it for firewall logs, and easily dumped out 100MB+ per day. (although I had it broken into 20MB files) Unless your CPU is pegged to begin with, I don't think this would be a problem. You'll essentially trade a little bit of CPU power, for a great deal of disk space. Sometimes, that's an extremely good tradeoff. Other times, notsomuch.

Obviously, testing is a good idea. But I didn't run into too many troubles. Just be advised the compression isn't transferrable, like an old fashioned zip file. So if you move this via FTP/CIFS/etc, you're moving the uncompressed amount. If you copy them outside of the compressed folder, you're moving the uncompressed amount. If you use backup software, you're backing up the uncompressed amount. Etc.

Might be worth noting there is the initial compression when you actually flip the flag for this folder. So you might want to compress a subset at a time, so you don't bring your server to it's knees while it compresses your logs for an hour. This may be significant depending on how much log info you have, and how important your application is. But all that being said, you can always reverse the process, and uncompress the directory just as easily. So if you find performance is too crappy, then you can always* flip it back.

As others have mentioned, compressing just the long term archived logs isn't a bad idea. If you wanted to put a bit more effort into the solution, you could always automate a zipping process on those logs, and spare yourself doing it through the OS.

**Nothing "always" works in IT. ;)*


--Christopher Karel

2

Personally I wouldn't configure any "live" logging to log to a compressed or encrypted folder. I would create an archive folder and move the log files from the application's log folder to this compressed archive folder after the application is done writing to them. I'm assuming that the log files "roll" on a daily basis, so today's log file is active and being written uncrompressed to the application's log folder and when the application starts a new log file, the previous log file is moved to the compressed folder.

2
  • Is there a specific reason you don't like logging to a compressed folder?
    – LBushkin
    Dec 6, 2010 at 21:20
  • I think that you're risking putting an unneccessary load on the application and the server by having it write "live" files to a compressed folder.
    – joeqwerty
    Dec 6, 2010 at 22:11
1

I don't see any warnings from MS when the recommend compression here: https://docs.microsoft.com/en-us/iis/manage/provisioning-and-managing-iis/managing-iis-log-file-storage

1
  • Great link, please add a bit more explanation next time! E.g. this quote: "Enable Folder Compression: IIS log files compress to about 2% of their original size. Enable compression of a log file as follows. You must be an administrator to perform this procedure."
    – Ekus
    Oct 13, 2020 at 18:39
0

If you have no heavy CPU load already, you will probably benefit from compressing your log files.

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .