Take the 2-minute tour ×
Server Fault is a question and answer site for professional system and network administrators. It's 100% free, no registration required.

Is it a good practice to use NTFS Compression on a IIS Log folders and files?

I was able to go down from 20GB to 7GB by doing this. The IIS logs are per day, and have an average size of 20MB but some extreme days have 200MB.

I'm wondering if IIS has to open the whole file in memory, forcing NTFS to unzip 20MB (or 200MB in extreme case) each time? Or is there some magic that allows IIS to append content? What's the system impact? Could it become a problem if we grow our traffic?

Should I split them per hour instead of per day?

Any official Microsoft paper on this? I could not locate one.

share|improve this question
2  
If you need to maintain the logs long term why not move them? Keep the current day's logs on the server and move/archive the rest somewhere else. What are you using the logs for anyway? –  joeqwerty Feb 7 '13 at 18:29
    
Moving is adding another process that can fail. I was trying to KISS. –  Malartre Feb 7 '13 at 22:16

2 Answers 2

up vote 8 down vote accepted

As Evan already gave a general answer, I like to address two of your sub-questions:

Does IIS flushes logs every X minutes?

http.sys, the kernel mode part of IIS is responsible for logging and it buffers the data in memory before writing it to the log files. I'm not certain but I don't think it does the flushing every x seconds, more likely after its buffer is getting full.

Does the whole file need to be read when adding a single line?

No, NTFS writes updates to a file into its own cache and then compresses and appends the data asynchronously to the file. Writing to a compressed file is not significantly slower than to an uncompressed file.

So there should be no problem with using NTFS compression on IIS log files.

Sources:

IIS 7 Resource Kit, Chapter 15:Logging - Microsoft Press 2008

Windows Internals 6th Edition Part2, Chapter 12: File Systems Microsoft Press 2012

share|improve this answer
    
Exactly the answer I was looking for, @peter-hahndorf! –  Malartre Feb 7 '13 at 22:48

I compress my IIS logs on a lot of IIS servers, albeit mainly servers that are hosting Outlook Web Access/App or low volume web sites. I have no problems doing it, and quite like the disk space savings.

In general, you're trading CPU for storage by making this decision. If you're CPU-bound to begin with then this probably isn't a good tradeoff. For my OWA servers, which can grow gigabytes of logs a day (thanks ActiveSync devices) I think the tradeoff is a good one.

The NTFS filesystem driver handles the compression, so it doesn't change how IIS writes to the files.

Edit:

You are, potentially, also trading-off some I/O bandwidth and IOPS, too. If you're a high enough volume that your log writes are a significant consumption of I/O resources you could see a decline in I/O consumption from enabling compression, too.

The only way you're going to tell how this impacts you is to benchmark it yourself. Take a baseline with compression disabled and then enabled and compare them. There's no magic wand to wave to know how it will impact you-- there's just too many non-deterministic factors in play.

share|improve this answer
1  
Beat me to it by a few seconds... don't you have work to do, instead of playing ServerFault? :p Anyway, +1. I've done the same thing for ages and yet to have a problem as a result of it. –  HopelessN00b Feb 7 '13 at 18:33
    
This answer is anecdotal and fun but I'm looking for more facts. I'm looking at specifics, like if IIS flush logs every X minutes and if it needs to read the whole file just to add one line. –  Malartre Feb 7 '13 at 18:49
    
@Malartre It's not anecdotal, really. As mentioned, NTFS file/folder compression is a tradeoff - disk space for CPU cycles (and a very minor memory use increase). That's about as specific and factual as is possible without doing benchmarking and running actual tests on your specific environment. –  HopelessN00b Feb 7 '13 at 20:56

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.