13

Is it a good practice to use NTFS Compression on a IIS Log folders and files?

I was able to go down from 20GB to 7GB by doing this. The IIS logs are per day, and have an average size of 20MB but some extreme days have 200MB.

I'm wondering if IIS has to open the whole file in memory, forcing NTFS to unzip 20MB (or 200MB in extreme case) each time? Or is there some magic that allows IIS to append content? What's the system impact? Could it become a problem if we grow our traffic?

Should I split them per hour instead of per day?

Any official Microsoft paper on this? I could not locate one.

Improve this question
2
  • 2
    If you need to maintain the logs long term why not move them? Keep the current day's logs on the server and move/archive the rest somewhere else. What are you using the logs for anyway?
    – joeqwerty
    Feb 7, 2013 at 18:29
  • 1
    Moving is adding another process that can fail. I was trying to KISS.
    – Malartre
    Feb 7, 2013 at 22:16

2 Answers 2

Reset to default
10

As Evan already gave a general answer, I like to address two of your sub-questions:

Does IIS flushes logs every X minutes?

http.sys, the kernel mode part of IIS is responsible for logging and it buffers the data in memory before writing it to the log files. I'm not certain but I don't think it does the flushing every x seconds, more likely after its buffer is getting full.

Does the whole file need to be read when adding a single line?

No, NTFS writes updates to a file into its own cache and then compresses and appends the data asynchronously to the file. Writing to a compressed file is not significantly slower than to an uncompressed file.

So there should be no problem with using NTFS compression on IIS log files.

Sources:

IIS 7 Resource Kit, Chapter 15:Logging - Microsoft Press 2008

Windows Internals 6th Edition Part2, Chapter 12: File Systems Microsoft Press 2012

Improve this answer
4
  • Exactly the answer I was looking for, @peter-hahndorf!
    – Malartre
    Feb 7, 2013 at 22:48
  • Interestingly, this article by Microsoft recommends otherwise: If you run a program that uses transaction logging and that constantly writes to a database or log, configure the program to store its files on a volume that is not compressed. If a program modifies data through mapped sections in a compressed file, the program can produce "dirty" pages faster than the mapped writer can write them. Question is, what is the definition of constantly?
    – Zero3
    Oct 30, 2016 at 22:36
  • 1
    @Zero3 Transaction logging is something a bit different from IIS logs. That would be where a function inside your program is actually not allowed to return with success until a transactional change has been made durable by having been written to disk, so application performance is directly related to your disk write speed for your transaction logs.
    – NReilingh
    Apr 11, 2018 at 19:52
  • @NReilingh You might be right. I actually don't know whether IIS writes to its log files sync/async. Either way, I think the overall point that the article raises (there are other examples in it, like user folders with a lot of reads and writes) is that heavy IO might be an issue with compressed folders.
    – Zero3
    Apr 24, 2018 at 17:51
13

I compress my IIS logs on a lot of IIS servers, albeit mainly servers that are hosting Outlook Web Access/App or low volume web sites. I have no problems doing it, and quite like the disk space savings.

In general, you're trading CPU for storage by making this decision. If you're CPU-bound to begin with then this probably isn't a good tradeoff. For my OWA servers, which can grow gigabytes of logs a day (thanks ActiveSync devices) I think the tradeoff is a good one.

The NTFS filesystem driver handles the compression, so it doesn't change how IIS writes to the files.

Edit:

You are, potentially, also trading-off some I/O bandwidth and IOPS, too. If you're a high enough volume that your log writes are a significant consumption of I/O resources you could see a decline in I/O consumption from enabling compression, too.

The only way you're going to tell how this impacts you is to benchmark it yourself. Take a baseline with compression disabled and then enabled and compare them. There's no magic wand to wave to know how it will impact you-- there's just too many non-deterministic factors in play.

Improve this answer
3
  • 1
    Beat me to it by a few seconds... don't you have work to do, instead of playing ServerFault? :p Anyway, +1. I've done the same thing for ages and yet to have a problem as a result of it.
    – HopelessN00b
    Feb 7, 2013 at 18:33
  • This answer is anecdotal and fun but I'm looking for more facts. I'm looking at specifics, like if IIS flush logs every X minutes and if it needs to read the whole file just to add one line.
    – Malartre
    Feb 7, 2013 at 18:49
  • @Malartre It's not anecdotal, really. As mentioned, NTFS file/folder compression is a tradeoff - disk space for CPU cycles (and a very minor memory use increase). That's about as specific and factual as is possible without doing benchmarking and running actual tests on your specific environment.
    – HopelessN00b
    Feb 7, 2013 at 20:56

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .