How to choose a correct value for open_file_cache in nginx configuration?

Question

I have a website with load of about 60 000 requests per minute served by nginx. Recently, I have enabled the open file cache and saw a dramatic improvement in performance. But in the evenenings, when loads is maximum, response times are still quite large and nginx uses a lot of IO.

Here are my current settings:

open_file_cache max=10000 inactive=30s;
open_file_cache_valid    60s;
open_file_cache_min_uses 2;
open_file_cache_errors   on;

And I still have some memory left:

free -m
             total       used       free     shared    buffers     cached
Mem:          7910       6917        992          0        439        885
-/+ buffers/cache:       5592       2317
Swap:         8099          0       8099

Question: now do I choose correct value for open_file_cache max=10000 and other parameters? Is 10000 enough, is it too small or big? Is there a way to monitor open file cache use?

Answer 1

The open file cache is a caching system for metadata operations (file mtime, file existence etc), not for file content, so it helps but not as much as you would expect.

Some workarounds you could try are:

• Lower the gzip compression ration if you have turned dynamic gzip on.
• If you are running a sufficiently recent kernel (> 2.6.22) try turning aio on. Check http://wiki.nginx.org/HttpCoreModule#aio
• Disable access logging. This alone can give you a very high performance jump. Make sure you don't need it though!
• Try an actual file content caching module such as https://github.com/bpaquet/ngx_http_enhanced_memcached_module. Warning: this will require recompiling nginx from source!

Answer 2

You could also try buffers as nginx won't write any temporary files when you have sufficient buffers