3

I've put Nginx in front of a php app server as a reverse proxy to cache the dynamic content and save some hits to php.

I've configured Nginx with the settings below with the hope that I will achieve the following behavior.

1) Nginx will cache 200 status code content for 20m before it attempts to refetch from the app servers

2) The cache will remain in place for up to 7 days to handle stale requests for infrequently accessed content.

3) Requests that come in 20 minutes after the content was originally cached will fire off a request to the backend for a fresh update, but will serve the stale version so the client get's an instant response.

proxy_cache_path  /var/lib/nginx/cache  levels=1:2   keys_zone=staticfilecache:512m inactive=7d max_size=15000m;
proxy_cache_use_stale timeout updating error invalid_header;
proxy_cache_valid 200 20m; 
proxy_cache_valid       404 1m;
proxy_cache_valid       any 15m;

UPDATE: After running some more tests and watching the server logs, it appears that content that is more than 20m old is not being served from the cache as stale, but instead is just building up in the cache_file_system.

Any ideas how to get Nginx to serve stale content for an extended period of time? The use case is basically caching infrequently accessed content that is expensive to generate on my app servers. Being able to serve stale items several days after they have expired in the cache when a hit finally comes in would be a great performance boost. If this isn't possible with Nginx, I'm open to other proxy/caching options (I'm only using this Nginx instance as a cache).

2 Answers 2

3

Turns out that this is not currently possible with Nginx. I ended up setting up Varnish with a long TTL (7 days) as well as a long grace setting (2 days). I then have a script hit each URL every few days and force a warming of the cache.

Given the amount of content that I need to cache, and how infrequently each item is accessed, I was at first concerned that the cost of running Varnish would be prohibitive (all that RAM!) -- which is why I was looking at Nginx's file based caching. However, Varnish does indeed have a file backed caching option that has been working very well for me. So far my cache is about 80GB and performing very nicely!

0

Those all look good. Were you trying to ask why this wasn't working?

One thing I would add is your proxy_use_stale should probably have some other HTTP error codes. I assume you'd want to continue serving stale from cache if the php is returning a 503 due to over-capacity issues.

2
  • It seems to be working (visual inspection). But I just want to verify that my understanding of the setup is correct. With regard to proxy_use_stale -- will the error keyword not catch 503's and similar?
    – erikcw
    Aug 31, 2011 at 5:18
  • No, error indicates a failure before making the request. Like if the server sends an RST after the three way handshake. After the request has been made and you want to recover you have to specify the http_XXX codes you want redirected to a new upstream.
    – polynomial
    Aug 31, 2011 at 6:34

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .