Take the 2-minute tour ×
Server Fault is a question and answer site for professional system and network administrators. It's 100% free, no registration required.

I'm not close to being well versed in using nginx or varnish but this is my setup at the moment.

I have a node.js server running which is serving either json, html templates, or socket.io events. Then I have nginx running in front of node which is serving all static content (css, js, etc.).

At this point I would like to cache both static content and dynamic content to memory.

It's to my understanding that varnish can cache static content quite well and it wouldn't require touching my application code. I also think it's capable of caching dynamic content too but there cannot be any cookie headers?

I do use redis at the moment for holding session data and planned to use it for other things in the future like keeping track of non-crucial but fun stats.

I just have no idea how I should handle caching everything on the site. I think it comes down to these options but there might be more:

  1. Throw varnish in front of nginx and let varnish cache static pages, no app code changes. Redis would cache dynamic db calls which would require modifying my app code.

  2. Ignore using varnish completely and let redis handle caching everything, then use one of the nginx-redis modules. I'm not sure if this would require a lot of app code changes (for the static files).

I'm not having any luck finding benchmarks that compare nginx+varnish vs nginx+redis and I'm too inexperienced to bench it myself (high chances of my configs being awful).

I'm basically looking for the solution that would be the most efficient in terms of req/sec and scalable in the future (throw new hardware at the problem + maybe adjust some values in a config = new servers up and running semi-painlessly).

share|improve this question

1 Answer 1

For the purely static content, you normally gain very little advantage by running Varnish in front of any mature web server such as nginx or Apache. (I can't speak for node.js because I have not yet used it.) The reason for this is that the kernel maintains a filesystem cache which stores recently accessed files in RAM. It's possible that Varnish does a slightly better job than the kernel at choosing exactly which files to keep in the cache and which to eject but it's also possible that it does a worse job. This will depend on what else your system does as well as the differences in the cache retention policies. The extra latency caused by having a proxy in front of your web server will far exceed any differences between Varnish and kernel filesystem caching.

You are correct about Varnish not caching responses sent with a Set-Cookie: header. It also skips looking at the cache if the request contains a Cookie: header. The reason for this is that there will be one such unique response for each and every user who visits any given page and each user is unlikely to re-visit the same page meaning that your cache would be full of entities that will never be used.

Varnish can cache dynamic content and this is where it shines. Let's say the front page of your website requires a PHP code compile and 20 database queries on cold caches. On warm caches (APC, memcached, Redis, MySQL query cache, etc.) it still requires lookups to memcached/Redis and doing a stat() on every PHP file included in the request. Assuming you have a reasonably well optimised site, this is still likely to be at least 1/10th of a second (and that's pretty well optimised; 1 - 3 whole seconds is far more common in my experience). Varnish will happily serve the same page in under 1/1000th of a second. But it means that your users can't be logged in on the front page of your website because they can't get their Cookie: headers but if this is acceptable to you, Varnish is an easy win.

Varnish also requires correct caching headers. If it isn't sure that it's safe to cache an object, it won't. If you meet all these requirements, Varnish might be the tool for you. That said, just putting the correct headers in place will cause the clients to cache the content themselves which can make a much larger difference than Varnish can.

If you lack the experience to benchmark your setup yourself, now is the time to get that experience. You are benchmarking to gauge the appropriateness of your configs. Try something and run ab over it, then change one thing and run ab in exactly the same way against that configuration. Now you know which one is "better". Rinse, repeat. (Also, always run ab twice and ignore the first result. This will make sure you only test warm caches and don't end up erroneously thinking one configuration is worse because it was tested against cold caches.)

Architecting a scalable solution for your website is a complex enough topic that it won't fit in a ServerFault answer. We can certainly help with individual parts of that (such as "How do I scale out my memcached server to a memcached cluster?") when you run into those problems.

Further reading:

I wrote an answer a while ago that had a section on finding the bottleneck when benchmarking.
Womble recently pointed me at an excellent post on finding bottlenecks.

share|improve this answer
    
Hmm thanks, so are you saying I shouldn't even bother using varnish to cache static files and instead do some nginx conf magic to set the headers of static files to include proper cache headers -- then linux will serve the cached files magically without nginx even being conf'd to cache them? Dynamic content seems like an entirely different animal. –  AntelopeSalad Apr 11 '12 at 17:00
    
That's pretty much the gist of it. But do your own benchmarking. That's the only way to be sure. –  Ladadadada Apr 11 '12 at 17:05
    
Node is capable of serving static files too but I figured nginx would be stronger. There's a logger utility for node that spits back the http code of each request. When using a pre-made static server lib, it routinely output 304 (content not modified) when it served cached static files, which is great. I'm just wondering how nginx can magically do this without it being conf'd to do so. Will linux handle this functionality by itself? That seemed a little crazy to me heh. –  AntelopeSalad Apr 11 '12 at 17:29
    
Nginx (and Apache) both set a Last-Modified header based on the modified time of static files. They also set ETag headers (usually based on the hash of the file). Clients will use the contents of these two headers in subsequent requests and if the modified time of a static file is not newer or if the ETag is the same then they will send a 304 response. This behaviour is configurable but is the default. –  Ladadadada Apr 12 '12 at 8:51

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.