13

I have a django app and I want to to setup Varnish on a server in front of it. In another serverfault thread somebody suggested putting Nginx in front of Varnish.

Should I put Nginx in front of Varnish on the caching server? If so, should I use Nginx on the app server?

4 Answers 4

10

We are talking 1 - 3 frontend servers in total, not a large server farm with load balancing between the tiers?

Putting nginx in front of Vanish enables you to do HTTP compression on the fly. That is a performance best practice, but it could be dispensed with. (Content in Varnish is often kept uncompressed, so that ESI Includes work, and so you don't have to deal with multiple cached versions of the same object depending on Vary header / browser matching.)

Regarding nginx on the app server -- is Apache with mod_wsgi not the recommended and most common way to deploy new Django installations nowadays? I'm not aware of a compelling reason for using nginx/fastcgi over Apache/mod_wsgi for Django; but you should get advice from a Django expert.

Regarding Varnish having attractive load balancing features that nginx doesn't have, I don't see what they are? Varnish has random and round-robin balancing. nginx has round-robin, client IP and consistent hashing -- I don't see a significant benefit for Varnish? Is it VCL or Varnish' graceful config reload or something else?

For a small 1-3 server setup I guess I would just do

Varnish --> Apache / mod_wsgi / Django

or maybe

Squid --> Apache / mod_wsgi / Django

and ignore HTTP compression for simplicity, unless bandwith is expensive.

Update:

Graham Dumpleton has written a valuable comment below. He mentions a very common setup for for example a blog on a VPS, or a small web farm without caching:

nginx --> Apache / mod_wsgi / Django

This is a very good solution, for a couple of reasons:

  1. Simple setup
  2. nginx, which has high speed and minimal overhead, handles static file serving and browser connection keepalive.
  3. Django runs in Graham Dumpleton's excellent mod_wsgi, the recommended platform for Django.

The reason I didn't mention this initially is that OP seemed to require Varnish, a very high performance caching solution. The nginx / Apache / mod_wsgi combo can not do caching with a level of performance and flexibility that matches Varnish.

7
  • 2
    You could even use 'nginx --> Apache/mod_wsgi/Django' as many people do for various good reasons. Nov 5, 2009 at 20:15
  • 4
    The other thing that nginx provides in that use case which many wouldn't realise is that nginx isolates Apache from slow clients. This is because request content up to certain size isn't streamed by nginx. Instead it will buffer up request headers and content and only proxy request when it has it all. This means it only hands it off to Apache when all information needed to handle request is available. Thus Apache/mod_wsgi will be busy for least time possible and processes/threads not tied up by slow client. A measure of buffering happens on reverse as well so Apache can finish quicker as well. Nov 6, 2009 at 1:21
  • 2
    @Graham Dumpleton: It is very nice to have you here, I hope you will stay around. :-) . Regarding nginx HTTP multiplexing / smart connection handling, I completely agree.
    – user2874
    Nov 6, 2009 at 3:47
  • First and foremost, thanks Jesper and Graham for taking the time to write such a comprehensive responses. I am planning on putting a load balancing server in front of two app servers. One server will handle most of the traffic with the second being used mostly for fail over and beta testing new features.
    – Enrico
    Nov 7, 2009 at 11:28
  • Varnish is appealing because it gives me full control over which requests get sent to which backend. Varnish also health checks backend servers, failover is simple to setup and it handles slow/dead backends gracefully (if both servers die).
    – Enrico
    Nov 7, 2009 at 11:29
4

You can use nginx without varnish to proxy and cache the content.

2
  • 2
    varnish has some attractive load balancing features that nginx doesn't out of the box
    – Enrico
    Nov 5, 2009 at 11:34
  • 4
    what features for example?
    – silent
    Nov 7, 2009 at 8:57
4

I've being using Nginx, Varnish, and Apache/mod_wsgi/Django successfully. I started with the following config:

Nginx -> Apache/mod_wsgi/Django

Once I started seeing significant load on Apache, I added Varnish:

Nginx -> Varnish -> Apache/mod_wsgi/Django

I use Nginx as a kind of "URL router". Django admin requests are sent directly from Nginx to Apache. Client requests are sent from Nginx to Varnish which caches the requests from Apache and also serves "graced" items from the cache if the app servers are unavailable.

My Nginx server also serves certain static content directly (e.g. images, CSS, and javascript files).

In general, performance has been excellent. I have noticed a couple of caveats that I should mention:

  1. On a busy site, restarting Varnish can cause the load to spike on the app servers, so it's best to keep Varnish restarts to a minimum. (Varnish doesn't seem to have a "reload" like Nginx/Apache where it just rereads its VCL files). Conversely, reloading an Nginx configuration has minimal impact. For this reason, I do most of the URL rewriting and "routing" in Nginx.
  2. Varnish is easy to drop in between Nginx and Apache. If you start noticing high load on your app servers, adding varnish with even the default config can really make a difference.
  3. If you do use Varnish, you definitely need to think about how you're going to handle cache invalidation.
  4. My experience has been that Varnish handles failed backends a bit more gracefully than Nginx (as you pointed out earlier).
1
  • I've never seen varnish behind something else. Varnish is usually the load balancer that does the url router. So you've got 2 webservers and 1 proxy or 2 proxies and 1 webserver?q
    – ioan
    Jan 11, 2017 at 12:18
2

I'm using Nginx->Varnish->uWSGI->Django

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .