Varnish load-balance with per-server request rewriting

Question

I'm trying to load-balance a 3rd-party web application with Varnish, and this 3rd-party application requires the 'real' server name (it appears to connect to this server by name in the background (and localhost doesn't work :() to be passed in the query string. I also need failover, which means I would like to use directors so my configuration can be simple and will scale.

Here's a basic example of what I'd like to do:

sub vcl_pass {
  set bereq.http.X-Varnish-Backend = bereq.backend.name
}

However backend.name is only available in beresp after we've already made the request. It appears that Varnish doesn't decide which backend to use until after vcl_pass finishes, and there's no other hook before the request is sent to the backend.

Is this correct? If so, is there another solution (other than fixing it on the web server, which I don't control)? Without loops, even if I implement a replacement client director in VCL, it's not going to scale beyond a couple servers.

Answer 1

First of all you have to made a strategy to differentiate between the used applications. You can use different host names (all of them point to your Varnish proxy server) or you can use different URLs to decide which request should be processed by which back-end system. If you have your strategy, set-up your Varnish:

Example A) Different Host Names

Prepare all required backends. It can be a single server or multiple servers by using a director:

backend example1 {
  .host = "...";
  .port = "...";
  ...
}

Set the correct back-end for incoming requests (in Varnish: req). I set an additional value req.http.backend, to use it in a later step. Here you can replace or unset any HTTP header of the client request:

sub vcl_recv {
  ...
  if (req.http.host == "example1.mysite.com") {
    set req.backend = example1;
    set req.http.backend = "example1";
    set req.http.host = "application1.internal.mysite.com";
    unset req.http.Cache-Control;
    ...
  }
  ...
}

Make some modifications with the returned back-end response (in Varnish: beresp). If you have a diffent host name for an application you might want to change redirects. Usually I remove some X-headers or replace caching headers of the back-end here:

sub vcl_fetch {
  ...
  if (req.http.backend == "example1") {
    ...
    unset beresp.http.X-Powered-By;
    ...
    if ((beresp.status == 301 || beresp.status == 302) && beresp.http.Location ~ "^http://application1\.internal\.mysite\.com") {
      set beresp.http.Location = regsub(beresp.http.Location, "^http://[^/]+", "http://example1.mysite.com");
    }
  }
  ...
}

Example B) Different URLs

Within Varnish the difference is in the method vcl_recv. Take care that your application is able to work with the pathname "/example1". Other solutions are possible but I wouldn't recommend it:

sub vcl_recv {
  ...
  if (req.http.host == "www.mysite.com") {
    if (req.url ~ "(?i)^/example1/") {
      set req.backend = example1;
      set req.http.backend = "example1";
      set req.http.host = "application1.internal.mysite.com";
      unset req.http.Cache-Control;
      ...
    }
    ...
  }
  ...
}

You can implement A or B or combine A and B. (More or less) Everything is possible with Varnish.