2

I have a working varnish cache for my websites, with the undesired side effect that requests for a domain with http are cached with a certain url, and requests for https with another url. That way I end up with double objects in the cache, and I wanted to ask some best practices on how to optimize this behaviour.

In detail I have two vhosts within nginx for the same domain listening on port 80 and 443 each. Requests are proxy passed to varnish:

proxy_pass http://varnish:8101/VirtualHostBase/http/example.com:80/path/VirtualHostRoot/;

and

proxy_pass http://varnish:8101/VirtualHostBase/https/example.com:443/path/VirtualHostRoot/;

In varnish.vcl I check for the requesting host and set the right backend, since there are multiple ones.

if (req.http.host == "example.com") {
  set req.backend = backend_0;
}

The backend is a Zope/Plone server. The pages are cached correctly in varnish, but I have an entry for /VirtualHostBase/http/example.com:80/path/VirtualHostRoot/logo.png and one for /VirtualHostBase/https/example.com:443/path/VirtualHostRoot/logo.png in my varnishlog (RxURL).

When Plone purges an entry, only the ssl version is purged, because every logged in user has to use https. The http entry remains until age invalidation.

Is it possible to combine http and https requests into one varnish object by rewriting urls? To save space and to do a successful purge. Maybe someone can give me a hint how to solve this!

2
  • Is there any specific reason to use http at all? I would simply 301 resirect everything to https. Sep 23, 2014 at 18:03
  • Your argument is valid and I had that in mind, but there are some minor reasons (less server load, some sites don't have a certificate, self signed ones produce alerts) that let me choose the other way.
    – boernie
    Sep 29, 2014 at 16:57

2 Answers 2

2

Varnish identifies different entities by their req.url and their req.http.Host (if present, otherwise, it uses server.ip). What you want is this:

sub vcl_hash {
  #Example URL is:
  # http://varnish:8101/VirtualHostBase/http/example.com:80/path/VirtualHostRoot/
  #req.url contains only:
  # /VirtualHostBase/http/example.com:80/path/VirtualHostRoot/
  #after transform, this will become:
  # /VirtualHostBase/fakescheme/example.com:fakeport/path/VirtualHostRoot/
  hash_data(regsub(regsub(req.url,":(80|443)/",":fakeport/"),"/https?/","/fakescheme/")); #equivalent of "hash URL"

  #Below here, copied from default.vcl
  if (req.http.host) {
    hash_data(req.http.host);
  } else {
    hash_data(server.ip);
  }
  return (hash);
}
5
  • I have to replace protocol and port number, therefore doing substitution for http/https and 80/443. I've read that varnish does not allow custom variables (to use as placeholder). Can both substitutions be done within one regsub function and how? Or how to set hash_data appropriately?
    – boernie
    Sep 29, 2014 at 16:50
  • req.url doesn't contain scheme (http vs. https). Thus, you only need to run the code, above, unless I'm missing something in your question.
    – BMDan
    Oct 2, 2014 at 19:18
  • Your code works, thx for that. But I have to replace the protocol too. Not the prefix, but the appearance inside the url that comes from rewriting in nginx. As the port changes with ssl, the "http(s)" before the hostname in the above example url changes as well.
    – boernie
    Oct 6, 2014 at 13:20
  • Boernie: Ah, you mean the scheme that is INSIDE of the req.url (.../http/example.com:80/... versus .../https/example.com:443/...). Code updated.
    – BMDan
    Oct 6, 2014 at 21:37
  • 1
    Just for completeness, one closing round bracket after ":fakeport/" should go to the end, then it works flawlessly, thx!
    – boernie
    Oct 7, 2014 at 13:08
3

An idea how you could implement this:

enter image description here

So you need to setup:

  • a Varnish reverse proxy
    • listening on port *:80 and using the backend localhost:8080
  • a Nginx webserver
    • listening on port *:443 and forwarding to localhost:80
    • listening on port localhost:8080 and serving a web site

Actually Nginx can already be configured as a caching reverse proxy. However if you'd like to have specific cache rules, the ability to purge single objects from cache and so on Varnish will be a better solution.

Hint:

You have to check if your website works well with same cache objects for http and https. It doesn't if the website provides HTML, CSS or JavaScript using absolute URLs for external content (e.g. embedded media assets). As you know browsers do not like embedding HTTP resources into HTTPS websites.

4
  • The idea is to have one cache for http and https. This does not solve that issue and in addition lacks ESI support for SSL.
    – user140044
    Sep 25, 2014 at 6:45
  • ESI will be handled by Varnish. So there are no lacks. Everything else (e.g. cache key for Varnish's objects) has to be defined in Varnish and there you can take care to have only one cache for one domain name. Sep 25, 2014 at 7:06
  • My apologies, missed an arrow. More coffee.
    – user140044
    Sep 25, 2014 at 7:12
  • Actually we have this in production and it works very well. Sep 25, 2014 at 7:17

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .