0

We have Varnish 3.0.6 sitting in front of a bunch of IIS servers.

The IIS servers don't care about case-sensitivity of the requests coming in, and despite my constant reminders to the devs, they of course use RandomCase in all their code. I was under the impression that the worst this would result in is duplication of resources in the cache, however what we are seeing is Varnish not caching at all, in certain circumstances.

I have a url such as http://www.example.com/Content/images/flags/gb.png.

I set a header in my VCL to tell me if the request was a hit or a miss in Varnish, and if I hit that using curl a few times from somewhere, I get the HITs I expect:

$ curl -Ss -X GET 'http://10.0.0.1/Content/images/flags/gb.png' --header 'Host: www.example.com' -I | grep X-Cache
X-Cache: HIT 15 Var01

However, if I use a URL with incorrect capitalisation, I get a MISS, regardless of how many times I fetch it:

$ curl -Ss -X GET 'http://10.0.0.1/content/images/flags/gb.png' --header 'Host: www.example.com' -I | grep X-Cache
X-Cache: MISS Var01

Where it gets strange, is that if both the filename and the path are wrong, it is cached:

$ curl -Ss -X GET 'http://10.0.0.1/content/images/flags/GB.png' --header 'Host: www.example.com' -I | grep X-Cache
X-Cache: HIT 15 Var01

So it seems that if both the path and the file are either both correct, or both incorrect, it is cached, but if one is correct and one is incorrect, it is never cached.

$curl -Ss -X GET 'http://10.0.0.1/content/images/flags/gb.png' --header 'Host: www.example.com' -I | grep X-Cache
X-Cache: MISS Var01

$curl -Ss -X GET 'http://10.0.0.1/content/images/flags/GB.png' --header 'Host: www.example.com' -I | grep X-Cache
X-Cache: HIT 7 Var01

$curl -Ss -X GET 'http://10.0.0.1/Content/images/flags/gb.png' --header 'Host: www.example.com' -I | grep X-Cache
X-Cache: HIT 7 Var01

$curl -Ss -X GET 'http://10.0.0.1/Content/images/flags/GB.png' --header 'Host: www.example.com' -I | grep X-Cache
X-Cache: MISS Var01

Here is a sample from the varnishlog output for a 'missing' request:

168 SessionOpen  c 192.168.0.1 60412 :80
168 ReqStart     c 192.168.0.1 60412 96071146
168 RxRequest    c GET
168 RxURL        c /content/images/flags/gb.png
168 RxProtocol   c HTTP/1.1
168 RxHeader     c User-Agent: curl/7.35.0
168 RxHeader     c Accept: */*
168 RxHeader     c Host: www.example.com
168 RxHeader     c X-Forwarded-Proto: https
168 VCL_call     c recv
168 VCL_return   c lookup
168 VCL_call     c hash
168 Hash         c /content/images/flags/gb.png
168 Hash         c www.example.com
168 VCL_return   c hash
168 HitPass      c 95565251
168 VCL_call     c pass pass
168 Backend      c 236 my_director web08
168 TTL          c 96071146 RFC 120 -1 -1 1462797933 0 1462797954 0 0
168 VCL_call     c fetch
168 TTL          c 96071146 VCL 28800 -1 -1 1462797933 -0
168 VCL_return   c deliver
168 ObjProtocol  c HTTP/1.1
168 ObjResponse  c OK
168 ObjHeader    c Content-Type: image/png
168 ObjHeader    c Last-Modified: Wed, 04 May 2016 07:56:47 GMT
168 ObjHeader    c Accept-Ranges: bytes
168 ObjHeader    c ETag: "809d381daa5d11:0"
168 ObjHeader    c Server: Microsoft-IIS/8.5
168 ObjHeader    c X-Powered-By: ASP.NET
168 ObjHeader    c Date: Mon, 09 May 2016 12:45:54 GMT
168 ObjHeader    c Content-Length: 599
168 ObjHeader    c X-Backend: web08
168 VCL_call     c deliver
168 VCL_return   c deliver
168 TxProtocol   c HTTP/1.1
168 TxStatus     c 200
168 TxResponse   c OK
168 TxHeader     c Content-Type: image/png
168 TxHeader     c Last-Modified: Wed, 04 May 2016 07:56:47 GMT
168 TxHeader     c Accept-Ranges: bytes
168 TxHeader     c ETag: "809d381daa5d11:0"
168 TxHeader     c X-Backend: web08
168 TxHeader     c Content-Length: 599
168 TxHeader     c Accept-Ranges: bytes
168 TxHeader     c Date: Mon, 09 May 2016 12:45:32 GMT
168 TxHeader     c Age: 0
168 TxHeader     c Connection: keep-alive
168 TxHeader     c X-Cache: MISS Var01
168 Length       c 599
168 ReqEnd       c 96071146 1462797932.596346378 1462797932.619358778 0.000039339 0.022963524 0.000048876

Update: To make things more confusing, another site we run here, also IIS-based, does not exhibit this issue, so I suspect there's something funky going on but I can't work out where. I can't see what Varnish might be seeing to make it think these requests are MISSes - all it sees is the URL, as my vcl_hash is just hash_data(req.url);hash_data(req.http.host)

|improve this question
  • It just got weirder. I'm looking at a different url, and now Varnish is refusing to cache the correct URL, but happily caches all the "wrong" URLs. – Cylindric May 9 '16 at 13:56
0

If all the servers behind Varnish are IIS, then the following should help you resolve this issue for you.

import std;
...
sub vcl_recv {
       set req.url = regsub(req.url, "\?.*", "");
       ...   
    }
...

This way you will not have to do regex case insensitive matches by using (?i) in each expression.

|improve this answer
  • Unfortunately I can't guarantee that the entire URL is case-insensitive - for example all the parameters passed in to the app by way of path elements (e.g. MVC) or even query-string parameters. Varnish has no way to know what bits are 'real' directories, and which bits IIS will hand off to the app. – Cylindric May 9 '16 at 13:03
  • You can use the following to strip the query string and parameters. (MVC Path elements are case-insensitive by default) set req.url = regsub(req.url, "\?.*$", ""); – Nasoo May 9 '16 at 13:34
  • I'd still rather find out why Varnish isn't caching everything it should be, rather than just renaming out the things it's ignoring. The URLs are always returning content to the end client, there is never a 404 or a redirect, and I can't see any reason for Varnish not to cache that response. – Cylindric May 9 '16 at 13:55
  • Please update your question with more information including your current vcl files. – Nasoo May 9 '16 at 14:11
  • I'll try, but that'll take a while, as I need to remove all the proprietary stuff. – Cylindric May 9 '16 at 14:35

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.