We have Varnish 3.0.6 sitting in front of a bunch of IIS servers.
The IIS servers don't care about case-sensitivity of the requests coming in, and despite my constant reminders to the devs, they of course use RandomCase in all their code. I was under the impression that the worst this would result in is duplication of resources in the cache, however what we are seeing is Varnish not caching at all, in certain circumstances.
I have a url such as http://www.example.com/Content/images/flags/gb.png.
I set a header in my VCL to tell me if the request was a hit or a miss in Varnish, and if I hit that using curl
a few times from somewhere, I get the HITs I expect:
$ curl -Ss -X GET 'http://10.0.0.1/Content/images/flags/gb.png' --header 'Host: www.example.com' -I | grep X-Cache
X-Cache: HIT 15 Var01
However, if I use a URL with incorrect capitalisation, I get a MISS, regardless of how many times I fetch it:
$ curl -Ss -X GET 'http://10.0.0.1/content/images/flags/gb.png' --header 'Host: www.example.com' -I | grep X-Cache
X-Cache: MISS Var01
Where it gets strange, is that if both the filename and the path are wrong, it is cached:
$ curl -Ss -X GET 'http://10.0.0.1/content/images/flags/GB.png' --header 'Host: www.example.com' -I | grep X-Cache
X-Cache: HIT 15 Var01
So it seems that if both the path and the file are either both correct, or both incorrect, it is cached, but if one is correct and one is incorrect, it is never cached.
$curl -Ss -X GET 'http://10.0.0.1/content/images/flags/gb.png' --header 'Host: www.example.com' -I | grep X-Cache
X-Cache: MISS Var01
$curl -Ss -X GET 'http://10.0.0.1/content/images/flags/GB.png' --header 'Host: www.example.com' -I | grep X-Cache
X-Cache: HIT 7 Var01
$curl -Ss -X GET 'http://10.0.0.1/Content/images/flags/gb.png' --header 'Host: www.example.com' -I | grep X-Cache
X-Cache: HIT 7 Var01
$curl -Ss -X GET 'http://10.0.0.1/Content/images/flags/GB.png' --header 'Host: www.example.com' -I | grep X-Cache
X-Cache: MISS Var01
Here is a sample from the varnishlog
output for a 'missing' request:
168 SessionOpen c 192.168.0.1 60412 :80
168 ReqStart c 192.168.0.1 60412 96071146
168 RxRequest c GET
168 RxURL c /content/images/flags/gb.png
168 RxProtocol c HTTP/1.1
168 RxHeader c User-Agent: curl/7.35.0
168 RxHeader c Accept: */*
168 RxHeader c Host: www.example.com
168 RxHeader c X-Forwarded-Proto: https
168 VCL_call c recv
168 VCL_return c lookup
168 VCL_call c hash
168 Hash c /content/images/flags/gb.png
168 Hash c www.example.com
168 VCL_return c hash
168 HitPass c 95565251
168 VCL_call c pass pass
168 Backend c 236 my_director web08
168 TTL c 96071146 RFC 120 -1 -1 1462797933 0 1462797954 0 0
168 VCL_call c fetch
168 TTL c 96071146 VCL 28800 -1 -1 1462797933 -0
168 VCL_return c deliver
168 ObjProtocol c HTTP/1.1
168 ObjResponse c OK
168 ObjHeader c Content-Type: image/png
168 ObjHeader c Last-Modified: Wed, 04 May 2016 07:56:47 GMT
168 ObjHeader c Accept-Ranges: bytes
168 ObjHeader c ETag: "809d381daa5d11:0"
168 ObjHeader c Server: Microsoft-IIS/8.5
168 ObjHeader c X-Powered-By: ASP.NET
168 ObjHeader c Date: Mon, 09 May 2016 12:45:54 GMT
168 ObjHeader c Content-Length: 599
168 ObjHeader c X-Backend: web08
168 VCL_call c deliver
168 VCL_return c deliver
168 TxProtocol c HTTP/1.1
168 TxStatus c 200
168 TxResponse c OK
168 TxHeader c Content-Type: image/png
168 TxHeader c Last-Modified: Wed, 04 May 2016 07:56:47 GMT
168 TxHeader c Accept-Ranges: bytes
168 TxHeader c ETag: "809d381daa5d11:0"
168 TxHeader c X-Backend: web08
168 TxHeader c Content-Length: 599
168 TxHeader c Accept-Ranges: bytes
168 TxHeader c Date: Mon, 09 May 2016 12:45:32 GMT
168 TxHeader c Age: 0
168 TxHeader c Connection: keep-alive
168 TxHeader c X-Cache: MISS Var01
168 Length c 599
168 ReqEnd c 96071146 1462797932.596346378 1462797932.619358778 0.000039339 0.022963524 0.000048876
Update: To make things more confusing, another site we run here, also IIS-based, does not exhibit this issue, so I suspect there's something funky going on but I can't work out where. I can't see what Varnish might be seeing to make it think these requests are MISSes - all it sees is the URL, as my vcl_hash
is just hash_data(req.url);hash_data(req.http.host)