How Varnish processes HEAD
requests
Varnish's default behavior is to accept a HEAD
request, and turn it into a GET
request.
The response is stored in cache, but the payload is stripped off before being returned to the client.
This is done by design, for efficiency reasons, and does not violate the RFCs. The output should be identical.
Varnish does not add a cache variation based on the request method. Because HEAD
is converted into GET
, it doesn't need that variation.
How to circumvent this default behavior
In your specific case, you've done optimizations at the origin level to process HEAD
requests. Apparently, receiving pure and unadulterated HEAD
requests at the origin matters.
To achieve this, and to circumvent default behavior, there are 2 common ways:
You can explicitly set the request method to HEAD
in vcl_backend_fetch
:
sub vcl_recv {
set req.http.method = req.method;
}
sub vcl_hash {
hash_data(req.http.method);
}
sub vcl_backend_fetch {
if(bereq.http.method == "HEAD") {
set bereq.method = "GET";
}
unset bereq.http.method;
}
You can bypass the cache:
sub vcl_recv {
if(req.method == "HEAD") {
return(pass);
}
}
The former solution is probably better than the latter in your case.
Something is going on in your VCL
Although you claim there's no VCL logic in place that keeps HEAD
requests intact, I suspect something is going on in your VCL.
I'd love to see your full VCL file, and give it a try myself. Feel free to redact any sensitive data.
I do agree with the solution to your problem.
Although I have a hard time believing that your Varnish installation keeps HEAD
requests intact without any special VCL, the solution you offer is a valid one.
Creating a cache variation per request method is a good way to tackle your issue.
Regardless whether or not you have special VCL in place for HEAD
requests, this is the behavior you need to satisfy your origin server. I'm not going to argue that.
The only reason I'm pursuing this topic, is because I'm interested in what caused this issue. You already provided the solution, I just want to make sure we both understand why things happened the way they did.
Looking forward to that VCL file, and thanks for your input.