I have a public website with some URLs/directories which are for private/internal use only. These private areas can only be accessed via certain IP addresses or with a known username/password.

Currently I achieve this via .htaccess files like this:

AuthType     Basic
AuthName     "Protected Area"
AuthUserFile /path/to/.htpasswd

SetEnvIf Remote_Addr trusted
SetEnvIf X-Forwarded-For trusted
# (Note I am aware X-Forwarded-For can be spoofed)

    Require env trusted
    Require valid-user

The problem is I want to add Varnish in front of my server to provide caching. Obviously the existing setup won't work with Varnish because it can't cache content restricted by .htaccess files in this way.

Is there a way I can continue to use my .htaccess files to protect my internal pages, or is there a similar approach I can use to place the responsibility for security on Varnish itself which doesn't require changing Varnish's VCL files every time I want to add or modify the restrictions?


You can definitely do this in Varnish by writing the following VCL:

vcl 4.0;

acl allowed {

sub vcl_recv {
    if(!client.ip ~ allowed && req.http.Authorization != "YWRtaW46YWRtaW4K") {
        return (synth(401, "Restricted"));
    return (hash);

sub vcl_synth {
    if (resp.status == 401) {
        set resp.http.WWW-Authenticate =
            {"Basic realm="Restricted area""};


Although this works fine, the username/password checks are done manually. If you have multiple usernames/passwords, you'll have to add each user in the if-statement.

Make sure you bypass the default VCL behavior

When handling Authorization headers in Varnish, it's important not to fall back on default behavior. By default Varnish will not lookup items in cache when an Authorization header is present.

So at some point in your logic, you'll have to execute return (hash);, which I did in the example. Otherwise you'll fall back on default behavior and items will not be looked up in cache.

Vmod_basicauth as an alternative

If that turns out to be a deal breaker, you can compile the following Varnish module that allows you to load an .htpasswd file and perform the authentication for you.

Still doing it all in Apache

If you decide to keep doing this in Apache, you'll need to change the VCL as well. I explained that default behavior will stop Varnish from looking up items in cache when an Authorization header is present.

If you again look at the default behavior again, you'll have to make sure a return (hash); is executed before you hit default behavior, or you have to reimplement part of it your self by adding the necessary if-statements

|improve this answer|||||
  • Brilliant, thank you once again. Just two questions: 1. I have different sets of restrictions for different pages/directories (currently scattered throughout many standalone .htaccess files); I assume I would have to use different if statements for each of those URLs? Something like if (req.url = "/secret1")? 2. Is there a way I can continue creating .htaccess files (so that directories remain secured if Varnish is ever turned off or bypassed) and still have Varnish work? My guess is I'd have to make Varnish send a base64-encoded username/password to any such protected URLs? – WackGet Mar 26 at 15:26
  • You might as well continue doing all of it in .htaccess& .htpasswd and not do it in Varnish, otherwise you'll be doing a lot of unnecessary work. The only thing you have to do is to make sure the VCL will cache pages that contain an Authorization header. As I explained: default Varnish behavior will not cache if such a header is present. See the Still doing it all in Apache section in the answer above. – Thijs Feryn Mar 28 at 15:21
  • The problem I have with that approach is at the moment the entire website is behind an Apache authorization setup (because it's a staging website which has to be accessible from anywhere but not to the general public). So currently I wouldn't be able to test Varnish on any part of the website if I keep my Apache setup. What I'll probably do is set up the "global" security restriction using Varnish and keep my other restricted pages using .htaccess since they don't need to be cached. Thanks again. I bought your book, by the way! – WackGet Mar 30 at 16:30

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.