Logging behind a reverse proxy?

Question

I am currently running Varnish as a reverse proxy in front of our development website, testing before deployment into production. One of the things I've had to come to grips with is logging: in a direct access world, Apache logs the client IP address to access_log and error_log. This is slightly less useful when every client connection is from our Varnish box.

I've done some customization with SetEnvIf and LogFormat, and now our access_log intelligently logs the appropriate IP from REMOTE_HOST or X-Forwarded-For, depending on the source of the incoming connection. This doesn't do anything for error_log though. As far as I can tell, I can't override the client IP in this log.

So, what are your solutions for logging in a reverse proxy world? Should I pretty much write off the standard Apache logging and focus my efforts somewhere else, ie. in code? I am interested in both usage statistics and security auditing here.

Answer 1

Tip:

mod_rpaf - rpaf is short for reverse proxy add forward.

It changes the remote address of the client visible to other Apache modules when two conditions are satisfied. First condition is that the remote client is actually a proxy that is defined in httpd.conf. Secondly if there is an incoming X-Forwarded-For header and the proxy is in its list of known proxies it takes the last IP from the incoming X-Forwarded-For header and changes the remote address of the client in the request structure.

http://stderr.net/apache/rpaf/

Answer 2

I log everything in Varnish (varnishlog to a file, rotated hourly for a couple of days). Lets me correlate exactly what's going on in Varnish with what's going on behind the scenes.

Your usage statistics, as you say, are mostly covered by judicious logging of X-Forwarded-For, although you can also use varnishncsa to do more complete access logging (so you know every hit on your site). "Security auditing" is too broad to be able to make concrete recommendations.

Answer 3

The error_log is report errors between Varnish and Apache, any errors that are occurring between the client and Varnish will be logged at Varnishs end, not Apaches. Your error_log should be empty, as you have control over both endpoints. Anything which does show up in that log should be considered serious.