I've setup a Varnish cache mirror to sit in front of a file server, but it seems to be endlessly re-downloading data from my file server.
There's about 100GB of data in total, but so far Varnish has downloaded 800GB from my file server.
I'm using the default VCL file that comes with Varnish and the response headers for files served by the file server are similar to the following:
HTTP/1.1 200 OK Cache-Control: max-age=290304000, public Content-Type: image/jpeg Expires: Wed, 29 Dec 2010 21:38:33 GMT Server: Microsoft-IIS/7.0 E-Tag: "8b4723296ab697530768f18b1378b269" Content-Disposition: inline; filename=image046.jpg; X-AspNet-Version: 4.0.30319 X-Powered-By: ASP.NET Date: Thu, 23 Dec 2010 05:38:33 GMT Content-Length: 100592
I'm starting varnishd with the following options:
varnish/sbin/varnishd -a 0.0.0.0:80 -f varnish/etc/varnish/default.vcl -s file,varnish/var/lib/varnish/varnish_storage.bin,100G