2

I want to use an intercepting squid server to cache specific large zip files that users in my network download frequently.

I have configured squid on a gateway machine and caching is working for "static" zip files that are served from an Apache web server outside our network.

The files that I want to have cached by squid are zip files >100MB which are served from a heroku-hosted Rails application. I set an ETag header (SHA hash of the zip file on the server) and Cache-Control: public header. However, these files are not cached by squid. This, for example, is a request that is not cached:

$ curl --no-keepalive -v -o test.zip --header "X-Access-Key: 20767ed397afdea90601fda4513ceb042fe6ab4e51578da63d3bc9b024ed538a" --header "X-Customer: 5" "http://MY_APP.herokuapp.com/api/device/v1/media/download?version=latest"
* Adding handle: conn: 0x7ffd4a804400
* Adding handle: send: 0
* Adding handle: recv: 0
...
> GET /api/device/v1/media/download?version=latest HTTP/1.1
> User-Agent: curl/7.30.0
> Host: MY_APP.herokuapp.com
> Accept: */*
> X-Access-Key: 20767ed397afdea90601fda4513ceb042fe6ab4e51578da63d3bc9b024ed538a
> X-Customer: 5
> 
  0     0    0     0    0     0      0      0 --:--:--  0:00:09 --:--:--     0< HTTP/1.1 200 OK
* Server Cowboy is not blacklisted
< Server: Cowboy
< Date: Mon, 18 Aug 2014 14:13:27 GMT
< Status: 200 OK
< X-Frame-Options: SAMEORIGIN
< X-Xss-Protection: 1; mode=block
< X-Content-Type-Options: nosniff
< ETag: "95e888938c0d539b8dd74139beace67f"
< Content-Disposition: attachment; filename="e7cce850ae728b81fe3f315d21a560af.zip"
< Content-Transfer-Encoding: binary
< Content-Length: 125727431
< Content-Type: application/zip
< Cache-Control: public
< X-Request-Id: 7ce6edb0-013a-4003-a331-94d2b8fae8ad
< X-Runtime: 1.244251
< X-Cache: MISS from AAA.fritz.box
< Via: 1.1 vegur, 1.1 AAA.fritz.box (squid/3.3.11)
< Connection: keep-alive

In the logs squid is reporting a TCP_MISS.

This is the relevant excerpt from my squid file:

# Squid normally listens to port 3128
http_port 3128
http_port 3129 intercept

# Uncomment and adjust the following to add a disk cache directory.
maximum_object_size 1000 MB
maximum_object_size_in_memory 1000 MB
cache_dir ufs /usr/local/var/cache/squid 10000 16 256
cache_mem 2000 MB

# Leave coredumps in the first cache dir
coredump_dir /usr/local/var/cache/squid
cache_store_log daemon:/usr/local/var/logs/cache_store.log

#refresh_pattern -i (/cgi-bin/|\?) 0    0%      0
refresh_pattern -i .(zip)       525600  100%    525600 override-expire ignore-no-cache ignore-no-store
refresh_pattern .               0       20%     4320

## DNS Configuration
dns_nameservers 8.8.8.8 8.8.4.4

After trying around for some time I realized that squid is sometimes deciding that my file is cacheable, sometimes not, depending on whether and when I enable/disable the dns_nameservers directive.

What could be wrong here?

1
  • 1
    Damn that is a massive obj_in_memory setting.. Try setting that much lower, like 1MB, or less.. You could also try using IMS refresh_patterns? refresh_pattern -i \.(dmg|msi|deb|rpm|exe|zip|tar|tgz|ram|rar|bin|ppt|doc|tiff)$ 10080 90% 43200 override-expire ignore-no-cache ignore-no-store ignore-private reload-into-ims
    – Grizly
    Sep 26, 2014 at 7:11

0

You must log in to answer this question.

Browse other questions tagged .