Take the 2-minute tour ×
Server Fault is a question and answer site for professional system and network administrators. It's 100% free, no registration required.

Instead of hot-linking to external images from my application I'd like to cache them on my server where my web app resides, e.g. instead of

<img src="http://api.domain.com/image/1234.jpeg" />

I'd like to call a cache, e.g.

<img src="http://dev:3128/myapp/image?href=http://api.domain.com/image/1234.jpeg">

So of Squid has the image it'll pass it through, otherwise it'll retrieve and then cache it for next time. Is this possible?

I've installed squid, configured it as a reverse proxy in front of Apache. My config is below (hostname is dev):

acl manager proto cache_object
acl localhost src 127.0.0.1/32 ::1
acl to_localhost dst 127.0.0.0/8 0.0.0.0/32 ::1
acl dev_users dstdomain dev
http_access allow dev_users
acl SSL_ports port 443
acl Safe_ports port 80          # http
acl Safe_ports port 21          # ftp
acl Safe_ports port 443         # https
acl Safe_ports port 70          # gopher
acl Safe_ports port 210         # wais
acl Safe_ports port 1025-65535  # unregistered ports
acl Safe_ports port 280         # http-mgmt
acl Safe_ports port 488         # gss-http
acl Safe_ports port 591         # filemaker
acl Safe_ports port 777         # multiling http
acl CONNECT method CONNECT
acl JPEG url_regex -i \.myapp/image?href=http://api.domain.com/image/*.jpeg$
#acl ALL dst 0.0.0.0/0.0.0.0
http_access allow manager localhost
http_access deny manager
http_access deny !Safe_ports
http_access deny CONNECT !SSL_ports
http_access allow localhost
http_access deny all
http_port 3128 accel defaultsite=dev vhost
cache_peer 127.0.0.1 parent 80 0 no-query originserver name=dev
cache_peer_access dev allow dev_users
cache_peer_access dev deny all
cache_dir ufs /var/spool/squid3 100 16 256
coredump_dir /var/spool/squid3
refresh_pattern ^ftp:           1440    20%     10080
refresh_pattern ^gopher:        1440    0%      1440
refresh_pattern -i (/cgi-bin/|\?) 0     0%      0
refresh_pattern (Release|Packages(.gz)*)$      0       20%     2880
refresh_pattern .               0       20%     4320
never_direct allow JPEG
#always_direct allow ALL

At this point the squid proxy seems to work at http://dev:3128/myapp as it serves my php app fine. But I have to comment out the ALL acl lines (otherwise I get no response) and requests for <img src="http://dev:3128/myapp/image?href=http://api.domain.com/image/1234.jpeg"> are still showing in the apache access log (whereas I'm looking for squid to cache/serve them).

http://dev:3128/myapp/image is actually a PHP script that retrieves and serves the image via fopen and fpassthru.

share|improve this question

5 Answers 5

up vote 0 down vote accepted
+50

I would try the following config. I cleaned up yours (removed stuff not referenced) and forced it to cache anything for a year and not cache failures.

Note, squid configs are very version specific, so what version are you running?

cache_dir ufs /var/spool/squid3 100 16 256
coredump_dir /var/spool/squid3

acl localhost src 127.0.0.1 ::1

acl PURGE method PURGE
http_access allow PURGE localhost

acl manager proto cache_object
http_access allow manager localhost

acl dev_users dstdomain dev
http_access allow dev_users

http_port 3128 accel defaultsite=dev vhost
cache_peer 127.0.0.1 parent 80 0 no-query originserver name=dev
cache_peer_access dev allow dev_users
cache_peer_access dev deny all

# Don't cache 404's
negative_ttl 0

# Cache everything for a year
refresh_pattern . 1440 100% 525949 override-expire ignore-reload ignore-no-cache ignore-no-store ignore-must-revalidate ignore-private ignore-auth 

#cache JPEG but not anything else
acl JPEG url_regex -i .*image\?href=http.*api.discogs.com.*image.*\.jpeg$
acl to_localhost dst 127.0.0.0/8
cache allow JPEG
cache deny to_localhost
share|improve this answer
    
ok, this is the only thing that's working for me (cache headers are not). How do I refine this to only cache .jpeg, .png and pass all others through? –  DaveO Jun 6 '13 at 4:16
    
Updated answer with working JPEG allow, rest deny –  DaveO Jun 6 '13 at 5:56
    
Glad it worked for you. I have to admit, it took me a week or two to get a similar cache everything squid config written. It's not always obvious what the proper way is to setup squid. –  kormoc Jun 6 '13 at 6:41

If squid is sitting in front of the web server, you can set up an ACL with a regex (though this would require you to set up a regex for all the external domains) either using url_regex, urlpath_regex could work, but it excludes the hostname.. which might be important if the path to the files is the same as ones hosted on your server.

acl JPGS url_regex -i \.somedomain.com/images/*.jpg$   #Matches JPGs from somedomain
acl GIFS url_regex -i \.somedomain.com/images/*.gif$   #Matches GIFs from somedomain
acl ALL dst 0.0.0.0/0.0.0.0                           #Matches Everything else 

never_direct allow JPGS
never_direct allow GIFS
always_direct allow ALL

I don't know exactly what situation you will be using Squid in here, so I added the classic forward everything else line, if you're not proxying all internet access, you may want to redefine that scope.

share|improve this answer
    
Hi, I've installed squid and configured as updated in my question and it seems to work, but as soon as I add the config lines shown I get no-response errors? –  DaveO May 30 '13 at 14:57
    
Obviously my example is way to broad for your specific usage, since it's only one site, you could probably use urlpath_regex \.gif$ instead (same for jpg), and see if you get somewhere? –  NickW May 30 '13 at 15:02
    
Hi, I change the acl to acl JPEG urlpath_regex \.jpeg$ but the requests are still getting through to apache. My complete config is above. To clarify, I'm looking for squid to pass through the image request to apache (PHP) the first time, cache it, then serve it itself from then onwards –  DaveO May 31 '13 at 2:38
    
So, is the actual path that long? (and remember ? is a special character in regexes) If it is only caching images from the server, make it simpler. Match on any JPG, if squid finds it in its cache, it will serve it from there, otherwise it will retrieve it. There's no need to get super fancy, my second option is probably going to be what you need, especially if the image is the last bit of the request string. –  NickW May 31 '13 at 8:27
    
Thanks NIck, I'm trying acl JPEG url_regex -i .*\.jpeg$ but it's sti;ll going to apache with the config above –  DaveO May 31 '13 at 15:57

If you want squid to behave like a cache, you need to configure your php script to generate header which allow caching.

The minimum for squid to actually cache anything is:

  • Date: ( generated automatically by php, so don't bother )
  • Last-Modifed: gmt date
  • Cache-Control: put here: "public, max-age="

The easy way is then to not issue a "Etag:" header so that you have only to handle "If-Modified-Since:" request ( if you issue Etag: you will have to handle If-None-Match also ).

And here you go.

If you want to go deeper you should read rfc 2616.

If you want to reduce efficiently the traffic between squid and your backend, implement correctly "If-Modified-Since" in your script. Also you can drop / rewrite numerous client headers which will prevent caching, but this is only a second step.

share|improve this answer

If with 'external' images you mean images not on your own server (so not immediately behind squid), you'll need to do some trickery, proxying and caching the images yourself. So let's set up an example to make sure I understand you correctly:

  • Your website is www.example.org, which is squid in front of apache
  • The images are on api.example.com, which is on another server
  • You want to embed those images in such a way that the requests go to your server

If the images are only on api.example.com this is easy and doesn't even need squid. You can do this with mod_proxy and mod_cache for apache. Make the <img ..> tag look like <img src="http://www.example.org/images/api/path/to/image/here" /> and use ProxyPass in apache to pass /images/api to proxy to api.example.com and mod_cache configuration to cache it.

If you really want to use squid, you'll need to make squid smarter. I would still make the <img ..> tag the same as above and use an ICAP helper to rewrite the requests for /images/api/ to go to api.example.com. Squid can than request and cache those.

share|improve this answer

By default, Squid doesn't cache URLs with the ? sign. Try with:

cache allow JPEG

Also pay attention of what @Oliver S mention, because you application should serve the contant as cacheable to work.

share|improve this answer

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.