up vote 2 down vote favorite
share [g+] share [fb]

Is there a Squid utility command to force a cache load from a local hard drive?

              xww.example.com
                /w /x /y /z
                  Apache
         ------------------------
         Slow Internet Connection
         ------------------------
          lan1  |  lan2  |  lan3
                |        |
          cache |  cache |  cache
          /x /y |  /x /z |  /y /z
          Squid |  Squid |  Squid
         ------ | ------ | ------
         Browser| Browser|Browser
         Browser| Browser|Browser
         Browser| Browser|Browser
         Browser| Browser|Browser

For example, plug the USB hard drive into the machine on lan1 that is running Squid. Then invoke a command like:

$ suqid-util\
 --force-cache-load\
 --url-root http://www.example.com/x\
 --local-root /mnt/usb1\
 --recursive

This would be useful when a 2TB hard drive can be hand carried to a location with slow Internet connection.

OR

A better solution using something other than Squid? I am still hunting.

link|improve this question
50% accept rate
What a perverse idea. Much more fun than a station wagon full of tapes. I love it. – quadruplebucky Mar 3 '10 at 3:41
What's the content? Is it audio and video files? HTML files with embedded MPEG? Content dictates options. – Warner Mar 3 '10 at 5:03
You could populate a squid cache (in a VM, say, or something easily hand-carried) with wget on a fast link, and move that to slow link side, set it up as a parent, wget the resident squid cache on the slow link, lather, rinse, repeat. Ugh. – quadruplebucky Mar 3 '10 at 6:10
feedback

1 Answer

up vote 1 down vote accepted

I have no experience with this but it seems others have used wget to spider the content via the proxy, thus populating the cache.

I can't think of any reason why you wouldn't be able to copy the cache_dir to your hard drive.

Out of curiosity, what exactly are you trying to do?

Using wget to populate SQUID cache

link|improve this answer
Mirror subsets of an increasing collection of static audio-video content to remote locations with slow Internet connections where the opportunistic delivery of microSD cards on the backs of pigeons can used to improve through-put. Well, actually people with hard drives may substitute for the pigeons under management control. – C.W.Holeman II Mar 3 '10 at 2:48
It seems like there should be a better solution than SQUID for something like this. I hear there's heavy latency with pigeons but with modern storage densities the throughput isn't too bad. – Warner Mar 3 '10 at 4:02
wget link should be placed in the answers to: serverfault.com/questions/118518/… – C.W.Holeman II Mar 3 '10 at 16:08
I appreciate you accepting. Did this work for you? – Warner Mar 4 '10 at 6:03
feedback

Your Answer

 
or
required, but never shown
discard

Not the answer you're looking for? Browse other questions tagged or ask your own question.