Take the 2-minute tour ×
Server Fault is a question and answer site for professional system and network administrators. It's 100% free, no registration required.

Is there a Squid utility command to force a cache load from a local hard drive?

              xww.example.com
                /w /x /y /z
                  Apache
         ------------------------
         Slow Internet Connection
         ------------------------
          lan1  |  lan2  |  lan3
                |        |
          cache |  cache |  cache
          /x /y |  /x /z |  /y /z
          Squid |  Squid |  Squid
         ------ | ------ | ------
         Browser| Browser|Browser
         Browser| Browser|Browser
         Browser| Browser|Browser
         Browser| Browser|Browser

For example, plug the USB hard drive into the machine on lan1 that is running Squid. Then invoke a command like:

$ suqid-util\
 --force-cache-load\
 --url-root http://www.example.com/x\
 --local-root /mnt/usb1\
 --recursive

This would be useful when a 2TB hard drive can be hand carried to a location with slow Internet connection.

OR

A better solution using something other than Squid? I am still hunting.

share|improve this question
    
What a perverse idea. Much more fun than a station wagon full of tapes. I love it. –  quadruplebucky Mar 3 '10 at 3:41
    
What's the content? Is it audio and video files? HTML files with embedded MPEG? Content dictates options. –  Warner Mar 3 '10 at 5:03
    
You could populate a squid cache (in a VM, say, or something easily hand-carried) with wget on a fast link, and move that to slow link side, set it up as a parent, wget the resident squid cache on the slow link, lather, rinse, repeat. Ugh. –  quadruplebucky Mar 3 '10 at 6:10

1 Answer 1

up vote 1 down vote accepted

I have no experience with this but it seems others have used wget to spider the content via the proxy, thus populating the cache.

I can't think of any reason why you wouldn't be able to copy the cache_dir to your hard drive.

Out of curiosity, what exactly are you trying to do?

Using wget to populate SQUID cache

share|improve this answer
    
Mirror subsets of an increasing collection of static audio-video content to remote locations with slow Internet connections where the opportunistic delivery of microSD cards on the backs of pigeons can used to improve through-put. Well, actually people with hard drives may substitute for the pigeons under management control. –  C.W.Holeman II Mar 3 '10 at 2:48
    
It seems like there should be a better solution than SQUID for something like this. I hear there's heavy latency with pigeons but with modern storage densities the throughput isn't too bad. –  Warner Mar 3 '10 at 4:02
    
wget link should be placed in the answers to: serverfault.com/questions/118518/… –  C.W.Holeman II Mar 3 '10 at 16:08
    
I appreciate you accepting. Did this work for you? –  Warner Mar 4 '10 at 6:03

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.