3

Is there a Squid utility command to force a cache load from a local hard drive?

              xww.example.com
                /w /x /y /z
                  Apache
         ------------------------
         Slow Internet Connection
         ------------------------
          lan1  |  lan2  |  lan3
                |        |
          cache |  cache |  cache
          /x /y |  /x /z |  /y /z
          Squid |  Squid |  Squid
         ------ | ------ | ------
         Browser| Browser|Browser
         Browser| Browser|Browser
         Browser| Browser|Browser
         Browser| Browser|Browser

For example, plug the USB hard drive into the machine on lan1 that is running Squid. Then invoke a command like:

$ suqid-util\
 --force-cache-load\
 --url-root http://www.example.com/x\
 --local-root /mnt/usb1\
 --recursive

This would be useful when a 2TB hard drive can be hand carried to a location with slow Internet connection.

OR

A better solution using something other than Squid? I am still hunting.

CW Holeman II
  • 421
  • 1
  • 8
  • 23
  • What a perverse idea. Much more fun than a station wagon full of tapes. I love it. – quadruplebucky Mar 03 '10 at 03:41
  • What's the content? Is it audio and video files? HTML files with embedded MPEG? Content dictates options. – Warner Mar 03 '10 at 05:03
  • You could populate a squid cache (in a VM, say, or something easily hand-carried) with wget on a fast link, and move that to slow link side, set it up as a parent, wget the resident squid cache on the slow link, lather, rinse, repeat. Ugh. – quadruplebucky Mar 03 '10 at 06:10

1 Answers1

1

I have no experience with this but it seems others have used wget to spider the content via the proxy, thus populating the cache.

I can't think of any reason why you wouldn't be able to copy the cache_dir to your hard drive.

Out of curiosity, what exactly are you trying to do?

Using wget to populate SQUID cache

Warner
  • 23,756
  • 2
  • 59
  • 69
  • Mirror subsets of an increasing collection of static audio-video content to remote locations with slow Internet connections where the opportunistic delivery of microSD cards on the backs of pigeons can used to improve through-put. Well, actually people with hard drives may substitute for the pigeons under management control. – CW Holeman II Mar 03 '10 at 02:48
  • It seems like there should be a better solution than SQUID for something like this. I hear there's heavy latency with pigeons but with modern storage densities the throughput isn't too bad. – Warner Mar 03 '10 at 04:02
  • wget link should be placed in the answers to: http://serverfault.com/questions/118518/configure-squid-to-permanently-cache-specific-directories – CW Holeman II Mar 03 '10 at 16:08
  • I appreciate you accepting. Did this work for you? – Warner Mar 04 '10 at 06:03