How can Squid be configured to cache all downloaded files?

Question

I am trying to setup a Squid proxy instance so that it can be used for Chef cookbook development with Test Kitchen.

When Test Kitchen spins up a virtual machine it installs chef-client by downloading it. I have configured it so that it downloads through the proxy server, however Squid does not cache the downloaded file, e.g.

I think this maybe down to it being accessed over SSL but I am not 100% sure. This is because apt in the Ubuntu VM is configured to use the same proxy instance and I get hits in the cache for those files:

Here is my squid.conf file:

http_port 3128

acl manager url_regex -i ^cache_object:// +i ^https?://[^/]+/squid-internal-mgr/

acl localhost src 127.0.0.1/32 ::1
acl to_localhost dst 127.0.0.0/8 0.0.0.0/32 ::1

acl localnet src 10.0.0.0/8     # RFC 1918 possible internal network
acl localnet src 172.16.0.0/12  # RFC 1918 possible internal network
acl localnet src 192.168.0.0/16 # RFC 1918 possible internal network
acl localnet src fc00::/7       # RFC 4193 local private network range
acl localnet src fe80::/10      # RFC 4291 link-local (directly plugged) machines

acl SSL_ports port 443
acl Safe_ports port 80          # http
acl Safe_ports port 21          # ftp
acl Safe_ports port 443         # https
acl Safe_ports port 70          # gopher
acl Safe_ports port 210         # wais
acl Safe_ports port 1025-65535  # unregistered ports
acl Safe_ports port 280         # http-mgmt
acl Safe_ports port 488         # gss-http
acl Safe_ports port 591         # filemaker
acl Safe_ports port 777         # multiling http
acl CONNECT method CONNECT

http_access allow manager localhost
http_access deny manager
http_access deny !Safe_ports
http_access deny CONNECT !SSL_ports
http_access allow localhost
http_access allow localnet
http_access deny all

maximum_object_size 1024 MB
cache_replacement_policy heap LFUDA
cache_dir aufs /var/spool/squid3 5000 24 256
coredump_dir /var/spool/squid3

refresh_pattern ^ftp:          1440    20%     10080
refresh_pattern ^gopher:       1440    0%      1440
refresh_pattern Packages\.bz2$ 0       20%     4320 refresh-ims
refresh_pattern Sources\.bz2$  0       20%     4320 refresh-ims
refresh_pattern Release\.gpg$  0       20%     4320 refresh-ims
refresh_pattern Release$       0       20%     4320 refresh-ims
refresh_pattern -i .(deb|rpm|exe|zip|tar|tgz|bz2|ram|rar|bin)$  129600 100% 129600 override-expire ignore-no-cache ignore-no-store
refresh_pattern .              0       20%     4320

Does anybody know what I need to add to the configuration so that it will cache the download of chef-client? Is it down to it being an SSL link?

score 4 · Accepted Answer · answered Jun 13 '17 at 21:26

As can be the case with these things, I have worked out that it is indeed to do with SSL.

The reason being is that SSL is meant to give the expectation of privacy so by default Squid does not cache secured pages because it cannot see them.

As I am only going to be using this for cache of packages I have two options:

Download chef-client using the plain http url
Create a self signed certificate for Squid to be a man in the middle so it can decrypt the incoming data and cache it.

Number 1 is easier at the moment whereas number 2 is probably more robust in the long term.

How can Squid be configured to cache all downloaded files?

1 Answers1