Varnish doesn't seem to be caching

Question

I've setup a Varnish cache mirror to sit in front of a file server, but it seems to be endlessly re-downloading data from my file server.

There's about 100GB of data in total, but so far Varnish has downloaded 800GB from my file server.

I'm using the default VCL file that comes with Varnish and the response headers for files served by the file server are similar to the following:

HTTP/1.1 200 OK
Cache-Control: max-age=290304000, public
Content-Type: image/jpeg
Expires: Wed, 29 Dec 2010 21:38:33 GMT
Server: Microsoft-IIS/7.0
E-Tag: "8b4723296ab697530768f18b1378b269"
Content-Disposition: inline; filename=image046.jpg;
X-AspNet-Version: 4.0.30319
X-Powered-By: ASP.NET
Date: Thu, 23 Dec 2010 05:38:33 GMT
Content-Length: 100592

I'm starting varnishd with the following options:

varnish/sbin/varnishd -a 0.0.0.0:80 -f varnish/etc/varnish/default.vcl -s file,varnish/var/lib/varnish/varnish_storage.bin,100G

Can you paste in your default.vcl (I can't find an unmodified one on the internets)? — Tom O'Connor, Dec 25 '10 at 08:54

score 4 · Accepted Answer · answered Dec 25 '10 at 08:50

4

sub vcl_recv {
    if (req.url ~ "\.(cur|gif|jpg|jpeg|swf|css|js|flv|mp3|mp4|pdf|ico|png)(\?.*|)$") {
      unset req.http.cookie;
      unset req.http.Accept-Encoding;
      unset req.http.Vary;
    }
}

If your requests contain cookies, they will be passed to the backend by default. Removing the encoding and vary will normalize things a little more so that you don't need to maintain a few copies of your images in cache. Also, I think 100G might be a bit excessive for your backing store, you might consider something closer to the amount of RAM you have in the machine.

answered Dec 25 '10 at 08:50

aaa

56
1

How do you know his Cache server doesn't have 100 GB of ram? It's quite easy to get 100+GB of ram in a server now. – Tom O'Connor Dec 25 '10 at 08:55
1

Thanks, the cookies were what was making Varnish not cache. I didn't think about it, but it turns out that because the Varnish mirror was on a subdomain of a site setting cookies for the whole domain, they were affecting requests to Varnish too. – anonymous coward Dec 25 '10 at 09:23
Funny that aaa didn't receive the bounty for answering the question. It really is a boys club around here. – aaa Jan 14 '11 at 11:59
1

@aaa - I need to make you clear on a few points sorry. This is a question and answer site not a chat board, can I ask you to read our FAQ please as you shouldn't comment in the form of an answer (I've converted your answer into the above comment). Secondly I don't understand your accusation, if you have a query regarding bounty management then you can contact us without being rude. Thirdly you appear to have used two different OpenID's to create two different accounts, each called 'aaa', this may have caused the bounty problem - would you like them merged? – Chopper3 Jan 14 '11 at 12:02

Varnish doesn't seem to be caching

1 Answers1