2

I'm trying to use varnish to cache rpms and other giant binaries. What I would've expected is that when an object is expired in the cache varnish would send a request with If-Not-Modified to the backend and then assuming the object didn't change, varnish would refresh the ttl on the local cached object without downloading a new one. I wrote a test backend to generate specific request (set small max-age and whatnot, as well as see the header varnish sends) but I never get anything else then full fetch. If-Not-Modified is never sent. My VCL is basically the default VCL. I tried playing around with setting small ttl/grace but never got any interesting behavior.

Is varnish even able to do what I want it to ? If so has anyone done anything similar and can give tips ?

  • On setting a small TTL, check out https://jakearchibald.com/2016/caching-best-practices/ for some good idea's on how to cache content. – Samuel Parkinson Aug 29 '17 at 12:15

2 Answers2

0

The request sent to the backend when an object is expired is the one that Varnish receives from the client.

So when testing your setup, are you sending an If-Not-Modified header in your requests to Varnish?

Have a look at https://www.varnish-software.com/wiki/content/tutorials/varnish/builtin_vcl.html to see what the built in VCL is.

Under vcl_backend_fetch, which will be called if there is no object in the cache, you can see there is no complex logic around stale objects, it is just passing on the request as is.

Samuel Parkinson
  • 2,992
  • 1
  • 27
  • 38
  • I've just tested this with Varnish 6 (a minimal default.vcl, and curl) and I can confirm that Varnish strips out the `If-None-Match` header in requests to the backend when refreshing a cached URL. When I replace Varnish with a nginx reverse-proxy config (with `proxy_cache_revalidate on`), the backend requests do contain the `If-None-Match` header and nginx handles 304s properly. Also, when directly talking to the backend with curl (with otherwise same headers), it returns 304, as expected. – maxschlepzig May 12 '19 at 11:58
0

First of all, quite a bit has happened in varnish-cache since this question was posted. I am answering the questions for varnish-cache 6.0 and later:

The behavior the OP expects is how varnish should behave now if the backend returns the Last-Modified and/or Etag headers.

Obviously, an object can only be refreshed if it still exist in cache. This is what beresp.keep is for. It extends the time an object is kept in cache after ttl and grace have expired. Note that objects are also LRU evicted if the cache is too small to keep all objects for their maximum lifetime.

On the comment by @maxschlepzig, it might be based on a misunderstanding:

When an object is not in cache but is to be cached, varnish can not forward the client request's conditional headers (If-Modified-Since, If-None-Match) because a 304 response would not be good for caching (it has not body and is relevant only for a particular request). Instead, varnish strips to conditional headers for this case to (potentially) get a 200 response with an object to put into cache.

As explained above, for a subsequent backend request after the ttl has expired, the conditional headers are constructed based on the cached response. The conditional headers from the client are not used for this case either.

All of this above applies for the case that an object is to be cached at all (Fetch, Hit-for-Miss (as created by setting beresp.uncacheable)).

For Pass and Hit-for-Pass (as created by return(pass(duration)) in vcl_backend_response), the client conditional headers are passed to the backend.

Nils Goroll
  • 148
  • 4