0

I have a legacy system that still serves customers and needs to be behind a fastly cache. However, our new system needs to stay in sync with our legacy system, so we make updates via the rest api. However, since fastly caches the results, this can cause our synchronization to start getting bad data.

I know it's possible to run a PURGE against the endpoint, and that seems to work a lot of the time, but not always (I'm not sure if our fastly servers are behind a load balancer. seems weird to me, but I'm not really a fastly person, so ¯\_(ツ)_/¯).

So. My question is, does anyone know if there is some sort of header I can send along with the request to force fastly to get me the latest info? I've tried Cache-Control: no-cache and no-store, however, these are not working.

If there is a setting on fastly itself to make it so these do work, that might be a solution, but our system folks are under high load and it can take a while for me to get things like this done sometimes. if I can provide them with the answer ahead of time, it might help.

CStroliaDavis
  • 392
  • 4
  • 14
  • I don't now fastly TBH, but after skimming through its documentation it seems to support HTTP caching. HTTP is rather specific that a cache should invalidate any stored representations if an unsafe operation is used on the caching-key, which is basically the URI the request is sent to. If representations aren't invalidated for such methods you should probably file an issue with fastly. Make sure though that you use the same URI for updating resources that you also use to retrieve the resource state representation otherwise cache invalidation wont work – Roman Vottner Oct 21 '22 at 15:00
  • So, we've found a workaround. We were able to get the address of the server that is behind fastly and we are now going to hit that directly (which is probably what we should have been doing all along, but I guess we don't always think of stuff like this when we're rushed). I'll leave the question up in case it helps me in some future endeavor. – CStroliaDavis Oct 21 '22 at 20:01

1 Answers1

0

Fastly's CDN product uses Varnish to cache content. You can find out more about Varnish and its configuration language (VCL) here: https://developer.fastly.com/learning/vcl/

Varnish supports HTTP caching, which means your origin server (e.g. your legacy service) should be sending the appropriate HTTP response header(s) to indicate how that resource should be cached (if at all) to any caching servers in front of it (e.g. a CDN like Fastly).

I would suggest reading through the following Fastly 'core concepts' reference which explains Fastly's caching behaviours in more detail: https://developer.fastly.com/learning/concepts/cache-freshness/

The following resource is also worth reading as it explains both the Cache-Control and Surrogate-Control HTTP headers (and their values) in more detail: https://www.integralist.co.uk/posts/http-caching/

With regard to your specific question, there isn't anything built-in to a CDN that would allow a client (e.g. the device making a request for an endpoint) to skip the CDN caching layer because if there were it would mean a malicious user (aka bad actor) could use that to side-step the cache as well and overload your origin server.

That said, one way to get fresh content would be to 'cache-bust' the endpoint you're requesting by appending a query string. For example:

https://www.example.com/foo?cachebust=1

A CDN will cache content using a 'key', and that key is typically a hash of the URL hostname and path/querystring (e.g. example.com and /foo). So by adding a query string (e.g. ?cachebust=1) you'll cause the cache lookup to fail and a request to be sent to the origin.

Another approach might be to set a custom HTTP request header assigned a secret key and then have the CDN configured with the same key. This would allow you to write some VCL that would execute something like:

sub vcl_recv {
  if (req.http.My-Secret == "123") {
    return(pass); // skip cache lookup
  }
  return(lookup); // lookup endpoint in the cache
}

Here is a working example using Fastly's "Fiddle" tool: https://fiddle.fastly.dev/fiddle/588f3c06

Integralist
  • 495
  • 3
  • 7