2

I have a REST API that I have built with Firebase Cloud Functions. No matter what I try, I can't get any of the endpoints to serve from the CDN cache.

A few pertinent details:

  • I am using the csurf middleware.
  • I have set (on select endpoints) a cache control header value of private, max-age=3600, s-maxage=86400
  • While the client application requires the user to be authenticated, the requests to the above-referenced endpoints omit any cookies and authentication-related data by setting the options of a typical fetch request like so:
{
  "method":"GET",
  "headers":{
    "Accept":"application/json",
    "Content-Type":"application/json"
  },
  "cache":"default",
  "credentials":"omit"
}

No matter how many times I make the same request from different browsers (or Postman), I never get any CDN cache hits. The response headers typically look like this:

accept-ranges: bytes
cache-control: private, max-age=3600, s-maxage=86400
content-encoding: gzip
content-type: application/json; charset=utf-8
date: Tue, 22 Mar 2022 20:23:18 GMT
etag: W/"410-SrNPDF/58eInOtNbbyxn6XXXXXXX"
expires: Tue, 22 Mar 2022 20:23:17 GMT
function-execution-id: XXXXXXXXXXXX
server: Google Frontend
set-cookie: _csrf=emL-XXXXXXXXXXXXXXXXXXXX; Path=/
set-cookie: XSRF-TOKEN=H3sVcdDA-XXXXXXXXXXXXXXXXXXXXXXXXXXX; Path=/
strict-transport-security: max-age=31556926
vary: cookie,need-authorization, x-fh-requested-host, accept-encoding
x-cache: MISS
x-cache-hits: 0
x-cloud-trace-context: b50952340f930d74ebfbebXXXXXXXXXX;o=1
x-country-code: US
x-orig-accept-language: en-US,en;q=0.9
x-powered-by: Express
x-served-by: cache-lax10660-LGB
x-timer: S1647980596.749133,VS0,VE2278

I am aware of the effect of the "vary" header, although Firebase doesn't seem to allow you to subtract items from it - only to add more to it.

What am I missing? TIA!

TheRealMikeD
  • 166
  • 1
  • 8

2 Answers2

2

Firebase doesn't seem to allow you to subtract items from it - only to add more to it.

This is security by design:

Note that Hosting adds Cookie and Authorization to the Vary header by default when a request is made for dynamic content. This ensures that any session or cookie authorization header you use is made part of the cache key, which prevents accidental leaks of content.

My guess is you'd have to make the clients (or downstream servers) not send the Cookie header.

You could do that by moving the public parts to another path or domain. But maybe there is also a Firebase Option to tick somewhere, marking the content static instead of dynamic.

Daniel W.
  • 31,164
  • 13
  • 93
  • 151
  • Yep, it prevents you from leaking data accidentally, but also does a pretty good job of preventing it from being cacheable for any other users. In which case, the CDN is not especially useful. You could just let the user's browser cache do the work. I think you are onto something with trying to prevent the server from sending cookies, though. I'm going to take a look at that. It may mean bypassing the csurf middleware for these particular endpoints. – TheRealMikeD Mar 23 '22 at 01:33
  • @TheRealMikeD if you just leave out the middleware, the browser will keep sending the cookie (it's bound to domain+path), that won't work. I am not familiar enough with Google Cloud services to give you a better suggestion. In theory, you could add another downstream cache that alters the `Vary` and `Cookie` headers. This works in the wild but I don't know if it's compatible with Google Services. – Daniel W. Mar 23 '22 at 10:13
  • 1
    Thanks for your comment, @Daniel W. I was out of town for a few days. Back now. I think you are on the right track with trying to eliminate cookies from the request. When I do a request from outside the authenticated site and allow cross-site requests (I have done this for one path as a test), I get cache hits. That implies that the cookies could be the problem. I'll see what I can do to get rid of them on fetch requests. – TheRealMikeD Mar 28 '22 at 22:12
1

OK, I got this to work. @Daniel W.'s answer got me thinking in the right direction. The issue was that the csurf middleware that I am using kept adding cookies to the response, and the CDN was using those as cache keys due to the inclusion of "cookie" in the "Vary" header (which Firebase will not allow you to remove). Basically, that means there were never going to be any CDN hits, because the cookies get changed by the csurf middleware on every response.

By bypassing the csurf middleware for selected endpoints (which return non-sensitive, publicly available data), the cookies are no longer added, and thus no longer involved in the CDN's cache key, which produces cache hits on the CDN for those endpoints.

You will also need to make sure not to send any cookies with the request. As I noted in the original question, you can do this by setting the "credentials":"omit" option in the config options for the fetch() call. No cookies in the request and no cookies in the response - that's the key.

For anyone else thinking of using this technique, use caution. @Daniel W. was right to quote the Firebase docs which state that the design is to use cookies as part of the cache key. In doing so, it ensures that sensitive data is not exposed. So if you go this route, you should be certain that you only bypass all cookies for endpoints which do not expose any sensitive data. Remember that if data for a given endpoint is cached on the CDN, any given request for that endpoint could get a cache hit on the CDN and not get back to the origin server for any sort of authentication or verification.

But if you have an endpoint which returns data that does not require authentication, this will work to make it cacheable on Google Cloud CDN servers.

TheRealMikeD
  • 166
  • 1
  • 8