Varnish: how to separately cache pages based on value of a specific cookie

Question

I manage a site that has a single cookie, which we have to use, but will always be one of 9 values (including no value). I'd like to use varnish in front of our application servers, with varnish separately caching a version of each page based on the cookie value.

So if we have page /page1, Varnish should separately manage a copy of what /page1 looks like with cookie values a, b, c, d, etc....

Assume we have plenty of memory on the Varnish server to handle storing all pages with all cookie combinations.

We have tried many VCL settings but can't figure out exactly how to make this work. Varnish needs to send that specific cookie to our application server as well, so our application knows which content to send back.

Thanks in advance!

score 5 · Answer 1 · edited Apr 12 '23 at 16:24

It's quite simple to achieve that in fact, you should add a custom vcl_hash:

sub vcl_hash {
  #...
  /* Hash cookie data */
  # As requests with same URL and host can produce different results when issued with  different cookies,
  # we need to store items hashed with the associated cookies. Note that cookies are already sanitized when we reach this point.
  if (req.http.Cookie) {
    /* Include cookie in cache hash */
    hash_data(req.http.Cookie);
  }
  #...
}

With this code, varnish will store a different cookie, for each cookie value... but I'll recommend you to also sanitize your cookies on vcl_recv, this is an excerpt of [1] containing cookie sanitization for Drupal sites:

sub vcl_recv {
  #...
  # Remove all cookies that backend doesn't need to know about.
  # See https://www.varnish-cache.org/trac/wiki/VCLExampleRemovingSomeCookies
  if (req.http.Cookie) {
    /* Warning: Not a pretty solution */
    /* Prefix header containing cookies with ';' */
    set req.http.Cookie = ";" + req.http.Cookie;
    /* Remove any spaces after ';' in header containing cookies */
    set req.http.Cookie = regsuball(req.http.Cookie, "; +", ";");
    /* Prefix cookies we want to preserve with one space */
    /* 'S{1,2}ESS[a-z0-9]+' is the regular expression matching a Drupal session cookie ({1,2} added for HTTPS support) */
    /* 'NO_CACHE' is usually set after a POST request to make sure issuing user see the results of his post */
    /* Keep in mind we should add here any cookie that should reach the backend such as splahs avoiding cookies */
    set req.http.Cookie = regsuball(req.http.Cookie, ";(S{1,2}ESS[a-z0-9]+|NO_CACHE|OATMEAL|CHOCOLATECHIP)=", "; \1=");
    /* Remove from the header any single Cookie not prefixed with a space until next ';' separator */
    set req.http.Cookie = regsuball(req.http.Cookie, ";[^ ][^;]*", "");
    /* Remove any '; ' at the start or the end of the header */
    set req.http.Cookie = regsuball(req.http.Cookie, "^[; ]+|[; ]+$", "");

    if (req.http.Cookie == "") {
      /* If there are no remaining cookies, remove the cookie header. */
      unset req.http.Cookie;
    }
  }
  #...
  return(hash);
  #...
}

[1] https://github.com/NITEMAN/varnish-bites/blob/master/varnish3/drupal-base.vcl

Thanks so much! So in your example, OATMEAL and CHOCOLATECHIP are the names of 2 cookies you want to keep for hashing purposes? — Matt, Sep 28 '13 at 16:26
In my example I hash all the cookies that are not filtered on vcl_recv, OATMEAL & CHOCOLATECHIP are only 2 of them. — NITEMAN, Sep 30 '13 at 09:42
I don't know if this answer is complete. Doesn't the builtin.vcl get appended to `vcl_recv` and `return (pass)` because the cookie is present? I think you have to `return (hash)` after sanitizing cookies, but then maybe also handle piping through non-get requests and everything else handled by the builtin.vcl - https://github.com/varnishcache/varnish-cache/blob/master/bin/varnishd/builtin.vcl — ckeeney, May 30 '19 at 19:42
Code bits are excerpts of a wider example for a prior version of VCL, not a complete drop-in solution. You're right, builtin (formerly called default) is always appended. For a more up-to-date example you can take a look to https://github.com/NITEMAN/varnish-bites/blob/master/varnish4/drupal-base.vcl Inline comments should clarify. Meantime I'll add propper return to the example — NITEMAN, Jun 03 '19 at 11:53
While this is functional, there is now an officially-supported cookie VMOD that makes the process of filtering out unwanted cookies far easier and cleaner. It can be found at: https://github.com/varnish/varnish-modules — Joe C., Jul 27 '19 at 02:07

score 0 · Answer 2 · answered Jul 27 '19 at 02:20

NITEMAN provided an excellent answer for the time this question was originally asked. However, the intervening timespan has resulted in an officially-supported VMOD that makes filtering unwanted cookies easier and cleaner. It can be found at: https://github.com/varnish/varnish-modules

From the example in the README (with slight tweaking):

import cookie;

sub vcl_recv {
        cookie.parse(req.http.cookie);
        cookie.filter_except("SESSIONID,PHPSESSID");
        set req.http.cookie = cookie.get_string();
        # Only SESSIONID and PHPSESSID are left in req.http.cookie at this point.
        # ...
        return (hash);
        # ...
}

Varnish: how to separately cache pages based on value of a specific cookie

2 Answers2

Linked