Syncing multiple Varnish Cache servers

Question

We have two servers, web1 and web2, each running Apache and Varnish. They are load balanced, with a persistence of 30 mins.

In our testing, we've found some cases were pages are cached on one instance of Varnish (say web1), but not on Varnish on web2.

Is there is a way to keep them in sync? So that when a page is cached (or purged) in web1, it also gets loaded into (or purged from) web2, and vice versa?

score 1 · Answer 1 · edited May 23 '17 at 11:55

1

You can use one varnish as a backend for another varnish.

You can do something like:

// use random director so you can fall back to web
director varnish random {
    {
        .backend = varnish2;
        .weight  = 100000000;
    }
    {
        .backend = web;
        .weight = 1;
    }
}

acl othervarnish {
        "{hostname}";
}

sub vcl_recv {
  set req.backend = web;

  // switch backend to varnish only if you haven't come from there
  if (!client.ip ~ othervarnish) {
     set req.backend = varnish;
  }
}

But note this seems like it doesn't work with ESIs.

edited May 23 '17 at 11:55

Community

1
1

answered Aug 17 '11 at 18:54

Pax

648
6
12

Thanks @Pax. So no way for having two varnish servers in 'parallel' if you will and synching them ... ? We have two upfront in case one half fails. Would be nice to retain that redundancy. – KM. Aug 17 '11 at 20:41

score 0 · Answer 2 · answered Aug 12 '17 at 15:56

I'm also looking for a solution to solve an equal problem, but didn't found it yet.

A workaround could be:

Put a HAProxy before your web-servers and activate sticky sessions. Therefore a user will only be directed to one varnish server which he visited before (and holds the same cached data).

To purge both varnish properly, your script or application should iterate over a list of all varnish servers and purge them one by one.

In my opinion, this solution isn't recommended. Varnish should support setting up a grid, where all servers can synchronize over time (eventual consistency).

Syncing multiple Varnish Cache servers

2 Answers2