Elastic Beanstalk purging Varnish across instances

Question

I’m a little stuck with how best to clear Varnish across all my instances on Elastic Beanstalk - unfortunately I’m not most knowledgeable with AWS.

I would like to keep things simple regarding the set up so I opted to have a Varnish cache on each server rather than separate Varnish servers.

Now if I have site which can clear the cache, it would go through the load balancer and only hit one server. From there I guess I could then hit the reach of my instances but at that stage I have no idea what the IPs are each instances are. So would I need to use AWS Cli to grab all instances then trigger a clear on each? Or is there a way when a certain URL is called (maybe with header) that it reaches all instances behind that load balancer.

I would really appreciate some guidance on this.

Below is my config, one thing that might look a little weird is the X-Purge-Token - I’ve decided it might make sense if a request comes from the load balancer or another instances check the header for token before allowing purge. I originally planned to use X-Forwarded-For but because IPs change I wasn’t sure the best route for this.

option_settings:
  - namespace: aws:elasticbeanstalk:application:environment
    option_name: COMPOSER_HOME
    value: /root

  - option_name: APP_ENV
    value: placeholder
  - option_name: APP_KEY
    value: placeholder

packages:
  yum:
    varnish: []

files:
  "/etc/varnish/default.vcl":
    owner: root
    group: root
    content: |
      backend default {
          .host = "127.0.0.1";
          .port = "8080";
      }

      acl elb {
        "172.31.0.0"/20;
        "172.31.16.0"/20;
        "172.31.32.0"/20;
      }

      acl purge {
        "localhost";
        "127.0.0.1";
        "::1";
        "86.19.111.22";
        "213.106.111.22";
      }

      sub vcl_recv {
        if (req.request == "PURGE") {

          set req.http.xff = regsub(req.http.X-Forwarded-For, "^[^,]+.?.?(.*)$", "\1");

          if(client.ip ~ elb) {
            if (req.http.X-Purge-Token == "tFjn3MnrunWzi49x") {
              return (lookup);
            }
          } else {
            if (client.ip ~ purge) {
              return (lookup);
            }
          }

           error 405 "Not allowed.";
        }
        if (!(req.url ~ "(preview=true|wp-login|wp-admin)")) {
          unset req.http.cookie;
        }
      }

      sub vcl_fetch {
        if (beresp.ttl == 120s) {
          set beresp.ttl = 24h;
        }

        if (!(req.url ~ "(preview=true|wp-login|wp-admin)")) {
          unset beresp.http.set-cookie;
        }
      }

      sub vcl_hit {
        if (req.request == "PURGE") {
          if(req.http.X-Purge-Method == "regex") {
            ban("req.url ~ " + req.url + " && req.http.host ~ " + req.http.host);
          }
          purge;
          error 200 "Purged.";
        }
      }

      sub vcl_miss {
        if (req.request == "PURGE") {
          if(req.http.X-Purge-Method == "regex") {
            ban("req.url ~ " + req.url + " && req.http.host ~ " + req.http.host);
          }
          purge;
          error 200 "Purged.";
        }
      }



commands:
  001_update_composer:
    command: export COMPOSER_HOME=/root && /usr/bin/composer.phar self-update
  010_httpd.conf:
    command: "sed -i 's/Listen 8080/Listen 80/g' /etc/httpd/conf/httpd.conf"
  011_httpd.conf:
    command: "sed -i 's/Listen 80/Listen 8080/g' /etc/httpd/conf/httpd.conf"
  040_varnish:
    command: "sed -i 's/VARNISH_LISTEN_PORT=6081/VARNISH_LISTEN_PORT=80/g' /etc/sysconfig/varnish"
  041_varnish:
    command: "sed -i 's/VARNISH_ADMIN_LISTEN_PORT=6082/VARNISH_ADMIN_LISTEN_PORT=2000/g' /etc/sysconfig/varnish"

container_commands:
   "1-storage-permissions":
      command: "mkdir -p public/content/uploads && chmod -R 755 public/content/uploads"
   "2-install-dependencies":
      command: "/usr/bin/composer.phar install"

services:
  sysvinit:
    varnish:
      enabled: true
      ensureRunning: true

score 0 · Answer 1 · answered Aug 07 '15 at 06:42

I think using the CLI is your best bet. Tag up your Elastic Beanstalk environments with appropriate tags which you can use to identify your EC2 instances. For example, an EnvironmentName and ServiceName tag.

Then you could do the following CLI call to get the private IP addresses of those instances which you then can use for sending individual purge requests to.

aws ec2 describe-instances \
    --region eu-west-1 \
    --filter Name=tag:EnvironmentName,Values=production \
    --filter Name=tag:ServiceName,Values=cache \
    --query "Reservations[].Instances[].PrivateIpAddress"

You would of course need to replace the values of the region, EnvironmentName and ServiceName so it fits your setup.

Elastic Beanstalk purging Varnish across instances

1 Answers1