1

I am trying to work out why my application keeps hitting the database while I have setup varnish infront of apache. I think I am missing some vital configuration, any tips are welcome

This is my curl result:

HTTP/1.1 200 OK
Server: Apache/2.2.16 (Debian)
Content-Language: en-us
Vary: Accept,Accept-Encoding,Accept-Language,Cookie
Cache-Control: s-maxage=60, no-transform, max-age=60
Content-Type: application/json; charset=utf-8
Date: Sat, 15 Sep 2012 08:19:17 GMT
Connection: keep-alive

My varnishlog:

   13 BackendClose - apache
   13 BackendOpen  b apache 127.0.0.1 47665 127.0.0.1 8000
   13 TxRequest    b GET
   13 TxURL        b /api/v1/events/?format=json
   13 TxProtocol   b HTTP/1.1
   13 TxHeader     b User-Agent: curl/7.19.7 (universal-apple-darwin10.0) libcurl/7.19.7 OpenSSL/0.9.8r zlib/1.2.3
   13 TxHeader     b Host: foobar.com
   13 TxHeader     b Accept: */*
   13 TxHeader     b X-Forwarded-For: 92.64.200.145
   13 TxHeader     b X-Varnish: 979305817
   13 TxHeader     b Accept-Encoding: gzip
   13 RxProtocol   b HTTP/1.1
   13 RxStatus     b 200
   13 RxResponse   b OK
   13 RxHeader     b Date: Sat, 15 Sep 2012 08:21:28 GMT
   13 RxHeader     b Server: Apache/2.2.16 (Debian)
   13 RxHeader     b Content-Language: en-us
   13 RxHeader     b Content-Encoding: gzip
   13 RxHeader     b Vary: Accept,Accept-Encoding,Accept-Language,Cookie
   13 RxHeader     b Cache-Control: s-maxage=60, no-transform, max-age=60
   13 RxHeader     b Content-Length: 6399
   13 RxHeader     b Content-Type: application/json; charset=utf-8
   13 Fetch_Body   b 4(length) cls 0 mklen 1
   13 Length       b 6399
   13 BackendReuse b apache
   11 SessionOpen  c 92.64.200.145 53236 :80
   11 ReqStart     c 92.64.200.145 53236 979305817
   11 RxRequest    c HEAD
   11 RxURL        c /api/v1/events/?format=json
   11 RxProtocol   c HTTP/1.1
   11 RxHeader     c User-Agent: curl/7.19.7 (universal-apple-darwin10.0) libcurl/7.19.7 OpenSSL/0.9.8r zlib/1.2.3
   11 RxHeader     c Host: foobar.com
   11 RxHeader     c Accept: */*
   11 VCL_call     c recv lookup
   11 VCL_call     c hash
   11 Hash         c /api/v1/events/?format=json
   11 Hash         c foobar.com
   11 VCL_return   c hash
   11 VCL_call     c miss fetch
   11 Backend      c 13 apache apache
   11 TTL          c 979305817 RFC 60 -1 -1 1347697289 0 1347697288 0 60
   11 VCL_call     c fetch deliver
   11 ObjProtocol  c HTTP/1.1
   11 ObjResponse  c OK
   11 ObjHeader    c Date: Sat, 15 Sep 2012 08:21:28 GMT
   11 ObjHeader    c Server: Apache/2.2.16 (Debian)
   11 ObjHeader    c Content-Language: en-us
   11 ObjHeader    c Content-Encoding: gzip
   11 ObjHeader    c Vary: Accept,Accept-Encoding,Accept-Language,Cookie
   11 ObjHeader    c Cache-Control: s-maxage=60, no-transform, max-age=60
   11 ObjHeader    c Content-Type: application/json; charset=utf-8
   11 Gzip         c u F - 6399 69865 80 80 51128
   11 VCL_call     c deliver deliver
   11 TxProtocol   c HTTP/1.1
   11 TxStatus     c 200
   11 TxResponse   c OK
   11 TxHeader     c Server: Apache/2.2.16 (Debian)
   11 TxHeader     c Content-Language: en-us
   11 TxHeader     c Vary: Accept,Accept-Encoding,Accept-Language,Cookie
   11 TxHeader     c Cache-Control: s-maxage=60, no-transform, max-age=60
   11 TxHeader     c Content-Type: application/json; charset=utf-8
   11 TxHeader     c Date: Sat, 15 Sep 2012 08:21:29 GMT
   11 TxHeader     c Connection: keep-alive
   11 Length       c 0
   11 ReqEnd       c 979305817 1347697288.292612076 1347697289.456128597 0.000086784 1.163468122 0.000048399

Varnish config

## Redirect requests to Apache, running on port 8000 on localhost
backend apache {
        .host = "127.0.0.1";
        .port = "8000";
}
## Receive
sub vcl_recv {
        if (req.url ~ "^/api") {
                set req.http.host = "foobar.com";
                set req.backend = apache;
        } else {
                set req.backend = apache;
        }
        remove req.http.Cookie;
        return( lookup );
}

## Fetch
sub vcl_fetch {
        ## Remove the X-Forwarded-For header if it exists.
        remove req.http.X-Forwarded-For;

        ## insert the client IP address as X-Forwarded-For. This is the normal IP address of the user.
        set    req.http.X-Forwarded-For = req.http.rlnclientipaddr;
        ## Added security, the "w00tw00t" attacks are pretty annoying so lets block it before it reaches our webserver
        if (req.url ~ "^/w00tw00t") {
                error 403 "Not permitted";
        }

        if(req.http.host == "foorbar.com") {
                ## force 1 day cache on any requests
                set beresp.ttl = 86400s;
                ## if clients will check for an update every hour, this will just come from varnish
                set beresp.http.Cache-Control = "max-age=3600";
        }

        ## Deliver the content
        return(deliver);
}

## Deliver
sub vcl_deliver {
        ## We'll be hiding some headers added by Varnish. We want to make sure people are not seeing we're using Varnish.
        ## Since we're not caching (yet), why bother telling people we use it?
        remove resp.http.X-Varnish;
        remove resp.http.Via;
        remove resp.http.Age;

        ## We'd like to hide the X-Powered-By headers. Nobody has to know we can run PHP and have version xyz of it.
        remove resp.http.X-Powered-By;
}
Hedde
  • 73
  • 6
  • Config of varnish please. – 3molo Sep 15 '12 at 09:33
  • I added my vcl config, note: I added the vcl_recv method after reading (of a places) a worldofwarcraft post about varnish http://us.battle.net/wow/en/forum/topic/3082211205 which seems to actually speed things up – Hedde Sep 15 '12 at 10:10
  • The config looks sane at first glance. I get the feeling that either it's not loaded, and the Max-age: 60 delivered by apache still stands, or it's simply not cached as '11 VCL_call c miss fetch' also indicates a cache miss. What if you run the same curl twice? – 3molo Sep 15 '12 at 11:00
  • 1
    @3molo you were right it didn't pick up the config file it still pointed to default I had to manually kill the process and restart varnish – Hedde Sep 15 '12 at 11:48

1 Answers1

1

Putting it in an answer then:

Your config looks fine, you need to load it either by using varnishadm or simply by restarting varnish. Using varnishadm's vcl.load and vcl.use you can have it reloaded without losing the cache:

$ varnishadm -T localhost:6081

vcl.load update1 /etc/varnish/default.vcl
vcl.use update1
3molo
  • 4,330
  • 5
  • 32
  • 46