0

I am running Varnish 3.0 in front of an Apache2 + PHP 5.3 server. My problem was that Varnish threw many 503 Service Unavailable errors, so I added a workaround, saying something like:

if http_code = 503 and cookie(REDIRECTED) = false
  set-cookie REDIRECTED=true
  header 307 redirect to same page // try again, maybe this time it doesn't fail
else if http_code = 503 and cookie(REDIRECTED) = true
  header 503 // we tried, but the server doesn't want to

This is a sketch in pseudo-code of the main idea what I did. However, when the 307 redirect is being made, the cookies sent by apache in the Set-Cookie header, are not sent. My main concern is that the PHPSESSID cookie isn't being sent. If I don't get an 503 error and Varnish doesn't have to make the redirect, all works fine. Any ideas?

Eduard Luca
  • 6,514
  • 16
  • 85
  • 137
  • Are the 503 errors being thrown by Apache or by Varnish? If they are thrown by Varnish, and your workaround is done in Apache, the cookies obviously won't be set. – Ketola Aug 29 '12 at 12:17
  • No, both the errors and the workaround is in Varnish. Apache is perfectly fine (I've checked the logs and no 503 are thrown in Apache whatsoever) – Eduard Luca Aug 29 '12 at 12:22

3 Answers3

1

Hopefully I understood your question right. I don't really see how PHPSESSID is related to handling the 503 errors, but you don't need to use cookies at all to re-try a request.

You can use restarts in Varnish to accomplish what you are trying to do. For instance to try the request 2 times you could do:

sub vcl_error {
    if (obj.status == 503 && req.restarts < 2) {
        return (restart);
    }
}

This should provide the functionality you are looking for without touching the cookies.

This is obvious, but I will say it anyway: what you really should do is find out what is causing the 503 errors. If they are not coming from the backend, they are caused either by timeouts or a problem with the Varnish configuration. Keep an eye on your Varnish log for "TxStatus c 503" lines and see what's going on with the requests.

Update regarding the possible cause of 503 errors:

Apparently you are receiving the 503 errors on POST requests (i.e. when a user tries to log in). This can occur if posts are done over slow links or the connection is terminated by the client for some other reason before POST body is completely transferred. However this should not show up as no error recorded in varnishlog. You might want to give the following a shot in any case to see if it fixes your problem.

Varnish defaults to return(pass) for POST requests, but this doesn't work as expected in all cases (see Varnish ticket #849 for more information).

We have opted to pipe all POST requests and append X-Forwarded-For to the requests as well. Here's the relevant VCL:

# PIPE all POST requests

sub vcl_recv {
 if (req.request == "POST") {
   return (pipe);
 }
}

# Set XFF for piped requests

sub vcl_pipe {
  set bereq.http.Connection = "close";
  set bereq.http.X-Forwarded-For = req.http.X-Forwarded-For;
  if (bereq.http.x-forwarded-for) {
    set bereq.http.X-Forwarded-For = bereq.http.X-Forwarded-For + ", " + client.ip;
  } else {
    set bereq.http.X-Forwarded-For = client.ip;
  }
  return (pipe);
}
Ketola
  • 2,767
  • 18
  • 21
  • You almost understood my question. The solution you gave me would work perfectly if my 503 errors would come from my backend, but they are coming from varnish itself, so that's why I have to do a 307 (temporary redirect) to the same page, so the 2nd request may work, however the cookies get deleted. We fixed the issue however by installing a varnish module. Will followup with a detailed answer. – Eduard Luca Aug 30 '12 at 12:47
  • obj.status _should_ be 503 even if the error is coming from Varnish instead of the backend. This can be easily simulated by defining a bogus backend or taking the backend offline completely. Furthermore if you are setting cookie on every response, you are not using Varnish for caching at all.. – Ketola Sep 03 '12 at 14:17
  • You are right, I've mistaken `obj.status` with `beresp.status`. I am not setting cookies at every request, only at login, but if the cookie doesn't get set at login, I'll have a big problem. I'll try your solution, I think it will work. Thanks! Edit: I also checked the logs, but they don't provide any info, the error message is: `no error recorded`. I also noticed these 503s are thrown when the server is under load, but I already have 3 load-balanced servers which should take care of the traffic. – Eduard Luca Sep 03 '12 at 17:35
  • Thank you for the update. I think I understand your problem better now. I will edit my answer to include another alternative. – Ketola Sep 04 '12 at 13:16
0

The only way we could fix this, was by installing a varnish module called vmod-headers, which allows you to append headers to other headers.

After that we just appended the cookie we wanted to set, to the already existing cookies which came from the backend.

Eduard Luca
  • 6,514
  • 16
  • 85
  • 137
-1

The default vcl shouldn't remove any cookies nor cache any pages without cache headers.

The reason you get 503 from varnish is because your apache is not responding in time. If you want it to be more forgiving you can change the following settings (awfully high below) for the backend:

backend www {
    .host = "www.example.com";
    .port = "http";
    .connect_timeout = 500s;    
    .first_byte_timeout = 500s;
    .between_bytes_timeout =500s;
}
Clarence
  • 2,944
  • 18
  • 16
  • I *can* search on the internet and I did that the first time I got 503s and the first thing I did was to increase the timeouts, which did exactly nothing. – Eduard Luca Aug 27 '12 at 10:00
  • And yet you did not bother to say you knew the reason of the 503. Nor did you post a single line of your VCL. Good luck to you, hopefully someone else still has patience with you. – Clarence Aug 27 '12 at 12:02
  • Because I didn't ask why I am getting these 503s, I just wanted to know about the overwritten cookies, that's all. – Eduard Luca Aug 27 '12 at 12:06