We have a set of 6 backends that serve our website. Because we use multiple datacenters we have found the best performance occurs for us when we have varnish send the request to the localhost httpd server (running on port 81). This of course is a very basic configuration well supported by varnish and can be accomplished using the fallback director:
director default fallback {
{ .backend = localbackend; }
{ .backend = web1; }
{ .backend = web2; }
{ .backend = web3; }
etc...
}
However the fallback director tries the other backends in order till it finds a healthy one. The BIG problem is that in the above config web1 will take all the traffic if localbackend fails !!! This will overload web1 and it will become sick. Then all requests go to web3... and now it will get 3x the normal traffic... cause a cascading failure.
Thus we need instead a config that will allow all requests to sent to the localhost httpd server if it is health, but if not, send the requests to the other healthy servers in a round robin type fashion.
Thank you in advance for any suggestions and solutions you might think of... Your help is GREATLY appreciated.