7

We have a system which has one Apache instance in front of multiple tomcats. These tomcats then connect to various databases. We balance the load to the tomcat with mod_proxy_balancer.

Currently we are receiving 100 requests a second, the load on the Apache server is quite low, but due to database heavy operations on the tomcats, the load there is roughly 25% (of what I estimate they can handle).

In a few weeks there is an event happening and we estimate that our requests will jump significant, maybe by a factor of 10.

I'm doing everything I can do reduce the load on our tomcats, but I know we are going to run out of capacity, so I would like to fail gracefully. By this I mean, instead of trying to deal with too many connections which all timeout, I would like Apache to somehow monitor average response time, and as soon as the response time to Tomcat is getting above some threshold, I would like a error page displayed.

This means that users who are lucky still get a page rendered quickly, and those who are unlucky get a error page quickly. Instead of everyone waiting far too long for their page, and eventually everyone timing out, and the database being swamped with queries which are never used.

Hopefully this makes sense, so I was looking for suggestions on how I could achieve this.

thanks

bramp
  • 230
  • 2
  • 8
  • so did it work the way you wanted it using the accepted answer? – agent provocateur Apr 28 '17 at 19:48
  • Not exactly, because I was hoping for some clever heuristic based on observed latency, but instead I have to make an assumption that under N connections my latency stays good. – bramp May 04 '17 at 05:46

2 Answers2

9

I refer to this as a "Sorry Server". If you're using Apache 2.2 you can add another host to your LB pool as a hot spare, and when the actual app servers reach capacity your balancer will direct requests to the "Sorry Server" until the application servers become available again. Here's a rough idea:

<Proxy balancer://yourapp>
    BalancerMember http://10.0.0.1:8080 retry=5 max=50
    BalancerMember http://10.0.0.2:8080 retry=5 max=50
    BalancerMember http://10.0.0.3:8080 retry=5 max=50
    BalancerMember http://10.0.0.4:8080 retry=5 max=50
    # the hot standby on server2
    BalancerMember http://10.0.0.5:80 status=+H
</Proxy>
<Location /app>
    ProxyPass           balancer://yourapp
</Location>

Actually you could alternatively set up an extra vhost on your loadbalancer machine and have it serve the "Sorry Server" page itself. Hope that helps :)

Bryan White
  • 606
  • 3
  • 5
  • thanks Bryan, but if I understand this, the hot standby will only be used if (in your example) all four main server die (or are overwhelmed). What I want is none of the server to be overwhelmed, as maintain a target level of service, and if that can't be reached, some users see the sorry page, whilst others keep browsing the site. What I don't want, are browsers timing out, while tomcat is still rendering their request. – bramp Oct 20 '10 at 02:12
  • If you have a proper max connections configured on Tomcat then it won't get to the point where it's dog slow. Limit the connections to each instance to a point before Tomcat starts crawling. Then when the LB can't connect to a particular instance it will pass that particular request to the hot spare sorry server. As soon as a Tomcat node becomes available, requests will start going to it again. You can also do this at the LB level instead by adding the "max" option to BalancerMember. See my edit above. – Bryan White Oct 20 '10 at 12:34
  • From Apache2 docs for `max`: "Maximum number of connections that will be allowed to the backend server. The default for this limit is the number of threads per process in the active MPM. **In the Prefork MPM, this is always 1**, while with other MPMs it is controlled by the `ThreadsPerChild` directive." – rxgx Aug 28 '12 at 23:50
2

A couple of notes:

The "max" parameter sets the maximum connections per child process, which, depending on the MPM you're using, will not create a hard maximum of concurrent connections. For example, prefork MPM will be almost totally useless for that.

Instead, I'd set it up using the "timeout" parameter and a customized 503 error page. Set the timeout to some sane value, beyond which you don't want your users to wait, and put some meaningful message in the 503 error page.

So:

ErrorDocument 503 /sitebusy.html

<Proxy balancer://yourapp>
    BalancerMember http://10.0.0.1:8080 timeout=15 retry=5
    BalancerMember http://10.0.0.2:8080 timeout=15 retry=5
    BalancerMember http://10.0.0.3:8080 timeout=15 retry=5
    BalancerMember http://10.0.0.4:8080 timeout=15 retry=5
</Proxy>

ProxyPass /app balancer://yourapp timeout=5

With this setup each worker will be put into a failed state if its response goes beyond 15 seconds and will be put back into the pool 5 seconds later. The balancer will wait at 5 seconds for a free worker.

rxgx
  • 103
  • 4