1

I'm currently running several different services (mostly spring boot apps) behind Apache httpd in a Reverse proxy configuration. Each service is on an individual instance with an auto-scaling group that feeds an internal elb. The internal elb hostname is the host in the ProxyPass and ProxyPassReverse to simplify host discovery.

Recently ran into an issue caused by the ip addresses behind the ELB changing. The httpd connection pool was keeping connections (or caching connections) to the old ip addresses which was fixed by bouncing httpd. Looking at the mod_proxy configure I saw the disableReuse flag and thought that would solve the problem however it caused a new one. Now my httpd instances are using 4-5x the CPU due to all the new connections it is now creating.

Does anyone have any recommendations for how to configure this setup? Should I re-enable reuse and set a ttl on the Proxy?

Robby Pond
  • 111
  • 3
  • Question: are you using a TCP or HTTP ELB? Two things I'd look at 1) Can httpd test the connection before it uses it? 2) Consider putting a TTL on the connection, it won't fix the problem but it will limit how long it will go on for. – Tim Mar 29 '17 at 00:29
  • Look at KeepAliveTimeout and MaxKeepAliveRequests https://httpd.apache.org/docs/2.4/mod/core.html#keepalivetimeout and https://httpd.apache.org/docs/2.4/mod/core.html#maxkeepaliverequests – Tim Mar 29 '17 at 00:34
  • *"The httpd connection pool was keeping connections (or caching connections) to the old ip addresses"* ... I think that may be a misdiagosis. When the ELB IPs change, those old addresses are unbound from the ELB and any connections would be severed. I'd be far more likely to believe that apache is caching the DNS lookup responses -- HAProxy and Nginx have had to solve that problem and Apache might have the same issue -- at least by default. Both of the other platforms formerly resolved them only once, at config parse time. Maybe Varnish, too. ELB made that an unacceptable optimization. – Michael - sqlbot Mar 29 '17 at 02:48
  • Why do you need apache in front of ELB at all? Of course I may be wrong but looks like you hold an axe from the wrong side. – Putnik Mar 29 '17 at 07:01

0 Answers0