In our infrastructure we have a load balancer(apache mod_balancer) and three backend servers that run apache as a web service and iptables as firewall, Lately ONE of them is giving a lot of " (110)Connection timed out: proxy: HTTP: attempt to connect to 10.0.0.15:80 (www.domain.com) failed" I checked the connections between this server and the load balancer through different port (9999) and it was working perfectly
I switch load balancer to contact the backend server through its public IP and the timeout error disappeared, which prove that the problem is not in backend apache server.
I tried to make the load balancer to get a file through port 80 from that backend server and it failed as well.
UPDATE :
<Proxy Balancer://cluster-fcgid lbmethod=byrequests timeout=10 maxattempts=2 >
BalancerMember http://backend.domain.com loadfactor=1 retry=30 max=130 timeout=40
BalancerMember http://backend2.domain.com loadfactor=1 retry=30 max=90 timeout=40
</Proxy>
The thrid backend server has the same configuration but I couldn't added it because of restrictions.
UPDATE :
the firewall table
*filter
:INPUT ACCEPT [0:0]
:FORWARD ACCEPT [0:0]
:OUTPUT ACCEPT [0:0]
:MYFIRE - [0:0]
-A INPUT -j MYFIRE
-A FORWARD -j MYFIRE
-A MYFIRE -i lo -j ACCEPT
-A MYFIRE -m state --state RELATED,ESTABLISHED -j ACCEPT
-A MYFIRE -p tcp -m state --state NEW -m tcp --dport ssh -j ACCEPT
-A MYFIRE -p tcp -m state --state NEW -m tcp --dport https -j ACCEPT
-A MYFIRE -i eth1 -p tcp -m state --state NEW -m tcp --dport http -j ACCEPT
-A MYFIRE -p icmp -m icmp --icmp-type any -j ACCEPT
-A MYFIRE -j DROP
COMMIT
What could be the source of the problem ? Any hints are greatly appreciated
Thanks