1

Setup that we're trying to evaluate is pretty straightforward - NginX on one machine (currently 1vCPU, 2GB RAM), two back-end Tomcats on two separate machines. Core configuration as follows: 

worker_processes  1;
events {
    worker_connections  2048;
    use epoll;
    # multi_accept on;
}
http {
    # define load-balancing upstream
    upstream tomcatUpstream {
        ip_hash;
        server 192.168.xx.51:8080;
        server 192.168.xx.52:8080;
    }

    include       /etc/nginx/mime.types;
    default_type  application/octet-stream;

    sendfile      on;
    tcp_nopush    on;
    tcp_nodelay   on;

    keepalive_timeout  0;     #  45;
    # keepalive_requests 4096;

    reset_timedout_connection on;
    client_body_timeout 30;
    send_timeout 15;
}

We tried running ab -c10 -n20 from 3 different machines (3 different public IPs - 2 different cities, 3 different ISPs, although within the same country) at the same time.

No matter what I changed or tried within configuration (epoll yes/no, multi_accept on/off, keep-alive 0 / !=0, restart, deleting var/cache/nginx...), only one back-end Tomcat was hit, and always the first one by order in the upstream list, as I tried changing the order as well.

For a bit more context - these two Tomcats serve dynamically generated binary files (binary per request, 60 - 200KB in size), and the response times are not very fast.

When ip_hash is disabled, requests are distributed on both back-ends. So, there must be something wrong either with our tests or NginX configuration...?

Less
  • 121
  • 1
  • 5
  • Does nginx listen to the world or you have some kind of gateway? – Alexey Ten Oct 22 '15 at 09:18
  • 1
    Does access log show different client IPs? BTW, it's unlikely but to impossible that 3 different IPs fall into same hash bucket. – Alexey Ten Oct 22 '15 at 09:21
  • 1
    Do you see different ip in your nginx access.log? – Guido Vaccarella Oct 22 '15 at 09:21
  • @Alexey Ten can't believe that it did not occur to me to check access log... you may be right, log shows only one IP (internal gateway, 192.168.xx.1), even though the NginX is opened to the world... but apparently via NAT..? – Less Oct 22 '15 at 09:29
  • @alexey-ten if you wanted to make an answer out of your comments, maybe with a suggestion how to properly configure this with 2 NICs, or something, I'd be glad to accept it. – Less Oct 23 '15 at 07:16

0 Answers0