1

I have a single Nginx server doing load balancing to several upstream servers.

What happens when a request comes in?

Does the load balancer keep the connection open and wait for an answer from the upstream servers? Or does it simply hand the request to an upstream server and "closes" the connection?

If the load balancer keeps the connection open until the request has been processed, it would mean that the load balancer is the bottleneck in my system, and it wouldn't matter if I increased the number of upstream servers.

jimmiw
  • 98
  • 7
  • Yes, nginx keeps connections. It will be bottleneck only for network subsystem. But usually bottleneck is CPU or storage. – Alexey Ten May 07 '15 at 08:40
  • But what about the number of incoming connections in the upstream servers are slow? – jimmiw May 07 '15 at 08:42
  • @jimmiw What do you mean ? – Xavier Lucas May 07 '15 at 08:44
  • Let's say that my load balancer can have 100 connections open at a time. If the upstream servers are too slow to respond, I would get an "website under heavy load" if the load balancer keeps the connection open if 101 requests come in. – jimmiw May 07 '15 at 08:46
  • Nginx could handle thousands of connections. While usual backend could only process tens/hundreds requests simultaneously – Alexey Ten May 07 '15 at 08:50
  • @jimmiw Nginx uses a very efficient programming pattern called reactor which is asynchronous and is based on events. I don't know which load you are expecting but nginx can manage a tremendous amount of connections due to this design. If you upstream servers are too slow to respond, then that's not due to nginx. – Xavier Lucas May 07 '15 at 08:51

1 Answers1

3

It depends what you tell nginx to do in your configuration and what proxying mode is set.

If your use case is proxy_pass or fastcgi_pass this is controlled by proxy_http_version and fastcgi_keep_conn combined to the keepalive directive in the upstream server block.

Xavier Lucas
  • 13,095
  • 2
  • 44
  • 50
  • I am using `proxy_pass`, but changing the `keepalive` to a lower value would only make my load balancer close the connection to the upstream server and return an error page? – jimmiw May 07 '15 at 08:44
  • @jimmiw Huh ? Absolutely not. It will simply keep a certain amount of open connections in each woker's cache and close least recently used **idle** connections but won't end up with any error. If more connections are needed due to a load peak, then nginx will open new connections. – Xavier Lucas May 07 '15 at 08:48
  • So that means that if `proxy_pass` is used, the load balancer simply sends the request to an upstream server and doesn't wait for the response? – jimmiw May 07 '15 at 08:52
  • @jimmiw Yes, it will only "wake up" when some data arrives back on the socket and thus will either buffer it in memory or in a temporary file (depends on the configuration). Once the upstream server has finished pushing the reply to nginx, thus nginx will send the reply back. If your upstream servers support chunked encoding you can even send any part of data received from the upstream server back without any buffering of the full reply content. – Xavier Lucas May 07 '15 at 08:55