I have a single Nginx server doing load balancing to several upstream servers.

What happens when a request comes in?

Does the load balancer keep the connection open and wait for an answer from the upstream servers? Or does it simply hand the request to an upstream server and "closes" the connection?

If the load balancer keeps the connection open until the request has been processed, it would mean that the load balancer is the bottleneck in my system, and it wouldn't matter if I increased the number of upstream servers.

  • Yes, nginx keeps connections. It will be bottleneck only for network subsystem. But usually bottleneck is CPU or storage. – Alexey Ten May 7 '15 at 8:40
  • But what about the number of incoming connections in the upstream servers are slow? – jimmiw May 7 '15 at 8:42
  • @jimmiw What do you mean ? – Xavier Lucas May 7 '15 at 8:44
  • Let's say that my load balancer can have 100 connections open at a time. If the upstream servers are too slow to respond, I would get an "website under heavy load" if the load balancer keeps the connection open if 101 requests come in. – jimmiw May 7 '15 at 8:46
  • Nginx could handle thousands of connections. While usual backend could only process tens/hundreds requests simultaneously – Alexey Ten May 7 '15 at 8:50

It depends what you tell nginx to do in your configuration and what proxying mode is set.

If your use case is proxy_pass or fastcgi_pass this is controlled by proxy_http_version and fastcgi_keep_conn combined to the keepalive directive in the upstream server block.

  • I am using proxy_pass, but changing the keepalive to a lower value would only make my load balancer close the connection to the upstream server and return an error page? – jimmiw May 7 '15 at 8:44
  • @jimmiw Huh ? Absolutely not. It will simply keep a certain amount of open connections in each woker's cache and close least recently used idle connections but won't end up with any error. If more connections are needed due to a load peak, then nginx will open new connections. – Xavier Lucas May 7 '15 at 8:48
  • So that means that if proxy_pass is used, the load balancer simply sends the request to an upstream server and doesn't wait for the response? – jimmiw May 7 '15 at 8:52
  • @jimmiw Yes, it will only "wake up" when some data arrives back on the socket and thus will either buffer it in memory or in a temporary file (depends on the configuration). Once the upstream server has finished pushing the reply to nginx, thus nginx will send the reply back. If your upstream servers support chunked encoding you can even send any part of data received from the upstream server back without any buffering of the full reply content. – Xavier Lucas May 7 '15 at 8:55

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.