I know that there are some situations that we encounter heavy http(s) load on a server and we should use a load balancer to distribute load among several back-end servers.
But I've a confusion here! Lets suppose that too many users (lets say 100,000) are trying to access simultaneously to example.com:80 and a load balancer (e.g Apache mod_proxy_balancer or any other stand-alone load balancer) wants to distribute this load among several back-end servers, but still all traffic has to be passed across the front-end server which brings about a heavy traffic on that server & even though all the requests are parsing at back-end servers, front-end server still have to manage them (suppose that in best case it has to create a single unique thread for every request which causes this server simply crash after creating 100,000 threads at a moment!).
Now my question (that may sounds funny to an expert!) is that how a load balancer can handle such situation? In other words, how front-end server can establish a link directly between clients & back-end servers without having to tolerate such heavy network load? (I know in this situation we can't call back-end servers really 'back-end'! but assume that its OK for now) Does this scenario need any application-specific alter? (e.g. in order to simply redirect clients with http Location header to other servers)


This depends on the purpose of the load balancer i.e. what type of workload it's supposed to balance. The reverse proxy based HTTP(S) load balancers are distributing the load between application back-ends generating the content.

Reverse proxies are not trying to balance network load, and compared to all the tasks dynamic applications do, passing network traffic (and possibly serving static content) is a relatively simple task.

For example, database access can be load balanced, too. Most of the database queries involves only reading from the database, so while writes needs to be replicated immediately across all the database servers, reads can be distributed evenly.

Network load balancing is a different area. For example, one common method for balancing (network) load between cluster servers is round-robin DNS records, where the same hostname simply has more than one A records.

  • Yup. Of course, if you have some webapp that is using sessions or similar to store data - authentication status, profile info, etc - then those need to stay on the same server instance, OR you need to use a centralized db for all of the servers (or both). So another possibility is to have one A record, which then redirects randomly (or in round robin order) to different hostnames that all serve the same content. User goes to example, is redirected to ww2.example or ww32.example or whatever and then stays there for the duration of their session. – ivanivan Aug 5 '17 at 14:31
  • 2
    A smart load balancer can always choose the same back-end for the same client. Among many other designs. This is just a quick overview. :) – Esa Jokinen Aug 5 '17 at 14:36
  • 1
    To add, There is also the redundancy aspect together with the load sharing aspect. A load balancer can allow backend nodes to go offline, without impact to end users if the application allows it. Moving the same thought up one layer brings the focus to load balancer redundancy: two or more load balancers may share a common cluster address, providing both load sharing and redundancy at the load balancer layer itself. This answers the 100,000 threads part. – ErikE Aug 6 '17 at 22:59

Your Answer

By clicking "Post Your Answer", you acknowledge that you have read our updated terms of service, privacy policy and cookie policy, and that your continued use of the website is subject to these policies.

Not the answer you're looking for? Browse other questions tagged or ask your own question.