I hope you don't mind me asking this as I couldn't find a definitive answer.

I don't have a problem, per-say. I am just wanting to research something for my own studies.

I was wondering just how Load Balancing worked. I know that a Load Balancer accepts requests and then handles all of the different connection requests and then will connect to a server that isn't failing or close to failing. What I'm interested in is, do all of the servers have to be mirrored in order for this to work? For example Facebook will have most probably peta-bytes of information stored on its servers. If they ran out of space on a Server how would they implement more space? If they added a new Server they wanted to balance the load to wouldn't that have to be mirrored as well and it would then become full. I know this isn't perhaps the best example to use but it's the only company apart from Google that will have so much data on their hands, there must be some cut off limit as to how much data you can store on a Server even for them.

Another query I have is is there a limit to the number of Load Balancing Servers to use? I know some balancers accept 7000 concurrent connections, so if I installed 3 Load Balancing Servers would it not theoretically be able to handle 21000 concurrent connections? I hope this makes sense. I'm new to the Server game.


The term "load balancer" is a very broad one, and there is no one answer to your questions as a result (for different sorts of load balancers, the answers will be different). For example, your description of a load balancer as "accept[ing] requests [...] and then [...] connect[ing] to a server that isn't failing or close to failing" isn't correct for some types of load balancer.

In general, a load balancer distributes work to more than one backend or "worker" servers or processes. How it does that varies by the sort of work to distribute (depending on the work and environment, it could use connection proxying, connection redirects, or various sort of work queues that the workers retrieve jobs from). The algorithm by which the load balancer determines which backend(s) are OK for service, and which one to direct a particular request to or when to retrieve a failed work request from a backend is, again, entirely application specific.

The state and nature of the backends is infinitely varied, too. Some architectures involve having a large pool of perfectly mirrored backends performing the same work, while others have a multi-tier architecture, where a pool of servers take requests and then make requests to other load-balanced services within the infrastructure. Large storage is usually a tier of it's own, and requests are fed into the storage cluster and handled in some other manner -- Google's GFS is an example of this. In very large cases, the workers making the requests for stored resources will themselves have a lookup mechanism to determine which storage cluster will have the information they need, as in Github's repository storage system.

As for the number of servers to use, it all comes down to per-backend performance and redundancy. As you surmise, if one server can handle 7000 concurrent connections, then three servers will be able to handle more-or-less 21000 concurrent connections -- assuming no bottlenecks in another tier (like your database server siezing up, or the load-balancer itself being unable to handle that many connections). However, with N servers each with a probability of failure P, the chances of a single machine failure is N*P; as you get larger, you will have a machine failure more often. As such, you typically want a certain amount of excess capacity -- which also deals with unexpected load peaks.

  • Thank you very much for the excellent response. I've had a look into the multitiered system that you mentioned, is that where there are many different layers of servers which share the load in order to make it more secure? I hope that made sense! Also is there any way to configure a Load Balancer so that if a request to say it.co/photos would point to a different Load Balancer which would balance between all of the photo servers? Or if you made a request to it.co/events, it would point to a different load balancer dealing with all of the events servers. Apologies for all the questions. – Jamie McLeish Oct 8 '11 at 11:50
  • A multi-tier system is about separation of responsibilities. You can configure a load balancer to do just about anything -- how to do a specific thing should be addressed as a separate question, not in a comment. – womble Oct 8 '11 at 23:19

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.