I'm currently investigating load balancing with Apache mod_load_balancer and mod_proxy. I will also be looking at other load balancers later but one thing has become clear. Why do hardly any of the load balancers (if any at all) make distribution decisions based on the actual load of the worker machines.

Apache, for example, distributes requests based on number of requests, amount of data throughput and length of request queue. Why don't they have some mechanism to distribute requests to the machine with the lowest CPU or memory usage.

I'm building a system where each request requires a lot of CPU to the point that 2 or 3 worker machines can only service 10 or 20 concurrent clients before I've max'ed out all their CPUs. Some requests for xml are really lightweight while others for 'stuff' are really heavy.

Does it really make any difference in the scheme of things? Does one find that even a CPU based distribution algorithm settles down into a round robin style eventually. Does it add extra overhead that makes it not worth it.

Are there any other load balancers that offer this facility, do they offer it and no one uses it for what ever reason.

It seems like something that would be really good but no one seems to implement it. I'm confused and need a little bit of advice on the subject.

2 Answers 2


One of the main issues with resource based load balancing is that the load information becomes stale by the time you would make the routing decision. There is an academic paper on the topic of staleness that you might want to read called Interpreting State Load Information. You can get nasty side effects like sending too much load to a box that seems under-utilized and then overwhelm it. In short, load based balancing seems like the best way to do it at first to everyone but it turns out simple methods tend to work better in practice.

In most load balancing simple algorithms are usually fine because either the transactions are short lived or they cause such low load that a round-robin or random distribution will be close enough to a good balance. There generally needs to be overhead to absorb the load from failed servers anyways (if you are close the max utilization on all 3, as soon as one dies the load will cascade and you lose the whole cluster).

One solution might be to create two queues, one for the "heavy stuff" and one for the "light stuff". I would call the "light stuff" load balancing and the "heavy stuff" job scheduling - in other words they seem like different problems. Then just have limit to max number of sessions per each client and a universal queue for them for the job scheduling. I don't know of an ideal tool for that off the top of my head though.


Generally I've found that "load" is a very specific term that is derived from a large number of metrics that vary from application to application. As no front end load balancer can know this detail (out of the box) and the fact that round robin with some carefully tuned connection limits basically works, it's not always worth the pain.

In environments I work where the applications are developed in house I try and push for a "monitor" page to be a part of the application that the load balancer's use to monitor the node status (i.e up/down). This can then be extended to include an integer load factor that a load balancer can use to adjust load to that node. All we define is the consistent interface and what that integer value will cause the load balancer to do. Then it's up to the apps and a lot of load testing to make sure the application isn't doing anything too funky to load distribution.

Following on from Kyle Brandt's mention of work queues, one of the alternate methods of load balancing is for workers to pull requests in from a queue rather than the standard load balancer pushing requests to a worker (or crammed into, as the case may be). An example of this setup is the Mongrel2 web server (http://mongrel2.org/) and it's use of 0MQ (http://www.zeromq.org/) as a distribution mechanism to application workers. As a machine slows due to heavy processing it won't retrieve new requests from the queue. You can still run into lots of 'heavy' requests being retrieved at once but the issue is short circuited more easily by the workers themselve's. It would be fairly trivial to divide work into 'typed' queues as Kyle suggested. This is a fairly big change for a backend application to support the pull of 0MQ requests but if your starting from scratch then it might be a way forward.

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.