Load balancing across application servers

Question

This is more a question of theory in order to help me as a developer understand the process our sysadmins are going to use in app deployment to distribute our app. I'd really appreciate some insight, as it would help me plan for possible eventual pitfalls (and might end up as a valuable resource to others who have entry-level questions regarding the topic.)

Assuming there are several valid host machines that could answer any incoming HTTP request, what is the most basic approach to load balancing between them? My basic conjecture as a programmer would suggest probably something utilizing DNS, until traffic gets so high something more robust is required. I assume proxies are leveraged in some degree to check which hosts are busy answering ongoing processing and which ones are vacant.

I have an application data tier that is built entirely in Python using WSGI, and in testing it we've just utilized mod_wsgi in Apache. If one was to theoretically deploy this on several identical machines, all connected to the same databases/resources in the application layer, what would be the first order of business in spreading out requests to the overall domain across them? Just listing multiple A records in the dns?

I am sorry for the vagueness, just haven't been able to find a decent primer on the subject from a developer's standpoint. I appreciate the help!

Answer 1

A typical setup will actually work in reverse of what you propose - first implementing a front-end load balancer (e.g. HAProxy, Nginx, even Varnish, etc - or hardware based), and only later using DNS.

Incoming requests will reach the load balancer, which will typically be able to perform several functions, including:

• Keep track of the status (i.e. health) of the back-end servers
• Intelligently decide which one to choose (e.g. lowest load, round-robin, specific request type, etc.)
• Try another server if the first server doesn't respond appropriately
• Mask your backend servers

There are a few advantages to this approach:

• You retain control over all aspects of the setup
• You can add and remove backend servers easily
• Load balancing is done 'intelligently'

The problem, however, is that all traffic has to flow through the single load balancer. On one level that makes it a significant point of failure (but you can failover to reduce the impact). More significantly though, at some point the single node will not be able to handle the traffic that needs to flow through it. The latter is the problem that is resolved with DNS - such that requests return different IPs, corresponding to different load balancers.

Answer 2

The load balancing approach completely depends on the design specifications that hopefully you've provided to the admins. There is no "this is how to load balance" guide because there are multiple ways to do it depending on the website and the robustness of the solution required. EG for a static website that only needs redundancies and has no business impact round robin DNS would be fine. If you just need redundancy then a cluster would be fine. If you need 99.999 reliability and loadbalancing based on response time you will need redundant dedicated hardware network loadbalancers.

As an admin I need to know what exactly your application does and where it will ramp up load to determine how best to implement loadbalancing. Additionally if this has a database backend it also needs to be redundant. These are all of the things that either you should have been asked or you should have provided to your admins in order to adequately design the server infrastructure.

Answer 3

Load balancing HTTP requests can be done using several ways. DNS round-robin is one to do it way, but you don't have much control over it.

Another and better approach will be to use a dedicated hardware for load balancing. For example, you can run a Linux box and install a load balancing software like haproxy.

One important issue you need to watch for is to have the same view from the client side regardless of chosen server from server cloud.