Active - Active Appserver Loadbalancing strategy

Question

Have a tomcat web application running across 6 servers.

3 In our primary site and 3 in the DR site

Our networking team setup 2 sets of load balancing

First layer is Site Level, splits traffic 50/50 between Primary and DR

Second layer is on each site to distribute traffic to each of the 3 servers in that site.

The problem I see occurring is 2 out of the 3 servers in a site will go down, and a single server is now handling 50% of our overall traffic.

Would it not make more sense to remove the first layer and just operate all 6 servers on a single loadbalancer?

There are no shared sessions or user data between the servers, If the server a user is on goes down they need to log out and back in.

The load balancing is also only used for login, everything else after that is strictly direct client to server communication.

Answer 1

It adds another layer of redundancy, and that may well be the thinking of your network team (although I recommend asking them!).

If you rely on only one load balancer, and it becomes unavailable (unit failure, or DC network /power outage), then you don't have a contingency. With your current config, if you had to you could (presumably) re-point your DNS records to a specific site, and still have traffic load balanced over 3 servers. I guess something similar could be done with hot-spare load balancers in each site that are not used under normal conditions (but unused network hardware makes me uneasy; will it work nicely when you hit the 'on' button?).

Another aspect is the method used for the load balancing. If you choose to use 1 load balancer to 2 different sites, it becomes much more likely that you will need the load balancer to act as a proxy, since you won't be able to perform other forms of network-trickery, such as MAC switching/Direct Routing. More here: http://www.linuxvirtualserver.org/VS-DRouting.html, which is very fast. And the other drawback with a proxy is that you normally need the return traffic to go back through it, whereas in a DR scenario the server can reply directly to do the client, meaning there is less of a bottleneck; sometimes you can make better use of bandwidth this way and reduce latency (dependent on the rest of the architecture obviously). So it's worth considering the methods that are currently used, at both tiers of your setup.