I'm new to load balancing and I'm wondering if it's possible to use multiple load balancers to redirect traffic to my application servers. I don't really understand how this can be done. Shouldn't a domain name match one to one with a certain server's IP address (in this case the IP of one load balancer)? If each load balancing server has a different IP, how can the request be received by both load balancers (or by 10 load balancers or 50 or 100)?
Using round robin DNS is not that great for high availability - if one server goes offline, clients will still try to connect to it and wait for a timeout.
There are other ways to achieve this.
1) Active/Passive load balancers
Basically one load balancer handles all traffic for one IP address.
If that balancer goes down, the passive node jumps in and takes over the IP.
Keep in mind that load balancers are pretty much only forwarding traffic, so for small to medium sized sites this can work out OK.
2) Active/Active load balancers
The same traffic IP is configured on both (or many more) load balancers.
Incoming traffic gets sent to all load balancers but a algorithm chooses which balancer should respond, all others discard that traffic.
Simple way to think of it, you have two load balancers:
When the requesting IP ends with an even number then load balancer A answers, otherwise load balancer B answers.
Of course your infrastructure must support this and there is overhead due to traffic getting sent but discarded.
More information, e.g. here: http://community.brocade.com/t5/SteelApp-Docs/Feature-Brief-Deep-dive-on-Multi-Hosted-IP-addresses-in-Stingray/ta-p/73867
High Availability with load balancers is commonly implemented using a virtual ip address (VIP) protocol which allows several hosts (i.e. load balancers) to answer to one common ip address in one of several possible ways (variations on active/passive, active/active).
There are a good number of these protocols, the ones I have seen most with regular load balancers are VRRP and NLB (as well as a good many nondescript blackboxed protocols in appliances). Expanding to routers and firewalls one may also encounter CARP, HRSP, GLSP for instance.
This strategy has a number of benefits over DNS load balancing which is a simpler strategy (and which is taken care of in another answer).
DNS load balancing is burdened for instance with:
- the slow turnover of dns caching mechanisms
- limited load balancing algorithms (typically just round-robin)
- the outsourcing of the load balancing decision to the client (through caching of the dns record)
- Slow drain of service queues when a server (i.e. a load balancer) is taken out of rotation (based on dns record TTLs as handled by ISPs and clients)
- Slow failover on load balancer failure
Using a virtual ip protocol for HA one may have a choice to achieve for instance:
- Choice of load balancing algorithm amongst the load balancers
- Server centric load balancing decisions (fascilitating for example service health based measures and routing)
- Quicker drain of service queues when a load balancer is taken out of rotation.
- Instant failover on load balancer failure
Only you know which strategy and protocol fits your scenario best.
The requirements: have a practical solution that works for cloud or any type of environment where no access to hardware load balancers, BGP protocols and all that stuff.
An application's income request number is unknown but should be high enough to meet an increased load expectation with no fear.
Let's find an application with similar nature of load, for instance logging store and search app. I found one.
What they want:
- Balance the load across collectors
- Offer fault tolerance, allowing us to continue ingesting data if one of the collectors dies or is experiencing issues
- Scale horizontally with the growth in our log volumes
What did they try and learnt about ELB:
- Doesn't work as expected
- Latency issues due to increased load
- Not enough monitoring facility
- Too many limitation (open ports and protocols number)
Why did they choose with Route53:
- "Round robin is pretty basic load balancing, but it works well for us from an efficiency standpoint"
- "We take advantage of Route 53 failover health checks."
- "If there is an issue with a collector, Route 53 automatically takes it out of the service; our customers won’t see any impact. "
- No Pre-Warmup Required with Route 53
Route 53 turned out to be the best way for Loggly to take advantage of our high-performance collectors given our huge log volumes, unpredictable variations, and constant growth in our business. It aligns with the collectors’ core purposes: To collect data at network line speed with zero loss, and it allows us to benefit from the elasticity of all of the AWS services we use at Loggly.
That particular example shows that in some scenarios (logs collector, adverts service or similar) load balancer is redundant and "DNS health-check round robin solution" does its job very well.
Let's see what AWS say re DNS failover:
With DNS Failover, Route 53 can detect an outage of your website and redirect your end users to alternate or backup locations that you specify. Route 53 DNS Failover relies on health checks-regularly making Internet requests to your applications endpoints from multiple locations around the world-to determine whether each endpoint of your application is up or down.
That technique also makes ELB (not required, just for a note) more robust, again it is based with RR + Health Check:
Route 53 DNS Failover handles all of these failure scenarios by integrating with ELB behind the scenes. Once enabled, Route 53 automatically configures and manages health checks for individual ELB nodes.
Let's now see how it works behind the scene. The obvious question is how to deal with DNS caching:
However, DNS caching can still be a problem here (see our previous post where "long tail" problem is covered) if TTL is not respected by all layers between your client and Route 53. You could then apply a "cache busting" technique: send a request to a unique domain
and define a wildcard Resource
Record "*.<your-domain>" to match it.
Algolia introduced "client retry strategy" which works pretty well if your client (JS in your case) can handle that:
We ended up implementing a basic retry strategy in our API clients. Each API client was developed to be able to access three different machines. Three different DNS records represented each user: USERIDID-1.algolia.io, USERID-2.algolia.io andUSERID-3.algolia.io. Our first implementation was to randomly select one of the records and then retry with a different one in case of failure.