9

I'm new to load balancing and I'm wondering if it's possible to use multiple load balancers to redirect traffic to my application servers. I don't really understand how this can be done. Shouldn't a domain name match one to one with a certain server's IP address (in this case the IP of one load balancer)? If each load balancing server has a different IP, how can the request be received by both load balancers (or by 10 load balancers or 50 or 100)?

  • Thank you for your response. So basically, if I want to use multiple load balancers to handle my traffic I only have to setup a different CNAME for each one of them? Specifically, if I need 10 load balancers to handle the traffic to my site, that's the only way to do it? – user3790827 Jul 12 '15 at 20:43
  • 1
    I recommend leaving questions open for at least a day before closing them. Even that is usually being hasty. Just because you've received an answer doesn't mean it's necessarily the only (or best) one, and marking your Q&A answered usually means it gets less attention. – Andrew B Jul 12 '15 at 22:16
  • 1
    @Anatoly I didn't make a decision yet. I've reviewed the solutions presented here and also talked with some of my friends who recommended me other solutions. I think that for my use case the best solution so far would be to use VPS servers from a cheap provider like DO or Vultr that don't offer Virtual IP and use the method used by Algolia with client load balancing. I need HA and scalability for the API only therefore there wouldn't be such a big deal if I create different subdomains for each load balancer. These end user of the widget will never notice them anyway. – user3790827 Jul 15 '15 at 15:53
  • @user3790827 sounds like a plan. Despite on the type of requirements having HA and Failover there are too many patterns around, everyone hits same problem but not anyone has SLA 99.9 (8 hours downtime a year) or higher. HA solutions are usually expensive and business trade off between availability and the cost. Clients usually accept 99.9 and aware of potential downtime or scheduled timeframe, even 100% uptime doesn't guarantee you zero bugs with development/deployment/security or human mistakes. – Anatoly Jul 15 '15 at 19:49
  • I investigated that Google Chrome forces DNS invalidation and query to happen in case of 3sec timeout. Not sure other browsers behaviour though. – Anatoly Aug 1 '15 at 12:24
12

Using round robin DNS is not that great for high availability - if one server goes offline, clients will still try to connect to it and wait for a timeout.

There are other ways to achieve this.
1) Active/Passive load balancers
Basically one load balancer handles all traffic for one IP address.
If that balancer goes down, the passive node jumps in and takes over the IP.
Keep in mind that load balancers are pretty much only forwarding traffic, so for small to medium sized sites this can work out OK.

2) Active/Active load balancers
The same traffic IP is configured on both (or many more) load balancers.
Incoming traffic gets sent to all load balancers but a algorithm chooses which balancer should respond, all others discard that traffic.
Simple way to think of it, you have two load balancers:
When the requesting IP ends with an even number then load balancer A answers, otherwise load balancer B answers.

Of course your infrastructure must support this and there is overhead due to traffic getting sent but discarded.
More information, e.g. here: http://community.brocade.com/t5/SteelApp-Docs/Feature-Brief-Deep-dive-on-Multi-Hosted-IP-addresses-in-Stingray/ta-p/73867

  • When you say "of course your infrastructure must support this" you mean I need an additional machine or VM that will send requests to the load balancers? – user3790827 Jul 13 '15 at 2:23
  • 2
    @user3790827 Infrastructure in this context is the network equipment, not servers.' – Jenny D Jul 13 '15 at 5:46
  • 1
    I'm planning use a cloud provider therefore I don't have direct control over the physical infrastructure. What should I ask my vps service provider for? – user3790827 Jul 13 '15 at 12:04
  • 1
    There is only abstract recommendations because it depends on a ton of details. We don't even know if it makes sense to have a multi hosted IP here - maybe his traffic is only a few hundred Mbit/s. If you need this, I would evaluate the proper software, check the requirements and figure out which provider supports it. Would DNS RR work? Sure. Would I use it? Depends on what kind of availability the owner of the business I'm working for is aiming for! – faker Jul 13 '15 at 13:59
  • @faker I'm sorry, I guess it's my fault because I didn't gave enough details. I want to build a javascript script that will be inserted into other people's websites and will gather traffic data( think Google Analytics), also it will access the server to display statistics for each page it's loaded on. Basically there would be a javascript file that will be loaded for each website it's used on. – user3790827 Jul 13 '15 at 14:31
6

High Availability with load balancers is commonly implemented using a virtual ip address (VIP) protocol which allows several hosts (i.e. load balancers) to answer to one common ip address in one of several possible ways (variations on active/passive, active/active).

There are a good number of these protocols, the ones I have seen most with regular load balancers are VRRP and NLB (as well as a good many nondescript blackboxed protocols in appliances). Expanding to routers and firewalls one may also encounter CARP, HRSP, GLSP for instance.

This strategy has a number of benefits over DNS load balancing which is a simpler strategy (and which is taken care of in another answer).

DNS load balancing is burdened for instance with:

  • the slow turnover of dns caching mechanisms
  • limited load balancing algorithms (typically just round-robin)
  • the outsourcing of the load balancing decision to the client (through caching of the dns record)
  • Slow drain of service queues when a server (i.e. a load balancer) is taken out of rotation (based on dns record TTLs as handled by ISPs and clients)
  • Slow failover on load balancer failure

Using a virtual ip protocol for HA one may have a choice to achieve for instance:

  • Choice of load balancing algorithm amongst the load balancers
  • Server centric load balancing decisions (fascilitating for example service health based measures and routing)
  • Quicker drain of service queues when a load balancer is taken out of rotation.
  • Instant failover on load balancer failure

Only you know which strategy and protocol fits your scenario best.

  • 1
    I'd also add that some load balancers support establishing BGP sessions with nearby routers, which allows you to set up Anycast solutions. If the load balancer goes down or otherwise stops advertising for the VIP (failed health check), the next best routing candidate wins. The last sentence of this answer is imperative though: you really need to talk to your company's network admins. – Andrew B Jul 12 '15 at 22:29
  • Here is a nice description of what you describe in the first paragraph cisco.com/c/en/us/support/docs/application-networking-services/… – Martin Podval Nov 11 '15 at 10:23
2

The requirements: have a practical solution that works for cloud or any type of environment where no access to hardware load balancers, BGP protocols and all that stuff.

An application's income request number is unknown but should be high enough to meet an increased load expectation with no fear.

Let's find an application with similar nature of load, for instance logging store and search app. I found one.

What they want:

  1. Balance the load across collectors
  2. Offer fault tolerance, allowing us to continue ingesting data if one of the collectors dies or is experiencing issues
  3. Scale horizontally with the growth in our log volumes

What did they try and learnt about ELB:

  1. Doesn't work as expected
  2. Latency issues due to increased load
  3. Not enough monitoring facility
  4. Too many limitation (open ports and protocols number)

Why did they choose with Route53:

  1. "Round robin is pretty basic load balancing, but it works well for us from an efficiency standpoint"
  2. "We take advantage of Route 53 failover health checks."
  3. "If there is an issue with a collector, Route 53 automatically takes it out of the service; our customers won’t see any impact. "
  4. No Pre-Warmup Required with Route 53

Route 53 turned out to be the best way for Loggly to take advantage of our high-performance collectors given our huge log volumes, unpredictable variations, and constant growth in our business. It aligns with the collectors’ core purposes: To collect data at network line speed with zero loss, and it allows us to benefit from the elasticity of all of the AWS services we use at Loggly.

That particular example shows that in some scenarios (logs collector, adverts service or similar) load balancer is redundant and "DNS health-check round robin solution" does its job very well.


Let's see what AWS say re DNS failover:

With DNS Failover, Route 53 can detect an outage of your website and redirect your end users to alternate or backup locations that you specify. Route 53 DNS Failover relies on health checks-regularly making Internet requests to your applications endpoints from multiple locations around the world-to determine whether each endpoint of your application is up or down.

That technique also makes ELB (not required, just for a note) more robust, again it is based with RR + Health Check:

Route 53 DNS Failover handles all of these failure scenarios by integrating with ELB behind the scenes. Once enabled, Route 53 automatically configures and manages health checks for individual ELB nodes.


Let's now see how it works behind the scene. The obvious question is how to deal with DNS caching:

However, DNS caching can still be a problem here (see our previous post where "long tail" problem is covered) if TTL is not respected by all layers between your client and Route 53. You could then apply a "cache busting" technique: send a request to a unique domain

("http://<unique-id>.<your-domain>") 

and define a wildcard Resource

Record "*.<your-domain>" to match it.

Algolia introduced "client retry strategy" which works pretty well if your client (JS in your case) can handle that:

We ended up implementing a basic retry strategy in our API clients. Each API client was developed to be able to access three different machines. Three different DNS records represented each user: USERIDID-1.algolia.io, USERID-2.algolia.io andUSERID-3.algolia.io. Our first implementation was to randomly select one of the records and then retry with a different one in case of failure.

  • 1
    I think Algolia's approach is the best for my budget and use cases. Normally I would be agains using different subdomain for each load balancer, but since only the JS widget uses them the end user will never notice the difference. – user3790827 Jul 15 '15 at 15:56
  • 1
    Someone also suggested to use Cloudflare's DNS cloudflare.com/features-optimizer to redirect traffic to the standby load balancer once a failure occurs with currently used load balancer. cloudflare.com/dns – user3790827 Jul 15 '15 at 16:01

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.