Take the 2-minute tour ×
Server Fault is a question and answer site for professional system and network administrators. It's 100% free, no registration required.

It has recently come to my attention that setting up multiple A records for a hostname can be used not only for round-robin load-balancing but also for automatic failover.

So I tried testing it:

  1. I loaded a page from our domain
  2. Noted which of our servers had served the page
  3. Turned off the web server on that host
  4. Reloaded the page

And indeed the browser automatically tried a different server to load the page. This worked in Opera, Safari, IE, and Firefox. Only Chrome failed to try a different server.

But after leaving that server offline for a few minutes and looking at the access logs, I found that the number of requests to the other servers had not significantly increased. With 1 out of 3 servers offline, I had expected accesses to each of the remaining 2 servers to roughly increase by 50%, but instead I only saw 7-10%. That can only mean in-browser DNS failover does not work for the majority of browsers/visitors, which directly contradicts what I had just tested.

Does anyone have an idea what is up with browsers' DNS failover behavior? What possible reason could there be why automatic failover works for me but not the majority of our visitors?

edit: To make myself clear, I made absolutely no change to our DNS settings; there's no TTL or propagation issue here, it's all about how the client handles the multiple A records.

share|improve this question
3  
Round robin DNS isn't load balancing or failover. It's, at best, a way to cause increased headaches when you have server availability problems. –  Evan Anderson Mar 16 '11 at 1:51
    
2  
It's also worth pointing out that the browser doesn't choose DNS entries, regardless of how you're seeing it. –  John Gardeniers Mar 16 '11 at 6:55
    
Don't do it! you're asking for trouble. –  Matt Apr 12 '13 at 4:30
add comment

2 Answers 2

OK I am going to start by saying DNS is not a good failover system in any way, you need a reverse proxy or load balancer. There are several reasons why the experience is not the same. First of all in chrome it uses The OS to grab DNS info so that is dependent on the OS for the IPs, so the OS in this case might only give it one IP.

As far as the other browsers its highly dependent on how they do DNS to how it'll work. So the browser itself might decide to not try the other IPs or even try the same one several times depending on the response the DNS server has.

This brings us to the DNS server itself, most do not respect your TTL records and keep then how ever long it feels, meaning Users could get your old IP for quite a while...

Fourthly user experience, do you want users to have to refresh 3 or 4 times to get your website? Do you have any session or login based stuff on your site, what happens if the browser gets another IP in the middle of the session. If you really need HA and uptime you really need to consider doing it right,honestly or it will end up more fractured than using just one server.

share|improve this answer
    
This answer is not related to my question. I've clarified the question; this is not about removing from DNS the IP address of a malfunctioning server. –  Daniel Mar 16 '11 at 2:17
    
Yeah, I think Its relevant, I am explaining why DNS fail over doesn't work, TTLs are a part of that. –  Jacob Mar 16 '11 at 2:43
    
TTL is irrelevant if the DNS records don't change, which is the case here. –  Daniel Mar 16 '11 at 2:46
2  
I don't see how you expect DNS to know not to hand out that IP with out changing it? Your DNS server has no clue your server is out and will happily keep handing out that IP... –  Jacob Mar 16 '11 at 2:48
    
Yes, exactly. The DNS server keeps handing out the same 3 IP addresses (A,B,C), including the bad one (A), and the browser fails over (to B/C) when it cannot connect (to A). At least that's what my browsers do, apart from Chrome. Please re-read my question, slowly. –  Daniel Mar 16 '11 at 2:54
show 7 more comments

Browsers are usually pretty aggressive about trying the alternate records when one is not responding.

A couple of things:

  1. Your issue with Chrome may be related to how it caches DNS - it does its own caching, and is pretty aggressive about it; could it have potentially still had the entry cached from before you had the multiple A records in place?
  2. Similarly, did you wait for at least the TTL of the DNS zone after the extra records were added to test the users coming in from the outside?
  3. Also, make sure that the load was fairly even between the servers in the first place; if one server only had 10% of the traffic, then you'd only expect a modest increase on the other node when it dies.

All that aside, DNS round robin is great for geographic redundancy and load balancing, but keep in mind that there are other good solutions out there for local failover.

share|improve this answer
    
1. AFAIK all browsers have DNS caches. I guess Chrome's issue is that it caches only one IP address even when a DNS query returns more than one. –  Daniel Mar 16 '11 at 2:23
    
2. I just edited my question to clarify that I made no change at all to DNS. So there's no TTL/propagation issue here; the records for the 3 servers were already there for a long time. –  Daniel Mar 16 '11 at 2:23
    
3. Yes, the load was fairly evenly split between the 3 servers to start with. –  Daniel Mar 16 '11 at 2:24
    
Chrome will fetch an IP and delete it and expect the OS to cache it,BTW.. –  Jacob Mar 16 '11 at 2:59
    
@Daniel Non-Chrome browsers cache for a short time period, and the OS does most of the heavy lifting. Chrome's a different animal, it does aggressive prefetching of DNS for pages you visit most when you first start the browser (enter about:dns in the address bar). However, all this is irrelevant to your question; Chrome's DNS cache has your multiple A records present.. Maybe grab a packet capture of Chrome's behavior? I'd be really surprised if it wasn't sending any requests at all to the other address. –  Shane Madden Mar 16 '11 at 5:05
add comment

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.