Server Fault is a question and answer site for system and network administrators. Join them; it only takes a minute:

Sign up
Here's how it works:
  1. Anybody can ask a question
  2. Anybody can answer
  3. The best answers are voted up and rise to the top

Mirrored servers in data centers nationwide -- how do millions of users simultaneously get instant results from their "" requests? I flunked my IT interview by getting this question wrong.

I thought that in the various metropolitan areas, an "" request goes to the ISP's DNS server, which somehow returns an IP address for one of several geographically-nearby http servers, and then something internally rolls over to the next available local Google server. But then, I could not explain where the table of available local Google servers is actually cached, or the details of the IP address rollover. Or how they could manually take some server out of the rotation, from anywhere.

So, what should I be reading now so I can ace this question next time?

Also, what daemons run on these machines 24/7 to keep all those mirrored database disks synchronized?

share|improve this question
I should stop looking at DNS, and start reading about multi-homed routing / BGP ! – Sysadmin Evstar Jan 8 '11 at 22:33

I am mixed here. What google does is standard anycast.

Basically you have your own internet addresses, in your own AS (a term a sysadmin should know) then assign all clusters the same IP ranges (all data centers). DNS will round robin, but you do not use this to direct to a specific addresss.... you use bgp anycast announcements for that. What you need to is access to the internet gbp level, which requires you own AS. The trick is that if a DNS says "address x" then your DNS anycast setup routes all requests to the closest data center.

That said, it is not standard knowledge for a nommal sysadmin to know about that. As you can see on the wrong answers further up who seem to not be even aware of Anycast, which is pretty much a standard approach. BGP / internet backbone routing protocols is not something 99% of the network admins have any need to know. In that, the question was unfair or for a senior position.

Also, what daemons run on these machines 24/7 to keep all those mirrored database disks synchronized?

None. Discs are not synced. To my knowlege google machines run in clusters of X machines each (forming one functional unit) and culusters dont get updated ever. They get regularly reinstalled with a new and updated image. Between installs, no sync happens.

share|improve this answer
Google is also using GeoIP DNS. This is easily visible by resolving '' from different parts of the country. From home and work, both in Washington State, it's the same list, but from a server in Utah it's a shorter and very different list. From the Utah server traceroute shows there is a 23ms difference between the WA IPs and the UT IPs. – sysadmin1138 Jan 9 '11 at 1:17

Was this interview for a sysadmin position? If so, I think the question is slightly unfair. Granted I don't know the full breadth or context of the question, and you should definitely know how components such as DNS and load balancing work in general, but I don't think you could be expected to know how Google (for example) manages to make it's services globally diverse and highly available.

Mirrored servers in data centers nationwide? That could mean anything.

What needs to be mirrored? A web site? A Windows domain infrastructure? An FTP service? Etc., etc.

share|improve this answer
How would you move Mount Everest? Nothing is fair, it seems. – Sysadmin Evstar Jan 8 '11 at 22:53
Actually you should know. THis is done by servers sharing the same IP and googles control over it's own peerings / bgp annoucements. Any Administratior who should know BGP (senio position or speciualized) should know this trick as it is standard knowledge. – TomTom Jan 8 '11 at 22:59
My point is, that without knowing the context and breadth of the question, none of us can give an accurate answer. Was the question related to BGP, DNS, mirroring a Windows domain, etc., etc. – joeqwerty Jan 8 '11 at 23:09

You may want to look into load-balancing and CDNs.

There is no one specific daemon that I'm aware of. There are however a number of ways of doing this.

Edit: And to answer your comment, DNS is not the ticket here. Its all about CDNs(content distribution networks) and load-balancers, redirecting to a number of available, mirrored hosts. These hosts are mirrored through any number of technologies, including(not likely in Google's case) rsync and things of that nature.

share|improve this answer
Sorr, no mirroring involved. What he looks for is BGP anycast. Yes, you mirror the content, but that is not making the routing. – TomTom Jan 8 '11 at 22:59

Your Answer


By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.