Take the 2-minute tour ×
Server Fault is a question and answer site for professional system and network administrators. It's 100% free, no registration required.

This is a Canonical Question about DNS (Domain Name Service).

If my understanding of the DNS system is correct, the .com registry holds a table that maps domains (www.example.com) to DNS servers.

  1. What is the advantage? Why not map directly to an IP address?

  2. If the only record that needs to change when I am configuring a DNS server to point to a different IP address, is located at the DNS server, why isn't the process instant?

  3. If the only reason for the delay are DNS caches, is it possible to bypass them, so I can see what is happening in real time?

share|improve this question

migrated from stackoverflow.com Feb 1 '12 at 17:27

This question came from our site for professional and enthusiast programmers.

17  
To all the people trying to migrate/close this question: Please leave it here. It has a home here, where it can be loved and tenderly cared for. And we can point all "How does DNS Work" questions here as a canonical answer, because this one is extremely well answered. –  Mark Henderson Feb 1 '12 at 22:40
    
You can not able full understand DNS unless you are name Paul Mockapetris, Paul Vixie or Cricket Liu. twitter.com/DEVOPS_BORAT/status/249006925767909376 –  Anthony Hatzopoulos Oct 16 '12 at 21:14
add comment

4 Answers

up vote 77 down vote accepted

Actually, it's more complicated than that - rather than one "central registry (that) holds a table that maps domains (www.mysite.com) to DNS servers", there are several layers of hierarchy

There's a central registry (the Root Servers) which contain only a small set of entries: the NS (nameserver) records for all the top-level domains - .com, .net, .org, .uk, .us, .au, and so on.

Those servers just contain NS records for the next level down. To pick one example, the nameservers for the .uk domain just has entries for .co.uk, .ac.uk, and the other second-level zones in use in the UK.

Those servers just contain NS records for the next level down - to continue the example, they tell you where to find the NS records for google.co.uk. It's on those servers that you'll finally find a mapping between a hostname like www.google.co.uk and an IP address.

As an extra wrinkle, each layer will also serve up 'glue' records. Each NS record maps a domain to a hostname - for instance, the NS records for .uk list nsa.nic.uk as one of the servers. To get to the next level, we need to find out the NS records for nic.uk are, and they turn out to include nsa.nic.uk as well. So now we need to know the IP of nsa.nic.uk, but to find that out we need to make a query to nsa.nic.uk, but we can't make that query until we know the IP for nsa.nic.uk...

To resolve this quandary, the servers for .uk add the A record for nsa.nic.uk into the ADDITIONAL SECTION of the response (response below trimmed for brevity):

jamezpolley@li101-70:~$dig nic.uk ns

; <<>> DiG 9.7.0-P1 <<>> nic.uk ns
;; global options: +cmd
;; Got answer:
;; ->>HEADER<<- opcode: QUERY, status: NOERROR, id: 21768
;; flags: qr rd ra; QUERY: 1, ANSWER: 11, AUTHORITY: 0, ADDITIONAL: 14

;; QUESTION SECTION:
;nic.uk.                IN  NS

;; ANSWER SECTION:
nic.uk.         172800  IN  NS  nsb.nic.uk.
nic.uk.         172800  IN  NS  nsa.nic.uk.

;; ADDITIONAL SECTION:
nsa.nic.uk.     172800  IN  A   156.154.100.3
nsb.nic.uk.     172800  IN  A   156.154.101.3

Without these extra glue records, we'd never be able to find the nameservers for nic.uk. and so we'd never be able to look up any domains hosted there.

To get back to your questions...

a) What is the advantage? Why not map directly to an IP address?

For one thing, it allows edits to each individual zone to be distributed. If you want to update the entry for www.mydomain.co.uk, you just need to edit the information on your mydomain.co.uk's nameserver. There's no need to notify the central .co.uk servers, or the .uk servers, or the root nameservers. If there was only a single central registry that mapped all the levels all the way down the hierarchy that had to be notified about every single change of a DNS entry all the way down the chain, it would be absolutely swamped with traffic.

Before 1982, this was actually how name resolution happened. One central registry was notified about all updates, and they distributed a file called hosts.txt which contained the hostname and IP address of every machine on the internet. A new version of this file was published every few weeks, and every machine on the internet would have to download a new copy. Well before 1982, this was starting to become problematic, and so DNS was invented to provide a more distributed system.

For another thing, this would be a Single Point of Failure - if the single central registry went down, the entire internet would be offline. Having a distributed system means that failures only affect small sections of the internet, not the whole thing.

(To provide extra redundancy, there are actually 13 separate clusters of servers that serve the root zone. Any changes to the top-level domain records have to be pushed to all 13; imagine having to coordinate updating all 13 of them for every single change to any hostname anywhere in the world...)

b) If the only record that needs to change when I am configuring a DNS server to point to a different IP address is located at the DNS server, why isn't the process instant?

Because DNS utilises a lot of caching to both speed things up and decrease the load on the NSes. Without caching, every single time you visited google.co.uk your computer would have to go out to the network to look up the servers for .uk, then .co.uk, then .google.co.uk, then www.google.co.uk. Those answers don't actually change much, so looking them up every time is a waste of time and network traffic. Instead, when the NS returns records to your computer, it will include a TTL value, that tells your computer to cache the results for a number of seconds.

For example, the NS records for .uk have a TTL of 172800 seconds - 2 days. Google are even more conservative - the NS records for google.co.uk have a TTL of 4 days. Services which rely on being able to update quickly can choose a much lower TTL - for instance, telegraph.co.uk has a TTL of just 600 seconds on their NS records.

If you want updates to your zone to be near-instant, you can choose to lower your TTL as far down as you like. The lower your set it, the more traffic your servers will see, as clients refresh their records more often. Every time a client has to contact your servers to do a query, this will cause some lag as it's slower than looking up the answer on their local cache, so you'll also want to consider the tradeoff between fast updates and a fast service.

c) If the only reason for the delay are DNS caches, is it possible to bypass them, so I can see what is happening in real time?

Yes, this is easy if you're testing manually with dig or similar tools - just tell it which server to contact.

Here's an example of a cached response:

jamezpolley@host:~$dig telegraph.co.uk NS

; <<>> DiG 9.7.0-P1 <<>> telegraph.co.uk NS
;; global options: +cmd
;; Got answer:
;; ->>HEADER<<- opcode: QUERY, status: NOERROR, id: 36675
;; flags: qr rd ra; QUERY: 1, ANSWER: 8, AUTHORITY: 0, ADDITIONAL: 0

;; QUESTION SECTION:
;telegraph.co.uk.       IN  NS

;; ANSWER SECTION:
telegraph.co.uk.    319 IN  NS  ns1-63.akam.net.
telegraph.co.uk.    319 IN  NS  eur3.akam.net.
telegraph.co.uk.    319 IN  NS  use2.akam.net.
telegraph.co.uk.    319 IN  NS  usw2.akam.net.
telegraph.co.uk.    319 IN  NS  use4.akam.net.
telegraph.co.uk.    319 IN  NS  use1.akam.net.
telegraph.co.uk.    319 IN  NS  usc4.akam.net.
telegraph.co.uk.    319 IN  NS  ns1-224.akam.net.

;; Query time: 0 msec
;; SERVER: 97.107.133.4#53(97.107.133.4)
;; WHEN: Thu Feb  2 05:46:02 2012
;; MSG SIZE  rcvd: 198

The flags section here doesn't contain the aa flag, so we can see that this result came from a cache rather than directly from an authoritative source. In fact, we can see that it came from 97.107.133.4, which happens to be one of Linode's local DNS resolvers. The fact that the answer was served out of a cache very close to me means that it took 0msec for me to get an answer; but as we'll see in a moment, the price I pay for that speed is that the answer is almost 5 minutes out of date.

To bypass Linode's resolver and go straight to the source, just pick one of those NSes and tell dig to contact it directly:

jamezpolley@li101-70:~$dig @ns1-224.akam.net telegraph.co.uk NS

; <<>> DiG 9.7.0-P1 <<>> @ns1-224.akam.net telegraph.co.uk NS
; (1 server found)
;; global options: +cmd
;; Got answer:
;; ->>HEADER<<- opcode: QUERY, status: NOERROR, id: 23013
;; flags: qr aa rd; QUERY: 1, ANSWER: 8, AUTHORITY: 0, ADDITIONAL: 0
;; WARNING: recursion requested but not available

;; QUESTION SECTION:
;telegraph.co.uk.       IN  NS

;; ANSWER SECTION:
telegraph.co.uk.    600 IN  NS  use2.akam.net.
telegraph.co.uk.    600 IN  NS  eur3.akam.net.
telegraph.co.uk.    600 IN  NS  use1.akam.net.
telegraph.co.uk.    600 IN  NS  ns1-63.akam.net.
telegraph.co.uk.    600 IN  NS  usc4.akam.net.
telegraph.co.uk.    600 IN  NS  ns1-224.akam.net.
telegraph.co.uk.    600 IN  NS  usw2.akam.net.
telegraph.co.uk.    600 IN  NS  use4.akam.net.

;; Query time: 9 msec
;; SERVER: 193.108.91.224#53(193.108.91.224)
;; WHEN: Thu Feb  2 05:48:47 2012
;; MSG SIZE  rcvd: 198

You can see that this time, the results were served directly from the source - note the aa flag, which indicates that the results came from an authoritative source. In my earlier example, the results came from my local cache, so they lack the aa flag. I can see that the authoritative source for this domain sets a TTL of 600 seconds. The results I got earlier from a local cache had a TTL of just 319 seconds, which tells me that they'd been sitting in the cache for (600-319) seconds - almost 5 minutes - before I saw them.

Although the TTL here is only 600 seconds, some ISPs will attempt to reduce their traffic even further by forcing their DNS resolvers to cache the results for longer - in some cases, for 24 hours or more. It's traditional (in a we-don't-know-if-this-is-really-neccessary-but-let's-be-safe kind of way) to assume that any DNS change you make won't be visible everywhere on the internet for 24-48 hours.

share|improve this answer
2  
+1 That is one awesome explanation. Will definitely bookmark this! –  Trollhorn Feb 2 '12 at 8:30
    
Thank you @Trollhorn :) –  James Polley Feb 2 '12 at 8:51
3  
The real answer to whether a DNS response came from a cache or not is in the "flags" section of the answer. There's no technical reason why one couldn't set a TTL of, say, 319 seconds. Instead, look for the aa (authoritative answer) flag in the response. If aa is present, the answer came directly from an authoritative name server. (If it's missing, the answer may still be fresh; some recursive name servers clear the aa flag before passing on the response to the client resolver.) –  Michael Kjörling Feb 2 '12 at 10:19
3  
You should note that some ISPs will cache DNS records for much longer than the TTL says that they should, so even with very short TTLs you can't guarantee that all at sites visitors will get the correct IP for a day or two after you relocate a site. –  Dan Neely Feb 2 '12 at 13:35
2  
@JamesPolley there's a (minor) error in your explanation with your use of the .uk servers. Currently the Nominet-managed second level domains are on the same servers as uk., so a query for example.co.uk to the uk servers will receive a delegation immediately without having to first go through a delegation to the co.uk servers. –  Alnitak Jan 7 '13 at 22:05
show 4 more comments

a) The number of possible domain names is way too large for one server to handle. And there's not just .com; there's .net, .org, .se, .info, and any number of others. Add to that, you can delegate responsibility for a subdomain (which is effectively what com does). DNS being less centralized makes all that easier to manage.

b) Machines along the way from the user to you have DNS caches in order to minimize the number of requests needed. It prevents, for example, spamming the network with requests for the address of "serverfault.com" every time you get a page from SF. Those servers can even cache "domain doesn't exist" results, which is why it can take a while for even a brand new domain to show up.

c) Although you can disable your cache, there are often other DNS servers between your machine and yourdomain.com's DNS server. Your ISP's DNS server, for example, will try to cache as much as it can. The only records that get updated relatively quickly around the net are ones with a short TTL (which basically says "i'm only valid for a few seconds; after that, ask me again for current info"). The reason TTLs are so high, though, is so that the server responsible for the domain can offload some of the work to other servers. If you had all the net contacting your one or two rinky-dink DNS servers for every hit to your web site, they'd be pretty much unusable the second someone saw your site on /., digg, etc.

share|improve this answer
    
Your (c) is wrong. It's a widespread misunderstanding of DNS. There's no chain of servers — local machine to router to ISP to … — each contacting the next in line. Requests go from a DNS client (library), through a single resolving proxy DNS server, to zero or more content DNS servers. Barring the existence of forwarding proxy DNS servers, that's it. –  JdeBP Feb 1 '12 at 18:03
2  
@JdeBP: It's a "widespread misunderstanding" because as far as i've seen in the real world, it's largely true. If you get your address via DHCP, as just about everyone does, then you've almost certainly been given the address of a DNS server that you're supposed to use. Which, in home networks, is almost always the router -- which is basically forwarding to your ISP's DNS. And in small business networks, is usually the domain controller -- which, again, is usually forwarding to the ISP's DNS. It's generally at that point that the iterative stuff takes over. –  cHao Feb 1 '12 at 18:42
    
You need to see more of the real world if you think "largely true" fits at all. I actually mentioned forwarding proxies as the one extra item. However, you're overestimating their use in business networks; and indeed overestimating how many change their domain controllers from the default, which is (after one has removed any root zone) a resolving DNS server using root hints. Your basic error in (c) remains uncorrected. Disabling a cache doesn't magically reveal a cache "behind" it. The DNS simply doesn't work that way. If you aren't misunderstanding it, then you are misrepresenting it. –  JdeBP Feb 2 '12 at 0:44
    
The fact is, if you disable the cache on your local machine, you still see the results that are cached by your local network's DNS server. And if you disable that, you're still likely to have your ISP's cache to worry about. Whether you accept that as a common case or not, it's the case i've seen just about every time -- which makes it common enough to be well worth mentioning. –  cHao Feb 2 '12 at 1:41
    
@cHao most CPE DNS servers only "forward" and don't cache. –  Alnitak Jan 7 '13 at 22:07
add comment

a) The number of IP --> host name mappings in the world is REALLY big. This system distributes the responsibility of hosting all the subdomain and MX records and every other DNS record out to the owner of the domain name. This was pretty much the point of the domain name. .com is held by one registry where as .uk can be held by another. Likewise example.com and otherexample.com can be hosted separately so the resources can be distributed.

b) It's cached, which reduces the number of hits on your DNS host down to a small fraction of what it would be otherwise. By default records live 2 days in the cache before being discarded. This can be changed by altering the TTL (time to live) of the record.

c) You can effectively stop records being cached by setting a really short TTL. This is NOT recommend unless you're using it for dynamic DNS. Caching drops the hits on the DNS server by a lot. To pick a guessed number out of the air we're talking about knocking off 95% of the requests.

share|improve this answer
    
Regarding 'C', be careful. Set that TTL low enough and your DNS server might get hammered. Not a huge issue if someone other than you is handling DNS. –  Publiccert Feb 1 '12 at 17:36
    
So if it weren't for subdomains, there wouldn't be much of an advantage –  sabof Feb 1 '12 at 17:47
4  
Perhapse, but remeber even yourdomain.com is a subdomain of .com. If it was just one big "hosts file" (as it was before DNS and elves still walked in middle earth) then yes. You'd just have one big file and everyone would cache that. –  couling Feb 1 '12 at 17:50
3  
The DNS namespace was flat, with no delegations, for a long time; and it was implemented as a single list, held in one place. That became unworkable and got replaced by DNS in 1982... –  James Polley Feb 1 '12 at 18:53
1  
@mfinni, Well, it's "domain name system", not "distributed name system" or anything like that. Of course it's designed to be distributable, but there's nothing saying that you absolutely have to run it that way. For some small office networks with no global connectivity, having everything in a root zone (or a single TLD such as local) probably makes some limited amount of sense. –  Michael Kjörling Feb 2 '12 at 10:27
show 2 more comments

If you're on a *nix system, download a copy of Dan Bernstein's djbdns from http://cr.yp.to/djbdns.html and run his dnstrace program to see how the recursive query system works. Its very informative.

share|improve this answer
    
Will it allow me to see the effect of a configuration change instantly? –  sabof Feb 1 '12 at 17:44
    
dnstrace (usually through dnstracesort) gives intense amounts of detail about the DNS configuration for any domain and query you make. If the server has made a change, they will show the change and how it has propagated. They're excellent for tracing propagation errors as well. –  mikebabcock Feb 1 '12 at 17:48
    
Thanks, I'll give it a try –  sabof Feb 1 '12 at 17:49
add comment

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.