Take the 2-minute tour ×
Server Fault is a question and answer site for professional system and network administrators. It's 100% free, no registration required.

I am trying to understand application layer protocols in TCP/IP stack. I know that both HTTP and DNS protocol stay at the top layer (Application Layer). So, when a browser wants to access a resource, it has to send a request to the HTTP server, as for example:

GET www.pippo.it/hello.htm HTTP/1.1

Making this request following the rules of HTTP protocol, it uses the page URL, not the IP address.

I know that DNS request is necessary to convert URL to IP. So my question is: does HTTP invoke the DNS protocol? It seems impossible to me, since both are top layer protocols (so DNS can't provide a service to HTTP). In the same way even TCP (which stays on a lower level) can't ask for a service at a higher level protocol like DNS.

So when does the DNS request happen? And who performs such a request?

share|improve this question
1  
Could you accept one of the answers to clarify which of these answers the question? –  utrecht Nov 16 at 15:57

3 Answers 3

HTTP request in question is actually not valid unless browser is talking to intermediary (proxy).

Your example would look bit more like following if browser was talking with web server directly:

GET /hello.htm HTTP/1.1
Host: www.pippo.it

Now to put it in perspective consider OSI layer model

enter image description here

So we have 3 systems in action.

  • Client running the browser
  • Web server serving the site
  • DNS server knowing IP address of the site

Protocols involved are; bottom up (minimum relevant set to OP)

  • IP
  • TCP, UDP
  • HTTP, DNS

HTTP communication is done over TCP protocol, and TCP communication is on top of IP protocol

DNS in this case is done over UDP protocol, and UDP protocol is on top of IP protocol.

communication sequence in short:

  1. CLIENT running browser asks DNS Server using UDP protocol for A record of www.pippo.it

    1.1. on client it is the operating system that does resolving part and talks back to browser, browser never talks to DNS directly, rather through OS by invoking gethostbyname() . On Windows order in which OS does resolving of addresses is likely defined by something like this, and on Linux resolving precedence is defined by /etc/nsswitch.conf

  2. DNS Serves using UDP protocol responds to CLIENT with record/IP address if it exists

  3. CLIENT opens TCP connection to port 80 of WEB server and writes following text

HTTP request:

GET /hello.htm HTTP/1.1
Host: www.pippo.it

You could mimic the same thing by doing something like this in your console/cmd prompt

> telnet www.pippo.it 80
Trying 195.128.235.49...
Connected to www.pippo.it.
Escape character is '^]'.
GET /hello.htm HTTP/1.1
Host: www.pippo.it

Followed by two empty lines. To this web server will respond, if requested content exist it will print it to screen, and in case of browser, that hypertext is actually parsed by browser and all tags, links, scripts and images are rendered in what we call a web page.

In reality there are some more details, e.g. browsers cache IP addresses if you already visited some domain, so DNS resolving may not be necessary. Also modern browsers may try to do the resolving before you actually need it (DNS prefetching) to speed up your browsing experience.

Additionally, your computer may have static record in say hosts file and no DNS server would ever be contacted, rather local static entry would be used first. This is configurable, and not necessarily true, but it is default for major operating systems I'm familiar with.

share|improve this answer
    
How does this work with regards to URLs that share an IP address? –  trlkly Nov 12 at 10:55
    
same thing, if URL has same IP address, Browser likely already has IP address cached from previous requests so the part where Browser asks for IP from OS, and OS from DNS is skipped and Browser directly starts talking with IP address making request like described. –  Hrvoje Špoljar Nov 12 at 11:01
3  
@trikly: that's what the 'Host' header is for. Without it you can only have one website per IP address. This is a key difference between HTTP/1.0 and HTTP/1.1. Thankfully HTTP/1.0 browsers are rare now - but if you want to cater for them then you need a different IP address for each site (they can still be hosted on the same server). –  A E Nov 12 at 11:06
1  
@AE Thanks. I think I was unclear in my question, and that's why Hrvoje didn't understand what I was saying. (I should have said domain rather than URL). I'm glad you still understood. –  trlkly Nov 12 at 11:08
1  
You say "This is not correct HTTP request", and that's mostly true, but it's closer than you imply: "To allow for transition to absoluteURIs in all requests in future versions of HTTP, all HTTP/1.1 servers MUST accept the absoluteURI form in requests". (RFC 2616 §5.1.2) So GET http://www.pippo.it/hello.htm HTTP/1.1 would be a valid, if unusual, request. It would also be a valid and usual request to an HTTP proxy. –  wfaulk Nov 12 at 22:44

HTTP is transported over TCP, which is a IP protocol. To make an HTTP request, the browser has to open a TCP connection, and do to that, it needs the destination IP address (i.e. the IP address of the server). To resolve the server's hostname, it has thus to issue a DNS request (generally the DNS request itself is sent by the operating system when a program calls its name resolution functions; however, nothing prevents a program from sending DNS requests by itself to the DNS server). Once the connection is established, it can send its HTTP request, which contains the path to the requested resource, and a Host field with the hostname of the server (e.g., Host: www.pippo.it). The hostname does not go on the request line (it would actually be GET /hello.htm HTTP/1.1), except when the request is sent to an HTTP proxy (and in this case, the full URL is present, including the protocol part, e.g. GET http://www.pippo.it/hello.htm HTTP/1.1),

share|improve this answer
    
Thanks, now it is clearer, but not completely. You write that browser has to issue a DNS request. Ok, but after having received the IP from DNS server, how does it use it? I mean, such IP doesn't appear in the HTTP request. So I assume there is still one more step before issuing the HTTP request and I think it is the opening of the connection. This point is not very clear to me... Thanks once more! –  Giancarlo Perlo Nov 11 at 13:59
4  
Indeed, the IP is needed to open the TCP connection, inside which the HTTP request is transported. Actually, the IP address of both client and server is sent along with ALL packets of the connection. The best way to learn how this works is probably to install a packet capture tool (Wireshark is an excellent multi-platform and open source one), capture a simple HTTP request, filter it out from the rest of network activity, and look at how all the packets were sent on the wire. You should actually be able to see the DNS request before the TCP connection, too. –  Ale Nov 11 at 14:47
1  
Proxied requests should be using the Host header, not putting the full URL in the GET line. –  OrangeDog Nov 12 at 11:32
1  
@OrangeDog: No, on the contrary. RFC 7230 (section 5.3.2) explicitly says that a client making a request to a proxy MUST use an absolute-URI in the request line. (There must still be a Host header duplicating information from the request line; section 5.4). –  Henning Makholm Nov 12 at 16:31

The procedure goes like this:

  1. The user (you) gives the browser a URL, like http://www.pippo.it/hello.htm
  2. The browser splits that into three parts:

    • Protocol http
    • Hostname www.pippo.it
    • URL path /hello.htm

    (a more complicated URL could have other parts too, I'll ignore that possibility for now)

  3. The browser knows that in order to create an IP connection, it needs an IP address. To obtain an IP address, it needs to use DNS (unless it has the address cached).

    1. The browser asks the operating system for the IP address of a DNS server; suppose it gets 8.8.8.8.
    2. The browser constructs the following multi-layered connection:

      • IP layer: connect to 8.8.8.8
      • UDP layer: set packet for destination port 53
      • DNS layer: create a DNS request for an A record for the hostname www.pippo.it

      Of course I'm omitting a lot of detail about e.g. the exact format of the packets involved.

    3. The browser receives a DNS response (layered on top of UDP layered on top of IP etc.) which gives the IP address for www.pippo.it, let's say it's 10.11.12.13
  4. The browser knows that in order to create a TCP connection, it needs a port number. To obtain a port number, it looks up the protocol http in its internal table and learns that it should use port 80.
  5. The browser constructs the following multi-layered connection:

    • IP layer: connect to 10.11.12.13
    • TCP layer: set packets to destination port 80
    • HTTP layer: create an HTTP request for the URL /hello.htm on the host www.pippo.it (because the computer at 10.11.12.13 might be hosting several domains, so it needs to know which one is desired)

      GET /hello.htm HTTP/1.1
      Host: www.pippo.it
      ...
      

    Of course I'm omitting all the details of the TCP handshake and such.

  6. The browser receives an HTTP response (layered on top of TCP layered on top of IP etc.) containing the contents of hello.htm

And for good measure, I'll mention that the browser now examines the content of that response, and identifies any additional resources needed: images, CSS, Javascript, etc. Then it repeats this whole process for each such resource.

share|improve this answer
4  
Step 3 really is not something that the application itself does. The application just uses something like getaddrinfo or gethostbyname to ask the OS to resolve the address for it. Also, the OS typically uses multiple mechanisms to try to look up names, not only DNS. (Typically at least the hosts file in addition to DNS.) –  Håkan Lindqvist Nov 11 at 17:28
    
Thanks! It is a really impressive and detailed answer and a very useful too! –  Giancarlo Perlo Nov 11 at 17:45

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.