Take the 2-minute tour ×
Server Fault is a question and answer site for professional system and network administrators. It's 100% free, no registration required.

I'm trying to fetch a page programatically and it takes exactly 10 seconds to resolve the host, every time. On another machine it takes exactly 30 seconds. Both Linux.

My code is in Java but the problem is reproducable using wget:

time wget -d  --header "User-Agent:Mozilla/5.0 (X11; Linux i686) AppleWebKit/534.24 (KHTML, like Gecko) Ubuntu/10.10 Chromium/11.0.696.65 Chrome/11.0.696.65 Safari/534.24" http://www.sportsdirect.com

This hangs for 10 secs on:

Resolving www.sportsdirect.com... 86.17.5.250

We're running on Linux.

To confuse things further browsers on the same machine fetch the same page immediately.

Any clues?

share|improve this question
    
No idea what the problem is but my lookup timed out but worked the second time –  Chopper3 May 11 '11 at 17:00

2 Answers 2

up vote 5 down vote accepted

This is because wget and lynx aren't caching the DNS results, and your browsers may even have IPv6 disabled.

The DNS for www.sportsdirect.com is broken, and times out on AAAA records.

You can use the -4 or --inet4-only option to wget to force IPv4 only.

$ time host -t A www.sportsdirect.com
www.sportsdirect.com is an alias for sd.gwcorp.sports-world.com.
sd.gwcorp.sports-world.com has address 86.17.5.250

real    0m0.009s
user    0m0.000s
sys 0m0.000s

$ time host -t AAAA www.sportsdirect.com
;; connection timed out; no servers could be reached

real    0m14.008s
user    0m0.000s
sys 0m0.000s
share|improve this answer

The browsers may be caching the DNS results. It might be worth trying the query directly to the configured DNS servers to see if they are the problem -- perhaps one of them is particularly slow or down. Take the IP address from the nameserver line in /etc/resolv.conf and run

dig +trace www.sportsdirect.com 8.8.8.8

(replace 8.8.8.8 with your DNS server's IP address)

share|improve this answer
    
dig returns pretty much instantly. I even have dnsmasq running, and if I don't specify the nameserver argument the response comes back from 127.0.0.1 as expected, using dig. Weird. It's only when I use an http client such as wget or lynx that it slows right down... –  Joel May 11 '11 at 17:04

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.