Take the 2-minute tour ×
Server Fault is a question and answer site for professional system and network administrators. It's 100% free, no registration required.

I'm facing a little question here. I have a long /etc/hosts file. (working in the web, so I'm hosting a lot of sites locally for developping). Let's says it is about 40, 50 lines long. All redirecting to 127.0.0.1.

Now when checking Chrome's inspector, it seems my local web pages are loading slowly because of the DNS Lookup part. On a mygreatwebsite.local url also defined in the /etc/hosts file.

After a quick check, it would seem that reducing the hosts file to a couple of lines, drastically speeds up the DNS lookup part. So my question, is it possible to be confronted with differences of about 5 seconds for 50 lines, and insignificant times for a couple of lines? Or am I completely missing something?

share|improve this question

6 Answers 6

up vote 13 down vote accepted

Your tld .local could be the problem, zeroconf uses this. There is no real standard, but the suggested tld for an internal tld is .site or .internal. See http://en.wikipedia.org/wiki/Top-level_domain#Pseudo-domains for this.

share|improve this answer
    
Indeed, I have noticed great performance boost when using .site extension. After a little research, it seems to be the apple's Bonjour protocol that uses .local tld. –  Zenklys Oct 19 '11 at 11:07
    
I joined StackExchange just to say THANK YOU for this excellent tip. My app was taking 10-15 seconds just to load assets and making my dev experience a nightmare. Using .site TLD has the pages loading all snappy again. Awesome. Sidenote: Someone needs to mention this tidbit to all the bloggers/screencasters that use .local! –  Steven Garcia Oct 26 '12 at 11:42

Can a long /etc/hosts file slow DNS lookup?

The answer is yes, in a Windows (XP/Vista/7) machine.

It is a known problem that a long hosts file does not work well with "DNS Client" service.

http://winhelp2002.mvps.org/hosts.htm:

in most cases a large HOSTS file (over 135 kb) tends to slow down the machine.

[Solution: Set "DNS Client" service to manual or disabled]

http://vlaurie.com/computers2/Articles/hosts.htm:

Note that a hosts file that is much over 100 KB can actually slow up browsing unless the service "DNS Client" is set to manual start.

www.ericphelps.com/scripting/samples/Hosts/:

The most important thing to do before using large HOSTS files is to disable the Domain Name Service client ("DNS Client" or "DnsCache") service on your machine. It sounds bizarre, but the DNS client isn't needed to do DNS. Worse, if you have a large HOSTS file (which is what I suggest), the DNS client chokes your entire PC.

... and I have personally witnessed this on three different Windows machines: XP Pro, Vista, and 7 Starter.

share|improve this answer
    
Interesting point of view. I didn't knew that. But this questions concerns Mac Os environments. ;) –  Zenklys Dec 1 '11 at 8:07

I would suggest using tcpdump to see whats actually going on. Without it you will be only guessing.

Basic syntax would be

tcpdump -n -i eth0 port 80 or port 53

(if you are interested in seeing ports 80 and 53, -n to prevent DNS lookups, supposing all comunication is taking place on eth0).

If this is too fast or too much information, redirect it to dumpfile with adding:

 -s 0 -w /tmp/dump

and use i.e. wireshark for looking into the dumpfile. You can see precise timing information in the file and much more...

share|improve this answer

For various reasons I have hosts files on some machines with hundreds of entries. This has no impact on DNS lookups simply because once the machine has found an entry in the hosts file it will not do a lookup for the same information. If everything else is configured correctly educing the size of the hosts file won't affect performance because even with a thousand entries that file can still be parsed faster than performing a DNS lookup. Check the order in which your machine is resolving addresses. The hosts file should be first.

share|improve this answer
    
Thanks, for the answer. How do I check the order? –  Zenklys Oct 19 '11 at 10:39
    
@Zenklys, I'll have to leave it up to someone far more expert in Linux to answer that for you. –  John Gardeniers Oct 19 '11 at 20:27

When i used Spybot it would insert around 50,000 hosts file entries that redirect known malicious sites to 127.0.0.1. I never noticed any issues from having a long hosts file of this nature. Likely has more to do with your dev setup or the code itself. How many sites are you serving locally (50?) Does everyone one of them take this long to respond when 50 entries are present?

share|improve this answer
    
Yes, I'm sure it has nothing to do with the sites itself, they ran perfectly on my other mac (Snow Leopard). Now on Lion, only the server configuration and hosts file have changed. And yes, they all take the same time to load. And always on DNS Lookup part. –  Zenklys Oct 19 '11 at 12:29

Based on my testing, a 34 MB hosts file with a few million entries slows down browsing in Windows a bit even when the "DNS Client" service is disabled. So, there is a maximum size beyond which you probably don't want to go.

share|improve this answer

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.