Take the 2-minute tour ×
Server Fault is a question and answer site for professional system and network administrators. It's 100% free, no registration required.

Heres the situation, I recently setup a few of my sites on a a CDN (specifically MediaTemple's ProCDN via Edgecast). Setup went perfectly normal but a couple of days later I notice this: (15K a day Average to 600K) Chart of DNS Query Usage

Now my domains / cdn is setup like so:

  • I have custom nameservers for digitaldawn.net
  • All supplementary domains point to these nameservers
  • All supplementary domains have cnames setup to cdn.domain.com
  • These cnames point to the Edgecast Origins ie. http://wpc.XXXX.edgecastcdn.net/XXXXXX
  • The edgecast origins themselves grab content from http://www.domain.com:80. ie. http://cdn.domain.com/an-image.jpg will be the content distributed version of http://www.domain.com/an-image.jpg

Firstly, would my setup account for this excessive increase in DNS queries? (The sites that have CDNS setup currently get minimal visitors (Probably less than 10 per day)).

Secondly, providing this is perfectly normal, is there any way to sidestep this? I have 10Million managed queries through DNS Made easy and at the current rate I'm going to be up a certain creek without a paddle...

I think I can sidestep half of it by hosting each domain on a static IP and setting the origins to static IP addresses, however this is far from ideal as I have quite a few sites. Are there any other options?

share|improve this question

1 Answer 1

up vote 3 down vote accepted

The primary cause of too many DNS requests is TTLs that are too low. Yours are low but not insanely low. (I've seen 60 and 1 as TTLs in production systems.)

digitaldawn.net.        1800    IN  A   109.73.163.166

www.digitaldawn.net.    3600    IN  A   208.94.146.71
www.digitaldawn.net.    3600    IN  A   208.94.146.70
www.digitaldawn.net.    3600    IN  A   208.94.146.80
www.digitaldawn.net.    3600    IN  A   208.94.146.81

cdn.digitaldawn.net.        1800    IN  CNAME   wpc.7b5c.edgecastcdn.net.
wpc.7b5c.edgecastcdn.net.   3600    IN  CNAME   gs1.wpc.edgecastcdn.net.
gs1.wpc.edgecastcdn.net.   14400    IN  A       93.184.221.133

Unless you are changing the IP address that these domains point to more often than once per day, you will be better off changing the TTL to something like 86400 (24 hours). You can go higher if you can be confident of having at least the time period in the TTL of advance waning that you might need to change the IP address.

For the cdn.digitladawn.net subdomain, even if you set that TTL to 86400, only that line in the above output will be cached for 24 hours. If the wpc.7b5c.edgecastcdn.net response changes, all clients should have picked up the new value after a maximum of one hour (ignoring for now those DNS servers that ignore your TTLs.)

The other two causes of too many DNS requests that I have seen are too many clients (say, thousands of edge CDN servers that are all hitting your authoritative name servers) or a single misbehaving client (possibly a script on your own server) that is doing lookups dozens of times per second. An example of this could be a reverse proxy that uses backend.digitaldawn.net as its upstream server and makes a DNS request for that domain for every HTTP request it has to proxy. Adding DNS caching to that server or running your own authoritative name server inside your production environment can solve this problem.

If you can get hold of better statistics for your name servers (such as the IP addresses of all the clients that did lookups) then you might be able to diagnose this sort of problem.

share|improve this answer
    
I usually don't tend to say "ask the vendor/provider", but with CDNs it often helps. –  rackandboneman May 17 '12 at 13:45
1  
What a difference! See cl.ly/Gijj - Thanks @Ladadadada –  Paul Millar May 18 '12 at 19:05
    
@PaulMillar I know it's been a while for you but I'm having the exact same issue! TTLs are 86,400 but have very high query counts. Did you find any reverse proxy issue or anything else ? –  Ashray Baruah Aug 26 '13 at 11:09

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.