Sporadically cannot connect to site from eastern U.S. and Europe

Question

We've been having an issue the last few days where users from the eastern U.S. and various European countries cannot connect to my company's sites, but only sporadically. I've been using this tool to test:

http://www.whatsmydns.net/#A/www.vitalmtb.com

http://www.whatsmydns.net/#A/www.vitalbmx.com

http://www.whatsmydns.net/#A/www.vitalmx.com

Please note that you may need to hit refresh a few times to see the red Xs.

The first domain listed above recently had its DNS changed to Amazon from GoDaddy in the hope that having distributed DNS servers (including one in the U.K.) would help. It does seem to have helped a little so far (not seeing red Xs in London at the same frequency), but there are still some locations (Berlin, Moscow, etc.) that are still showing as such. The second and third domain above are still using GoDaddy DNS.

My best guess is that there is a problem server (routing or DNS) on the eastern part of the U.S., since (1) that is the gateway to the U.S. from Europe and (2) I've seen lots of red Xs in Boston and a few in New York, but no other U.S. cities. Unfortunately, any traceroutes I've received from users so far show the complete route to our server with no signs of failure (problem is sporadic, as mentioned above).

Any recommendations on how to go about troubleshooting something of this magnitude?

Edit:

I tested each of the URLs above 10x and got the following:

http://www.whatsmydns.net/#A/www.vitalmtb.com = 20 failures (Berlin, Germany x 4; Merzig Saarland, Germany x 6; London, United Kingdom x 5; Moscow, Russia x 2; Portland OR, U.S. x 3)

http://www.whatsmydns.net/#A/www.vitalbmx.com = 21 failures (Rome, Italy x 7; Moscow, Russia x 2; Berlin, Germany x 5; New York NY, U.S. x 3; Boston MA, U.S. x 4)

http://www.whatsmydns.net/#A/www.vitalmx.com = 18 failures (Rome, Italy x 4, Berlin, Germany x 6; Ankara, Turkey x 1; London, United Kingdom x 1; Rome, Italy x 2; Moscow, Russia x 4)

You may disregard my above statement about how changing DNS from GoDaddy to Amazon helped, because based on these results it did not. Again, the first one is using Amazon's Route53 DNS and the latter two are currently using GoDaddy DNS.

Answer 1

From what I can see there the DNS queries simply time out sometimes. This is not necessarily a problem with your DNS service, it might just be a problem with the querying servers, which might be overloaded, have a query quota defined for recursive lookups or simply experiencing connectivity problems.

It is nearly impossible to troubleshoot without having actual access to the user's network. Flawless traceroutes to your hosts's destination address only show that the path to your web servers seems to be okay. They do not tell anything about occasional failures in name resolution.

Apart from all the connectivity stuff, there is a problem with your vitalmtb.com zone:

> set q=soa
> vitalmtb.com
Server:  ns-29.awsdns-03.com
Address:  205.251.192.29

vitalmtb.com
        primary name server = ns-29.awsdns-03.com
        responsible mail addr = awsdns-hostmaster.amazon.com
        serial  = 1
        refresh = 7200 (2 hours)
        retry   = 900 (15 mins)
        expire  = 1209600 (14 days)
        default TTL = 86400 (1 day)

The "serial=1" in the SOA record is deviating from a common practice to use a datestamp/index combination as the value for this attribute. It also indicates that this is the very first revision of your zone. If you've ever made changes to the zone after the first publication without increasing the serial, caching nameservers are likely not to notice and still hand out stale information. You should change it to something like "2011121101" and update it accordingly on every zone data change.