For web app troubleshooting, you need to check each step individually. This means using dig
to check the DNS request, curl
and tcpdump
(on both client and server) to check the TCP connection, strace
to check what the app does when it gets the request. You should stop whenever you get the first unexpected behaviour
You also need to understand (and add to your question) the difference between (for example) a timeout at the DNS request stage compared to a response with an incorrect answer such as NXDOMAIN or the wrong IP address.
If your nameserver returns the wrong answer you will probably get a timeout when trying to connect to the IP address but if the IP address is active on the network and its firewall isn't dropping packets then you will get a RST packet back instantly. The difference between these is important.
If you are getting timeouts at the TCP connect stage, it can be due to the client's outbound firewall rules, the server's inbound rules, the server's outbound rules or the client's inbound rules. Running tcpdump
on both client and server and looking at the firewall logs will help you distinguish between these cases.
If you manage to connect and send a request but your web app is doing something that takes a long time and hence never sends a response, you will get yet another timeout. You might even get a RST packet at this stage.
The difference between a timeout at the DNS stage, a timeout at the TCP connect stage and a timeout after the HTTP request has been sent is an important first step. It narrows down where the problem could be, it may even tell you exactly what the problem is.