How to debug “HTTP request sent, awaiting response”?

Question

I have a Linux server on an ADSL connection, and I just noticed that sometimes I get

Connecting to example.com|xxx.xxx.xxx.122|:80... connected.
HTTP request sent, awaiting response...

when I use wget or e.g. a browser like firefox.

In the case of wget pressing CTRL-C and then try again "solves the problem", and in the case of firefox, reloading the page also makes the page load.

It seams to be random when it happens. I have tried to change the DNS to Google's DNS, but I still get the same problems there.

Question

How do I debug something like that, so I can find the source of the problem or perhaps a pattern?

Answer 1

Have you ever tried waiting to see what HTTP status code returns?

To debug this case, open 2 terminals: in the first, ssh to your server and type:

# tcpdump -vv -s0 tcp port 80 -w /tmp/example.pcap

(you can also append and src host <client_IP> to filter junk requests)

and in the second, use wget to browse your website, replicate this issue and switch to the first terminal and press Ctrl+C.

Copy this .pcap file to the client and open with Wireshark to see what happens.

Answer 2

If you see:

HTTP request sent, awaiting response...

This means that the DNS works fine, that the TCP connection is in established state and the request was send. I think that the cause of your problems is the remote server that might be busy.

As @quanta said a network capture will give you better details. In Wireshark just use Follow TCP Stream to see the HTTP conversation. Pay attention to ICMP messages too.