4

I have a server that generally works fine, but gets stuck for 20 seconds when trying to connect with SSL (either SSH or HTTPS display the same pattern.)

I tried various connections without SSL, such as a telnet:

telnet server-name 80

Entered a GET command

GET http://server-name/
Host: server-name
Accept: text/html, */*
Accept-Language: en-us
User-Agent: Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.1)

And the reply is 100% instantaneous.

However, when I try to connect to the server via HTTPS in my browser, or use SSH to connect to that same server, it sits for about 20 seconds before connecting.

For SSH, it will then work nicely (i.e. not slow at all anymore). For HTTPS, it will be slow each time it has to connect again. A connection that does not get closed will continue to work fast, however.

I'm adding a screenshot of the Net info appearing in Firebug. As we can see, each time a new connection is attempted, 20s. Looking at the server usage with htop, the CPU is at 0% and when something happens it goes to a whooping 0.01% for the 1 min. usage report. So the server overall is not being pegged in any way (i.e. it is a test server at this point and we do not get hits by others yet.)

enter image description here

So my question is: What could cause such a slowdown?

I thought maybe it could be that OpenSSL is attempting to use /dev/random, but I never heard of such a problem before. The random device does not output much at all on that VPS. However, /dev/urandom works very well. I can get 1Mb of random data in a few seconds. What could I look into to fix this problem?


/etc/resolv.conf looks like Google DNS. Should be fast...

# Dynamic resolv.conf(5) file for glibc resolver(3) generated by resolvconf(8)
#     DO NOT EDIT THIS FILE BY HAND -- YOUR CHANGES WILL BE OVERWRITTEN
nameserver 8.8.8.8
nameserver 8.8.4.4

Note that the Apache2 setup in itself should not matter directly since it also happens with SSH...


Also I tried with ssh -vvv. The connection is instantaneous.

OpenSSH_6.6.1, OpenSSL 1.0.1f 6 Jan 2014
debug1: Reading configuration data /home/alexis/.ssh/config
debug1: /home/alexis/.ssh/config line 202: Applying options for do-nia2match
debug1: Reading configuration data /etc/ssh/ssh_config
debug1: /etc/ssh/ssh_config line 19: Applying options for *
debug1: Hostname has changed; re-reading configuration
debug1: Reading configuration data /home/alexis/.ssh/config
debug1: Reading configuration data /etc/ssh/ssh_config
debug1: /etc/ssh/ssh_config line 19: Applying options for *
debug2: ssh_connect: needpriv 0
debug1: Connecting to 178.62.213.172 [178.62.213.172] port 22.
debug1: Connection established.
...
debug1: SSH2_MSG_NEWKEYS received
debug1: SSH2_MSG_SERVICE_REQUEST sent
debug2: service_accept: ssh-userauth
debug1: SSH2_MSG_SERVICE_ACCEPT received
debug2: key: /home/alexis/.ssh/do-nia2match_rsa, explicit

SUPER LONG PAUSE HAPPENS HERE (~20s)

debug1: Authentications that can continue: publickey,password
debug3: start over, passed a different list publickey,password
debug3: preferred publickey,keyboard-interactive
debug3: authmethod_lookup publickey
...

I'm not too sure why when sending the key the SSH server would hang for 20s...

8
  • 1
    Chrome's timings are more detailed, and show SSL negotiation is completing quickly. Your problem occurs after that is complete. Please post a copy of the server's /etc/resolv.conf and check your Apache configuration for the HostnameLookups setting. – Michael Hampton Feb 14 '16 at 0:45
  • It's not the TLS handshake, it's the time to first byte, according to this test webpagetest.org/result/… . Have a look at the details graph. Look at your resources, look at what's different about the slow ones. Is it hitting PHP which is using too much CPU? RAM constrained? – Tim Feb 14 '16 at 0:56
  • 2
    Can your server perform DNS lookups against Google DNS? Are there log entries from Apache or ssh regarding the problem? – Michael Hampton Feb 14 '16 at 1:12
  • Oh! It looks like the DNS is blocked by the firewall. It worked before, but someone else may have changed something there... Let me make sure port 53 is open properly and try again. – Alexis Wilke Feb 14 '16 at 1:18
  • Wow! That was it! I can understand that the DNS is required for Apache2, but I have to say that the fact that SSH was also bugged down was super surprising to me and did not ring the bell! Ah! I see that the SSH server does a reverse lookup by default. Now I understand why it would react like Apache2. – Alexis Wilke Feb 14 '16 at 1:20
1

I found out (way back—see comments) that I was blocking all 127.0.0.0 except 127.0.0.1 addresses.

Debian and thus Ubuntu defines an entry in your /etc/hosts at 127.0.1.1 with your domain name. You should see something like this at the beginning of your /etc/hosts file:

127.0.1.1     hostname.example.com hostname

I had to review my firewall and actually I decided to open all of 127.x.x.x since all are part of the private network and these are all safe IPs.

2
  • 1
    This might not be digital ocean specific if you're running Debian or Ubuntu. Debian uses that loopback address as a workaround for some gnome related issues. Bug for reference: bugs.debian.org/cgi-bin/bugreport.cgi?bug=719621 – Spooler Oct 3 '18 at 22:57
  • Well... I noticed with a server at DigitalOcean. But I see that indeed it's a Debian thing. I fixed my answer, though and use Debian & Ubuntu instead. – Alexis Wilke Oct 3 '18 at 23:10

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.