first of all, I apologize for my bad english, I'm not a native speaker.
I'm currently having troubles using SSH remote port forwarding. I've got customer's systems (running Debian 8) that keep active remote SSH tunnel on one of my servers. Every connection are quite good (and the connection speed is also good enough) for all this customer's system except for one but the configuration is the same for all.
Here is the command I use to mount the tunnel (same for all customers) :
autossh -f -i myKeyFile -q -N -R [::]:10201:localhost:22 remoteUser@remoteServer -p remotePort
The point is that for only one specific customer, the connection via the remote port forwarding is very slow before login time (about 3-5 minutes), and after, all is back to normal speed. When I try to connect directly to the customer server (without SSH remote port forwarding), the connection is ok.
I try to check the firewall on the system, but even if I remove all rules (iptables -P and -F on INPUT/OUTPUT chain, and iptables -t nat -F too), the connection still remain slow.
Here are the log messages with ssh -v :
$ ssh localUser@localhost -p 10201 -4C -v OpenSSH_6.7p1 Debian-5+deb8u2, OpenSSL 1.0.1t 3 May 2016 debug1: Reading configuration data /etc/ssh/ssh_config debug1: /etc/ssh/ssh_config line 19: Applying options for * debug1: Connecting to localhost [127.0.0.1] port 10201. debug1: Connection established. debug1: permanently_set_uid: 0/0 debug1: key_load_public: No such file or directory debug1: identity file /home/remoteUser/.ssh/id_rsa type -1 debug1: key_load_public: No such file or directory debug1: identity file /home/remoteUser/.ssh/id_rsa-cert type -1 debug1: key_load_public: No such file or directory debug1: identity file /home/remoteUser/.ssh/id_dsa type -1 debug1: key_load_public: No such file or directory debug1: identity file /home/remoteUser/.ssh/id_dsa-cert type -1 debug1: key_load_public: No such file or directory debug1: identity file /home/remoteUser/.ssh/id_ecdsa type -1 debug1: key_load_public: No such file or directory debug1: identity file /home/remoteUser/.ssh/id_ecdsa-cert type -1 debug1: key_load_public: No such file or directory debug1: identity file /home/remoteUser/.ssh/id_ed25519 type -1 debug1: key_load_public: No such file or directory debug1: identity file /home/remoteUser/.ssh/id_ed25519-cert type -1 debug1: Enabling compatibility mode for protocol 2.0 debug1: Local version string SSH-2.0-OpenSSH_6.7p1 Debian-5+deb8u2 # HERE IS THE 3-5 MIN DELAY debug1: Remote protocol version 2.0, remote software version OpenSSH_6.7p1 Debian-5+deb8u7 debug1: match: OpenSSH_6.7p1 Debian-5+deb8u7 pat OpenSSH* compat 0x04000000 debug1: SSH2_MSG_KEXINIT sent debug1: SSH2_MSG_KEXINIT received debug1: kex: server->client aes128-ctr email@example.com firstname.lastname@example.org debug1: kex: client->server aes128-ctr email@example.com firstname.lastname@example.org debug1: sending SSH2_MSG_KEX_ECDH_INIT debug1: expecting SSH2_MSG_KEX_ECDH_REPLY debug1: Server host key: ECDSA 85:e6:aa:0c:3a:15:87:ec:98:c3:9b:c5:73:06:f4:fd debug1: Host '[localhost]:10201' is known and matches the ECDSA host key. debug1: Found key in /home/remoteUser/.ssh/known_hosts:40 debug1: SSH2_MSG_NEWKEYS sent debug1: expecting SSH2_MSG_NEWKEYS debug1: SSH2_MSG_NEWKEYS received debug1: SSH2_MSG_SERVICE_REQUEST sent debug1: SSH2_MSG_SERVICE_ACCEPT received debug1: Authentications that can continue: publickey,password debug1: Next authentication method: publickey debug1: Trying private key: /home/remoteUser/.ssh/id_rsa debug1: Trying private key: /home/remoteUser/.ssh/id_dsa debug1: Trying private key: /home/remoteUser/.ssh/id_ecdsa debug1: Trying private key: /home/remoteUser/.ssh/id_ed25519 debug1: Next authentication method: password localUser@localhost's password:
As commented in the log, the delay occurs right after this message :
debug1: Local version string SSH-2.0-OpenSSH_6.7p1 Debian-5+deb8u2
As I didn't find anything particulary wrong, I also check hard drive health, CPU load, RAM usage etc without success. Please notice that all my customers have got the same hardware.
Here is my /etc/ssh/sshd_config file :
# Package generated configuration file # See the sshd_config(5) manpage for details # What ports, IPs and protocols we listen for Port 22 # Use these options to restrict which interfaces/protocols sshd will bind to #ListenAddress :: #ListenAddress 0.0.0.0 Protocol 2 # HostKeys for protocol version 2 HostKey /etc/ssh/ssh_host_rsa_key HostKey /etc/ssh/ssh_host_dsa_key HostKey /etc/ssh/ssh_host_ecdsa_key HostKey /etc/ssh/ssh_host_ed25519_key #Privilege Separation is turned on for security UsePrivilegeSeparation yes # Lifetime and size of ephemeral version 1 server key KeyRegenerationInterval 3600 ServerKeyBits 1024 # Logging SyslogFacility AUTH LogLevel INFO # Authentication: LoginGraceTime 120 PermitRootLogin without-password StrictModes yes RSAAuthentication yes PubkeyAuthentication yes #AuthorizedKeysFile %h/.ssh/authorized_keys # Don't read the user's ~/.rhosts and ~/.shosts files IgnoreRhosts yes # For this to work you will also need host keys in /etc/ssh_known_hosts RhostsRSAAuthentication no # similar for protocol version 2 HostbasedAuthentication no # Uncomment if you don't trust ~/.ssh/known_hosts for RhostsRSAAuthentication #IgnoreUserKnownHosts yes # To enable empty passwords, change to yes (NOT RECOMMENDED) PermitEmptyPasswords no # Change to yes to enable challenge-response passwords (beware issues with # some PAM modules and threads) ChallengeResponseAuthentication no # Change to no to disable tunnelled clear text passwords #PasswordAuthentication yes # Kerberos options #KerberosAuthentication no #KerberosGetAFSToken no #KerberosOrLocalPasswd yes #KerberosTicketCleanup yes # GSSAPI options #GSSAPIAuthentication no #GSSAPICleanupCredentials yes X11Forwarding yes X11DisplayOffset 10 PrintMotd no PrintLastLog yes TCPKeepAlive yes #UseLogin no #MaxStartups 10:30:60 #Banner /etc/issue.net # Allow client to pass locale environment variables AcceptEnv LANG LC_* Subsystem sftp /usr/lib/openssh/sftp-server # Set this to 'yes' to enable PAM authentication, account processing, # and session processing. If this is enabled, PAM authentication will # be allowed through the ChallengeResponseAuthentication and # PasswordAuthentication. Depending on your PAM configuration, # PAM authentication via ChallengeResponseAuthentication may bypass # the setting of "PermitRootLogin without-password". # If you just want the PAM account and session checks to run without # PAM authentication, then enable this but set PasswordAuthentication # and ChallengeResponseAuthentication to 'no'. UsePAM yes AllowTCPForwarding yes
I gave a try to "UseDNS no" in sshd_config without success. I also tried to change remotePort / localPort and this didn't change anything.
The point is, I can connect to the host in both way, but I was wondering why is the connection that slow for this specific customer, only when using SSH remote port forwarding ?
Thank you for your help. If you need any more information, I can provide it to you.