31

It's a sort of regular occurrence that, after SSHing to a server (or even opening a terminal on my Mac), the login banner prints immediately but it takes ~10 seconds to a minute for the shell prompt to appear. After that, performance is fine and network latency is not unusual.

This doesn't seem like a computationally difficult, memory-intensive, or IO-heavy task. What's it doing with all those billions of CPU cycles?

Improve this question
9
  • 9
    ssh -v -v -v and ur_shell -x might be prudent debugging steps.
    – thrig
    Sep 15, 2015 at 17:26
  • 2
    Is there a lot of lines in your .bash_history?
    – kasperd
    Sep 15, 2015 at 17:37
  • 2
    Look at your .profile and related files (sorry, not sure which shell you're using) and remove temporarily to see if that improves things. You probably have a command(s) that are timing out.
    – TheFiddlerWins
    Sep 15, 2015 at 17:45
  • 3
    Alternatively to Moby Disk's suggestion, is it still equally slow if you start the shell itself from within the shell? If it's slow when (re)connecting but fast when starting a second shell from within an existing, connected session, that will tell you that it's not the shell itself that is causing the slowness; if it is equally slow in the two situations, then something the shell does on startup is taking a lot of time. Either way you learn some about which aspect of "from zero to shell prompt" is slow.
    – user
    Sep 15, 2015 at 20:55
  • 2
    Often this is GSSAPI authentication trying to occur (which, if you're not a Kerberos shop, is likely completely useless). Other times, it's DNS reverse lookups.
    – Charles Duffy
    Sep 15, 2015 at 22:32

8 Answers 8

Reset to default
33

Quite a few things could be happening here. You can find most of the answers in your shell's manual, but those are usually incredibly long and oblique, so...

Chances are your problem boils down to one of a few things.

If your profile or bashrc have expensive things, consider trimming them back.

If your profile or bashrc use a reverse DNS lookup (to set the prompt or something), fix DNS or use hostname instead.

Shells open a lot of files, among other things, while initialising. If system load is high, it will often show up here.

If the banner is pre-authentication, it might also actually be authentication (pam, LDAP, etc.) that is slow.

It might be none of these things, though. A surprising amount of stuff happens right before displaying the prompt!

Improve this answer
4
  • 1
    Another one I've seen block for a substantial while is attempted GSSAPI authentication (if not turned off on either the SSH client or server config). Using a full-system tracing tool such as sysdig is probably the best silver bullet available to get to the bottom of this kind of cross-concern issue.
    – Charles Duffy
    Sep 15, 2015 at 22:33
  • +1 Thorough answer. This site has a nice flowchart for logins, which files they execute/source, and other info.
    – Tim S.
    Sep 15, 2015 at 23:22
  • 15
    +1. 99% of the time for me it's the reverse DNS lookup.
    – mpontillo
    Sep 15, 2015 at 23:38
  • @Mike: same here - I would recommend to start with this one as it is easy to fix. It was 100% in my case.
    – WoJ
    Sep 16, 2015 at 15:19
22

It's probably either waiting for DNS or trying to authenticate via LDAP or such.

Try to add UseDNS no to /etc/ssh/sshd_config

If it also does it on local logons, check if any LDAP servers or DNS servers you have configured are slow or unresponsive.

Improve this answer
6

One possibility (covered by other answers) is that the process of setting up the SSH session itself is where the time is lost.

Another alternative is that your shell startup scripts running on the remote machine after SSH session establishment have something that is taking a long time (perhaps attempting to access some broken network mount). You can debug this second possibility as follows:

Temporarily add the following to the top of your ~/.bash_profile:

set -x
PS4='+ $(date "+%s.%N")\011 '

The set -x turns on some debugging for every shell command executed. The PS4 variable controls how that debugging is presented - specifically in this case we use date to add timestamps.

You may then analyze the timestamps of the debugging output to see which commands in your startup scripts are taking too long.

Improve this answer
2
  • 1
    True if and only if the issue happens after the remote session is opened. Many of the potential causes are during SSH handshake and authentication.
    – Charles Duffy
    Sep 15, 2015 at 22:32
  • 2
    Much improved. :)
    – Charles Duffy
    Sep 15, 2015 at 23:09
4

If it's an Ubuntu server, the default login setup checks if any packages are updatable every time a login shell runs. If the package lists aren't in disk cache, this can take a second or two even on a fast idle desktop.

$ ssh localhost 
Welcome to Ubuntu 15.04 (GNU/Linux 3.19.0-26-generic x86_64)

 * Documentation:  https://help.ubuntu.com/

*** System restart required ***
Last login: Sat Sep 12 01:38:38 2015 from localhost

To generate that "restart required" message, it had to check that the currently running kernel is not the default kernel currently installed. (i.e. there was a kernel and I didn't reboot yet.) It will also print a count of security updates available, if there are any.

I think this is the major slowdown in logging in to Ubuntu that's been introduced recently.

If not that, then your ~/.bash_profile / ~/.bashrc might be the problem.

Have you tried logging in to the server from itself (ssh localhost)? Or logging in a 2nd time right away? (To see if it's a lot faster when stuff is cached.)

Improve this answer
2

This is in most cases a timeout of a DNS request.

Cause: The server attempts a reverse DNS lookup using the IP address of the client, and does not get a reply. If A connects to B, B tries to convert the IP address of A to a name.

Workaround: Enter the IP-Address and name of the client to the hosts file of the server.

Solution: make all hosts known to the DNS server.

Improve this answer
0

Learned alot from this thread. In my case the offender was nvm:

export NVM_DIR="$HOME/.nvm"
[ -s "/usr/local/opt/nvm/nvm.sh" ] && . "/usr/local/opt/nvm/nvm.sh"  # This loads nvm
[ -s "/usr/local/opt/nvm/etc/bash_completion" ] && . "/usr/local/opt/nvm/etc/bash_completion"  # This loads nvm bash_completion
Improve this answer
0

for future visitors, i had miniconda on my machine and its .bashrc entry was the one causing my delayed shell prompt. i no longed needed the conda so removed its entry from .bashrc that looked kike this

# >>> conda initialize >>>
# !! Contents within this block are managed by 'conda init' !!
__conda_setup="$('/home/<YOUR_USER>/miniconda3/bin/conda' 'shell.bash' 'hook' 2> /dev/null)"
if [ $? -eq 0 ]; then
    eval "$__conda_setup"
else
    if [ -f "/home/<YOUR_USER>/miniconda3/etc/profile.d/conda.sh" ]; then
        . "/home/<YOUR_USER>/miniconda3/etc/profile.d/conda.sh"
    else
        export PATH="/home/<YOUR_USER>/miniconda3/bin:$PATH"
    fi
fi
unset __conda_setup
# <<< conda initialize <<<
Improve this answer
0

a common issue i have is, that the server is Reachable by IPv4 and IPv6 and due the fact that IPv6 will be prefered even if its unsupported to the internet of the LAN, it tries to connect.

On Putty you can solve this, that you use instead AUTO, prefer ipv4 as default

this behaviour seems a windows issue mostly, but found also on some linux boxes also

Improve this answer

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .