When you use something like putty to connect to a linux box, and you setup your SSH keys etc.
When connecting, how does it tell the server that you want to connect using your SSH keys?
Is SSH running as a service on a particular port or does it simply pass your private-key and then the login service sees that and tries to connect using it?
Just looking for a fairly high level understanding (with maybe some detail if you want to...)
The server-side is configured for the authentication types it allows. And then the client reacts accordingly. From my linux box, I ran an ssh session to one of my hosts with debug output to show what it does:
OpenSSH_5.3p1, OpenSSL 1.0.0-fips-beta4 10 Nov 2009
debug1: Reading configuration data /etc/ssh/ssh_config
debug1: Applying options for *
debug1: Connecting to blahblahblah [ip_address] port 22.
debug1: Connection established.
debug1: identity file /home/username/.ssh/identity type -1
debug1: identity file /home/username/.ssh/id_rsa type 1
debug1: identity file /home/username/.ssh/id_dsa type -1
debug1: Remote protocol version 2.0, remote software version OpenSSH_4.7
debug1: match: OpenSSH_4.7 pat OpenSSH_4*
debug1: Enabling compatibility mode for protocol 2.0
debug1: Local version string SSH-2.0-OpenSSH_5.3
debug1: SSH2_MSG_KEXINIT sent
debug1: SSH2_MSG_KEXINIT received
debug1: kex: server->client aes128-ctr hmac-md5 none
debug1: kex: client->server aes128-ctr hmac-md5 none
debug1: SSH2_MSG_KEX_DH_GEX_REQUEST(1024<1024<8192) sent
debug1: expecting SSH2_MSG_KEX_DH_GEX_GROUP
debug1: SSH2_MSG_KEX_DH_GEX_INIT sent
debug1: expecting SSH2_MSG_KEX_DH_GEX_REPLY
debug1: Host 'hostname' is known and matches the RSA host key.
debug1: Found key in /home/username/.ssh/known_hosts:13
debug1: ssh_rsa_verify: signature correct
debug1: SSH2_MSG_NEWKEYS sent
debug1: expecting SSH2_MSG_NEWKEYS
debug1: SSH2_MSG_NEWKEYS received
debug1: SSH2_MSG_SERVICE_REQUEST sent
debug1: SSH2_MSG_SERVICE_ACCEPT received
debug1: Authentications that can continue: publickey,gssapi-with-mic,password
debug1: Next authentication method: gssapi-with-mic
debug1: Unspecified GSS failure. Minor code may provide more information
Credentials cache file '/tmp/krb5cc_500' not found
debug1: Unspecified GSS failure. Minor code may provide more information
Credentials cache file '/tmp/krb5cc_500' not found
debug1: Unspecified GSS failure. Minor code may provide more information
debug1: Next authentication method: publickey
debug1: Offering public key: /home/username/.ssh/id_rsa
debug1: Server accepts key: pkalg ssh-rsa blen 277
debug1: Authentication succeeded (publickey).
debug1: channel 0: new [client-session]
debug1: Entering interactive session.
debug1: Sending environment.
debug1: Sending env XMODIFIERS = @im=none
debug1: Sending env LANG = en_US.UTF-8
So you can see it negotiated a protocol, and then started offering authentication types. When it hit publickey authentication, it found a key, sent a key and then the key was accepted by the server, which then authentication succeeded, and didn't proceed with any more auth mechanisms. If it had failed to handle the key, I would have gotten the password prompt, because that would have been the only method left to try.
The great thing about internet protocols is that most of them have detailed engineering documents you can go to. In this case, RFC 4251, which references a number of other SSH related RFCs. The one you're after is RFC 4252, The Secure Shell (SSH) Authentication Protocol. From the Framework section:
The server drives the authentication by telling the client which authentication methods can be used to continue the exchange at any given time. The client has the freedom to try the methods listed by the server in any order. This gives the server complete control over the authentication process if desired, but also gives enough flexibility for the client to use the methods it supports or that are most convenient for the user, when multiple methods are offered by the server.
At that point your client (PuTTY) will check to see if it has any keys to use (via Pageant). If it finds a private key applicable to that host, it uses it. If it finds an encrypted private key Pageant will ask for a password and cache things for you. Then putty tells the server "publickey" and the process goes by RFC 4252 Section 7. The SSH server MAY fall back to password authentication if configured to do so (typically via PAM).
With putty you need to either specify the key in the configuration or be running the putty agent (pageant.exe) and have your keys loaded. The server really doesn't know anything your client doesn't tell it, and what the client communicates depends heavily on your configuration.
Putty docs:
It doesn't send your private key, that's the beauty of private/public keypairs.
In short, there's a pair of keys, a private one that is totally secret, never send to anybody; and the public key, that you can send to the winds, or post to the web, whatever.
The interesting property is that even if you know a public key, it's astronomically hard to figure the corresponding private key. At the same time, any message encrypted with one key can only be decrypted with the other one.
the handshake used by ssh is roughly like this:
sshd server is listening at a TCP port, usually port 22.ssh client connects with the sshd server.I'm not near PuTTY right now, but under Auth or Login you can choose what keys to use for each saved session. There's no service(s) running.
Basically, the SSH handshake compares the methods the client and server support for authentication, data encryption, etc. until they find ones they can agree on, or give up. The standard port is tcp/22, but some folks change it to reduce nuisance hack attempts.
My favorite explainer of how keypairs work is Simon Tatham: Public key authentication - an introduction
