I have a central server where I stored all the private ssh keys to the different machines that I want to ssh to. Currently, only sysadmins have access to this 'central' server.

Given the above scenario, I like to ask the following questions:

  1. How do you protect your private ssh keys? I read about ssh-agent but I am not sure how to use it or if it can be used in this situation.
  2. If a sysadmin leaves and he copies all the private ssh keys, then he has access to all the servers. How do you deal with this situation?

2 Answers 2


SSH private keys should never leave the machine they were generated on, unless to be moved into a more secure platform, such as an HSM. Likewise, they should never be accessible to anyone other than the person who generated them and will be using them, e.g. an administrator's workstation, or better, an HSM/smartcard. If you're going to be accessing a server from multiple machines, generate a unique key on each machine and deploy the public keys accordingly, or put a public key from a smart card or other strong authenticator into the authorized keys.

For starters, I always recommend protecting private keys with a passphrase. This will mitigate the chances of unauthorized key usage if it falls into the wrong hands. A higher security measure is to embed the private key into a smartcard, and make use of the cryptographic functions on it via a middleware. Redhat makes it easy to do all this.

How do you protect your private ssh keys? I read about ssh-agent but I am not sure how to use it or if it can be used in this situation.

ssh-agent is used in a terminal session, to cache the unencrypted version of the private key, making it unnecessary for you to enter the key's passphrase each time it is used. You can also use an external authenticator by which you make use of an authentication challenge/response, but never have access to the cleartext. Please take a peek at http://www.gooze.eu/howto/using-openssh-with-smartcards/using-ssh-authentication-agent-ssh-add-with-smartcards in order to review how you can use ssh with a secure authenticator.

If a sysadmin leaves and he copies all the private ssh keys, then he has access to all the servers. How do you deal with this situation?

This is a very strong argument as to why private keys should never be shared, and certainly why multiple users' keys should never be stored together. Plain and simple, there is no reason to do so. No one else has any business getting your private key. That said, even if you were in a situation where someone got their hands on a bunch of active private keys, if they were properly passphrase-protected by a strong passphrase, it would largely be a non-issue.

Again, by credentialing your users with a strong authenticator having cryptographic functions, you can avoid this situation altogether. When a machine needs to perform cryptographic functions, rather than store a private key on the filesystem, you can store it on an HSM to prevent exfiltration (theft) of the private keys.

SSH keys private keys can be a component of multi-factor authentication system, such as a smart card, or another type of hardware security module (USB form factor, PCIe card, fortezza, networked HSM, etc) This is pretty standard now for security conscious businesses, governments and the military operate simply because one compromise can lead to massive exposure; passwords are too weak.

A smart card, depending on the security of it, can be either (A) unvalidated, a (B) FIPS 140-2 level 2 device, all the way up to a (C) controlled cryptographic device usable for Top Secret SCI information and Suite B algorithms. In the case of (B), a government authorized lab has independently validated the cryptography, and (C), the NSA has evaluated the device in question. B is considered equivalent to best commercial grade security, and C is considered military / intelligence community grade. Prices go up for the robust stuff!

  • 1
    The very act of storing an SSH private key on the filesystem is an incorrect security practice. Your comment didn't even deal with the massive security gap in the OP's question.
    – Brennan
    Commented Apr 19, 2012 at 21:22
  • @Brennan - feel free to edit my answer to your liking. Yes, there are always ways and means of doing things in a more secure manner, but we need to strike a balance between security, cost, and ease-of-use. In 99% of cases, keeping password-encrypted private keys on the fs is perfectly fine, especially in single-user machines.
    – EEAA
    Commented Apr 19, 2012 at 21:24
  • the problem is the idea of performing key storage on the server, not on the adminstrator's workstation. I'll make some updates :)
    – Brennan
    Commented Apr 19, 2012 at 21:31

I agree with ErikA. However, there are situations when you don't want passphrase-protected private keys (e.g. for automatical backup). In that case, you and every sysadmin each has a private key.

If you want a shared private key, then you can limit access by IP: the central server is the unique place from that someone can connect to other machines.

Depend on the importance of each machine, you can have a different solution for each.

  • For completeness, if you do use passphraseless keys for automated tasks like backup, then you should also ensure that those keys are limited in their scope on the servers they are configured to log in to. Read the AUTHORIZED_KEYS FILE FORMAT section of the sshd man page for information on how to restrict what a specific key can do on a server. In the case of backups, you would want to ensure that the user can only run specific backup tasks (via sudo for example, again with limited access), and you probably want to do things like disable pty allocation as well Commented Apr 19, 2012 at 23:49

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .