I assume you mean you want to use multiple private keys on a client, not on a server.
By default, the ssh client loads:
for protocol version 1
~/.ssh/identity
for protocol version 2:
~/.ssh/id_dsa
~/.ssh/id_ecdsa
~/.ssh/id_ed25519
~/.ssh/id_rsa
It will try all those keys (until the first succeeds).
If you want to add other keys you have two options:
1. add a per host selection via ~/.ssh/config
You have already mentioned this, so I assume you are familiar with it. In short it looks like this:
Host host1
HostName host1.example.com
IdentityFile ~/.ssh/id_rsa_host1
Host host2
HostName host2.example.com
IdentityFile ~/.ssh/id_rsa_host2
2. use the ssh-agent
Configure the ssh-agent (lots of docs on google, eg. http://mah.everybody.org/docs/ssh ), it comes by default with OpenSSH. You can add via
ssh-add -i $KEY_FILE
each private key.
3. do it manually
start the ssh client with the -i option to select which key you want to use.
ssh -i $KEY_FILE
I would suggest to use option 2 (ssh-agent), it gives you several benefits in addition, but it all depends on what your situation is.
man 5 ssh_config