Best way to use multiple SSH private keys on one client

You can instruct ssh to try multiple keys in succession when connecting. Here's how:

$ cat ~/.ssh/config
IdentityFile ~/.ssh/id_rsa
IdentityFile ~/.ssh/id_rsa_old
IdentityFile ~/.ssh/id_ed25519
# ... and so on

$ ssh server.example.com -v
....
debug1: Next authentication method: publickey
debug1: Trying private key: /home/example/.ssh/id_rsa
debug1: read PEM private key done: type RSA
debug1: Authentications that can continue: publickey
debug1: Trying private key: /home/example/.ssh/id_rsa_old
debug1: read PEM private key done: type RSA
....
[server ~]$

This way you don't have to specify what key works with which server. It'll just use the first working key.

Also you would only enter a passphrase if a given server is willing to accept the key. As seen above ssh didn't try to ask for a password for .ssh/id_rsa even if it had one.

Surely it doesn't outbeat a per-server configuration as in other answers, but at least you won't have to add a configuration for all and every server you connect to!

From my .ssh/config:

Host myshortname realname.example.com
    HostName realname.example.com
    IdentityFile ~/.ssh/realname_rsa # private key for realname
    User remoteusername

Host myother realname2.example.org
    HostName realname2.example.org
    IdentityFile ~/.ssh/realname2_rsa  # different private key for realname2
    User remoteusername

Then you can use the following to connect:

ssh myshortname

ssh myother

And so on.