Load private key manually with ssh

Another way is to use ssh-agent and ssh-add commands manually before using ssh.

ssh-agent (if not running already)
ssh-add /path/to/private_key

example:
ssh-agent
ssh-add ~/.ssh/id_rsa

You can also add a specific configuration to each host you access, which is pretty much the same as persisting the usage of the flags available in ssh.

There's an entire world of flags available, and there are some mappings for each different service specialization provided. In your case, using specific id_rsa files, you could write down to your ~/.ssh/config file:

...

Host host_alias
    HostName host_name
    IdentityFile ~/.ssh/id_rsa_you_want

...

Then, you can simply use:

ssh host_alias

And the id_rsa_you_want will be used -- as well as any further configurations you may apply to the connection. See man ssh_config for the entire list of available directives.

You can use the -i option.

Source: man ssh

-i identity_file
    Selects a file from which the identity (private key) for public key authentication is read.  The default is ~/.ssh/identity for protocol
    version 1, and ~/.ssh/id_dsa, ~/.ssh/id_ecdsa and ~/.ssh/id_rsa for protocol version 2.  Identity files may also be specified on a per-
    host basis in the configuration file.  It is possible to have multiple -i options (and multiple identities specified in configuration
    files).  ssh will also try to load certificate information from the filename obtained by appending -cert.pub to identity filenames.