I used to specify per-host entries in ~/.ssh/config file, but I recently found an excellent and automatic solution:
Place each key in a directory named after its host and user, like:
~/.ssh/ids/github.com/git/id_rsa
~/.ssh/ids/myhost/me/id_rsa
~/.ssh/ids/myhost/mygf/id_rsa
Then set up ~/.ssh/config file this way:
IdentityFile ~/.ssh/ids/%h/%r/id_rsa
IdentityFile ~/.ssh/id_rsa
Now once you ssh me@myhost (or ssh mygf@myhost), the correct key will be used. Adding a new key can now be done by simply creating appropriate folders and copying the keys there.
While it works "out of the box" on Linux, things get a bit harder (but also a bit more secure) on Mac OS X: if your key requires a passphrase (which it should), then ssh client won't prompt for it, expecting to get it from ssh-agent (this is described in detail here).
For some reason which I considered not worth investigating, OS X's ssh-agent did not want to pop up its nice and very secure message box. I worked around this by adding the needed keys manually via ssh-add (like, ssh-add <path-to-key>). However, I'm not satisfied with this solution and will probably need to take a second look at it, especially if agent doesn't start to work after I reboot the Mac.
