How to get pinentry-curses to start on the correct tty?

The gpg-agent man page explains under the option --enable-ssh-support that the ssh agent protocol is not able to provide the name of the tty to the agent, so it defaults to using the original terminal it was started in. Before running the ssh command that requires a passphrase in a new terminal you need to type

gpg-connect-agent updatestartuptty /bye

in the new terminal to update the agent's view of which tty or display to use.

As per the upstream bug against openssh, the proper way to this is adding the following to your ~/.ssh/config:

Match host * exec "gpg-connect-agent UPDATESTARTUPTTY /bye"

This has worked for me perfectly so far.