When, exactly, should old password reset tokens be deleted?

You should not reset the token when the user clicks on it, because the user might be disturbed in the process (e.g. his cat jumped on the keyboard -- mine does that on an all-too-regular basis).

(Two days ago I was using such a link -- not for password reset, but similar -- which was deactivated as soon as I clicked on it, and it turned out that the page behind it was not compatible with Chrome. So I had to request a new link, doing the whole process one more time, and I cursed them for that. When dealing with passwords, you want and need to make a cooperative ally of the user, and certainly not to make him angry.)

When the password is actually reset, all pending password reset links should be deactivated. It is simpler if you allow only one reset link at a time; if the user requests a password reset while the previous link is still valid, just send it again (possibly, reset the timeout counter). There is no need for supporting several simultaneously valid, distinct password reset links. One link at a time means an easier database design, and thus less scope for bugs.