Download installer for old version of Git

Where can I get an older version of git for Windows?

You're right, the download page links to old source releases on kernel.org.

However, the GitHub page for git-for-windows provides download links, where you can get 2.3.4 portable dev preview, for example.

(note, I don't see the final release of 2.3.4, but as per Ramhound's comment there is no guarantee this was released; what is presented should hopefully fit your needs)