Why are position and momentum space examples of Pontryagin duality?

Practically speaking, the full machinery of Pontryagin duality is way more advanced than physicists need to understand the uncertainty principle. There are several ways to "derive" that the momentum-space wavefunction is the Fourier transform of the position-space wavefunction, which depend somewhat on your choice of starting postulates. Here's one common path:

One common starting fundamental postulate is the commutation relation $[\hat{x}, \hat{p}] = i \hbar.$ The most common position-space representation of this commutation relation is $\hat{x} \to x,\ \hat{p} \to -i \hbar \frac{\partial}{\partial x}$. In this representation, taking the inner product of $\langle x |$ and the eigenvalue equation $\hat{p} |p\rangle = p | p \rangle$ gives the differential equation $$-i \hbar \frac{d\, \psi_p(x)}{dx} = p\, \psi_p(x),$$ which has solution $\psi_p(x) = \langle x | p \rangle \propto e^{(i p x)/\hbar}$. Then to express an arbitrary state $| \psi \rangle$ in the momentum basis, we can use the resolution of the identity $$ \psi(p) = \langle p | \psi \rangle = \int dx\ \langle p | x \rangle \langle x | \psi \rangle \propto \int dx\ e^{-ipx/\hbar} \psi(x),$$ which is just the Fourier transform. This generalizes straightforwardly into higher dimensions.

BTW, the fact that position-space and momentum-space wavefunctions are Fourier transforms of each other (or more precisely, can be chosen to be Fourier transforms of each other) gives some nice intuition for the uncertainty relation but isn't actually necessary to derive it. All you need is the commutation relation, as I explain here.

Seems that one may have a bit of a chicken-and-egg situation here. What comes first, quantum mechanics or the Fourier transform? According to tparker's answer it seem that one should take quantum mechanics as more fundamental and the Fourier transform then follows from it. However, I suspect it is the other way around.

The Fourier properties were more or less imposed at the point where Planck discovered the relationship between energy and frequency $E=\hbar\omega$, which was later extended to a relationship between momentum and the propagation vector ${\bf p}=\hbar{\bf k}$. Due to these relationships, the fathers of quantum mechanics expanded everything in terms of plane waves. Well, plane waves form an orthogonal basis. Hence, such an expansion comes down to a Fourier analysis. As a consequence, one then obtains the Heisenberg uncertainty relationship.

However, one also finds in quantum mechanics some Heisenberg-type uncertainty relations that does not seem to follow directly from a Fourier relationship. For example, consider the uncertainty relationship associated with spin. This begs the question, what is the underlying principle that leads to an uncertainty relationship, which is shared by Fourier analysis?

This underlying principle, in my view, is the notion of mutually unbiased bases. Any inner product between elements from the respective mutually unbiased bases $\langle x|k\rangle$ gives a constant magnitude, independent of the choice of elements (the phase could be different). Any state with a particular representation in one basis will have a representation in a mutually unbiased basis that obeys a Heisenberg-type uncertainty relationship; the width in terms of one representation would be inverse proportional to the width in the other representation.

What does this have to do with Fourier analysis? Well, a Fourier transform is a link between representations in two mutually unbiased bases. This follows from the fact that for these bases $\langle x|k\rangle=\exp(-ixk)$, which means that $|\langle x|k\rangle|=constant$. This property, ultimately leads to the uncertainty relationship as we know it.