Momentum of a photon equals Planck's constant over wavelength

It seems like your are not satisfied by answers involving axioms. I think that you instead want to know the motivation behind the axiom beyond just saying that it works. I am not sure if my answer is the original motivation, but I think it can be viewed as a good motivation for the validity of $p= \frac{h}{\lambda}$. While other answers do a great job at going into the theory, I will tackle the question using more of an experimental motivation.

We will first start with the double slit experiment. This experiment is usually first introduced as evidence of the wave-like nature of light, where light emanating from one slit interferes with light emanating from the other (of course a different interpretation is found if we send single photons through the slits and the same interference pattern arises, but I digress). However, this experiment also works with electrons. You get an interference pattern consistent with treating the electrons as waves with wavelength $$\lambda=\frac hp$$

You get maxima in intensity such that $$\sin\theta_n=\frac{n\lambda}{d}$$

Where $\theta$ is the angle formed by the central maximum, the slit, and the maximum in question, $d$ is the slit separation, and $n$ is an integer.

This would then be a way to experimentally motivate/verify this relationship between momentum and wavelength for matter, but what about photons? The double slit experiment does not give us a way to validate $p=\frac h\lambda$ (that I know of. Maybe you could determine the radiation pressure on the detector?). Let's look at a different experiment.

We know that the energy of a photon from special relativity is $$E=pc$$

So, if our momentum relation is true, it must be that $$E=\frac{hc}{\lambda}=hf$$ which is something that can be verified experimentally to be true. The photoelectric effect is one such experiment we could do, where shining light onto a material causes electrons or other charge carriers to become emitted from that material. The higher the frequency of the light, the more energetic the electrons coming from the material are, and the maximum kinetic energy of an electron can be shown to follow $K_{max}=h(f-f_0)$ where $f$ is the frequency of the light and $f_0$ is the material-dependent threshold frequency (i.e. we need $f>f_0$).

I know that my answer does not get to a fundamental explanation of this relation in question, but I hope it shows why one would want it to be a fundamental idea that holds true when formulating QM. If you want a more fundamental explanation, then I will edit or remove this answer due to some pretty good fundamental answers already here.


Glossing over lots of subtleties:

A fundamental axiom of quantum mechanics is the canonical commutation relation $[\hat{X}, \hat{P}] = i \hbar$. In this position basis, this becomes $\hat{X} \to x$ and $\hat{P} \to -i \hbar \frac{\partial}{\partial x}$ (modulo lots of technical details involving the Stone-von Neumann theorem, etc.).

Another fundamental axiom of quantum mechanics is that states with definite values of a physical observable must be eigenstates of the corresponding Hermitian operator. So a particle with momentum $p$ is described by a wave function $|p\rangle$ satisfying $\hat{P} |p\rangle = p |p\rangle$. (Whether we can legitimately talk about the wavefunction of a massless relativistic particle is another subtlety that I'll gloss over.)

Putting this together, we have that in the position basis $$-i \hbar \frac{\partial \psi}{\partial x} = p \psi(x) \implies \psi(x) \propto e^{i p x / \hbar}.$$ So the wavefunction is spatially periodic with period $\lambda = 2 \pi \hbar / p = h / p$, so $p = h / \lambda$. This "derivation" works equally well whether or not the particle is massive or massless.


For convenience, let $k=2\pi/\lambda$ and $\omega=2\pi f$. Here $k$ is called the wavevector and $\omega$ is a version of the frequency that is in units of radians per second rather than oscillations per second.

Then we have the following two completely analogous relationships:

$$p=\hbar k $$

$$E=\hbar \omega .$$

The analogy holds because in relativity, momentum is to space as energy is to time.

If you assume $p=\hbar k$, then there are straightforward arguments that lead to $E=\hbar \omega$. If you assume $E=\hbar \omega$, there are similar aguments that get you to $p=\hbar k$. They're not independent of each other. If you believe in one, and you believe in relativity, then you have to believe in the other.

These are fundamental relationships that hold true in all of quantum mechanics. They're not just true for photons, they're true for electrons and baseballs.

With "why" questions like this, you have to decide what you want to take as a fundamental assumption. There are treatments of quantum mechanics that take various sets of axioms. Depending on what set of axioms you choose, these relations could be derived or they could be axioms. If someone tells you they have a proof of one of these relationships, you should ask them what assumptions they started from, and then ask yourself whether you find the assumptions more solid than these relations. Are the assumptions more intuitively reasonable? Better verified by experiment? Aesthetically preferable?