Equivalence of two characterizations of the norm of a quadratic integer.

For an integer $n$, the quotient group ${\mathbf Z}[i]/(n)$ clearly has order $n^2$. Now consider the chain of ideals $({\rm N}(\alpha)) \subset (\alpha) \subset {\mathbf Z}[i]$. The index $[{\mathbf Z}[i]:{\rm N}(\alpha)]$ is $({\rm N}(\alpha))^2$. Through multiplication by $\alpha$, ${\mathbf Z}[i]/(\overline{\alpha}) \cong (\alpha)/({\rm N}(\alpha))$ as additive groups. Thus $$[{\mathbf Z}[i]:({\rm N}(\alpha))] = [{\mathbf Z}[i]:(\alpha)][(\alpha):({\rm N}(\alpha))] = [{\mathbf Z}[i]:(\alpha)][{\mathbf Z}[i]:(\overline{\alpha})].$$ Complex conjugation induces an isomorphism of ${\mathbf Z}[i]/(\alpha)$ with ${\mathbf Z}[i]/(\overline{\alpha})$ (as rings or, sufficient for our purposes, as additive groups). Thus the above displayed equation implies $$({\rm N}(\alpha))^2 = [{\mathbf Z}[i]:(\alpha)]^2.$$ Now take positive square roots of both sides and you're done.

This method applies without essential changes to show for any quadratic ring ${\mathcal O}$ (by which I mean a subring of rank 2 inside a quadratic field, not necessarily the full ring of integers) and nonzero $\alpha$ in ${\mathcal O}$ that $\#({\mathcal O}/(\alpha)) = |{\rm N}(\alpha)|$. In the proof replace complex conjugation with whatever the conjugation operation is on the quadratic ring. The absolute value is essential if you're in a real quadratic field, where the norm function can take negative values. For example, $\#({\mathbf Z}[\sqrt{2}]/(1+5\sqrt{2})) = |-49| = 49$. This approach does not tell you a set of representatives for the cosets mod $\alpha$, but you didn't ask for that.

In order to go beyond quadratic rings, say to integers in a number field, this method breaks down because it depends on the crutch of the conjugation operation. It's true that in any order ${\mathcal O}$ in a number field $K$, $\#({\mathcal O}/(\alpha)) = |{\rm N}_{K/\mathbf Q}(\alpha)|$, but the method above doesn't directly extend to that setting. You could first prove the result for a larger Galois extension containing $K$ and then use that to derive the case of interest, or you just need other techniques (e.g., structure theorem for torsion modules over a PID). The trick above for the quadratic case is nice since it gives you the result fairly painlessly (no need to find an explicit set of representatives, which can be tedious for ${\mathbf Z}[i]/(\alpha)$ if the real and imaginary parts of $\alpha$ are not relatively prime, and then would you have to start all over again if you wanted to work in ${\mathbf Z}[\sqrt{10}]\dots$).


It is worth emphasis that the standard proof in KCd's answer has a very suggestive presentation in "fractional" form, since the "cancellations" are analogous to cancellation of fractions, viz.

$$\rm\ \alpha\bar\alpha R\ \subset\ \alpha R\ \subset\ R $$

$$\rm \Rightarrow\ \ N(\alpha )^2 =\: \left|\frac{R}{\alpha\bar\alpha R}\right|\: =\: \left|\frac{R}{\alpha R}\right|\: \left|\frac{\alpha R}{\alpha\bar\alpha R}\right|\: =\: \left|\frac{R}{\alpha R}\right|\: \left|\frac{R}{\bar\alpha R}\right|\: =\: \left|\frac{R}{\alpha R}\right|^{\:\!2}$$

This is one reason for the use of quotient-like notation for quotient structures.

A more general method is to rewrite the ideal as a module in Hermite normal form, via well-known algorithms, e.g. see H. Cohen's book (vol. 1) or Lemmermeyer's lecture notes, e.g. see this answer.