Why is $a^n - b^n$ divisible by $a-b$?

They are all special cases of the Polynomial Factor Theorem: $\rm\: x-y\:$ divides $\rm\:f(x)-f(y),\:$ here for $\rm\,f\,$ a polynomial with integer coefficients, so $\rm\:f(x)-f(y) = (x-y)\:g(x,y)\:$ for a polynomial $\rm\:g\:$ with integer coefficients. The prior equation (so the divisibility) remains true when we evaluate the indeterminates $\rm\:x,y\:$ at integer values. This yields your examples by choosing $\rm\:f(z) = z^n.$

Said simpler: $\rm\: mod\,\ x\!-\!y\!:\ \ x\equiv y\,\Rightarrow f(x)\equiv f(y)\ $ by the Polynomial Congruence Rule.

for example: $\ \rm\ mod\,\ 9\!-\!4\!:\ \ 9\equiv 4\:\Rightarrow\ 9^n\equiv\, 4^n\ $ (alternatively by the Congruence Power Rule).

Note that the all linked proofs proceed by induction on $\rm\,n\,$ (= polynomial degree).

It can be generalized as:

$$a^n-b^n = (a-b)(a^{n-1}+a^{n-2}b+\cdots +b^{n-1})$$

If you are interested in a modular arithmetic point of view, since $a \equiv b \pmod{a-b},$ $a^n \equiv b^n \pmod{a-b}.$

Your last two examples are true because what you are essentially doing is plugging in $n=1.$

Consider the polynomial $f(x)=x^n-b^n.$ Then $f(b)=b^n-b^n=0.$ So $b$ is a root of $f$ and this implies $(x-b)$ divides $f(x)$. Put $x=a$, then $a-b$ divides $f(a)=a^n-b^n.$