Why do the factorials appear in differences of consecutive powers?
For any polynomial of degree $n$, taking $n$ divided differences will get you to a constant. That constant is $n!$ times the coefficient of the leading term. As your polynomial, $k^n$, has a leading coefficient of $1$ the constant will be $n!$.