Why do we use von Neumann ordinals and not Zermelo ordinals?

There is no real, deep, fundamental reason. You can find a bijection between the set of Von Neumann naturals and Zermelo naturals, so anything you can do with the one set you can do with the other.

However, Von Neumann naturals are more convenient in practice for a lot of reasons. For one, the element we call $n$ also has exactly $n$ elements. That means that we can use the actual set $n$ as a cardinality, defining "the set $A$ has $n$ elements" to mean that there is a bijection between $A$ and $n$. For another, a convenient way to define the ordinals is to say that they are the transitive sets which are linearly ordered by $\in$. Then the Von Neumann naturals are precisely the finite ordinals, which is a natural and important way to think about the finite ordinals.

A simple motivation for the Von Neumann style is to write it in this way: $$ n := \{m |\ m<n\}. $$ I.e., the ordinal $n$ is the set of all the ordinals up to $n-1$. This is, I'd say, the idea behind the Von Neumann ordinals, though obviously it isn't quite a proper definition ($m$ from what base set? What is $<$?).

Alternative way to express it: $$\begin{align} 0 &:= \{\} \\ n+1 &:= \{0 \ldots n\} \end{align}$$