How do we know our definitions don't lead to contradictions

A definition like that can't lead to a contradiction since it is just a definition: the meaning you've assigned to the symbols "$0!$".

A definition might lead to a problem with some rules of arithmetic you like. For example, if you chose to define $x^0 = 0$ instead of $1$ then the identlty $$ x^{a+b}= x^ax^b $$ would fail sometimes. So it would be a bad definition, but not a contradiction.

All the usual suggestions for how to define division by $0$ break some identity in ordinary arithmetic, which is why we don't use any of them.

Defining $0! = 1$ preserves the identity $$ n! = n(n-1)! $$ No other definition would do that.

It depends what you mean by a definition. A definition such as $$ P \mathrel{:=} \{ i \in \Bbb{N} \mid \mbox{$i$ is prime}\} $$ where the defining property of $P$ simply states that it is equal to something that already exists is completely unproblematic. But in addition to explicit definitions like this, we often use various kinds of implicit definition. A recursive definition such as: $$ \begin{align*} 0! &= 1 \\ (n+1)! & = n! \times (n + 1) \end{align*} $$ is a fairly explicit form of implicit definition that can be shown to be consistent using the recursion theorem as discussed in Marcel's answer.

A recursive "definition" (of a function $f : \Bbb{N} \to \Bbb{N}$) such as $$ \begin{align*} f(0) &= 1 \\ f(n+1) & = f(n+2) \end{align*} $$

would be considered undefined in programming languages, but from a mathematical perspective it is just a loose definition (i.e., an implicit definition that does not uniquely determine the defined object): it says that $f(0) = 1$ and $f(i) = f(j)$ for $i, j > 0$ and this system of equations has many solutions. If we had instead:

$$ \begin{align*} f(0) &= 1 \\ f(n+1) & = 2(f(n+2)+1) \end{align*} $$

and still required $f : \Bbb{N} \to \Bbb{N}$, then this "recursive" definition would be inconsistent, i.e., there are no solutions and asserting the existence of such an $f$ would lead to a contradiction (do you see why?).

Another form of implicit definition would be to define the function $\exp : \Bbb{R} \to \Bbb{R}$ as the solution of the following system of equations:

$$ \begin{align*} \exp(0) &= 1 \\ \exp'(x) &= \exp(x) \end{align*} $$

Here an explicit construction of a witness (say using a power series) shows that the definition is consistent (and a simple argument using properties of the derivative shows that the definition characterises $\exp$, i.e., the equations have only one solution).

In general, there are many forms of implicit definition. To show that it is consistent to take a property $\phi(X)$ to be the defining property of a new object $X$, you have to prove $\exists x\phi(x)$. You will probably also want to prove that the existence is unique, i.e., $\forall x\forall y(\phi(x) \land \phi(y) \Rightarrow x = y)$, unless you really did intend your definition to be loose. If you are lucky, then something like the recursion theorem or the theory of differential equations will help with these proofs.

Certain types of definitions can lead to contradictions. It is up to the person proposing the definition to show that this can't happen. Look up well-defined.