Why is $1$ not a prime number?

One of the whole "points" of defining primes is to be able to uniquely and finitely prime factorize every natural number.

If 1 was prime, then this would be more or less impossible.

It's important to understand that this is not something that can be proved: it's a definition. We choose not to regard 1 as a prime number, simply because it makes writing lots of theorems much easier.

Noah gives the best example in his answer: Euclid's theorem that every positive integer can be written uniquely as a product of primes. If 1 is defined to be a prime number, then we'd have to change that theorem to: "every positive integer can be written uniquely as a product of primes, except for infinite multiplications by 1". So we choose to go with the easier path of defining 1 to not be a prime.

The main point of talking about prime numbers is Euclid's theorem that every positive integer can be written uniquely as a product of primes. As Justin remarks, this would break horribly if $1$ were considered prime, for example we could factor $2$ as $2\times1\times1\times1\times1\times1$. Instead we say that $1$ is not a prime, but it is the product of zero primes (see Why is $x^0 = 1$ except when $x = 0$? to understand why any prime multiplied by itself $0$ times is $1$) so Euclid's theorem works out nicely!