Definition of the gamma function for non-integer negative values

The definition you gave is valid only for $x >0$, has you have pointed out. However, you can extend $\Gamma$ to negative non integer values by defining

$$\Gamma(x) := \frac{1}{x}\Gamma(x+1) $$

whenever $x <0$, $x \notin \mathbb Z$. For example you get

$$\Gamma\left(-\frac{1}{2}\right) = -2 \Gamma\left(\frac{1}{2}\right) $$

and $\Gamma\left(\frac{1}{2}\right)$ is given by the integral you have presented.


The integral $\int_0^\infty t^{x-1}e^{-t}\,dt$ is a "representation" of the Gamma function for $x>0$. That is, for $x>0$

$$\int_0^\infty t^{x-1}e^{-t}\,dt=\Gamma(x)$$

But the Gamma Function exists for all complex values of $x$ provided $x$ is not $0$ or a negative integer.

The idea of "representing" a function on a subset of the domain of definition is introduced in elementary calculus courses. For example, we can represent the function $f(x)=\log(1+x)$ as the series

$$f(x)=\log(1+x)=\sum_{n=1}^\infty\frac{(-1)^{n-1}x^n}{n} \tag 1$$

which is valid for $-1<x\le 1$. However, $f(x)$ exists for all $-1<x$.


To add on the other answers :


This is one of the 1st example of analytic continuation. It is clear that $$\Gamma(z) = \frac{\Gamma(z+1)}{z}=\frac{\Gamma(z+2)}{z(z+1)}=\frac{\Gamma(z+n+1)}{z(z+1)\ldots(z+n)}$$ makes $\Gamma(z)$ well-defined for $z \in \mathbb{C}, -z \not \in \mathbb{N}$.

But it is not so obvious (without a lot of theorems in complex analysis) that this continuation is the only one being analytic, in the same way that $\frac{1}{1-z}$ is the only one analytic continuation of $\sum_{n=0}^\infty z^n$ beyond $|z|< 1$