Deriving Moment Generating Function of the Negative Binomial?

To derive the mgf of the negative binomial distribution we are going to use the following identity:

$$\binom{-r}{y}=\left( -1 \right)^y \binom{r+y-1}{y} $$

We can prove that in the following way:

$$\begin{align} \binom{-r}{y} & = \frac{ \left( -r \right) \left(-r-1 \right) \ldots \left(-r-y+1 \right)}{y!}\\ & = \left(-1 \right)^y \frac{ \left(r+y-1 \right) \ldots \left( r+1 \right)r}{y!} \\ & = \left(-1 \right)^y \binom{r+y-1}{y} \end{align}$$

Now

$$M \left( t \right)=\sum_{y=0}^{\infty} e^{ty} \binom{y+r-1}{r-1} \left( 1-p \right)^y \times p^r $$

Grouping terms and using the above idenity we get:

$$\begin{align} M \left( t \right)& =p^r \sum_{y=0}^{\infty} \binom{y+r-1}{r-1} \left[ e^t \left( 1-p \right) \right]^y \\&=p^r\sum_{y=0}^{\infty} \binom{-r}{y}\left( -1 \right)^y\left[ e^t \left( 1-p \right) \right]^y \\& =p^r\sum_{y=0}^{\infty} \binom{-r}{y}\left[ -e^t \left( 1-p \right) \right]^y \end{align} $$

Then using Newton's Binomial Theorem: $\left( x+1 \right)^r= \sum_{i=0}^\infty {r\choose i}x^i$ provided that $|x|<1$, the last term becomes:

$$M \left(t \right)= \frac{p^r}{\left[ 1- \left(1-p \right)e^t \right]^r}$$

provided that $t<-\log(1-p)$

Note that the negative binomial distribution can come with a slightly different parameterization as well, as it has been pointed out in the comments. I leave it to you to derive the mgf for the other case.

Hope this helps.

The m.g.f. of a sum of independent random variables is just the product of their m.g.f.s, so $$ M_{Y_1+\cdots+Y_r}(t) = \left( M_{Y_1}(t) \right)^r. $$ \begin{align} M_{Y_1}(t) & = \operatorname E(e^{tY_1}) = \sum_{y=1}^\infty e^{ty} \Pr(Y_1=y) \\[10pt] & = \sum_{y=1}^\infty e^{ty} p(1-p)^{y-1} = \frac{p}{1-p} \sum_{y=1}^\infty \left(e^t(1-p)\right)^y \\[10pt] & = \frac{p}{1-p} \cdot \frac{\text{first term}}{1-\text{common ratio}} \\[10pt] & = \frac{p}{1-p}\cdot\frac{e^t(1-p)}{1-e^t(1-p)} \\[10pt] & = \frac{e^tp}{1-e^t(1-p)}. \end{align}

(Then remember to raise the whole thing to the power $r$.)