Bound for $\sum_{k=0}^{n}(-1)^k{3n\choose k}{n\choose k}$.

Using the "coefficient-of" notation, we have $$S_n:=\sum_{k=0}^{n}(-1)^k\binom{3n}{k}\binom{n}{k}=\sum_{k=0}^{n}[z^k](1-z)^{3n}\times[z^{n-k}](1+z)^n=[z^n]\big((1-z)^3(1+z)\big)^n.$$ Let $f(z)=(1-z)^3(1+z)/z$; then, by Cauchy integral formula, for any $r>0$ we have $$S_n=\frac{1}{2\pi\mathrm{i}}\oint_{|z|=r}\big(f(z)\big)^n\frac{dz}{z}\implies|S_n|\leqslant\Big(\max_{|z|=r}\big|f(z)\big|\Big)^n.$$ The key to a solution is to choose $\color{blue}{r=1/\sqrt{3}}$ (this is suggested by saddle points of $\big|f(z)\big|$, that is, the solutions $z=(-1\pm\mathrm{i}\sqrt{2})/3$ of $f'(z)=0$; alternatively, we can let $r$ be arbitrary, and minimize the result w.r.t. $r$ in the end). Using $|1+z|^2+|1-z|^2=2(1+|z|^2)$ again, we arrive at $$\text{maximize}\quad a^3 b\quad\text{subject to}\quad a^2+b^2=8/3.$$ Solving it the way you know, we find $a^2=2$, $b^2=2/3$, $a^3 b=4/\sqrt{3}$ and $\color{blue}{\max\limits_{|z|=r}\big|f(z)\big|=4}$.