Calculate $\int_0^{\pi} \frac{ \cos ( kx ) }{ 1 - 2 \tau \cos (x ) } dx$

If contour integration is acceptable, let $\alpha=\frac{1-\sqrt{1-4\tau^2}}{2\tau}$ and $1/\alpha=\frac{1+\sqrt{1-4\tau^2}}{2\tau}$. $$\newcommand{\Res}{\operatorname*{Res}} \begin{align} \int_0^\pi\frac{\cos(kx)}{1-2\tau\cos(x)}\,\mathrm{d}x &=\frac12\int_{-\pi}^\pi\frac{\cos(kx)+i\sin(kx)}{1-2\tau\cos(x)}\,\mathrm{d}x\tag1\\ &=\frac12\oint_{|z|=1}\frac{z^k}{1-\tau z-\tau z^{-1}}\,\frac{\mathrm{d}z}{iz}\tag2\\ &=\frac1{2i}\oint\frac{z^k}{z-\tau z^2-\tau}\,\mathrm{d}z\tag3\\ &=\frac i{2\tau}\oint\frac{z^k}{(z-\alpha)(z-1/\alpha)}\,\mathrm{d}z\tag4\\ &=\frac1{2i\sqrt{1-4\tau^2}}\oint\left(\frac{z^k}{z-\alpha}-\frac{z^k}{z-1/\alpha}\right)\mathrm{d}z\tag5\\ &=\frac{\pi\alpha^k}{\sqrt{1-4\tau^2}}\tag6 \end{align} $$ Explanation:
$(1)$: since cosine is even, we can double the domain and divide by $2$
$\phantom{(1)\text{:}}$ since sine is odd, adding it to the integral has no effect
$(2)$: write trig functions as functions of $z$ on the unit circle ($z=e^{ix})$
$(3)$: simplify
$(4)$: factor the denominator
$(5)$: partial fractions
$(6)$: $1/\alpha$ is outside the unit circle and $\Res\limits_{z=\alpha}\left(\frac{z^k}{z-\alpha}\right)=\alpha^k$

Using the fact that for all $r\in (-1,1)$ and all real $x$, \begin{align*} \sum_{n=-\infty}^\infty r^{|n|}e^{inx} =\frac{1-r^2}{1+r^2 - 2r\cos x} \end{align*} (for background, this is the series representation of the Poisson kernel) we get \begin{align*} \int_0^{\pi} \frac{\cos(kx)}{1+r^2 -2r\cos x} dx =& \frac 1 2 \int_{-\pi}^{\pi} \frac{e^{ikx}}{1+r^2 -2r\cos x} dx\\ =&\frac 1{2}\int_{-\pi}^{\pi} \left(\frac 1 {1-r^2}\sum_{n=-\infty}^\infty r^{|n|}e^{inx}\right)e^{ikx} dx\\ =&\frac 1{2(1-r^2)}\sum_{n=-\infty}^\infty r^{|n|}\underbrace{\int_{-\pi}^{\pi} e^{i(n+k)x} dx}_{=0 \ \text{ if }\ n+k\ne 0}\\ =&\frac { \pi r^k}{1-r^2}. \end{align*} Now solving for $$\frac r{1+r^2} = \tau \ \ \Longrightarrow \ \ r=\frac {1-\sqrt{1-4\tau^2}}{2\tau} \in (-1,1) $$ we finally get \begin{align*} \int_0^{\pi} \frac{\cos(kx)}{1 -2\tau\cos x} dx = &\frac{\pi r^k(1+r^2)}{1-r^2} =\frac{\pi}{ \sqrt{1-4\tau^2}}\left(\frac{1-\sqrt{1-4\tau^2}}{2\tau}\right)^{k}. \end{align*}

If $k\in\mathbb{N}$, the exercise immediately boils down to the determination of the coefficient of $\cos(kx)$ in the Fourier cosine series of $$(\text{assuming }|\tau|<\tfrac{1}{2})\qquad \frac{1}{1-2\tau\cos(x)}=\sum_{n\geq 0}\tau^n\left(e^{ix}+e^{-ix}\right)^n=\sum_{n\geq 0}\tau^n\sum_{m=0}^{n}\binom{n}{m}e^{(2m-n)ix} $$ which, by applying $\text{Re}$ to both sides, equals $$ \sum_{n\geq 0}\tau^n \sum_{m=0}^{n}\binom{n}{m}\cos\left((2m-n)x\right). $$ Therefore the coefficient of $\cos(kx)$ is the sum of $\tau^n\binom{n}{m}$ where $2m-n=\pm k$ and $m\leq n$, and $$\int_{0}^{\pi}\frac{\cos(kx)}{1-2\tau\cos(x)}\,dx = \frac{\pi}{2}\cdot\left[\sum_{\substack{n\geq k \\ n\equiv k\!\!\pmod{2}}}\!\!\!\binom{n}{\frac{k+n}{2}}\tau^n+\sum_{\substack{n\geq k \\ n\equiv k\!\!\pmod{2}}}\!\!\!\binom{n}{\frac{n-k}{2}}\tau^n\right]. $$ By mapping $n$ into $k+2r$ we get $$ \pi\tau^k\sum_{r\geq 0}\binom{2r+k}{r}\tau^{2r}$$ which is clearly related to $\frac{1}{\sqrt{1-4\tau^2}}=\sum_{r\geq 0}\binom{2r}{r}\tau^{2r}$.