Compute $\int_0^{\pi} \frac{\cos(nx)\cos(x) - \cos(nt)\cos(t)}{\cos(x) -\cos(t)}dt$

Here's a solution that only rests on the following simple trigonometric identity: $$\cos(a+b)+\cos(a-b)=2\cos(a)\cos(b)\tag{1}$$ We'll get back to it later, but for now, notice that $$\begin{split} I_n(x)&=\int_0^{\pi} \frac{\cos(nx)\cos(x) - \cos(nt)\cos(t)}{\cos(x) -\cos(t)}dt\\ &=\int_0^{\pi}\frac{[\cos(nx)-\cos(nt)]\cos(x) + \cos(nt)[\cos(x)-\cos(t)]}{\cos(x) -\cos(t)}dt\\ &=\cos(x)\int_0^{\pi}\frac{\cos(nx)-\cos(nt)}{\cos(x) -\cos(t)}dt+\int_0^\pi\cos(nt)dt \end{split}$$ In other words, $$I_n(x)=\cos(x)J_n(x)+\pi\delta_{n=0}\tag{2}$$ where we define $$J_n(x)=\int_0^\pi \frac{\cos(nx)-\cos(nt)}{\cos(x)-\cos(t)}dt$$ and the Kronecker symbol $\delta_{n=0}$, which is equal $0$, unless $n=0$, in which case it's equal to $1$.

Now, let's go back to (1). Plugging $a=nx$ and $b=x$ into that identity implies that $$\cos((n+1)x)+\cos((n-1)x)=2\cos x \cos(nx)$$ Subtracting the same equation with $t$ to this one yields $$ \begin{split} \cos((n+1)x)-\cos((n+1)t) \\ +\cos((n-1)x)-\cos((n-1)t)=\\ 2\cos x \cos(nx)-2\cos(t)\cos(nt) \end{split}$$ Dividing by $\cos(x)-\cos(t)$, and integrating over $[0,\pi]$ leads to $$J_{n+1}(x)+J_{n-1}(x)=2I_n(x)\tag{3}$$ Finally, combining [2] and [3] gets us, for $n\geq 0$, $$J_{n+2}(x)-2\cos(x)J_{n+1}(x)+J_{n}(x)=0$$

The solution to this second-order recurrence relation is $$J_n(x)=\alpha e^{inx}+\beta e^{-inx}$$ Since, $J_0=0$ and $J_1=\pi$, $$J_n(x)=\frac {\pi \sin(nx)}{\sin x}$$ and $$I_n(x)=\pi\cos(x)\frac{\sin(nx)}{\sin(x)} \mbox{ for } n\geq 1 \mbox{, and }I_0=\pi$$

A nice way to evaluate generalized integrals is to consider them as the coefficients of an infinite series. Therefore, the coefficient of the $n$th term is simply the integral under question. Before we begin though, there is one identity to note:

$$\sum\limits_{n\geq0}z^n\cos nx=\frac {1-z\cos x}{z^2-2z\cos x+1}$$ Proof: Rewrite $\cos nx$ as the real part of $e^{nix}$. Using the infinite geometric sequences, we get that$$\sum\limits_{n\geq0}\left(ze^{ix}\right)^n=\frac 1{1-ze^{ix}}$$Now, take the real part of both sides. Clearly, the left - hand side becomes $z^n\cos nx$. Meanwhile, the right - hand side becomes, through some clever rationalization$$\begin{align*}\operatorname{Re}\left[\frac 1{1-ze^{ix}}\right] & =\operatorname{Re}\left[\frac 1{1-z\cos x-zi\sin x}\right]\\ & =\operatorname{Re}\left[\frac {1-z\cos x+zi\sin x}{(1-z\cos x)^2+z^2\sin^2x}\right]\\ & =\frac {1-z\cos x}{z^2-2z\cos x+1}\end{align*}$$completing the proof.

---

With that in mind, we are ready to begin. Since the OP has stated in the comments that he is trying to evaluate the integral$$I_n=\int\limits_0^{\pi}\mathrm dt\,\frac {\cos nx-\cos nt}{\cos x-\cos t}$$I will show a way to evaluate it in this answer. To wit, denote the generating function of the integral as $G(z)$

$$G(z)=\sum\limits_{n\geq0}I_nz^n$$

And remember that the coefficient of $z^n$ simply gives $I_n$. Interchange the sum and the integral, and using the identity we've derived above, get

$$\begin{align*}G(z) & =\int\limits_0^{\pi}\frac {\mathrm dt}{\cos x-\cos t}\sum\limits_{n\geq0}z^n\biggr[\cos nx-\cos nt\biggr]\\ & =\int\limits_0^{\pi}\frac {\mathrm dt}{\cos x-\cos t}\left[\frac {1-z\cos x}{z^2-2z\cos x+1}-\frac {1-z\cos t}{z^2-2z\cos t+1}\right]\end{align*}$$

Combining the two fractions, and recalling that any terms in $z$ are constants, the function becomes

$$G(z)=\frac {z(1-z^2)}{z^2-2z\cos x+1}\int\limits_0^{\pi}\frac {\mathrm dt}{z^2-2z\cos t+1}$$

The remaining integral can be easily evaluated using a Weierstrass substitution. Substitute $w=\tan\left(\tfrac t2\right)$ so that

$$\begin{array}{|c|c|c|}\hline w=\tan\left(\dfrac t2\right) & \mathrm dt=\dfrac {2\,\mathrm dw}{1+w^2} & \cos t=\dfrac {1-w^2}{1+w^2}\\\hline\end{array}$$

The remaining rational function can be evaluated in an elementary fashion

$$\begin{align*}G(z) & =\frac {2z(1-z^2)}{z^2-2z\cos x+1}\int\limits_0^{\infty}\frac {\mathrm dw}{w^2(1+z)^2+(1-z)^2}\\ & =\frac {2z}{z^2-2z\cos x+1}\arctan\left(\frac {1+z}{1-z}w\right)\,\Biggr\rvert_0^{\infty}\\ & =\frac {\pi z}{z^2-2z\cos x+1}\end{align*}$$

From the second line, it's important to observe that the argument of the inverse tangent will remain positive if and only if $|z|<1$. When $|z|<1$, then the denominator is positive, as well as the numerator. Therefore, $\tfrac {1+z}{1-z}>0$. However, if $|z|>1$, then the argument is less than zero and there is an extra negative sign. For the purpose of this question, we'll consider when $|z|<1$.

Now all we have to do is find the coefficient of $z^n$. There is a nice and convenient way to do this by using

$$2\cos x=e^{ix}+e^{-ix}$$

Factoring the denominator by grouping gives

$$\begin{align*}\frac z{z^2-2z\cos x+1} & =\frac z{(1-ze^{ix})(1-ze^{-ix})}\\ & =z\sum\limits_{k\geq0}z^k e^{kix}\sum\limits_{l\geq0}z^l e^{-lix}\end{align*}$$

Now observe what happens when we expand the products together$$\begin{multline}(1+ze^{ix}+z^2e^{2ix}+\cdots)(1+ze^{-ix}+z^2e^{-ix}+\cdots)\\=1+z(e^{ix}+e^{-ix})+z^2(e^{2ix}+1+e^{-2ix})+\cdots\end{multline}$$

The sum within the parenthesis seems to start off at the index of the $n$th term and decrease by a factor of two! Using this, it's possible to rewrite the coefficients conveniently as

$$a_k=\sum\limits_{m=0}^ke^{(k-2m)ix}=\frac {\sin x(k+1)}{\sin x}$$

Hence$$\frac {\pi z}{z^2-2z\cos x+1}=\pi\sum\limits_{k\geq1}\frac {\sin xk}{\sin x}z^k$$

And setting $k$ as $n$ gives the term $z^n$. Therefore, our integral is simply$$\int\limits_0^{\pi}\mathrm dt\,\frac {\cos nx-\cos nt}{\cos x-\cos t}\color{blue}{=\frac {\pi\sin xn}{\sin x}}$$

Completing Frank's solution:

$$ [z^n]\frac{\pi z}{z^2-2z\cos x+1} = \frac{\pi}{2}[z^{n}]\left(\frac{1}{z-e^{ix}}+\frac{1}{z-e^{-ix}}\right) $$ equals, by geometric series, $$ \frac{\pi}{2}\left(-e^{-(n+1)ix}-e^{(n+1)ix}\right)=-\pi\cos((n+1)x). $$