Evaluating $\int^{\pi}_0\arctan\left(\frac{p\sin x}{1-p\cos x}\right)\sin(nx) dx$ by differentiation under integral?

For $p^{2} <1$, $$\frac{\sin x}{1+p^{2}-2p \cos (x)} = \sum_{k=1}^{\infty} p^{k-1} \sin(kx). $$

This can be derived by evaluating the geometric series $\sum_{k=0}^{\infty}(p e^{ix})^{k} $.

So assuming $n$ is a positive integer, we have

$$ \begin{align} I'(p) &= \int_{0}^{\pi} \sin (nx) \sum_{k=1}^{\infty} p^{k-1} \sin(kx) \, dx \\ &= \sum_{k=1}^{\infty}p^{k-1} \int_{0}^{\pi} \sin(nx) \sin(kx) \, dx \\ &= p^{n-1} \int_{0}^{\pi} \sin^{2}(nx) \, dx \tag{1} \\ &= \frac{p^{n-1}}{2} \left(\int_{0}^{\pi} \, dx - \int_{0}^{\pi} \cos(2nx) \, dx \right) \\&= \frac{p^{n-1}}{2} (\pi-0) \\ &= \frac{\pi}{2}p^{n-1}. \end{align}$$

$(1)$ The functions $\sin(nx)$ and $\sin(kx)$ are orthogonal on $[0, \pi]$ unless $k=n$.

We assume $p^2<1$.

First observe that $$ 2I^{\prime} (p)=\!\!\int^{\pi}_{0}\!\! \frac{2\sin x \sin (nx)}{1+p^2-2p \cos x}dx =\!\!\int^{\pi}_{0}\!\! \frac{\cos((n-1)x)}{1+p^2-2p \cos x}dx-\!\!\int^{\pi}_{0}\!\! \frac{\cos((n+1)x)}{1+p^2-2p \cos x}dx. \tag1 $$ Then, by the change of variable $t=\tan (x/2)$, one gets $$ \int^{\pi}_{0} \frac{1}{1+p^2-2p \cos x}\:dx =\int_0^{\infty} \frac{2 \:dt}{(1-p)^2+(1+p)^2t^2}=\frac{\pi}{1-p^2}.\tag2 $$ From the standard geometric sum identity, one may prove that $$ \frac{1-p^2}{1+p^2-2p \cos x}=1+2\sum_{k=1}^{n-1}p^k\cos(kx)+\frac{2\:p^n\cos(nx)}{1+p^2-2p \cos x}-\frac{2\:p^{n+1}\cos((n-1)x)}{1+p^2-2p \cos x}.\tag3 $$Since $\displaystyle \int^{\pi}_{0}\cos(kx)dx=0$, for $k=1,2,\cdots$, then integrating $(3)$ from $x=0$ to $x=\pi$ gives, using $(2)$, $$ \require{cancel} \cancel{(1-p^2)}\frac{\pi}{\cancel{(1-p^2)}}=\pi+2\:p^n\int^{\pi}_{0}\!\! \frac{\cos(nx)}{1+p^2-2p \cos x}dx-2\:p^{n+1}\int^{\pi}_{0}\!\! \frac{\cos((n-1)x)}{1+p^2-2p \cos x}dx. \tag4 $$ From $(4)$ we deduce $$ \int^{\pi}_{0}\!\! \frac{\cos(nx)\:dx}{1+p^2-2p \cos x}\!=\!p\!\int^{\pi}_{0}\!\! \frac{\cos((n-1)x)\:dx}{1+p^2-2p \cos x}\!=\cdots=\!p^n\!\int^{\pi}_{0}\!\! \frac{\:dx}{1+p^2-2p \cos x}=\!\frac{\pi\:p^n}{1-p^2}, \tag5 $$ using $(2)$.

Finally, from $(5)$, $(1)$ rewrites $$ \require{cancel} 2I^{\prime} (p)=\frac{\pi\:p^{n-1}}{1-p^2}-\frac{\pi\:p^{n+1}}{1-p^2}=\frac{\pi\:p^{n-1}}{\cancel{1-p^2}}(\cancel{1-p^2}) $$ or $$ I^{\prime} (p)=\frac{\pi}2 \:p^{n-1} $$ then integrating with respect to $p$, using $I(0)=0$, we obtain $$ I(p)=\frac{\pi}{2n} \:p^n,\quad n\geq1, $$ that is

$$ \int^{\pi}_{0}\arctan \left(\frac{p \sin x}{1-p \cos x}\right) \sin(nx) \:dx=\frac{\pi}{2n}\: p^n,\qquad p^2<1,\, n\geq1. $$