Prove $\int_{0}^{\pi/2} x\csc^2(x)\arctan \left(\alpha \tan x\right)\, dx = \frac{\pi}{2}\left[\ln\frac{(1+\alpha)^{1+\alpha}}{\alpha^\alpha}\right]$

This approach is similar to Ron Gordon's answer. We introduce a new variable $\beta$ before differentiating.

By substitution $\tan x=u$, $$\int_{0}^{\!\Large \frac{\pi}{2}} x\csc^2(x)\arctan \left(\! \alpha \tan x\right)\, \mathrm{d}x=\int_{0}^{\infty}\frac{\tan^{-1}u\tan^{-1}\alpha u}{u^2}\,\mathrm{d}u$$

We will prove $$\int_0^{\infty}\frac{\tan^{-1}\alpha u\tan^{-1}\beta u}{u^2}\,\mathrm{d}u=\frac\pi2\log\left(\frac{(\alpha+\beta)^{\alpha+\beta}}{\alpha^\alpha\beta^\beta}\right)\tag{1}$$

Differentiation gives \begin{align}\partial_\alpha\partial_\beta\int_0^{\infty}\frac{\tan^{-1}\alpha u\tan^{-1}\beta u}{u^2}\,\mathrm{d}u&=\int_{0}^{\infty}\frac{\mathrm{d}u}{(1+\alpha^2 u^2)(1+\beta^2 u^2)}\\&=\frac1{\alpha^2-\beta^2}\int_{0}^{\infty}\frac{\alpha^2}{1+\alpha^2u^2}-\frac{\beta^2}{1+\beta^2u^2}\,\mathrm{d}u\\&=\frac1{\alpha^2-\beta^2}\int_{0}^{\infty}\frac{\alpha}{1+u^2}-\frac{\beta}{1+u^2}\,\mathrm{d}u\\&=\frac\pi{2(\alpha+\beta)}\end{align} for $\alpha\ne\beta$. The case $\alpha=\beta$ can be proved by letting $\alpha\to\beta$. Then $$\partial_\beta\int_0^{\infty}\frac{\tan^{-1}\alpha u\tan^{-1}\beta u}{u^2}\,\mathrm{d}u=\frac{\pi}{2}(\log(\alpha+\beta)-C_1(\beta))$$ If $\alpha\to0$ we have $0=\frac\pi2(\log\beta-C_1(\beta))$ so $C_1(\beta)=\log\beta$. Integrating by $\beta$,$$\int_0^{\infty}\frac{\tan^{-1}\alpha u\tan^{-1}\beta u}{u^2}\,\mathrm{d}u=\frac{\pi}{2}((\alpha+\beta)\log(\alpha+\beta)-\beta\log\beta-C_2(\alpha))$$ Let $\beta\to0$ and we find $C_2(\alpha)=\alpha\log\alpha$. Thus $(1)$ is proven and putting $\beta=1$ gives the conclusion $$\int_{0}^{\!\Large \frac{\pi}{2}} x\csc^2(x)\arctan \left(\! \alpha \tan x\right)\, \mathrm{d}x=\frac\pi2\log\left(\frac{(\alpha+1)^{\alpha+1}}{\alpha^\alpha}\right)$$

Note that $$ \begin{align} 1-(1-\alpha^2)\sin^2(x) &=1+(1-\alpha^2)\frac{e^{2ix}-2+e^{-2ix}}{4}\\ &=\frac{1-\alpha^2}4\left[e^{2ix}+2\frac{1+\alpha^2}{1-\alpha^2}+e^{-2ix}\right]\\ &=\frac{1-\alpha^2}4e^{-2ix}\left[e^{2ix}+\frac{1+\alpha}{1-\alpha}\right]\left[e^{2ix}+\frac{1-\alpha}{1+\alpha}\right]\\ &=\frac{(1+\alpha)^2}4\left[1+\frac{1-\alpha}{1+\alpha}e^{2ix}\right]\left[1+\frac{1-\alpha}{1+\alpha}e^{-2ix}\right]\tag{1} \end{align} $$ Taking $\frac12$ of the log of $(1)$ yields $$ \frac12\log\left(1-(1-\alpha^2)\sin^2(x)\right) =\log\left(\frac{\alpha+1}2\right)-\sum_{n=1}^\infty\left(\frac{\alpha-1}{\alpha+1}\right)^n\frac{\cos(2nx)}n\tag{2} $$ Subtracting $\log(\alpha)$ from $(2)$ and sending $\alpha\to\infty$ gives $$ \log(\sin(x))=-\log(2)-\sum_{n=1}^\infty\frac{\cos(2nx)}n\tag{3} $$ Now, we are ready to tackle the integral given. Taking the derivative, using partial fractions, integrating by parts, then applying $(2)$ and $(3)$, yields $$ \begin{align} &\frac{\mathrm{d}}{\mathrm{d}\alpha}\int_0^{\pi/2}x\csc^2(x)\arctan(\alpha\tan(x))\,\mathrm{d}x\\ &=\int_0^{\pi/2}\frac{x}{\sin(x)}\frac{1}{1-(1-\alpha^2)\sin^2(x)}\,\mathrm{d}\sin(x)\tag{4a}\\ &=\int_0^{\pi/2}\left[\frac1{\sin(x)}-\frac{\raise{5mu}{\frac12}\sqrt{1-\alpha^2}}{1+\sqrt{1-\alpha^2}\sin(x)}+\frac{\raise{5mu}{\frac12}\sqrt{1-\alpha^2}}{1-\sqrt{1-\alpha^2}\sin(x)}\right]x\,\mathrm{d}\sin(x)\tag{4b}\\ &=\int_0^{\pi/2}x\,\mathrm{d}\left[\log(\sin(x))-\tfrac12\log\left(1-(1-\alpha^2)\sin^2(x)\right)\right]\tag{4c}\\ &=-\frac\pi2\log(\alpha)-\int_0^{\pi/2}\left[\log(\sin(x))-\tfrac12\log\left(1-(1-\alpha^2)\sin^2(x)\right)\right]\mathrm{d}x\tag{4d}\\ &=-\frac\pi2\log(\alpha)+\frac\pi2\log(2)+\frac\pi2\log\left(\frac{\alpha+1}2\right)\tag{4e}\\ &=\frac\pi2\log\left(\frac{\alpha+1}\alpha\right)\tag{4} \end{align} $$ Explanation:
$\mathrm{(4a)}$: differentiate with respect to $\alpha$
$\mathrm{(4b)}$: partial fractions
$\mathrm{(4c)}$: prepare to integrate by parts
$\mathrm{(4d)}$: integrate by parts
$\mathrm{(4e)}$: apply $(2)$ and $(3)$ noting that $\int_0^{\pi/2}\frac{\cos(2nx)}{n}\,\mathrm{d}x=\left[\frac{\sin(2nx)}{2n^2}\right]_0^{\pi/2}=0$

Integrating $(4)$ gives $$ \int_0^{\pi/2}x\csc^2(x)\arctan(\alpha\tan(x))\,\mathrm{d}x =\frac\pi2\log\left(\frac{(\alpha+1)^{\alpha+1}}{\alpha^\alpha}\right)\tag{5} $$

This integral may be done by observing the following: Let

$$I(\alpha) = \int_0^{\pi/2} dx \, x\, \csc^2{x} \, \arctan{\left ( \alpha \tan{x}\right)} $$

Then $I(\alpha)$ satisfies the differential equation

$$\begin{align}\frac{d}{d\alpha}\left ( \frac{I(\alpha)}{\alpha} \right ) &= -\frac1{\alpha^2} \int_0^{\pi/2} dx \frac{\arctan{\left (\alpha \tan{x} \right )}}{\tan{x}} \\ &= -\frac1{\alpha^2} \int_0^{\infty} du \frac{\arctan{\alpha u}}{u (1+u^2)} \\ &= -\frac1{\alpha^2} J(\alpha)\end{align}$$

One may prove this by integrating the original integral by parts on the one hand, and differentiating with respect to $\alpha$ on the other.

The integral on the RHS may be evaluated by, again, differentiating with respect to $\alpha$ as follows:

$$\begin{align} J'(\alpha) &= \int_0^{\infty} \frac{du}{(1+u^2) (1+\alpha^2 u^2)}\end{align}$$

This integral is easily done using partial fractions; the result is

$$J'(\alpha) = \frac{\pi}{2} \frac1{1+\alpha}$$

which implies that

$$J(\alpha) = \frac{\pi}{2} \log{(1+\alpha)}$$

because $J(0)=0$. We are now left to solve

$$\frac{d}{d\alpha}\left ( \frac{I(\alpha)}{\alpha} \right ) = -\frac{\pi}{2} \frac{\log{(1+\alpha)}}{\alpha^2}$$

This reduces to a simple integration by parts, the result of which is

$$I(\alpha) = \frac{\pi}{2} \left [(1+\alpha) \log{(1+\alpha)} - \alpha \log{\alpha} \right ] + C \alpha $$

Now we just need to show that $C=0$. This is equivalent to showing that

$$\int_0^{\pi/2} dx \frac{x^2}{\sin^2{x}} = \pi \log{2} $$

which is indeed true, as the integral may be shown, by integrating by parts, to be

$$-2 \int_0^{\pi/2} dx \log{\sin{x}} = -\int_0^{\pi} dx \log{\sin{x}}$$

which is indeed $\pi \log{2}$.

ADDENDUM

This last integral may in fact be done by elementary methods. For example, consider that

$$\begin{align}\int_0^{\pi} dx \log{\sin{x}} &= 2 \int_0^{\pi/2} dx \, \log{\sin{2 x}} \\ &= 2 \int_0^{\pi/2} dx \, \log{(2 \sin{x} \cos{x})} \\ &= \pi \log{2} + 2 \int_0^{\pi/2} dx \, \log{\sin{x}} + 2 \int_0^{\pi/2} dx \, \log{\cos{x}} \\&= \pi \log{2} + 4 \int_0^{\pi/2} dx \, \log{\sin{x}} \\&= \pi \log{2} + 2 \int_0^{\pi} dx \, \log{\sin{x}}\end{align}$$

A simple rearrangement proves the result.