How to evaluate the closed form for $\int_{0}^{\pi/2}{\mathrm dx\over \sin^2(x)}\ln\left({a+b\sin^2(x)\over a-b\sin^2(x)}\right)=\pi\cdot F(a,b)?$

Let $u=\cot{x}$. Then $du = dx/\sin^2{x}$ and $\sin^2{x}=1/(1+u^2)$, and the limits become $\infty$ and $0$, so the integral becomes $$ \int_0^{\infty} \log{\left( \frac{a+b/(1+u^2)}{a-b/(1+u^2)} \right)} \, du = \int_0^{\infty} \log{\left( \frac{a+b+au^2}{a-b+au^2} \right)} du $$ One can now integrate this by parts to get a couple of easy rational integrals, although we should be a bit careful about the upper limit: $$ \int_0^{M} \log{\left( \frac{a+b+au^2}{a-b+au^2} \right)} du = \left[ u\log{\left( \frac{a+b+au^2}{a-b+au^2} \right)} \right]_0^M - \int_0^M u \left( \frac{2au}{a+b+au^2}-\frac{2au}{a-b+au^2} \right) du $$ The boundary term disappears: the bottom limit is clear, and the top limit follows because $$ M\log{(a+(a+b)M^{-2})} - M\log{(a+(a-b)M^{-2})} \sim \frac{2b}{aM} \to 0 $$ as $M \to \infty$. The remaining integral is just a rational function; some partial fractions trickery lets us rewrite it as $$ \int_0^M \left( \frac{2(a+b)}{a+b+au^2}-\frac{2(a-b)}{a-b+au^2} \right) du, $$ which we can then use the arctangent integral on to find the final answer $$ \left( \sqrt{1+\frac{b}{a}} - \sqrt{1-\frac{b}{a}} \right)\pi $$ or $$ \frac{\sqrt{a+b}-\sqrt{a-b}}{\sqrt{a}}\pi $$ or $$ \frac{2b\pi}{\sqrt{a(a+b)}+\sqrt{a(a-b)}}, $$ depending on preference.

You may like this method. Let $$I(b)=\int_{0}^{\pi/2}{\mathrm dx\over \sin^2(x)}\ln\left({a+b\sin^2(x)\over a-b\sin^2(x)}\right) $$ and then \begin{eqnarray} I'(b)&=&\int_{0}^{\pi/2}\bigg(\frac1{a+b\sin^2(x)}+\frac1{a-b\sin^2(x)}\bigg) \mathrm dx\\ &=&\frac{\pi}{2\sqrt a}(\frac1{\sqrt{a+b}}+\frac{1}{\sqrt{a-b}}). \end{eqnarray} So $$ I(b)=\frac{\pi}{2\sqrt{a}}\int_0^b(\frac1{\sqrt{a+t}}+\frac{1}{\sqrt{a-t}})dt=\pi\frac{\sqrt{a+b}-\sqrt{a-b}}{\sqrt a}$$ and hence $$　F(a,b)=\frac{\sqrt{a+b}-\sqrt{a-b}}{\sqrt a}. $$

The closed form you are looking for is $$ F(a,b) = \sqrt{\frac{a+b}{a}}-\sqrt{\frac{a-b}{a}} $$ This was hidden because for $F(2,1)$ $$ \left(\sqrt{\frac32}-\sqrt{\frac12}\right)^2 = 2-\sqrt{3} $$ so the answer $\sqrt{2-\sqrt{3}}$ is correct, but it is not in that uniform form.