How to evaluate $\int_0^\frac{1}{\sqrt2}\frac{dx}{(1+x^2)\sqrt{1-x^2}}$

hint: $$\frac{1}{1+\sin^2 x}=\frac{\sec^2 x}{1+2\tan^2 x}$$

We see that $$\frac1{1+\sin(x)^2}=\frac{\sec(x)^2}{1+2\tan(x)^2}$$ as was hinted by @E.H.E. We then preform the sub $u=\tan(x)$ to get $$\int_0^1 \frac{du}{1+2u^2}$$ Then we may in fact compute the integral $$I(x;a,b,c)=\int\frac{ dx}{ax^2+bx+c}=\int\frac{dx}{a(x+\frac{b}{2a})^2+g}$$ Here $g=c-\frac{b^2}{4a}$. If we assume that $4ac>b^2$, then we may make the substitution $x+\frac{b}{2a}=\sqrt{\frac{g}{a}}\tan u$ which gives $$I(x;a,b,c)=\sqrt{\frac{g}{a}}\int\frac{\sec^2u\, du}{g\tan^2u+g}$$ $$I(x;a,b,c)=\frac{u}{\sqrt{ag}}$$ $$I(x;a,b,c)=\frac2{\sqrt{4ac-b^2}}\arctan\frac{2ax+b}{\sqrt{4ac-b^2}}+C$$ And by noting that your integral is given by $I(1;2,0,1)-I(0;2,0,1)$ we have your integral at the value $$\frac{\arctan\sqrt2}{\sqrt2}\approx 0.675510858856$$

Hint

Divide numerator & denominator by $\sin^2\theta$

and set $\cot\theta =u$

Or divide numerator & denominator by $\cos^2\theta$

and set $\tan\theta=v$