Extremely ugly integral $\int_{-\pi}^{\pi} \frac{\operatorname{sign}(x)\arctan(x^2)-|x|}{\sin^2(x)+|\cos(x)|}\,dx$

I can't take full credit for this, Wolfram can be very helpful when you know how to coax it along.

For the integral

$$\int{dx\over\sin^2x+\cos x},$$

let

\begin{align*} u &= \tan\left(\frac{x}{2}\right)\\ du &= \frac{1}{2}\sec^{2}\left(\frac{x}{2}\right)\,dx. \end{align*}

Then we have \begin{align*} \sin x &= \frac{2\sin x}{1 +\cos x}\cdot \frac{1+\cos x}{2} = \frac{2\tan^{2}\left(\frac{x}{2}\right)}{\sec^{2}\left(\frac{x}{2}\right)} = \frac{2\tan^{2}\left(\frac{x}{2}\right)}{\tan^{2}\left(\frac{x}{2}\right)+1}=\frac{2u^{2}}{u^{2}+1}\\ \cos x&= \frac{2\cos x}{1+\cos x}\cdot\frac{1+\cos x}{2}=\frac{1+\cos x - 1 + \cos x}{(1+\cos x)\sec^{2}\left(\frac{x}{2}\right)}=\frac{1-\frac{1-\cos x}{1+\cos x}}{\sec^{2} \left(\frac{x}{2}\right)}= \frac{1-u^{2}}{u^{2}+1}\\ dx &= \frac{2\,du}{u^{2}+1}. \end{align*}

The integral then becomes \begin{align*} 2\int \frac{du}{(u^{2}+1)\left[\frac{4u^{2}}{(u^{2}+1)^{2}}+\frac{1-u^{2}}{u^{2}+1}\right]} &=-2\int\frac{u^{2}+1}{u^{4}-4u^{2}-1}\,du\\ &=-2\int\frac{u^{2}+1}{\left(u^{2}-\sqrt{5}-2\right)\left(u^{2}+\sqrt{5}-2\right)}\,du. \end{align*}

We can then use a partial fraction decomposition, which I will omit, to change the integral to \begin{align*} -\int\frac{\sqrt{5}-3}{\sqrt{5}\left(u^{2}+\sqrt{5}-2\right)}\,du -\int\frac{-3-\sqrt{5}}{\sqrt{5}\left(-u^{2}+\sqrt{5}+2\right)}\,du, \end{align*}

which becomes \begin{align*}\frac{3-\sqrt{5}}{\sqrt{5}}\int\frac{du}{u^{2}+\sqrt{5}-2} + \frac{3+\sqrt{5}}{\sqrt{5}}\int\frac{du}{-u^{2} + \sqrt{5} +2}. \end{align*}

We can rewrite this as $$\frac{3-\sqrt{5}}{\sqrt{5}(\sqrt{5}-2)}\int\frac{du}{1+\left(\frac{u}{\sqrt{\sqrt{5}-2}}\right)^{2}}+\frac{3+\sqrt{5}}{\sqrt{5}(\sqrt{5}+2)}\int\frac{du}{1-\left(\frac{u}{\sqrt{\sqrt{5}+2}}\right)^{2}}. $$

This then becomes

$$\frac{3-\sqrt{5}}{\sqrt{5}\sqrt{\sqrt{5}-2}}\tan^{-1}\left(\frac{u}{\sqrt{\sqrt{5}-2}}\right)+\frac{3+\sqrt{5}}{\sqrt{5}\sqrt{\sqrt{5}+2}}\tanh^{-1}\left(\frac{u}{\sqrt{\sqrt{5}+2}}\right)+C, $$

or substituting back

$$\frac{3-\sqrt{5}}{\sqrt{5}\sqrt{\sqrt{5}-2}}\tan^{-1}\left(\frac{\tan\left(\frac{x}{2}\right)}{\sqrt{\sqrt{5}-2}}\right)+\frac{3+\sqrt{5}}{\sqrt{5}\sqrt{\sqrt{5}+2}}\tanh^{-1}\left(\frac{\tan\left(\frac{x}{2}\right)}{\sqrt{\sqrt{5}+2}}\right)+C.$$

Since this expression vanishes at $x=0$, your integral is given by

$$-2\pi\left(\frac{3-\sqrt{5}}{\sqrt{5}\sqrt{\sqrt{5}-2}}\tan^{-1}\left(\frac{\tan\left(\frac{\pi}{4}\right)}{\sqrt{\sqrt{5}-2}}\right)+\frac{3+\sqrt{5}}{\sqrt{5}\sqrt{\sqrt{5}+2}}\tanh^{-1}\left(\frac{\tan\left(\frac{\pi}{4}\right)}{\sqrt{\sqrt{5}+2}}\right)\right)\approx -8.734.$$

Noting $$ \int_0^{\pi/2}\frac{1}{\sin^2x+\cos x}dx=\int_0^{\pi/2}\frac{1}{\cos^2x+\sin x}dx=\int_0^{\pi/2}\frac{1}{1-\sin^2x+\sin x}dx $$ and using $u=\tan(\frac{x}{2})$, one has \begin{eqnarray} \int_0^{\pi/2}\frac{1}{1-\sin^2x+\sin x}dx&=&\int_0^{\pi/2}\frac{1}{1-(\frac{2\tan(\frac{x}{2})}{1+\tan^2(\frac{x}{2})})^2+\frac{2\tan(\frac{x}{2})}{1+\tan^2(\frac{x}{2})}}dx\\ &=&\int_0^1\frac{1}{1-(\frac{2u}{1+u^2})^2+\frac{2u}{1+u^2}}\frac{2du}{1+u^2}\\ &=&\int_0^1\frac{u^2+1}{u^4+2u^3-2u^2+2u+1}du\\ &=&\int_0^1\frac{u^2+1}{[(u-\phi)^2+\phi][(u+\frac1\phi)^2-\frac1\phi]}du \end{eqnarray} Here $\phi=\frac{-1+\sqrt5}{2}$. Noting $$ \frac{u^2+1}{[(u-\phi)^2+\phi][(u+1/\phi)^2-1/\phi]}=-\frac{\phi}{\sqrt5}\frac{1}{(u-\phi)^2+\phi}+\frac{1}{\sqrt5\phi}\frac{1}{(u+\frac1\phi)^2-\frac1\phi} $$ one has \begin{eqnarray} \int_0^{\pi/2}\frac{1}{1-\sin^2x+\sin x}dx &=&\int_0^1\frac{u^2+1}{[(u-\phi)^2+\phi][(u+\frac1\phi)^2-\frac1\phi]}du\\ &=&-\frac{\phi}{\sqrt5}\int_0^1\frac{1}{(u-\phi)^2+\phi}du+\frac{1}{\sqrt5\phi}\int_0^1\frac{1}{(u+\frac1\phi)^2-\frac1\phi}du\\ &=&-\frac{\sqrt\phi}{\sqrt5}\bigg[\arctan(\frac{u-\phi}{\sqrt\phi})-\text{arctanh}(\frac{1+u\phi}{\sqrt\phi})\bigg]\bigg|_0^1\\ &=&-\frac{\sqrt\phi}{\sqrt5}\bigg[\arctan(\frac{1-\phi}{\sqrt\phi})+\arctan\sqrt\phi-\text{arctanh}(\frac{1}{\phi\sqrt\phi})+\text{arctanh}\sqrt\phi\bigg]. \end{eqnarray}

You've done most part of the problem. I can't solve the last integral, but I believe I can provide an approximate solution. Let us take $f(x) = \sin^2 x + |cos x|$. The graphs of $|\cos x|$ and $\cos^2 x$ are pretty close. So $f(x)$ will always be quite close to $1$. Now, to find the maxima and minima values of the function, we need only look at one part where $\cos x$ is positive. To find the maxima and minima, then,

$ 2\sin x\cos x - \sin x = 0 \implies \sin x = 0 \text{ or } \cos x = \frac12. $

Hence, $\min f = 1$ and $\max f = \frac54$. We're interested in the integral of $g(x) = \frac{1}{f(x)}$. Clearly $g$ oscillates between $\frac45$ and $1$. The values being pretty close, let us approximate $g(x)$ as a constant function $g(x) = \frac12 \left( \frac45 + 1\right) = \frac9{10}$.

Thus, $I \approx -\pi \int_0^\pi \frac9{10} dx = -\frac{9}{10}\pi^2$.