Integral $\int_0^\infty\frac{\operatorname{arccot}\left(\sqrt{x}-2\,\sqrt{x+1}\right)}{x+1}\mathrm dx$

Define the function $I(s)$ for $s > 0$ by

$$ I(s) = \int_{0}^{\infty} \frac{\operatorname{arccot}( \sqrt{x} - e^{s}\sqrt{x+1} )}{x+1} \, dx. $$

By observing that $I(\infty) = 0$, we have

$$ I(s) = -\int_{s}^{\infty} I'(t) \, dt. $$

Applying Leibniz's integral rule,

$$ I'(s) = \int_{0}^{\infty} \frac{e^{s}\sqrt{x+1}}{1 + (\sqrt{x} - e^{s}\sqrt{x+1} )^{2}} \, \frac{dx}{x+1}. $$

Now with the substitution $x = \tan^{2}\theta$,

\begin{align*} I'(s) &= \int_{0}^{\frac{\pi}{2}} \frac{\sin\theta}{\cosh s - \sin\theta} = \frac{1}{2} \int_{-\frac{\pi}{2}}^{\frac{\pi}{2}} \frac{\cos\theta}{\cosh s - \cos\theta}. \end{align*}

Then with the substitution $z = e^{i\theta}$, it follows that

\begin{align*} I'(s) &= \frac{i}{2} \int_{\Gamma} \frac{z^{2} + 1}{z^{2} - 2z \cosh s + 1} \frac{dz}{z}, \end{align*}

where the contour $\Gamma$ denotes the counter-clockwise semicircular arc joining from $-i$ to $i$. Note that we have $z^{2} - 2z \cosh s + 1 = 0$ if and only if $\cosh s = \cos \theta$ if and only if $z = e^{i\theta} = e^{\pm s}$. Thus deforming the contour to the straight line joining from $-i$ to $i$ with infinitesimal indent at the origin, we obtain

\begin{align*} I'(s) &= \frac{i}{2} \left\{ \operatorname{PV}\! \int_{-i}^{i} f(z) \, dz + 2\pi i \operatorname{Res} (f, e^{-s}) + \pi i \operatorname{Res} (f, 0) \right\}, \end{align*}

where $f$ denotes the integrand

$$ f(z) = \frac{z^{2} + 1}{z (z^{2} - 2z \cosh s + 1)}. $$

Proceeding the calculation,

\begin{align*} I'(s) &= \frac{i}{2} \left\{ i \operatorname{PV} \! \int_{-1}^{1} f(ix) \, dx + \pi i - 2 \pi i \coth s \right\} \\ &= - \frac{1}{2} \int_{0}^{1} \{ f(ix) + f(-ix) \} \, dx - \frac{\pi}{2} + \pi \coth s \\ &= \cosh s \int_{0}^{1} \frac{\frac{1}{2} (1 - x^{-2}) }{\{ \frac{1}{2}(x + x^{-1}) \}^{2} + \sinh^{2} s} \, dx - \frac{\pi}{2} + \pi \coth s. \end{align*}

Now with the substitution $u = \frac{1}{2} \{ x + x^{-1} \}$, it follows that

\begin{align*} I'(s) &= - \cosh s \int_{1}^{\infty} \frac{du}{u^{2} + \sinh^{2} s} - \frac{\pi}{2} + \pi \coth s \\ &= - \arctan(\sinh s) \coth s - \frac{\pi}{2} + \pi \coth s. \end{align*}

Finally, with the substitution $x = \sinh t$,

\begin{align*} I(s) = - \int_{s}^{\infty} I'(t) \, dt &= \int_{s}^{\infty} \left\{ \arctan(\sinh t) \coth t + \frac{\pi}{2} - \pi \coth t \right\} \, dt \\ &= \int_{\sinh s}^{\infty} \left( \frac{\arctan x}{x} + \frac{\pi}{2\sqrt{x^{2} + 1}} - \frac{\pi}{x} \right) \, dx. \end{align*}

Our problem corresponds to $s = \log 2$, or equivalently, $a := \sinh s = \frac{3}{4}$.

\begin{align*} &\int_{a}^{\infty} \left( \frac{\arctan x}{x} + \frac{\pi}{2\sqrt{x^{2} + 1}} - \frac{\pi}{x} \right) \, dx \\ &= - \int_{a}^{\infty} \frac{\arctan (1/x)}{x} \, dx + \frac{\pi}{2} \int_{a}^{\infty} \left( \frac{1}{\sqrt{x^{2} + 1}} - \frac{1}{x} \right) \, dx \\ &= - \int_{0}^{1/a} \frac{\arctan x}{x} \, dx + \frac{\pi}{2} \lim_{R\to\infty} \left[ \log\left( \frac{x + \sqrt{x^{2} + 1}}{x} \right) \right]_{a}^{R} \\ &= - \operatorname{Ti}\left(\frac{1}{a}\right) - \frac{\pi}{2} \log\left( \frac{a + \sqrt{a^{2} + 1}}{2a} \right), \end{align*}

where the function

$$ \operatorname{Ti}(x) = \frac{\operatorname{Li}_{2}(ix) - \operatorname{Li}_{2}(-ix)}{2i} $$

is the inverse tangent integral. Using the following simple identity

$$ \operatorname{Ti}(x) = \operatorname{Ti}\left(\frac{1}{x}\right) + \frac{\pi}{2}\log x, $$

it follows that

\begin{align*} \int_{a}^{\infty} \left( \frac{\arctan x}{x} + \frac{\pi}{2\sqrt{x^{2} + 1}} - \frac{\pi}{x} \right) \, dx = - \operatorname{Ti}(a) - \frac{\pi}{2} \log\left( \frac{a + \sqrt{a^{2} + 1}}{2a^{2}} \right). \end{align*}

Therefore, plugging $a = \frac{3}{4}$ gives

$$ \int_{0}^{\infty} \frac{\operatorname{arccot}( \sqrt{x} - 2\sqrt{x+1} )}{x+1} \, dx = \pi \log(3/4) - \operatorname{Ti}(3/4) $$

as Cleo pointed out without explanation. More generally, for $k > 1$

$$ \int_{0}^{\infty} \frac{\operatorname{arccot}( \sqrt{x} - k \sqrt{x+1} )}{x+1} \, dx = \frac{\pi}{2} \log \left( \frac{(k^{2} - 1)^{2}}{2k^{3}} \right) - \operatorname{Ti}\left( \frac{k^{2} - 1}{2k} \right). $$

By putting $x=\sinh(z)^2$ we have:

$$ I = \int_{0}^{+\infty}2\tanh z\operatorname{arccot}\left(\sinh z-2\cosh z\right)dz; $$

integrating by parts we have:

$$ I = \int_{0}^{+\infty} 2\frac{-1+2\tanh z}{5\cosh z-4\sinh x}\log\left(\cosh z\right)dz;$$

now putting $z=\log t$ we get:

$$ I = 4\int_{1}^{+\infty} \frac{t^2-3}{(t^2+1)(t^2+9)}\log\left(\frac{t+t^{-1}}{2}\right)dt.$$

By putting $y=\frac{2}{t+t^{-1}}$ we can split the integral in two parts, i.e.

$$ I_1 = 24\int_{0}^{1}\frac{\log x}{9+16x^2} dx,$$ $$ I_2 = -4\int_{0}^{1}\frac{(3-8x^2)\log x}{(9+16x^2)\sqrt{1-x^2}}dx.$$

By considering the Taylor series of $\frac{1}{9+16x^2}$ and integrating term-by-term we get $I_1=\Im\operatorname{Li}_2\frac{3i}{4}$. The really mysterious thing is that $I_2=\pi\log\frac{3}{4}$. Not so mysterious, anyway. We have that: $$\int_{0}^{1}\frac{\log x}{\sqrt{1-x^2}}=\int_{0}^{\pi/2}\log\cos x\, dx=-\frac{1}{2}\int_{0}^{+\infty}\frac{\log(1+t^2)}{1+t^2}=-\frac{\pi}{2}\log 2,$$ (through $x=\arctan t$) in virtue of the magic formula: $$\int_{\mathbb{R}}\frac{\log\left(A^2 x^2 + B^2\right)}{x^2+1}dx=2\pi\log(A+B),\tag{1}$$ that can be proved through standard complex-analytic techniques. So we have only to deal with: $$\int_{0}^{1}\frac{\log x}{(16x^2+9)\sqrt{1-x^2}}dx=\int_{0}^{\pi/2}\frac{\log\cos\theta}{16\cos^2\theta+9}d\theta,$$ or, by putting $\theta=\arctan t$, $$\int_{0}^{\infty}\frac{\log(t^2+1)}{25+9t^2}dt=\frac{1}{15}\int_{0}^{+\infty}\frac{\log(25t^2+9)-\log(9)}{t^2+1}dt,$$ and the magic formula tell us that the last expression is equal to $\frac{1}{15}\left(\pi\log 8-\pi\log 3\right)$, completing the proof.

$$\int_0^\infty\frac{\operatorname{arccot}\left(\sqrt{x}-2\,\sqrt{x+1}\right)}{x+1}dx=\pi\ln\frac34-\Im\operatorname{Li}_2\frac{3i}4$$