Show that $I=J={\ln(7+4\sqrt{3})\over 4\sqrt3}?$

To prove that $I = J$ , note that

$$I-J = \int^{\pi/2}_0 \frac{\tan x-\tan^3 x}{\sqrt{(1+\tan^2 x)(1+\tan^6 x)}}\mathrm dx = \int^{\pi/4}_{-\pi/4} \frac{\tan (\pi/4-x)-\tan^3 (\pi/4-x)}{\sqrt{(1+\tan^2 (\pi/4-x))(1+\tan^6 (\pi/4-x))}}\mathrm dx$$

We need to prove the follwing function is odd

$$f(x) = \frac{\tan (\pi/4-x)-\tan^3 (\pi/4-x)}{\sqrt{(1+\tan^2 (\pi/4-x))(1+\tan^6 (\pi/4-x))}}$$

Now use that

$$\tan\left(\frac{\pi}{4} -x\right) = \frac{\cos x - \sin x}{\cos x + \sin x}$$

After expanding the denominator and nominator we realize

$$\tan (\pi/4-x)-\tan^3 (\pi/4-x)=\frac{-\cos(x) + \cos(3 x) + \sin(x) + \sin(3 x)}{(\cos x + \sin x)^3}$$ $$(1+\tan^2 (\pi/4-x))(1+\tan^6 (\pi/4-x))=\frac{10 - 6 \cos(4 x)}{(\cos x + \sin x )^8}$$

Hence

$$f(x) = \frac{(-\cos(x) + \cos(3 x) + \sin(x) + \sin(3 x))(\sin x+ \cos x)}{\sqrt{10 - 6 \cos(4 x)}}$$

Interestingly this simplifies to

$$f(x) = \frac{\sin(4x)}{\sqrt{10 - 6 \cos(4 x)}}$$

Hence $f(x)$ is odd which imlies

$$I-J = \int^{\pi/4}_{-\pi/4} f(x) \,dx = 0$$

Now using lab bhattacharjee result and $I=J$

$$4I=\int_0^{\infty}\dfrac{4du}{(3+2u-1)\sqrt{3+(2u-1)^2}} = {\log(7+4\sqrt{3})\over \sqrt3}$$

which implies

$$J=I= {\log(7+4\sqrt{3})\over 4\sqrt3}$$

HINT:

$$2(I+J)=\int_0^{\pi/2}\dfrac{2\tan x(1+\tan^2x)}{\sqrt{(1+\tan^2x)(1+\tan^6x)}}dx$$

Set $\tan^2x=u$ $$2(I+J)=\int_0^{\infty}\dfrac{du}{\sqrt{(1+u)(1+u^3)}}$$

$$=\int_0^{\infty}\dfrac{4du}{(3+2u-1)\sqrt{3+(2u-1)^2}}$$

Set $2u-1=\sqrt3\tan v$ $$2(I+J)=\int_{-\pi/6}^{\pi/2}\dfrac{2\sqrt3\sec v\ dv}{3+\sqrt3\tan v}=\int_{-\pi/6}^{\pi/2}\dfrac{2\ dv}{\sqrt3\cos v+\sin v}=\int_{-\pi/6}^{\pi/2}\csc(v+\pi/3)dv$$

On the path of Lab Bhattacharjee,

$J=\displaystyle \int_0^{+\infty} \dfrac{1}{\sqrt{(1+x)(1+x^3)}}dx$

Perform the change of variable $y=\dfrac{1-x}{1+x}$,

$\begin{align}\displaystyle J&=\int_{-1}^{1} \dfrac{1}{\sqrt{1+3x^2}}dx\\ \end{align}$

Perform the change of variable $y=\sqrt{3}x$,

$\begin{align} J&=\dfrac{1}{\sqrt{3}} \int_{-\sqrt{3}}^{\sqrt{3}}\dfrac{1}{\sqrt{1+x^2}}dx\\ &=\dfrac{1}{\sqrt{3}}\Big[\text{arcsinh}(x)\Big]_{-\sqrt{3}}^{+\sqrt{3}}\\ &=\dfrac{1}{\sqrt{3}}\Big[\ln\left(x+\sqrt{1+x^2}\right)\Big]_{-\sqrt{3}}^{+\sqrt{3}}\\ &=\dfrac{1}{\sqrt{3}}\ln\left(\dfrac{2+\sqrt{3}}{2-\sqrt{3}}\right)\\ &=\boxed{\dfrac{1}{\sqrt{3}}\ln\left(7+4\sqrt{3}\right)} \end{align}$