How can I integrate $\int\frac{e^{2x}-1}{\sqrt{e^{3x}+e^x} } \mathop{dx}$?

Take out $e^x$ from numerator and denominator as follows $$\int\dfrac{e^{2x}-1}{\sqrt{e^{3x}+e^x} } \ dx=\int\dfrac{e^x(e^{x}-e^{-x})}{\sqrt{e^{2x}(e^{x}+e^{-x})} } dx$$ $$=\int\dfrac{e^x(e^{x}-e^{-x})}{e^x\sqrt{e^{x}+e^{-x}} } dx$$ $$=\int\dfrac{(e^{x}-e^{-x})dx}{\sqrt{e^{x}+e^{-x}} } $$ $$=\int\dfrac{d(e^{x}+e^{-x})}{\sqrt{e^{x}+e^{-x}} } $$ $$=2\sqrt{e^{x}+e^{-x}}+C $$

I used the same steps you did follow but I stopped at $$I=\int \left(1-\cot ^2(\theta )\right) \sec (\theta )\sqrt{\tan (\theta )}\, d\theta$$ Rewrite it as $$I={\displaystyle\int}\dfrac{\cos^2\left(\theta\right)-\sin^2\left(\theta\right)}{\cos^\frac{3}{2}\left(\theta\right)\sin^\frac{3}{2}\left(\theta\right)}\,d\theta$$ Now $$u=\cos\left(\theta\right)\sin\left(\theta\right)\implies du=\cos^2\left(\theta\right)-\sin^2\left(\theta\right)\implies d\theta=\dfrac{du}{\cos^2\left(\theta\right)-\sin^2\left(\theta\right)}$$ $$I=\int\dfrac{du}{u^\frac{3}{2}}u==-\dfrac{2}{\sqrt{u}}+C$$ Back to $\theta$ $$I=\frac 2{\sqrt{\sin(\theta)\cos(\theta)}}=\frac {2\sqrt 2}{\sqrt{\sin(2\theta)}}+C$$

$$ \int\!\dfrac{e^{2x}-1}{\sqrt{e^{3x}+e^x}}\mathop{dx} =\int\!\dfrac{e^{x}-e^{-x}}{\sqrt{e^{x}+e^{-x}}}\mathop{dx} =\int\!\dfrac{2\sinh x}{\sqrt{2\cosh x}}\mathop{dx} =2\sqrt{2\cosh x} + C = 2\sqrt{e^{x}+e^{-x}} + C $$