Integration of $\int\frac{x^2-1}{\sqrt{x^4+1}} \, dx$

For any real number of $x$ ,

When $|x|\leq1$ ,

$\int\dfrac{x^2-1}{\sqrt{x^4+1}}dx$

$=\int(x^2-1)\sum\limits_{n=0}^\infty\dfrac{(-1)^n(2n)!x^{4n}}{4^n(n!)^2}dx$

$=\int\sum\limits_{n=0}^\infty\dfrac{(-1)^n(2n)!x^{4n+2}}{4^n(n!)^2}dx-\int\sum\limits_{n=0}^\infty\dfrac{(-1)^n(2n)!x^{4n}}{4^n(n!)^2}dx$

$=\sum\limits_{n=0}^\infty\dfrac{(-1)^n(2n)!x^{4n+3}}{4^n(n!)^2(4n+3)}-\sum\limits_{n=0}^\infty\dfrac{(-1)^n(2n)!x^{4n+1}}{4^n(n!)^2(4n+1)}+C$

When $|x|\geq1$ ,

$\int\dfrac{x^2-1}{\sqrt{x^4+1}}dx$

$=\int\dfrac{x^2-1}{x^2\sqrt{1+\dfrac{1}{x^4}}}dx$

$=\int\left(1-\dfrac{1}{x^2}\right)\sum\limits_{n=0}^\infty\dfrac{(-1)^n(2n)!x^{-4n}}{4^n(n!)^2}dx$

$=\int\sum\limits_{n=0}^\infty\dfrac{(-1)^n(2n)!x^{-4n}}{4^n(n!)^2}dx-\int\sum\limits_{n=0}^\infty\dfrac{(-1)^n(2n)!x^{-4n-2}}{4^n(n!)^2}dx$

$=\sum\limits_{n=0}^\infty\dfrac{(-1)^n(2n)!x^{1-4n}}{4^n(n!)^2(1-4n)}-\sum\limits_{n=0}^\infty\dfrac{(-1)^n(2n)!x^{-4n-1}}{4^n(n!)^2(-4n-1)}+C$

$=\sum\limits_{n=0}^\infty\dfrac{(-1)^n(2n)!}{4^n(n!)^2(4n+1)x^{4n+1}}-\sum\limits_{n=0}^\infty\dfrac{(-1)^n(2n)!}{4^n(n!)^2(4n-1)x^{4n-1}}+C$

As per the solution I derived here:

\begin{align} I(x,a,k,n,m) &=\int_0^x \frac{t^k}{\left(t^n + a\right)^m}\:dt\\ & = \frac{1}{n}a^{\frac{k + 1}{n} - m} \left[B\left(m - \frac{k + 1}{n}, \frac{k + 1}{n}\right) - B\left(m - \frac{k + 1}{n}, \frac{k + 1}{n}, \frac{1}{1 + ax^n} \right)\right] \end{align}

We position your integral as:

\begin{align} I &= \int \frac{x^2 - 1}{\sqrt{x^4 + 1}}\:dx = \int_{0}^{x} \frac{t^2}{\sqrt{t^4 + 1}}\:dt = \int_{0}^{x} \frac{t^2 - 1}{\sqrt{t^4 + 1}}\:dt - \int_{0}^{x} \frac{1}{\sqrt{t^4 + 1}}\:dt \\ &= I\left(x,1,2,4, \frac{1}{2}\right) - I\left(x,1,0,4, \frac{1}{2}\right) \\ &= \frac{1}{4}1^{\frac{2 + 1}{4} - \frac{1}{2}} \left[B\left(\frac{1}{2} - \frac{2 + 1}{4}, \frac{2 + 1}{4}\right) - B\left(\frac{1}{2} - \frac{2 + 1}{4}, \frac{2 + 1}{4}, \frac{1}{1 + 1\cdot x^4} \right)\right] - \frac{1}{4}1^{\frac{0 + 1}{4} - \frac{1}{2}} \left[B\left(\frac{1}{2} - \frac{0 + 1}{4}, \frac{0 + 1}{4}\right) - B\left(\frac{1}{2} - \frac{0 + 1}{4}, \frac{0 + 1}{4}, \frac{1}{1 + 1\cdot x^4} \right)\right] \\ &= \frac{1}{4}\left[B\left(-\frac{1}{4}, \frac{3}{4}\right) - B\left(-\frac{1}{4}, \frac{3}{4}, \frac{1}{1 + x^4}\right) \right] - \frac{1}{4}\left[B\left(\frac{1}{4}, \frac{1}{4}\right) - B\left(\frac{1}{4}, \frac{1}{4}, \frac{1}{1 + x^4}\right) \right] \end{align}

Using the relationship between the Beta and Gamma Functions, we have:

\begin{align} B\left(-\frac{1}{4}, \frac{3}{4}\right) &=\frac{\Gamma\left(-\frac{1}{4}\right)\Gamma\left(\frac{3}{4}\right)}{\Gamma\left(-\frac{1}{4} + \frac{3}{4}\right)} & B\left(\frac{1}{4}, \frac{1}{4}\right)&=\frac{\Gamma\left(\frac{1}{4}\right)\Gamma\left(\frac{1}{4}\right)}{\Gamma\left(\frac{1}{4} + \frac{1}{4}\right)} \\ &= \frac{\Gamma\left(-\frac{1}{4}\right)\Gamma\left(\frac{3}{4}\right)}{\Gamma\left(\frac{1}{2}\right)} & &= \frac{\Gamma\left(\frac{1}{4}\right)\Gamma\left(\frac{3}{4}\right)}{\Gamma\left(\frac{1}{2} \right)} \\ &= \frac{\Gamma\left(-\frac{1}{4}\right)\Gamma\left(\frac{3}{4}\right)}{\sqrt{\pi}} & &= \frac{\Gamma\left(\frac{1}{4}\right)\Gamma\left(\frac{3}{4}\right)}{\sqrt{\pi}} \end{align}

Thus:

\begin{align} I &= \frac{1}{4}\left[B\left(-\frac{1}{4}, \frac{3}{4}\right) - B\left(-\frac{1}{4}, \frac{3}{4}, \frac{1}{1 + x^4}\right) \right] - \frac{1}{4}\left[B\left(\frac{1}{4}, \frac{1}{4}\right) - B\left(\frac{1}{4}, \frac{1}{4}, \frac{1}{1 + x^4}\right) \right] \\ &= \frac{1}{4}\left[B\left(-\frac{1}{4}, \frac{3}{4}\right) - B\left(\frac{1}{4}, \frac{1}{4}\right) \right] + \frac{1}{4}\left[B\left(\frac{1}{4}, \frac{1}{4}, \frac{1}{1 + x^4}\right) - B\left(-\frac{1}{4}, \frac{3}{4}, \frac{1}{1 + x^4}\right) \right] \\ &= \frac{\Gamma\left(\frac{3}{4}\right)}{4\sqrt{\pi}}\left[ \Gamma\left(-\frac{1}{4}\right) - \Gamma\left(\frac{1}{4}\right)\right]+ \frac{1}{4}\left[B\left(\frac{1}{4}, \frac{1}{4}, \frac{1}{1 + x^4}\right) - B\left(-\frac{1}{4}, \frac{3}{4}, \frac{1}{1 + x^4}\right) \right] \end{align}