Real-Analysis Methods to Evaluate $\int_0^\infty \frac{x^a}{1+x^2}\,dx$, $|a|<1$.
I'll assume $\lvert a\rvert < 1$. Letting $x = \tan \theta$, we have
$$\int_0^\infty \frac{x^a}{1 + x^2}\, dx = \int_0^{\pi/2}\tan^a\theta\, d\theta = \int_0^{\pi/2} \sin^a\theta \cos^{-a}\theta\, d\theta$$
The last integral is half the beta integral $B((a + 1)/2, (1 - a)/2)$, Thus
$$\int_0^{\pi/2}\sin^a\theta\, \cos^{-a}\theta\, d\theta = \frac{1}{2}\frac{\Gamma\left(\frac{a+1}{2}\right)\Gamma\left(\frac{1-a}{2}\right)}{\Gamma\left(\frac{a+1}{2} + \frac{1-a}{2}\right)} = \frac{1}{2}\Gamma\left(\frac{a+1}{2}\right)\Gamma\left(\frac{1-a}{2}\right)$$
By Euler reflection,
$$\Gamma\left(\frac{a+1}{2}\right)\Gamma\left(\frac{1-a}{2}\right) = \pi \csc\left[\pi\left(\frac{1+a}{2}\right)\right] = \pi \sec\left(\frac{\pi a}{2}\right)$$
and the result follows.
Edit: For a proof of Euler reflection without contour integration, start with the integral function $f(x) = \int_0^\infty u^{x-1}(1 + u)^{-1}\, du$, and show that $f$ solves the differential equation $y''y - (y')^2 = y^4$, $y(1/2) = \pi$, $y'(1/2) = 0$. The solution is $\pi \csc \pi x$. On the other hand, $f(x)$ is the beta integral $B(1+x,1-x)$, which is equal to $\Gamma(x)\Gamma(1-x)$. I believe this method is due to Dedekind.
In the same spirit as the previously posted answers, we enforce the substitution $x\to\sqrt{\frac{1-x}{x}}$ to reveal
$$\begin{align} \int_0^\infty \frac{x^a}{1+x^2}\,dx&=\frac12\int_0^1 x^{(a-1)/2}(1-x)^{-(a+1)/2}\,dx\\\\ &=\frac12 B\left(\frac{1+a}{2},\frac{1-a}{2}\right) \end{align}$$
which after applying the relationships $B(x,y)=\frac{\Gamma(x)\Gamma(y)}{\Gamma(x+y)}$ and $\Gamma(z)\Gamma(1-z)=\pi \csc(\pi z)$ recovers the closed form $\frac{\pi}{2}\sec(\pi a/2)$.
NOTES:
In order to be more self-contained, I thought it might be useful to provide herein proofs of the mechanisms used in the evaluation of the integral of record. To that end, we proceed.
Relationship Between Beta and Gamma
Note that we can write the product $\Gamma(x)\Gamma(y)$ for $x>0$, $y>0$ as
$$\begin{align} \Gamma(x)\Gamma(y)&=\int_0^\infty s^{x-1}e^{-s}\,ds\,\int_0^\infty t^{x-1}e^{-t}\,dt\\\\ &=\int_0^\infty \, \int_0^\infty s^{x-1}t^{y-1}e^{-(s+t)}\,ds\,dt\\\\ &=\int_0^\infty t^{y-1} \int_t^\infty (s-t)^{x-1}e^{-s}\,ds\,dt\\\\ &=\int_0^\infty e^{-s}\int_0^s t^{y-1}(s-t)^{x-1}\,dt\,ds\\\\ &=\int_0^\infty s^{x+y-1}e^{-s} \int_0^1 t^{y-1}(1-t)^{x-1}\,dt\,ds\\\\ &=\Gamma(x+y)B(x,y) \end{align}$$
as was to be shown.
Limit Definition of Gamma
Let $G_n(x)$ be the sequence of functions given by
$$G_n(x)=\int_0^n s^{x-1}\left(1-\frac{s}{n}\right)^n\,ds$$
I showed in THIS ANSWER, using only Bernoulli's Inequality, that the sequence $\left(1-\frac{s}{n}\right)^n$ monotonically increases for $s\le n$. Therefore, $\left|s^{x-1} \left(1-\frac{s}{n}\right)^n\right|\le s^{x-1}e^{-s}$ for $s\le n$. The Dominated Convergence Theorem guarantees that we can write
$$\begin{align} \lim_{n\to \infty} G_n(x)=&\lim_{n\to \infty}\int_0^n s^{x-1}\left(1-\frac{s}{n}\right)^n\,ds\\\\ &=\lim_{n\to \infty}\int_0^\infty \xi_{[0,n]}\,s^{x-1}\left(1-\frac{s}{n}\right)^n\,ds\\\\ &=\int_0^\infty \lim_{n\to \infty} \left(\xi_{[0,n]}\,\left(1-\frac{s}{n}\right)^n\right)\,s^{x-1}\,\,ds\\\\ &=\int_0^\infty s^{x-1}e^{-s}\,ds\\\\ &=\Gamma(x) \end{align}$$
ALTERNATIVE PROOF: Limit Definition of Gamma
If one is unfamiliar with the Dominated Convergence Theorem, then we can simply show that
$$\lim_{n\to \infty}\int_0^n s^{x-1}e^{-s}\left(1-e^s\left(1-\frac{s}{n}\right)^n\right)=0$$
To do this, we appeal again to the analysis in THIS ANSWER. Proceeding, we have
$$\begin{align} 1-e^s\left(1-\frac{s}{n}\right)^n &\le 1-\left(1+\frac{s}{n}\right)^n\left(1-\frac{s}{n}\right)^n\\\\ &=1-\left(1-\frac{s^2}{n^2}\right)^n\\\\ &\le 1-\left(1-\frac{s^2}{n}\right)\\\\ &=\frac{s^2}{n} \end{align}$$
where Bernoulli's Inequality was used to arrive at the last inequality. Similarly, we see that
$$\begin{align} 1-e^s\left(1-\frac{s}{n}\right)^n &\ge 1-e^se^{-s}\\\\ &=0 \end{align}$$
Therefore, applying the squeeze theorem yields to coveted limit
$$\lim_{n\to \infty}\int_0^n s^{x-1}e^{-s}\left(1-e^s\left(1-\frac{s}{n}\right)^n\right)=0$$
which implies $\lim_{n\to \infty}G_n(x)=\Gamma(x)$.
Integrating by parts repeatedly the integral representation of $G_n(x)$ reveals
$$G_n(x)=\frac{n^x\,n!}{x(x+1)(x+2)\cdots (x+n)}$$
so that
$$\bbox[5px,border:2px solid #C0A000]{\Gamma(x)=\lim_{n\to \infty}\frac{n^x\,n!}{x(x+1)(x+2)\cdots (x+n)}}$$
Reflection Formula
Finally, we can write
$$\begin{align} \Gamma(x)\Gamma(1-x)&=\lim_{n\to \infty}\frac{n\,(n!)^2}{x(1-x^2)(4-x^2)\cdots (n^2-x^2)(n+1-x) }\\\\ &=\lim_{n\to \infty}\frac{1}{x\prod_{k=1}^n \left(1-\frac{x^2}{k^2}\right)}\\\\ &=\frac{\pi}{\sin(\pi x)} \end{align}$$
In arriving at the last equality, we used the infinite product representation of the sine function $\sin(\pi x)=\pi x\,\prod_{k=1}^n \left(1-\frac{x^2}{k^2}\right)$, which was proven in THIS ANSWER using real analysis.
APPENDIX:
Again, to be self-contained, we will show in this appendix that Equation $(1)$ of the OP is indeed the partial fraction representation of $(2)$.
We begin by expanding the function $\cos(ax)$ in a Fourier series,
$$\cos(xy)=a_0/2+\sum_{n=1}^\infty a_n\cos(nx) \tag{A1}$$
for $x\in [-\pi/\pi]$. The Fourier coefficients are given by
$$\begin{align} a_n&=\frac{2}{\pi}\int_0^\pi \cos(xy)\cos(nx)\,dx\\\\ &=\frac1\pi (-1)^n \sin(\pi y)\left(\frac{1}{y +n}+\frac{1}{y -n}\right)\tag {A2} \end{align}$$
Substituting $(A2)$ into $(A1)$, setting $x=0$, and dividing by $\sin(\pi y)$ reveals
$$\begin{align} \pi \csc(\pi y)&=\frac1y +\sum_{n=1}^\infty (-1)^n\left(\frac{1}{y -n}+\frac{1}{y +n}\right)\\\\ &=\sum_{n=-\infty}^\infty \frac{(-1)^n}{y-n}\tag {A3} \end{align}$$
Next, letting $y=(1+a)/2$ in $(A3)$,then letting $y=(1-a)/2$ in $(A3)$ we find after combining results and dividing by $2$
$$\begin{align} \pi \sec(\pi a/2)&=\sum_{n=-\infty}^\infty (-1)^n\left(\frac{1}{a-(2n-1)}-\frac{1}{a+(2n-1)}\right)\\\\ &=\sum_{n=1}^\infty (-1)^n\left(\frac{1}{a-(2n-1)}-\frac{1}{a+(2n-1)}\right)\\\\ &+\sum_{n=-\infty}^0 (-1)^n\left(\frac{1}{a-(2n-1)}-\frac{1}{a+(2n-1)}\right)\\\\ &=\sum_{n=0}^\infty (-1)^{n+1}\left(\frac{1}{a-(2n+1)}-\frac{1}{a+(2n+1)}\right)\\\\ &+\sum_{n=\infty}^{0} (-1)^n\left(\frac{1}{a-(-2n-1)}-\frac{1}{a+(-2n-1)}\right)\\\\ &=2\sum_{n=0}^\infty (-1)^n\left(\frac{1}{a-(-2n-1)}-\frac{1}{a+(-2n-1)}\right)\\\\ \end{align}$$
Finally, dividing by $2$ yields the coveted representation
$$\frac{\pi}{2}\sec\left(\frac{\pi a}{2}\right)=\sum_{n=0}^\infty (-1)^n\left(\frac{1}{a-(-2n-1)}-\frac{1}{a+(-2n-1)}\right)$$
as was to be shown!
Hint. Assume $|a|<1$. Another equivalent approach would be to write $$\begin{align} \int_0^\infty \frac{x^a}{1+x^2}\,dx&=\int_0^\infty x^a \left(\int_0^\infty e^{-(1+x^2)t}dt\right)dx \\\\&=\int_0^\infty e^{-t}\left(\int_0^\infty x^a e^{-tx^2}dx\right)dt\\\\ &=\frac12\Gamma\left(\frac{1+a}{2}\right)\int_0^\infty t^{\frac{1-a}{2}-1}e^{-t}dt\\\\ &=\frac12\Gamma\left(\frac{1+a}{2}\right)\Gamma\left(\frac{1-a}{2}\right)\\\\ &=\frac{\pi}{2}\sec\left(\frac{\pi a}{2}\right) \end{align}$$ by using the standard integral representation of the $\Gamma$ function and (6.1.30).