What contour should be used to evaluate $\int_0^\infty \frac{\sqrt{t}}{1+t^2} dt$

Call $\displaystyle K=\int_0^\infty \frac{\sqrt{t}}{1+t^2} \mathrm dt$ and $\displaystyle J_\gamma=\int_\gamma\frac{\sqrt{t}}{1+t^2} \mathrm dt$. Basically, you made the same mistake twice.

(1) Evaluating $J_\gamma$ thanks to the residue theorem yields $$ J_\gamma=2\pi\mathrm i\cdot\left.\left(\frac{\mathrm e^{(\log t)/2}}{t+\mathrm i}\right)\right|_{t=\mathrm i}=\pi\mathrm e^{(\log \mathrm i)/2}=\pi\mathrm e^{\mathrm i\pi/4}=\pi(1+\mathrm i)/\sqrt2. $$ Here the mistake is to believe that (the principal value of) $\log\mathrm i$ is $\pi/2$ instead of $\mathrm i\pi/2$: consider that $\mathrm e^{\mathrm i\pi/2}=\cos(\pi/2)+\mathrm i\sin(\pi/2)=\mathrm i$ and that $\mathrm e^{\pi/2}$ is... well, a real number close to $4.81$.

(2) Evaluating $J_\gamma$ through an integral on the real line yields $$ \int_{-\infty}^\infty \frac{\sqrt{t}}{1+t^2} \mathrm dt=K+\int_{-\infty}^0 \frac{\sqrt{t}}{1+t^2} \mathrm dt=K+\int_0^{+\infty} \frac{\mathrm i\sqrt{t}}{1+t^2} \mathrm dt=(1+\mathrm i)\,K. $$ Here, the mistake is to believe that, for every positive real number $t$, $\sqrt{-t}$ is $\mathrm e^{\pi/2}\sqrt{t}$ instead of $\mathrm e^{\mathrm i\pi/2}\sqrt{t}=\mathrm i\sqrt{t}$.

Correcting steps (1) and (2), your reasoning yields the relation $$ (1+\mathrm i)K=\lim\limits_{R\to+\infty}J_{\gamma}=\pi(1+\mathrm i)/\sqrt2, $$ hence the (exact) value $K=\pi/\sqrt2$.

Note: A "purely real" road is available to compute $K$. To do that, first use in succession the changes of variables $u=\sqrt{t}$, $v=\pm u$, and $w=1/v$ to get $$ K=\int_0^\infty\frac{2u^2}{1+u^4}\mathrm du=\int_{-\infty}^\infty\frac{v^2}{1+v^4}\mathrm dv=\int_{-\infty}^\infty\frac{\mathrm dw}{1+w^4}. $$ Thus, $$ K=\frac12\int_{-\infty}^\infty\frac{x^2+1}{x^4+1}\mathrm dx. $$ The factorisation $x^4+1=(x^2+\sqrt2x+1)(x^2-\sqrt2x+1)$ implies that $$ \frac{x^2+1}{x^4+1}=\frac1{(\sqrt2x+1)^2+1}+\frac1{(\sqrt2x-1)^2+1}, $$ hence $K=\frac12(L_1+L_{-1})$ where, for every real number $a$, $$ L_a=\int_{-\infty}^\infty\frac{\mathrm dx}{(\sqrt2x+a)^2+1}. $$ Using the change of variable $u=\sqrt2x+a$, one sees that, for every real number $a$, $$ L_a=\frac1{\sqrt2}\int_{-\infty}^\infty\frac{\mathrm du}{u^2+1}=\frac{\pi}{\sqrt2}, $$ where the last equality is direct if one recognizes the density of a standard Cauchy distribution. This proves finally that $K=\pi/\sqrt2$.

Your contour would work if you took into account the behavior of $\sqrt{z}$: your larger integral is not twice the original, but $1+i$ times it.

More generally, your integral and others $\int_0^\infty {t^s\,dt \over 1+t^2}$ (and with other denominators...) can be integrated by the Hankel or keyhole contour: come along the positive reals from $+\infty$ to near $0$, go counter-clockwise around $0$ on a small circle, then back out to $+\infty$ along the positive reals. The trick is that with $s$ non-integral this gives $(1-e^{-2\pi i s})$ times the original integral. Then use residues at the zeros of the denominator.

What contour, do you ask? Well, the best contour is no contour at all! :-) All integrals of the

form $\displaystyle\int_0^\infty\frac{t^{^{n-1}}}{a^{^m}+t^{^m}}dt$ can be shown to equal $a^{^{n-m}}\cdot\dfrac\pi m\cdot\csc\bigg(n\cdot\dfrac\pi m\bigg),~$ where $\csc x=\dfrac1{\sin x}~,$

in the following manner: First, let $t=au$, then, after factoring $a^{^{n-m}}$ outside the integral, let

$x=\dfrac1{1+u^{^m}}~,~$ and recognize the expression of the beta function in the resulting integral.

Afterwards, apply Euler's reflection formula for the $\Gamma$ function, in order to arrive at the

aforementioned result. Identifying the various parameters, we have $I=\dfrac\pi{\sqrt2}$ . $\big($I'm posting

this in the answer section because it's too long for a comment $)$.