An improper integral : $\int_{0}^\infty {\ln(a^2+x^2)\over{b^2+x^2}}dx$

Let us assume for definiteness that $a> b>0$ and use parity to write the integral as $$I=\frac12\int_{-\infty}^{\infty}\frac{\ln(a^2+x^2)}{b^2+x^2}dx.$$ In the complex $x$-plane, the integral has two poles $x=\pm i b$ and two logarithmic branch points $x=\pm ia$. We introduce two branch cuts running from these points to $\pm i\infty$, and deform the contour of integration trying to pull it to e.g. $i\infty$. The result will be determined by two contributions:

• the residue at $x=ib$, equal to $$\frac12\cdot 2\pi i\cdot \frac{\ln(a^2-b^2)}{2ib}=\frac{\pi}{2b}\ln(a^2-b^2),$$

• the jump on the logarithmic branch cut emanating from $x=ia$, producing

$$-\frac12\cdot 2\pi \int_{0}^{\infty}\frac{ds}{(a+s)^2-b^2}=-\frac{\pi}{2b}\ln\frac{a-b}{a+b}.$$ The sum of the two contributions gives $$I=\frac{\pi}{b}\ln(a+b).$$

Another approach using contour integration is to consider the function $$ f(z) = \frac{\ln(z+ia)}{b^{2}+z^{2}} \, , \quad a, b >0.$$

Since the branch point of $f(z)$ is in the lower half-plane, we can integrate $f(z)$ around a contour consisting of the line segment $[-R,R]$ and the upper half of the circle $|z|=R$.

Letting $R \to \infty$, the integral vanishes along the upper half of the circle $|z|=R$.

So we have

$$ \begin{align} \int_{0}^{\infty} \frac{\ln(a^{2}+x^{2})}{b^{2}+x^{2}} \, dx &= \text{Re} \int_{-\infty}^{\infty} \frac{\ln(x+ia)}{b^{2}+x^{2} } \, dx \\ &= \text{Re}\, \left( 2\pi i \ \text{Res}[f(z),ib] \right) \\ &= \frac{\pi}{b} \, \ln(a+b) . \end{align}$$

Based on your calculations we have

$$f(a)={\pi\over b}\ln(b+a)+C\implies C=f(0)-{\pi\over b}\ln(b).$$

So, we need to find $f(0)$ which can be found using the original integral as

$$f(0)= 2\int_{0}^{\infty} \frac{\ln(x)}{b^2+x^2}dx.$$

To evaluate the last integral see here.