How do I show that $\int_0^\infty \frac{\sin(ax) \sin(bx)}{x^{2}} \, \mathrm dx = \pi \min(a,b)/2$

In several steps:

• Trigonometric relation $$\left( \frac{\sin ax}{x}\right)\left( \frac{\sin bx}{x}\right)=\frac{1-\cos(a+b)x}{2x^2}-\frac{1-\cos(a-b)x}{2x^2}$$
• Dirichlet integral $$\int_0^\infty \frac{\sin \alpha t}{t}dt=\frac{\pi}{2}\mathrm{sgn}(\alpha)$$
• Integration by parts $$\int_0^\infty\frac{1-\cos(\alpha)t}{t^2}=\alpha\int_0^\infty \frac{\sin \alpha t}{t}dt=\frac{\pi}{2}|\alpha|$$
• Combine $$\int_0^\infty \left( \frac{\sin ax}{x}\right)\left( \frac{\sin bx}{x}\right) \mathrm{d}x=\frac{\pi}{4}(|a+b|-|a-b|)=\frac{\pi}{2}\min(a,b)$$

One way is to to this by residues. Another way to integrate once by parts to get $$I=\int_0^{\infty}\frac{b\sin ax\cos bx+a\cos ax \sin bx}{x}dx,$$ then to use the formula $2\sin \alpha\cos\beta=\sin(\alpha+\beta)+\sin(\alpha-\beta)$ and the mentioned integral (note that your formula needs to be corrected on the left and on the right) $$\displaystyle \int_0^{\infty}\frac{\sin xy}{x}\,dx=\frac{\pi}{2}\mathrm{sgn}(y).$$ This gives \begin{align}I&=\frac{\pi}{4}\Bigl[b\,\mathrm{sign}(a+b)+b\,\mathrm{sign}(a-b)+a\,\mathrm{sign}(a+b)+a\,\mathrm{sign}(b-a)\Bigr]=\\ &=\frac{\pi}{4}\left(|a+b|-|a-b|\right), \end{align} For $a,b>0$ the last expression is obviously equal to $\pi\min\{a,b\}/2$.

A very easy way to see this is to use Parseval's theorem for Fourier transforms. In general, Parseval's theorem states that, for two functions $f$ and $g$, each having respective FTs $\hat{f}$ and $\hat{g}$, related by

$$\hat{f}(k) = \int_{-\infty}^{\infty} dx \, f(x) \, e^{i k x}$$

etc., then

$$\int_{-\infty}^{\infty} dx \, f(x) \bar{g}(x) = \frac{1}{2 \pi} \int_{-\infty}^{\infty} dk \, \hat{f}(k) \bar{\hat{g}}(k)$$

The FT of $\sin{(a x)}/x$ is given by

$$\int_{-\infty}^{\infty} dx \, \frac{\sin{a x}}{x} e^{i k x} = \begin{cases} \pi & |k| \lt a \\ 0 & |k| \gt a\end{cases}$$

Similarly,

$$\int_{-\infty}^{\infty} dx \, \frac{\sin{b x}}{x} e^{i k x} = \begin{cases} \pi & |k| \lt b \\ 0 & |k| \gt b\end{cases}$$

By Parseval, we take the integral of the product of the transforms, which is the product of two rectangles. The product is clearly nonzero over the smaller of the two widths, i.e. $2 \min\{a,b\}$. Thus,

$$\int_{-\infty}^{\infty} dx \, \frac{\sin{a x}}{x} \, \frac{\sin{b x}}{x} = \frac{\pi^2}{2 \pi} 2 \min\{a,b\} $$

Therefore

$$\int_{0}^{\infty} dx \, \frac{\sin{a x}}{x} \, \frac{\sin{b x}}{x} = \frac{\pi}{2} \min\{a,b\}$$