$\int_{-\infty}^{\infty} \frac{\sin(x)}{x} dx $ with integration by parts?

The issue is the discontinuity at $0$ in the cosine terms when you perform the partial integration, which is overlooked if you integrate over $\mathbb{R}$. To get around this, consider:

$$\int_{-\infty}^{\infty}\frac{\sin x}{x}dx=2\int_0^{\infty}\frac{\sin x}{x}dx$$

By parts:

$$\int_{0}^{\infty} \frac{\sin(x)}{x} dx = \left.-\frac{\cos(x)}{x} \right|_{0}^{\infty} - \int_{0}^{\infty} \frac{\cos(x)}{x^2} dx$$

And here, both terms on the RHS diverge. No contradiction.

The definite integration by parts formula

$$\int_a^b f(x) g(x)\ dx = F(x) g(x)\bigg|_a^b - \int_a^b F(x) g'(x)\ dx$$

is justified by the product rule for derivative and the fundamental theorem of calculus

$$\int_a^b (F g)'(x)\ dx = F(x) g(x) \bigg|_a^b $$

But the "fine print" there is that $F(x) g(x)$ is assumed to be continuously differentiable on the interval $(a,b)$ and continuous on $[a,b]$. In this case with $F(x) = -\cos(x)$ and $g(x) = 1/x$, $F(x) g(x)$ is undefined at $x=0$, and no definition of it there will make it continuous at $x=0$, let alone differentiable. So you can't use the integration by parts formula for this integral on any interval containing $0$.