What is $\Delta\frac{1}{|\mathbf{x}|^2}$, as a distribution?

Taking Fourier transform, we have

$$ \left( \Delta \frac{1}{|x|^{2}} \right)^{\wedge}(\xi) = -|\xi|^{2} \left( \frac{1}{|x|^{2}} \right)^{\wedge}(\xi). $$

Indeed, for any $\varphi \in \mathcal{S}(\Bbb{R}^{3})$,

\begin{align*} \langle (\Delta |x|^{-2})^{\wedge}, \varphi \rangle &= \langle \Delta |\xi|^{-2}, \hat{\varphi}(\xi) \rangle = \langle |\xi|^{-2}, \Delta \hat{\varphi}(\xi) \rangle = \langle |\xi|^{-2}, - (|x|^{2}\varphi)^{\wedge}(\xi) \rangle \\ &= - \langle (|x|^{-2})^{\wedge}(\xi), |\xi|^{2}\varphi(\xi) \rangle = - \langle |\xi|^{2} (|x|^{-2})^{\wedge}(\xi), \varphi(\xi) \rangle. \end{align*}

But we have

\begin{align*} \left( \frac{1}{|x|^{2}} \right)^{\wedge}(\xi) &= \int_{\Bbb{R}^{3}} \frac{e^{-i\xi\cdot x}}{|x|^{2}} \, dx = \int_{0}^{\infty} \int_{S^{2}} e^{-i\xi\cdot r \omega} d\sigma_{\omega} dr \\ &= \int_{0}^{\infty} \int_{0}^{2\pi} \int_{0}^{\pi} e^{-i|\xi| r \cos\phi} \sin\phi \, d\phi d\theta dr \\ &= 2\pi \int_{0}^{\infty} \int_{-1}^{t} e^{i|\xi| r t}\, dt dr \qquad (t = -\cos\phi) \\ &= \frac{4\pi}{|\xi|} \int_{0}^{\infty} \frac{\sin |\xi|r}{r} \, dt = \frac{2\pi^{2}}{|\xi|}. \end{align*}

So it follows that

$$ \left( \Delta \frac{1}{|x|^{2}} \right)^{\wedge}(\xi) = -|\xi|^{2} \left( \frac{1}{|x|^{2}} \right)^{\wedge}(\xi) = -2\pi^{2} |\xi| $$

and hence

$$ \Delta \frac{1}{|x|^{2}} = -2\pi^{2} \sqrt{-\Delta} \delta_{0}. $$

The relevant distribution theory here was surely known long before L. Schwartz, but he certainly fully legitimized it, and a bit of further work by A. Grothendieck on holomorphic vector-valued functions (e.g., distribution-valued functions) makes the following viewpoint completely rigorous. Gelfand-Shilov's "Generalized Functions" volume I does such things, and many more complicated ones, at length.

For $\Re(s)>2$, on $\mathbb R^n$, $\Delta |x|^s = s(s+n-2)\cdot |x|^{s-2}$, noting that the right-hand side is locally integrable, hence, gives a distribution by integration-against-it.

This relation also gives the meromorphic continuation of the distribution $|x|^s$, by dividing through by $s(s+n-2)$ and replacing $s$ by $s+2$, and induction: $|x|^s=(\Delta |x|^{s+2})/(s+2)(s+n)$. Since $|x|^{s+2}$ is locally integrable for $\Re(s)>-n$, for $n>2$ there is in fact no pole at $s=-2$, and the first pole is at $s=-n$.

The first pole of that meromorphic family of distributions at $s=-n$ can be shown to be a constant multiple (depending on dimension $n$) of Dirac $\delta$ by applying $|x|^s$ to $e^{-\pi |x|^2}$ (for example) and looking at the numerical meromorphic continuation... and regularizing a general $f$ by $f-f(0)\cdot e^{-\pi |x|^2}$...

For $n=3$, the first pole is at $s=-3$, with residue a multiple of $\delta$. The relation above shows that there is no pole at $s=-4$ for $n=3$, so we'd imagine that the value is a constant multiple of $|x|^{-4}$. However this is not a locally integrable function! At this point, one might think of J. Hadamard's "finite part" trick, which is to integrate by parts and ignore the terms that blow up. In fact, as Riesz showed a few years later, this is the same outcome as by analytic continuation. So one has a choice about how to present that distribution. Yes, for $f$ vanishing sufficiently at $0$, it is just (a constant multiple of) integration against $|x|^{-4}$.