What is a "vanishing moment"?

The basic idea is that a wavelet has $p$ vanishing moments if and only if the wavelet scaling function can generate polynomials up to degree $p-1$. The "vanishing" part means that the wavelet coefficients are zero for polynomials of degree at most $p-1$, that is, the scaling function alone can be used to represent such functions. More vanishing moments means that the scaling function can represent more complex functions. Loosely, you can think of it as

$$\textrm{more vanishing moments } \rightarrow \textrm{ complex functions can be represented with a sparser set of wavelet coefficients.}$$

The "moments" part comes from the fact that this is all equivalent to saying that the first $p$ derivatives of the Fourier transform of the wavelet filter all are zero when evaluated at 0. This is perfectly analogous to the probabilistic idea of a "moment generating function" of a random variable, which is basically the Fourier transform, and the $n$-th derivative evaluated at zero gives the $n$-th moment of the variable (i.e. the expected value, the expected value of the square, of the cube, etc.) So these Fourier transform derivative-zeros correspond to integrals back in the time/space domain that must be zero for the wavelet. In a sense, these conditions mean that the wavelet is "unbiased." It doesn't skew the function that is being transformed because the wavelet itself has no expected effect on a function until that function has a non-trivial $p+1$ order derivative.

Added: Section 5.2.1 at this link shows the integrals that I'm referring to and, I think, does a good job illustrating why you might refer to this as a "decay toward infinity" kind of property.

The n-th moment of a function is equal to the n-th derivative of its Fourier transform at zero frequency. So higher the number of zero moments, higher the number of zero derivatives and smoother the signal decays from mid frequency to DC in the frequency domain.