Singular Distribution

It turns out that there are two common definitions for singular distribution; see this article (Singular distribution - Springer Online Reference Works).

According to one definition, it is a probability distribution on $\mathbb{R}^n$ concentrated on a set of Lebesgue measure zero and giving probability zero to every one-point set. This is the case in the Wikipedia article you linked. Sometimes, a singular distribution is defined without the latter requirement; under this definition every discrete distribution is singular (with respect to Lebesgue measure). In any case, a singular distribution does not have a probability density function. (Distributions with probability density functions are called absolutely continuous.) As for the case of a multivariate Gaussian distribution, if the covariance matrix is singular, then the distribution is continuous singular (hence has no density, and gives probability zero to every one-point set).

I find only the expression "this Gaussian is singular" on page 14 of your reference, but not the definition of "singular distribution".

But to answer your question:

The delta distribution is not a singular distribution, it is a discrete probability distribution. It does not have a Radon-Nikodym density with respect to the Lesbegue measure, because the Lesbegue measure of a single point is zero, and the delta distribution is concentrated on a single point.

Don't get confused if people write stuff like $$ \int_{\mathbb{R}} \delta_0(x) d x = 1 $$ This is not correct in the strict sense. Instead, the "density function" of the delta distribution concentrated on zero - which is not a density in the sense of Radon-Nikodym - would be $$ f(x) = 0 \; \text{for} \; x \neq 0 $$ and $$ f(0) = \infty $$ and therefore we would have $$ \int_{\mathbb{R}} f(x) d x = 0 $$

But: For a discrete probability distribution, it is possible to name an at most countable set of points such that each point can be assigned a finite probability, such that the probability of any set is equal to the sum of the probabilities of the points it does contain.

This is not possible for a singular probability distribution like the Cantor distribution. The Cantor distribution is not concentrated on a countable set of points. Therefore the terms "singular distribution" and "discrete probability distribution" are different, and the delta distribution is a discrete one, not a singular one.