If a measure only assumes values 0 or 1, is it a Dirac's delta?

As delfonics says, the answer is: yes, if and only if there does not exist a measurable cardinal, and an outline of the proof was given by hot_queen in an answer to a question of mine from 2014 (Consistency strength of 0-1 valued Borel measures), which I somehow completely forgot about. In the meantime, I had written out the argument below, which essentially fills in the details in hot_queen's argument.

One direction is easy. Suppose $\kappa$ is a measurable cardinal, so that there exists a $\kappa$-additive (in particular, countably additive) $\mu : 2^{\kappa} \to \{0,1\}$ which is not a Dirac mass. Let $M$ be $\kappa$ equipped with the discrete metric (or for that matter, any other metric that $\kappa$ might happen to admit). Then $\mu$ (or its restriction to the Borel sets, if we are using a non-discrete metric) is the desired counterexample.

The other direction I found in the paper [1]. The argument, for the current case, goes as follows.

Working in ZFC, suppose there is a metric space $M$ and a 0-1 valued Borel measure $\mu$ which is not a point mass. Using the axiom of choice, $M$ is in one-to-one correspondence with some cardinal $\kappa$, so we can write $M = \{x_\alpha : \alpha \in \kappa\}$. (In other words, we have well-ordered $M$.) Note each $\alpha \in \kappa$ is itself an ordinal.

For every $\alpha$, we have $\mu(\{x_\alpha\})=0$, and $\{x\}$ is a decreasing intersection of the open balls $B(x_\alpha,1/n)$. So by continuity from above, there is an open ball $B_\alpha$ centered at $x_\alpha$ with $\mu(B_\alpha) = 0$. Set $H_\alpha = B_\alpha \cap \left( \bigcup_{\beta \in \alpha} B_\beta\right)^c$. (Some of the $H_\alpha$ might be empty but that is okay.) Note that $H_\alpha$ is the intersection of an open (hence $F_\sigma$) set and a closed set, so $H_\alpha$ is Borel (indeed $F_\sigma$). And since $H_\alpha \subset B_\alpha$ we have $\mu(H_\alpha) = 0$. By construction, the $H_\alpha$ are pairwise disjoint, and $\bigcup_{\alpha \in \kappa} H_\alpha = M$. Now they cite a result of D. Montgomery [2] which asserts that any arbitrary union of the $H_\alpha$ is in fact an $F_\sigma$; in particular it is Borel.

Now define a measure $\nu : 2^{\kappa} \to \{0,1\}$ by $\nu(Y) = \mu\left(\bigcup_{\alpha \in Y} H_\alpha\right)$. Since $\mu$ is countably additive and the $H_\alpha$ are disjoint, we have that $\nu$ is countably additive. Moreover, for any $\alpha$ we have $\nu(\{\alpha\}) = \mu(H_\alpha) = 0$, and $\nu(\kappa) = \mu(M) = 1$. Thus $\kappa$ admits a nontrivial countably additive 0-1 valued measure on all its subsets.

A measurable cardinal $\lambda$ has to have a measure which is not only countably additive but actually $\lambda$-additive, so to finish, we use an argument due to Ulam, mentioned on the Wikipedia page above. Since we have shown there is a cardinal (namely $\kappa$) with a nontrivial countably additive 0-1 valued measure on all its subsets, there is a minimal cardinal with this property; call it $\kappa_1$ and let $\nu_1 : 2^{\kappa_1} \to \{0,1\}$ be the corresponding countably additive measure.

Suppose $\nu_1$ were not $\kappa_1$-additive; that means there is a collection $\mathcal{C} \subset 2^{\kappa_1}$, having $\kappa_0 := |\mathcal{C}| < \kappa_1$, such that $\mathcal{C}$ consists of pairwise disjoint sets of $\nu_1$-measure zero, and yet $\nu_1\left(\bigcup \mathcal{F}\right) = 1$. Fix a bijection $\phi : \kappa_0 \to \mathcal{C}$ and define a measure $\nu_0 : 2^{\kappa_0} \to \{0,1\}$ by $\nu_0(B) = \nu_1\left(\bigcup_{\beta \in B} \phi(\beta)\right)$. Then $\nu_0$ is countably additive; for any $\beta \in \kappa_0$ we have $\nu_0(\{\beta\}) = \nu_1(\phi(\beta)) = 0$ since every set in $\mathcal{C}$ had measure zero; and $\nu_0(\kappa_0) = \nu_1(\bigcup \mathcal{F}) = 1$. So $\nu_0$ is nontrivial, and this contradicts the minimality of $\kappa_1$.

We have thus shown that $\kappa_1$ is a measurable cardinal.

[1] Marczewski, E.; Sikorski, R. Measures in non-separable metric spaces. Colloquium Math. 1, (1948). 133–139. MR 25548

[2] Montgomery, D. Non-separable metric spaces. Fundamenta Mathematicae 25, (1935). 527–533.

Note: I wasn't able to find a copy of Montgomery's paper online, and it doesn't seem to be indexed in MathSciNet. If someone has a copy of this paper, or knows where to find another proof of the result, I would be interested to hear it. I found a number of other references mentioning this result, so it seems to be fairly well established.

The answer is yes iff there is no measurable cardinal. For the less trivial direction, see this.