Space of probability measures "complete"? (In the other sense)

For any measurable space $(X,\mathscr B)$ let's denote by $\mathscr M$ the linear space of all bounded (signed) measures. This is known to be a Banach space w.r.t. the total variation norm $$ \|\mu\| = \sup_{B\in \mathscr B}(|\mu(B)|+|\mu(B^c)|). $$ so that this space is complete. Define $f:\mathscr M\to\Bbb R$ as $f(\mu) = \mu(X)$. This map is clearly continuous: $$ |f(\mu) - f(\nu)|\leq \|\mu - \nu\| $$ and thus $\mathscr M_1:=\{\mu:f(\mu) = 1\}$ is a closed subspace of $\mathscr M$. The space of all non-negative measures $\mathscr M_+:=\{\mu:\mu\geq 0\}$ can be characterized as $$ \mathscr M_+ = \{\mu:\|\mu\| - f(\mu) = 0\} $$ and thus is a closed subspace of $\mathscr M$ as well. The space of all probability measures $\mathscr P = \mathscr M_1\cap\mathscr M_+$ is thus a closed subspace of a complete space, hence complete itself.

Besides of the total variation distance which can be introduced regardless the structure of the underlying measurable space, there are other sorts of metric spaces of measures. The relation of completeness for them are more involved.

This is related to Prokhorov's theorem and tightness. If you have a seperable metric space $(X,d)$ that is complete, then $(\mathcal{P},d_P)$ is complete, where $\mathcal{P}(X)$ is the set of all Borel probability measures and $d_P$ is the Prokhorov metric.

You can find a good detailed discussion of this here., section 9 pg 26.