Weak convergence of probability measure

You got the idea. Let $F_n(t,\omega):=\frac 1n\sum_{j=1}^n\chi_{(-\infty,t]}(X_k(\omega))$, where $X_j$ are independent random variable of law $\mu$. By Glivenko-Cantelli theorem, also known as fundamental theorem of statistics, we know that for almost all $\omega\in\Omega$, we have $$\sup_{t\in\mathbb R}|F_n(t,\omega)-F(t)|\to 0.$$

Fix one of these $\omega$, and let $\mu_n$ the probability measure associated with the cumulative distribution function $F_n(t)=F_n(t,\omega)$. Since $F_n(t)\to F(t)$ at all points of continuity of $F$, we have that $\mu_n\to \mu$ weakly. Since $\mu$ is supported by the finite set $\{X_1(\omega),\ldots,X_n(\omega)\}$, we are done.

Note that the result we used is maybe not the most simple way, and that pointwise convergence is not enough to conclude (since the almost everywhere depend on $t$).

Sorry if this is not exactly an answer to your original question. But I get a little bit itchy when I see the way some probabilists treat weak convergence of measures...

The Riesz Representation Theorem states that when $X$ is locally compact Hausdorff, we have an isometry between the dual of $C_0(X)$ and the normed vector space of signed finite regular measures on the Borel sets of $X$. The norm of a measure is its total variation. In this context, weak convergence of measures is simply the weak-* convergence in $C_0(X)$.

The Banach-Alaoglu Theorem states that the closed unit ball is compact in the weak-* topology. It is very easy to show that the set of (positive) measures such that $0 \leq \mu(X) \leq 1$ is a closed (in the weak-* topology) subset of the unit ball, and therefore is also compact. Let's represent the set of those measures by $\mathcal{M}$. Notice that if $X$ is not compact, then the set of probability measures (on the Borel sets of $X$) is not compact in the weak-* topology. In fact, if we take a sequence $x_n \in X$ "convergin to $\infty$", then $$ \frac{1}{n} \sum_{j=1}^n \delta_{x_n} \rightarrow 0. $$

Since $\mathcal{M}$ is compact and convex, the Krein-Milman Theorem says that $\mathcal{M}$ is the weak-* closure of the convex combination of extremal points of $\mathcal{M}$. The extremal points are points that are not non-trivial convex combination of other points of $\mathcal{M}$. Notice that these are the measure $0$, and the Dirac deltas $\delta_x$. Therefore, the convex combinations of those are the measures $\mu_n$ with finite support. Now, it only remains to show that we can use $\frac{1}{\mu_n(X)} \mu_n$ instead of $\mu_n$, to conclude that any probability measure is the weak-* limit of probability measures with finite support.

I realize that this is quite complicated. But IMHO, concepts borrowed from functional analysis should be regarded as such. Of course, this is a matter of taste, but the definition using $F_n(t)$ is too artificial EVEN when compared to the definition using the weak-* topology on $C_0(X)$.