What is the purpose of : in math mode?

The : is a relational symbol in TeX. In the TeXbook it is used in conjunction with =.

Plain \TeX\ treats the four characters |=|, |<|, |>|, and |:|\ as
``^{relations}'' because they express a relationship between two
quantities. For example, `${x<y}$' means that $x$~is less than~$y$.
Such relationships have a rather different meaning from binary
operations like $+$, and the symbols are typeset somewhat differently:
\beginmathdemo
|$x=y>z$|&x=y>z\cr
|$x:=y$|&x:=y\cr
|$x\le y\ne z$|&x\le y\ne z\cr
|$x\sim y\simeq z$|&x\sim y\simeq z\cr
|$x\equiv y\not\equiv z$|&x\equiv y\not\equiv z\cr
|$x\subset y\subseteq z$|&x\subset y\subseteq z\cr
\endmathdemo

In the comments, egreg pointed out another example in the TeXbook, where : is used to separate the condition from the set variable.

\beginmathdemo
|$\{\,x\mid x>5\,\}$|&\{\,x\mid x>5\,\}\cr
|$\{\,x:x>5\,\}$|&\{\,x:x>5\,\}\cr
\endmathdemo
(Some authors prefer to use a ^{colon} instead of `$\mid$', as in the second
example here.)

The colon has the exact same meaning in Plain TeX and LaTeX, as can be verified using this simple document:

\showthe\mathcode`:
\csname @@end\endcsname
\bye

The output is

$ pdftex test.tex
This is pdfTeX, Version 3.14159265-2.6-1.40.20 (TeX Live 2019) (preloaded format=pdftex)
 restricted \write18 enabled.
entering extended mode
(./test.tex
> 12346.
l.1 \showthe\mathcode`:

? 
 )
No pages of output.
Transcript written on test.log.
$ pdflatex test.tex
This is pdfTeX, Version 3.14159265-2.6-1.40.20 (TeX Live 2019) (preloaded format=pdflatex)
 restricted \write18 enabled.
entering extended mode
(./test.tex
LaTeX2e <2018-12-01>
> 12346.
l.1 \showthe\mathcode`:

? 
 )
No pages of output.
Transcript written on test.log.

---

Here the same table rendered in LaTeX:

\documentclass{article}
\usepackage{array}
\usepackage{amsmath}
\usepackage{listings}
\lstset{basicstyle=\ttfamily}
\begin{document}
\lstMakeShortInline|
\begin{tabular}{l>{$}l<{$}}
|$x=y>z$|                 & x=y>z                 \\
|$x:=y$|                  & x:=y                  \\
|$x\le y\ne z$|           & x\le y\ne z           \\
|$x\sim y\simeq z$|       & x\sim y\simeq z       \\
|$x\equiv y\not\equiv z$| & x\equiv y\not\equiv z \\
|$x\subset y\subseteq z$| & x\subset y\subseteq z \\
\end{tabular}
\end{document}

Quite often, one sees : employed in place of a vertical bar (|) to denote conditioning on some event or condition. E.g., both $E(X : X>0)$ and $E(X\mid X>0)$ would express the "expectation of the random variable X conditional on this random variable taking on positive values".

By default, TeX treats both : and \mid -- but not | and \vert -- as "relational" operators and inserts an amount of whitespace around them that's also used for other relational operators such as <, =, and >.

For other, i.e., non-conditioning math uses of the colon symbol, one should employ \colon, \coloneq (or, depending on the package that's in use, \coloneqq, etc.