Free ornaments font

You can't miss adforn font. Use \usepackage{adforn} to access the glyphs. See manual of the package for the commands.

There are still other fonts including ornaments (not in The Comprehensive LaTeX Symbol List), for example, linux-libertine (libertine package).

fourier offers ornamental glyphs, specifically fourier-orns. A short example:

\documentclass{article}
\usepackage{fourier-orns}
\begin{document}
\Huge
\textxswup\textxswdown
\decoone\decotwo
\decothreeleft\decothreeright
\decofourleft\decofourright
\floweroneleft\floweroneright
\end{document}

There are more, have a look at the fourier-orns example pdf file.

There is also pgfornament. This is not a font - any more than pst-vectorian is a font. Rather, it consists of code for drawing ornaments using PGF.

While not on CTAN, it is readily available and, I think, quite popular because, unlike its pst- predecessor, it can be straightforwardly used with pdfTeX. It is available as pgfornament.zip.

A sampler:

\documentclass{article}
\usepackage{pgfornament}

\begin{document}
  \tikzset{
    pgfornamentstyle/.style={scale=.25}
  }
  \foreach \i in {1,...,89} {\expandafter\pgfornament\expandafter{\i}\ }
\end{document}