coloring the page numbers in the table of contents

This happens because the links are only the titles of entries in the ToC. The page numbers themselves are not hyperlinked. Add the option

linktoc=all

to your set of hyperref options, which should link both the section names as well as the page number.

\documentclass{book}
\usepackage{hyperref,xcolor}% http://ctan.org/pkg/{hyperref,xcolor}
\definecolor{wine-stain}{rgb}{0.5,0,0}
\hypersetup{
  colorlinks,
  linkcolor=wine-stain,
  linktoc=all
}
\begin{document}
\tableofcontents

\chapter{First chapter}
\section{First section}
\subsection{First subsection}
\subsubsection{First subsubsection}
\subsubsection{Second subsubsection}
\subsection{Second subsection}
\subsubsection{First subsubsection}
\subsubsection{Second subsubsection}

\section{Second section}
\subsection*{An unnumbered first subsection}
\subsubsection{First subsubsection}
\subsubsection{Second subsubsection}
\subsection*{An unnumbered second subsection}
\subsubsection{First subsubsection}
\subsubsection{Second subsubsection}

\chapter{Second chapter}
\section{First section}
\subsection*{An unnumbered first subsection}
\subsubsection{First subsubsection}
\subsubsection{Second subsubsection}
\subsection*{An unnumbered second subsection}
\subsubsection{First subsubsection}
\subsubsection{Second subsubsection}

\section{Second section}
\subsection{First subsection}
\subsubsection{First subsubsection}
\subsubsection{Second subsubsection}
\subsection{Second subsection}
\subsubsection{First subsubsection}
\subsubsection{Second subsubsection}

\end{document}

Adding linktocpage is not sufficient, since the sectional units are then not hyperlinked. Unless, of course, this is what you're after in the first place. That is, if used, it provides a mutually exclusive switch between hyperlinking in the ToC. In fact, options related to hyperlinking from the ToC include:

• linktoc=none: No hyperlinking in the ToC;
• linktoc=section: Only the section titles are hyperlinked;
• linktoc=page: Only the page numbers are hyperlinked;
• linktoc=all: Both section titles and page numbers are hyperlinked;
• linktocpage: Similar to linktoc=page

The linktoc key-value is undocumented, but stem from the following code in hyperref.dtx:

\chardef\Hy@linktoc@none=0 %
\chardef\Hy@linktoc@section=1 %
\chardef\Hy@linktoc@page=2 %
\chardef\Hy@linktoc@all=3 %
\ifHy@linktocpage
  \let\Hy@linktoc\Hy@linktoc@page
\else
  \let\Hy@linktoc\Hy@linktoc@section
\fi
\define@key{Hyp}{linktoc}{%
  \@ifundefined{Hy@linktoc@#1}{%
    \Hy@Warning{%
      Unexpected value `#1' of\MessageBreak
      option `linktoc' instead of `none',\MessageBreak
      `section', `page' or `all'%
    }%
  }{%
    \expandafter\let\expandafter\Hy@linktoc
    \csname Hy@linktoc@#1\endcsname
  }%
}

Here's another option, using the tocloft package and its \cftXpagefont family of commands:

\documentclass{book}
\usepackage{xcolor}
\usepackage{tocloft}
\usepackage{hyperref}

\colorlet{wine-stain}{orange!80!black}% just for the example

\hypersetup{
  colorlinks,
  linkcolor=wine-stain % custom color
}

\renewcommand\cftchappagefont{\color{wine-stain}}
\renewcommand\cftsecpagefont{\color{wine-stain}}
\renewcommand\cftsubsecpagefont{\color{wine-stain}}

\begin{document}

\tableofcontents
\chapter{Test Chapter}
\section{Test Section}
\subsection{Test Subsection}

\end{document}

If the leaders (the filling dots) should alos receive color, you can also redefine the \cftXleader family of commands:

\documentclass{book}
\usepackage{xcolor}
\usepackage{tocloft}
\usepackage{hyperref}

\colorlet{wine-stain}{red!80!black}

\hypersetup{
  colorlinks,
  linkcolor=wine-stain % custom color
}

\renewcommand\cftchappagefont{\color{wine-stain}}
\renewcommand\cftsecpagefont{\color{wine-stain}}
\renewcommand\cftsecleader{\color{wine-stain}\cftdotfill{\cftsecdotsep}}
\renewcommand\cftsubsecpagefont{\color{wine-stain}}
\renewcommand\cftsubsecleader{\color{wine-stain}\cftdotfill{\cftsubsecdotsep}}

\begin{document}

\tableofcontents
\chapter{Test Chapter}
\section{Test Section}
\subsection{Test Subsection}

\end{document}

Notice that both these approaches just add color.