How to provide more horizontal space between cases in a system?

Use \\[3ex] for adding vertical space. But, in this case, you can even do better: if you notice, the matrices are not perfectly aligned to each other. So I propose a slightly different solution:

\documentclass{article}
\usepackage{amsmath,amssymb}
\newcommand{\C}{\mathbb{C}}
\newcommand{\Aaa}{%
  \begin{bmatrix}1&0\\0&-1\end{bmatrix}%
}
\newcommand{\Ba}{%
  \begin{bmatrix}0&1\\-1&0\end{bmatrix}%
}

\begin{document}

% Simplistic solution with `\\[3ex]`
\begin{displaymath}
\rho_1: V_{24}\longrightarrow GL_2(\C):
\begin{cases}
a\longmapsto \Aaa\\[3ex]
b\longmapsto \Ba
\end{cases}
\end{displaymath}

%% Better solution with horizontal alignment
\begin{displaymath}
\rho_1: V_{24}\longrightarrow GL_2(\C):
\begin{cases}
\begin{aligned}
a&\longmapsto \Aaa\\
b&\longmapsto \Ba
\end{aligned}
\end{cases}
\end{displaymath}
\end{document}

In the second solution, not only aligned puts the two matrices one over each other, but it also succeeds to leave some vertical space between the rows, without explicit spacing.

You can manually adjust the height between rows if you have super-tall rows using \\[<len>]. The following solution also adjusts the location of b to make it align with a, thereby aligning the matrices (using some \phantom and overlaping magic):

\documentclass{article}
\usepackage{amsmath}% http://ctan.org/pkg/amsmath
\newcommand{\mymatrix}[1]{%
  \begin{bmatrix}#1\end{bmatrix}
}
\begin{document}
\[
  \rho_1: V_{24}\longrightarrow GL_2(\mathcal{R}):
  \begin{cases}
    a \longmapsto \mymatrix{1 & 0 \\ 0 & -1} \\[12pt]
    \hphantom{a}\llap{$b$} \longmapsto \mymatrix{0 & 1 \\ -1 & 0}
  \end{cases}
\]
\end{document}

Using the tabstackengine package introduced at Writing a table with equally spaced columns, based on the widest column, you can control the vertical gap between the matrix rows as well as between the "cases", and the horizontal gap between matrix elements.

\documentclass{article}
\usepackage{tabstackengine}
\usepackage{amssymb}
\begin{document}
\setstackgap{L}{1.3\baselineskip}%BASELINE SHIFT BETWEEN MATRIX ELEMENTS
\setstacktabbedgap{2ex}% HORIZONTAL GAP BETWEEN MATRIX ELEMENTS
\savestack\Mone{$\bracketMatrixstack[c]{1&0\\0&-\!1}$}
\savestack\Mtwo{$\bracketMatrixstack[c]{0&1\\-\!1&0}$}
\setstackgap{L}{2.8\baselineskip}% BASELINE SHIFT BETWEEN CASE ROWS
\( f: G\longrightarrow GL_2(\mathbb{R}):
\left\{\raisebox{.3\baselineskip}{% .3 IS AN AD HOC CORRECTION
\alignVectorstack{a&\longmapsto \Mone\\b&\longmapsto \Mtwo}}
\right. \)
\end{document}