How can I use color in a mathematical expression without losing horizontal spacing?

You have to work a bit harder, telling TeX the kind of object you want:

\documentclass{article}
\usepackage{xcolor}
\definecolor{red}{HTML}{dd0000}
\begin{document}
\[3+(-5)-2+3\]
\[3+(-5)\mathbin{\textcolor{red}{-}}\textcolor{red}{2}+3\]
\end{document}

For a one shot application this is probably the easiest way. If the emphasis color is always the same, you could define a new command:

\newcommand{\mathem}[2][\mathord]{%
  #1{\textcolor{red}{#2}}}

and input the above as

3 + (-5) \mathem[\mathbin]{-} \mathem{2} + 3

You lose the horizontal spacing because of the brace group {}. Besides egreg's solution there are two ways that don't require telling TeX the kind of object you want:

1. Use \begingroup ... \endgroup instead of {...},

2. Use \color{black} to switch back to black.

Both yields the same result:

\documentclass{article}
\usepackage{amsmath} % for gather
\usepackage{xcolor}
\definecolor{red}{HTML}{dd0000}
\begin{document}
\begin{gather*}
3+(-5)-2+3 \\
3+(-5) \color{red} -2 \color{black} +3 \\
3+(-5)\begingroup\color{red}-2\endgroup+3
\end{gather*}
\end{document}