Convert Mathematica formula to TeX (MathJax) with color

The short answer is that, yes, it definitely is feasible. The most basic approach is to use the "BoxRules" option of Export[_,"TeX",___] to generate the TeX color commands. To do this with your example, we might use the following BoxRules function:

Clear[myBoxRule];
myBoxRule[s_String] := s;
myBoxRule[RowBox[stuff_List, ___]] := StringJoin[myBoxRule /@ stuff];
myBoxRule[FractionBox[numerator_String, denominator_]]  := 
  "\\frac{" <> numerator <> "}{" <> myBoxRule[denominator] <> "}";
myBoxRule[StyleBox[s_String, fColor_RGBColor, opts___]] := 
  "\\color[rgb]" <> latexColor[fColor] <> s <> "}";
myBoxRule[StyleBox[stuff_RowBox, fColor_RGBColor, opts___]] := 
  "\\color[rgb]" <> latexColor[fColor] <> myBoxRule[stuff] <> "}";

We can apply this to an example like yours as follows. (I modified your example a bit to emphasize the nested structure.)

SeedRandom[2];
expr = Nest[1 + 1/Style[#, 
   RGBColor[Random[], Random[], Random[]]] &, 
   x, 5];
ExportString[expr, "TeX", "UsePackages" -> {"color"}, "BoxRules" ->
  {box : (_StyleBox | _RowBox | _FractionBox) :> myBoxRule[box]}]

(* Out: 
%% AMS-LaTeX Created by Wolfram Mathematica 9.0 : www.wolfram.com

\documentclass{article}
\usepackage{color}

\newcommand{\mathsym}[1]{{}}
\newcommand{\unicode}[1]{{}}

\newcounter{mathematicapage}
\begin{document}

\[1+\frac{1}{\color[rgb]{0.902785,0.672727,0.419794}{1+\frac{1}{\color[rgb]{0.632356,0.0725195,0.251276}{1+\frac{1}{\color[rgb]{0.529257,0.0507012,0.74907}{1+\frac{1}{\color[rgb]{0.421515,0.512692,0.0228035}{1+\frac{1}{\color[rgb]{0.238705,0.844529,0.473928}{x}}}}}}}}}}\]

\end{document}
*)

The TeXed output looks like so:

This does not seem to work with MathJaX on this site. It's possible that the color package simply isn't installed.

You can modify a TeXForm internal function to get color output. Here is a function that converts an RGBColor object into the equivalent hex string:

tohex[RGBColor[r_, g_, b_]] := StringJoin["#", IntegerString[Round[255 {r,g,b}], 16, 2]]

Then, modify an internal function:

System`Convert`TeXFormDump`maketex[StyleBox[boxes_, ___, rgb_RGBColor, ___]] := StringJoin[
    "\\color{", 
    tohex[rgb],
    "}{", 
    System`Convert`TeXFormDump`maketex[boxes],
    "}"
]

Now, using TeXForm with color works:

Style[Style[3, Blue]/Style[5, Green] + 2, Orange] //TeXForm

$\color{#ff8000}{\frac{\color{#0000ff}{3}}{\color{#00ff00}{5}}+2}$

Mark's example:

SeedRandom[2];
expr = Nest[1 + 1/Style[#, RGBColor[Random[], Random[], Random[]]] &, x, 5];
HoldForm @ Evaluate @ expr //TeXForm

$1+\frac{1}{\color{#e6ac6b}{1+\frac{1}{\color{#a11240}{1+\frac{1}{\color{#870dbf}{1+\frac{ 1}{\color{#6b8306}{1+\frac{1}{\color{#3dd779}{x}}}}}}}}}}$