Continuous function whose right hand derivative equals 0 is constant?

Yes, $f$ is necessarily constant.

One can proceed similarly as in the proofs of the mean value theorem and Rolle's theorem for differentiable functions.

It suffices to show that $f(c) = f(d)$ for $a < c < d < b$. The continuity of $f$ then implies that $f$ is constant on $[a, b]$.

Assume that $f(c) \ne f(d)$, without loss of generality $f(c) < f(d)$. Consider the function $$ g(x) = f(x) - (x-c)\frac{f(d)-f(c)}{d-c} $$ for $x \in [c, d]$. The right-hand derivative of $g$ is $$ g_+'(x) = f_+'(x) - \frac{f(d)-f(c)}{d-c} = - \frac{f(d)-f(c)}{d-c} < 0 $$ for all $x \in [c, d)$.

But $g(c) = g(d)$, so that $g$ attains its minimum at a point $x_0 \in [c, d)$, where $$ g_+'(x_0) = \lim_{x \to x_0^+} \frac{g(x)-g(x_0)}{x-x_0} \ge 0 \, , $$ which is a contradiction. This completes the proof.