Prove that $C\exp(x)$ is the only set of functions for which $f(x) = f'(x)$

Of course $C e^x$ has the same property for any $C$ (including $C = 0$). But these are the only ones.

Proposition: Let $f : \mathbb{R} \to \mathbb{R}$ be a differentiable function such that $f(0) = 1$ and $f'(x) = f(x)$. Then it must be the case that $f = e^x$.

Proof. Let $g(x) = f(x) e^{-x}$. Then

$$g'(x) = -f(x) e^{-x} + f'(x) e^{-x} = (f'(x) - f(x)) e^{-x} = 0$$

by assumption, so $g$ is constant. But $g(0) = 1$, so $g(x) = 1$ identically.

N.B. Note that it is also true that $e^{x+c}$ has the same property for any $c$. Thus there exists a function $g(c)$ such that $e^{x+c} = g(c) e^x = e^c g(x)$, and setting $c = 0$, then $x = 0$, we conclude that $g(c) = e^c$, hence $e^{x+c} = e^x e^c$.

This observation generalizes to any differential equation with translation symmetry. Apply it to the differential equation $f''(x) + f(x) = 0$ and you get the angle addition formulas for sine and cosine.

Let $f(x)$ be a differentiable function such that $f'(x)=f(x)$. This implies that the $k$-th derivative, $f^{(k)}(x)$, is also equal to $f(x)$. In particular, $f(x)$ is $C^\infty$ and we can write a Taylor expansion for $f$:

$$T_f(x) = \sum_{k=0}^\infty c_k x^k.$$

Notice that the fact that $f(x)=f^{(k)}(x)$, for all $k\geq 0$, implies that the Taylor series $T_f(x_0)$ converges to $f(x_0)$ for every $x_0\in \mathbb{R}$ (more on this later), so we may write $f(x)=T_f(x)$. Since $f'(x) = \sum_{k=0} (k+1)c_{k+1}x^k = f(x)$, we conclude that $c_{k+1} = c_k/(k+1)$. The value of $c_0 = f(0)$, and therefore, $c_k = f(0)/k!$ for all $k\geq 0$. Hence:

$$f(x) = f(0) \sum_{k=0}^\infty \frac{x^k}{k!} = f(0) e^x,$$

as desired.

Addendum: About the convergence of the Taylor series. Let us use Taylor's remainder theorem to show that the Taylor series for $f(x)$ centered at $x=0$, denoted by $T_f(x)$, converges to $f(x)$ for all $x\in\mathbb{R}$. Let $T_{f,n}(x)$ be the $n$th Taylor polynomial for $f(x)$, also centered at $x=0$. By Taylor's theorem, we know that $$|R_n(x_0)|\leq |f^{(n+1)}(\xi)|\frac{ |x_0 - 0|^{n+1}}{(n+1)!},$$ where $R_n(x_0)=f(x) - T_{f,n}(x)$ and $\xi$ is a number between $0$ and $x_0$. Let $M=M(x_0)$ be the maximum value of $|f(x)|$ in the interval $I=[-|x_0|,|x_0|]$, which exists because $f$ is differentiable (therefore, continuous) in $I$. Since $f(x)=f^{(n+1)}(x)$, for all $n\geq 0$, we have: $$|R_n(x_0)|\leq |f^{(n+1)}(\xi)|\frac{ |x_0|^{n+1}}{(n+1)!}\leq |f(\xi)|\frac{ |x_0|^{n+1}}{(n+1)!}\leq M \frac{|x_0|^{n+1}}{(n+1)!} \longrightarrow 0 \ \text{ as } \ n\to \infty.$$ The limit goes to $0$ because $M$ is a constant (once $x_0$ is fixed) and $A^n/n! \to 0$ for all $A\geq 0$. Therefore, $T_{f,n}(x_0) \to f(x_0)$ as $n\to \infty$ and, by definition, this means that $T_f(x_0)$ converges to $f(x_0)$.

Yet another way: By the chain rule, ${\displaystyle {d \over dx} \ln|f(x)| = {f'(x) \over f(x)} = 1}$. Integrating, you get $\ln |f(x)| = x + C$. Taking $e$ to both sides, you obtain $|f(x)| = e^{x + C} = C'e^x$, where $C' > 0$. As a result, $f(x) = C''e^x$, where $C''$ is an arbitrary constant.

If you are worried about $f(x)$ being zero, the above shows $f(x)$ is of the form $C''e^x$ on any interval for which $f(x)$ is nonzero. Since $f(x)$ is continuous, this implies $f(x)$ is always of that form, unless $f(x)$ is identically zero (in which case we can just take $C'' = 0$ anyhow).