Proof of $\frac{d}{dx}e^x = e^x$

Everything in the "proof" will depend on your definition of the function $e^x$. I will choose the definition $$ e^h \overset{\text{def}}{=} \sum_{k=0}^\infty \frac{h^k}{k!}. $$ Using this, one sees that $$ \frac{e^h - 1}{h} = \frac{\sum_{k=0}^\infty \frac{h^k}{k!} - 1}{h} = \sum_{k=1}^\infty \frac{h^{k-1}}{k!} = 1 + h \sum_{k=0}^\infty \frac{h^k}{(k+2)!}. $$ If you have studied convergence tests, you know that the last series on the right converges for all $h \in \mathbb R$, hence taking the limit when $h \to 0$, the RHS goes to $1$ because the series will converge to something and the $h$ factored out will make the product go to zero.

Another way to do this would be to show that this series is an analytic function and is its own Taylor expansion around zero (this is not hard to do using convergence tests), so to differentiate it you can go term by term and readily see that its derivative is itself.

A third approach, which will sound a little stupid and meaningless but is nonetheless funny, is choosing another definition for $e^x$ : consider the differential equation $$ f'(x) = f(x), \quad f(0) = 1. $$ Using differential equation theory it is really not hard at all to show that the solutions to the equation (without the initial condition) is a one-dimensional vector space and there exists an unique element of this vector space which satisfies the initial condition $f(0) = 1$, because the solutions are of the form $Cg(x)$ for some solution $g(x)$. Let $exp(x)$ be defined as a solution to this differential equation satisfying the initial condition. Then clearly $exp'(x) = exp(x)$. Then you can easily see that $exp'(x)$ is a differentiable function, so by induction $exp(x)$ is an infinitely differentiable function with Taylor expansion $$ \exp(x) = \sum_{k=0}^\infty \frac{x^k}{k!}. $$ I mentioned it to show the importance of which definition one decides to choose ; it can change the whole structure of an argument.

Hope that helps,

Suppose you have an exponential function, like $f(x)=2^x$.

The derivative is $$ f'(x) = \lim_{h\to0} \frac{f(x+h)-f(x)}{h} = \lim_{h\to0}\frac{2^{x+h}-2^x}{h} = \lim_{h\to0}\left(2^x\cdot\frac{2^h-1}{h} \right). $$ So far, just algebra. Now watch this: $$ = 2^x\lim_{h\to0}\frac{2^h-1}{h}. $$ This can be done because $2^x$ is "constant" and "constant" means "not depending on $h$".

But this is equal to $(2^x\cdot\text{constant})$. But in this case "constant" means "not depending on $x$". "Constant" always means "not depending on something", but "something" varies with the context.

What's the "constant"? In the case above, it's not hard to show that the constant is somewhere between $1/2$ and $1$. It we'd started with $4^x$ instead, then it would be fairly easy to show that the "constant" would be more than $1$. For a base somewhere between $2$ and $4$, the "constant" is $1$. That base is $e$.

If you want to talk about how it is knonw that $2$ is too small and $4$ is too big, to be the base of the "natural" exponential function (i.e., the one for which the "constant" is $1$), I can post further on this.

Later edit: OK, how do we know that $2$ is too small and $4$ is too big, to serve in the role of the base of the "natural" exponential function? Look at the graph of $y=2^x$. It gets steeper as you go from left to right. As $x$ goes from $0$ to $1$, $y$ goes from $1$ to $2$. So the average slope between $x=1$ and $x=2$ is $$\dfrac{\text{rise}}{\text{run}} = \frac{2-1}{1-0} = 1.$$ Since it gets steeper going from left to right, the slope at the left end of this interval, i.e. at $x=0$, must be less than that. Thus we have $\dfrac{d}{dx}2^x = (2^x\cdot\text{constant})$ and the "constant" is less than $1$. (Thinking about the interval from $x=-1$ to $x=0$ in the same way shows that the "constant" is more than $1/2$.)

Now look at $y=4^x$, and use the interval from $x=-1/2$ to $x=0$, and do the same thing, and you see that when you write $\dfrac{d}{dx}4^x = (4^x\cdot\text{constant})$, then that "constant" is more than $1$.

This should suggest that $4$ is too big, and $2$ is too small.

You can show that $3$ is too big by using the interval from $x=-1/6$ to $x=0$ and doing the same thing. But the arithmetic is messy.

Apostol's approach in his Calculus text is to define the natural logarithm by $$\log x=\int_1^xt^{-1}\,dt$$ from which you immediately get that the derivative of $\log x$ is $1/x$. Then Apostol defines the exponential function as the functional inverse of the logarithm. So if $y=e^x$, then $x=\log y$; now differentiate with respect to $x$, using the chain rule, to get $$1={1\over y}{dy\over dx}$$ So, ${dy\over dx}=y=e^x$, as desired.