How to prove that the exponential function is the inverse of the natural logarithm by power series definition alone

I enjoyed Christian's answer, and if you want the shortest proof starting from the series definitions, then that is the way to go. But, like you, I was curious about this question because I like to manipulate power series, and because this problem ended up being harder than I thought it would be.

Here is a solution that meets your requirements. The only thing I have omitted is proving that exchanging the order of summation is valid when I do it below, which is probably not the part of the proof you are interested in.

A few lemmas are needed, which I will list first. The first two are well-known combinatorial identities, and I will omit the proofs. I'll follow your approach, but I'll delay the expansion of $e^x$ a little bit longer than you did, because otherwise you get sums over an arbitrarily large number of variables, which end up being difficult to handle.

Lemma 1: For $m,n \in \mathbb{Z}$ with $0 \le m < n$, $$ \sum_{k=0}^n (-1)^k \binom{n}{k} k^m = 0 $$ The easiest way to prove this is with calculus, but it can be proven by induction on $m$ from the result that for $n \ge 1$, $$ \sum_{k=0}^n (-1)^k \binom{n}{k} = 0 $$ which itself can be proven by induction on $n$. So, no calculus required.

Lemma 2: $$ \sum_{n=k}^m \binom{n}{k} = \binom{m+1}{k+1} $$ This can also be proven by induction.

Lemma 3: $ e^x e^y = e^{x+y} $

Proof: This one is a lot easier than the main result. Expand the right hand side using the series definition and the binomial theorem to get $$ e^{x+y} = \sum_{m=0}^\infty \frac{(x + y)^m}{m!} = \sum_{m=0}^\infty \sum_{i=0}^m \frac{1}{m!} \binom{m}{i} x^i y^{m-i} = \sum_{i=0}^\infty \sum_{m=i}^\infty \frac{x^i y^{m-i}}{i! (m-i)!} $$ Let $j=m-i$ and the sums become independent: $$ \sum_{i=0}^\infty \sum_{m=i}^\infty \frac{x^i y^{m-i}}{i! (m-i)!} = \sum_{i=0}^\infty \sum_{j=0}^\infty \frac{x^i y^j}{i!j!} = \left( \sum_{i=0}^\infty \frac{x^i}{i!} \right) \left( \sum_{j=0}^\infty \frac{x^j}{j!} \right) = e^x e^y $$

Lemma 4: $ (e^x)^y = e^{xy} $ when $y$ is a nonnegative integer.

Proof: This is obvious using Lemma 3.

Main result: $ \log(e^x) = x $

Proof: Apply the power series for the logarithm, then a binomial expansion, and then the power series for the exponential.

$$ \begin{align} \log(e^x) &= \sum_{n=1}^\infty \frac{(-1)^{n+1}}{n} ( e^x - 1 )^n \\ &= \sum_{n=1}^\infty \frac{(-1)^{n+1}}{n} \sum_{k=0}^n \binom{n}{k} (-1)^{n-k} e^{kx} \\ &= \sum_{n=1}^\infty \frac{(-1)^{n+1}}{n} \sum_{k=0}^n \binom{n}{k} (-1)^{n-k} \sum_{m=0}^\infty \frac{k^m x^m}{m!} \\ &= -\sum_{m=0}^\infty \frac{x^m}{m!} \sum_{n=1}^\infty \frac{1}{n} \sum_{k=0}^n \binom{n}{k} (-1)^k k^m \\ &= -\sum_{m=0}^\infty \frac{x^m}{m!} a_m \tag{1} \label{1} \end{align} $$

where $a_m$ has been defined as

$$ a_m = \sum_{n=1}^\infty \frac{1}{n} \sum_{k=0}^n \binom{n}{k} (-1)^k k^m $$

Split the sum over $n$ into two parts, from $1$ to $m$ and from $m+1$ to $\infty$.

$$ a_m = \sum_{n=1}^m \frac{1}{n} \sum_{k=0}^n \binom{n}{k} (-1)^k k^m + \sum_{n=m+1}^\infty \frac{1}{n} \sum_{k=0}^n \binom{n}{k} (-1)^k k^m $$

On the right, the summation over $k$ is zero by Lemma 1 because $n > m$. This gives us

$$ a_m = \sum_{n=1}^m \frac{1}{n} \sum_{k=0}^n \binom{n}{k} (-1)^k k^m \label{2} \tag{2} $$

In this expression, when $k \ge 1$, we can apply the identity

$$ \frac{1}{n} \binom{n}{k} = \frac{1}{k} \binom{n-1}{k-1} \label{3} \tag{3} $$

Fortunately, the $k=0$ terms in \eqref{2} are all zero because of the $k^m$ term (since we are only considering $a_m$ where $m \ge 1$). Dropping the $k=0$ terms, moving the summation over $n$ to the right, and applying \eqref{3} in \eqref{2}, we get

$$ \begin{align} a_m &= \sum_{k=1}^m (-1)^k k^m \sum_{n=k}^m \frac{1}{k} \binom{n-1}{k-1} \\ &= \sum_{k=1}^m (-1)^k k^{m-1} \sum_{n=k}^m \binom{n-1}{k-1} \end{align} $$

Applying Lemma 2 to the summation over $n$ gives $$ a_m = \sum_{k=1}^m (-1)^k k^{m-1} \binom{m}{k} $$

This can be simplified by again applying Lemma 1. However, note that Lemma 1 requires the $k=0$ terms, which we dropped earlier so that we could apply \eqref{3}. Now the $k=0$ term in the sum must be replaced by including $k=0$ in the sum and subtracting it outside the sum.

$$ a_m = \left( \sum_{k=0}^m (-1)^k k^{m-1} \binom{m}{k} \right) - 0^{m-1} $$

Obviously this term on the right is zero when $m>1$, but the $m=1$ case results in $0^0$. Here the appropriate convention is that $0^0=1$. (For example, when we expanded the exponential as $e^{kx} = \sum_m k^m x^m/m!$, the $m=0$ term is interpreted as $1$ even when $k=0$.) Since the $0^{m-1}$ term yields 0 for $m \ne 1$ and 1 for $m=1$, we can write this using the Kronecker delta as $\delta_{m,1}$.

$$ a_m = \left( \sum_{k=0}^m \binom{m}{k} (-1)^k k^{m-1} \right) - \delta_{m,1} $$

Now Lemma 1 can be applied again. (Note that the conditions of the lemma are satisfied because $m-1 < m$.) This gives $ a_m = -\delta_{m,1} $. Inserting this into $\eqref{1}$ gives the final result,

$$ \log(e^x) = x $$

Consider the two functions $$f(x):=\sum_{k=1}^\infty{(-1)^{k-1}\over k}x^k\qquad\bigl(=\log(1+x)\bigr)$$ and $$g(y):=\sum_{j=1}^\infty{1\over j!}y^j\qquad\bigl(=e^y-1\bigr)\ .$$ Both are defined in a neighborhood of the orgin, one has $f(0)=g(0)=0$, and termwise differentiation of the two series reveils that $$\eqalign{f'(x)&=\sum_{k=1}^\infty(-1)^{k-1}x^{k-1}={1\over 1+x}\ ,\cr g'(y)&=\sum_{j=1}^\infty{j\over j!}y^{j-1}=\sum_{j'=0}^\infty{1\over j'!}y^{j'}=1+g(y)\ .\cr}$$ We now consider the composition $$p(y):=f\bigl(g(y)\bigr)$$ of these two functions. According to the chain rule (no further series manipulation required!) we obtain $$p'(y)=f'\bigl(g(y)\bigr)\>g'(y)={1\over 1+g(y)}\bigl(1+g(y)\bigr)\equiv 1\ .$$ As $p(0)=0$ we can conclude that $p(y)\equiv y$, which proves that $f$ and $g$ are indeed inverses of each other.