Physical intuition for the solution to $y' = y$.
Suppose each step $k$ gives you $x^k$ solutions. However, of those $x^k$ solutions, not all are unique, because some are permutations of others, or in other words re-ordering of others. So, if we count solutions that are permutations of one another as same solutions, then there are $x^k/k!$ solutions at each step $k$. This explains dividing by $k!$. https://en.wikipedia.org/wiki/Permutation#k-permutations_of_n
Why would each step produce $x^k$ solutions? Assume your set $S$ has $x$ elements. If you're asking how many $k$-tuplets there are on $S$, or in how many ways you can arrange $x$s in packets of $k$ of them, then the answer is $x^k$. https://en.wikipedia.org/wiki/Permutation#Permutations_with_repetition
So, if you pack $x$ elements in packets of $k$, but don't mind the way elements are ordered in packets, then you can make this many packets of all the different lengths altogether:
$\sum\limits_{k=1}^{\infty} \frac{x^k}{k!}$
The fact that $x$ is not an integer in general is just an echo of the weird possibility that this result makes sense for any $x\in\mathbb{R}$ and even $x\in\mathbb{C}$.
The weirder possibility is that $x$ represents the number of infinitely many elements. One can form packets of any sizes then. This would explain why $k$ may range over $\mathbb{N}$. And still weirder, the smallness of $x$ may be explained as re-normalization: you squeeze entire $\mathbb{N}$ line so that it becomes $\mathbb{R}$ or, say, interval $\left[0,1\right]$. This is possible in non-standard analysis where infinitesimals live, numbers that are smaller than any positive real number. So $x$ is just re-normalized number of elements of the set $S$. And $S$ has infinitely many discrete elements, countably many if you will. https://en.wikipedia.org/wiki/Non-standard_analysis
Suppose you have $x$ of some weird rabbits in heaven, who are all hermaphrodites: can act as male and female, but cannot breed with themselves, only with other rabbits. Suppose every rabbit breeds with every other rabbit. How many geneticly diverse offspring will be created? Well, you pack them in pairs, and don't mind if the pair is (male,female) or (female, male), obviously. In how many ways can you pack them this way? Well, $x^2/2!$.
Now, what if weird rabbits can breed in trios too, each possible trio producing just 1 offspring? You count that in too, and get $x^3/3!$ of genetic varieties, if you assume that a baby rabbit can have 2 fathers. I told you these rabbits are weird...
And so on. When you count all the possible varieties in, you get $e^x$. Number of rabbits $x$ has to be infinite for this to make sense, but if when passing from heaven to Earth numbers become squeezed, so that infinite numbers become finite, this becomes ordinary usual $e^x$.
So, if the number of rabbits $x$ grows, and it surely does after all this, then the number of genetic varieties among rabbits, grows as $e^x$ as weird rabbits keep reproducing in heaven.
To demonstrate the way one may squeeze an infinite natural number of rabbits $x$ to become a real finite number $r$, consider a unit interval $\left[ 0,1 \right] $ of real numbers on the real number line, and assume this:
$\bf{Assumption}$: The number of elements $c$ of the unit interval $\left[ 0,1 \right] $ is constant.
This seems a fairly reasonable assmuption. Why shouldn't the number of reals in the unit interval be constant? Do real numbers multiply?
What's the minimal distance between numbers of the unit interval? Well, the length of the unit interval is $1$, the number of elements is $c$, which is obviously an infinitely large number, and hence the minimal distance of equidistributed, equadistant elements is the length divided by the number of elelments, $1/c=0$. Hence, there is no minimal distance. This is so because $\mathbb{R}$ is dense: there's a real number between any two real numbers.
OK, now stretch the unit interval by factor $c$! The new, stretched interval is now of length $c$. Number of elements is constant, so still $c$. What is the minimal length between elements of this new interval? Length divided by the number of elements is $c/c=1$. The distance between two consecutive elements is now $1$. So, by stretching the unit interval by $c$, we have created $\mathbb{N}$.
So, any infinitely large natural number $x$ is just some real number $r$ from the unit interval, stretched by $c$, really...
Now, newly created $\mathbb{N}$ has a unit interval as well, so if we stretch this new unit interval by, say, $c$, then...
So, you see, $e^x$ is a function from heaven...
This is a question suitable for physics forums too.
Maybe you may find this answer suitable? Click a like or something if you do, please, I need 50 reputations in order to be able to comment, and I have serious questions... Thanks! :D