Cardinal Arithmetic versus Ordinal Arithmetic
The reason is that the definitions of ordinal arithmetics and cardinal arithmetics are very different.
Whereas the ordinal arithmetics operations are concerned with order types, the cardinal arithmetics are concerned with certain sets.
For example, $\alpha+\beta$ as ordinals is the order type of $\alpha$ concatenated with $\beta$. Whereas cardinality strips naked any possible structure, and considering the cardinality of the set $\{0\}\times\alpha\cup\{1\}\times\beta$, which is equal to the maximum of $|\alpha|$ and $|\beta|$ (granted one is infinite).
Exponentiation, which is the strangest one, is defined very differently, again, from ordinals and cardinals.
- In cardinals $\alpha^\beta$ is the cardinality of the set of all functions from $\beta$ to $\alpha$.
In the ordinals, we care about the order, so $\alpha^\beta$ is the order type of the reverse lexicographic order of functions from $\beta$ into $\alpha$ which are non-zero only in finitely many coordinates.
Equivalently, and perhaps more clearly, we can define this by induction, $\alpha^0=1$, $\alpha^{\beta+1}=\alpha^\beta\cdot\alpha$, and for a limit ordinal $\delta$, $\alpha^\delta=\sup\{\alpha^\beta\mid\beta<\delta\}$.
Now we easily see that $2^\omega=\omega$ when we are talking about ordinal exponentiation. $2^\omega$ is the limit of $2^n$ for finite $n$, but $2^n$ has a finite order type, and it's a strictly increasing sequence. The limit of a strictly increasing sequence of finite ordinals is $\omega$ itself.
Well, it sounds weird, isn't it? But it's not really weird. We have this sort of phenomenon in other - more familiar - systems of arithmetics.
In the natural numbers $n\cdot m$ can be defined as repeated addition of $n$, $m$ times. Addition itself can be defined as repeated successor operation. On the other hand, when we consider the real numbers $\sqrt2\cdot\sqrt2$ cannot be thought as repeated addition. What does it even mean to add something $\sqrt2$ times? It is true that in this case, if we restrict back to the natural numbers then the operations become repeated application of the "previous operation", but this is because of the nature of the natural numbers as a corner stone of modern mathematics (in many many ways). For infinite, and less-corner stoney this is not the case, as exhibited in ordinals and cardinals.