Commutativity of Ordinal Multiplication

Statement (2) is true, though I don't know any way to prove it without intricate analysis of Cantor normal forms. The rough idea is that it is extremely rare for ordinal multiplication to commute, so $\alpha^2\beta^2=\beta^2\alpha^2$ forces $\alpha$ and $\beta$ to have a certain highly restricted form that implies $\alpha\beta=\beta\alpha$ as well.

Let me start with a version of (2) for addition.

Lemma: Let $\alpha$ and $\beta$ be ordinals such that $\alpha\cdot 2+\beta\cdot 2=\beta\cdot 2+\alpha\cdot 2$. Then $\alpha+\beta=\beta+\alpha$.

Proof: If $\alpha$ or $\beta$ is finite this is easy (a nonzero finite ordinal commutes only with other finite ordinals), so we assume they are both infinite. Write them in Cantor normal form as $$\alpha=\omega^{\alpha_n}\cdot a_n+\cdots+\omega^{\alpha_1}\cdot a_1+a_0$$ and $$\beta=\omega^{\beta_m}\cdot b_m+\cdots+\omega^{\beta_1}\cdot b_1+b_0.$$ We then have Cantor normal forms $$\alpha\cdot 2=\omega^{\alpha_n}\cdot (2a_n)+\cdots+\omega^{\alpha_1}\cdot a_1+a_0$$ and $$\beta\cdot 2=\omega^{\beta_m}\cdot (2b_m)+\cdots+\omega^{\beta_1}\cdot b_1+b_0.$$ By the criterion for commutativity of addition in this answer, $\alpha\cdot 2+\beta\cdot 2=\beta\cdot 2+\alpha\cdot 2$ implies that $m=n$, $\alpha_k=\beta_k$ for all $k$, and $a_k=b_k$ for $k<n$. But then using that criterion in the other direction (or just directly adding the Cantor normal forms), we conclude that $\alpha+\beta=\beta+\alpha$.

Now we prove statement (2).

Theorem: Let $\alpha$ and $\beta$ be ordinals such that $\alpha^2\beta^2=\beta^2\alpha^2$. Then $\alpha\beta=\beta\alpha$.

Proof: If $\alpha$ is finite, it is easy to see that $\alpha^2\beta^2=\beta^2\alpha^2$ iff either $\beta$ is finite or $\alpha$ is $0$ or $1$, in which case we have $\alpha\beta=\beta\alpha$. So we assume $\alpha$ is infinite, and in the same way we may assume $\beta$ is infinite.

Write $\alpha$ and $\beta$ in Cantor normal form as $$\alpha=\omega^{\alpha_n}\cdot a_n+\cdots+\omega^{\alpha_1}\cdot a_1+a_0$$ and $$\beta=\omega^{\beta_m}\cdot b_m+\cdots+\omega^{\beta_1}\cdot b_1+b_0.$$ Then we have the Cantor normal form $$\alpha^2=\omega^{\alpha_n+\alpha_n}\cdot a_n+\dots+\omega^{\alpha_n+\alpha_1}\cdot a_1 + \omega^{\alpha_n}\cdot a_na_0+\omega^{\alpha_{n-1}}\cdot a_{n-1}+\dots+\omega^{\alpha_1}\cdot a_1+a_0$$ if $\alpha$ is a successor (i.e., $a_0>0$) and $$\alpha^2=\omega^{\alpha_n+\alpha_n}\cdot a_n+\dots+\omega^{\alpha_n+\alpha_1}\cdot a_1$$ if $\alpha$ is a limit (i.e., $a_0=0$), and similarly for $\beta^2$.

If $\alpha$ and $\beta$ are both successor ordinals, then by my answer here there exists an ordinal $\gamma$ and natural numbers $i$ and $j$ such that $\alpha^2=\gamma^i$ and $\beta^2=\gamma^j$. If $i$ and $j$ are even we get $\alpha=\gamma^{i/2}$ and $\beta=\gamma^{j/2}$ (we can recover $\alpha$ from the formula for $\alpha^2$ above so $\alpha$ is the unique square root of $\alpha^2$) and so $\alpha\beta=\beta\alpha$. To handle the odd case, note that a successor ordinal whose Cantor normal form has $N+1$ terms can be written (uniquely) as an $i$th power iff its Cantor normal form is "$N/i$-periodic" in the sense that $N=di$ for some $d$, with term $dx+y$ from the right (starting from $0$) having the form $\omega^{\gamma_d\cdot x+\gamma_y}\cdot c_y$ (except that when $y=0$, the coefficient is instead $c_{N}c_0$ for $0<x<i$ and $c_{N}$ for $x=i$). Indeed, this is exactly what you get from raising $\omega^{\gamma_d}\cdot c_N+\omega^{\gamma_{d-1}}\cdot c_{d-1}+\dots+c_0$ to the $i$th power. It follows from this description that if a successor ordinal is both an $i$th power and an $i'$th power, it must also be an $\operatorname{lcm}(i,i')$th power (since the Cantor normal form will be "$\gcd(N/i,N/i')$-periodic", by an argument similar to the proof in the answer linked above). So if $i$ is odd, $\alpha^2=\gamma^i$ implies that $\gamma^i$ is actually a $2i$th power and so $\gamma$ has a square root $\delta$. We then see that $\alpha=\delta^i$ and $\beta=\delta^j$ and so again $\alpha\beta=\beta\alpha$.

If one if $\alpha$ and $\beta$ is a successor and the other is a limit, let us say $\alpha$ is a successor. Then the Cantor normal form for $\alpha^2$ has $2n+1$ terms including a final $a_0$ term and the Cantor normal form for $\beta^2$ has $m$ terms. When we multiply $\alpha^2\beta^2$, we get one term for each term of $\beta^2$, for just $m$ terms. When we multiply $\beta^2\alpha^2$, we get one term for each term of $\alpha^2$ before the last one, and then also $m$ more terms from $\beta^2\cdot a_0$, for a total of $2n+m$ terms. Since we are assuming $\alpha$ is infinite, $n>0$, so $2n+m\neq m$ and we cannot have $\alpha^2\beta^2=\beta^2\alpha^2$.

Finally, if $\alpha$ and $\beta$ are both limit ordinals, let us see what we can conclude from $\alpha^2\beta^2=\beta^2\alpha^2$ using the criterion for commutativity of multiplication in this answer (note that the criterion there is necessary only for limit ordinals; see my comments there). We find that $m=n$, and for each $k$ from $1$ to $n$, $a_k=b_k$ and $$\alpha_n+\alpha_n+\beta_n+\beta_k=\beta_n+\beta_n+\alpha_n+\alpha_k.$$ In the case $k=n$, this tells us $\alpha_n+\beta_n=\beta_n+\alpha_n$ by the Lemma. So we can rearrange the terms above to get $$\alpha_n+\beta_n+\alpha_n+\beta_k=\alpha_n+\beta_n+\beta_n+\alpha_k.$$ Since addition is left-cancellative, this implies $\alpha_n+\beta_k=\beta_n+\alpha_k$. Using the commutativity criterion in the other direction, we conclude that $\alpha\beta=\beta\alpha$.

I might have overlooked something, but it seems that one only needs four simple properties of ordinals ($\alpha$, $\beta$ and $\gamma$ will denote ordinals):

• Comparison trichotomy: $(\alpha<\beta)$ or $(\alpha=\beta)$ or $(\alpha>\beta)$.
• Multiplication associativity: $\alpha(\beta\gamma)=(\alpha\beta)\gamma$
• If $\alpha\leq\beta$, we have $\alpha\gamma \leq \beta\gamma$ and $\gamma\alpha\leq \gamma\beta$ for any $\gamma$.
• If $\alpha<\beta$ and $0<\gamma$, we have $\gamma\alpha<\gamma\beta$.

Let's start by assuming $\alpha\beta<\beta\alpha$ and see what we can conclude about relation between $\alpha^2\beta^2$ and $\beta^2\alpha^2$.

Multiplying the assumed inequality $\alpha\beta<\beta\alpha$ by $\alpha$ on the left turns it into a non-strict one and a subsequent multiplication by $\beta$ on the right keeps it so. Therefore, we have

$$\alpha^2\beta^2=\alpha\alpha\beta\beta=\left(\alpha(\alpha\beta)\right)\beta \leq \left(\alpha(\beta\alpha)\right)\beta=\alpha\beta\alpha\beta$$

Another application of the third property yields $$\alpha\beta\alpha\beta=(\alpha\beta)(\alpha\beta)\leq (\beta\alpha)(\alpha\beta)$$

Now we can use the fourth property to get $$(\beta\alpha)(\alpha\beta)<(\beta\alpha)(\beta\alpha)=\beta\alpha\beta\alpha$$ (note that $\beta\alpha$ cannot be zero due to being strictly greater than $\alpha\beta$)

The last step follows the same reasoning as the first one and gives us $$\beta\alpha\beta\alpha=(\beta(\alpha\beta))\alpha\leq (\beta(\beta\alpha))\alpha=\beta\beta\alpha\alpha=\beta^2\alpha^2$$

Putting it all together, we have $$\alpha\beta <\beta\alpha \implies \alpha^2\beta^2<\beta^2\alpha^2$$

Since the same kind of reasoning applies if we swap $\alpha$ and $\beta$, we can use the first property to finish the proof: $\alpha^2\beta^2=\beta^2\alpha^2$ can only be true if $\alpha\beta=\beta\alpha$.

I'm not really answering the question, just providing a reference, but maybe that will be of some use. Namely, the answer to your question can be found in W. Sierpiński's book Cardinal and Ordinal Numbers, Second Edition Revised, 1963, particularly in section 26 (pp. 351–362), "On ordinal numbers commutative with respect to multiplication".

Quoting from p. 354:

THEOREM 1. Transfinite ordinal numbers $\alpha$ and $\beta$ are multiplicatively commutative if and only if there exist natural numbers $m$ and $n$ for which $\alpha^n=\beta^m$.

Quoting from p. 359:

THEOREM 3. If $\alpha$, $\beta$ and $\gamma$ are arbitrary ordinal numbers such that $\alpha\gt1$, $\alpha\beta=\beta\alpha$ and $\alpha\gamma=\gamma\alpha$, [then $\beta\gamma=\gamma\beta$].

The conclusion of Theorem 3 is in square brackets because I've corrected an obvious typo: in my copy the conclusion reads $\beta\gamma=\gamma\alpha$.

Quoting from p. 360:

COROLLARY 2. Ordinal numbers $\gt1$ form denumerable disjoint sets such that two ordinal numbers $\gt1$ are multiplicatively commutative if and only if they belong to the same set.

Statement (2) is easily deducible from any of the quoted propostions. It seems too difficult for an exam question, but maybe there is a simpler direct proof of statement (2).

I don't see a counterexample to $\alpha\beta^2=\beta^2\alpha\implies\alpha\beta=\beta\alpha$ even for order types $\alpha$ and $\beta$, but that would be another question.