How to show that any rectangle in ellipse must be oriented parallel to axes?

Without loss of generality $a>b$. Take an inscribed rectangle with two sides gradient $k\ne0$ and two sides gradient $-\frac{1}{k}\ne0$. Shrink along the $x$-axis by a factor $\frac{b}{a}$. The two sides gradient $k$ now have gradient $\frac{ka}{b}$ and the two sides gradient $-\frac{1}{k}$ now have gradient $-\frac{a}{kb}$. So the rectangle is now a non-rectangular parallelogram with opposite angles not summing to $\pi$. But it is inscribed in a circle. Contradiction.

EDIT: This picture illustrated the basic idea of the proof below. (Although it is written using the parametrization of ellipse, it roughly corresponds to the picture.) I would say that the idea behind almagest's answer is similar.

The basic idea can be explained like this: Relation between ellipse and circle is that we scaled the coordinate along one of the axes using some factor. This is an affine transformation, so it does not influence whether two vectors are parallel. So this gives us one-to-one correspondence between parallelograms in the circle and parallelograms in the ellipse. Every parallelogram inscribed in a circle is a rectangle. However, if the rectangle does not have sides parallel to the axes, this transformation changes the right angle to a different angle.

Although I only realized this later. (And I have to admit that the way the argument is written out below is much clumsier than the idea illustrated by this picture.)

---

We can use the well-known parametrization of ellipse given by \begin{align*} \newcommand{\vp}{\varphi} x&=a\cos\vp\\ y&=b\sin\vp \end{align*}

This means that any four points on the ellipse can be expressed as $A=(a\cos\vp_1,b\sin\vp_1)$, $B=(a\cos\vp_2,b\sin\vp_2)$, $C=(a\cos\vp_3,b\sin\vp_3)$, $D=(a\cos\vp_4,b\sin\vp_4)$, where $0\le\vp_1<\vp_2<\vp_3<\vp_4$.

If they should form the rectangle, then we necessarily have $\overrightarrow{AB}=\overrightarrow{DC}$ and $\overrightarrow{AD}=\overrightarrow{BC}$. This condition gives us \begin{align*} a(\cos\vp_2-\cos\vp_1)&=a(\cos\vp_3-\cos\vp_4)\\ b(\sin\vp_2-\sin\vp_1)&=b(\cos\vp_3-\sin\vp_4)\\ a(\cos\vp_3-\cos\vp_2)&=a(\cos\vp_4-\cos\vp_1)\\ b(\sin\vp_3-\sin\vp_2)&=b(\cos\vp_4-\sin\vp_1) \end{align*} Of course, this is equivalent to \begin{align*} \cos\vp_2-\cos\vp_1&=\cos\vp_3-\cos\vp_4\\ \sin\vp_2-\sin\vp_1&=\cos\vp_3-\sin\vp_4\\ \cos\vp_3-\cos\vp_2&=\cos\vp_4-\cos\vp_1\\ \sin\vp_3-\sin\vp_2&=\cos\vp_4-\sin\vp_1 \end{align*} This means that the points given by the angles $\vp_1,\dots,\vp_4$ on the unit circle form a parallelogram.¹ Hence we get $\vp_2-\vp_1=\vp_4-\vp_3$ and $\vp_3=\pi+\vp_1$.

If we denote $\vp=\frac{\vp_1+\vp_2}2$ and $\alpha=\frac{\vp_2-\vp_1}2$, then wee can rewrite our four points as \begin{align*} A&=(a\cos(\vp-\alpha),b\sin(\vp-\alpha))\\ B&=(a\cos(\vp+\alpha),b\sin(\vp+\alpha))\\ C&=(-a\cos(\vp-\alpha),-b\sin(\vp-\alpha))\\ D&=(-a\cos(\vp+\alpha),-b\sin(\vp+\alpha))\\ \end{align*} In this way we have describe all parallelorams inscribed in an ellipse.

In order to get a rectangle, the length of the diagonals must be the same. This gives us the condition \begin{align*} a^2\cos^2(\vp-\alpha)^2+b^2\sin^2(\vp-\alpha)&=a^2\cos^2(\vp+\alpha)^2+b^2\sin^2(\vp+\alpha)\\ a^2(\cos\vp\cos\alpha+\sin\vp\sin\alpha)^2+b^2(\sin\vp\cos\alpha-\cos\vp\sin\alpha)^2&= a^2(\cos\vp\cos\alpha-\sin\vp\sin\alpha)^2+b^2(\sin\vp\cos\alpha+\cos\vp\sin\alpha)\\ a^2\cos\vp\cos\alpha+\sin\vp\sin\alpha&=b^2\cos\vp\cos\alpha+\sin\vp\sin\alpha\\ (a^2-b^2)\cos\vp\cos\alpha\sin\vp\sin\alpha&=0\\ \cos\vp\cos\alpha\sin\vp\sin\alpha&=0\\ \sin2\vp\sin2\alpha&=0 \end{align*} (Notice that we have used $a^2-b^2\ne0$.)

So we either get $\sin2\vp=0$, which means $\vp=k\frac\pi2$. (And $\alpha$ is arbitrary.) This solution leads to a rectangle with sides parallel to axis.

The other possibility is $\sin2\alpha=0$, which means $\alpha=k\frac\pi2$. In this case two pairs of points will coincide and we will get line segments instead of rectangles.

---

¹The whole argument can be viewed geometrically. We simply "stretch" the ellipse to a circle. The shear transformation does not change which vectors are parallel to each other. The argument is relying on the fact that we "know" what the parallelograms inscribed in a circle look like. (Maybe it would be interesting to check whether the equalities $\vp_2-\vp_1=\vp_4-\vp_3$ and $\vp_3=\pi+\vp_1$ can also be derived by some algebraic manipulation and trigonometric identities instead of a geometric argument involving unit circle and inscribed parallelogram.)