How to get the limits of rotated ellipse?

The previous answerer had the right idea, but the wrong execution. We have, for the rotated ellipse

$$\frac{(x\cos\,\theta-y\sin\,\theta)^2}{a^2}+\frac{(x\sin\,\theta+y\cos\,\theta)^2}{b^2}=1$$

the following expression for the derivative:

$$\frac{\mathrm dy}{\mathrm dx}=-\frac{a^2 x\,\sin^2\theta+y(a-b)(a+b)\sin\,\theta \cos\,\theta+b^2 x\,\cos^2\theta}{a^2 y\,\cos^2\theta+x(a-b)(a+b)\sin\,\theta \cos\,\theta+b^2 y\,\sin^2\theta}$$

We can separate the numerator and denominator of this expression, and equate any of the two to zero, and then solve as a simultaneous equation with the equation of the ellipse. (Zero numerator corresponds to horizontal tangents, and zero denominator corresponds to vertical tangents.)

After much algebra and tears, we find that the required limits are $x=\pm\sqrt{a^2\cos^2\theta+b^2\sin^2\theta}$ and $y=\pm\sqrt{a^2\sin^2\theta+b^2\cos^2\theta}$.

This could also have been done starting from the parametric equations; for that route, one will need to use the Weierstrass substitution so that one has the rational parametrization of the ellipse, and the problem is reduced again to the solution of algebraic equations.

---

Here is a Mathematica demonstration:

With[{a = Sqrt[2], b = 1, th = Pi/6}, 
 Block[{xa = Sqrt[a^2 Cos[th]^2 + b^2 Sin[th]^2], 
        ya = Sqrt[a^2 Sin[th]^2 + b^2 Cos[th]^2]}, 
       ParametricPlot[
         RotationMatrix[th].{a Cos[u], b Sin[u]}, {u, 0, 2 Pi}, 
         Epilog -> {AbsoluteThickness[4], 
         Line[{{-xa, -ya}, {xa, -ya}, {xa, ya}, {-xa, ya}, {-xa, -ya}}]}
      ]]]

You don't need calculus here, just a little geometry.

Let $c$ be the (positive) $x$-coordinate of the vertical tangent, and let $d$ be the (positive) $y$-coordinate where the rotated ellipse meets the $y$-axis. Apply a vertical shear transformation, of the form $(x,y) \mapsto (x,y+ux)$, to transform the rotated ellipse into an ellipse with horizontal major axis $c$ and vertical minor axis $d$. This transformation is area-preserving, so $\pi cd = \pi ab$. Thus $c = ab/d$.

To find $d$, take the equation of the rotated ellipse (filched from J.M.'s answer):

$$\frac{(x\cos\theta-y\sin\theta)^2}{a^2}+\frac{(x\sin\theta+y\cos\theta)^2}{b^2}=1$$

and set $x = 0$:

$$\frac{(d\sin\theta)^2}{a^2}+\frac{(d\cos\theta)^2}{b^2}=1$$

Hence

$$\frac{d^2}{a^2b^2}(b^2\sin^2\theta+a^2\cos^2\theta) = 1$$

giving

$$c = \frac{ab}{d} = \sqrt{b^2\sin^2\theta+a^2\cos^2\theta}$$

Similarly, using a horizontal shear transformation, the $y$-coordinate of the horizontal tangent is $\sqrt{a^2\sin^2\theta+b^2\cos^2\theta}$.