Finding the Number of Zeros of a Function in a Given Annulus

Problems of this type which come from books are often vulnerable to some meta-mathematical arguments. One basic principle is that, in order to apply Rouché's theorem, you are never expected to count the number of zeros of any polynomial which you cannot solve explicitly. Also, in the actual application of Rouché's theorem, you'll rarely need to use anything more complicated than the triangle inequality to establish the required inequality. In practice this means that you're hoping to show something like

$$ |\text{complicated}| < |\text{simple}|. $$

Another guiding rule is that larger powers of $z$ are smaller in the unit circle and smaller powers of $z$ are smaller outside of the unit circle when compared to the rest of the polynomial.

For this particular problem, if I wanted to know how many zeros are inside the unit circle, I would guess that the dominating polynomial $q(z)$ in the inequality $|p(z)| < |q(z)|$ is either $2$ or $10z+2$ or $10z$, since smaller powers are more important in the unit circle. For convenience we'll try $10z$ first.

The polynomial $10z$ has one zero at $z=0$ which happens to lie inside the unit circle. Now on $|z| = 1$ we have

$$ \begin{align} \left|z^6-6z^2+2\right| &\leq 1+6+2 \\ &= 9 \\ &< 10 \\ &= 10 |z| \\ &= |10z|, \end{align} $$

and we can conclude from Rouché's theorem that $z^6 - 6z^2 + 10z + 2$ has exactly one zero in the disk $|z| \leq 1$. If you'd like you can check that it's also true that

$$ \left|z^6-6z^2\right| < |10z+2| $$

on the unit circle, so we could have gone that way instead.

To find the number of zeros in $|z| \leq 2$ we'll start from the other end and hope that $z^6$ dominates on the circle $|z| = 2$. Note that $z^6$ has a zero of multiplicity $6$ at $z=0$ (so, effectively, it has $6$ zeros), which happens to lie inside the circle $|z| = 2$. On $|z| = 2$ we have

$$ \begin{align} \left|-6z^2+10z+2\right| &\leq 6 \cdot 2^2 + 10 \cdot 2 + 2 \\ &= 46 \\ &< 64 \\ &= 2^6 \\ &= \left|z^6\right|, \end{align} $$

so from Rouché's theorem we know that $z^6 - 6z^2 + 10z + 2$ has all $6$ zeros in the disk $|z| < 2$.

Hence the number of zeros of the polynomial in the annulus $1 < |z| < 2$ is $6-1 = 5$.