Must eigenvalues be numbers?

Of course. The definition of an eigenvalue does not require that the field in question is that of the real or complex numbers. In fact, it doesn't even need to be a matrix. All you need is a vector space $V$ over a field $F$, and a linear mapping $$L: V\to V.$$

Then, $\lambda\in F$ is an eigenvalue of $L$ if and only if there exists a nonzero element $v\in V$ such that $L(v)=\lambda v$.

Eigenvalues need to be elements of the field. The most common examples of fields contain objects that we usually call numbers, but this is not part of the definition of an eigenvalue. As a counterexample, consider the field $\mathbb R(x)$ of rational expressions in a variable $x$ with real coefficients. The $2\times 2$ matrix over that field

$$\left(\begin{matrix} x & 0 \\ 0 & \frac1x \end{matrix}\right)$$

has eigenvalues $x$ and $1/x$: not unknown numbers, but known elements of the field $\mathbb R(x).$

Very interesting question. Consider real valued column "vectors" which are matrices $2N \times 2$, each 2x2 block on the form $\left[\begin{array}{rr}a&-b\\b&a\end{array}\right]$. This is a famous representation for complex numbers $a+bi$. So if we have a block-matrix on the same form we can find "eigenvalues" which will be 2x2 blocks complex numbers (in the same block-representation). This idea can then be extended to more advanced types of "numbers". This leads to the idea that more advanced "numbers" themselves are matrices - if the elements are "simpler" numbers in some sense. Instead of an eigenvalue decomposition with respect to the simplest numbers, we get a block-eigenvalue decomposition. If you are interested in this I would encourage you to read more about Representation Theory where more advanced types of numbers are written as matrices with elements of a less complicated type of number. For instance the field of quaternions can in turn be written as a matrix of $2 \times 2$ complex numbers - and since those in turn are $2 \times 2$ real numbers we could see them as a matrix with $4 \times 4$ real numbers. So we can get a "hierarchy" of more or less "advanced" numbers, depending on which size of matrix blocks we look.