What is the dimension of a matrix?

The dimension of a vector space is the number of coordinates you need to describe a point in it. Thus, a plane in $\mathbb{R}^3$, is of dimension $2$, since each point in the plane can be described by two parameters, even though the actual point will be of the form $(x,y,z)$.

If you take the rows of a matrix as the basis of a vector space, the dimension of that vector space will give you the number of independent rows. For a vector space whose basis elements are themselves matrices, the dimension will be less or equal to the number of elements in the matrix, this $\dim[M_2(\mathbb{R})]=4$

The following literature, from Friedberg's "Linear Algebra," may be of use here:

---

Definitions. A vector space is called finite-dimensional if it has a basis consisting of a finite number of vectors. The unique number of vectors in each basis for $V$ is called the dimension of $V$ and is denoted by $\dim(V)$.

---

A basis, if you didn't already know, is a set of linearly independent vectors that span some vector space, say $W$, that is a subset of $V$.