Prime/undecomposable matrices

For a different point of view, you might like to take a look at Section 12.5 and Appendix A in the free on-line version of the following book (in which you'll find some interesting open questions related to "prime matrices" of the type you described):

N. Pytheas Fogg, Substitutions in dynamics, arithmetics and combinatorics, Lecture Notes in Mathematics, vol. 1794, Springer-Verlag, Berlin, 2002.

I would suggest to have a look at books on fuchsian groups, e.g. Beardon, "the geometry of discrete groups", or S. Katok, "fuchsian groups".

They don't talk about primitive matrices in your sense, but really, that's just because that's not the terminology used in the field.

For example, the fact that any integer matrices can be expressed as a product of powers of finitely many elements, is a consequence of the following general result.

The group $SL_2(Z)$ has finite covolume in $SL_2(R)$. That means that the volume of $SL_2(Z)\backslash SL_2(R)$ is finite, and that implies that $SL_2(Z)$ is finitely generated: there exists a finite number of generators such that any element in $SL_2(Z)$ is a product of positive powers of these generators. There may be several ways to express an element in term of these generators, though.

Any discrete subgroup of $SL_2(Z)$ with finite covolume is in fact finitely generated. Uniqueness of the decomposition is not always granted, one has to dive a little deeper into the structure of $SL_2(Z)$ to understand why that holds in that particular case. Let me point out two remarkable properties of that group that ultimately explain why there is uniqueness. First the two standard generators are conjuguated to their inverses, and that allows to take positive powers in the decomposition. And second, $SL_2(Z)$ contains a subgroup of index six which is a free group, so that uniqueness holds trivially for that subgroup. That subgroup is the set of integer matrices that are congruent to the identity modulo 2.

So I think you can get much insight from these books, even if they don't talk about primitive matrices per se.