Conjugacy classes of $\mathrm{SL}_2(\mathbb{Z})$
One can proceed as follows for $SL_2(\mathbb{Z})$.
- First, the trace is a conjugacy invariant.
- For trace $0$ there are two conjugacy classes represented by $\pmatrix{0 & 1 \\ -1 & 0}$ and $\pmatrix{0 & -1 \\ 1 & 0}$. These representatives can be thought of as $90^\circ$ and $270^{\circ}$ degree rotations of a lattice generated by the corners of a square centered on the origin.
- For trace $1$ and $-1$ there are two conjugacy classes each, represented by the matrices $$M=\pmatrix{1 & -1 \\ 1 & 0}, M^2=\pmatrix{0 & -1 \\ 1 & -1}, M^4=\pmatrix{-1 & 1 \\ -1 & 0}, M^5 = \pmatrix{0 & 1 \\ -1 & 1} $$ These representatives can be thought of as $60^\circ$, $120^\circ$, $240^\circ$, and $300^\circ$ degree rotations of a lattice generated by the vertices of a regular hexagon centered at the origin.
- For trace $2$ there is a $\mathbb{Z}$-indexed family of conjugacy classes, represented by $\pmatrix{1 & n \\ 0 & 1}$; these are all "shear" transformations except for the identity. For trace $-2$ there is a similar $\mathbb{Z}$-indexed family of conjugacy classes represented by $\pmatrix{-1 & n \\ 0 & -1}$.
- In general, for nonzero trace the conjugacy classes come in opposite pairs, represented by a matrix $M$ with trace $t>0$ and an opposite representative $-M$ with trace $-t<0$.
- For trace of absolute value $> 2$, there is one conjugacy class for each word of the form $$\pm R^{j_1} L^{k_1} R^{j_2} L^{k_2} \cdots R^{j_I} L^{k_I} $$ up to cyclic conjugacy, where $I \ge 1$ and all the exponents are positive integers. A matrix representing this form is obtained from the above word by making the replacements $$R=\pmatrix{1 & 1 \\ 0 & 1}, \quad L=\pmatrix{1 & 0 \\ 1 & 1} $$ The transformations represented by such words are all "hyperbolic" transformations, having an independent pair of real eigenvectors. The slope of the expanding eigenvector is a quadratic irrational, and hence has eventually repeating continued fraction expansion. The cyclic sequence $(j_1,k_1,j_2,k_2,\ldots,j_I,k_I)$ can be thought of as the fundamental repeating portion of the continued fraction expansion of the slope of the expanding eigenvector, or, better, as an appropriate power of the fundamental repeating portion where the power is equal to the exponent of the given matrix.
Number theorists will tell you that the number of conjugacy classes of each trace $t>2$ is closely related to the class number of the number field generated by $\sqrt{t^2-4}$.
This is the subject of Gauss' reduction theory, as discussed in Karpenkov's book (among many other places). In this 2007 paper, Karpenkov also extends the method to $SL(n, \mathbb{Z}).$
The conjugacy classes of elements of ${\rm SL}(2,\mathbb{Z})$ with given trace are counted in:
S. Chowla, J. Cowles and M. Cowles: On the number of conjugacy classes in SL(2,Z). Journal of Number Theory 12(1980), Issue 3, Pages 372-377.