Is this group of matrices cyclic?

The group is indeed cyclic and is generated by the matrix $$ J=\left( {\begin{array}{cc} 1 & 1 \\ 0 & 1 \\ \end{array} } \right) $$

As you have for $n \in \mathbb Z$

$$ J^n =\left( {\begin{array}{cc} 1 & n \\ 0 & 1 \\ \end{array} } \right) $$

if we define: $$ B=\left( {\begin{array}{cc} 0 & 1 \\ 0 & 0 \\ \end{array} } \right) $$ then $$ A = I+B \\ B^2 = 0 $$ so by the binomial theorem, for $n \ge 0$: $$ A^n =(I+B)^n = I+nB $$ note also that: $$ (I+B)(I-B) = I-B^2=I $$ so $$ A^{-1}=I-B \\ A^{-n}=(I-B)^n =I-nB $$

Hint:

Compute the product $\begin{pmatrix}1&n\\0&1\end{pmatrix}\begin{pmatrix}1&p\\0&1\end{pmatrix}$.