How can I construct a nilpotent matrix with the property $A^2 \not= 0$ but $A^3=0$

This is only possible in a space of dimension $\ge k$. In such a space take the matrix

$$N_k=\begin{pmatrix} 0&1&0& \dots&0\\ 0&0&1& \dots&0\\ 0&\vdots&\ddots& \ddots&0\\ 0&0&\dots& 0&1\\ 0&0&0& \dots&0\\ \end{pmatrix}$$