$A$ and $A^2$ have same characteristic polynomial

Yes: consider $$A=\begin{bmatrix}\omega&0\\0&\omega^2\end{bmatrix}$$ where $\omega$ is a primitive 3rd root of unity.

Let $a$ and $b$ are eigenvalues of $A$ then eigenvalues of $A^2$ are $a^2$ and $b^2$. You need

$t^2-(a+b)t+ab=t^2-(a^2+b^2)t+a^2b^2$

So $ab=a^2b^2\implies ab=1$ as $A^{-1}$ exists.

And $a+b=a^2+b^2\implies a+1/a=a^2+1/a^2\implies a^4-a^3-a+1=0$

$\implies (a-1)(a^3-1)=0\implies a=1$,$\omega$,$\omega^2$

Discard $a=1$ as $A\ne I$ and diagonalizable. Discard the case of both eigenvalues equal as $A$ is not diagonalizable for repeated eigenvalues. The only possibility is $\{\omega,\omega^2\}$ as the eigenvalue set for $A$. So $$A=\begin{bmatrix}\omega&0\\0&\omega^2\end{bmatrix}$$