If every eigenvalue of $A$ is zero, does this mean $A$ is a zero matrix?

No, any strictly upper triangular matrix, such as:

$$\begin{pmatrix}0&1\\0&0\end{pmatrix}$$

will have all eigenvalues zero.