$A^{-1}$ has integer entries if and only if the ${\rm det}\ (A) =\pm 1$
The direction you've done is correct to a t and is exactly how I'd do it. As for the other direction, you might be tempted to try to reverse your argument but that definitely isn't going to go your way. Here's a hint: suppose that $A^{-1}$ has integer entries, then we know that
$$\det(A^{-1}A) = \det(I) = 1$$
but we also have that $\det(A^{-1}A) = \det(A^{-1})\det(A)$. Can you take it from here?
You did one direction well. Here the other direction:
If $A$ only contains integers then $\det(A)$ is an integer (computing a determinant only uses $+$ and $\cdot$). Assume $A^{-1}$ is also an integer matrix, then $\det(A^{-1})$ is also integer. Then note that
$$\Bbb Z\ni\det(A^{-1})=\frac 1{\det(A)}$$
implies $\det(A)\in\{-1,1\}$ because no other integers then $\pm 1$ have integer reciprocals.