Why is the inverse matrix of $A^TA$ is guaranteed to exists?

Edit: After adding the rank condition, see this post: Prove rank $A^TA$ = rank $A$ for any $A_{m \times n}$

If you can prove that $\text{rank}( A^T A) = \text{rank}(A) = m$, you get that $A^T A$, being an $m \times m$ matrix now, has full rank and hence is invertible.

Here is a counter example for the quadratic case:

There, it is not guaranteed to exist. Consider the matrix $$ A = \begin{pmatrix} 0 & 1 \\ 0 & 0 \end{pmatrix}, $$ which is obviously not invertible. Then $$ A^T A = \begin{pmatrix} 0 & 0 \\ 1 & 0 \end{pmatrix} \cdot \begin{pmatrix} 0 & 1 \\ 0 & 0 \end{pmatrix} = \begin{pmatrix} 0 & 0 \\ 0 & 1 \end{pmatrix} $$ is also not invertible.

In fact, using the determinant formulas $$ \det A^T = \det A $$ and $$ \det AB = \det A \det B $$ you get that $$ \det A^T A = \det A^T \det A = (\det A)^2 $$ and this is not equal to zero if and only if $\det A \neq 0$, i.e. $A$ is invertible.