Prove that if $A$ is invertible matrix then $AA^T$ and $A^T A$ are also invertible.

Let B be a matrix, so that $AB = I$ and $BA = I$. Then $A^T B^T = (BA)^T = I^T = I$ and $B^T A^T = (AB)^T = I^T = I$. So the inverse for $A^T$ is $B^T$. Now we get :

$(A A^T)(B^T B) = A(A^T B^T) B = A(I)B = AB = I$ and $(A^T A)(B B^T) = I$. So the inverse for $AA^T$ is $B^TB$ and for $A^T A$ we have $B B^T$.

Using your notation, you've shown that $(AB)^T = I$, so far so good; but also $(AB)^T = B^TA^T = (A^{-1})^TA^T$, and this tells you that $A^T$ is invertible and $(A^T)^{-1}=(A^{-1})^T$. Can you see where to go with this?

Recall a matrix is invertible if and only if its determinant is non-zero.

Now what can you say about

A) $\text{det}(A^T)$

B) $\text{det}(AB)$?

Work these out, and it becomes clear