Is the Square Root of an Inverse Matrix Equal to the Inverse of the Square Root Matrix?

It is generally true that if $A$ is an $n\times n$ invertible and if $A^{-1}$ has a "square root" $C$, also $n\times n$, such that:

$$ A^{-1} = C^2 $$

then $C^{-1} A^{-1} C^{-1} = I$ holds.

The first fact we need is that since $A$ is invertible, $A^{-1}$ is invertible, and this implies $C$ is invertible. For if not, then there would exist a nonzero vector $x$ in the nullspace of $C$, and $Cx=0$ would imply $A^{-1}x=0$, contradicting the invertibility of $A^{-1}$. Thus $A = (C^2)^{-1} = (C^{-1})^2$.

The second fact we need is that a one-sided inverse of a matrix is a two-sided inverse, so that:

$$ A C^2 = I \; \implies \; C A C = I $$

That is, using associativity of matrix multiplication, the left hand side tells us $(AC)C = I$, so that $C$ is a (right) inverse of $AC$. Thus it must also be a (left) inverse of $AC$, which is what the right hand equation states.

Finally by the same reasoning:

$$ C A C = I \; \implies \; C^{-1} A^{-1} C^{-1} = I $$

In this discussion/proof we have not invoked the symmetry of $A$ nor the uniqueness of a symmetric positive definite square root $C$ for $A^{-1}$, which also would be symmetric positive definite. The reasoning above is correct even if $A$ is not symmetric, and even if $C$ is not positive definite, and relies only on $A^{-1} = C^2$.