Positive definiteness of difference of inverse matrices

For ease of notation I'll use $\ge$ instead of $\succeq$ and so on.

Lemma 1: $A \le B \Rightarrow C^T A C \le C^T B C$ for any conformable matrix $C.$

Proof. We have $x^TC^T(B-A)C x = (Cx)^T(B-A)C x \ge 0$ for any conformable vector $x$ so that $C^T(B-A)C \ge 0.$

Lemma 2: $I \preceq B \Rightarrow$ $B$ is invertible and $B^{-1} \preceq I.$

Proof.

• Since $B$ is symmetric we can find an orthogonal matrix $G$ and a diagonal matrix $D$ such that $B=GDG^T.$ Hence $I = G^T I G \le G^T B G = D.$ Thus all eigenvalues of $B$ are $\ge 1 >0,$ hence $B$ is invertible.
• Now write $B^{-1} = B^{-1/2}IB^{-1/2} \le B^{-1/2}BB^{-1/2} = I,$ where $B^\alpha := GD^{\alpha}G^T.$

Proposition: $0 < A \le B \Rightarrow$ $B$ invertible and $B^{-1} \le A^{-1}.$

Proof. \begin{align*} A\le B &\Rightarrow B-A \ge 0 \\ &\Rightarrow A^{-1/2}(B-A)A^{-1/2} \ge 0\\ &\Rightarrow A^{-1/2}BA^{-1/2} \ge I\\ &\Rightarrow A^{1/2}B^{-1}A^{1/2} \le I,\\ \end{align*} hence \begin{align*} B^{-1} &= A^{-1/2}A^{1/2}B^{-1}A^{1/2}A^{-1/2}\\ &= A^{-1/2}(A^{-1/2}B^{-1}A^{-1/2})^{-1}A^{-1/2}\\ &\le A^{-1/2}IA^{-1/2}\\ &= A^{-1}. \end{align*}

HINT:

Try proving that if $A$ is positive definite and $B-A$ is non-negative definite, then $\det(B-\lambda A)=0$ has all its roots $\lambda\ge 1$ and conversely if $\lambda\ge1$, then $B-A$ is non-negative definite. (it's an exercise from a book by the way)

Then, $\det(A^{-1}-\mu B^{-1})=0$

$\Rightarrow \det B\det(A^{-1}-\mu B^{-1})\det A\ge0$ $\quad($ as $A$ and $B$ are both p.d., $\det A,\det B>0)$

$\Rightarrow \det(B-\mu A)=0$

So the roots of $\det(A^{-1}-\mu B^{-1})=0$ are the same as the roots of $\det(B-\mu A)=0$.

As $B-A$ is n.n.d. we have $\mu\ge1$, which in turn makes $A^{-1}-B^{-1}$ n.n.d. (by the claim above)