What does a positive definite matrix have to do with Cauchy-Schwarz Inequality?

As a complement to the other answers, I just wanted to note that reasoning a bit more abstractly you can get this result from the ordinary Cauchy-Schwarz inequality for free, without any computation.

The Cauchy-Schwarz inequality is valid in any inner product space (Euclidean vector space if you are working over the real numbers). In a given an inner product space (such as $\Bbb R^n$ with the standard inner product), what a positive definite symmetric linear operator $B$ (given by a positive definite symmetric matrix in the case of $\Bbb R^n$) gives you is precisely another inner product on the same space, defined by $(x\mid y)_B\overset{\rm def}=(x\mid By)=( Bx\mid y)$. Now the Cauchy-Schwarz inequality holds for this inner product as well: $$ (x\mid y)_B^2 \leq (x\mid x)_B ~~(y\mid y)_B \qquad\text{with equality only if $x \parallel y$} $$ (the notation $x \parallel y$ here means the vectors $x,y$ are linearly dependent, one being a scalar multiple of the other). Now taking $\def\b{\mathbf b}x=\b$ and $\def\d{\mathbf d}y=B^{-1}\d$, this becomes $$ (\b\mid B^{-1}\d)_B^2 \leq (\b\mid \b)_B ~~(B^{-1}\d\mid B^{-1}\d)_B \qquad\text{with equality only if $\b \parallel B^{-1}\d$.} $$ which expanding the definition of $(\mid )_B$ simplifies to $$ (\b\mid\d)^2 \leq (\b\mid B\b) ~~(\d\mid B^{-1}\d) \qquad\text{with equality only if $\b \parallel B^{-1}\d$,} $$ which is your "extended" Cauchy-Schwarz inequality.


The sentence you underlined in red abbreviates these steps:

$$(\mathbf{B}^{1/2} \mathbf{b} \cdot \mathbf{B}^{-1/2} \mathbf{d})^2 \leq (\mathbf{B}^{1/2} \mathbf{b} \cdot \mathbf{B}^{1/2} \mathbf{b})(\mathbf{B}^{-1/2} \mathbf{d} \cdot \mathbf{B}^{-1/2} \mathbf{d}) $$

This is the Cauchy-Schwarz inequality applied as requested. Now we use the transpose-based definition of the dot product $(\mathbf{a} \cdot \mathbf{b}) = \mathbf{a}^T \mathbf{b}$:

$$(\mathbf{b}^T (\mathbf{B}^{1/2})^T \mathbf{B}^{-1/2} \mathbf{d})^2 \leq (\mathbf{b}^T (\mathbf{B}^{1/2})^T \mathbf{B}^{1/2} \mathbf{b}) (\mathbf{d}^T (\mathbf{B}^{-1/2})^T \mathbf{B}^{-1/2} \mathbf{d})$$

Finally we take advantage of the the following observations:

  • $\mathbf{B}$ is positive definite and therefore symmetric
  • $\mathbf{B}^{1/2}$ is also positive definite and symmetric
  • The inverse of a positive definite matrix is positive definite

As a result:

$$(\mathbf{B}^{1/2})^T \mathbf{B}^{-1/2} = \mathbf{B}^{1/2} \mathbf{B}^{-1/2} = \mathbf{I}$$

$$(\mathbf{B}^{1/2})^T \mathbf{B}^{1/2} = \mathbf{B}^{1/2} \mathbf{B}^{1/2} = \mathbf{B}$$

$$(\mathbf{B}^{-1/2})^T \mathbf{B}^{-1/2} = \mathbf{B}^{-1/2} \mathbf{B}^{-1/2} = \mathbf{B}^{-1}$$

Completing the proof.

You can see how this relates to the regular Cauchy-Schwarz inequality (for the dot product) by considering what it states when $\mathbf{B} = \mathbf{I}$ (Since $\mathbf{I}$ is a positive definite matrix). In this case we have

$$(\mathbf{b} \cdot \mathbf{d})^2 \leq (\mathbf{b} \cdot \mathbf{b}) (\mathbf{d} \cdot \mathbf{d})$$

Which is precisely the usual Cauchy-Schwarz inequality for the dot product.


Isn't CSI $x'y\leq \sqrt{x'\cdot x}\sqrt{y'\cdot y}$? Square it to get $$(x'\cdot y)^2\leq (x'\cdot x)(y'\cdot y)$$ Now apply this inequality by letting the vectors $x=B^{1/2} b$ and $y=B^{-1/2}d.$ You get $$b'\cdot d \leq (b'Bb)(d'B^{-1}d)$$ as required.