Relationship between inner product and norm

In $\mathbb{R}^n$ and $\mathbb{C}^n$ we can actually present a characterization of all inner products. Fix an inner product $\langle \cdot , \cdot \rangle$, probably the Euclidean one. Then for any positive definite matrix $P$, $(x,y) = \langle x,Py \rangle$ is also an inner product. Here positive definiteness is defined in terms of $\langle \cdot , \cdot \rangle$.

Conversely, any inner product on a finite dimensional space over $\mathbb{R}$ or $\mathbb{C}$ has such a representation. In particular, given an inner product $(\cdot , \cdot)$, we have $P_{ij}=( e_i,e_j )$, where $\{ e_i \}_{i=1}^n$ is an orthonormal system with respect to $\langle \cdot , \cdot \rangle$. This gives the representation of $P$ in terms of the orthonormal coordinates given by $\langle \cdot , \cdot \rangle$. If the base inner product is the Euclidean one, then $\{ e_i \}_{i=1}^n$ can be taken to be the standard basis, in which case this is the usual representation.

The fact that that every inner product induces a norm is nearly a simple consequence of the definition of an inner product. Specifically, $\| x \| \geq 0$ with $\| x \| = 0$ iff $x = 0$ is built completely into the definition. $\| a x \| = |a| \| x \|$ is also trivial, because $\| a x \| = (ax,ax) = a \overline{a} \| x \|^2$, then $|a| = \sqrt{a \overline{a}}$. The tricky part is the triangle inequality. This is typically proven with the Cauchy-Schwarz inequality.


Yes, there are many different types of inner products. Consider the inner product on $L^2$ given by $\langle f, g \rangle = \int f(x) \overline{g(x)} dx$.

An inner product $\langle , \rangle$ always defines a norm by the formula $||x||^2 = \langle x, x \rangle$. You can check that all the conditions of a norm are satisfied. However, the converse is not true, that is, not every norm gives rise to an inner product. Norms which satisfy the parallelogram law can be used to define inner products via the polarization identity.


The weighted sum formula for the inner product, $$(x,y) = \sum_i w_i x_i y_i,$$ captures the inner product's essential meaning - in finite dimensions all inner products act like a weighted sum in some basis (also the idea can be generalized to infinite dimensions using the spectral theorem, the sum basically becoming an integral)

Why? Well by definition inner products are symmetric positive definite bilinear forms, so on finite dimensions they always have a representation by a symmetric positive definite matrix, $$(x,y) = x^T M y.$$

Taking the singular value decomposition $M = U \Sigma U^T$ (or you could do eigenvalue decomposition since it's a SPD matrix), that gives $$(x,y) = x^T U \Sigma U^T y$$

So, if $x_i,y_i$ are the components of $x,y$ in the basis of the singular vectors of $M$ then you can write the inner product in a weighted sum type form $$(x,y) = \sum_i \sigma_i x_i y_i,$$

where the $\sigma_i$'s are the singular values from the diagonal matrix $\Sigma$ .


In terms of norms, the unit balls for a norm induced by an inner product are ellipsoids, with axes given by the singular vectors, and axis lengths determined by the singular values. So in a more geometric sense, there is a direct correspondence between ellipsoids and inner products.