Understanding of the theorem that all norms are equivalent in finite dimensional vector spaces

To answer the question in the update:

If $(X,\|\cdot\|)$ is a normed space of infinite dimension, we can produce a non-continuous linear functional: Choose an algebraic basis $\{e_{i}\}_{i \in I}$ which we may assume to be normalized, i.e., $\|e_{i}\| = 1$ for all $i$. Every vector $x \in X$ has a unique representation $x = \sum_{i \in I} x_i \, e_i$ with only finitely many nonzero entries (by definition of a basis).

Now choose a countable subset $i_1,i_2, \ldots$ of $I$. Then $\phi(x) = \sum_{k=1}^{\infty} k \cdot x_{i_k}$ defines a linear functional on $x$. Note that $\phi$ is not continuous, as $\frac{1}{\sqrt{k}} e_{i_k} \to 0$ while $\phi(\frac{1}{\sqrt{k}}e_{i_k}) = \sqrt{k} \to \infty$.

There can't be a $C \gt 0$ such that the norm $\|x\|_{\phi} = \|x\| + |\phi(x)|$ satisfies $\|x\|_\phi \leq C \|x\|$ since otherwise $\|\frac{1}{\sqrt{k}}e_k\| \to 0$ would imply $|\phi(\frac{1}{\sqrt{k}}e_k)| \to 0$ contrary to the previous paragraph.

This shows that on an infinite-dimensional normed space there are always inequivalent norms. In other words, the converse you ask about is true.

You are going to need something of this nature. A Banach Space is a complete normed linear space (over $\mathbb{R}$ or $\mathbb{C}$). The equivalence of norms on a finite dimensional space eventually comes down to the facts that the unit ball of a Banach Space is compact if the space is finite-dimensional, and that continuous real-valued functions on compact sets achieve their sup and inf. It is the Bolzano Weirstrass theorem that gives the first property.

In fact, a Banach Space is finite dimensional if and only if its unit ball is compact. Things like this do go wrong for infinite-dimensional spaces. For example, let $\ell_1$ be the space of real sequences such that $\sum_{n=0}^{\infty} |a_n| < \infty $. Then $\ell_1$ is an infinite dimensional Banach Space with norm $\|(a_n) \| = \sum_{n=0}^{\infty} |a_n|.$ It also admits another norm $\|(a_n)\|' = \sqrt{ \sum_{n=0}^{\infty} |a_{n}|^2}$ , and this norm is not equivalent to the first one.

Here's a proof of the equivalence of norms in finite dimension that does not use compactness arguments. It breaks down into three steps.

We set some notation first: given a normed finite dimensional real vector space $E$, write $E'$ for its algebraic dual space (set of all linear forms) and $E^{*}$ for its topological dual space (set of all continuous linear forms).

• All linear forms on $E$ are continuous, i.e $E'=E^{*}$, whatever the norm $\|.\|$ on $E$.

The Hahn-Banach theorem states that every continuous form on a subspace of $E$ can be extended to all of $E$. In particular, given $x\in E$, there is a $\varphi:E\rightarrow \mathbb{R}$ for which $\varphi(x)\neq 0$, since there are non trivial forms $\mathbb{R}x\rightarrow \mathbb{R}$ and these can be extended. As a consequence, the map \begin{align*} E &\rightarrow (E^{*})' \\ x &\mapsto (\phi\rightarrow \phi(x)) \end{align*} is injective. We also have isomorphisms $E'\simeq E$ and $E^{*}\simeq (E^{*})'$. Composing the three maps we have an injection $E' \rightarrow E^{*}$ so that $\dim E' \leqslant \dim E^{*}$. On the other hand since $E^{*}\subset E'$ we must have $\dim E^{*}=\dim E'$ and $E^{*}=E'$ as desired.

• All linear maps between finite dimensional normed spaces are continuous.

Let $A:F\rightarrow E$ be such a map and $e_{i},i\in I$ a basis for $E$. Let $(e_{i}^{*})$ be the dual basis of $(e_{i})$ and $\tilde{e_{i}}$ the linear map $\mathbb{R}\rightarrow E$ that maps $1$ to $e_{i}$. Then $Id_{E}=\sum_{i\in I}\tilde{e_{i}}e_{i}^{*}$. In particular, $A=\sum \tilde{e_{i}}e_{i}^{*}A$. The the $\tilde{e_{i}}$ are continuous and so are the $e_{i}^{*}A$, since they are linear forms. Hence $A$ is continuous as a sum of composites of continuous maps.

• All norms are equivalent.

Given two norms $\|.\|_{1}, \|.\|_{2}$ on $E$, the identity, being linear, defines an homeomorphism between $E,\|.\|_{1}$ and $E,\|.\|_{2}$. It is easy to check that this means that the norms are equivalent.

This proof is really a way of saying that the topology induced by a norm on a finite-dimensional vector space is the same as the topology defined by open half-spaces; in particular, all norms define the same topology and all norms are equivalent. There are other ways to prove that using the Hahn-Banach theorem.