How do you prove that $tr(B^{T} A )$ is a inner product?
For every $A=(A_{ij}) \in \mathbb{R}^{m\times n}$ we have $$ \langle A,A\rangle=\text{tr}(A^TA)=\sum_{i=1}^n(A^TA)_{ii}=\sum_{i=1}^n\sum_{j=1}^mA^T_{ij}A_{ji}=\sum_{i=1}^m\sum_{j=1}^nA_{ij}^2 \ge 0, $$ and $$ \langle A,A\rangle=\sum_{i=1}^m\sum_{j=1}^nA_{ij}^2 = 0\iff (A_{ij}=0 \quad \forall i,j) \iff A=0 $$ Since $$ \text{tr}(X^T)=\text{tr}(X), \quad \text{tr}(X+Y)=\text{tr}(X)+\text{tr}(Y), \quad \text{tr}(\lambda X)=\lambda\text{tr}(X) $$ for every $X,Y \in \mathbb{R}^{n\times n}$, and $\lambda \in \mathbb{R}$, therefore, for every $A,B, C \in \mathbb{R}^{m\times n}$, and $\lambda \in \mathbb{R}$ we have \begin{eqnarray} \langle A,B\rangle&=&\text{tr}(B^TA)=\text{tr}((B^TA)^T)=\text{tr}(A^TB)=\langle B,A\rangle,\\ \langle \lambda A+B,C\rangle&=&\text{tr}(C^T(\lambda A+B))=\text{tr}(\lambda C^TA+C^TB)=\lambda\text{tr}(C^TA)+\text{tr}(C^TB)\\ &=&\lambda\langle A,C\rangle+\langle B,C\rangle. \end{eqnarray}
If $A=(a_{ij})$ and $B=(b_{ij})$ and $C=B^TA=(c_{ij})$ then $$(c)_{ij}=\sum_{k=1}^m b_{ki}a_{kj}$$
$$\mathrm{tr}(B^TA)=\sum_{i=1}^n c_{ii}=\sum_{i=1}^n\sum_{k=1}^m b_{ki}a_{ki}$$ so we see that $\langle.,.\rangle$ is an inner product (Euclidian) by identifying $\mathcal M_{m\times n}(\mathbb R)$ to $\mathbb R^{m\times n}$.
You want to verify all the properties of a real inner product (since we're looking at a real vector space). Using your notation $\langle A,B \rangle = \mathrm{tr}(B^T A)$, we want to check that:
- $\langle A,B \rangle = \langle B,A \rangle $ (symmetry)
- $\langle cA + B, C \rangle = c \langle A,C \rangle + \langle B,C \rangle$ (linearity)
- $\langle A, A \rangle > 0$ unless $A = 0$ (definiteness)
You'll need to use some properties about the trace in order to prove some of these conditions hold.