What new insights does numerical analysis give on linear algebra?
I think the big thing you'd learn is that computing and theory are quite different, and this might cause you to look at your theoretical knowledge in a different way. In particular, computing involves issues of performance and numerical stability that are typically ignored in theoretical studies.
A couple of specific examples are:
Computing a matrix inverse is almost never a good way to solve a system of linear equations.
Calculating the zeros of the characteristic polynomial is almost never a good way to find eigenvalues. Interestingly, the reverse is true — computing eigenvalues is a very good way to find the zeros of a polynomial.
Also, I think you can learn a lot by "playing" with numerical examples, which is only feasible if you have a good toolbox of numerical methods and a convenient way to access them.
Some examples, which may appear more often in numerical analysis than in linear algebra courses.
Condition numbers of matrices. This number roughly describes how sensitive the solution of $A \mathbf x = \mathbf b$ is with respect to pertubations in $\mathbf b$. Pertubations are always present in numerical analysis, due to numerical errors or different influences. This leads to the field of pre-conditioners: How can we multiply our system, i.e. $C A \mathbf x = C \mathbf b$, such that the condition number decreases? Both systems would be treated to be equal as pure linear systems, but as numerical linear systems they differ, since their numerical solutions are different.
Constructive proofs and approximative sequences. If unique, the largest eigenvector of a matrix can be found via the recursive sequence $\mathbf x_k = \frac{A \mathbf x_{k-1}}{||A \mathbf x_{k-1}||}$. (If the starting vector $\mathbf x_0$ is not orthogonal to the eigenspace of the largest eigenvalue.) This example shows how a good intuition in linear algebra helps to build numerical algorithms. This even goes further: If the iteration is ill-conditioned in direction of the largest eigenvector, numerical errors will even improve the convergence of the iteration. This leads to new algorithms, where this property is used. This is again a property, which would less interesting in a non-numerical setting.
Estimates. A typical theorem here is the Gershgorin circle theorem, which tells you more about the distance of eigenvalues from the diagonal entries of a matrix.
Special classes of matrices. For example Hessenberg-matrices are useful to build stable algorithms for eigenvalue approximation. Some numerical analysts are pure magicians when it comes to decompositions and useful matrix calculus identities.
- Sparse matrices. For parallel algorithms, matrices should be almost diagonal, here the questions also differ from typical questions in linear algebra. How can we decrease the bandwidth of a matrix, i.e. the maximum distance of non-zero entries from the diagonal?
I have the feeling that advanced linear algebra topics are spread around in almost any mathematical branch. You can analyse matrix groups, go to infinite dimensions, etc. In this sense, numerical analysis is also an extension, with a special focus on certain topics, like for example stability or sparsity. But it is not the only way to learn more about linear algebra!