Can Mathematica do symbolic linear algebra?

Initially, Mathematica is not designed for such abstract calculations.

But, Mathematica is a powerful programming language, so that one can add such functionality easily.

See the following examples in related area of differential geometry:

• calculations in symbolic dimensions
• Abstract calculations

Indeed this is a one liner in NCAlgebra:

<< NC`
<< NCAlgebra`
NCGrad[1/2 (x ** z - y)^T ** (x ** z - y), z]

which results in

-y^T ** x + z^T ** x^T ** x

I am not sure, but maybe this software for Mathematica http://www.math.ucsd.edu/~ncalg/ could somehow help. The software is for a package called NCAlgebra developed by UC San Diego. I am not familiar with the detailed usage, but it claims to implement capability to study noncommutative inequalities, linear controls, and semidefinite programming within Mathmeatica.