Text suggestion for linear algebra and geometry
In my opinion, having a basic knowlegde of algebra (Axler is very good, for sure), I would bet on learning different small topics from different books, because it is rather difficult to find everything in one book. If you want to have deeper insight in linear algebra with applications to geometry, I would suggest you to study the following (advanced) topics:
Tensor, symmetric and antisymmetric vector spaces (that is, given vector spaces $V$ and $W$, to construct the objects $V\otimes W$, $Sym^r(V)$, $\bigwedge^n V$) together with a review of basic algebra from a higher point of view. For this item, I have found an extremely good source, Multilinear Algebra and Applications. It will help you have a solid ground on linear algebra, being a quite nice book.
Basic Lie Theory: I strongly reccomend you Naive Lie Theory. It is a must for someone wishing to relate linear algebra and geometry. Exceptional good book.
(Optional) Clifford Algebras: this is a bit more difficult but nevertheless useful if you have time enough, at least to have a rough idea. Please read the article here and then try to read the first 20 pages of Spin Geometry.
I think this is a good planning to study in summer. The references are nice and easily readable, but also having a higher level.
I spent nearly 3 years looking for an understanding as to how linear algebra related to geometry and how this approach was supposed to unify the subject, I have looked at every single one of the books mentioned here and none of them answered my question.
The best versions of what all these books are trying to do (in terms of relating all of linear algebra to geometry) are two rare books, one by Fekete "Real Linear Algebra", whose introduction absolutely must be read to get a sense on how to view linear algebra as a whole, and another by Dieudonné "Linear Algebra and Geometry", which is extremely geometric in spirit, uses very rigorous notation and whose introduction stresses the distinction between affine and metric properties of Euclidean geometry,
Comments such as Artin's famous comments on linear transformations, or the notion that a determinant should be interpreted as a homothety of volumes (best done in Dieudonné!) etc... make it seem like some deep unified view of linear algebra in terms of geometry exists, e.g. that something that explains all the theorems with pictures may exist. Similarly chapters such as "Unitary Geometry" in Weyl's 'Group Theory and Quantum Mechanics' would only lead one to push harder in finding some unified interpretation of linear algebra, so who wouldn't want to check every good reference?
I've since found there is only a partial answer to this question, and the answer is Gelfand, once you have general relativity and quantum mechanics to actually guide you into seeing this. You can basically view the first chapter (on vector spaces and inner product spaces) as developing a geometric formalism, modelled on putting a vector space into a curved space (manifold), applicable to general relativity (and Euclidean geometry by extension), and the second chapter (on operators and linear transformations) as developing an algebraic formalism, modelled on complex numbers and polynomials (which Axler also mentions, as I'm sure you've read) mainly applicable to quantum mechanics (remember QM is not going to demand pretty geometric interpretations! Hence the importance of discarding the necessity for geometric interpretations here, and it unifies the subject when one does this!).
In rough overview, in Ch. 1 you first begin discussing the affine geometry of parallelism through the concepts of linear independence, bases, changing bases & isomorphisms (refer to Dieudonne for a lovely reason to see why you can think of this as affine geometry, more generally it follows Klein's view of affine/Euclidean/projective geometry as a geometry invariant under parallel/orthogonal/central projections, but there is a more technical sense in which the word Affine is used in linear algebra (see below) so be careful), then we add perpendicularity via an inner product to get Euclidean geometry (or Hermitian geometry if you want complex numbers) and discuss the general principles, then we strictly focus on orthogonal geometry (i.e. w.r.t. an orthogonal basis) via Gram-Schmidt & least squares and stuff. After this linear, bilinear and quadratic forms enter the picture, which is just another (general) way to talk about non-orthogonal (curvilinear) geometry, it just allows you to do Euclidean, Hermitian, Minkowskian & Symplectic geometry in one fell swoop using the same ideas, and then the final sections discuss Lagrange's & Jacobi's methods for reducing a quadratic form to a sum of squares, which is just another way of saying Einstein's equivalence principle, i.e. that locally at any point of spacetime we can work as if we are working in an orthogonal geometry, but globally this will not be true, i.e. we can diagonalize our metric (a quadratic form) locally using these methods, but globally it will not hold! (The very last section is on Hermitian geometry, i.e. doing all this stuff over the complex numbers). Thus the picture is all motivated by imagining putting a vector (tangent) space to a curved manifold and invoking the equivalence principle locally. You can ignore all this and pretend we're studying the algebra for it's own sake as the other books do, but you don't get a unified motivation/explanation for what you're doing that way...
There is a similar way to naturally motivate all the Jordan normal forms, eigenvalues, adjoint, self-adjoint, normal, unitary etc... concepts of linear algebra & I'll write a big explanation up if you like this description so far.
Based on all this, I can now mention a more advanced version of Gelfand & Shafarevich, namely Kostrikin, who has an exercises book does the affine stuff the way it's usually done & really does link operators with geometry (!), but I recommend this view as a secondary outgrowth/application of the algebraic interpretation to actually carrying out the Euclidean, Hermitian, Minkowskian & Symplectic geometry I mentioned above, because it doesn't unify the subject the way thinking about complex numbers/polynomials does. Basically you can view sections from chapter 2 of Kostrikin as like a chapter 3 of Gelfand mixing your ideas together, or perhaps just extended versions of his section on bilinear forms.
Thus, I recommend a mix of Dieudonné, Fekete, Gelfand & Kostrikin + exercise book. Hope this helps!
For more on the geometry, take a look at Berger's wonderful two-volume text Geometry. It's a very sophisticated treatment of classical geometries (not differential geometry), full of linear algebra. You might also look at Pedoe's beautiful book, Geometry, A Comprehensive Course, which uses multilinear algebra as well as linear algebra. There's also the beginnings of Lie groups, so you could look at Curtis's Matrix Groups.
Perhaps if you added a bit more focus to your question, we might be able to help you a bit more.