How can I get feedback for my work in math if I'm not in academia?

Nowadays, you can get feedback and answers to questions from places like MathStackExchange, and MathOverflow. Most of the responses are (in my observation) either entirely competent or quickly corrected by others. Good quality control. No, that wouldn't be the same as having a good advisor (ideally a world-class scholar on your subject) that you talk to in person every week. Still, these sites can give guidance.

Also, quite a few of the more-senior people who like to contribute to such sites may be amenable to direct email contact, etc.

(Grad school is not about "taking notes in lectures" per se, but is about having access to the informal observations of very good mathematicians, and being able to ask them questions immediately, in the moment.)

Disclaimer: I am a mathematician, but I do research in CS.

My observation is that all mathematics, but especially research-grade mathematics has a lot of features of a folklore. It is much easier when someone explains it to you, even more if informally. Reading books helps, of course, but for me a quick informal explanation done in five minutes could replace a tedious drilling through formalism for understanding for few days. Of course, you'd still need to be able to get through the formal and completely correct description. But it's much easier, when you already have an informal understanding.

To give an example, the determinant is actually an oriented volume of the n-dimensional parallelotope spanned with matrix component vectors. Compare this with a definition from almost any undergrad book on linear algebra.

I’ll give an answer based on personal experience. My situation is not exactly like yours, but may be similar enough to provide some insight.

I recently completed my undergrad in physics and applied math. Upon finishing this I decided that I was more interested in pure math, and wanted to pursue this in grad school. I ended up not getting into any of the Ph.D programs I wanted (probably because of a lack of pure math experience), so I was left in limbo, so to speak, still wanting to study math but not yet able to enroll on a graduate program.

My solution was to email some professors at a local university who I had found doing things somewhat related to what I was interested in. I told them my situation and explained that I was just looking to learn more about their research and math in general. This ended up working out and now I attend regular seminars at the university and I meet up with them every once in awhile to just discuss math.

I have found it very helpful to be able to have face to face conversations with other mathematicians, even though I do most studying on my own at home. Though Stack Exchange is a valuable resource, I don’t think it can adequately replace this interaction.

I’d like to point out that I am not sure if my experience is typical. After all, this is a somewhat small university that I am talking about, and it may be harder to find professors at larger universities who are able to take extra time to talk to you (I certainly found this to be the case at UC Berkeley, which is where I did my undergrad). Still, I think that this is an option you may be interested in considering.