mathematical maturity

I thought I was hopeless at applied maths when I graduated from my undergrad. But after 3 years of graduate research, I still won't say I am good at it, but at least I can actually read papers in reasonable time now.

If you keep doing it, you will get better as you acquire more practice and experience. E.g. 1 year ago I was completely hopeless at cooking, but after stubbornly trying to cook over and over again, I can actually produce some tasty dishes. Thinking back, I wonder why I was ever stuck in these two activities. Same pattern happens for all my other hobbies.

In general, I have three broad advice:

1) Consistency

Keep doing it and do it often. Being good at something is about doing it a lot over a long period of time.

2) Repetition

Keep repeating the same thing. E.g. I used to be really bad at baking scones. Every time I failed, I would try to figure out why I failed and experiment with fixes. After 7 batches of sad looking squashed scones, I fixed all my mistakes and is now able to consistently bake nice looking scones!

3) Don't Delve on Specific Details

In my humble opinion, the biggest thing that has been holding me back from all my activities (mathematics or otherwise) was my inability to move ahead. I tend to get stuck trying to solve specific problems.

I find that it is much better to move on and make as much progress as possible, then come back to the stuck part later to try at it again. If I am unable to resolve it, I would move on again and then come back later. E.g. if you get stuck at a part of the paper, it might help to move on and read the rest of the paper. Or even put this paper aside and read another one.

A mathematics specific advice: it helps a great deal if you have supervision and/or feedback from a professor.

Not having a foundation in proofs based classes (like analysis and abstract algebra) is a drawback because they teach you the basic tools/arguments/definitions typically used to prove other things in higher math. How important is all of this? It depends in part on what your career goals are. If you want to do mostly research or work in academics, esp at a good institution, then it will probably be important that you improve your background, esp given your comments about your past experiences trying to learn upper-level material. If you work in industry, maybe it isn't important. In my case, where I want to work in statistics in the private sector, I got to know other people already working in the field, and found that many of them did not have nor did they really need many of the upper level math grad courses. Maybe you can similarly survey people in your field. If you do need to remedy your background, consider a Master's program, where you can take some of these courses, en route to your PhD or whatever job you plan to go into. It is not so easy to get a good job with just a BS in math, so this is probably not a bad route for you anyhow. Good Luck

Since you are interested in machine learning:

For machine learning, what you need is a good understanding of linear algebra, multivariate calculus, probability, algorithms/complexity and perhaps some convex optimization.

Depending on the algorithms you are interested in, convex optimization will probably be the most challenging subject, but you usually won't need deep understanding of this subject to understand the algorithms that use this (e.g. support vector machines). Convex Optimization by Boyd et al. is a great free book on the subject, although it contains much more than you need to understand machine learning.

Algorithms and complexity can also be complicated if you are not used to dealing with these things. There are several good books on algorithms, Introduction to Algorithms by Cormen et al. is probably the most famous.

Finally, depending on the subject, some of the probability material can be quite advanced, for example in probabilistic graphical models. Again there are many great books on the subject (e.g. Probabilistic Graphical Models by Koller et al.).

BTW for all of these you can find free online video courses.