Self study Control Theory

I have a background in EE and I am studying control theory, therefore I have some different insight than someone from pure or applied mathematics.

Let me preface my answer with a warning: Most people are turned on by the IDEA of studying complicated subjects, but these are the same people who are turned off actually having to do the work.

You see these questions on Mathstackexchange all the time, people listing dozens of books each 700 pages long and asking whether if it is feasible to go through them during the course of their undergrad. I think engineering students in general are more prone to burn out while studying if the material is not tethered to applications.

To gain a basic background in control theory for application or research here are the courses that are must:

1. The basics: Linear algebra, complex analysis, multivariable calculus, ODE
2. Frequency domain control theory, and State Space control theory
3. Nonlinear dynamical system
4. Linear operator theory

These are the prerequisites that most control books will contain or teach you such as one by Sontag, Dullerud, Sastry, Khalil, Vidyasagar.

To do more advanced work you need a course on (usually in grad school):

1. Optimization
2. Real analysis and topology
3. Classical mechanics
4. Probability and random processes

These will cover optimal control, control of time varying systems, robotics and stochastic control and coming up with new results. There are other fields that I haven't even mentioned and requires (more) sophisticated tools like finite automaton, machine learning, etc.

Now the question is which field do you want to apply your control in. This opens to the door to biology, quantum mechanics, circuit theory, signal and image processing

Hope this helps.

From a purely mathematical perspective, I think taking courses (with the Math department at your university, not the engineering department) in Real Analysis, Complex Analysis, Functional Analysis, Abstract Algebra, Algebraic Topology, Smooth Manifolds, Lie Theory, Dynamical Systems and Differential Geometry should really set you up for the mathematics to do graduate level work in control theory.

Admittedly that is a lot of courses, but since you have already taken some, you could probably skip the first three. As for the rest, you can take some of them now and the others in your first year of graduate school.

Of course there also controls courses in Linear Systems, Non-linear and Adaptive Control, Robust Control, Geometric Control, etc. but for those, the offerings might differ from university to university, and it really depends on what area of control theory you want to work in.