How to deal with an advisor that offers you nearly no advising at all?

I second Nate's suggestion to look at https://academia.stackexchange.com, there are already many similar questions (with answers, some of them specifically from mathematicians) on that site that may help you.

But since "go somewhere else" is not exactly the kind of answer someone in your situation needs, here are a few thoughts from a random person on the internet:

First, it sounds to me as if your PhD is actually going rather well: You've already obtained independent results, written papers, and initiated fruitful collaborations on your own. If that is not what you should demonstrate for this degree, I don't know what is. (Of course it could -- always! -- be going better, and the experience could be more pleasant for you.) So I wouldn't worry about your chances of graduating.

In fact, one possible (but certainly not the only) explanation is that your advisor is thinking the same thing: "They're doing well on their own, they don't need my help, and it's better for their career if they're working independently anyway." (Of course, this can also be a convenient rationalization of laziness or poor time management on their end...) If this is the case, I'd sit down with them, explain to them that you in fact do need their help, and negotiate exactly what kind and on what schedule. If that doesn't work (and you haven't graduated by then), switching advisors or getting a formal co-advisor (either in the same department or a different university) is certainly not unheard of.

Finally (and this is the reason I am writing this answer now), you write

And I also feel that this whole situation damages me mentally to a point where I frequently get anxiety attacks.

It's completely normal to have doubts and frustrations during your PhD (and the timing seems about on schedule for it, as well), but this is a strong emotional response that you should take seriously and seek help dealing with. Here I don't necessarily mean professional help (although there are certainly professionals that can help with this), but finding a trusted person you can talk to about these issues on a regular basis to prevent them from building up. (Here especially, https://academia.stackexchange.com can give you much better recommendations since this is something that happens in all disciplines.)

In the German tradition a PhD advisor is called a "Doktorvater/Doktormutter" --- a thesis parent. It might help to think about your relationship with your advisor along those lines. Many parents like to leave their kids on their own to explore the world, and see their role as "cheer leader" --- your biggest fan. From what you describe, "He gave this thesis the best grade possible", I do get the impression that your advisor admires you and is supportive, while letting you explore the world of mathematics on your own. And you definitely seem to be making good progress in that exploration, initiating collaborations, co-authoring papers. These are all signs of a great scientific maturity, you can be proud of how far you have come. Whether you continue in academia or not, these accomplishments will stand out on your CV.

If your advisor would "force" you to work on problems that do not interest you, I would definitely recommend you to seek another advisor. But you seem to have an advisor who lets you pursue your own research interests and the scientific environment in which you find yourself does seem to be intellectually stimulating. Perhaps it's true that you are climbing a mountain via the steepest route, but that may well be the most rewarding route.

Whatever you do, don't quit because of your supervisor. If you want to quit, quit, but not because of your supervisor, maths is too good for that!

For what it's worth, I also struggled with a supervisor offering no help (actually no help would have been better, the little that my supervisor did say was always counterproductive and often simply wrong). I don't know if it's a German thing or if it's institute dependent (I was in Göttingen), maybe a bit of both. I'll try give a longer reply later with more concrete things that helped me and so hopefully can help you, but for now just take my first two sentences! :)