Asking a professor to teach better

If you want to bring this up with them, you need to focus on concrete things you want done differently. "Try harder" is going to (a) antagonize them and (b) not change anything for you (except maybe for the worse). The professor, as you say, is aware that they are not putting in much effort, and it is very unlikely that they will change this habit unless forced to.

I suggest thinking about what, concretely, would help you most if done differently. If you need more in-class examples, raise the issue as "I feel the class would profit from working through more examples in class". If the inconsistent notation is a problem, ask him if it is possible to unify notation. These are not things that are an incredible effort for the teacher, so they have a realistic chance of getting addressed. Complaining and hoping that the teacher somehow sees the error of their ways isn't.

That said, there is a good chance that all complaints and requests coming from you directly to the teacher will be brushed aside. If the teacher is as badly prepared and unmotivated as you say, they are probably not the kind of person who would care deeply about student satisfaction. In that case, going one level higher (i.e., complaining to the programme manager or equivalent) is often more useful. While this will not typically improve things rapidly, it will likely improve the situation of this course over time, especially if complaints are raised from different classes (it's hard to continue to brush off student complaints if the same complaints come again and again for multiple years).

The best approach to a situation such as this is definitely not to bluntly tell him "I think you are doing a poor job teaching this course. You need to try harder."

Building rapport with this professor may go a long way to encouraging him to "do better." From the perspective of the professor, I always felt more compelled to help students have a positive experience in my class when I knew them and when they made a good faith effort to engage with me in and out of class. Perhaps you could read the material in advance of class and come prepared with some questions to ask in class. After a few times of doing this, the professor will likely come prepared to engage with students a bit more.

From the perspective of a student, I know there are (hopefully rare) times where a professor is overwhelmed with administrative responsibilities, personal issues, burn-out, etc. As a student I had to take it upon myself to learn the material and succeed in the subject. Especially as I matured in my field, I saw that the successful students in my field did not rely on a professor to take them from ground zero to the stratosphere on a subject. The professor ultimately was there to supplement and formalize our learning. As this question is tagged graduate school, I even more emphatically feel that this is the case. When you are in graduate school, you have to be to the point where you can somewhat teach yourself the material as necessary. Once you are out of graduate school, there is no longer a professor to hold your hand and lead you along to the correct solution. You need to be able to find the answers for yourself.

I say all of this with the acknowledgement that some professors are stuck in their ways and that just building rapport and supplementing your in class experience with personal study outside of the classroom will not always change how a professor teaches. Note of course that if a professor does not respond at all to his students trying to engage with him, he is extremely unlikely to respond positively to a stern lecture from students or administrators about his sub-par teaching.

In addition to the excellent answer of xLeitix here, you could also employ an independent approach. If it isn't forbidden by the rules at your university, you might want to form a study group of your peers who will jointly explore the topics of the class. You could take your joint class notes and try to merge them into something more comprehensible, for example. Law students are famous for this sort of thing, for example.

Since things like mathematics are best learned through practice, find ways to solve a lot of problems in the particular field, sharing ideas about them if possible. Take the professor as just one resource for your education.

I likewise agree with Vladhagen that a direct approach to the professor will likely be counterproductive. But you can ask for specifics, as in "I'm not getting the definition of limit. Can you say more about that?", or "Can you give some additional exercises to help me get it?" Make it about the subject matter, not the approach of the professor.

This will actually serve you well as you go on to higher level studies and need to find ways to dig deeper, even with a great professor.

Take the attitude that how he teaches is less important than that you learn.