Why do students go for a PhD even when chances of getting academic jobs on its completion are slim?

The issue is that the frame of your question is wrong. You’re assuming that the purpose of a PhD is to get an academic job, when that is not the case: Many people who get PhD’s have no desire for an academic job, even from the time they apply!

The number of academic positions is indeed insufficient to absorb all the PhD’s. But there are lots of other “landing spots”:

• Industrial research and development
• Start-ups
• Non-university government research institutes (like the US NIH or DOE lab system, or the Max-Planck-Instituts in Germany)
• Finance and consulting
• Administrative and Supervisory positions in other disciplines.

Overall, PhD’s have a lower unemployment rate than non-PhD’s. It’s all a question of where they want to go.

As for doing a PhD, the basic skill it teaches you—the one that makes you valuable to lots of people—is the ability to learn new fields quickly and become an expert in those fields. You don’t always need specialized knowledge in your PhD topic to get into an area.

But ultimately the reason for doing a PhD is the love of research and discovery. People pursue research careers because that’s what they want to do.

Some of my observations:

1. They don't know what they're going into. Most PhD students have some idea of how hard it is to get a job afterwards, but don't actually know. It's similar to how one can imagine what skydiving is like, but don't actually know until after trying it.
2. They're confident they can succeed. PhD students are some of the smartest of their generation. They were top of their high school class, top of their undergraduate class, they've never met an obstacle they can't overcome. Why can't they also top their graduate class, top the postdoc chain, and top the applicants for tenured positions?
3. The process worked for their supervisors, which makes them think it'll also work for the students. The supervisors are probably the people most influential in convincing undergraduates to do graduate studies. These are the people who've already did all of the things in #2. If they can do it, so can their students, and hence they encourage their students to try.
4. They don't know what they want to do. They haven't really thought about future careers and only have a vague idea of how their studies enable them to find a job if the academic path fails. The hidden reason they did undergraduate studies is because that's what everyone who did well in high school went on to do, and the same applied for graduate studies. "The best undergraduates do PhD studies, so I'll do it too".
5. They have romantic ideas about what people with PhDs do. When scientists talk about what they do, they don't usually say "I write funding proposals". The not-exactly-accurate answers propagate and influence the next generation of students to try to become scientists themselves.

Because it's fun!

No, really. I went to do a PhD because I thought it was fun. It allowed me to live in an awesome location, travel around the world to conferences and summer schools, to spend years doing exciting research with nice colleagues, and even getting paid for all of it (and in Sweden, the pay is not bad at slightly above the national median salary, plenty for a sufficiently large apartment (compared to an undergraduate dormitory) where I did my PhD).

I postponed growing up and thinking about the future for after the PhD, which means now, 4½ years into a couple of postdocs.