Is there a lack of oversight of how professors interact with students?
Much abuse in the academic world is directed against students. They are in an ambiguous position. They are often not considered working professionals, though in practice they are. I.e. they often teach and/or conduct research. They are usually not rewarded financially in line with their training and skills. They cannot easily move to another place till they have completed their degree. Also, they are more dependent on their mentor/supervisor than a working professional would typically be. And they are not accorded the rights that working persons normally are. Students are also less likely to complain because they need to have good relations with the faculty for their future careers. Often they are working with faculty members on projects. While similar situations/circumstances can arise outside academia, this combination is relatively unusual. The closest analogy I can think of is the institution of interns, who probably also endure abuse, though their stays are usually much shorter. Anyway, this is fertile ground for abuse.
While this is not directly relevant, it's symptomatic that there is often much controversy/difficulty when students try to start Unions to protect their rights. This has been an issue for a good many years, at least in the United States. There was a famous case at Yale some years ago, for example.
Additionally, the institution of tenure is quite rare outside academia, though it exists. For example, judges have tenure. Tenured professors are hard to fire, and therefore are more likely to get away with abuse without consequences.
There is another issue, which is more speculative. But I'll mention it, anyway. This may also be country dependent. Foreigners entering Western countries, at least, are subject to employment restrictions. There is one notable exception to this restriction - namely educational institutions, which means universities in practice. This may not be universally true, but for example in the US, being a student is probably the easiest way to enter the country, and the main exception to the H1B work visa cap, the last I heard, were educational and non-profit institutions.
Anyway, in practice this means there are many foreigners in Western universities, both at a student and post-doc level, and of course at faculty level. Here I am only concerned about junior temporary employees like students and post-docs. In the US, at least, there is much institutional discrimination against such persons. They can't easily move to another job or university because of employment restrictions. Getting a non-university job is often very difficult. Getting employed involves an additional burden of paperwork. So, such people are, again, targets for abuse. This is not theoretical - I've seen and heard much to support this. Of course, foreigners get employment outside academia too, but for the reasons I have discussed, I believe the density of foreigners employed is particularly high in Western universities.
To supplement a great answer by Faheem Mitha:
I would add that there are professions with very structured career (e.g. in medicine, law and military), and problems there are at least as severe. In general, the thing is a about the guild system, where the only way to become a professional is to get acceptance of a small community of professionals (cf. free jobs, where anyone can start and it is the market who decides).
Such systems have good quality control (i.e. there is no way around internal checks), but also generate huge gap between the already established people and the newcomers (which is an ample room for abuse - conflict with supervisors may not only jeopardize one's position in a particular company or institute, but the whole career).
As a side note, internships in the guild system are not only meant to teach, but also to control supply, so to maintain prices or prestige.
The question makes a salient point, namely why it keeps happening (to whichever degree) even though everyone knows that it happens.
I think there are two answers to this question:
Professors don't really have any kind of training with regard to human resource management. As a professor, at least at research intensive universities, you are hired and promoted almost exclusively based on your research credentials. You may be managing a group of 20 grad students and postdocs, but almost never will these managers have gone through any kind of training that would teach them how to actually do this -- neither from the operational viewpoint, nor from the point of view of how to manage the humans that make up your group. It is certainly true for me that I have felt unsure how to deal with situations in my own research group many times. There is no formal structure in universities where you have to go through (or could even choose to go through) any kind of training that would prepare you for being in the position of a group leader before you get into it. Consequently, many professors essentially wing it day in day out and "shit happens". (The same happens, by the way, within the professorial ranks -- there is no training you can take before you become a department head.)
Among colleagues, it is usually reasonably well known which of the professors treat their students well and which don't. But what are you going to do about it? Once someone has tenure, the only stick you have is to open post-tenure proceedings or to have formal hearings to revoke someone's tenure because of abuse of students. Unless a professor sexually assaults a student, it is almost inconceivable to think of evidence that would stand in the court of law upon which a university could base revoking someone's tenure. So it isn't done. A department head might talk to a professor in a case where students keep complaining, but as there are no formal training programs or requirements for professors (see above), there is in fact very little that can be done in practice. (I'm not trying to make this into an excuse, it's just a cold hard look at the realities of a university, sad as it is in these cases.)