How to proceed when PhD advisor leaves academia or moves to a new university midway?

One issue is what your advisor is doing. If your advisor is leaving academia (e.g., going to work on Wall Street or for the government), then you are probably on your own, but moving to another university is generally not a big problem.

If you haven't started doing research yet, you should probably just switch to a new advisor, but if you are already making progress, then staying with your current advisor is often the best approach.

One possibility that wasn't mentioned in your question is moving physically to the new university as a visiting student while staying enrolled at the old university. You still work on research with your advisor, and at the end receive a degree from the university you started at. Maybe it varies between fields, but this is pretty common in mathematics, and it avoids some of the difficulties like transferring credit or different requirements.

If your advisor is supportive (and they should be!), then this is generally not hard to arrange. It's easiest if they are technically going on leave from the old university, rather than resigning immediately, but that's a pretty common way to arrange moving between universities, partly because it simplifies situations like this. (And even if your advisor is not going on leave, you just need a colleague to step in as the formal advisor, while letting your advisor handle the day to day interaction.) The major reason for difficulty would be if there was some serious problem at the old university, such as a personality conflict with the department chair, which could make the department unwilling to be flexible.

I went through this experience, and I have friends who've gone through this experience, and it's never fun. You ask a lot of questions; I'll try to answer as many as I can.

1. What should be the student's considerations when the professor offers him a choice to move along with him?

   Consider the following:

   1. How far along are you in your work? If you haven't proposed yet, it's probably easier just to find a new professor and start anew. If you haven't done any serious research yet (1-2 yrs), definitely find a new advisor and start fresh.
   2. Did you have a good relationship with this person? Do you want to continue working with them?
   3. Often, your credits will not follow you. (I don't have a source for this statement, other than I've been told by numerous people that graduate credits rarely transfer between institutions.) Make sure they will transfer, or that you will be given some sort of pass, before transferring.

   Other stuff (rankings, location, collaborators) should obviously be taken into account as well. In my experience, most students do not move along with their advisor.

2. Do universities offer coursework waiver and faster-than-usual graduation for students accompanying new professors?

   Almost certainly not.

   Can papers published in the older university be considered a part of the thesis that will be written for the newer department?

   Probably, talk with the university before transferring.

3. What should a student do if such an option is not available to the professor and also there are not other professors in the department who could/are willing to guide the student? Will the university offer a compensation for him?

   That's pretty unusual. This happens all the time; people are familiar with the situation. In many cases, the department will be willing to help you find someone new. You should view the ordeal as identical to when you chose your initial advisor; you'll probably do (shortened) interviews with a few profs, talk to lab members, look into research, etc. The difference is that, by now, you should be familiar with those people who do research similar to what you've been doing, so your search will be easier; you'll know them, and they should know you, even if only because you've taken a class with them or something.

   You will almost definitely not receive compensation. C'est la vie, my friend... welcome to the real world.

I generally agree with @AnonymousMathematician that the easiest thing to do may simply be to visit the New University while staying an enrolled student at Old University. It may require checking with both departments, but then again so will and out and out transfer.

Another option that hasn't been mentioned:

It may be possible to convert your current advisor to an "outside reader" or simply a committee member who happens to have an appointment at another institution. The formal title of your "advisor" can then be switched to a member of your committee at your current institution - preferably one sympathetic and supportive of your situation.