What should I do if the first author wants me to write the whole paper?

First of all: Good call to talk about authorship issues at the beginning, rather than only once troubles started! Now authorship issues do occasionally have to be re-negotiated depending on the course the project takes.

There do seem to be potentially two separate aspects in your question:

A. What to do if I have contributed as much as I am willing as a secondary author, yet the first author keeps asking for more? - Point out that you have done X, Y and Z, and that you are only willing to do significantly more if you are the first author (fundamental reading + commenting remain unaffected of course).

B. I started a side-project, and I no longer want to work on this AT ALL. - Make clear to your co-author that you have no intention to work on this project anymore, and that this pertains to the future, too. Either give them free rein to continue the project without you (either with you on the author list or not, depending on your preferences); OR make sure you are still available for basic reading/commenting, but simply nothing more extensive.

In both cases, if the coauthor shows a significantly stronger reaction than "medium annoyance", then maintaining a positive relationship to them is futile anyway.

Since I suppose bad language is not allowed here, I'll be restrained. Suffice to say, as an academic I think that is a basic professional principle is lacking here: if you, with existing career, wage and pedagogical duties, are writing with a student, they should be first author. Always. Every time. No matter what.

Two reasons: first, you already have all the power and privileges. Students are not there to be consumed. Second, you don't want to be seen to be exploiting someone who is vulnerable to your power over them.

Short answer: this person sounds like a dick. Put them off, finish your PhD, get a job, then tell anyone following on after you to avoid writing with them.

The professor is putting it off until you present your dissertation. What happens then? Are you still employed by your institution?

If you have an position somewhere else, congratulations! Just tell him that you have other obligations, and that you will be available for comments and discussion, but your new employer would want you to work on their projects.

If you are going to remain at your institution, and don't have anything else that will pay your bills, you could use this project to improve your CV. I don't know in your field, but only two conference publications sounds on the weaker end (in technical fields in Sweden people usually finish with four first author journal papers after four years). If you don't have anything else (yet), you can ask him to hire you. If funding is not available, you may decide to work with him anyway, but letting him know that your main priority is finding a new job, and you will leave as soon as someone offers to pay you. You may also decide not to work for free, and you could tell him that.

In the case that you decide to actually continue with the project, you should discuss authorship again. You have done most of the job so far, and you have the work he wants you to do as a leverage. You could appeal to his humanity (as if professors had any!) and say that you are in a critical point in your career and need to strengthen your CV.