Do I really need the N-MOS driver here?

The P-Channel MOSFET needs to switch up to 20V.

And that's the catch, when making the MCU output high impedance, you could have 20 V at that MCU's output. You would be violating the maximum ratings of the MCU.

But actually you would not get to 20 V at that output as the ESD protection diodes in the MCU will pull down that voltage (to roughly Vdd + one diode forward voltage or some safe clamping voltage, for example 6 V) so your PMOS will never be off.

Do not operate the MCU pin the way you propose. Applying 20V to a pin that nominally operates from 0 to 3.3V is operating the device outside of recommended guidelines.

Any statement in the data sheet that may say you can inject 1mA through the outputs protection diodes would be of a sort that says you may be able to get away with that momentarily but for normal operating practice it is foolhardy to do this. There is aging of the silicon to consider but even greater concern is the potential for MCU circuit latchup and resulting catastrophic silicon damage that can occur when the current injection happens when the MCU is powering up or down or even in it's off state.

So NO; do not be foolhardy for the sake of a minuscule resistor and a transistor that is at most a SOT-23 package.