How do you remove the buzzer from this PCB whose solder side is inaccessible?

You could quieten the buzzer by filling the hole in the top of the plastic. If you pour in something rubbery (hot glue could be good) then it would almost certainly muffle the sound.

To remove it, I'd crack the plastic with some pliers removing enough to heat the legs with a soldering iron.

I know the OP has probably solved this problem, but this is for anyone who might land here through google like I did.

I just disabled a piezoelectric buzzer exactly like this one.

Looking at the first picture included in the OP's post, the buzzer is the black cylinder with the hole at the top. See that seam running all around the cylinder? Take a very small flat head screwdriver, insert it into the seam, and wiggle the top off.

Beneath that is a shiny silver disk. Lift that out and throw it away. You'll see copper wire beneath that, but there's no need to touch that.

Lastly, snap the top of the cylinder back on. Now you'll barely be able to hear the buzz.

There may be an alternative choice. These devices are often resonant, and very loud at resonance. Putting a piece of cellophane tape over the opening may upset the resonance and cause it to be so quiet that you no longer need to remove it. And of course, the process is reversible.