How does a "permanent" magnet with switch work?

The mechanical button lowers or raises the magnet, which changes the effective force between it and the steel plate that the base attaches to. You are correct, one can not turn a permanent magnet off this way, but one can greatly change the force between it and a magnetic material by changing the distance.

After looking it up I have to correct myself. One particular design doesn't actually lower or raise the magnet but it turns a cylindrical magnet in an assembly between two magnetic jokes. This is a very important difference to my original suggestion, which would never completely eliminate the force between the magnet and the base plate. In the design shown in https://en.wikipedia.org/wiki/Magnetic_base it's the change in symmetry that can essentially completely cancel the field. It's quite ingenious, really.