Is it safe to bypass an LDO by shorting its input and output?

As far as I know this is actually a fairly common solution. I know I've used this setup in the past without any problems.

This application note from LT even shows how to do this automatically, using a FET as the bridge: 5V to 3.3V Regulator with Fail-Safe Switchover – Design Note 82.

Not a good idea bypassing, it will just sit there and eat power.

Instead, use a single LD1117A in adjustable mode, and switch out the resistor.

^{simulate this circuit – Schematic created using CircuitLab}