Why does hot plugging blow stuff up, and how to prevent it?

Notice the two pins on both sides are longer, and the two in the center are shorter. This ensures connections are made in the proper order (and also broken in the proper order when unplugging).

If the connector is not designed for hot plugging, there is no such guarantee.

The order you want is:

• First, Ground/shield.

This ensures both sides agree on what "0V" is, and also discharges any static electricity safely. Sometimes a tiny spark is visible. You do not want to connect the ESD sensitive pins first!

• Second, power supply.

• Third, signals

Order is very important. You really want to avoid applying voltage to the signal pins of an unpowered chip, as current will then flow through the ESD protection diodes, and the chip will be powered from its IO pins. This can damage the chip.

Also, if ground connects last, then the signal lines will act as ground instead, and current will flow in them. If the device contains 3V3 chips powered by a LDO from the +5V from USB and ground isn't connected, who knows what the voltages inside the device are going to be...

An excellent example of how NOT to do it are audio RCA connectors.

Notice how the tip makes contact first. I'm sure you've done this before. The loudspeakers let out a very loud hum, until the grounds are connected.

why do things not like being hot plugged?

It's because the pins connect in the wrong order.

Since you mention an ESC, I guess you have voltages and currents large enough to fry some chips. In this case not connecting the ground first can really hurt...

is there an easy way I protect againest this?

Use a connector that is hotplug-safe. If it does not carry the power supply, only signals and ground, then you could get away with large value resistors on the signal lines instead... but it's a hack.

Unfortunately these connectors are very uncommon. Headers like the ones used with arduino's are designed to be a part of a finished product that will only be plugged during manufacture, so they will not be hotplug-safe.

Hotplug-safe connectors will be available for the usual standards (USB, HDMI, whatever) but this will not be what you need for your application.

So, I guess you're stuck doing it carefully, powering down before messing with the circuit...

Hot swaping is bad for a few reasons:
1) If you connect the Vcc before ground current can flow into your circuit in anomalous ways. For example if Vcc is connected and a digital or analog pin before ground, power can flow into Vcc and out of the pin potentially short circuiting that pin and burning out that part of the circuit.

2) Can cause a "brown out" from the voltage dropping temporarily on a system bus or power supply.

3) Upon disconnect inductors in the circuit or cables can exhibit high voltages if disconnected.

(I had a cable that was not hot-swappable in a product that service reps would hot swap on accident. Because of the mutual inductance in the cable (and an improper cable design with straight wires running next to each other for a meter or so) it would blow out the digital drivers on both sides of the cable. Upon further inspection it was discovered that when the cable was disconnected a digital CMOS line would rise to 7V!)

I have also had great success in implementing both strategies below. One thing you can do in design if your rolling your own hot swap system is find a standard connector (I used sca2 for my bus, but you could use sata, or another industry standard connector, just make sure people understand they can't plug other stuff into it).

Precharge Circuitry:

A long pin and current limiting resistor may be used to limit inrush current to a device. The long pin mates first; the current limit must be set so that the host system power rails stay within specification, but the device charges up adequately before the power and signal pins make connection. Care must be taken when choosing a pre-charge resistor value, the following scenarios show some common problems: If the pre-charge resistor value is too small, the device will still draw too much current on insertion, causing the system power rails to drop out of regulation.

Hot swap controller

A hot swap controller IC controls inrush current to a device. Hot swap controllers typically incorporate electronic fusing, and in high current applications it may be difficult to distinguish between inrush current and a short circuit. The components are more expensive than pre-charge resistors and in some cases the use of more active components in the system may introduce reliability concerns.

Image and text source: Design Considerations for Hot Swap

It really depends on the circuit and in some cases the connector itself.

When you disconnect or connect something, the connections do not all happen at the same time. That means there is an unpredictable connection state during the process. Some of those connections can put large voltages or large current where you really do not want them to be. Worse, connectors are usually tight, meaning the user wiggles them around to get them to come apart, creating even more random makes and breaks in the process.

Some connectors, like card edge connectors, are also notorious for shorting adjacent pins out while inserting or removing before they mate properly. One should never even think about plugging or unplugging one of these in hot.

If whatever is being disconnected does not care, for example, a two pin plug going to a LED with a simple pull-up pulldown drive, nothing bad will happen assuming you do not zap it with ESD. But most things are not that robust.

You can of course design things to be hot-pluggable, but that is complicated and expensive, and non-functional for most of the life of the product and is hard to justify if it is not a specific design requirement.

Having said that, systems should always be designed such that if the thing is powered up when sensor A is not plugged in, output B should not go into some state that depends on that sensor. If the loss of that sensor will cause a failure or danger, then appropriate measures need to be added to detect that loss and gracefully enter a safe state.

But generally, unless you REALLY know what MIGHT happen, do not hot plug!