What's the difference between a potentiometer and a rheostat?

The correct term for the common terminal of a potentiometer is the slider.

A rheostat is simply a variable resistance used to control power to a load and you are correct about the wiring. Only the slider and one other terminal are used.

A potentiometer uses all three terminals, enabling a variable voltage or signal to be tapped off from the slider.

Potentiometers and rheostats are made the same way, but rheostats are usually much "beefier", as they are generally used in high-power situations.

The slider is often connected to one or other terminal for safety reasons, in case it loses contact with the track.

One difference not yet mentioned between devices intended for rheostats versus those intended for use as pots: if a device will be used as a rheostat, it is important that the wiper resistance be fairly small, and even more important that it be 'well-behaved'. If the device will be used as a pot, and the amount of current flowing through the wiper will be minimal, wiper resistance is relatively unimportant. A 100-ohm rheostat with a wiper resistance of 10 ohms in some spots and 1,000 ohms in other spots would be totally useless. A 100-ohm pot with such behavior could be just fine, however, if it was being used to drive a high-impedance input. Note that such a pot would be roughly equivalent to a 1,000-ohm pot in parallel that had a 110-ohm fixed resistor tied between the ends.

A Rheostat is used to vary the amount of current in the circuit but a potentiometer used to vary the voltage between the second terminal and one of the outside terminal