What is Overmodulation?

In the image below a amplitude modulated sine wave:

• 0% unmodulated, the sine envelope is not visible at all;
• < 100% modulation depth is normal AM use;
• 100% modulation depth, the sine envelope touch at y=0. Maximum modulation that can be retrieved with an envelope detector without distortion;
• > 100% modulation depth, "overmodulation", the original sine wave can no longer be detected with an envelope detector.

A still overview of the most important modulation depths (0, 50, 100 and 200%):

The animation was created using gnuplot using the following script:

unset xtics
set yrange [-3:3]
set samples 10000
do for [d=0:200] { plot sin(2*pi*3*x)*(1+(sin(2*pi*x/10))*d/100) title sprintf("%3i%%",d); }

AM over modulation causes the carrier wave to invert it's phase when the modulating signal has an amplitude that is above a certain level. Broadcast AM typically never does this because the complexity of an accurate demodulator is too great for the thousands and millions of receivers.

AM is just the mathematical multiplication of two signals and regular broadcast AM remains as a 2 quadrant multiplier whereas full modulation uses all four quadrants.

100% modulation is where the modulating signal drives the carrier to zero and is theoretically the maximum modulation that can be successfully demodulator by a regular AM envelope detector.

Over modulation isn't really of any significance to FM systems (unlike AM). If the modulation signal amplitude is too great, any decent frequency modulator will limit the signal so that it can't push the bandwidth of the modulated signal too wide in the frequency spectrum. In effect, the modulating signal becomes clipped.

With amplitude modulation, the carrier signal is alternately driven above and below its resting (unmodulated) value. At 100% modulation, the peaks will be driven to twice the resting power, and the "valleys" will reach zero. If the modulation level is increased beyond this value, the valleys will attempt to go below zero. Since the power level can't go below zero, the signal will "clip" instead, producing distortion. The peaks may clip as well, but that will be dependent on the capability of the power stage.

For frequency modulation, going past the designed deviation value will produce a signal whose bandwidth exceeds that for which the receiver was designed. The receiver will experience distortion because part of the signal has been filtered out, and the excessive bandwidth of the transmitted signal may interfere with stations on adjacent frequencies.