Why on-shell vs. off-shell matters?
It's important to distinguish them because on-shell and off-shell are opposite to each other, in a sense. On-shell describes fields that obey the equations of motion and real particles; off-shell describes fields that don't have to obey the equations of motion and virtual particles.
On-shell are momenta with $p^2=m^2$ with the right $m^2$ for the given field; off-shell are momenta that don't obey the condition.
Amplitudes with external particles that are on-shell, i.e. on-shell amplitudes, express the scattering amplitudes and may be directly used to calculate cross sections etc. Off-shell amplitudes i.e. amplitudes with external off-shell momenta encode much more general correlators.
In some theories, i.e. quantum gravity i.e. string theory in the flat space, only on-shell amplitudes for the external particles such as gravitons may be calculated. On the contrary, the analogous quantities to these on-shell amplitudes in the AdS space may be expressed by off-shell correlators in the corresponding CFT.
It's always important to know whether the 4-momenta etc. we are attaching are obliged to be on-shell or not, i.e. whether they're on-shell or off-shell. If we substitute off-shell momenta to on-shell-only formulae, we get meaningless or ill-defined results. If we only substitute on-shell momenta to off-shell formulae, we miss a significant portion of the physical information.
I) The meaning of the word on-shell depends on context. We are aware of at least three meanings.
The original meaning of the word on-shell refers to (as Lubos Motl writes in his correct answer) that the mass-shell condition $$ p_0^2 - p_1^2-p_2^2-p_3^2 ~=~m_0^2 $$ is satisfied for a 4-momentum $p_{\mu}\in \mathbb{R}^4$. The mass-shell is therefore a 3-dimensional hyperboloid in 4-dimensional energy-momentum space $\mathbb{R}^4$.
Nowadays more generally, the word on-shell is also used in the sense that equations of motion of the system are satisfied.
In constrained systems, the word on-shell refers to the constrained surface of configurations of the dynamical variables that obey the constraints.
II) The word off-shell originally means not on-shell. (Note however that the word off-shell sometimes effectively means not on-shell and on-shell. E.g. an off-shell formulation also describes physics on-shell.)