Poles and Zeros in English
In short, poles and zeros are a way of analyzing the stability of a feedback system.
I'll try not to get too math-heavy but I'm not sure how to explain without at least some math.
Here is the basic structure of a feedback system:
In this form there is no gain or compensation in the feedback path, it is placed entirely in the forward path, however, the feedback portion of more general systems can be transformed to look like this and analyzed in the same way.
The transfer function in the box is called \$L(s)\$ because the analysis is often done in the Laplace transform space. Laplace transforms are similar to fourier transforms, so you can think of L(s) as a frequency response. For example, a perfect low-pass filter has \$L(s) = 1\$ for \$s\$ less than the cutoff frequency, and \$L(s) = 0\$ above the cutoff frequency.
\$L(0)\$ is the DC gain of the system. For a feedback control system, a large DC gain is desirable because it reduces the steady-state tracking error of the system.
Poles and Zeros
\$L(s)\$ is a complex-valued function. Usually the polar form \$A e^{i \theta}\$ is used; $A$ is the magnitude and \$\theta\$ is the phase. The magnitude of \$L(s)\$ is also called the gain.
The poles and zeros provide a convenient, fast way to think about the properties of \$L(s)\$. When doing a rough plot of \$L(s)\$, poles contribute -90° of phase above the pole frequency and cause the magnitude to "roll off" (decrease). Zeros do the opposite — they contribute +90° of phase and the magnitude increases. This will probably make a lot more sense looking at the pictures and the "Rules for hand-made Bode plot" section of http://en.wikipedia.org/wiki/Bode_plot.
For a system to be stable, the magnitude of \$L(s)\$ needs to drop below unity before (at a lower frequency) the phase reaches -180°. Typically some margin is required here; "gain margin" and "phase margin" are two ways of measuring how far \$L(s)\$ is from the (1, -180°) point.
As a simple example, an op-amp might have \$L(s) = \frac{10^{6}}{s}\$. In this case there is a pole at zero and no zeros. As you would expect for an op-amp, there is a large DC gain. The gain drops off as the frequency increases from DC (due to the pole at zero). According to this model, the system cannot be unstable because the phase is never less than -90°.
When reading an app note that talks about poles and zeros, you may need to figure out the general form of \$L(s)\$ for the system in question, or you may be able to draw some conclusions just from a list of poles and zeros. Adding either a pole or a zero to a system will change both the gain and phase margin; adding a pole and a zero together (at different frequencies, both below the -180° crossover) will change the gain margin but not the phase margin. Adding two zeros and two poles can create a hump in \$L(s)\$ (think bandpass filter), without changing either the gain or the phase margin.
Hope this helps. In general I would expect datasheets and app notes to suggest values for the compensation components so that the user doesn't need to analyze the stability unless there are special requirements. If you have a specific part in mind that you're having trouble using and you post a link the datasheet, I might be able to offer something.
A feedback system (like any other AC circuit) can be described using a complex function \$L(s)\$. It is called the transfer function of the system and describes all of its linear behavior.
\$L(s)\$ can be plotted as two plots: one for magnitude and and one for phase both vs. frequency (the Body plots). These plots let us easily determine the stability of the system. An unstable system gets a 180° phase shift (so a negative feedback suddenly starts to be a positive one) while still having some gain.
Every complex function describing an electrical circuit is completely defined by it's poles and zeros. If you write the function as a ratio of two polynomials of \$j\omega\$ then zeros are points where the numerator equals \$0\$ and poles are zeros of the denominator.
It is quite easy to draw Bode plots from poles and zeros so they are the preferred method for specifying control systems. Also if you can ignore output loading (because you separated the various stages with op amps) then you can just multiply transfer functions without doing all the normal circuit calculations. Multiplication of polynomial ratios means you can just concatenate the lists of poles and zeros.
So back to your question:
Check the Wikipedia page for an introduction and this tutorial for a reference on how to draw Bode plots from a list of poles and zeros.
Read a little about the practical stuff in the Laplace transform. Short version: you just calculate the circuit as with complex numbers but substituting \$s\$ where you would write \$j\omega\$. Then you find \$V_{out} \over V_{in}\$ and you have your transfer function.
From an open loop transfer function (imagine cutting the loop with scissors and putting some kind of frequency response meter in there) you draw Bode plots and verify the stability. The Feedback, Op Amps and Compensation application note is short and dense but has all the theory you need for this part. Try at least skimming through it.
A pole is a frequency where a filter resonates and would, at least mathematically, have infinite gain. A zero is where it blocks a frequency - zero gain.
A simple DC blocking capacitor, such as for coupling audio amplifiers, has a zero at the origin - it blocks 0Hz signals, that is, blocks constant voltage.
Generally, we're dealing with complex frequencies. We consider not just signals that are sums of sine/cosine waves, like Fourier did; we theorize about exponentially growing or decaying sines/cosines. Poles and zeros representing such signals can be anywhere in the complex plane.
If a pole is close to the real axis, which represents normal steady sine waves, that represents a sharply tuned bandpass filter, like a high quality LC circuit. If it's far, it's a mushy soft bandpass filter with a low 'Q' value. The same kind of intuitive reasoning applies to zeros - sharper notches in the response spectrum occur where zeros are close to the real axis.
The transfer function L(s) describing a filter's response should have equal numbers of poles and zeros. This is a basic fact in complex analysis, valid because we're dealing with linear lumped components described by simple algebra, derivatives and integrals, and we can describe sines/cosines as complex exponential functions. This kind of math is analytic everywhere. It is common to not mention poles or zeros at infinity, however.
Either entity, if not on the real axis, will appear in pairs - at a complex frequency and at its complex conjugate. This relates to the fact that a real signals in result in real signals out. We don't measure complex number voltages. (Things get more interesting in the microwave world.)
If L(s)= 1/s, that is a pole at the origin and a zero at infinity. This is the function for an integrator. Apply a constant voltage, and the gain is infinity - the output climbs without limit (until it reaches the supply voltage or the ciruit smokes). At the opposite end, putting a very high frequency into an integrator won't have any effect; it gets averaged out to zero over time.
Poles in the "right half plane" represents a resonance at some frequency that makes a signal grow exponentially. So you want poles in the left half plane, meaning that for any arbitrary signal put into the filter, the output will ultimately decay to zero. That's for a normal filter. Of course, oscillators are supposed to oscillate. They maintain a steady signal due to nonlinearities - transistors can't put out more than Vcc or less than 0 volts for output.
When you look at a frequency response plot, you might guess that every bump corresponds to a pole, and every dip to a zero, but that's not strictly true. and poles and zeros far from the real axis have effects that aren't apparent in that way. It would be nice if someone invented a Flash or java web applet that let you move several poles and zeros around anywhere, and plot the response.
All this is oversimplified, but should give some intuitive idea about what poles and zeros mean.