Slow face detection on OpenCV?

What is the size of the input image. I am guessing 640x480. Generally people who post YouTube videos resize the image to 160x120. IN full resolution of 640x480 it is very difficult to get more than 2-3 fps. Try to send 160x120 image. You should be getting at least 10fps.

Adding to the previous answers:

you can also speed things up by setting the Max and most importantly the Min size for detectMultiScale.

[Also, as the previous answers say, heavy scaling-down is in order as Haar detector uses very simple features (for the relations of upto 6 pixels; on larger scales you add up rectangle-like areas as if it was just one pixel). On standard mac/mbp2011 I could get around 60fps that is more than enough.]

For an even better speedup you could also eliminate non-changing areas, using say templateMatching.

Recently I had found a Simd Library, which has an implementation of HAAR and LBP cascade classifiers. It can use standard HAAR and LBP casscades from OpenCV. This implementation has SIMD optimizations with using of SSE4.1, AVX2 and NEON(ARM), so it works in 2-3 times faster then original OpenCV implementation.