Do editors ever "overbook" referees?

I think this one will be very hard to answer with hard data. So, I'll just throw in some anecdotal evidence, along with a few things learnt from some editors that I know.

From a reviewer point of view, it has happened to me exactly twice in a decade (i.e. very rarely) to receive an editor's review saying “I've reached a decision based on input from other reviewers, and you do not need to review the manuscript”.

• The first time, the email came as my review was already overdue (a week or ten days), so I suppose the editor asked another reviewer when I didn't reply on time, and the new reviewer was fast to reply.
• The second time, it was sort of the other way around: I was asked to review a paper, then 5 days later the editor wrote, saying the reviewer who was uncommunicative had finally replied, and my review was no longer needed. He apologized profusely, and offered to actually wait for my review if I had started doing it and wanted to finish. (I was happy to let it go.)

I think for an editor, growing a list of trustworthy and willing reviewers is crucial. So, they simply cannot make them work for nothing! (pun intended)

An editor that cancels the review for a referee within the "normal" referee period will end up angering the referee and likely losing her as a reviewer for future papers. So this is a very unlikely practice.

Many journals will request multiple reviews (three is not uncommon for many of the journals I submit to), and will wait until the end of the standard review period to return any comments. If two of the three (or four) reviewers have returned reviews, and the others haven't, then the process can reasonably truncated at that point. (If you're behind schedule, you don't really have a right to complain in this case!)

I know this question was asked and answered long ago, but I thought it might be useful to contribute some real data from an editor / program chair point of view.

Because reviewers are volunteer labor, they are unreliable. I personally find that I get about a 75% rate of return on useful reviews. For journals, pretty much every review that is actually returned is useful, but many do not return. For conferences, where the PC members have signed up in advance, the rate of return is much higher, but there are a significant number of essentially useless 1-sentence reviews that give a score but no justification.

At the same time, an important dirty secret of the reviewing process is that there is rarely a fixed number of reviews that are actually necessary. In the venues in which I am involved, typically 4 reviews is best practice, 3 is acceptable, and 2 is only supportable if there is very strong agreement on a clear accept or reject.

Moreover, when you are recruiting reviewers for a journal submission, the reviewers don't necessarily respond to the request to review immediately. Thus, I will typically significantly overbook the initial set of requests, asking 6-8 reviewers. Generally somewhere between 2 and 6 of those will accept, from which typically 3-4 will actually return reviews, giving me enough for a well-justified decision. If things go unusually well and end up with five reviews, that's just fine and will make the author feel we've taken them very seriously, but I'm not sure such an overabundance has ever actually happened to me.

If not enough reviewers accept, I have to send out additional batches of requests, all of which can lead to a significant skew in the times at which reviews arrive. Moreover, it is an extremely rare reviewer who will return a review significantly in advance of the due date. So for me at least, overbooking isn't to get quick turnaround time, but to prevent excessively slow turnaround time that can happen when you need one more review and a reviewer drops out at the last minute, forcing you to start the clock all over again on a new reviewer.

Because it is all volunteer labor, however (and because I do my own turns as a reviewer in other venues as well), I am very mindful of the importance of not taking a reviewer for granted. If somebody has promised me a review, I want that review. It is only when they have become significantly late that I will send a question along the lines of: "I am currently only waiting on your review before I can send a decision to these authors; are you still able to provide a review?" The only time when I simply cancel a review request is when the reviewer has become repeatedly unresponsive, and then they get a black mark for unreliability in the appropriate set of organizational records.