How to decide which papers to review?
I've found that after you get a few decent papers, you'll soon get far more requests than you can possibly handle (e.g. over the last few years I've consistently received one every week or two).
Besides the obvious issues of whether you're competent AND interested in reading the paper AND are currently available, my calculus usually goes like this:
Is this a very high end journal? E.g. in my field that would be JAMA, NEJM, Science, etc. Accept the request. It's good to have input on some of the most influential papers in your field.
If not, is it a journal you've never reviewed for before? It's always good to be able to add to the list of journals on your CV. If I haven't, but I HAVE heard of the journal (e.g. I know it's not a predatory journal), I'll usually accept.
Is it an editor you have a history with/have a decent reviewer/editor relationship with? Accept the request. It's good to have arms-length senior academics who like you and are aware that you exist and know your expertise--e.g. for external non-collaborator letters when you're up for tenure.
If the answer to all three of those is "No", I'll usually decline and suggest another colleague.
It comes down to personal preferences to some extent how you'd like to weight the various factors that go into deciding which papers to review. The two major points are:
- Personal Interest: Does the paper's content interest you personally? While you can review a paper that doesn't, it's much easier to get through one fully that does interest you. As for why it does, it could be you simply enjoy the topic, or perhaps it directly impacts your research.
- Scope: Can you fully assess the merits of this paper for publication? In other words, is it beyond your scope or not? If you cannot confidently say you can understand and assess the paper, then you may not want to accept to review it. While you normally have the option to decide this during the review process and inform the editor, it is best to make a determination of this prior to accepting to review, for convenience sake.
My answer is similar to the ones above with a small twist. I accept papers based on interest (is it an area I work in or would like to work or know), quality of journal/conference, and familiarity with the editor.
However, I also have a "paper budget" to prevent me from drowning myself in reviews (I used to do it in the past). In my area, it is usual to have 3 reviewers per paper. So, I target to review 3 times the number of papers I have submitted. It helps me focus my selection.