Does Impact Factor reflect the quality of a journal?
No.
To answer Paul's comment:
The title of the post asks about journals. For a well-worn list of criticisms of Impact Factor as an indicator of journal quality, see the Wikipedia article and the sources it cites. In many disciplines, important papers receive most of their citations well outside the IF's two-year window. Raw citation data can be manipulated by editorial policies, some more nefarious than others, or even by individual papers. Thomson-Reuters' calculations of Impact Factor are not reproducible, even using their own citation data. Et cetera ad nauseam. But most importantly: Having lots of citations is not the same thing as quality.
The body of the post asks about journal articles. Impact Factor is (roughly) the average number of citations to all papers published by a journal in a given time window. Even if it were a reliable measure of average quality (which it isn't), it would say nothing at all about the quality of any individual paper.
If the other two answers haven't made this obvious. The answer is no, but it can be taken even further than what the above answers suggests. We can study this!
The above image is the coefficient of determination between the impact factor of journals and the 2-year citation rate of their papers from 1902 to 2009, for all natural and medical sciences journals. Basically this means that even if all you care about is the number of citations you get then even then the impact factors of journals is starting to matter less and less.
Of course, what matters is how influential your ideas are and not how much they are cited, but we don't really a have a good metric for that.
Clearly, people here don't seem to like the idea that impact factor would be used to evaluate journal quality. I'm not a big fan of impact factor either. It can be gamed (and the degree to which it is gamed will increase as greater importance is placed on it). It also becomes particularly problematic when you compare across disciplines with different citation practices and citation half-lives. Furthermore, there's also the risk that people start using impact factor to evaluate article quality, which is a lot more questionable.
That said, within a discipline, I generally find that better journals have higher impact factors. It's not perfect. But there is a strong correlation. There's also research that shows very high correlations between various indices of journal quality including peer ratings, impact factor, and various other citation based indicators.
So in short, within a field, impact factor is one of many variables that typically correlates highly with journal quality. If you know nothing about the quality of a journal, you'll know more about it by looking at its impact factor. However, That shouldn't stop you from looking at other and most likely better indicators of journal quality (e.g., what is your own evaluation of the content published in the journal in recent years).
It is also really important to ask yourself why you want to evaluate journal quality. Is it to select a journal? to reward, promote, or hire academics? Using impact factor in these cases can be problematic for a wide variety of reasons, but these are separate issues.
Here's one article that I found providing empirical evidence by Saha et al (2003). To quote the abstract:
Objectives: Impact factor, an index based on the frequency with which a journal's articles are cited in scientific publications, is a putative marker of journal quality. However, empiric studies on impact factor's validity as an indicator of quality are lacking. The authors assessed the validity of impact factor as a measure of quality for general medical journals by testing its association with journal quality as rated by clinical practitioners and researchers.
Methods: We surveyed physicians specializing in internal medicine in the United States, randomly sampled from the American Medical Association's Physician Masterfile (practitioner group, n = 113) and from a list of graduates from a national postdoctoral training program in clinical and health services research (research group, n = 151). Respondents rated the quality of nine general medical journals, and we assessed the correlation between these ratings and the journals' impact factors.
Results: The correlation between impact factor and physicians' ratings of journal quality was strong (r2 = 0.82, P = 0.001). The correlation was higher for the research group (r2 = 0.83, P = 0.001) than for the practitioner group (r2 = 0.62, P = 0.01).
Conclusions: Impact factor may be a reasonable indicator of quality for general medical journals.
References
- Saha, S., Saint, S. & Christakis, D.A. (2003). Impact factor: a valid measure of journal quality?. Journal of the Medical Library Association, 91, 42. PDF