Originally conceived to help libraries and researchers be selective in journal management, the influence of IF has grown and spread significantly, raising issues about the process of the calculation and its current uses and abuses.
You Might Also Like
Explore This IssueAugust 2007
Also By This Author
According to Thomson Scientific, which computes journal IFs, the number is “a measure of the frequency with which the ‘average article’ in a journal has been cited in a particular year or period.” (See Figure 1)
In 2004, the fully automated system applied its algorithms to 27 million citations in 5,968 science journals and 1,712 social science journals. Originally, IF was intended as a practical quantitative tool to rank, evaluate, categorize, and compare journals worldwide. Now it is seen as a measure of the quality of a journal, its contents, and its authors. Recently, however, IFs have been used, or abused according to Dr. Garfield, as a dynamic in hiring decisions and in awarding grants and tenure. “It never occurred to me that ‘impact’ would become so controversial,” says Dr. Garfield.
Over the past 10 years, IF has come to routinely be applied in ways that Dr. Garfield never imagined. “The term ‘impact factor’ has gradually evolved, especially in Europe, to describe both journal and author impact,” he says.
Repercussions of Rank
According to Michael D. Lockshin, MD, editor of A&R, in countries such as England, Germany, France, and China, laboratory space, promotions, and budgets can be based on the IF of the journals in which an author publishes “Academics in these countries are desperate to be in high–impact factor journals,” says Dr. Lockshin, who is professor of medicine and obstetrics-gynecology at Weill Medical College and an attending physician at the Hospital for Special Surgery in New York City. In Spain, researchers have a legal right to be rewarded for publishing in high-impact journals, and in China some universities require PhD candidates to publish two articles in high-impact journals before they are allowed to earn their doctorate.
Although this kind of application is not as prevalent in the United States, other issues are influenced by IF. Dr. Garfield has wryly referred to “Citation Sanity and Insanity: The Obsession and Paranoia of Citations and Impact Factor” as an alternative title to an address he gave in 2005 to the International Congress of Peer Review and Biomedical Publication.1
The allegations against this system run from claims that IF reporting can be artificially manipulated by editors who are trying to raise their numbers to complaints that while academics vie for limited space in high-IF journals, their work is being delayed from reaching the community in the most timely fashion.
European critics say there is an English-language bias to the reporting. With American journals overwhelmingly citing research reported in English, the mean journal impact of American science is 30% above the world average.2 This puts pressure on authors around the world to eschew low-IF national journals in favor of a chance to appear in high-IF English-language publications. Papers that do appear in low-IF national journals are often underrated by the scientific community and policy makers.
The ability of pharmaceutical companies and other potential advertisers to employ IF in their buying decisions ups the ante. Anecdotal reports of self-citations have editors, eager to boost their reputation, citing their own publication in editorials and suggesting that authors add citations from their journal to upcoming articles. Fortunately, according to Dr. Lockshin, Thomson Scientific sees these efforts as transparent and seeks to discount self-citations.
A&R, AC&R Make an Impact
The ACR’s influential journals A&R and Arthritis Care & Research (AC&R) achieved a 2006 IF of 7.751, up 0.33 from the 2005 score of 7.421. (When calculating IF, AC&R is considered a section of A&R, and citable articles from both journals are used to calculate a single IF.) A&R was already the top-ranked rheumatology publication indexed by Thomson Scientific before the increase. The 2006 IFs were announced in June.
“We were delighted to see that the impact factor rose,” says A&R editor Michael D. Lockshin, MD, in an e-mail. “Since the impact factor lags two years behind the publication dates, the 2007 impact factor reflects publications in 2005 and 2006,” he noted.
What will the higher IF mean for the journal? “It enhances our reputation and our attractiveness for top-flight authors,” says Dr. Lockshin, who attributes the increase to the “outstanding articles we publish, of course.”
Another issue is the mix of articles in a publication and manner in which Thomson Scientific arrives at the final calculation. Clinical studies, clinical definitions/consensus reports, and review articles are highly desirable because they generate the most citations. A single sensational article can increase IF more than several well-regarded basic science papers, so editors can be tempted to alter their publication mix to raise their IF. Consideration as to which authors and articles will contribute the most to IF could affect the information that ultimately gets to the readership.
Letters to the editor, news articles, editorials, book reviews, and abstracts of meetings are not counted as citable articles, but can raise the IF by increasing the equation’s numerator. In June 2006, the Public Library of Science (PLoS) issued this editorial opinion: “We conclude that science is currently rated by a process that is itself unscientific, subjective, and secretive.”3
Although, according to Dr. Lockshin, the definition of “citable” is arbitrary, because journals are not uniform in article structure, he takes exception to the PLoS view. “Allowing for the many variations of journal and citation style, we judge that [the Thomson process] is an honest and fair attempt, and as sound as any we could devise, to rate journals by an open and explicable metric,” he says.
Perhaps the most resonant argument is that IF does not take into account the quality of the work in individual articles or the utility of a journal to readers. Citations may come from controversy, retractions, and authors who refute the findings of an article. Antony Rosen, MD, director of the rheumatology division at Johns Hopkins University in Baltimore and chair of the ACR Journal Publications Committee, views IF in a pragmatic light. “In terms of relevance, quality, and importance, there are lots of useful articles that never get cited,” Dr. Rosen says. “Impact factor is just one measure of a journal, nothing more.”
Francine Kaplan is a medical journalist based in Atlanta.
- Garfield E. The Agony and the Ecstasy: The history and meaning of the journal impact factor. International Congress on Peer Review and Biomedical Publication. Chicago, September 16, 2005.
- Braun T, Glänzel W, Grupp H. The scientometric weight of 50 nations in 27 science areas, 1989–1993. Part II. Life sciences. Scientometrics. 1996;34:207-237.
- The Impact Factor Game. It is time to find a better way to assess the scientific literature. PLoS Med. 2006;3(6):e291.