Is "Impact" the "Factor" that matters…? (Part II).

In continuity of my Editorial from last issue, Impact Factors (IF) were intended to indicate the quality of journals and not individual articles. Its use in the process of academic evaluation have made IF very controversial. The use of IF to assess the worth of individual papers and their authors is very contentious.[1]

The most controversial, flawed and threatening aspect of IF usage is to determine 'author impact'. The use of IFs by academic officials as an useful tool to indicate person's scientific merit and for deciding on promotions, as are its undesirable and detrimental applications. IFs of journals in which individual's research is published cannot be used to define the impact of their research.[1]

The limitations of the journal IF to determine impact of a particular article published in that journal was very aptly highlighted by Stephen Curry and his team consisting of Marcia McNutt, who was as editor-in-chief of Science, personnel at Springer Nature, eLife, PLoS, the Royal Society and EMBO Press.[2]

Curry along with his co-researchers plotted the citations from 11 journals for articles published in 2013–14 (used to calculate 2015 impact factors). The eleven journals comprised of Science, Nature, eLife and 3 Public Library of Science (PLoS) journals.[2]

Majority of articles published in these journals had fewer citations than the impact factor for their journal. 74.8% of Nature articles, 75.5% of Science papers and 65.3% PLoS Genetics were cited below their impact factor of 38.1, 34.7 and 6.7 respectively. This means that 74. 8% of articles published in Nature had citations less than 38 in two years (its impact factor of 38). 75.5% of Science papers were cited less than 35 times in 2 years (its impact factor was 34.7). 65.3% of papers published in PLoS Genetics had citations less than 7 in 2 years (its impact factor of 6.7).[2]

Similarly, in 2005, 89% of Nature's impact factor was a result of 25% of the articles.[3]

So, the IF is of disputed value when IF of a journal is used for the appraisal of the class of a specific article, author or individual's research achievements.[456789] There are other parameters to judge these qualities.

Despite all the reservations about impact factor, and till we have a better system in place, Hoeffel [10] articulated the position aptly by stating that “Impact Factor is not a perfect tool to measure the quality of articles but there is nothing better and it has the advantage of already being in existence and is, therefore, a good technique for scientific evaluation. Experience has shown that in each specialty the best journals are those in which it is most difficult to have an article accepted, and these are the journals that have a high impact factor. Most of these journals existed long before the impact factor was devised. The use of impact factor as a measure of quality is widespread because it fits well with the opinion we have in each field of the best journals in our specialty.”[10]