S Lehrl1. 1. Abteilung für Medizinische Psychologie und Psychopathometrie, Universität Erlangen-Nürnberg.
Abstract
BACKGROUND: Regularly the Institute of Scientific Information publishes the impact factor (IF) that plays an increasing role when the scientific quality of scientific performances of journals, single publications, scientists, and research groups have to be evaluated in order to support them. QUESTIONS: How valid is the IF assigned to journals, single publications, scientists, and research groups? Have all these the same chance to be evaluated? How can fairness of evaluation be increased? Can its validity be improved? RESULTS: The value of IF equals the average number of citations per article published in the preceding 2 calendar years in a journal. The criteria for selection of citing journals and of those with an "official" IF are not fully explicated. Although the citations have no equal units of measurement, empirical findings confirm their pragmatic applicability. IF of journals and even the citation rates of its articles are skewedly distributed to right hand. Additionally, the citation rates of the articles within a journal vary. Therefore, the IF of journals rarely equal the actual citation rates of their articles. Usually, IF overestimates the citation rate and quality of the articles. Its tendency not to recognize low and high quality even increases when IF is administered to individual scientists and small research groups, whereas it decreases in large research groups. Under the premise that the extent of scientific quality corresponds to the amount of information a paper adds to the state of science, language, actuality etc. are confounders because English, reviewing, biomedical, and actual articles have preferred citation rates. CONCLUSIONS: Evaluation of scientific performances by IF is to be restricted to journals and large research groups. Fairness demands comparisons to homogeneous journals with respect to confounders such as language. Principally, no journal should be excluded to obtain an IF if it fulfills the minimum criteria of an internationally communicating science. For this purpose they have to provide a title, key words, and an abstract in English, a peer review system etc. Often journals are the centre of science cultures that are able to generate research of highest levels. The users can contribute to increase the IF of "their" journal and to care for the valid application of this indicator.
BACKGROUND: Regularly the Institute of Scientific Information publishes the impact factor (IF) that plays an increasing role when the scientific quality of scientific performances of journals, single publications, scientists, and research groups have to be evaluated in order to support them. QUESTIONS: How valid is the IF assigned to journals, single publications, scientists, and research groups? Have all these the same chance to be evaluated? How can fairness of evaluation be increased? Can its validity be improved? RESULTS: The value of IF equals the average number of citations per article published in the preceding 2 calendar years in a journal. The criteria for selection of citing journals and of those with an "official" IF are not fully explicated. Although the citations have no equal units of measurement, empirical findings confirm their pragmatic applicability. IF of journals and even the citation rates of its articles are skewedly distributed to right hand. Additionally, the citation rates of the articles within a journal vary. Therefore, the IF of journals rarely equal the actual citation rates of their articles. Usually, IF overestimates the citation rate and quality of the articles. Its tendency not to recognize low and high quality even increases when IF is administered to individual scientists and small research groups, whereas it decreases in large research groups. Under the premise that the extent of scientific quality corresponds to the amount of information a paper adds to the state of science, language, actuality etc. are confounders because English, reviewing, biomedical, and actual articles have preferred citation rates. CONCLUSIONS: Evaluation of scientific performances by IF is to be restricted to journals and large research groups. Fairness demands comparisons to homogeneous journals with respect to confounders such as language. Principally, no journal should be excluded to obtain an IF if it fulfills the minimum criteria of an internationally communicating science. For this purpose they have to provide a title, key words, and an abstract in English, a peer review system etc. Often journals are the centre of science cultures that are able to generate research of highest levels. The users can contribute to increase the IF of "their" journal and to care for the valid application of this indicator.