The impact factor of rheumatology journals: an analysis of 2008 and the recent 10 years.

Despite various weaknesses, the impact factor (IF) is still used as an important indictor for scientific quality in specific subject categories. In the current study, the IFs of rheumatology journals over the past 10 years were serially analyzed and compared with that from other fields. For the past 10 years (1999-2008), the IFs published by the Institute for Scientific Information in the Science Citation Index-Journal Citation Report were analyzed. For the majority of rheumatology journals, the IF shows a gradually increasing trend. The mean and median level of increase of IF from 1999 to 2008 is 233.9 and 66.5%, respectively. The increase in IF from 1999 or the first year with IF documentation to that in 2008 was higher for European journals than for the USA journals. The aggregate IF and the median IF of rheumatology journals remained within the top 30% and top 15% in clinical medical and all the scientific categories, respectively. Over the past 10 years, rheumatology journals showed a general increase in IF and rheumatology remained a leading discipline. For journals in the English language, those from Europe had an even higher increase than those from USA.

Introduction

The impact factor (IF), an attempt to quantify and rank journal quality, has been published since 1961 by the Institute for Scientific Information (ISI). It is a measure of the frequency with which an average article in a journal has been cited in a particular year. The IF of a journal, computed annually by the Science Citation Index (SCI), is the average number of times that articles published in that journal in two consecutive years are cited during the following year. It has been widely used as an index for evaluating the quality of publications in scientific journals [1, 2]. Although it has been criticized for various weaknesses [3-7], the IF does provide an objective measure of the citation rate of the average published article in a specific journal, and it has not been replaced by any other index of rating the quality of journals.

To give a further basis for the discussion on the significance of the IF, the IF of rheumatology journals of 2008 as well as those of the past 10 years was serially analyzed and compared with that from other fields given the relative paucity of information in this area.

Methods

For the past 10 years (1999–2008), the IFs published by the ISI in the SCI—Journal Citation Report (JCR) [8] were descriptively analyzed. All journals listed in the 2008 SCI under the subheading “Rheumatology” were included. For each journal, the IF for the years 1999–2008 (if available), language (i.e., English, multiple languages (including English), or non-English) and country of origin (according to the item of “Journal Country/Territory” of each journal in the Journal Citation Report) were determined. The aggregate IF and median IF of each category were also extracted from the JCR each year (only available since 2003).

Additionally, all categories of scientific journals in the “JCR Science Edition” and categories related to clinical medicine (so-called clinical medicine journals listed under the following subheadings: Allergy; Andrology; Anesthesiology; Cardiac & cardiovascular system; Clinical neurology; Critical care medicine; Dentistry, oral surgery & medicine; Dermatology; Emergency medicine; Endocrinology & metabolism; Gastroenterology & hepatology; Geriatrics & gerontology; Hematology; Infectious diseases; Medicine, general and internal; Obstetrics & gynecology; Oncology; Ophthalmology; Orthopedics; Otorhinolaryngology; Pediatrics; Peripheral vascular disease; Psychiatry; Psychology; Radiology, nuclear medicine & medical imaging; Rehabilitation; Respiratory system; Rheumatology; Surgery; Transplantation; Tropical medicine; Urology & nephrology) were also analyzed with regard to changes in the ranking of the top 50% rheumatology journals (according to the IF in 2008), as well as to the ranking of the aggregate and median IF of rheumatology journals over the past 10 years.

Results

General data

Twenty-two journals in rheumatology were analyzed according to the 2008 JCR Science Edition (Table 1). Among the 22 journals, 20 were in English; one was in German, and the other one was in “multiple languages”. Eight journals originated from USA, 7 from UK, and the other 7 originated from 5 different countries.

Table 1

Change in IF of rheumatology journals from 1999 to 2008

	Language	Country	IF in 2008	Change from lowest to 2008	Change from 1999 or the first IF to 2008
Aktuelle Rhematologie	German	Germany	0.084	0.000 (0.0%)	−0.151 (−64.3%)
Annals of the Rheumatic Diseases	English	England	7.188	5.220 (265.2%)	5.220 (265.2%)
Arthritis and Rheumatism	English	USA	6.787	0.000 (0.0%)	−0.551 (−3.8%)
Arthritis Research & Therapy	English	England	4.485	1.003 (28.8%)	−0.267 (−10.9%)
Best Practice & Research in Clinical Rheumatology	English	England	3.066	2.806 (1079.2%)	2.806 (1079.2%)
BMC Musculoskeletal Disorders	English	England	1.987	0.987 (98.7%)	0.987 (98.7%)
Clinical and Experimental Rheumatology	English	Italy	2.364	1.080 (84.1%)	1.016 (75.4%)
Clinical Rheumatology	English	USA	1.559	0.944 (153.5%)	0.944 (153.5%)
Current Opinion in Rheumatology	English	USA	4.689	1.705 (57.1%)	1.089 (30.3%)
Journal of Musculoskeletal Pain	English	USA	0.500	0.289 (137.0%)	−0.287 (−36.5%)
Journal of Rheumatology	English	Canada	3.282	0.691 (26.7%)	0.403 (14.0%)
JCR—Journal of Clinical Rheumatology	English	USA	1.416	1.142 (416.8%)	1.032 (268.8%)
Joint Bone Spine	English	France	1.953	1.583 (427.8%)	1.583 (427.8%)
Lupus	English	England	2.244	0.780 (53.3%)	0.780 (53.3%)
Nature Clinical Practice Rheumatology	English	USA	5.677	5.455 (2457.2%)	5.455 (2457.2%)
Osteoarthritis and Cartilage	English	England	4.082	2.002 (96.3%)	2.216 (84.7%)
Rheumatic Disease Clinics of North America	English	USA	1.770	0.000 (0.0%)	1.872 (−14.1%)
Rheumatology International	English	Germany	1.327	0.434 (48.6%)	−0.290 (19.8%)
Rheumatology	English	England	4.136	1.599 (63.0%)	0.219 (63.0%)
Scandinavian Journal of Rheumatology	English	Norway	2.345	1.176 (100.6%)	1.599 (100.6%)
Seminars in Arthritis and Rheumatism	English	USA	4.379	1.803 (70.0%)	1.176 (70.0%)
Zeitschrift fur Rhematologie	Multiple	Germany	0.533	0.069 (14.9%)	1.803 (14.9%)

Among the 22 journals in the field of rheumatology, the only non-English one was entitled “Aktuelle Rhematologie”, whose IF remains at the bottom throughout the 10 years.

“Arthritis and Rheumatism”, the official journal of the American College of Rheumatology, always remained as the top one with respect to IF among rheumatology journals until 2008, when it was replaced by “Annals of the Rheumatic Diseases”, the official journal of the European League Against Rheumatism.

Changes in IFs of rheumatology journals

Compared with 1999 (or the first year of IF documentation), the IFs in 2008 of 17/22 journals increased, with a median level of +84.7% (range +14.0 to +2457.2%). Only five journals had their IFs decreased, with a median level of −14.1% (ranging −3.8 to −64.3%). Among all the 22 journals, the mean and median level of increase in IF from 1999 (or the first year of IF documentation) to 2008 were 233.9% and 66.5%, respectively.

Journals in English language from USA and Europe

For the journals in the English language, the mean IF of those originating from the USA was higher than that from Europe before 2002. Since 2003, however, the mean IF of European journals has begun to catch up with that of USA journals. Most of the 12 non-USA journals in English originated from European countries, except one from Canada (“Journal of Rheumatology”). If we include this journal and compare the IF of English journals from the USA and that from non-USA countries, the results are similar (Fig. 1).

Fig. 1

Comparisons of the IFs of rheumatology journals from 1999 to 2008 in the English language. a Between USA and European countries; b between USA and non-USA countries

The mean and median increase of IF in 2008 from 1999 or the first year with IF documentation were higher in non-USA journals than in USA journals [1.426 ± 1.471 vs. 1.185 ± 1.896, 1.096 (range −0.551 to +5.220) vs. 0.988 (range −0.267 to +5.455)]. The median percentage of increase of IF in 2008 from 1999 or the first year with IF documentation was higher in non-USA journals than in USA journals [80.0% (range −10.9 to +1079.2%) vs. 50.1% (range −36.5 to +2457.2%)].

Effect of review articles

Among the 20 journals in the English language, 4 journals publish almost exclusively reviews rather than original papers (so-called review-journals). These are “Best Practice & Research in Clinical Rheumatology”, “Current Opinion in Rheumatology”, “Rheumatic Disease Clinics of North America”, and “Nature Clinical Practice Rheumatology”. Compared with the other 16 journals that primarily publish original papers, the “review-journals” showed more increase in their IFs. The mean and median increase of IF in 2008 from the lowest level was 2.492 vs. 1.296 and 2.256 vs. 0.944, respectively. The mean and median percentage of increase of IF in 2008 from the lowest level was 898.4 vs. 129.4% and 568.2 vs. 90.2%.

Ranking among clinical medicine journals and all scientific journals

Since 2003, the “category data” of journals of each discipline has been available, including total citation, aggregate IF, and median IF. There were 170 categories in 2003 rising to 173 categories in 2008. The aggregate IF and the median IF of rheumatology journals ranked 12–19 and 17–31 among these categories through these years, respectively. Among the 32 categories of journals related to clinical medicine, the aggregate IF and the median IF of rheumatology journals ranked 5–9 and 5–12 through the period of 2003–2008, respectively (Fig. 2).

Fig. 2

The percentile rankings of the aggregate IF and median IF of rheumatology journals from 2003 to 2008. a among clinical medicine journals; b among all scientific journals

From 1999 to 2008, 1,291–1,553 journals were registered as “clinical medicine journals” and 5,550–6,598 journals as scientific journals in the “JCR Science Edition”. The ranking of the top 50% rheumatology journals (according to the IF in 2008) in clinical medicine journals and all scientific journals in the “JCR Science Edition” is listed in Table 2.

Table 2

Absolute rankings of the IF of the top 50% rheumatology journals among all the scientific journals and clinical medicine journals

	1999	2000	2001	2002	2003	2004	2005	2006	2007	2008
No. of all scientific journals	5,550	5,686	5,752	5,876	5,907	5,969	6,088	6,166	6,426	6,598
Annals of the Rheumatic Diseases	966	732	509	431	416	444	184	262	217	203
Arthritis and Rheumatism	114	119	125	124	148	158	169	166	162	225
Nature Clinical Practice Rheumatology								5726	623	308
Current Opinion in Rheumatology				428	622	757	400	378	482	465
Arthritis Research & Therapy					262	324	648	583	559	504
Seminars in Arthritis and Rheumatism	613	501	530	708	881	744	616	718	676	530
Rheumatology		693	533	511	431	398	429	515	552	603
Osteoarthritis and Cartilage	806	951	917	700	695	535	434	525	629	623
Journal of Rheumatology	515	550	704	600	832	820	814	916	894	961
Best Practice & Research in Clinical Rheumatology	1,602	4,770	2,930	3,538	2,044	3,404	1,664	1,791	1,745	1,102
Clinical and Experimental Rheumatology		1,337	1,438	2,029	1,370	1,987	1,202	1,496	1,544	1,684
Aggregate IF of rheumatology journals*					19/170	17/170	12/171	12/172	13/172	17/173
Median IF of rheumatology journals*					21/170	31/170	17/171	19/172	17/172	18/173
No. of clinical medicine journals	1,293	1,291	1,320	1,338	1,346	1,363	1,397	1,419	1,478	1,553
Annals of the Rheumatic Diseases	332	250	165	132	132	151	54	75	62	57
Arthritis and Rheumatism	28	27	27	27	37	38	48	44	43	65
Nature Clinical Practice Rheumatology								1,389	222	104
Current Opinion in Rheumatology				131	207	264	129	125	176	167
Arthritis Research & Therapy					79	101	232	207	205	180
Seminars in Arthritis and Rheumatism	206	151	174	236	306	259	214	253	244	187
Rheumatology		237	176	163	139	127	138	179	203	211
Osteoarthritis and Cartilage	277	321	310	230	238	178	140	185	225	219
Journal of Rheumatology	167	172	241	192	285	289	280	321	318	343
Best Practice & Research in Clinical Rheumatology		1,186	876	1,049	684	1,025	563	622	616	392
Clinical and Experimental Rheumatology	522	448	482	678	466	658	417	522	545	585
Aggregate IF of rheumatology journals#					9/32	7/32	5/32	5/32	7/32	8/32
Median IF of rheumatology journals#					6/32	12/32	6/32	6/32	5/32	7/32

* Within all categories, # within categories of clinical medicine

Since 2005, “Arthritis and Rheumatism” and “Annals of the Rheumatic Diseases” hold the top 2 IF among the rheumatology journals. The ranking of these two journals among clinical medicine journals and among all the scientific journals remained roughly stable or kept consistently rising within the last 10 years.

Discussion

The reliability of this IF is questionable [3-7]. For example, a journal’s IF, derived from citations to all articles in a journal, cannot tell the quality of any specific research article, nor of the quality of the work of author; review articles are heavily cited and increase the IF of journals. Moreover, there are numerous ways by which the IF can be manipulated. However, since so far there is no alternative, IFs are nowadays used as an important indicator for scientific quality of journals in specific subject categories [9].

The current study analyzed the change in IFs of rheumatology journals in the last 10 years. For the majority of these journals, the IF shows a gradual increase. This is consistent with trends in other fields [10]. Possible reasons for the increase in the IF of scientific journals include the increasing number of references per published article nowadays, modifications in the relative frequency of publication of review articles vs. original research articles, and the availability of full text of articles on the Internet [11]. The IF of the only non-English journal in rheumatology, “Aktuelle Rhematologie”, remains at the bottom throughout this 10-year period. This is in part due to bias against non-English journals noted by ISI [12, 13].

Since 2003 when the “category data” in journals of each discipline became available, the aggregated IF of rheumatology journals has stably remained within the top 30% and top 15% of categories related to clinical medicine and all scientific journals, respectively. This indicates that rheumatology is one of the active fields in research [12].

The most important finding in the current study is that, among the rheumatology journals in the English language, the IF of journals originating from European countries has gradually caught up with that originating from the USA. This phenomenon is different from what occurs in many other fields of medicine, e.g., nephrology and cardiology, and it also contrasts with some previous reports stating that journals from the USA generally show higher IF levels than journals from Europe [14]. This may be partially attributed to the citing behavior which favors journals from the USA in some fields, i.e., studies originating in the USA rarely cite research from other nations, while European researchers tend to cite more frequently paper from the other side of the Atlantic [14, 15]. Despite such bias, endeavor by the editorial boards of many European rheumatology journals is crucial for the increase in journals’ IF.

The current study also observed that “review-journals” have more rapid increase in IFs than those publishing original papers. This is consistent with findings in other fields since reviews tend to be cited more frequently [9, 12].

In conclusion, over the past 10 years, rheumatology journals showed a general increase in IF and rheumatology remains a leading discipline in clinical medicine and all scientific categories. For journals in the English language, those from Europe had an even higher increase in IF than those from the USA.