RELATION BETWEEN IMPACT FACTOR IN ORTHOPEDIC JOURNALS AND LEVEL OF EVIDENCE.

OBJECTIVE: This study aims to assess the quality of articles published in the leading orthopedic surgery journals, by measuring the relation between the impact factor and the number studies with a high level of evidence.
METHODS: A literature review was performed of articles published in four previously selected journals. A score of journal evidence (RER - Relation between Randomized clinical trials and Systematic reviews) was calculated, considering the number of RCTs and SR published and the total number of full-text articles.
RESULTS: The selected journals were JBJS-Am, ASMJ, BJJ-Br and Arthroscopy, with Impact factors of 5.280, 4.362, 3.309 and 3.206 respectively in 2015. In the study, the RER Scores, in the same order, were 9.408, 6.153, 7.456 and 7.779.
CONCLUSION: The journal JBJS-Am is the best available source of information on orthopedic surgery from this point of view. It has the highest Impact Factor and clearly the highest RER Score. On the other hand, we could conclude that the number of published RCT and good quality SR is very low, with less than 10% of all the articles. Level of evidence III, Analyses based on limited alternatives and costs, and poor estimates.

INTRODUCTION

An enormous number of articles are published annually by each orthopedic journal, which leads to a progressive increase of new information, new surgical techniques, updates of diseases and case reports that can be easily accessed.

With too much data available in the literature, effective and judicious analysis of these data should be done in order to guide and complement surgeon's decision-making process, defining therefore, the central point of practicing Evidence-Based Medicine (EBM). It should involve integration of clinical expertise, patients’ perceptions and values, and the best available research evidence. However, most of knowledge of orthopedic surgeons is based only on clinical experience of some experts.

Aiming for the best quality of information, students, researchers, clinical practitioners and surgeons may choose an article based on the journal according to its relevance, which can be measured by Impact Factor (IF). The IF was created as a measurement of the number of citations of scientific articles published in a determined period. The IF shows indirectly the relevance and quality of the publications by a journal, and it is calculated by a relation between published articles and articles that are mentioned, quoted or cited_in the biennium prior to the year in which the IF is calculated.

Another method to assess the quality of the study is the analysis its design, considering the level of evidence, which is categorized in six levels: (Figure 1)

Figure 1

The Hierarchy of evidence – Rockwood and Green's Fracture in Adults, 7Ed.

OBJECTIVE

This study aims to assess the quality of published articles in the most important orthopedic surgery journals, by measuring the relation between the impact factor and the number of high-evidence level studies.

METHODS

Inclusion criteria used was the four orthopedic surgery journals with the highest impact factor (IF) included in the list of Journal of Citation Report of Web of Knowledge.

A systematic assessment of published articles of these four journals was performed. Full articles published from January 1st 2013 to December 31st 2014 were selected. A score of journal evidence (RRS - Relation of Randomized clinical trials and Systematic Revision) was calculated considering the number of RCTs and SR published and total of full articles (Figure 2).

Figure 2

RRS score calculation formula.

Editorials, letters and communications were excluded from the calculation. Also, narrative reviews were not considered high-evidence articles and were not included in the sum of RCTs and SRs. From that point, we could estimate the RRS score of each journal selected and compare with IF.

RESULTS

The selected journals in orthopedic surgery field with higher IF selected were:

Journal of Bone and Joint Surgery - American Volume (JBJS-Am);

American Journal of Sports Medicine (AJSM);

The Bone and Joint Journal - British Volume (BJJ-Br) and

Arthroscopy - The Journal of Arthroscopic and Related Surgery. A total of 210 level 1 and 2 (high-evidence level) articles were identified in the four selected journals. Of these 26 were systematic reviews / meta-analysis and 184 RCTs (Tabela 1).

Table 1

Demonstrates relation of total published articles to number to high-evidence articles, RRS Score and Impact factor.

	Total published articles 2013 – 2014	RCTs/SRs	RRS Score	IF
JBJS-Am	659	62	9.408	5.280
AJSM	650	40	6.153	4.362
BJJ-Br	523	39	7.456	3.309
Arthroscopy	887	69	7.779	3.206

RCT: Randomized Controlled Trial; SR: Systematic Review; RRS***; IF: Impact Factor (JCR-Web of Knowledge); JBJS-AM: Journal of Bone and Joint Surgery - American Volume; AJSM: American Journal of Sports Medicine; BJJ-Br: The Bone and Joint Journal - British Volume; Arthroscopy - The Journal of Arthroscopic and Related Surgery.

DISCUSSION

Since 2003, the main orthopedic journal - The Journal of Bone and Joint Surgery - American Volume (JBJS-Am), have been using a scale based on the pyramid of evidence, ranking their published articles according to level of evidence. Actions like this promote readers and authors to practice EBM, improve the quality of studies to be published and demonstrate the increasing influence of this practice in science of orthopedic surgery. Some journals publish a large number of letters, communications and editorials. In order to not compromise results, these publications were excluded from the analysis in this study.

The proposed Score assesses the proportion of high-evidence articles published by a journal each year. However, the highest obtained RRS score in this study of 9.408 of JBJS-Am means that less than 10% of all articles published in the journal of highest impact factor represents high-evidence quality. Other studies also demonstrate even lower rates of quality articles published. Rodrigues have found 0.84% in plastic surgery journals, Moraes found 2.4% in orthopedic literature and Rosales found from zero to 8.3% in hand surgery journals.

The impact factor of a scientific journal reflects its importance and relevance in the literature. Despite the fact that there are some criticisms about the validity of IF, once many authors self-citation and policies practiced by journal editors to increase its IF rating. Limitation of this study is lack of assessment of quality of published RCTs. Tools like the one described by Jadad demonstrate fails and risks of biases in these studies. This further and deeper evaluation may be postponed and be the next step when a larger amount of high-quality evidence studies is published.

Another limitation is that prospective cohort studies for prognosis and accuracy studies for diagnostics were not included as high quality evidence articles even though they represent the highest evidence for each study design.

CONCLUSION

The conclusion of this study was that the JBJS-Am is the best available source of information in orthopedic surgery. It has the highest Impact Factor and clearly the highest RRS Score. On the other hand, we could conclude that the number of published RCT and good quality SR is very low, with less than 10% of all the articles.