Usama Ahmed Ali1,2, Beata M M Reiber3, Joren R Ten Hove2, Pieter C van der Sluis2, Hein G Gooszen4, Marja A Boermeester1, Marc G Besselink1. 1. Department of Surgery, G4-190, Academic Medical Center, Meibergdreef 9, 1105 AZ, Amsterdam, The Netherlands. 2. Department of Surgery, University Medical Center Utrecht, Heidelberglaan 100, 3584 CX, Utrecht, The Netherlands. 3. Department of Surgery, G4-190, Academic Medical Center, Meibergdreef 9, 1105 AZ, Amsterdam, The Netherlands. bmmreiber@gmail.com. 4. Center for Evidence Based Surgery, Radboud University Nijmegen Medical Center, Geert Grooteplein Zuid 10, 6525 GA, Nijmegen, The Netherlands.
Abstract
PURPOSE: The journal impact factor (IF) is often used as a surrogate marker for methodological quality. The objective of this study is to evaluate the relation between the journal IF and methodological quality of surgical randomized controlled trials (RCTs). METHODS: Surgical RCTs published in PubMed in 1999 and 2009 were identified. According to IF, RCTs were divided into groups of low (<2), median (2-3) and high IF (>3), as well as into top-10 vs all other journals. Methodological quality characteristics and factors concerning funding, ethical approval and statistical significance of outcomes were extracted and compared between the IF groups. Additionally, a multivariate regression was performed. RESULTS: The median IF was 2.2 (IQR 2.37). The percentage of 'low-risk of bias' RCTs was 13% for top-10 journals vs 4% for other journals in 1999 (P < 0.02), and 30 vs 12% in 2009 (P < 0.02). Similar results were observed for high vs low IF groups. The presence of sample-size calculation, adequate generation of allocation and intention-to-treat analysis were independently associated with publication in higher IF journals; as were multicentre trials and multiple authors. CONCLUSION: Publication of RCTs in high IF journals is associated with moderate improvement in methodological quality compared to RCTs published in lower IF journals. RCTs with adequate sample-size calculation, generation of allocation or intention-to-treat analysis were associated with publication in a high IF journal. On the other hand, reporting a statistically significant outcome and being industry funded were not independently associated with publication in a higher IF journal.
PURPOSE: The journal impact factor (IF) is often used as a surrogate marker for methodological quality. The objective of this study is to evaluate the relation between the journal IF and methodological quality of surgical randomized controlled trials (RCTs). METHODS: Surgical RCTs published in PubMed in 1999 and 2009 were identified. According to IF, RCTs were divided into groups of low (<2), median (2-3) and high IF (>3), as well as into top-10 vs all other journals. Methodological quality characteristics and factors concerning funding, ethical approval and statistical significance of outcomes were extracted and compared between the IF groups. Additionally, a multivariate regression was performed. RESULTS: The median IF was 2.2 (IQR 2.37). The percentage of 'low-risk of bias' RCTs was 13% for top-10 journals vs 4% for other journals in 1999 (P < 0.02), and 30 vs 12% in 2009 (P < 0.02). Similar results were observed for high vs low IF groups. The presence of sample-size calculation, adequate generation of allocation and intention-to-treat analysis were independently associated with publication in higher IF journals; as were multicentre trials and multiple authors. CONCLUSION: Publication of RCTs in high IF journals is associated with moderate improvement in methodological quality compared to RCTs published in lower IF journals. RCTs with adequate sample-size calculation, generation of allocation or intention-to-treat analysis were associated with publication in a high IF journal. On the other hand, reporting a statistically significant outcome and being industry funded were not independently associated with publication in a higher IF journal.
Keywords:
Impact factor; Methodologic quality and randomized controlled surgical trials
Authors: Carin M Olson; Drummond Rennie; Deborah Cook; Kay Dickersin; Annette Flanagin; Joseph W Hogan; Qi Zhu; Jennifer Reiling; Brian Pace Journal: JAMA Date: 2002-06-05 Impact factor: 56.272
Authors: Kanu Okike; Mininder S Kocher; Jennifer L Torpey; Benedict U Nwachukwu; Charles T Mehlman; Mohit Bhandari Journal: J Clin Epidemiol Date: 2010-10-13 Impact factor: 6.437
Authors: Pascal Probst; Phillip Knebel; Kathrin Grummich; Solveig Tenckhoff; Alexis Ulrich; Markus W Büchler; Markus K Diener Journal: Ann Surg Date: 2016-07 Impact factor: 12.969
Authors: P J Devereaux; Peter T-L Choi; Samer El-Dika; Mohit Bhandari; Victor M Montori; Holger J Schünemann; Amit X Garg; Jason W Busse; Diane Heels-Ansdell; William A Ghali; Braden J Manns; Gordon H Guyatt Journal: J Clin Epidemiol Date: 2004-12 Impact factor: 6.437
Authors: Lesley Wood; Matthias Egger; Lise Lotte Gluud; Kenneth F Schulz; Peter Jüni; Douglas G Altman; Christian Gluud; Richard M Martin; Anthony J G Wood; Jonathan A C Sterne Journal: BMJ Date: 2008-03-03