Rahul Mhaskar1, Benjamin Djulbegovic, Anja Magazin, Heloisa P Soares, Ambuj Kumar. 1. Division and Center for Evidence-Based Medicine and Outcomes Research, Department of Internal Medicine, Clinical and Translational Science Institute, Morsani College of Medicine, University of South Florida, Tampa, FL 33612, USA.
Abstract
OBJECTIVES: To assess whether the reported methodological quality of randomized controlled trials (RCTs) reflects the actual methodological quality and to evaluate the association of effect size (ES) and sample size with methodological quality. STUDY DESIGN AND SETTING: Systematic review. This is a retrospective analysis of all consecutive phase III RCTs published by eight National Cancer Institute Cooperative Groups up to 2006. Data were extracted from protocols (actual quality) and publications (reported quality) for each study. RESULTS: Four hundred twenty-nine RCTs met the inclusion criteria. Overall reporting of methodological quality was poor and did not reflect the actual high methodological quality of RCTs. The results showed no association between sample size and actual methodological quality of a trial. Poor reporting of allocation concealment and blinding exaggerated the ES by 6% (ratio of hazard ratio [RHR]: 0.94; 95% confidence interval [CI]: 0.88, 0.99) and 24% (RHR: 1.24; 95% CI: 1.05, 1.43), respectively. However, actual quality assessment showed no association between ES and methodological quality. CONCLUSION: The largest study to date shows that poor quality of reporting does not reflect the actual high methodological quality. Assessment of the impact of quality on the ES based on reported quality can produce misleading results.
OBJECTIVES: To assess whether the reported methodological quality of randomized controlled trials (RCTs) reflects the actual methodological quality and to evaluate the association of effect size (ES) and sample size with methodological quality. STUDY DESIGN AND SETTING: Systematic review. This is a retrospective analysis of all consecutive phase III RCTs published by eight National Cancer Institute Cooperative Groups up to 2006. Data were extracted from protocols (actual quality) and publications (reported quality) for each study. RESULTS: Four hundred twenty-nine RCTs met the inclusion criteria. Overall reporting of methodological quality was poor and did not reflect the actual high methodological quality of RCTs. The results showed no association between sample size and actual methodological quality of a trial. Poor reporting of allocation concealment and blinding exaggerated the ES by 6% (ratio of hazard ratio [RHR]: 0.94; 95% confidence interval [CI]: 0.88, 0.99) and 24% (RHR: 1.24; 95% CI: 1.05, 1.43), respectively. However, actual quality assessment showed no association between ES and methodological quality. CONCLUSION: The largest study to date shows that poor quality of reporting does not reflect the actual high methodological quality. Assessment of the impact of quality on the ES based on reported quality can produce misleading results.
Authors: P J Devereaux; Peter T-L Choi; Samer El-Dika; Mohit Bhandari; Victor M Montori; Holger J Schünemann; Amit X Garg; Jason W Busse; Diane Heels-Ansdell; William A Ghali; Braden J Manns; Gordon H Guyatt Journal: J Clin Epidemiol Date: 2004-12 Impact factor: 6.437
Authors: Lesley Wood; Matthias Egger; Lise Lotte Gluud; Kenneth F Schulz; Peter Jüni; Douglas G Altman; Christian Gluud; Richard M Martin; Anthony J G Wood; Jonathan A C Sterne Journal: BMJ Date: 2008-03-03
Authors: An-Wen Chan; Jennifer M Tetzlaff; Peter C Gøtzsche; Douglas G Altman; Howard Mann; Jesse A Berlin; Kay Dickersin; Asbjørn Hróbjartsson; Kenneth F Schulz; Wendy R Parulekar; Karmela Krleza-Jeric; Andreas Laupacis; David Moher Journal: BMJ Date: 2013-01-08
Authors: An-Wen Chan; Jennifer M Tetzlaff; Douglas G Altman; Andreas Laupacis; Peter C Gøtzsche; Karmela Krle A-Jerić; Asbjørn Hrobjartsson; Howard Mann; Kay Dickersin; Jesse A Berlin; Caroline J Dore; Wendy R Parulekar; William S M Summerskill; Trish Groves; Kenneth F Schulz; Harold C Sox; Frank W Rockhold; Drummond Rennie; David Moher Journal: Rev Panam Salud Publica Date: 2015-12
Authors: An-Wen Chan; Jennifer M Tetzlaff; Douglas G Altman; Andreas Laupacis; Peter C Gøtzsche; Karmela Krleža-Jerić; Asbjørn Hróbjartsson; Howard Mann; Kay Dickersin; Jesse A Berlin; Caroline J Doré; Wendy R Parulekar; William S M Summerskill; Trish Groves; Kenneth F Schulz; Harold C Sox; Frank W Rockhold; Drummond Rennie; David Moher Journal: Ann Intern Med Date: 2013-02-05 Impact factor: 25.391