An-Wen Chan1, Douglas G Altman. 1. University Health Network, Department of Medicine, Suite RFE 3-805, 190 Elizabeth Street, Toronto, ON M5G 2C4, Canada. anwen.chan@utoronto.ca
Abstract
OBJECTIVE: To examine the extent and nature of outcome reporting bias in a broad cohort of published randomised trials. DESIGN: Retrospective review of publications and follow up survey of authors. Cohort All journal articles of randomised trials indexed in PubMed whose primary publication appeared in December 2000. MAIN OUTCOME MEASURES: Prevalence of incompletely reported outcomes per trial; reasons for not reporting outcomes; association between completeness of reporting and statistical significance. RESULTS: 519 trials with 553 publications and 10,557 outcomes were identified. Survey responders (response rate 69%) provided information on unreported outcomes but were often unreliable--for 32% of those who denied the existence of such outcomes there was evidence to the contrary in their publications. On average, over 20% of the outcomes measured in a parallel group trial were incompletely reported. Within a trial, such outcomes had a higher odds of being statistically non-significant compared with fully reported outcomes (odds ratio 2.0 (95% confidence interval 1.6 to 2.7) for efficacy outcomes; 1.9 (1.1 to 3.5) for harm outcomes). The most commonly reported reasons for omitting efficacy outcomes included space constraints, lack of clinical importance, and lack of statistical significance. CONCLUSIONS: Incomplete reporting of outcomes within published articles of randomised trials is common and is associated with statistical non-significance. The medical literature therefore represents a selective and biased subset of study outcomes, and trial protocols should be made publicly available.
OBJECTIVE: To examine the extent and nature of outcome reporting bias in a broad cohort of published randomised trials. DESIGN: Retrospective review of publications and follow up survey of authors. Cohort All journal articles of randomised trials indexed in PubMed whose primary publication appeared in December 2000. MAIN OUTCOME MEASURES: Prevalence of incompletely reported outcomes per trial; reasons for not reporting outcomes; association between completeness of reporting and statistical significance. RESULTS: 519 trials with 553 publications and 10,557 outcomes were identified. Survey responders (response rate 69%) provided information on unreported outcomes but were often unreliable--for 32% of those who denied the existence of such outcomes there was evidence to the contrary in their publications. On average, over 20% of the outcomes measured in a parallel group trial were incompletely reported. Within a trial, such outcomes had a higher odds of being statistically non-significant compared with fully reported outcomes (odds ratio 2.0 (95% confidence interval 1.6 to 2.7) for efficacy outcomes; 1.9 (1.1 to 3.5) for harm outcomes). The most commonly reported reasons for omitting efficacy outcomes included space constraints, lack of clinical importance, and lack of statistical significance. CONCLUSIONS: Incomplete reporting of outcomes within published articles of randomised trials is common and is associated with statistical non-significance. The medical literature therefore represents a selective and biased subset of study outcomes, and trial protocols should be made publicly available.
Authors: John P A Ioannidis; Stephen J W Evans; Peter C Gøtzsche; Robert T O'Neill; Douglas G Altman; Kenneth Schulz; David Moher Journal: Ann Intern Med Date: 2004-11-16 Impact factor: 25.391
Authors: Diana R Elbourne; Douglas G Altman; Julian P T Higgins; Francois Curtin; Helen V Worthington; Andy Vail Journal: Int J Epidemiol Date: 2002-02 Impact factor: 7.196
Authors: Ana Macura; Iosief Abraha; Jamie Kirkham; Gian Franco Gensini; Lorenzo Moja; Alfonso Iorio Journal: Intern Emerg Med Date: 2010-03-19 Impact factor: 3.397