BACKGROUND: Randomized trials without reported adequate allocation concealment have been shown to overestimate the benefit of experimental interventions. We investigated the robustness of conclusions drawn from meta-analyses to exclusion of such trials. MATERIAL: Random sample of 38 reviews from The Cochrane Library 2003, issue 2 and 32 other reviews from PubMed accessed in 2002. Eligible reviews presented a binary effect estimate from a meta-analysis of randomized controlled trials as the first statistically significant result that supported a conclusion in favour of one of the interventions. METHODS: We assessed the methods sections of the trials in each included meta-analysis for adequacy of allocation concealment. We replicated each meta-analysis using the authors' methods but included only trials that had adequate allocation concealment. Conclusions were defined as not supported if our result was not statistically significant. RESULTS: Thirty-four of the 70 meta-analyses contained a mixture of trials with unclear or inadequate concealment as well as trials with adequate allocation concealment. Four meta-analyses only contained trials with adequate concealment, and 32, only trials with unclear or inadequate concealment. When only trials with adequate concealment were included, 48 of 70 conclusions (69%; 95% confidence interval: 56-79%) lost support. The loss of support mainly reflected loss of power (the total number of patients was reduced by 49%) but also a shift in the point estimate towards a less beneficial effect. CONCLUSION: Two-thirds of conclusions in favour of one of the interventions were no longer supported if only trials with adequate allocation concealment were included.
BACKGROUND: Randomized trials without reported adequate allocation concealment have been shown to overestimate the benefit of experimental interventions. We investigated the robustness of conclusions drawn from meta-analyses to exclusion of such trials. MATERIAL: Random sample of 38 reviews from The Cochrane Library 2003, issue 2 and 32 other reviews from PubMed accessed in 2002. Eligible reviews presented a binary effect estimate from a meta-analysis of randomized controlled trials as the first statistically significant result that supported a conclusion in favour of one of the interventions. METHODS: We assessed the methods sections of the trials in each included meta-analysis for adequacy of allocation concealment. We replicated each meta-analysis using the authors' methods but included only trials that had adequate allocation concealment. Conclusions were defined as not supported if our result was not statistically significant. RESULTS: Thirty-four of the 70 meta-analyses contained a mixture of trials with unclear or inadequate concealment as well as trials with adequate allocation concealment. Four meta-analyses only contained trials with adequate concealment, and 32, only trials with unclear or inadequate concealment. When only trials with adequate concealment were included, 48 of 70 conclusions (69%; 95% confidence interval: 56-79%) lost support. The loss of support mainly reflected loss of power (the total number of patients was reduced by 49%) but also a shift in the point estimate towards a less beneficial effect. CONCLUSION: Two-thirds of conclusions in favour of one of the interventions were no longer supported if only trials with adequate allocation concealment were included.
Authors: Marco Vinceti; Gabriele Dennert; Catherine M Crespi; Marcel Zwahlen; Maree Brinkman; Maurice P A Zeegers; Markus Horneber; Roberto D'Amico; Cinzia Del Giovane Journal: Cochrane Database Syst Rev Date: 2014-03-30
Authors: David Moher; Sally Hopewell; Kenneth F Schulz; Victor Montori; Peter C Gøtzsche; P J Devereaux; Diana Elbourne; Matthias Egger; Douglas G Altman Journal: BMJ Date: 2010-03-23
Authors: Susan Armijo-Olivo; Jorge Fuentes; Maria Ospina; Humam Saltaji; Lisa Hartling Journal: BMC Med Res Methodol Date: 2013-09-17 Impact factor: 4.615
Authors: Michael H Bloch; Kaitlyn E Panza; Jon E Grant; Christopher Pittenger; James F Leckman Journal: J Am Acad Child Adolesc Psychiatry Date: 2013-03 Impact factor: 8.829