Michal Kicinski1, David A Springate2,3, Evangelos Kontopantelis2,4. 1. Faculty of Science, Hasselt University, Diepenbeek, Belgium. 2. Centre for Primary Care, National Institute for Health Research School for Primary Care Research, Institute of Population Health, University of Manchester, Manchester, U.K. 3. Centre for Biostatistics, Institute of Population Health, University of Manchester, Manchester, U.K. 4. Centre for Health Informatics, Institute of Population Health, University of Manchester, Manchester, U.K.
Abstract
UNLABELLED: We used a Bayesian hierarchical selection model to study publication bias in 1106 meta-analyses from the Cochrane Database of Systematic Reviews comparing treatment with either placebo or no treatment. For meta-analyses of efficacy, we estimated the ratio of the probability of including statistically significant outcomes favoring treatment to the probability of including other outcomes. For meta-analyses of safety, we estimated the ratio of the probability of including results showing no evidence of adverse effects to the probability of including results demonstrating the presence of adverse effects. RESULTS: In the meta-analyses of efficacy, outcomes favoring treatment had on average a 27% (95% Credible Interval (CI): 18% to 36%) higher probability to be included than other outcomes. In the meta-analyses of safety, results showing no evidence of adverse effects were on average 78% (95% CI: 51% to 113%) more likely to be included than results demonstrating that adverse effects existed. In general, the amount of over-representation of findings favorable to treatment was larger in meta-analyses including older studies. CONCLUSIONS: In the largest study on publication bias in meta-analyses to date, we found evidence of publication bias in Cochrane systematic reviews. In general, publication bias is smaller in meta-analyses of more recent studies, indicating their better reliability and supporting the effectiveness of the measures used to reduce publication bias in clinical trials. Our results indicate the need to apply currently underutilized meta-analysis tools handling publication bias based on the statistical significance, especially when studies included in a meta-analysis are not recent.
UNLABELLED: We used a Bayesian hierarchical selection model to study publication bias in 1106 meta-analyses from the Cochrane Database of Systematic Reviews comparing treatment with either placebo or no treatment. For meta-analyses of efficacy, we estimated the ratio of the probability of including statistically significant outcomes favoring treatment to the probability of including other outcomes. For meta-analyses of safety, we estimated the ratio of the probability of including results showing no evidence of adverse effects to the probability of including results demonstrating the presence of adverse effects. RESULTS: In the meta-analyses of efficacy, outcomes favoring treatment had on average a 27% (95% Credible Interval (CI): 18% to 36%) higher probability to be included than other outcomes. In the meta-analyses of safety, results showing no evidence of adverse effects were on average 78% (95% CI: 51% to 113%) more likely to be included than results demonstrating that adverse effects existed. In general, the amount of over-representation of findings favorable to treatment was larger in meta-analyses including older studies. CONCLUSIONS: In the largest study on publication bias in meta-analyses to date, we found evidence of publication bias in Cochrane systematic reviews. In general, publication bias is smaller in meta-analyses of more recent studies, indicating their better reliability and supporting the effectiveness of the measures used to reduce publication bias in clinical trials. Our results indicate the need to apply currently underutilized meta-analysis tools handling publication bias based on the statistical significance, especially when studies included in a meta-analysis are not recent.
Authors: Clare F Heal; Jennifer L Banks; Phoebe D Lepper; Evangelos Kontopantelis; Mieke L van Driel Journal: Cochrane Database Syst Rev Date: 2016-11-07
Authors: Pierre Côté; Jan Hartvigsen; Iben Axén; Charlotte Leboeuf-Yde; Melissa Corso; Heather Shearer; Jessica Wong; Andrée-Anne Marchand; J David Cassidy; Simon French; Gregory N Kawchuk; Silvano Mior; Erik Poulsen; John Srbely; Carlo Ammendolia; Marc-André Blanchette; Jason W Busse; André Bussières; Carolina Cancelliere; Henrik Wulff Christensen; Diana De Carvalho; Katie De Luca; Alister Du Rose; Andreas Eklund; Roger Engel; Guillaume Goncalves; Jeffrey Hebert; Cesar A Hincapié; Maria Hondras; Amanda Kimpton; Henrik Hein Lauridsen; Stanley Innes; Anne-Laure Meyer; David Newell; Søren O'Neill; Isabelle Pagé; Steven Passmore; Stephen M Perle; Jeffrey Quon; Mana Rezai; Maja Stupar; Michael Swain; Andrew Vitiello; Kenneth Weber; Kenneth J Young; Hainan Yu Journal: Chiropr Man Therap Date: 2021-07-20