George F Borm1, A Rogier T Donders. 1. Department of Epidemiology, Biostatistics and HTA (EBH 133), Radboud University Nijmegen Medical Centre, Nijmegen, The Netherlands. g.borm@ebh.umcn.nl
Abstract
OBJECTIVE: To estimate the extent to which the practice of periodically updating meta-analyses causes inflation of the type I error and then to compare the estimate with the inflation caused by publication bias. We also present a simple method to adjust for the inflation associated with updating meta-analyses. STUDY DESIGN AND SETTING: Simulations were used to estimate the error rates. RESULTS: In general, updating meta-analyses caused 2- to 5-fold inflation of the type I error rates, which exceeded the inflation caused by publication bias. As a rule of thumb, the results of a meta-analysis are robust up to 5, 10, 15, or 22 updates, if the P-value multiplied by 4, 6, 8, or 10 remains below the desired significance level. CONCLUSION: Meta-analyses are likely to be updated until a clear conclusion is reached. Therefore, it is important to take the inflation of the error rate into account to interpret the results correctly.
OBJECTIVE: To estimate the extent to which the practice of periodically updating meta-analyses causes inflation of the type I error and then to compare the estimate with the inflation caused by publication bias. We also present a simple method to adjust for the inflation associated with updating meta-analyses. STUDY DESIGN AND SETTING: Simulations were used to estimate the error rates. RESULTS: In general, updating meta-analyses caused 2- to 5-fold inflation of the type I error rates, which exceeded the inflation caused by publication bias. As a rule of thumb, the results of a meta-analysis are robust up to 5, 10, 15, or 22 updates, if the P-value multiplied by 4, 6, 8, or 10 remains below the desired significance level. CONCLUSION: Meta-analyses are likely to be updated until a clear conclusion is reached. Therefore, it is important to take the inflation of the error rate into account to interpret the results correctly.
Authors: Kristian Thorlund; Georgina Imberger; Michael Walsh; Rong Chu; Christian Gluud; Jørn Wetterslev; Gordon Guyatt; Philip J Devereaux; Lehana Thabane Journal: PLoS One Date: 2011-10-18 Impact factor: 3.240