Lifeng Lin1, Haitao Chu2, Mohammad Hassan Murad3, Chuan Hong4, Zhiyong Qu5, Stephen R Cole6, Yong Chen7. 1. Department of Statistics, Florida State University, Tallahassee, FL, USA. 2. Division of Biostatistics, School of Public Health, University of Minnesota, Minneapolis, MN, USA. chux0051@umn.edu. 3. Evidence-Based Practice Center, Mayo Clinic, Rochester, MN, USA. 4. Department of Biostatistics, Harvard School of Public Health, Boston, MA, USA. 5. School of Social Development and Public Policy, Beijing Normal University, Beijing, China. 6. Department of Epidemiology, UNC Gillings School of Global Public Health, Chapel Hill, NC, USA. 7. Department of Biostatistics and Epidemiology, University of Pennsylvania, Philadelphia, PA, USA.
Abstract
BACKGROUND: Decision makers rely on meta-analytic estimates to trade off benefits and harms. Publication bias impairs the validity and generalizability of such estimates. The performance of various statistical tests for publication bias has been largely compared using simulation studies and has not been systematically evaluated in empirical data. METHODS: This study compares seven commonly used publication bias tests (i.e., Begg's rank test, trim-and-fill, Egger's, Tang's, Macaskill's, Deeks', and Peters' regression tests) based on 28,655 meta-analyses available in the Cochrane Library. RESULTS: Egger's regression test detected publication bias more frequently than other tests (15.7% in meta-analyses of binary outcomes and 13.5% in meta-analyses of non-binary outcomes). The proportion of statistically significant publication bias tests was greater for larger meta-analyses, especially for Begg's rank test and the trim-and-fill method. The agreement among Tang's, Macaskill's, Deeks', and Peters' regression tests for binary outcomes was moderately strong (most κ's were around 0.6). Tang's and Deeks' tests had fairly similar performance (κ > 0.9). The agreement among Begg's rank test, the trim-and-fill method, and Egger's regression test was weak or moderate (κ < 0.5). CONCLUSIONS: Given the relatively low agreement between many publication bias tests, meta-analysts should not rely on a single test and may apply multiple tests with various assumptions. Non-statistical approaches to evaluating publication bias (e.g., searching clinical trials registries, records of drug approving agencies, and scientific conference proceedings) remain essential.
BACKGROUND: Decision makers rely on meta-analytic estimates to trade off benefits and harms. Publication bias impairs the validity and generalizability of such estimates. The performance of various statistical tests for publication bias has been largely compared using simulation studies and has not been systematically evaluated in empirical data. METHODS: This study compares seven commonly used publication bias tests (i.e., Begg's rank test, trim-and-fill, Egger's, Tang's, Macaskill's, Deeks', and Peters' regression tests) based on 28,655 meta-analyses available in the Cochrane Library. RESULTS: Egger's regression test detected publication bias more frequently than other tests (15.7% in meta-analyses of binary outcomes and 13.5% in meta-analyses of non-binary outcomes). The proportion of statistically significant publication bias tests was greater for larger meta-analyses, especially for Begg's rank test and the trim-and-fill method. The agreement among Tang's, Macaskill's, Deeks', and Peters' regression tests for binary outcomes was moderately strong (most κ's were around 0.6). Tang's and Deeks' tests had fairly similar performance (κ > 0.9). The agreement among Begg's rank test, the trim-and-fill method, and Egger's regression test was weak or moderate (κ < 0.5). CONCLUSIONS: Given the relatively low agreement between many publication bias tests, meta-analysts should not rely on a single test and may apply multiple tests with various assumptions. Non-statistical approaches to evaluating publication bias (e.g., searching clinical trials registries, records of drug approving agencies, and scientific conference proceedings) remain essential.
Entities:
Keywords:
Cochrane Library; funnel plot; meta-analysis; publication bias; statistical test
Authors: Pablo Alonso-Coello; Holger J Schünemann; Jenny Moberg; Romina Brignardello-Petersen; Elie A Akl; Marina Davoli; Shaun Treweek; Reem A Mustafa; Gabriel Rada; Sarah Rosenbaum; Angela Morelli; Gordon H Guyatt; Andrew D Oxman Journal: BMJ Date: 2016-06-28
Authors: Ashwini Arvind; Zoe N Memel; Lisa L Philpotts; Hui Zheng; Kathleen E Corey; Tracey G Simon Journal: Metabolism Date: 2021-04-21 Impact factor: 13.934