Chang Xu1, Ling Li1, Lifeng Lin2, Haitao Chu3, Lehana Thabane4, Kang Zou1, Xin Sun5. 1. Chinese Evidence-Based Medicine Center & Cochrane China, West China Hospital, Sichuan University, Chengdu, China. 2. Department of Statistics, Florida State University, Tallahassee, FL, USA. 3. Division of Biostatistics, School of Public Health, University of Minnesota, Minneapolis, MN, USA. 4. Department of Health Research Methods, Evidence, and Impact, McMaster University, 1280 Main Street West, Hamilton, Ontario L8S 4K1, Canada; Biostatistics Unit, Father Sean O'Sullivan Research Centre, St Joseph's Healthcare, 3rd Floor, Martha Wing, Room H-325, 50 Charlton Avenue East, Hamilton, Ontario L8N 4A6, Canada. 5. Chinese Evidence-Based Medicine Center & Cochrane China, West China Hospital, Sichuan University, Chengdu, China. Electronic address: sunx79@hotmail.com.
Abstract
OBJECTIVES: Classical meta-analyses routinely treated studies with no events in both arms noninformative and excluded them from analyses. This study assessed whether such studies contain information and have an influence on the conclusions of meta-analyses. STUDY DESIGN AND SETTING: We collected meta-analyses of binary outcomes with at least one study having no events in both arms from Cochrane systematic reviews (2003-2018). We used the generalized linear mixed model to reanalyze these meta-analyses by two approaches: one including studies with no events in both arms and one excluding such studies. The magnitude and direction of odds ratio (OR), P value, and width of 95% confidence interval (CI) were compared. A simulation study was conducted to examine the robustness of results. RESULTS: We identified 442 meta-analyses. In comparing paired meta-analyses that included studies with no events in both arms vs. those not, 8 (1.80%) resulted in different directions on OR; 41 (9.28%) altered conclusions on statistical significance. Substantial changes occurred on P value (55.66% increased and 44.12% decreased) and the width of 95% CI (50.68% inflated and 49.32% declined) when excluding studies with no events. Simulation study confirmed these findings. CONCLUSION: Studies with no events in both arms are not necessarily noninformative. Excluding such studies may alter conclusions.
OBJECTIVES: Classical meta-analyses routinely treated studies with no events in both arms noninformative and excluded them from analyses. This study assessed whether such studies contain information and have an influence on the conclusions of meta-analyses. STUDY DESIGN AND SETTING: We collected meta-analyses of binary outcomes with at least one study having no events in both arms from Cochrane systematic reviews (2003-2018). We used the generalized linear mixed model to reanalyze these meta-analyses by two approaches: one including studies with no events in both arms and one excluding such studies. The magnitude and direction of odds ratio (OR), P value, and width of 95% confidence interval (CI) were compared. A simulation study was conducted to examine the robustness of results. RESULTS: We identified 442 meta-analyses. In comparing paired meta-analyses that included studies with no events in both arms vs. those not, 8 (1.80%) resulted in different directions on OR; 41 (9.28%) altered conclusions on statistical significance. Substantial changes occurred on P value (55.66% increased and 44.12% decreased) and the width of 95% CI (50.68% inflated and 49.32% declined) when excluding studies with no events. Simulation study confirmed these findings. CONCLUSION: Studies with no events in both arms are not necessarily noninformative. Excluding such studies may alter conclusions.