Yanan Ren1, Lifeng Lin2, Qinshu Lian1, Hui Zou3, Haitao Chu4. 1. Division of Biostatistics, School of Public Health, University of Minnesota, Minneapolis, MN, USA. 2. Department of Statistics, Florida State University, Tallahassee, FL, USA. 3. School of Statistics, University of Minnesota, Minneapolis, MN, USA. 4. Division of Biostatistics, School of Public Health, University of Minnesota, Minneapolis, MN, USA. chux0051@umn.edu.
Abstract
BACKGROUND: Meta-analysis combines multiple independent studies, which can increase power and provide better estimates. However, it is unclear how best to deal with studies with zero events; such studies are also known as double-zero-event studies (DZS). Several statistical methods have been proposed, but the agreement among different approaches has not been systematically assessed using real-world published systematic reviews. METHODS: The agreement of five commonly used methods (i.e., the inverse-variance, Mantel-Haenszel, Peto, Bayesian, and exact methods) was assessed using the Cohen's κ coefficients using 368 meta-analyses with rare events selected from the Cochrane Database of Systematic Reviews. Three continuity corrections, including the correction of a constant 0.5, the treatment arm continuity correction (TACC), and the empirical (EMP) correction, were used to handle DZS when applying inverse-variance and Mantel-Haenszel methods. RESULTS: When the proportion of DZS studies was lower than 50% in a meta-analysis, different methods had moderately high agreement. However, when this proportion was increased to be over 50%, the agreement among the methods decreased to different extents. For the Bayesian, exact, and Peto methods and the inverse-variance and Mantel-Haenszel methods using the EMP correction, their agreement coefficients with the inverse-variance and Mantel-Haenszel methods using a constant 0.5 and TACC decreased from larger than 0.70 to smaller than 0.30. In contrast, the agreement coefficients only decreased slightly among the Bayesian, exact, and Peto methods and the inverse-variance and Mantel-Haenszel methods using the EMP correction. CONCLUSIONS: To utilize all available information and reduce research waste and avoid overestimating the effect, meta-analysts should incorporate DZS, rather than simply removing them. The Peto and other conventional methods with continuity correction should be avoided when the proportion of DZS is extremely high. The exact and Bayesian methods are highly recommended, except when none of the included studies have an event in one or both treatment arms.
BACKGROUND: Meta-analysis combines multiple independent studies, which can increase power and provide better estimates. However, it is unclear how best to deal with studies with zero events; such studies are also known as double-zero-event studies (DZS). Several statistical methods have been proposed, but the agreement among different approaches has not been systematically assessed using real-world published systematic reviews. METHODS: The agreement of five commonly used methods (i.e., the inverse-variance, Mantel-Haenszel, Peto, Bayesian, and exact methods) was assessed using the Cohen's κ coefficients using 368 meta-analyses with rare events selected from the Cochrane Database of Systematic Reviews. Three continuity corrections, including the correction of a constant 0.5, the treatment arm continuity correction (TACC), and the empirical (EMP) correction, were used to handle DZS when applying inverse-variance and Mantel-Haenszel methods. RESULTS: When the proportion of DZS studies was lower than 50% in a meta-analysis, different methods had moderately high agreement. However, when this proportion was increased to be over 50%, the agreement among the methods decreased to different extents. For the Bayesian, exact, and Peto methods and the inverse-variance and Mantel-Haenszel methods using the EMP correction, their agreement coefficients with the inverse-variance and Mantel-Haenszel methods using a constant 0.5 and TACC decreased from larger than 0.70 to smaller than 0.30. In contrast, the agreement coefficients only decreased slightly among the Bayesian, exact, and Peto methods and the inverse-variance and Mantel-Haenszel methods using the EMP correction. CONCLUSIONS: To utilize all available information and reduce research waste and avoid overestimating the effect, meta-analysts should incorporate DZS, rather than simply removing them. The Peto and other conventional methods with continuity correction should be avoided when the proportion of DZS is extremely high. The exact and Bayesian methods are highly recommended, except when none of the included studies have an event in one or both treatment arms.
Authors: Ian Shrier; Jean-François Boivin; Russell J Steele; Robert W Platt; Andrea Furlan; Ritsuko Kakuma; James Brophy; Michel Rossignol Journal: Am J Epidemiol Date: 2007-08-21 Impact factor: 4.897
Authors: Joshua D Wallach; Kun Wang; Audrey D Zhang; Deanna Cheng; Holly K Grossetta Nardini; Haiqun Lin; Michael B Bracken; Mayur Desai; Harlan M Krumholz; Joseph S Ross Journal: BMJ Date: 2020-02-05