Aiwen Xing1, Haitao Chu2, Lifeng Lin3. 1. Department of Statistics, Florida State University, Tallahassee, FL, USA. 2. Division of Biostatistics, School of Public Health, University of Minnesota, Minneapolis, MN, USA. 3. Department of Statistics, Florida State University, Tallahassee, FL, USA. Electronic address: linl@stat.fsu.edu.
Abstract
BACKGROUND AND OBJECTIVES: The network meta-analysis (NMA) is frequently used to synthesize evidence for multiple treatment comparisons, but its complexity may affect the robustness (or fragility) of the results. The fragility index (FI) is recently proposed to assess the fragility of the results from clinical studies and from pairwise meta-analyses. We extend the FI to NMAs with binary outcomes. METHODS: We define the FI for each treatment comparison in NMAs. It quantifies the minimal number of events necessary to be modified for altering the comparison's statistical significance. We introduce an algorithm to derive the FI and visualizations of the process. A worked example of smoking cessation data is used to illustrate the proposed methods. RESULTS: Some treatment comparisons had small FIs; their significance (or nonsignificance) could be altered by modifying a few events' status. They were related to various factors, such as P-values, event counts, and sample sizes, in the original NMA. After modifying event status, treatment ranking measures were also changed to different extents. CONCLUSION: Many NMAs include insufficiently compared treatments, small event counts, or small sample sizes; their results are potentially fragile. The FI offers a useful tool to evaluate treatment comparisons' robustness and reliability.
BACKGROUND AND OBJECTIVES: The network meta-analysis (NMA) is frequently used to synthesize evidence for multiple treatment comparisons, but its complexity may affect the robustness (or fragility) of the results. The fragility index (FI) is recently proposed to assess the fragility of the results from clinical studies and from pairwise meta-analyses. We extend the FI to NMAs with binary outcomes. METHODS: We define the FI for each treatment comparison in NMAs. It quantifies the minimal number of events necessary to be modified for altering the comparison's statistical significance. We introduce an algorithm to derive the FI and visualizations of the process. A worked example of smoking cessation data is used to illustrate the proposed methods. RESULTS: Some treatment comparisons had small FIs; their significance (or nonsignificance) could be altered by modifying a few events' status. They were related to various factors, such as P-values, event counts, and sample sizes, in the original NMA. After modifying event status, treatment ranking measures were also changed to different extents. CONCLUSION: Many NMAs include insufficiently compared treatments, small event counts, or small sample sizes; their results are potentially fragile. The FI offers a useful tool to evaluate treatment comparisons' robustness and reliability.
Authors: Orestis Efthimiou; Thomas P A Debray; Gert van Valkenhoef; Sven Trelle; Klea Panayidou; Karel G M Moons; Johannes B Reitsma; Aijing Shang; Georgia Salanti Journal: Res Synth Methods Date: 2016-01-11 Impact factor: 5.273
Authors: Milo A Puhan; Holger J Schünemann; Mohammad Hassan Murad; Tianjing Li; Romina Brignardello-Petersen; Jasvinder A Singh; Alfons G Kessels; Gordon H Guyatt Journal: BMJ Date: 2014-09-24
Authors: Clément Palpacuer; Karima Hammas; Renan Duprez; Bruno Laviolle; John P A Ioannidis; Florian Naudet Journal: BMC Med Date: 2019-09-16 Impact factor: 8.775
Authors: Benjamin R Baer; Mario Gaudino; Mary Charlson; Stephen E Fremes; Martin T Wells Journal: Proc Natl Acad Sci U S A Date: 2021-12-07 Impact factor: 12.779