Elliott E Ridgeon1, Paul J Young, Rinaldo Bellomo, Marta Mucchetti, Rosalba Lembo, Giovanni Landoni. 1. 1Intensive Care Unit, Wellington Regional Hospital, Wellington, New Zealand. 2Medical Research Institute of New Zealand, Wellington, New Zealand. 3Department of Intensive Care, Austin Hospital, Faculty of Medicine, the University of Melbourne and Australian and New Zealand Intensive Care Research Centre, Monash University School of Public Health and Preventive Medicine, Melbourne, VIC, Australia. 4Department of Anesthesia and Intensive Care, IRCCS San Raffaele Scientific Institute, Milan, Italy. 5Vita-Salute San Raffaele University, Milan, Italy.
Abstract
OBJECTIVES: Recent literature has drawn attention to the potential inadequacy of frequentist analysis and threshold p values as tools for reporting outcomes in clinical trials. The fragility index, which is a measure of how many events the statistical significance of a result depends on, has been suggested as a means to aid the interpretation of trial results. This study aimed to calculate the fragility index of clinical trials in critical care medicine reporting a statistically significant effect on mortality (increasing or decreasing mortality). DATA SOURCES: Literature search (PubMed/MEDLINE) to identify all multicenter randomized controlled trials in critical care medicine. STUDY SELECTION: We identified 862 trials; of which 56 fulfilled eligibility criteria and were included in our analysis. DATA EXTRACTION: Calculation of fragility index for trials reporting a statistically significant effect on mortality, and analysis of the relationship between trial characteristics and fragility index. DATA SYNTHESIS: The median fragility index was 2 (interquartile range, 1-3.5), and greater than 40% of trials had a fragility index of less than or equal to 1. 12.5% of trials reported loss to follow-up greater than their fragility index. Trial sample size was positively correlated, and reported p value was negatively correlated, with fragility index. CONCLUSIONS: In critical care trials reporting statistically significant effects on mortality, the findings often depend on a small number of events. Critical care clinicians should be wary of basing decisions on trials with a low fragility index. We advocate the reporting of fragility index for future trials in critical care to aid interpretation and decision making by clinicians.
OBJECTIVES: Recent literature has drawn attention to the potential inadequacy of frequentist analysis and threshold p values as tools for reporting outcomes in clinical trials. The fragility index, which is a measure of how many events the statistical significance of a result depends on, has been suggested as a means to aid the interpretation of trial results. This study aimed to calculate the fragility index of clinical trials in critical care medicine reporting a statistically significant effect on mortality (increasing or decreasing mortality). DATA SOURCES: Literature search (PubMed/MEDLINE) to identify all multicenter randomized controlled trials in critical care medicine. STUDY SELECTION: We identified 862 trials; of which 56 fulfilled eligibility criteria and were included in our analysis. DATA EXTRACTION: Calculation of fragility index for trials reporting a statistically significant effect on mortality, and analysis of the relationship between trial characteristics and fragility index. DATA SYNTHESIS: The median fragility index was 2 (interquartile range, 1-3.5), and greater than 40% of trials had a fragility index of less than or equal to 1. 12.5% of trials reported loss to follow-up greater than their fragility index. Trial sample size was positively correlated, and reported p value was negatively correlated, with fragility index. CONCLUSIONS: In critical care trials reporting statistically significant effects on mortality, the findings often depend on a small number of events. Critical care clinicians should be wary of basing decisions on trials with a low fragility index. We advocate the reporting of fragility index for future trials in critical care to aid interpretation and decision making by clinicians.
Authors: Muhammad Shahzeb Khan; Rohan Kumar Ochani; Asim Shaikh; Muhammad Shariq Usman; Naser Yamani; Safi U Khan; M Hassan Murad; John Mandrola; Rami Doukky; Richard A Krasuski Journal: Circ Cardiovasc Qual Outcomes Date: 2019-12-11