Youfeng Zhu1, Rui Zhang2, Xiaoling Ye3, Houqiang Liu4, Jianrui Wei5. 1. Department of Intensive Care Unit, Guangzhou Red Cross Hospital, Jinan University, Guangzhou 510220, Guangdong Province, China. Electronic address: 151276953@qq.com. 2. Department of Intensive Care Unit, Guangzhou Red Cross Hospital, Jinan University, Guangzhou 510220, Guangdong Province, China. Electronic address: 63281796@qq.com. 3. Department of Intensive Care Unit, Guangzhou Red Cross Hospital, Jinan University, Guangzhou 510220, Guangdong Province, China. Electronic address: 1718768452@qq.com. 4. Department of Intensive Care Unit, Guangzhou Red Cross Hospital, Jinan University, Guangzhou 510220, Guangdong Province, China. Electronic address: 15975406024@139.com. 5. Guangzhou Women and Children's Medical Center, Guangzhou 510220, Guangdong Province, China. Electronic address: Jianruiw@163.com.
Abstract
INTRODUCTION: The discrimination and calibration accuracy of prediction models tends to become poor over time. The performance of predictive models should be reevaluated periodically. The aim of this study was to reassess the discrimination of the six commonly used models for predicting 28-day mortality in patients with sepsis based on the Sepsis 3.0 criteria. METHODS: Patient data were extracted from the fourth edition of the Medical Information Mart for Critical Care (MIMIC IV) database. The systemic inflammatory response syndrome (SIRS), Sequential Organ Failure Assessment (SOFA), Oxford Acute Severity of Illness Score (OASIS), Logistic Organ Dysfunction System (LODS), and Simplified Acute Physiology Score II (SAPS II) and III (SAPS III) scores were calculated and collected. The area under the receiver operating characteristic curve (AUROC) was used to compare the discrimination abilities of the models using non-parametric Wilcoxon statistics. The Delong method was used to perform pairwise comparisons of the AUROCs of the models. Multiple subgroup analyses for age, body mass index, and sex were performed with regard to the 28-day mortality prediction of the models. RESULTS: A total of 12 691 patients were included. The mean age of the patients was 65.97 ± 15.77 years; 7673 patients (60.50%) were male. The mean SIRS, SOFA, OASIS, SAPS II, LODS, and SAPS III scores were higher in the non-survivor group than in the survivor group. The discrimination for 28-day mortality with the SAPS III (AUROC 0.812, 95% confidence interval (CI) 0.802-0.822) and LODS (AUROC 0.804, 95% CI 0.743-0.765) models was superior to that of the SIRS (AUROC 0.575, 95% CI 0.562-0.589), SOFA (AUROC 0.612, 95% CI 0.598-0.626), OASIS (AUROC 0.753, 95% CI 0.742-0.764), and SAPS II (AUROC 0.754, 95% CI 0.743-0.765) models. The Youden index of the SAPS III model was 0.484, which was the highest among the models. Subgroup analyses showed similar results to the overall results. CONCLUSIONS: The discrimination for 28-day mortality with the SAPS III and LODS models was superior to that of the SIRS, SOFA, OASIS, and SAPS II models. The SAPS III model showed the best discrimination capacity for 28-day mortality compared with the other models.
INTRODUCTION: The discrimination and calibration accuracy of prediction models tends to become poor over time. The performance of predictive models should be reevaluated periodically. The aim of this study was to reassess the discrimination of the six commonly used models for predicting 28-day mortality in patients with sepsis based on the Sepsis 3.0 criteria. METHODS: Patient data were extracted from the fourth edition of the Medical Information Mart for Critical Care (MIMIC IV) database. The systemic inflammatory response syndrome (SIRS), Sequential Organ Failure Assessment (SOFA), Oxford Acute Severity of Illness Score (OASIS), Logistic Organ Dysfunction System (LODS), and Simplified Acute Physiology Score II (SAPS II) and III (SAPS III) scores were calculated and collected. The area under the receiver operating characteristic curve (AUROC) was used to compare the discrimination abilities of the models using non-parametric Wilcoxon statistics. The Delong method was used to perform pairwise comparisons of the AUROCs of the models. Multiple subgroup analyses for age, body mass index, and sex were performed with regard to the 28-day mortality prediction of the models. RESULTS: A total of 12 691 patients were included. The mean age of the patients was 65.97 ± 15.77 years; 7673 patients (60.50%) were male. The mean SIRS, SOFA, OASIS, SAPS II, LODS, and SAPS III scores were higher in the non-survivor group than in the survivor group. The discrimination for 28-day mortality with the SAPS III (AUROC 0.812, 95% confidence interval (CI) 0.802-0.822) and LODS (AUROC 0.804, 95% CI 0.743-0.765) models was superior to that of the SIRS (AUROC 0.575, 95% CI 0.562-0.589), SOFA (AUROC 0.612, 95% CI 0.598-0.626), OASIS (AUROC 0.753, 95% CI 0.742-0.764), and SAPS II (AUROC 0.754, 95% CI 0.743-0.765) models. The Youden index of the SAPS III model was 0.484, which was the highest among the models. Subgroup analyses showed similar results to the overall results. CONCLUSIONS: The discrimination for 28-day mortality with the SAPS III and LODS models was superior to that of the SIRS, SOFA, OASIS, and SAPS II models. The SAPS III model showed the best discrimination capacity for 28-day mortality compared with the other models.