William P T M van Doorn1,2, Patricia M Stassen3,4, Hella F Borggreve3, Maaike J Schalkwijk3, Judith Stoffers3, Otto Bekers1,2, Steven J R Meex1,2. 1. Department of Clinical Chemistry, Central Diagnostic Laboratory, Maastricht University Medical Center, Maastricht, The Netherlands. 2. CARIM School for Cardiovascular Diseases, Maastricht University, Maastricht, The Netherlands. 3. Division of General Internal Medicine, Section Acute Medicine, Department of Internal Medicine, Maastricht University Medical Centre, Maastricht University, Maastricht, The Netherlands. 4. CAPHRI School for Care and Public Health Research Institute, Maastricht University, Maastricht, The Netherlands.
Abstract
INTRODUCTION: Patients with sepsis who present to an emergency department (ED) have highly variable underlying disease severity, and can be categorized from low to high risk. Development of a risk stratification tool for these patients is important for appropriate triage and early treatment. The aim of this study was to develop machine learning models predicting 31-day mortality in patients presenting to the ED with sepsis and to compare these to internal medicine physicians and clinical risk scores. METHODS: A single-center, retrospective cohort study was conducted amongst 1,344 emergency department patients fulfilling sepsis criteria. Laboratory and clinical data that was available in the first two hours of presentation from these patients were randomly partitioned into a development (n = 1,244) and validation dataset (n = 100). Machine learning models were trained and evaluated on the development dataset and compared to internal medicine physicians and risk scores in the independent validation dataset. The primary outcome was 31-day mortality. RESULTS: A number of 1,344 patients were included of whom 174 (13.0%) died. Machine learning models trained with laboratory or a combination of laboratory + clinical data achieved an area-under-the ROC curve of 0.82 (95% CI: 0.80-0.84) and 0.84 (95% CI: 0.81-0.87) for predicting 31-day mortality, respectively. In the validation set, models outperformed internal medicine physicians and clinical risk scores in sensitivity (92% vs. 72% vs. 78%;p<0.001,all comparisons) while retaining comparable specificity (78% vs. 74% vs. 72%;p>0.02). The model had higher diagnostic accuracy with an area-under-the-ROC curve of 0.85 (95%CI: 0.78-0.92) compared to abbMEDS (0.63,0.54-0.73), mREMS (0.63,0.54-0.72) and internal medicine physicians (0.74,0.65-0.82). CONCLUSION: Machine learning models outperformed internal medicine physicians and clinical risk scores in predicting 31-day mortality. These models are a promising tool to aid in risk stratification of patients presenting to the ED with sepsis.
INTRODUCTION:Patients with sepsis who present to an emergency department (ED) have highly variable underlying disease severity, and can be categorized from low to high risk. Development of a risk stratification tool for these patients is important for appropriate triage and early treatment. The aim of this study was to develop machine learning models predicting 31-day mortality in patients presenting to the ED with sepsis and to compare these to internal medicine physicians and clinical risk scores. METHODS: A single-center, retrospective cohort study was conducted amongst 1,344 emergency department patients fulfilling sepsis criteria. Laboratory and clinical data that was available in the first two hours of presentation from these patients were randomly partitioned into a development (n = 1,244) and validation dataset (n = 100). Machine learning models were trained and evaluated on the development dataset and compared to internal medicine physicians and risk scores in the independent validation dataset. The primary outcome was 31-day mortality. RESULTS: A number of 1,344 patients were included of whom 174 (13.0%) died. Machine learning models trained with laboratory or a combination of laboratory + clinical data achieved an area-under-the ROC curve of 0.82 (95% CI: 0.80-0.84) and 0.84 (95% CI: 0.81-0.87) for predicting 31-day mortality, respectively. In the validation set, models outperformed internal medicine physicians and clinical risk scores in sensitivity (92% vs. 72% vs. 78%;p<0.001,all comparisons) while retaining comparable specificity (78% vs. 74% vs. 72%;p>0.02). The model had higher diagnostic accuracy with an area-under-the-ROC curve of 0.85 (95%CI: 0.78-0.92) compared to abbMEDS (0.63,0.54-0.73), mREMS (0.63,0.54-0.72) and internal medicine physicians (0.74,0.65-0.82). CONCLUSION: Machine learning models outperformed internal medicine physicians and clinical risk scores in predicting 31-day mortality. These models are a promising tool to aid in risk stratification of patients presenting to the ED with sepsis.
Authors: Scott M Lundberg; Gabriel Erion; Hugh Chen; Alex DeGrave; Jordan M Prutkin; Bala Nair; Ronit Katz; Jonathan Himmelfarb; Nisha Bansal; Su-In Lee Journal: Nat Mach Intell Date: 2020-01-17
Authors: Mervyn Singer; Clifford S Deutschman; Christopher Warren Seymour; Manu Shankar-Hari; Djillali Annane; Michael Bauer; Rinaldo Bellomo; Gordon R Bernard; Jean-Daniel Chiche; Craig M Coopersmith; Richard S Hotchkiss; Mitchell M Levy; John C Marshall; Greg S Martin; Steven M Opal; Gordon D Rubenfeld; Tom van der Poll; Jean-Louis Vincent; Derek C Angus Journal: JAMA Date: 2016-02-23 Impact factor: 56.272
Authors: Mitchell M Levy; Mitchell P Fink; John C Marshall; Edward Abraham; Derek Angus; Deborah Cook; Jonathan Cohen; Steven M Opal; Jean-Louis Vincent; Graham Ramsay Journal: Crit Care Med Date: 2003-04 Impact factor: 7.598
Authors: Scott M Lundberg; Bala Nair; Monica S Vavilala; Mayumi Horibe; Michael J Eisses; Trevor Adams; David E Liston; Daniel King-Wai Low; Shu-Fang Newman; Jerry Kim; Su-In Lee Journal: Nat Biomed Eng Date: 2018-10-10 Impact factor: 25.671
Authors: Fereshteh S Bashiri; John R Caskey; Anoop Mayampurath; Nicole Dussault; Jay Dumanian; Sivasubramanium V Bhavani; Kyle A Carey; Emily R Gilbert; Christopher J Winslow; Nirav S Shah; Dana P Edelson; Majid Afshar; Matthew M Churpek Journal: J Am Med Inform Assoc Date: 2022-09-12 Impact factor: 7.942
Authors: Samuel W Terman; Herm J Lamberink; Geertruida Slinger; Willem M Otte; James F Burke; Kees P J Braun Journal: Epilepsia Date: 2021-10-11 Impact factor: 6.740
Authors: Stephen Chi; Aixia Guo; Kevin Heard; Seunghwan Kim; Randi Foraker; Patrick White; Nathan Moore Journal: Med Care Date: 2022-05-01 Impact factor: 2.983
Authors: Amin Naemi; Thomas Schmidt; Marjan Mansourvar; Mohammad Naghavi-Behzad; Ali Ebrahimi; Uffe Kock Wiil Journal: BMJ Open Date: 2021-11-02 Impact factor: 2.692