Joon Lee1, David M Maslove2. 1. School of Public Health and Health Systems, University of Waterloo, Waterloo, Ontario, Canada joon.lee@uwaterloo.ca. 2. Department of Medicine & Critical Care Program, Queen's University, Kingston, Ontario, Canada.
Abstract
PURPOSE: Severity of illness (SOI) scores are traditionally based on archival data collected from a wide range of clinical settings. Mortality prediction using SOI scores tends to underperform when applied to contemporary cases or those that differ from the case-mix of the original derivation cohorts. We investigated the use of local clinical data captured from hospital electronic medical records (EMRs) to improve the predictive performance of traditional severity of illness scoring. METHODS: We conducted a retrospective analysis using data from the Multiparameter Intelligent Monitoring in Intensive Care II (MIMIC-II) database, which contains clinical data from the Beth Israel Deaconess Medical Center in Boston, Massachusetts. A total of 17 490 intensive care unit (ICU) admissions with complete data were included, from 4 different service types: medical ICU, surgical ICU, coronary care unit, and cardiac surgery recovery unit. We developed customized SOI scores trained on data from each service type, using the clinical variables employed in the Simplified Acute Physiology Score (SAPS). In-hospital, 30-day, and 2-year mortality predictions were compared with those obtained from using the original SAPS using the area under the receiver-operating characteristics curve (AUROC) as well as the area under the precision-recall curve (AUPRC). Test performance in different cohorts stratified by severity of organ injury was also evaluated. RESULTS: Most customized scores (30 of 39) significantly outperformed SAPS with respect to both AUROC and AUPRC. Enhancements over SAPS were greatest for patients undergoing cardiovascular surgery and for prediction of 2-year mortality. CONCLUSIONS: Custom models based on ICU-specific data provided better mortality prediction than traditional SAPS scoring using the same predictor variables. Our local data approach demonstrates the value of electronic data capture in the ICU, of secondary uses of EMR data, and of local customization of SOI scoring.
PURPOSE: Severity of illness (SOI) scores are traditionally based on archival data collected from a wide range of clinical settings. Mortality prediction using SOI scores tends to underperform when applied to contemporary cases or those that differ from the case-mix of the original derivation cohorts. We investigated the use of local clinical data captured from hospital electronic medical records (EMRs) to improve the predictive performance of traditional severity of illness scoring. METHODS: We conducted a retrospective analysis using data from the Multiparameter Intelligent Monitoring in Intensive Care II (MIMIC-II) database, which contains clinical data from the Beth Israel Deaconess Medical Center in Boston, Massachusetts. A total of 17 490 intensive care unit (ICU) admissions with complete data were included, from 4 different service types: medical ICU, surgical ICU, coronary care unit, and cardiac surgery recovery unit. We developed customized SOI scores trained on data from each service type, using the clinical variables employed in the Simplified Acute Physiology Score (SAPS). In-hospital, 30-day, and 2-year mortality predictions were compared with those obtained from using the original SAPS using the area under the receiver-operating characteristics curve (AUROC) as well as the area under the precision-recall curve (AUPRC). Test performance in different cohorts stratified by severity of organ injury was also evaluated. RESULTS: Most customized scores (30 of 39) significantly outperformed SAPS with respect to both AUROC and AUPRC. Enhancements over SAPS were greatest for patients undergoing cardiovascular surgery and for prediction of 2-year mortality. CONCLUSIONS: Custom models based on ICU-specific data provided better mortality prediction than traditional SAPS scoring using the same predictor variables. Our local data approach demonstrates the value of electronic data capture in the ICU, of secondary uses of EMR data, and of local customization of SOI scoring.
Authors: Christopher M Horvat; Henry Ogoe; Sajel Kantawala; Alicia K Au; Ericka L Fink; Eric Yablonsky; Patrick M Kochanek; Srinivasan Suresh; Robert S B Clark Journal: Pediatr Crit Care Med Date: 2019-08 Impact factor: 3.624
Authors: Christine W Nibbelink; Janay R Young; Jane M Carrington; Barbara B Brewer Journal: Crit Care Nurs Clin North Am Date: 2018-04-04 Impact factor: 1.326
Authors: Eline Stenwig; Giampiero Salvi; Pierluigi Salvo Rossi; Nils Kristian Skjærvold Journal: BMC Med Res Methodol Date: 2022-02-27 Impact factor: 4.615
Authors: Daniel E Leisman; Michael O Harhay; David J Lederer; Michael Abramson; Alex A Adjei; Jan Bakker; Zuhair K Ballas; Esther Barreiro; Scott C Bell; Rinaldo Bellomo; Jonathan A Bernstein; Richard D Branson; Vito Brusasco; James D Chalmers; Sudhansu Chokroverty; Giuseppe Citerio; Nancy A Collop; Colin R Cooke; James D Crapo; Gavin Donaldson; Dominic A Fitzgerald; Emma Grainger; Lauren Hale; Felix J Herth; Patrick M Kochanek; Guy Marks; J Randall Moorman; David E Ost; Michael Schatz; Aziz Sheikh; Alan R Smyth; Iain Stewart; Paul W Stewart; Erik R Swenson; Ronald Szymusiak; Jean-Louis Teboul; Jean-Louis Vincent; Jadwiga A Wedzicha; David M Maslove Journal: Crit Care Med Date: 2020-05 Impact factor: 7.598