Eduardo A Trujillo Rivera1, James M Chamberlain2, Anita K Patel3, Hiroki Morizono4, Julia A Heneghan3, Murray M Pollack3. 1. George Washington University School of Medicine and Health Sciences, Washington, DC. 2. Department of Pediatrics, Division of Emergency Medicine, Children's National Hospital and George Washington University School of Medicine and Health Sciences, Washington, DC. 3. Department of Pediatrics, Division of Critical Care Medicine, Children's National Hospital and George Washington University School of Medicine and Health Sciences, Washington, DC. 4. Children's National Research Institute, Associate Research Professor of Genomics and Precision Medicine, George Washington University School of Medicine and Health Sciences, Washington, DC.
Abstract
OBJECTIVES: Assess a machine learning method of serially updated mortality risk. DESIGN: Retrospective analysis of a national database (Health Facts; Cerner Corporation, Kansas City, MO). SETTING: Hospitals caring for children in ICUs. PATIENTS: A total of 27,354 admissions cared for in ICUs from 2009 to 2018. INTERVENTIONS: None. MAIN OUTCOME: Hospital mortality risk estimates determined at 6-hour time periods during care in the ICU. Models were truncated at 180 hours due to decreased sample size secondary to discharges and deaths. MEASUREMENTS AND MAIN RESULTS: The Criticality Index, based on physiology, therapy, and care intensity, was computed for each admission for each time period and calibrated to hospital mortality risk (Criticality Index-Mortality [CI-M]) at each of 29 time periods (initial assessment: 6 hr; last assessment: 180 hr). Performance metrics and clinical validity were determined from the held-out test sample (n = 3,453, 13%). Discrimination assessed with the area under the receiver operating characteristic curve was 0.852 (95% CI, 0.843-0.861) overall and greater than or equal to 0.80 for all individual time periods. Calibration assessed by the Hosmer-Lemeshow goodness-of-fit test showed good fit overall (p = 0.196) and was statistically not significant for 28 of the 29 time periods. Calibration plots for all models revealed the intercept ranged from--0.002 to 0.009, the slope ranged from 0.867 to 1.415, and the R2 ranged from 0.862 to 0.989. Clinical validity assessed using population trajectories and changes in the risk status of admissions (clinical volatility) revealed clinical trajectories consistent with clinical expectations and greater clinical volatility in deaths than survivors (p < 0.001). CONCLUSIONS: Machine learning models incorporating physiology, therapy, and care intensity can track changes in hospital mortality risk during intensive care. The CI-M's framework and modeling method are potentially applicable to monitoring clinical improvement and deterioration in real time.
OBJECTIVES: Assess a machine learning method of serially updated mortality risk. DESIGN: Retrospective analysis of a national database (Health Facts; Cerner Corporation, Kansas City, MO). SETTING: Hospitals caring for children in ICUs. PATIENTS: A total of 27,354 admissions cared for in ICUs from 2009 to 2018. INTERVENTIONS: None. MAIN OUTCOME: Hospital mortality risk estimates determined at 6-hour time periods during care in the ICU. Models were truncated at 180 hours due to decreased sample size secondary to discharges and deaths. MEASUREMENTS AND MAIN RESULTS: The Criticality Index, based on physiology, therapy, and care intensity, was computed for each admission for each time period and calibrated to hospital mortality risk (Criticality Index-Mortality [CI-M]) at each of 29 time periods (initial assessment: 6 hr; last assessment: 180 hr). Performance metrics and clinical validity were determined from the held-out test sample (n = 3,453, 13%). Discrimination assessed with the area under the receiver operating characteristic curve was 0.852 (95% CI, 0.843-0.861) overall and greater than or equal to 0.80 for all individual time periods. Calibration assessed by the Hosmer-Lemeshow goodness-of-fit test showed good fit overall (p = 0.196) and was statistically not significant for 28 of the 29 time periods. Calibration plots for all models revealed the intercept ranged from--0.002 to 0.009, the slope ranged from 0.867 to 1.415, and the R2 ranged from 0.862 to 0.989. Clinical validity assessed using population trajectories and changes in the risk status of admissions (clinical volatility) revealed clinical trajectories consistent with clinical expectations and greater clinical volatility in deaths than survivors (p < 0.001). CONCLUSIONS: Machine learning models incorporating physiology, therapy, and care intensity can track changes in hospital mortality risk during intensive care. The CI-M's framework and modeling method are potentially applicable to monitoring clinical improvement and deterioration in real time.
Authors: Nathan Brajer; Brian Cozzi; Michael Gao; Marshall Nichols; Mike Revoir; Suresh Balu; Joseph Futoma; Jonathan Bae; Noppon Setji; Adrian Hernandez; Mark Sendak Journal: JAMA Netw Open Date: 2020-02-05
Authors: Melissa D Aczon; David R Ledbetter; Eugene Laksana; Long V Ho; Randall C Wetzel Journal: Pediatr Crit Care Med Date: 2021-06-01 Impact factor: 3.971
Authors: Eduardo A Trujillo Rivera; Anita K Patel; James M Chamberlain; T Elizabeth Workman; Julia A Heneghan; Douglas Redd; Hiroki Morizono; Dongkyu Kim; James E Bost; Murray M Pollack Journal: Pediatr Crit Care Med Date: 2021-01-01 Impact factor: 3.971
Authors: Eduardo A Trujillo Rivera; Anita K Patel; Qing Zeng-Treitler; James M Chamberlain; James E Bost; Julia A Heneghan; Hiroki Morizono; Murray M Pollack Journal: Pediatr Crit Care Med Date: 2021-01-01 Impact factor: 3.971