Jesse D Raffa1, Alistair E W Johnson1, Zach O'Brien2, Tom J Pollard1, Roger G Mark1,3, Leo A Celi1,3, David Pilcher1,2,3,4,5,6,7, Omar Badawi7. 1. Institute for Medical Engineering and Science, Massachusetts Institute of Technology, Cambridge, MA. 2. Austin Health, Melbourne, VIC, Australia. 3. Beth Israel Deaconess Medical Center, Boston, MA. 4. Department of Intensive Care and Hyperbaric Medicine, Alfred Hospital, Melbourne, VIC, Australia. 5. Australian and New Zealand Intensive Care Research Centre, School of Public Health and Preventive Medicine, Monash University, Alfred Hospital, Melbourne, VIC, Australia. 6. Centre for Outcome and Resource Evaluation, Australian and New Zealand Intensive Care Society, Melbourne, VIC, Australia. 7. Connected Care Informatics, Philips Healthcare, Baltimore, MD.
Abstract
OBJECTIVES: To develop and demonstrate the feasibility of a Global Open Source Severity of Illness Score (GOSSIS)-1 for critical care patients, which generalizes across healthcare systems and countries. DESIGN: A merger of several critical care multicenter cohorts derived from registry and electronic health record data. Data were split into training (70%) and test (30%) sets, using each set exclusively for development and evaluation, respectively. Missing data were imputed when not available. SETTING/PATIENTS: Two large multicenter datasets from Australia and New Zealand (Australian and New Zealand Intensive Care Society Adult Patient Database [ANZICS-APD]) and the United States (eICU Collaborative Research Database [eICU-CRD]) representing 249,229 and 131,051 patients, respectively. ANZICS-APD and eICU-CRD contributed data from 162 and 204 hospitals, respectively. The cohort included all ICU admissions discharged in 2014-2015, excluding patients less than 16 years old, admissions less than 6 hours, and those with a previous ICU stay. INTERVENTIONS: Not applicable. MEASUREMENTS AND MAIN RESULTS: GOSSIS-1 uses data collected during the ICU stay's first 24 hours, including extrema values for vital signs and laboratory results, admission diagnosis, the Glasgow Coma Scale, chronic comorbidities, and admission/demographic variables. The datasets showed significant variation in admission-related variables, case-mix, and average physiologic state. Despite this heterogeneity, test set discrimination of GOSSIS-1 was high (area under the receiver operator characteristic curve [AUROC], 0.918; 95% CI, 0.915-0.921) and calibration was excellent (standardized mortality ratio [SMR], 0.986; 95% CI, 0.966-1.005; Brier score, 0.050). Performance was held within ANZICS-APD (AUROC, 0.925; SMR, 0.982; Brier score, 0.047) and eICU-CRD (AUROC, 0.904; SMR, 0.992; Brier score, 0.055). Compared with GOSSIS-1, Acute Physiology and Chronic Health Evaluation (APACHE)-IIIj (ANZICS-APD) and APACHE-IVa (eICU-CRD), had worse discrimination with AUROCs of 0.904 and 0.869, and poorer calibration with SMRs of 0.594 and 0.770, and Brier scores of 0.059 and 0.063, respectively. CONCLUSIONS: GOSSIS-1 is a modern, free, open-source inhospital mortality prediction algorithm for critical care patients, achieving excellent discrimination and calibration across three countries.
OBJECTIVES: To develop and demonstrate the feasibility of a Global Open Source Severity of Illness Score (GOSSIS)-1 for critical care patients, which generalizes across healthcare systems and countries. DESIGN: A merger of several critical care multicenter cohorts derived from registry and electronic health record data. Data were split into training (70%) and test (30%) sets, using each set exclusively for development and evaluation, respectively. Missing data were imputed when not available. SETTING/PATIENTS: Two large multicenter datasets from Australia and New Zealand (Australian and New Zealand Intensive Care Society Adult Patient Database [ANZICS-APD]) and the United States (eICU Collaborative Research Database [eICU-CRD]) representing 249,229 and 131,051 patients, respectively. ANZICS-APD and eICU-CRD contributed data from 162 and 204 hospitals, respectively. The cohort included all ICU admissions discharged in 2014-2015, excluding patients less than 16 years old, admissions less than 6 hours, and those with a previous ICU stay. INTERVENTIONS: Not applicable. MEASUREMENTS AND MAIN RESULTS: GOSSIS-1 uses data collected during the ICU stay's first 24 hours, including extrema values for vital signs and laboratory results, admission diagnosis, the Glasgow Coma Scale, chronic comorbidities, and admission/demographic variables. The datasets showed significant variation in admission-related variables, case-mix, and average physiologic state. Despite this heterogeneity, test set discrimination of GOSSIS-1 was high (area under the receiver operator characteristic curve [AUROC], 0.918; 95% CI, 0.915-0.921) and calibration was excellent (standardized mortality ratio [SMR], 0.986; 95% CI, 0.966-1.005; Brier score, 0.050). Performance was held within ANZICS-APD (AUROC, 0.925; SMR, 0.982; Brier score, 0.047) and eICU-CRD (AUROC, 0.904; SMR, 0.992; Brier score, 0.055). Compared with GOSSIS-1, Acute Physiology and Chronic Health Evaluation (APACHE)-IIIj (ANZICS-APD) and APACHE-IVa (eICU-CRD), had worse discrimination with AUROCs of 0.904 and 0.869, and poorer calibration with SMRs of 0.594 and 0.770, and Brier scores of 0.059 and 0.063, respectively. CONCLUSIONS: GOSSIS-1 is a modern, free, open-source inhospital mortality prediction algorithm for critical care patients, achieving excellent discrimination and calibration across three countries.
Authors: Antonio Paulo Nassar; Amilcar Oshiro Mocelin; André Luiz Baptiston Nunes; Fabio Poianas Giannini; Leonardo Brauer; Fabio Moreira Andrade; Carlos Augusto Dias Journal: J Crit Care Date: 2011-10-26 Impact factor: 3.425
Authors: Peter J Stow; Graeme K Hart; Tracey Higlett; Carol George; Robert Herkes; David McWilliam; Rinaldo Bellomo Journal: J Crit Care Date: 2006-06 Impact factor: 3.425
Authors: Michael W Kuzniewicz; Eduard E Vasilevskis; Rondall Lane; Mitzi L Dean; Nisha G Trivedi; Deborah J Rennie; Ted Clay; Pamela L Kotler; R Adams Dudley Journal: Chest Date: 2008-04-10 Impact factor: 9.410
Authors: John C Marshall; Laura Bosco; Neill K Adhikari; Bronwen Connolly; Janet V Diaz; Todd Dorman; Robert A Fowler; Geert Meyfroidt; Satoshi Nakagawa; Paolo Pelosi; Jean-Louis Vincent; Kathleen Vollman; Janice Zimmerman Journal: J Crit Care Date: 2016-07-25 Impact factor: 3.425