Andre L Holder1, Supreeth P Shashikumar, Gabriel Wardi, Timothy G Buchman, Shamim Nemati. 1. 1 Division of Pulmonary, Critical Care, Allergy and Sleep Medicine, Department of Medicine, Emory University School of Medicine, Atlanta, GA. 2 The Emory Critical Care Center, Emory Healthcare, Atlanta, GA. 3 Department of Biomedical Informatics, UC San Diego Health, University of California, San Diego, CA. 4 Department of Emergency Medicine, UC San Diego Health, University of California, San Diego, CA. 5 Division of Pulmonary, Critical Care, and Sleep Medicine, Department of Medicine, UC San Diego Health, University of California, San Diego, CA. 6 Department of Surgery, Emory University School of Medicine, Atlanta, GA.
Abstract
OBJECTIVES: To train a model to predict vasopressor use in ICU patients with sepsis and optimize external performance across hospital systems using domain adaptation, a transfer learning approach. DESIGN: Observational cohort study. SETTING: Two academic medical centers from January 2014 to June 2017. PATIENTS: Data were analyzed from 14,512 patients (9,423 at the development site and 5,089 at the validation site) who were admitted to an ICU and met Center for Medicare and Medicaid Services definition of severe sepsis either before or during the ICU stay. Patients were excluded if they never developed sepsis, if the ICU length of stay was less than 8 hours or more than 20 days or if they developed shock up to the first 4 hours of ICU admission. MEASUREMENTS AND MAIN RESULTS: Forty retrospectively collected features from the electronic medical records of adult ICU patients at the development site (four hospitals) were used as inputs for a neural network Weibull-Cox survival model to derive a prediction tool for future need of vasopressors. Domain adaptation updated parameters to optimize model performance in the validation site (two hospitals), a different healthcare system over 2,000 miles away. The cohorts at both sites were randomly split into training and testing sets (80% and 20%, respectively). When applied to the test set in the development site, the model predicted vasopressor use 4-24 hours in advance with an area under the receiver operator characteristic curve, specificity, and positive predictive value ranging from 0.80 to 0.81, 56.2% to 61.8%, and 5.6% to 12.1%, respectively. Domain adaptation improved performance of the model to predict vasopressor use within 4 hours at the validation site (area under the receiver operator characteristic curve 0.81 [CI, 0.80-0.81] from 0.77 [CI, 0.76-0.77], p < 0.01; specificity 59.7% [CI, 58.9-62.5%] from 49.9% [CI, 49.5-50.7%], p < 0.01; positive predictive value 8.9% [CI, 8.5-9.4%] from 7.3 [7.1-7.4%], p < 0.01). CONCLUSIONS: Domain adaptation improved performance of a model predicting sepsis-associated vasopressor use during external validation.
OBJECTIVES: To train a model to predict vasopressor use in ICU patients with sepsis and optimize external performance across hospital systems using domain adaptation, a transfer learning approach. DESIGN: Observational cohort study. SETTING: Two academic medical centers from January 2014 to June 2017. PATIENTS: Data were analyzed from 14,512 patients (9,423 at the development site and 5,089 at the validation site) who were admitted to an ICU and met Center for Medicare and Medicaid Services definition of severe sepsis either before or during the ICU stay. Patients were excluded if they never developed sepsis, if the ICU length of stay was less than 8 hours or more than 20 days or if they developed shock up to the first 4 hours of ICU admission. MEASUREMENTS AND MAIN RESULTS: Forty retrospectively collected features from the electronic medical records of adult ICU patients at the development site (four hospitals) were used as inputs for a neural network Weibull-Cox survival model to derive a prediction tool for future need of vasopressors. Domain adaptation updated parameters to optimize model performance in the validation site (two hospitals), a different healthcare system over 2,000 miles away. The cohorts at both sites were randomly split into training and testing sets (80% and 20%, respectively). When applied to the test set in the development site, the model predicted vasopressor use 4-24 hours in advance with an area under the receiver operator characteristic curve, specificity, and positive predictive value ranging from 0.80 to 0.81, 56.2% to 61.8%, and 5.6% to 12.1%, respectively. Domain adaptation improved performance of the model to predict vasopressor use within 4 hours at the validation site (area under the receiver operator characteristic curve 0.81 [CI, 0.80-0.81] from 0.77 [CI, 0.76-0.77], p < 0.01; specificity 59.7% [CI, 58.9-62.5%] from 49.9% [CI, 49.5-50.7%], p < 0.01; positive predictive value 8.9% [CI, 8.5-9.4%] from 7.3 [7.1-7.4%], p < 0.01). CONCLUSIONS: Domain adaptation improved performance of a model predicting sepsis-associated vasopressor use during external validation.
Authors: Sophie E Ack; Shamelia Y Loiseau; Guneeti Sharma; Joshua N Goldstein; India A Lissak; Sarah M Duffy; Edilberto Amorim; Paul Vespa; Joseph Randall Moorman; Xiao Hu; Gilles Clermont; Soojin Park; Rishikesan Kamaleswaran; Brandon P Foreman; Eric S Rosenthal Journal: Neurocrit Care Date: 2022-06-10 Impact factor: 3.532