Hyoungju Yun1, Jinwook Choi1,2,3, Jeong Ho Park4,5. 1. Interdisciplinary Program of Medical Informatics, College of Medicine, Seoul National University, Seoul, KR. 2. Department of Biomedical Engineering, College of Medicine, Seoul National University, Seoul, KR. 3. Institute of Medical and Biological Engineering,, Seoul National University Medical Research Center, 103 Daehak-Ro, Jongno-Gu, Seoul, KR. 4. Department of Emergency Medicine, College of Medicine, Seoul National University, Seoul, KR. 5. Laboratory of Emergency Medical Services, Seoul National University Hospital Biomedical Research Institute, Seoul, KR.
Abstract
BACKGROUND: Emergency department (ED) triage system to classify and prioritize patients at high risk from less urgent continues to be a challenge. OBJECTIVE: This study, comprising 80,433 patients, aims to develop a machine learning algorithm prediction model of critical care outcome for adult patients using information collected during ED triage and compare the performance with that of the baseline model using Korean Triage and Acuity Scale (KTAS). METHODS: To predict the need of critical care, we used 13 predictors from triage information: age, gender, mode of ED arrival, time interval between onset and ED arrival, reason of ED visit, chief complaints, systolic blood pressure, diastolic blood pressure, pulse rate, respiratory rate, body temperature, oxygen saturation and level of consciousness. The baseline model with KTAS was developed using logistic regression and the machine learning model with 13 variables was generated using extreme gradient boosting (XGB) and deep neural network (DNN) algorithms. The discrimination was measured by area under the receiver operating characteristic curve(AUC). The ability of calibration with Hosmer-Lemeshow test and reclassification with net reclassification index (NRI) were evaluated. The calibration plot and partial dependence plot were used in analysis. RESULTS: The AUC of the model with the full set of variables (0.833-0.861) was better than that of the baseline model (0.796). The XGB model of AUC 0.861 (0.848, 0.874, 95% CI) showed a higher discriminative performance than DNN model of 0.833(0.819, 0.848). The XGB and DNN models proved better reclassification than the baseline model with positive net reclassification index. The XGB models was well calibrated (Hosmer-Lemeshow test p>0.05); however, the DNN showed poor calibration power (H-L test p<0.001). We further interpreted non-linear association between variables and critical care prediction. CONCLUSIONS: Our study demonstrated that the performance of the XGB model using initial information at ED triage for predicting patients in need of critical care outperformed the conventional model with KTAS.
BACKGROUND: Emergency department (ED) triage system to classify and prioritize patients at high risk from less urgent continues to be a challenge. OBJECTIVE: This study, comprising 80,433 patients, aims to develop a machine learning algorithm prediction model of critical care outcome for adult patients using information collected during ED triage and compare the performance with that of the baseline model using Korean Triage and Acuity Scale (KTAS). METHODS: To predict the need of critical care, we used 13 predictors from triage information: age, gender, mode of ED arrival, time interval between onset and ED arrival, reason of ED visit, chief complaints, systolic blood pressure, diastolic blood pressure, pulse rate, respiratory rate, body temperature, oxygen saturation and level of consciousness. The baseline model with KTAS was developed using logistic regression and the machine learning model with 13 variables was generated using extreme gradient boosting (XGB) and deep neural network (DNN) algorithms. The discrimination was measured by area under the receiver operating characteristic curve(AUC). The ability of calibration with Hosmer-Lemeshow test and reclassification with net reclassification index (NRI) were evaluated. The calibration plot and partial dependence plot were used in analysis. RESULTS: The AUC of the model with the full set of variables (0.833-0.861) was better than that of the baseline model (0.796). The XGB model of AUC 0.861 (0.848, 0.874, 95% CI) showed a higher discriminative performance than DNN model of 0.833(0.819, 0.848). The XGB and DNN models proved better reclassification than the baseline model with positive net reclassification index. The XGB models was well calibrated (Hosmer-Lemeshow test p>0.05); however, the DNN showed poor calibration power (H-L test p<0.001). We further interpreted non-linear association between variables and critical care prediction. CONCLUSIONS: Our study demonstrated that the performance of the XGB model using initial information at ED triage for predicting patients in need of critical care outperformed the conventional model with KTAS.
Authors: Jae Yong Yu; Feng Xie; Liu Nan; Sunyoung Yoon; Marcus Eng Hock Ong; Yih Yng Ng; Won Chul Cha Journal: Sci Rep Date: 2022-10-19 Impact factor: 4.996