Predicting outcomes after trauma: Prognostic model development based on admission features through machine learning.

ABSTRACT: In an overcrowded emergency department (ED), trauma surgeons and emergency physicians need an accurate prognostic predictor for critical decision-making involving patients with severe trauma. We aimed to develope a machine learning-based early prognostic model based on admission features and initial ED management.We only recruited patients with severe trauma (defined as an injury severity score >15) as the study cohort and excluded children (defined as patients <16 years old) from a 4-years database (Chi-Mei Medical Center, from January 2015, to December 2018) recording the clinical features of all admitted trauma patients. We considered only patient features that could be determined within the first 2 hours after arrival to the ED. These variables included Glasgow Coma Scale (GCS) score; heart rate; respiratory rate; mean arterial pressure (MAP); prehospital cardiac arrest; abbreviated injury scales (AIS) of head and neck, thorax, and abdomen; and ED interventions (tracheal intubation/tracheostomy, blood product transfusion, thoracostomy, and cardiopulmonary resuscitation). The endpoint for prognostic analyses was mortality within 7 days of admission.We divided the study cohort into the early death group (149 patients who died within 7 days of admission) and non-early death group (2083 patients who survived at >7 days of admission). The extreme Gradient Boosting (XGBoost) machine learning model provided mortality prediction with higher accuracy (94.0%), higher sensitivity (98.0%), moderate specificity (54.8%), higher positive predict value (PPV) (95.4%), and moderate negative predictive value (NPV) (74.2%).We developed a machine learning-based prognostic model that showed high accuracy, high sensitivity, and high PPV for predicting the mortality of patients with severe trauma.

Introduction

Major trauma is one of the leading causes of morbidity and mortality in adults worldwide.[ The management of patients with major trauma requires a multidisciplinary team and consumes considerable resources.[ In an overcrowded emergency department (ED), trauma surgeons and emergency physicians often encounter the dilemma regarding the distribution of limited medical resources among patients, including; medical staff, priority for imaging studies, intensive care, and surgery. An accurate prognostic predictor can support critical decision-making involving patients with severe trauma.

The Glasgow Coma Scale (GCS) and Glasgow Outcome Scale can provide predictions within 24 hours after the injury, but not prediction on admission.[ Prognostic models with admission features are essential to support healthcare workers (HCWs) in early clinical decision-making and in enhancing resource utilization. Various trauma scoring systems based on anatomical classification, physiological data, and a combination of both have been developed.[ Several of these scoring systems have been used to predict the outcomes of patients with trauma.[ Nevertheless, patterns of traumatic injuries and management for patients with trauma differ between different areas or countries and can be altered under different insurance systems.[ The prognostic predictor derived from a universal trauma scoring system may not fit the unique situation in a single institution.[

In previous studies, machine learning algorithms have provided a new opportunity for estimating the results of medical interventions and predicting patient outcomes.[ A well-designed algorithm uses real-time data to help physicians with decision-making, and system efficacy can be reinforced with further data input.[ A medical institution can apply an algorithm to predict patient outcomes, to distribute medical resources, and to improve the quality of care of patients with severe trauma.[

We aimed to develop a machine learning-based early prognostic model based on admission features and initial ED management. We hope the model can help to predict mortality on patients with severe trauma. Healthcare workers can utilize the model before initiating definite therapeutic interventions on patients and anticipate the improvement of care for patients with trauma.

Materials and methods

Data management

In the Chi-Mei Medical Center, Taiwan, a database was set up by Division of Traumatology, Department of Surgery to record the clinical features of all admitted patients with trauma. The original design of the database was for quality improvement of the care for patients with trauma and the patient data come from the medical records of patients. The database was collected the patients data of demographics, prehospital presentations, ED presentations, various trauma scores, hospital course, and prognosis. The database were included the data of 11 816 patients from January 1, 2015, to December 31, 2018. We only recruited patients with severe trauma (defined as injury severity score >15[) as the study cohort and excluded children (defined as patients <16 years old[), which yielded a study sample of 2232 patients (Fig. 1). We considered patient features that could be determined easily and reliably within the first 2 hours after arrival to the ED. These variables included the GCS score; heart rate; respiratory rate; mean arterial pressure (MAP); prehospital cardiac arrest; AIS of head and neck, thorax, and abdomen; and ED interventions (tracheal intubation/tracheostomy, blood product transfusion, thoracostomy, and cardiopulmonary resuscitation). Because delayed hospital death is often attributed to complications, comorbidities, and preinjury health conditions of patients,[ the endpoint for prognostic analyses was mortality within 7 days of admission. We divided the study cohort into the early death group (patients who died within 7 days of admission) and non-early death group (patients who survived at >7 days of admission).

Figure 1

Study flowchart. N: numbers of the patients.

Development of machine learning model

We used 80% of patient's data to develop the prognostic model and remaining 20% of patient's data to validate its accuracy (Fig. 1). In this study, we tested 12 analyzers, namely; decision trees, random forest, artificial neural networks, k-nearest neighbors algorithm, Naïve Bayes, k-means clustering, logistic regression, support vector machine, AdaBoost, quadratic discriminant analysis, gradient boosting, and XGBoost. EXtreme Gradient Boosting was selected as the final machine learning algorithm technique because it showed the highest accuracy among these analyzers. In addition, we presented the other patient data that were not included in the predictor model to portray the whole picture of the study cohort and compared the collected data between the early and non-early death groups.

The Institutional Review Board of Human Research, Chi-Mei Medical Center granted this study exemption from approval because the researchers used deidentified data. The study was conducted in accordance with the Declaration of Helsinki.

Statistical analysis

Statistical analyses were performed using Statistical Product and Service Solutions (version 15) (SPSS, Inc., Chicago, IL), and the early and non-early death groups were compared. Data were reported as proportions for categorical variables. Continuous data were presented as mean ± standard deviation or median and interquartile range (Q). We used the Chi-Squared test to evaluate the differences in categorical variables. Continuous data between groups were compared using Student t test and Mann–Whitney U test. Overall, statistical significance was set at a P value of ≤.05.

Results

General descriptions of the study cohort

In total, 2232 (18.9%) patients with severe trauma were selected from 11 816 patients in the database. In total, 233 deaths occurred during hospitalization, which yielded a mortality rate of 10.4%. Among the deaths reported, 149 (6.7%) patients had died within 7 days of admission and were included in the early death group. Another 2083 (93.3%) patients were included in the non-early death group. The study flowchart was presented in Figure 1. The age of the patients in the study cohort was 57.0 ± 20.2 years (Table 1), and 66% of the patients were men (Table 2). In total, 14 (0.6%) patients arrived at the ED in a cardiac arrest state. The median GCS score in the ED was 15 (Q1, Q3: 11, 15). The heart rate, respiratory rate, and MAP were 89.0 ± 21.8 beat/min, 17.5 ± 4.0 respiration/min, and 102.4 ± 26.1 mm Hg, respectively (Table 1). We documented head and neck, thorax, and abdomen injuries in 81.1%, 36.6%, and 17.7% of the patients, respectively (Fig. 2A). If AIS was ≥3 injuries, the ratio of these injuries became 77.4%, 30.0%, and 10.5%, respectively (Fig. 2B). Figure 2A showed the comparisons of injuries of the head and neck, thorax, and abdomen between early death and non-early death groups. The early death group exhibited a significantly higher rates of injury on head and neck than the non-early death group. (early vs non-early: head and neck: 91.9% vs 80.4%, P < .001, thorax: 34.9% vs 36.7%, P = .655, abdomen: 20.8% vs 17.5%, P = .311). Regarding injuries which AIS ≥3 of the head and neck, thorax, and abdomen, the early death group revealed significantly higher rates of injury on head and neck as well as abdomen than the non-early death group (early vs non-early: head and neck: 89.9% vs 76.5%, P < .001, thorax: 27.5% vs 30.1%, P = .498, abdomen: 16.1% vs 10.1%, P = .022) (Fig. 2B).

Table 1

Characteristics of early and non-early death groups.

	All (n = 2232)	Non-early death group (n = 2083)	Early death group (n = 149)	P value
Age (yr)†	57.0 ± 20.2	56.0 ± 20.2	62.0 ± 19.0	.053
Glasgow coma scale∗	15 (11–15)	15 (12–15)	4 (7–8)	.000
Heart rate (beat/min)†	89.0 ± 21.8	89.4 ± 19.8	83.8 ± 40.3	.000
Respiratory rate (breath/min, mean ± standard deviation)†	17.5 ± 4.0	17.7 ± 3.6	15.2 ± 7.5	.000
Mean arterial pressure (mmHg)†	102.4 ± 26.1	103.4 ± 23.2	87.9 ± 49.6	.000
Out-of-hospital cardiac arrest (percentage)‡	14 (0.6%)	6 (0.3%)	8 (5.4%)	.000

Table 2

Data not included in the machine learning model.

	All (n = 2232)	Non-early death group (n = 2083)	Early death group (n = 149)	P value
Sex (male) (percentage)	1473 (66.0%)	1368 (65.7%)	105 (70.5%)	.233
Comorbidity (percentage)	969 (43.4%)	883 (42.4%)	86 (57.7%)	.000
Surgery (percentage)	970 (43.5%)	917 (44.0%)	53 (35.6%)	.044
Hospital stay (day) (mean ± SD)	16.9 ± 18.2	17.9 ± 18.5	4.0 ± 4.5	.000
Requirement for ICU (percentage)	1417 (63.5%)	1282 (61.5%)	135 (90.6%)	.000
ICU stay (day) (mean ± SD)	8.5 ± 9.7	9.0 ± 10.1	4.0 ± 7.5	.000
AIS∗ face (percentage)	543 (24.3%)	521 (25.0%)	22 (17.8%)	.004
AIS extremity (percentage)	971 (43.5%)	931 (44.7%)	40 (26.8%)	.000
AIS external (percentage)	42 (1.9%)	38 (1.8%)	4 (2.7%)	.455
ISS† (mean ± SD)	22.8 ± 9.7	21.9 ± 8.1	35.1 ± 18.5	.000
NISS‡ (mean ± SD)	27.2 ± 9.8	26.5 ± 9.0	37.1 ± 13.5	.000
RTS§ (mean ± SD)	701411 ± 1.3524	7.3187 ± 1.0440	4.2579 ± 2.3630	.000
TRISS (mean ± SD)	0.8562 ± 0.2204	0.8864 ± 0.1745	0.4347 ± 0.3346	.000

Figure 2

The percentages for all patients, patients in the early death group, and those in the non-early death group with (A) any injuries of the head and neck, thorax, and abdomen and (B) abbreviated injury scales ≥3 of the head and neck, thorax, and abdomen.

For the study cohort, the requirements for tracheal intubation/tracheostomy, blood product transfusion, thoracostomy, and cardiopulmonary resuscitation were 16.7%, 15.6%, 5.2%, and 0.8%, respectively (Fig. 3). Figure 3 also demonstrated higher requirements for tracheal intubation, transfusion, thoracostomy, and cardiopulmonary resuscitation in early death group than those in non-early death group (early vs non-early: tracheal intubation: 57.0% vs 13.8%, P < .001, transfusion: 36.2% vs 14.1%, P < .001, thoracostomy: 6.7% vs 5.1%, P = .405, cardiopulmonary resuscitation: 8.7% vs 0.2%, P < .001).

Figure 3

Data on emergency department management of the 2 groups; percentages of patients who underwent tracheal intubation/tracheostomy, blood product transfusion, thoracostomy, and cardiopulmonary resuscitation.

Results of the machine learning algorithm

The machine learning model was exhibited higher accuracy, sensitivity, and positive predictive value (PPV) (94.0%, 98.0%, and 95.4%, respectively) with moderate specificity and negative predictive value (NPV) (54.8% and 74.2%, respectively) for predicting the mortality of patients with severe trauma. The importance of each feature from high to low was as follows: GCS, AIS of head and neck, AIS of abdomen, ED interventions, age, MAP, respiratory rate, heart rate, presence of prehospital cardiac arrest, and AIS of thorax.

Comparison between early and non-early death groups

Regarding prehospital data, the early death group showed low GCS scores, low heart rate, low respiratory rate, low MAP, and high incidence of out-of-hospital cardiac arrest (early death vs non-early death: GCS: 4 vs 15, heart rate: 83.8 vs 89.4, respiratory rate: 15.2 vs 17.7, MAP: 87.9 vs 103.4, out-of-hospital cardiac arrest: 5.4% vs 0.3%, P = .00, Table 1).

Table 2 presents features for all patients, the early death group, and the non-early death group that were not included in the machine learning model. In most features, differences between the early and non-early death groups were significant. The early death group showed more comorbidities, less requirement for surgery, shorter stay in the hospital and in the intensive care unit, higher requirement for intensive care, and lower trauma scores than the non-early death group. (early vs non-early: male sex: 70.5% vs 65.7%, P = .233, comorbidity: 57.7% vs 42.4%, P = .000, requirement for surgery: 35.6% vs 44.0%, P = .044, days of hospital stay: 4.0 ± 4.5 vs 17.9 ± 18.5, P = .000, requirement for intensive care: 90.6% vs 61.5%, P = .000, days of intensive care: 4.0 ± 7.5 vs 9.0 ± 10.1, P = .000, AIS face: 17.8% vs 25.0%, P = .004, AIS extremity: 26.8% vs 44.7%, P = .000, AIS external: 2.7% vs 1.8%, P = .455, Injury Severity Score: 35.1 ± 18.5 vs 21.9 ± 8.1, P = .000, New Injury Severity Score: 37.1 ± 13.5 vs 26.5 ± 9.0, P = .000, Revised Trauma Score: 4.2579 ± 2.3630 vs 7.3187 ± 1.0440, P = .000, Trauma Injury Severity Score: 0.4347 ± 0.3346 vs 0.8864 ± 0.1745, P = .000).

Discussion

The developed machine learning-based prognostic model could predict the mortality of patients with severe trauma. The high sensitivity and PPV of this model indicates that almost all patients with mortality risk will be identified. Regarding specificity and the NPV, although they were unsatisfactory, we believe that both values will be improved after inputting more data for machine learning. Additionally, an unsatisfactory NPV indicates that the clinical condition of a patient with severe trauma changes constantly. Healthcare workers should never make their judgement solely relying on a single predictive model.

To prospectively validate this model, we are currently designing an algorithm for model integration into the hospital information system. This predictive model can help the HCWs in ED on decision making. They can either concentrate all available resource to accelerate the resuscitation or distribute limited resources on patients with higher chance of survival. In events with multiple casualties, this model could play a role of repeated triage of patients.

In this machine learning model, the most challenging problem was choosing appropriate features. The chosen features should be easily accessible within a short ED stay, clinically relevant to prognosis, and widely available in most patients with trauma. Age is a well-known prognostic factor of patients with trauma and, therefore, should be provided.[ Regarding physiological data, we included the GCS score, heart rate, MAP, and respiratory rate. These data can be obtained at the ED triage of every patient with trauma, and they constitute the major components of the revised trauma score and trauma and injury severity score.[ From the comparison between early and non-early death groups, we also found significant differences of most chosen features and this finding re-enforced the inclusion of these features. Some scholars opine that patients with severe trauma experience continuous changes as disease progressed and vital signs, and the GCS score determined at the ED triage cannot demonstrate changes in the patient condition.[ However, most injuries from trauma were determined immediately after the traumatic event. The triage data represent a considerable portion of injury severity.[ Additionally, not all patients with trauma are all closely monitored, and continuous data are lacking in a considerable number of patients with trauma. Furthermore, if all patients with trauma receive close monitoring and continuous observation by HCWs, a prognostic model for patients with trauma is unnecessary.

The anatomical classification provided by trauma scores characterized the distribution and severity of injuries. They are traditionally prognostic indicators, and the AIS of head and neck, thorax, and abdomen are significantly related to mortality.[ In the management of patients with trauma, injuries to head, neck, thorax, and abdomen are prioritized for physical examinations and imaging studies.[ We believe that data on the AIS of head and neck, thorax, and abdomen of most patients with severe trauma can be obtained within 2 hours on arrival to the ED and were included in the model. In fact, from the comparisons between early death and non-early death groups, we did discover many patients in both groups sustained injuries of head and neck, thorax, and abdomen. Furthermore, the early death group showed statistically higher rates of injuries on certain categories than the non-early death group, which infers AIS of the 3 parts can be prognostic indicators of trauma patients.

Furthermore, we included ED management to develop this machine learning algorithm. Tracheal intubation/tracheostomy, transfusion of blood product, thoracostomy, and cardiopulmonary resuscitation were common life-saving interventions conducted in the ED. A standardized trauma protocol study conducted in Colombia showed that ED interventions are crucial for the management of patients with severe trauma.[ The requirement for emergent interventions indicate treatment urgency and injury severity for patients. Additionally, ED management data partly represent the disease progress and can overcome the shortcoming that continuous physiological data were not used when developing this model. For the study cohort, we noticed that patients in early death group required more ED management than patients in non-early death group, which further support the inclusion of ED management in this model.

Patients with traumatic out-of-hospital cardiac arrest often have poor prognosis.[ Therefore, we included prehospital cardiac arrest as a feature to develop the model. However, this feature did not notably affect the result of the prognostic model. The reason may be that only a small number of patients with traumatic out-of-hospital cardiac arrest survived after hospital admission; therefore, the influence of this feature became insignificant. Additionally, most patients in the cardiac arrest state present with poor vital signs and often undergo ED interventions[; the influence of prehospital cardiac arrest may be partly neutralized by these factors.

A survey conducted in the Netherlands for the causes of traumatic death showed that the mortality rate due to exsanguinations decreased and death resulting from central nervous system injury increased.[ The finding is in accordance with our study results. Both GCS scores and AIS of head and neck played significant roles in the present prognostic model. Elderly people are susceptible to minor traumatic events and are fragile to even minor injuries, especially fall-related traumatic brain injury.[ We anticipate that the influence of central nervous system injury on mortality will increase in the future.

We did not include features in Table 2 to develop this predictive model because we knew if we used more features, the accuracy of this prognostic model would increase because the differences in these features were significant between the early and non-early death groups. However, if more features are included, longer time will be required for evaluating patient outcomes using the model. In a clinically practical prognostic model, a balance must be maintained between efficacy and efficiency.

Limitations

This study had several limitations. First, this system was developed for prognostic prediction in the ED; therefore, the cohort should include patients with trauma in the ED instead of admitted patients with trauma. Some critically injured patients may not have been included in this study. In our opinion, if a patient cannot survive for more than 2 hours in the ED, the patient is extremely critical and does not require the assistance of a prognostic model. Second, the model was developed based on a retrospective database, and the initial AIS of body parts in ED may be different from the final AIS during hospital discharge. In the ED, emergency physicians and trauma surgeons always prioritize the most severely wounded part in every body region. As the AIS records the most severe injury, we believe the AIS in the ED should be similar to the final AIS during discharge in most patients. Finally, this machine learning-based prognostic model was developed and validated using only the data of the tested trauma center. The application of this model to patients with trauma in other hospitals needs further validation.

Conclusions

We developed a machine learning-based prognostic model from a database to predict the mortality of patients with trauma. Extreme Gradient Boosting was selected as the final machine learning algorithm technique, and only patient's features that could be determined within 2 hours of arrival to the ED were used in the model development. This model showed high accuracy, high sensitivity, and high PPV for predicting the mortality of patients with severe trauma.

Acknowledgments

This manuscript was edited by Wallace Academic Editing. We appreciated their excellent work.

Author contributions

Conceptualization: Kuo-Chang Lee.

Data curation: Hsiu-Fen Chiang, Hsiu-Chen Su, Kuo-Tai Chen.

Formal analysis: Chien-Chin Hsu.

Investigation: Chien-Chin Hsu.

Methodology: Kuo-Chang Lee, Nan-Chun Wu.

Project administration: Tzu-Chieh Lin, Gwo-Jiun Horng.

Resources: Kuo-Tai Chen.

Software: Tzu-Chieh Lin.

Supervision: Gwo-Jiun Horng.

Writing – original draft: Kuo-Chang Lee.

Writing – review & editing: Kuo-Tai Chen.