Validation of CRASH Model in Prediction of 14-day Mortality and 6-month Unfavorable Outcome of Head Trauma Patients.

INTRODUCTION: To date, many prognostic models have been proposed to predict the outcome of patients with traumatic brain injuries. External validation of these models in different populations is of great importance for their generalization. The present study was designed, aiming to determine the value of CRASH prognostic model in prediction of 14-day mortality (14-DM) and 6-month unfavorable outcome (6-MUO) of patients with traumatic brain injury.
METHODS: In the present prospective diagnostic test study, calibration and discrimination of CRASH model were evaluated in head trauma patients referred to the emergency department. Variables required for calculating CRASH expected risks (ER), and observed 14-DM and 6-MUO were gathered. Then ER of 14-DM and 6-MUO were calculated. The patients were followed for 6 months and their 14-DM and 6-MUO were recorded. Finally, the correlation of CRASH ER and the observed outcome of the patients was evaluated. The data were analyzed using STATA version 11.0.
RESULTS: In this study, 323 patients with the mean age of 34.0 ± 19.4 years were evaluated (87.3% male). Calibration of the basic and CT models in prediction of 14-day and 6-month outcome were in the desirable range (P < 0.05). Area under the curve in the basic model for prediction of 14-DM and 6-MUO were 0.92 (95% CI: 0.89-0.96) and 0.92 (95% CI: 0.90-0.95), respectively. In addition, area under the curve in the CT model for prediction of 14-DM and 6-MUO were 0.93 (95% CI: 0.91-0.97) and 0.93 (95% CI: 0.91-0.96), respectively. There was no significant difference between the discriminations of the two models in prediction of 14-DM (p = 0.11) and 6-MUO (p = 0.1).
CONCLUSION: The results of the present study showed that CRASH prediction model has proper discrimination and calibration in predicting 14-DM and 6-MUO of head trauma patients. Since there was no difference between the values of the basic and CT models, using the basic model is recommended to simplify the risk calculations.

Introduction:

Trauma is the third cause of mortality in developed countries and the most common cause of death in people under 40 years old (1, 2). Statistics show that Iran is among the countries with the highest incidence of road traffic accidents worldwide with more than 21000 traffic-related deaths annually. In Iran, accidents are the second cause of mortality and the most important one for hospitalization (3, 4). Based on the existing reports, about 70% of mortality and disabilities caused by accidents are due to head trauma (5, 6). Evaluations done in the United States indicate that every year 150-200 out of each million people are affected with severe physical and mental disabilities due to traumatic brain injuries (7, 8). Physicians believe that accurate evaluation of prognosis is a very important, yet difficult task (9, 10). To date, many prognostic models have been proposed to predict the outcome of patients with traumatic brain injuries, but none have been widely used (11-13). CRASH (corticosteroid randomization after significant head injury) prognostic model is one of them, which has been developed in recent years and has two separate outcome prediction models for high and low-middle income countries. CRASH model has been proposed by Medical Research Council during CRASH trial project with a sample size greater than 10000. The model claims to be able to predict 14-day mortality (14-DM) and 6-month unfavorable outcome (6-MUO) of head trauma patients (14). Although discrimination and external validation of this model have been evaluated in multiple studies, the results have been contradicting (15-19). Therefore, the present study was designed, aiming to determine the value of CRASH prognostic model in prediction of 14-DM and 6-MUO of patients with traumatic brain injury.

Methods:

In the present prospective diagnostic test study, head trauma patients referred to Imam Hossein and Shohadaye Tajrish Hospitals, Tehran, Iran, from 2012 to 2014 were evaluated. Using census sampling, all the head trauma patients (isolated or multiple trauma) who were referred to the emergency department (ED) within 8 hours of trauma with a GCS (Glasgow coma scale) ≤ 14 were included. Patients with missing follow-up data or not willing to participate were excluded. Initially, a senior emergency medicine resident gathered the variables required for calculating CRASH expected risks (ER), and observed 14-DM and 6-MUO. Then ER regarding 14-DM and 6-MUO were calculated for both computed tomography (CT) scan and basic models (4 scores for each patient) using the web based calculator. The patients were subject to follow-up for 6 months and their 14-DM and 6-MUO were gathered and recorded. Finally, the correlation of CRASH ER and the observed outcome of the patients was evaluated. The present study was approved by the Ethics Committee of Shahid Beheshti University of Medical Sciences. Researchers adhered to the principles of Helsinki Declaration over the course of the study and before being included in the study, informed written consent form was signed by the patient or their relatives.

Definition of CRASH model

This model reports the ER of 14-DM and 6-MUO of head trauma patients as percentage. It has a basic model for low and middle-income countries that predicts the outcome without the need for head imaging and a CT model for high-income countries based on head imaging findings (14). Data needed in the basic model consists of age, level of consciousness based on GCS, pupil reactivity, and presence of major extracranial injury. The CT model needs brain CT scan findings in addition to the 4 mentioned factors in the basic model (14). In the two models, 6-month outcome is determined based on Glasgow outcome scale (GOS). Therefore, the patients are divided into 2 groups of 6-month favorable (good recovery or moderate disability) and unfavorable (severe disability, vegetative state, and death) outcome.

Statistical analyses:

The data were analyzed using STATA version 11.0. Qualitative factors were reported as frequency and percentage, and quantitative ones as mean and standard deviation. In the present study, discrimination and calibration were used on the data for evaluation of model performance. Discriminatory power of the model was evaluated using the area under the receiver operating characteristic (ROC) curve and calculation of likelihood ratio. Area under the ROC curve in the basic and CT models were compared to identify the best model. Best cut-off points for classifying patients into low, moderate, and high-risk groups were determined. In addition, calibration of model was assessed using logistic regression analysis and calculation of intercept, and slope of the line. The ER line was then compared with the observed one using Hosmer-Lemeshow test. The non-significant result of this test means that there is a correlation between the expected outcomes by the model and those observed in reality, a representative of proper calibration. In all analyses, p < 0.05 was considered as significance level.

Results:

Out of the 403 patients entering the study, 80 (19.85%) were excluded because of missing follow-up data (80.14% follow-up rate). In the present study 323 patients with the mean age of 34.0 ± 19.4 years (1-90 years) were evaluated (87.3% male). Table 1 shows demographic characteristics of the participants. 65.3% of the patients were referred to the hospital within an hour of the accident. Road traffic collision with 58.8% prevalence and falling down with 28.8% were the most important mechanisms of injury. In total, 64 (19.8%) patients died in the 14-day and 7 (2.2%) in the 6-month follow up. In addition, 10 (3.1%) patients had severe disability in the 6-month follow up.

Table 1

Baseline characteristics of the studied population

Variables	N (%)
Age
< 20	62 (20.2)
20-29	81 (26.5)
30-39	52 (17.0)
40-49	44 (14.4)
50-59	26 (8.5)
≥ 60	41 (13.4)
Gender
Male	282 (87.3)
Female	41 (12.7)
Time since injury (hour)
< 1	211 (65.3)
1-3	82 (25.4)
> 3	30 (9.3)
Mechanism of trauma
Road traffic collision	190 (58.8)
Falling down > 2 meters	93 (28.8)
Other	40 (12.4)
Glasgow coma scale
Mild (14-15)	130 (40.5)
Moderate (9-13)	112 (34.9)
Severe (3-8)	79 (24.6)
Pupil reaction to light stimulation
None	32 (9.9)
One side	30 (9.3)
Both sides	261 (80.8)
Major extracranial injuries
Yes	70 (21.7)
No	253 (78.3)
Computed Tomography scan
Petechial hemorrhage
Yes	126 (39.0)
No	197 (61.0)
Obliteration of the third ventricle or basal cisterns
Yes	15 (4.6)
No	308 (95.4)
Subarachnoid hemorrhage
Yes	77 (23.8)
No	246 (76.2)
Midline shift
Yes	33 (10.2)
No	290 (89.8)
Non-evacuated hematoma
Yes	215 (66.6)
No	108 (33.4)

Figure1 (A and B) shows the calibration curve of CT and basic models. Slope of the regression line in basic and CT models in prediction of 14-DM were 1.1 (p = 0.07) and 1.3 (p = 0.065), respectively. In addition, these slopes were 1.05 (p = 0.15) and 1.23 (p = 0.065) respectively for 6-month outcome of the patients (figure 1, C and D).

Figure 1

Calibration of Basic and computed tomography (CT) scan model in prediction of 14-day mortality (A and B) and 6-month unfavorable outcome (C and D

Discriminatory power

The mean ER of 14-DM was 46.3 ± 28.16% in those who died and 7.78 ± 10.3% in those who survived based on this model (p < 0.001). In addition, mean ER of 6-MUO were 41.56 ± 13.46% and 17.9 ± 19.3% for observed unfavorable and favorable outcome groups, respectively (p < 0.001). Area under the ROC curve in prediction of 14-DM and 6-MUO were 0.92 (95% CI: 0.89-0.96) and 0.92 (95% CI: 0.90-0.95), respectively (figure 2). Based on the calculated likelihood ratio the patients with < 3.9%, 3.9-63.89%, and ≥ 64% ER of 14-DM were in the observed low, moderate, and high risk groups, respectively. In addition, patients with < 18%, 18-43.19%, and ≥ 43.2% ER of 6-MUO were in the observed low, moderate, and high risk groups, respectively (table 3 and 4).

Figure 2

Area under the receiver operating characteristics curve (AUC) of Basic and computed tomography (CT) scan models in prediction of 14-day mortality (A) and 6-month unfavorable outcome (B

Table 3

Risk stratification of 6-months unfavorable outcome in head trauma based on CRASH model expected risk (ER

	Outcome		Likelihood ratio	P
	Favorable	Unfavorable
Basic model (%)
Low risk (ER < 18.0)	174 (100)	0 (0.0)	0.0*	<0.0001
Moderate risk (18.0 ≥ ER < 43.2)	46 (75.4)	15 (24.6)	---
High risk (ER ≥ 43.2)	25 (28.4)	63 (71.6)	7.5†
CT model
Low risk (ER < 17.8)	166 (100)	0 (0.0)	0.0*	<0.0001
Moderate risk (17.8 ≥ ER < 78.7)	74 (60.2)	49 (39.8)
High risk (ER ≥78.7)	4 (11.8)	30 (88.2)	23.9†

Negative likelihood ratio

Positive Likelihood ratio

The mean ER of 14-DM was 44.0 ± 26.2% in those who died and 7.9 ± 8.9% in the survivors (p < 0.001), based on CT model. In addition, mean ER of 6-MUO were 45.9 ± 16.5% and 20.6 ± 19.3% for observed unfavorable and favorable outcome groups, respectively (p < 0.001). Area under the ROC curve in the CT model in prediction of 14-DM and 6-MUO were 0.93 (95% CI: 0.91-0.97) and 0.93 (95% CI: 0.91-0.96), respectively (figure 2). Based on the calculated likelihood ratio the patients with < 4.7%, 4.7-51.19%, ≥ 51.2% ER of 14-DM were in the observed low, moderate, and high risk groups, respectively. In addition, patients with < 17.8%, 17.8-78.69%, and ≥ 78.7% ER of 6-MUO were in the observed low, moderate, and high risk groups, respectively (table 2 and 3).

Table 2

Risk stratification of 14-day mortality in head trauma patients based on CRASH model expected risk (ER

	Outcome		Likelihood ratio	P
	Survived	Died
Basic model
Low risk (ER < 3.9)	146 (100)	0 (0.0)	0.0*	<0.0001
Moderate risk (3.9 ≥ ER < 63.9)	111 (73.5)	40 (26.5)	---
High risk (ER ≥ 63.9)	1 (3.8)	25 (96.2)	99.2†
CT model
Low risk (ER < 4.7)	144 (100)	0 (0.0)	0.0*	<0.0001
Moderate risk (4.7 ≥ ER < 51.2)	113 (73.9)	40 (26.1)
High risk (ER ≥ 51.2)	2 (7.7)	24 (92.3)	47.6†

Negative likelihood ratio.

Positive Likelihood ratio.

Discussion:

The results of the present study showed that CRASH model has proper discrimination and calibration in predicting 14-DM and 6-MUO of the patients referred to the ED with head trauma. Calculation of likelihood ratio showed that patients with an ER of < 3.9% in the basic model are at low risk for mortality in 14 days, while those with ≥ 63.9% are at high risk. These cut-off points are 4.7% and 51.2%, respectively for the CT model. In addition, when the ER of 6-month outcome is < 17.8% in the CT model and < 18 in the basic one, observation of unfavorable outcome is very unlikely. On the other hand, if ER of 6-MUO is > 43.19 in basic and > 78.69 in CT model, observation of unfavorable outcome is very likely.

The present study confirmed the proper value of CRASH models in predicting the outcome of head trauma patients. This result is in line with the CRASH trial study from which the model was extracted (14). Wong et al. also expressed that calibration and discrimination of CRASH model are desirable for predicting 14-day outcome. They also added that CRASH model shows a mild underestimation in predicting 6-month outcome (19). Majdan et al. also showed that the CRASH model has a good ability in predicting patients’ outcome but it shows underestimation in prediction of 6-month outcome (17). In contrast, Honeybul et al. believed that CRASH model overestimated the 6-month outcome, especially when the ER was <80% (15). These dissimilarities might be due to different mechanisms of trauma. On the other hand, the definition of major extracranial injury in the Majdan et al. study is different from the present study and the CRASH trial.

Although the population that CRASH model was extracted from consisted of people over 16 years old (14), children were also included in this study. Even though 36 (11.1%) of the patients were under 16 years old, calibration and discrimination of the CRASH model was desirable in predicting the outcome of head trauma patients. In sub group analysis, area under the ROC curve was calculated to be 0.95 (95% CI: 0.88-1.0) in the basic model and 0.96 (95% CI: 0.92-1.0) in the CT model in prediction of 14-DM of those under 16 years of age. They were 0.95 (95% CI: 0.90-1.0) and 0.96 (95% CI: 0.91-1.0), respectively in prediction of their 6-MUO. The findings reported here were obtained by studying only 36 children and teenagers; therefore, further studies and more data are needed to confirm the predictive value of this model in them. Previous studies report that complex models that contain accurate data and diagnostic tests such as CT scan, have better discrimination in predicting patient outcome (16, 18). However, the present study showed that the CT model does not provide better calibration or discrimination compared to the basic model. In line with these findings, Majdan et al. also showed that CT models have no advantage over basic models in predicting patient outcome (17). Therefore, it seems that the basic model has appropriate power for predicting outcomes without needing CT scan results.

One of the limitations of the CRASH trial is lack of an accurate definition for some prognostic factors in the CRASH dataset. For example, the definition for extracranial injury covers a very wide range, which includes all the cases that need hospitalization. This general and somehow unclear definition leads to alteration of the ER based on the decision of the physician. This is the reason that Majdan et al. defined extracranial injury as abbreviated injury scale > 2 in their study. However, the researchers of the present study used the definitions of the CRASH trial to ensure minimum alterations from the original study.

Conclusion:

The results of the present study showed that CRASH model has proper discrimination and calibration in predicting 14-DM and 6-MUO of the head trauma patients. Since there was no difference between the values of the basic and CT models, using the basic model is recommended to simplify the risk calculations.

Author contribution:

Behrooz Hashemi and Saeed Safari designed the study method and analyzed the data. Mahnaz Amanat, Maryam Motamedi, and Farhad Rahmati gathered the data. Alireza Baratloo and Mohammad Mehdi Forouzanfar contributed in study design and data gathering. All authors have contribution in drafting and writing the article and accept responsibility for all parts of the study.