The ability of two scoring systems to predict in-hospital mortality of patients with moderate and severe traumatic brain injuries in a Moroccan intensive care unit.

AIM OF STUDY: We aim to assess and to compare the predicting power for in-hospital mortality (IHM) of the Acute Physiology and Chronic Health Evaluation-II (APACHE-II) and the Simplified Acute Physiology Score-II (SAPS-II) for traumatic brain injury (TBI). PATIENTS AND METHODS: This retrospective cohort study was conducted during a period of 2 years and 9 months in a Moroccan intensive care unit. Data were collected during the first 24 h of each admission. The clinical and laboratory parameters were analyzed and used as per each scoring system to calculate the scores. Univariate and multivariate analyses through regression logistic models were performed, to predict IHM after moderate and severe TBIs. Areas under the receiver operating characteristic curves (AUROC), specificities and sensitivities were determined and also compared.
RESULTS: A total of 225 patients were enrolled. The observed IHM was 51.5%. The univariate analysis showed that the initial Glasgow coma scale (GCS) was lower in nonsurviving patients (mean GCS = 6) than the survivors (mean GCS = 9) with a statistically significant difference (P = 0.0024). The APACHE-II and the SAPS-II of the nonsurviving patients were higher than those of the survivors (respectively 20.4 ± 6.8 and 31.2 ± 13.6 for nonsurvivors vs. 15.7 ± 5.4 and 22.7 ± 10.3 for survivors) with a statistically significant difference (P = 0.0032 for APACHE-II and P = 0.0045 for SAPS-II). Multivariate analysis: APACHE-II was superior for predicting IHM (AUROC = 0.92).
CONCLUSION: The APACHE-II is an interesting tool to predict IHM of head injury patients. This is particularly relevant in Morocco, where TBI is a greater public health problem than in many other countries.

Introduction

Traumatic brain injury (TBI) remains a major health problem in Morocco. It is a leading cause of morbimortality in the young adult.[1]

Early prediction of mortality after TBI is essential and may be useful in several areas. For instance, it can support early clinical decision-making, facilitate reliable stratification of these patients based on their prognoses, allocate accurately the resources and also it can help in communicating with patients families.[2345]

To reach this purpose, many studies have identified various prognostic factors that affect the outcome after head injury with varying degrees of correlation.[67] Furthermore, a multiple prognostic models and scales, which vary in complexity and accuracy of prediction,[89] have been created in order to predict clinical outcome for TBI, but they are not widely used,[3] because they were developed from relatively small samples of patients originating from a single center or region, their methodological quality is generally poor (with the exception of those developed by Hukkelhoven et al. and Signorini et al.)[1011] with lack of external validation,[12] and only a few are developed using populations from low and middle income countries, where most of trauma occurs.[3] Furthermore, they are not presented to physicians in a user-friendly way as they are not easy or simple to use and they are time-consuming to make.

Prognostic models such as Corticosteroid Randomization after Significant Head Injury (CRASH) and International Mission for Prognosis and Analysis of Clinical Trials in TBI (IMPACT) were found as useful tools, based on admission characteristics, to predict the risk of 6 months mortality and unfavorable outcomes in individual patients after moderate or severe TBI.[4] However, Medical Research Council CRASH Trial Collaborators found that the strength of predictors of outcomes in TBI varies according to whether patients are from high or low-middle income countries.[13]

We attempt to evaluate the discriminating capability of two scores daily used and more familiar in our practice, Acute Physiology and Chronic Health Evaluation-II (APACHE-II) and Simplified Acute Physiology Score-II (SAPS-II),[1415] to predict mortality in specific patient populations such those with TBI. The potential validities of these predicting systems can allow their application before in-hospital therapeutic interventions, especially in low- and middle-income countries where the resources are limited and where other prognostic models haven’t been generalized yet.

Patients and Methods

Data collection

Our hospital (one of the three sites of our university teaching hospital) is a 409-bed tertiary care center in Marrakech, Morocco. It offers care to the population of the South. The 10-bed medical-surgical intensive care unit (ICU) has approximately 570 admissions a year. The hospital also has a coronary care unit and a cardiac surgical ICU. The ICU is run by full-time intensivists and has 24 h immediate access to other medical and surgical specialties. The nurse-to-patient ratio is approximately 1:2.

Records of 431 consecutive patients, who were admitted to our adult medical and surgical ICU during a period of 2 years and 9 months, were retrospectively examined, and 225 of these were included in this study. Inclusion criteria were: Patients admitted with a diagnosis of moderate or severe TBI (Glasgow coma scale [GCS] ≤12) at the time of hospital admission; patients aged 16 years or more; and patients admitted to the emergency room no more than 2 h after their injuries. Exclusion criteria were: Polytrauma (n = 109) and incomplete data gathering (n = 17). Furthermore, readmissions to ICU (n = 40), patients who died within the first 24 h of admission and who were transferred from a different hospital were excluded from the study (n = 40).

A careful review of all medical charts including laboratory results was carried out. Patients data observed during the first 24 h of their hospital stays were collected by experienced doctors to obtain the following variables: Demographics, neurological injury, temperature (°C), systolic and mean arterial blood pressure (mmHg), heart rate, respiratory rate, PaO2 (mmHg) or FiO2, arterial pH and bicarbonate, serum sodium, potassium, urea and creatinine, urine output, serum white blood cell count, hematocrit, platelet count and bilirubin, age, type of admission, GCS score, presence of chronic diseases (chronic organ insufficiency) or immunocompromised state. Consequently, we selected the most abnormal value of each variable during the period between admission and the hour 24 of hospitalization.[1415] Assessment of the severity using GCS is problematic when ongoing sedation is needed. As poor GCS carries a heavy weight in both the APACHE-II and SAPS-II scores, the lowest initial recording of GCS before sedation was used.

For all patients, APACHE-II and SAPS-II scores were calculated. The risk of death was calculated as described in the original literature.[1415] The associated risks of in-hospital mortality (IHM) were derived using data from each patient's ICU stay and predictive equations of the respective scoring system.

The endpoint for the prognostic analysis was IHM in connection with the predicted mortalities of the scores.

Statistical analysis

Continuous variables were expressed as mean ± standard deviation and were compared using standard t-test. Categorical values were expressed in absolute and relative frequencies, and were analyzed using commercially available statistical software (Epi Info™ 6 developed by Centers for Disease Control and Prevention (CDC), USA). P > 0.05 were considered as nonstatistically significant. Predicted mortality was calculated using logistic regression formulae described in the original articles.[1415]

A receiver operating characteristic (ROC) curve was built for each severity index, and area under the ROC curve (AUROC) was used to test the ability of models to discriminate between patients who survived or patients who did not.[1617]

Results

Traumatic brain injury accounted for 27.5% of all admissions to ICU in the study period (431 out of 15.64 of total admissions). Study population demographics are shown in Table 1. 4.4% of all patients had one or more severe chronic illnesses. Cardiovascular disease was the leading chronic illness, followed by metabolic illness (diabetes mellitus) and immunosuppression [Table 1].

Table 1

General characteristics of 225 patients (distributed in two groups: Survivors and nonsurvivors) including demographic data, comorbidity, scoring system data, ICU length of stay and ICU mortality

Mean APACHE-II and SAPS-II values were 20.56 (range: 2-36) and 28.12 (range: 5-60), respectively. In comparison with survivors, nonsurvivors were older, had a longer lead time, ICU and hospital lengths of stay. They had higher APACHE-II, SAPS-II scores, and they had also lower GCS score. Univariate analysis [Table 1] showed that age, APACHE-II, SAPS-II and GCS scores were predictors for IHM in TBI patients.

Observed mortality during the hospital stay was 51.55% (116/225) with 40% mortality in ICU (90/225 patients). Both APACHE-II and SAPAS-II underestimated mortality for our patients’ sample. APACHE-II system strongly correlated (Spearman's rank correlation coefficient, 0.98, P < 0.01). The mean values for the overall patients population, survivors and nonsurvivors, resulting from multivariate analysis, are listed in Table 2.

Table 2

Univariate and multivariate analysis of the three severity scoring systems and comparison of their results

Both APACHE-II and SAPS-II systems were accurate for predicting mortality with statistical significance. Furthermore, the higher score the greater risk of mortality. Figures 1 and 2 show the ROC curves for the two scoring systems. APACHE-II showed increased total adjustment, with the highest AUROC (0.92), compared to 0.843 for the SAPS-II, sensitivity and specificity (respectively 76.4%; 82.7% against 69.8%; 78.5% for the SAPS-II). These data reflect the better discriminative power of the first system.

Figure 1

Discriminative ability of clinical prediction rules (outcome = death) derived from Acute Physiology and Chronic Health Evaluation-II scoring system

Figure 2

Receiver operating characteristic curve for the Simplified Acute Physiology Score-II scoring system

Discussion

Several scoring systems and prognostic models have been developed in the field of intensive care and emergency medicine over the last two decades to enable caregivers to quantify early the probability of survival of trauma patients and estimate the patients’ severity illness. Furthermore, outcome prediction systems have become key tools to evaluate the care quality and the ICU performance.[1618]

Acute Physiology and Chronic Health Evaluation II and SAPS-II are two general scoring systems currently in common use for measuring the condition of individual ICU patients by numeric scores including multiple physiologic variables selected because of their impact on mortality: The sicker the patient, the more deranged the values and the higher the score.[1415]

Since its creation, APACHE-II has been designated as an accurate and exact predictor of mortality across a wide range of diagnostic populations, particularly in the ICU settings.

While APACHE-II score has been considered “invalid” in the trauma population, we hypothesized that APACHE-II would more accurately predict outcomes in critically brain injured patients in whom commonly used specific trauma scores have inherent limitations.

Due to the presence of cerebral insults, patients admitted to ICU tended to show more unfavorable outcomes compared with non-ICU patients and many studies have identified various prognostic factors that affect the outcome after head injury with varying degrees of correlation.[19]

The main findings in this study are that patient age and presedation GCS score were individually related to hospital death. Many previous studies have demonstrated that older patients have generally poor outcomes, and a low GCS score was associated with a more severe TBI and higher mortality, which were consistent with our results.[202122232425] Comorbidity has also been found to be a predictor of mortality in earlier studies.[202627]

The preresuscitation GCS is still one of the important determinants in predicting mortality of TBI patients. The presedation GCS, is a clinical, simple, and practical tool to predict mortality in neurosurgical and head trauma patients.[28293031] In fact, it has been the gold standard for effective assessment in these patients against, which the other grading systems are usually compared.[30] However, TBI remain an extremely heterogeneous disorder with unpredictable evaluative patterns.[202122] Numerous studies documented the superiority of APACHE-II compared with GCS in predicting mortality due to its recognition of age and significant underlying problems.[333435] In addition to that, APACHE-II and SAPS-II scores include this important factor (GCS) in their automated calculation tables. For those reasons, Zali et al. found that APACHE-II was better than GCS for mortality prediction in neurosurgical ICU patients.[36] That was true in two works conducted in Turkey and Korea.[3738]

Cho and Wang concluded that APACHE-III was better in assessing early outcomes than either the GCS or APACHE-II in neurosurgical intensive care: The AUROC curves of these scores were, respectively, 0.892, 0.868 et 0.826.[34]

In this paper, we investigated the discriminative power of APACHE-II and SAPS-II in predicting hospital mortality of moderate and severe TBI patients. In both systems, predicted mortality (the average of APACHE-II and SAPS-II were respectively 20.56 and 28.12) was much lower than actual mortality (51.55%). There are many reasons that could explain this result: On the one hand, most scoring systems have been constructed in general ICU populations and were therefore not validated for specific patients or groups. This has been especially true for TBI patients, who are younger and have fewer chronic health problems frequently seen in older patients, resulting in underestimated predicted mortality.[39] On the other hand, the higher rate of actual as compared to predicted mortality is attributable to the severity of intracranial injury on initial computed tomography scan, the longer mean admission time (1 h 48 min after the accident), and the poor quality of management on the field (sub optimal prehospital management).

Simplified Acute Physiology Score II was the weakest predicting system with an underestimation of mortality and a low number of correctly categorized patients [Table 2]. In fact, GCS alone had a better predicting ability than SAPS-II. This may be considered surprising, since the impact of the GCS score is included in the SAPS-II system. We suggest that the SAPS-II system assigns more points to emergency surgical patients than to medical patients, supposing that the surgical patients are at a higher risk of death. However, for TBI patients the opposite is true: Patients managed surgically have better severity of illness-adjusted outcomes than patients medically managed.

In this study, the illness severity and death rate among medical patients were higher than those recorded for surgical patients. Despite the stratification ability of the APACHE-II system, it lacked accuracy in predicting death rates because the recorded death rate was higher than the predicted rate. Another multicenter study proved that for the overall estimation of aggregate ICU mortality of trauma patients, the APACHE-III system was the most reliable; however, performance was most accurate for subsets of patients with head trauma.[3435]

Wong et al. and Vassar et al. found that both the Trauma and Injury Severity Score (TRISS) and the APACHE-II systems were both poor predictors of the risk for hospital death among ICU trauma patients.[4041] On the other hand, in a recent work, the authors suggest that there is no difference between the sequential organ failure assessment (SOFA) score and APACHE-II and TRISS in predicting the outcomes of ICU trauma patients. Although their conclusion was limited by the fact that this work was a single center study; however, the method for calculating SOFA scores is easier and simpler than APACHE-II and TRISS.[42]

Overall, APACHE-II and TRISS did not meet acceptable thresholds of performance.[41] Furthermore, critically injured patients have physiologic derangements not accurately accounted for by commonly used trauma scores (TRISS, ISS etc.).

In this subset a more general ICU scoring system, for instance:APACHE-II is useful for risk adjustment, for research, administrative and quality improvement purposes. Similarly, a prospective cohort study of 1019 critically injured patients enrolled in the United States showed that APACHE-II was the best predictor of mortality (AUROC 0.77 vs. AUROC 0.54 for ISS and 0.64 for TRISS).[43]

Trauma and Injury Severity Score, as well as IMPACT and CRASH models, has shown a good discriminatory power in measuring outcomes in a Chinese work enrolling TBI patients in Hong Kong.[44]

Furthermore, the generalizability of the many published prognostic systems for early prediction of outcome in TBI requires external validation.[1245] Thus, the two prognostic models (IMPACT and CRASH) were recently cross-validated and externally validated in numerous trials where the samples were composed of patients from Western countries.[46474849]

Our work was a contribution from a part of the world where TBI is a greater health problem than in many Western countries with high socioeconomic costs. However, there are some concerns. First the data came from a retrospective study, with its inherent biases. Second, our study was conducted in a single center. Third, we used admission parameters, which could vary in time. Furthermore, the main limitations were related to the high rates of patients’ exclusion and death.

Conclusion

In our context, there is a need for a simple, easily applicable, objective bedside scoring system, based on well-established prognostic factors, to predict the short-term outcome of head injury. This will help in the efficient use of resources and in communicating with the families of the victims. This study demonstrated that the APACHE-II score had a better discriminative power for early accurate prediction of outcome after moderate and severe head injuries, especially in the Moroccan health system. However, due to the underestimation of mortality and to the extremely high death and exclusion rates, the results of this work cannot be generalized to any other populations. Thus, there is a need for further multicenter studies to allow trauma care providers to set uniform standards for predicting outcomes in traumatic brain injury particularly in low-and middle-income countries.