Developing a prognostic model for traumatic brain injury--a missed opportunity?

Traumatic brain injury (TBI) is a major cause of death and disability throughout the world. The annual incidence of TBI in the United States has been estimated at between 180 and 250 per 100,000 population per year [1]. TBI is the cause of one third to one half of all trauma deaths, and the leading cause of disability in people under 40, severely disabling 15 to 20 per 100,000 population per year [2]. Injuries, including TBI, are projected to account for 20% of the worldwide burden of death and disability by 2020 [3].

Research into new therapies for TBI has been disappointing. At least 21 multi-centre clinical trials have been conducted since 1985, none of which have shown a convincing benefit in the treatment of TBI [4]. Clinical trials into TBI provide significant challenges: trauma is a neglected research topic worldwide, consent in unconscious patients requires careful ethical consideration, and the injuries are very heterogeneous in terms of mechanism and pathology [4,5].

Prognostic Models

Prognostic models are statistical models that combine data from patients to predict clinical outcome. Such models based on data collected soon after presentation could in theory be used to aid early clinical decision making and to allow more accurate counselling of patients' families. They could also have a pivotal role in the future design and analysis of randomised controlled trials, and assist in clinical audit by allowing adjustment for case mix [5].

However, while many prognostic models have been developed, none are widely used. A recent systematic review of prognostic models in TBI found that they were developed from small samples of patients, had poor methodology (for example, in over half of the models, loss to follow-up was not reported), were rarely externally validated, and were not clinically practical [5].

Linked Research Article

This Perspective discusses the following new study published in PLoS Medicine:

Steyerberg EW, Mushkudiani N, Perel P, Butcher I, Lu J, et al. (2008) Predicting outcome after traumatic brain injury: Development and international validation of prognostic scores based on admission characteristics. PLoS Med 5(8): e165. doi:10.1371/journal.pmed.0050165

Ewout Steyerberg and colleagues describe a prognostic model for the prediction of outcome of traumatic brain injury using data available on admission.

Developing a New Prognostic Model for TBI

In recent years, there have been two milestones that helped forward research into TBI: the formation of the IMPACT (International Mission on Prognosis and Analysis of Clinical Trials in TBI) database [4], and the CRASH (Corticosteroid Randomisation After Significant Head Injury) trial [6]. The IMPACT database combined patient data from eight randomised controlled clinical trials and three observational studies to give a patient population of over 9,000. The CRASH trial was a randomised controlled trial of the effect of early steroid administration on outcome after TBI. The CRASH trial enrolled 10,008 patients and is the largest clinical trial ever conducted in patients with TBI.

This year, the CRASH trial collaborators published in the BMJ a series of prognostic models for predicting outcome after TBI, based on the 10,008 patients enrolled in the CRASH trial [7]. As well as being the largest prognostic models developed for TBI, the new models display other key differences from previous models: they were externally validated, included patient data from low- and middle-income countries, and are publicly available. The external validation for these models took place against patient data in the IMPACT database.

In a paper published in this issue of PLoS Medicine, Ewout Steyerberg and colleagues describe the development and validation of new prognostic models for TBI [8]. The authors review patient characteristics from 8,509 patients in the IMPACT database. From an initial examination of 26 potential predictors, the authors define a core prognostic model based on three clinical predictors: age, motor component of Glasgow coma score (GCS), and pupillary reactivity. They also develop an extended model, which includes the three core predictors plus information on secondary insults and computerized tomography characteristics, and a laboratory model, which also includes haemoglobin and glucose measurement.

The endpoint was Glasgow outcome scale (GOS) at six months (the GOS runs from 1 to 5, with 1 representing death and 5 representing recovery; see http://www.dundee.ac.uk/medther/Stroke/Scales/Gos.htm). For validation, the authors focus on prediction of death (GOS 1) versus survival (GOS 2–5) and of unfavourable (GOS 1–3) versus favourable (GOS 4–5) outcome. The model was externally validated on data from 6,272 patients from the CRASH trial. All predictors had statistically significant associations with six-month GOS. The discriminatory ability of the models was shown to increase with increasing complexity. External validation confirmed the discriminatory ability of the core model and a modified computerized tomography model with 6,681 and 5,309 patients, respectively, from the CRASH trial. The calibration was not as good, in that outcomes from the CRASH trial were systematically poorer; however, calibration was near perfect when only the patients from high-income countries were included.

Steyerberg and colleagues' paper in PLoS Medicine and the recent BMJ paper use very similar methodology to develop their predictive models (one used the IMPACT dataset as the training dataset and the other the CRASH dataset, externally validating against the other). It is debatable whether publishing two papers separately added value—they could have possibly been combined in a single publication.

What Are the Strengths and Limitations of This Model?

While the predictors of age, motor score, and pupillary reactivity have been included in many prognostic models for TBI, Steyerberg and colleagues' models [8] and the models produced by the CRASH collaborators [7] are the largest and most robustly validated that have been developed so far. They are also readily accessible to clinicians via a Web-based calculator [9,10].

Caution must be advised in applying population-based estimates of outcome to individual patients. While both of these groups of prognostic models have been externally validated, they have essentially been validated against the populations of each other. All of the patients from CRASH and most of the patients from IMPACT were enrolled in clinical trials. Work is needed to establish the accuracy of these models prospectively in patients not enrolled in clinical trials.

The CRASH dataset is more heterogeneous than many of the phase III neuroprotection studies because inclusion GCS included moderate and severe injury and included a high number of patients from low- and middle-income countries. Steyerberg and colleagues' models seemed less accurate at predicting outcome following TBI in low- and middle-income countries.

Logistic regression, used in the PLoS Medicine and BMJ papers [7,8], treats the patient cohorts as homogeneous and therefore has a number of limitations. In contrast, decision tree analysis takes a group of subjects and divides them into subgroups according to the best discriminating factor within the original group. The level at which each discriminating factor (blood pressure, age, GCS, gender, etc.) is important is given, and the process is repeated recursively producing subgroups with less variability or entropy than the original. If followed to the conclusion of the process, each patient would end up in a category of his or her own, a situation that is closer to clinical reality than treating all patients as a homogenous group and thus the same. Decision tree analysis requires a finite number of target categories to be of value. Logistic regression analysis was previously compared with decision tree analysis for predicting recovery from traumatic brain injury, using a small dataset from 124 adult head-injured patients in an intensive care unit [11].

Comparison between the logistic regression analysis and decision tree analysis shows a number of similarities, particularly that hypotension is a strong indicator of bad outcome and that age, gender, and brainstem syndromes are important indicators. Differences include the creation of smaller patient groups by the decision tree, each of which is characterized, whereas the regression analysis suggests factors that discriminate for the set of patients as a whole [12,13]. Decision tree analysis identifies more homogenous subgroups and predictors that are important to that subgroup. Such subgroups might include, for example, a good prognosis group with the following discriminating factors: young age, extradural haematoma, and initial good GCS. A possible poor prognosis group might be identified as: elderly patients, acute subdural haematoma, low GCS, and severe comorbidity. It is not plausible that the same prognostic factors will carry the same importance in such disparate categories of TBI.

The logistic regression analysis and decision tree analysis show age is an important co-variant in prediction of outcome, but the decision tree gives threshold values of 50 years for the prediction of death and 30 years for prediction of good or poor outcome. It might be clinically more informative to apply decision tree analysis to the IMPACT and CRASH datasets for prediction and hypothesis generation/revision.

Clinical Implications

We now have new prognostic models for TBI developed from large numbers of patients, which have been externally validated to allow prediction between favourable or unfavourable outcome at six months [7,8]. These models may provide useful additional information in regard to clinical decision making and the counselling of patients' relatives, but it must be remembered that their outcomes apply to populations—and so great caution is needed if applying them to individual patients. These models will undoubtedly be of great use in clinical audit, allowing comparison between different units and changes in management over time, while allowing for different case mix. The models should also allow better trial design and analysis: many patients with TBI previously included in clinical trials have expected outcomes that are so good or bad that no intervention could be expected to alter the outcome [14]. Future clinical trials could focus on those patients with a more uncertain prognosis.