Development and temporal external validation of a simple risk score tool for prediction of outcomes after severe head injury based on admission characteristics from level-1 trauma centre of India using retrospectively collected data.

OBJECTIVE: To develop and validate a simple risk scores chart to estimate the probability of poor outcomes in patients with severe head injury (HI).
DESIGN: Retrospective.
SETTING: Level-1, government-funded trauma centre, India. PARTICIPANTS: Patients with severe HI admitted to the neurosurgery intensive care unit during 19 May 2010-31 December 2011 (n=946) for the model development and further, data from same centre with same inclusion criteria from 1 January 2012 to 31 July 2012 (n=284) for the external validation of the model. OUTCOMES: In-hospital mortality and unfavourable outcome at 6 months.
RESULTS: A total of 39.5% and 70.7% had in-hospital mortality and unfavourable outcome, respectively, in the development data set. The multivariable logistic regression analysis of routinely collected admission characteristics revealed that for in-hospital mortality, age (51-60, >60 years), motor score (1, 2, 4), pupillary reactivity (none), presence of hypotension, basal cistern effaced, traumatic subarachnoid haemorrhage/intraventricular haematoma and for unfavourable outcome, age (41-50, 51-60, >60 years), motor score (1-4), pupillary reactivity (none, one), unequal limb movement, presence of hypotension were the independent predictors as its 95% confidence interval (CI) of odds ratio (OR)_did not contain one. The discriminative ability (area under the receiver operating characteristic curve (95% CI)) of the score chart for in-hospital mortality and 6 months outcome was excellent in the development data set (0.890 (0.867 to 912) and 0.894 (0.869 to 0.918), respectively), internal validation data set using bootstrap resampling method (0.889 (0.867 to 909) and 0.893 (0.867 to 0.915), respectively) and external validation data set (0.871 (0.825 to 916) and 0.887 (0.842 to 0.932), respectively). Calibration showed good agreement between observed outcome rates and predicted risks in development and external validation data set (p>0.05).
CONCLUSION: For clinical decision making, we can use of these score charts in predicting outcomes in new patients with severe HI in India and similar settings. © Author(s) (or their employer(s)) 2021. Re-use permitted under CC BY-NC. No commercial re-use. See rights and permissions. Published by BMJ.

This is the first study from Indian settings to develop and to internally and externally validate a tool to predict poor outcome in patients with severe head injury.

The prognostic models were reported according to the most recent relevant guidelines available.

The events per variable (EPV) ratio was more than the required EPV ensured the lack of overfitting.

There is a need of external validation of models from multiple centres to enhance the models’ generalisability.

Missing data for outcome at 6 months follow-up were prominent largely (27.9% and 32.4%, in development and validation data set, respectively).

Background

Traumatic brain injury (TBI) poses a leading cause of disability and mortality worldwide with an estimate of more than 50 million people affected annually and it is expected that, over their lifetime, about half the world’s population will have one or more TBIs.1 2 It is predicted to surpass many diseases as a major cause of death and disability by the year 2020.3It is the main cause of one-third to one half of all trauma deaths and the leading cause of disability in people under 40, severely disabling 15–20 per 100 000 populations per year.4 About 90% of trauma-related deaths take place in low-income and middle-income countries (LMICs).5 Disability-adjusted life years due to injury progressively climb with decreasing national income levels.6 Until 2030, TBI as neurological injury is expected to remain the most important cause of disability from neurological disease across all ages.7 Moreover, the relative proportion of TBI in injury cases is larger and the odds of dying are more than doubled in low-income settings.8 TBI is a leading cause of mortality, morbidity, disability, socioeconomic losses and poor quality of life among survivors in India and other LMIC countries.9–11

Prognostic models are essential for many purposes in case of TBI, which help clinicians to provide reliable information to patients and relatives on expectation of outcomes, and to facilitate comparative audit of care between centres and countries, and to risk adjustment when evaluating other patient characteristics and treatment strategies.2 12 TBI is a heterogeneous and complex disease, in which there are several factors that may affect mortality and poor long-term outcome.13–16 The risk scores are the simplified version of a prognostic model, which are assigned to each risk factor based on regression coefficients,17 18 which would be easy to implement in the emergency room before therapeutic interventions for clinical decision making with limited clinical data. Accurate prognostication using risk score can help in justifiable transfer to neurosurgical specialist services, early management of the individual patient and appropriate resource allocation.19–21

Titterington et al demonstrated that it was the choice of variables and the setting in which models were applied which is more important rather than the formulae.22 Most of the trauma happens in LMICs, but only a few prediction models have been developed using these populations.17 19 23 Prognostic models are normally published for patients with moderate or severe TBI, but very few of these models are being used widely, as many of them were developed using small samples and poor modelling strategy, and they are rarely validated on external populations, which limit its generalisation.17 24 25 Furthermore, very few of these were clinically practical as they were not presented to clinicians in a user friendly way.17

Our previous study from the same study centre showed that the outcomes like in-hospital mortality and unfavourable outcome at 6 months were significantly higher in severe TBI as compared with moderate. It also showed that a significant number of patients with poor admission Glasgow Coma Scale (GCS) score may have good outcome, therefore, aggressive management should be done for all patients with severe TBI.26 However, if a better multivariate model is developed which can mirror the outcome, management can be tailored accordingly, especially in a resource-constrained setting. Despite several identified predictors, there is no simple evaluation tool to quickly assess the likelihood of short-term and long-term poor outcomes in case of severe TBI alone in our settings based on routinely available variables. Keeping in view of the above gaps, our aim was to develop and validate a simple score chart based on prognostic models for prediction of in-hospital mortality and unfavourable outcome according to Glasgow Outcome Scale (GOS) at 6 months post-trauma using admission characteristics in patients with severe HI.

Methods

Design

In this retrospective study, we considered all patients with severe HI admitted to the neurosurgery intensive care unit (ICU) of Jai Prakash Narayan Apex Trauma Centre (JPNATC) under All India Institute of Medical Sciences (AIIMS), New Delhi (India) during 19 May 2010–31 December 2011 for the model development set and further, retrospectively collected data from same centre with same inclusion criteria from 1 January 2012 to 31 July 2012 was used for the temporal (external) validation of the developed model. Presently, JPNATC, AIIMS, New Delhi, a largest fully fledged integrated level I, tertiary care trauma centre in the Indian subcontinent, has 30 triage and 36 ICU beds, and it is the apex referral centre for TBI in New Delhi as well as for much of the other parts of the country.26

There were two sources of data collection: computerised patient record system (soft copy) and patient’s medical record file (hard copy). Trained research nursing and other clinical staffs including medical doctors maintained these both sources. Under the supervision of neurosurgeon DA (one of co-authors), author VKK collected all study variables from both these existing sources in a prefixed pro forma for all treated patients enrolled during the study period. Later on, a library was created in software EPI Info V.7.1.2, Centers for Disease Control and Prevention, Atlanta, Georgia (USA) for entry of collected data.

Patients, study site and inclusion criteria

Figure 1 summarises the flow of participants through the study in both development and validation data set. This study included all patients had postresuscitation GCS ≤8 at the time of admission at emergency department (ED) and subsequently, admitted to ICU under the Department of Neurosurgery at JPNATC, AIIMS, New Delhi, India within 72 hours of injury. Clinical injury severity was based on the GCS score at admission to ED of the participating hospital and defined as severe (GCS 3–8). During the study period, there were a total of 1230 consecutive eligible patients admitted to the ICU of the neurosurgery department, for whom at least one CT scan was performed after admission to the ED. From a total of 1230 patients, a total of 948 were used for model development and a total of 284 were used for model’s external validation. However, data on unfavourable outcome at 6 months were available for 874 patients. All patients were treated and evaluated according to the ‘Guidelines for the Management of Severe Head Injury’.27

Figure 1

Flow chart of patients’ population used in model’s development and validation. GCS, Glasgow Coma Scale.

Predictors and outcome(s)

We considered patients’ characteristics, which were previously reported as important predictors in literature and/or could be determined easily and reliable within the first few hours after injury.16 These comprised the information based on demographics (age, gender), clinical severity (the motor GCS at admission, pupillary reactivity, limb movement and major extracranial injuries), secondary insult (hypotension) and various CT findings (midline shift, subdural haematoma, epidural haematoma (EDH), basal cistern effaced, presence of traumatic subarachnoid haemorrhage/intraventricular haematoma (tSAH/IVH)). As soon as the patients were admitted to the neurosurgery casualty ward, the total GCS of the patient was assessed. As it may be impossible to assess eye opening as a result of periorbital swelling and a verbal response cannot be adequately assessed in intubated patients, motor GCS is more reliable and can be used as a predictor that reflects clinical severity instead of total GCS.28 Pupillary reactivity, limb movement (hemiparesis), motor score and others admission characteristics were also measured on admission in the neurosurgery casualty ward. Since only a few patients were classified as ‘Obeys commands’ category, this category was combined with the ‘localising pain’ category of motor score. Hypotension was considered to be present when a patient had systolic blood pressure below 90 mm Hg at least once within the first 24 hours of admission.

Two outcomes, (1) in-hospital mortality and (2) unfavourable outcome according to GOS29 at 6 months post-trauma were measured. The 6-month GOS was dichotomised into favourable outcome (moderate disability, good recovery or GOS=4, 5) and unfavourable outcome (death, persistent vegetative state and severe disability or GOS=1–3). Neurosurgeons and the author (VKK) were involved in determining the 6-month follow-up data through direct interview of patients in clinic or by telephonic interview of patients or their care takers.

Sample size consideration

For developing a prognostic model or risk score, it is widely suggested that the data set used for model development should contain a minimum of 10 outcome events per variable (EPV) included as a predictor in the model.30–32 A total of 12 candidate predictors were chosen for this study. We ended with 21 candidate parameters as some of these predictors were categorical variables with more than two levels. This means the EPV ratio was approximately 18 and 23 for in-hospital mortality and 6 months unfavourable outcome, respectively, which are far more than the required EPV. The large sample size ensured that there would not be a problem of overfitting.33

Statistical analysis

To see the association between two categorical variables, Pearson χ2 test was used. Non-normally distributed continuous variable (ie, age) (examined using Shapiro-Wilk test) between two independent groups were compared using Wilcoxon rank-sum test. To find out the potential factors associated with outcomes, and hence, for developing prognostic models for both the outcomes, that is, in-hospital mortality (no/yes), and 6 months outcome (favourable/unfavourable), logistic regression procedure was used. Variables which were found to be significant at the level of 5% under crude association analysis (univariable analysis) were included in the multivariable regression analysis. Results are presented in the form of unadjusted OR, and adjusted OR (AOR) and corresponding 95% CI using univariable and multivariable logistic regression analysis, respectively.33 For development of the risk score and clinical interpretation, the continuous variable age was categorised into meaningful categories after studying the shape of the relationship between age and outcome(s) using linear prediction. A linear relationship of age with outcomes was found to be a good approximation (figure 2). There was no any missing value in all predictor variables and outcome ‘in-hospital mortality’. We performed complete-case analysis for both the outcomes. The information lost about 6-month follow-up outcome data (27.9% and 32.4%, in development and validation data set, respectively) restricted our multivariable analysis to n=682 for model development and external validation analysis to n=192 for unfavourable outcome at 6 months. All the p values<0.05 were taken as significant. Data analysis was done using software Stata V.12.1. Internal validation was performed using R V.3.6.1.

Figure 2

Relationship of age with in-hospital mortality and unfavourable outcome.

Performance of the models

The performance of models was assessed in terms of discrimination and calibration.30 Discrimination was quantified by using area under the receiver operating characteristic curve (AUC) which determines whether those with higher predicted risks are more likely to have a poor outcome among all possible pairs of patients with different outcomes.30 Calibration was assessed graphically and tested with the Hoshmer-Lemeshow (H-L) goodness of fit test, which determines agreement between predicted and observed risks over the full range of predicted probabilities.34

Clinical score chart development

We developed a simple risk score chart in a user-friendly manner for estimating the probability of in-hospital mortality and unfavourable outcome at 6 months post-trauma based on multivariable regression coefficients to facilitate application of the models in clinical practice. The regression coefficients were rescaled and rounded to the whole number.35 The rescaling and rounding were such that the performance of the models remained similar to that of the original models.

For calculating exact probability of outcomes, excel based calculator and formula based on regression coefficients have been provided and are available online as additional online supplemental material.

Validation of developed models

Multivariable models and its risk scores were internally and externally validated. The bootstraps resampling technique was used to assess the internal validation of the models.36 Regression models were estimated in 200 bootstrap samples, as 200 bootstrap samples are often sufficient to obtain stable estimates.30 For obtaining an estimate of the predictive accuracy as corrected for bias, we refitted and tested the model on the original samples for each of the 200 bootstrap samples. For strengthening the generalisability of prognostic models, we did temporal external validation using an independent data set.

Patient and public involvement

There was no such involvement of patients and public in this study, but outcome at 6 months was measured with patients or their care takers in clinic or by telephonic interview. The output tool arrived from this research might be disseminated in new patients with severe HI and trauma centres in Indian settings.

Reporting

We followed Transparent Reporting of a multivariable prediction model for Individual Prognosis Or Diagnosis (TRIPOD) guidelines for our current research work.37

HIs are also generally referred to as brain injury or TBI. Therefore, in this paper, we have used both the words interchangeably.

Results

Patient characteristics and outcomes

Table 1 shows the patient characteristics and outcomes in the development and validation dataset. The p values for distribution of each of the characteristics between development and validation groups signified that they were almost similar in both the patient population except two variables (major extracranial injury and midline shift on CT). The median age was 30 years in both the groups and most of the patients were men in both development and validation datasets (87.0% and 87.3%, respectively). The proportion of patients with in-hospital mortality and unfavourable outcome in the development dataset were 39.5% and 70.7%, respectively.

Table 1

Patient characteristics and outcome in the development and validation data set

Characteristics	Development data set (n=946) n (column %)	Validation data set (n=284) n (column %)	P value
Demographic
Age (years)
Median (IQR)	30 (21–41)	30 (22–40)	0.999
Male	823 (87.0)	248 (87.3)	0.886
Clinical
Motor score
No motor response (1)	124 (13.1)	30 (10.6)	0.467
Extension to pain (2)	191 (20.2)	54 (19.0)
Flexion to pain (3)	70 (7.4)	29 (10.2)
Withdrawal from pain (4)	145 (15.3)	42 (14.8)
Localising pain/obeys commands (5/6)	416 (44.0)	129 (45.4)
Pupil reactivity
Both pupils reacted	668 (70.6)	195 (68.7)	0.742
One pupil reacted	96 (10.2)	33 (11.6)
No pupil reacted	182 (19.2)	56 (19.7)
Limb movement			0.103
Bilateral well	671 (70.9)	204 (71.8)
Unequal	156 (16.5)	56 (19.7)
Bilateral Absent	119 (12.6)	24 (8.5)
Major extracranial Injury (yes)	277 (29.3)	58 (20.4)	0.003
Secondary insult
Hypotension (yes)	260 (27.5)	87 (30.6)	0.301
Various CT findings
Midline shift (>5 mm) (yes)	389 (41.1)	138 (48.6)	0.026
SDH (yes)	450 (47.6)	137 (48.2)	0.843
EDH (yes)	159 (16.8)	38 (13.4)	0.167
Basal cistern effaced (yes)	171 (18.1)	59 (20.8)	0.306
tSAH/IVH (yes)	381 (40.3)	81 (28.5)	0.001
Outcome(s)
In-hospital mortality (yes)	374 (39.5)	98 (34.5)	0.127
	(n=682)	(n=192)
Unfavourable outcome (yes)	482 (70.7)	133 (69.3)	0.707

EDH, epidural haematoma; IVH, intraventricular haemorrhage; SDH, Subdural haematom; tSAH, traumatic subarachnoid haemorrhage.

Development of the models

Tables 2 and 3 show the results of univariable and multivariable logistic regression for in-hospital mortality and unfavourable outcome at 6 months, respectively. For both outcomes, univariate analysis revealed that all the candidate predictors were statistically significant except two variables, viz., gender and major extracranial injury. The multivariable analysis revealed that for in-hospital mortality, age (51–60, >60 years), motor score (1, 2, 4), pupillary reactivity (none), presence of hypotension, basal cistern effaced and tSAH/IVH; and for unfavourable outcome, age (41–50, 51–60, >60 years), motor score (1–4), pupillary reactivity (none, one), unequal limb movement and presence of hypotension were the independent predictors as its 95% CI of OR does not contain one. Among these, for in-hospital mortality, motor score (AOR (95% CI)=7.8 (3.4 to 17.7) for score=1, and 4.6 (2.7, 7.6) for score=2), presence of hypotension (AOR (95% CI)=8.3 (5.3 to 13.2)) and none pupillary reactivity (AOR (95% CI)=4.6 (2.5 to 8.2)) were the stronger prognostic factors; while for unfavourable outcome at 6 months, motor score (AOR (95% CI)=4.4 (1.3 to 14.9) for score=1, and 7.6 (3.8 to 15.3) for score=2, 8.0 (3.2 to 20.3) for score=3, 3.3 (1.9 to 5.8) for score=4), age (AOR (95% CI)=3.5 (1.6 to 8.3) for 41–50 years, 5.3 (1.6 to 17.5) for 51–60 years, and 12.8 (3.0 to 53.7) for >60 years), presence of hypotension (AOR (95% CI)=7.2 (3.4 to 15.0)) and none pupillary reactivity (AOR (95% CI)=5.8 (2.1 to 16.4)) were the stronger prognostic factors. Other variables including CT variables possessed moderate prognostic effects for both outcomes. For both outcomes, an EDH on a CT turned out to be a relatively protective factor in multivariable analysis, although it was not a statistically significant (AOR (95% CI)=0.7 (0.4 to 1.2) for in-hospital mortality and 0.6 (0.4 to 1.1) for unfavourable outcome).

Risk score chart development for clinical application

Table 4 shows a simple risk score chart, in which a numeric score has been assigned to each level of all predictors in multivariable models, which was developed to predict the probability of in-hospital mortality and unfavourable outcome at 6 months in patients with severe HI. The scores in the chart need to be added for predicting outcome for an individual patient and the sum of this score corresponds to probability of poor outcome in figure 3. For example, a 43-year-old patient, with a motor score of 3 (abnormal flexion) and one pupil reacting, having unequal limb movement, a hypotension episode at admission, and his/her CT report shows presence of basal cistern effaced and tSAH has a score of 1+1 + 1+1 + 4+2 + 2=12 points for in-hospital mortality and 3+4 + 2+1 + 4+1 + 1=16 for unfavourable outcome. According to figure 3, these scores correspond to the risks of in-hospital mortality and unfavourable outcome at 6 months, ~94% and 100%, respectively. Vertical lines at the graph indicate 95% CI of point estimate.

Table 4

Risk score chart for in-hospital mortality and unfavourable outcome at 6 months

Prognostic variables	*SMortality	†SUO
Age (years)
≤18	0	0
19–30	0	1
31–40	0	1
41–50	1	3
51–60	2	3
>60	2	5
Motor score
None (1)	4	3
Extension (2)	3	4
Abnormal flexion (3)	1	4
Withdrawal from pain (4)	2	2
Localises/ obeys (5/6)	0	0
Pupillary reactivity
Both reacted	0	0
One reacted	1	2
No reacted	3	4
Limb movement
Bilateral well	0	0
Unequal	1	1
Bilateral absent	1	2
Hypotension
No	0	0
Yes	4	4
Midline shift
No	0	0
Yes	1	0
Subdural haematoma
No	0	0
Yes	0	0
Epidural haematoma
No	0	0
Yes	−1	−1
Basal cistern effaced
No	0	0
Yes	1	1
tSAH/IVH
No	0	0
Yes	2	1

*SMortality: risk score for in-hospital mortality.

†SUO: risk score for unfavourable outcome.

IVH, intraventricular haematoma; tSAH, traumatic subarachnoid haemorrhage.

Figure 3

Predicted probability (with 95% CI, indicated as vertical line) of in-hospital mortality and unfavourable outcome corresponding to the sum scores from table 4.

Table 5 shows the performances of multivariable models and risk scores in terms of discrimination and calibration ability in development, internal and external validation data sets. The performance of the risk score was almost similar to the original models in development, internal and external validation data set. The discriminative ability (AUC (95% CI)) of the score chart for in-hospital mortality and 6 months outcome was excellent in the development data set (0.890 (0.867 to 912) and 0.894 (0.869 to 0.918), respectively), internal validation data set (0.889 (0.867 to 909) and 0.893 (0.867 to 0.915), respectively) and external validation data set (0.871 (0.825 to 916) and 0.887 (0.842 to 0.932), respectively). Both internal (AUC: 0.880–0.893) and external validation (AUC: 0.871–0.893) of the models and risk scores confirmed the discriminative ability of the models for both outcomes and these models could be used for predicting the outcome of new patients in similar settings.

Table 5

Performance of the models

	In-hospital mortality			Unfavourable outcome
		Discrimination	Calibration		Discrimination	Calibration
	n	AUC (95% CI)	P value*	N	AUC (95% CI)	P value*
Development data set
Original model	946	0.893 (0.871 to 0.915)	0.957	682	0.897 (0.873 to 0.921)	0.605
Risk score	946	0.890 (0.867 to 912)	0.558	682	0.894 (0.869 to 0.918)	0.587
Internal validation
Original model	946	0.881 (0.858 to 0.901)†	–	682	0.880 (0.868 to 0.909)†	–
Risk score	946	0.889 (0.867 to 0.909)†	–	682	0.893 (0.867 to 0.915)†	–
External validation
Original model	284	0.875 (0.831 to 0.919)	0.391	192	0.893 (0.849 to 0.937)	0.516
Risk score	284	0.871 (0.825 to 0.916)	0.360	192	0.887 (0.842 to 0.932)	0.415

*Hoshmer-Lemeshow test.

†Optimism corrected using bootstrap method.

AUC, the area under the curve.

As evident from the table 5, calibration ability for both models and risk score was good in the development as well as external validation dataset as p>0.05 evaluated with the H-L test for goodness of fit. The calibration or goodness of fit plots for risk score is shown in figures 4 and 5. In all these figures, the observed proportions (indicated in circles) are more or less near to predicted proportions or nearer to the straight line passing through origin. In figures 6 and 7 (calibration plots in internal validation data set), we show good agreement of the predictions from the risk scores with observed in-hospital mortality and unfavourable outcome, respectively, using bootstrap resampling methods.

Calibration plot of risk score for in-hospital mortality.

Calibration plot of risk score for unfavourable outcome.

Calibration plot of risk score for in-hospital mortality in internal validation data set.

Calibration plot of risk score for unfavourable outcome in internal validation data set.

Discussion

In this paper, using multivariable prognostic models, we proposed a methodologically valid, simple and robust evaluation tool, in the form of risk score, in user friendly manner to quickly assess the likelihood of short-term (in-hospital mortality) and long-term outcomes (unfavourable outcome at 6 months) in case of severe HI in our settings based on routinely available admission variables. The models discriminated well between patients with poor and good outcomes in development and validation data sets. The calibration ability confirmed the good agreement between observed proportions and predicted risks of outcomes in independent data sets. The present risk score chart may be used for predicting the outcome of new patients in similar settings.

Our findings of independent predictors are in the line of previous findings reported in literatures as most of these have previously been identified as prognostic factors for poor outcome in cases of HI or TBI.16 17 25 34–44 For 6 months outcome, OR for motor scores does not commensurate with severity of motor score. Interestingly, the same results were observed in other two studies also.38 41 It means, in other words, for whatever reason, a patient with poor motor score could have a better prognosis with good outcome. To the best of my knowledge, present study is the first to use and show that unequal limb movement (hemiparesis) as an independent prognostic factor with marginal significance for unfavourable outcome at 6 months in severe HI. However, further study is needed to strengthen our result.

Many studies used Marshall’s CT classification and other classification.,24 38 39 but this study used the major individual CT characteristics, which are easily and readily available at our study centre. Our study results confirm the prognostic value of the CT characteristics, specially the presence of tSAH /IVH in severe HI and are found to be consistent across the studies.45 In present study, the presence of EDH in the CT scan was associated with a better outcome after severe HI, although it was statistically not significant, which may be explained by the possibility of surgical evacuation at emergency basis of such type of haematomas.38 Although there is generally little intrinsic brain damage, an EDH often disturbs brain function because of compression. There is a possibility of occurring of full recovery if compression is relieved in time.

There are multivariable models for outcome predictions in TBI, but only a few that focused on severe head trauma,40 44 46 47 in fact, this is first from our setting. We have adopted TRIPOD guidelines and done both, internal and external validation. Although many previous studies did not report calibration of models, we reported both, discrimination and calibration ability and this study are based on relatively lager sample size as compared with previously reported studies on severe head trauma. Prognostic modelling on small samples may suffer with biased estimates and imprecise regression coefficients.33 48 Our present study confirms that the model developed on larger samples discriminated and calibrated best in new patients. Also, a large sample size assured more statistical power to support the conclusion. A good model should have a balance between these two dimensions of performance, that is, discrimination and calibration ability of models, in order to find the best trade-off.49 For predictive modelling approach, overfitting may occur with small sample, in which a model predicts an outcome well in the development sample but tend to predict extreme probabilities for new patients in external validation sample.24 The risk of overfitting can be minimised with adaptation of larger sample size, and in our study, we found that performances of models for both outcome are very good in internal and external validation data set which means there is no overfitting. The discriminative ability will be boosted upwards, if the population includes subsets with a more extreme prognosis.50 The reason for high discriminative ability could be the data set from the same centre being used in this study for development and validation of models. There are various issues, which may affect the validity and applicability of the prognostic models. The local level of care may vary between centres or cities or countries or regions, which may result in differences in outcome. For generalisability of findings, external validation is essential, which indicates how a model performs in new patients. Most of the time, models perform worse during external validation than during the initial assessment because validation samples are totally independent of the development sample.

For making prognostic models clinically useful, risk score charts were developed in the user-friendly manner, which may support clinician in the initial assessment of the severity and prognosis of a patient with TBI and this information may also be useful for relatives of the patients as well. The certain structural assumptions and statistical interaction terms were not included regarding the prognostic models, because we wanted to make our model simple, interpretable and clinically useful. Hence, it is possible that specific patterns of risk factors reflected insufficiently in the model prediction. However, we should keep in mind that statistical models can only augment, not replace clinical judgement. Prediction should be made with care and not directly be applied when making clinical decisions.51 In the UK, four centres study, it was found that making predictions available as part of a routine clinical service reshaped deployment of resources.52 It should be brought in mind that the prognostic models can never replace the clinical assessment.

Our present study has several limitations which should be acknowledged. As it was our primary intent to focus on only variables which are easily, and reliably assessed at our centre in routine manner and we wanted to focus on the baseline situation, without influences of subsequent clinical treatment, it is likely that prognostic models including information from later time periods will perform better.53 Conclusions can be drawn only concerning models using baseline characteristics. In present study, only those patients were included which came to the study centre (ie, JPNATC) within the first 72 hours of injury. The specialised status of JPNATC, AIIMS may have introduced bias towards inclusions of more severely injured patients. Missing data for outcome at 6 months follow-up were prominent largely due to lack of contact information. Therefore, selection bias might have occurred for 6 months outcome’s model because we excluded patients with missing outcome data and restricted the multivariate analysis to the 682 patients. Most of the patients were from the northern part or northern states of India, especially from Delhi or NCR regions. Findings of present study, therefore, should be interpreted with cautions. Models were developed from single-centre data and temporal validation of developed models was done using data from the same hospital. Hence, there is a need for development and validation of prognostic models based on patient populations from multiple centres to enhance the models’ generalisability. Also, some factors affecting external validation may have altered since this data collection, we, therefore, recommend repeatedly validating prognostic models on new patient populations. Also, we could do further research to evaluate the impact of using this risk score in clinical practice to predict patient outcomes and to assess the acceptability of the tool by clinicians in the EDs in case of patients with severe TBI.

Conclusion

We have developed and validated practical models using routinely available admission characteristics to predict in-hospital mortality and unfavourable outcome at 6 months in patients with severe HI. The models’ performance is good, and we recommend for the use of these models and its score charts in predicting outcomes in new patients with severe HI in India and similar settings. These models may be useful for both clinicians as well as patients to assess the outcomes after severe HI for providing realistic information to relatives on expectation of outcomes, for quantifying and classifying the severity of patients with TBI, for risk adjustment when evaluating other patient characteristics on treatment strategies, for designing future clinical trials, and for supporting clinical decision making and limited resource allocation.

Table 2

Association between predictors and in-hospital mortality in development data set

Prognostic variables	Total (n=946)	In-hospital Mortality (n=374) n (row %)	P value	Unadjusted OR (95% CI)	Adjusted OR (95% CI)
Age (years)		60 (37.3)	<0.001
≥18*	161	106 (31.7)		1	1
19–30	334	78 (37.5)		0.8 (0.5 to 1.2)	0.9 (0.5 to 1.6)
31–40	208	60 (45.8)		1.1 (0.7 to 1.5)	1.2 (0.6 to 2.3)
41–50	131	39 (63.9)		1.4 (0.9 to 2.3)	1.6 (0.8 to 3.1)
51–60	61	31 (60.8)		2.9 (1.6 to 5.5)	2.8 (1.2 to 6.4)
>60	51			2.6 (1.4 to 5.0)	2.1 (0.9 to 5.3)
Gender			0.064		–
Male*	823	316 (38.4)		1
Female	123	58 (47.2)		1.4 (0.9 to 2.01
Motor score			<0.001
1	124	112 (90.3)		53.3 (27.7 to 102.4)	7.8 (3.4 to 17.7)
2	191	130 (86.1)		12.2 (8.1 to 18.3)	4.6 (2.7 to 7.6)
3	70	26 (37.1)		3.4 (1.9 to 5.9)	1.8 (0.9 to 3.7)
4	145	44 (30.3)		2.5 (1.6 to 3.9)	2.1 (1.3 to 3.5)
5/6*	416	62 (14.9)		1	1
PR			<0.001
Both reacted*	668	170 (25.5)		1	1
One reacted	96	47 (49.0)		2.8 (1.8 to 4.3)	1.4 (0.8 to 2.5)
No reacted	182	157 (86.3)		18.4 (11.6 to 29.0)	4.6 (2.5 to 8.2)
Limb movement			<0.001
Bilateral well*	671	206 (30.70)		1	1
Unequal	156	68 (43.6)		1.7 (1.2 to 2.5)	1.5 (0.9 to 2.3)
Bilateral absent	119	100 (84.0)		11.9 (7.1 to 19.9)	1.6 (0.8 to 3.3)
MEI			0.512		–
No*	669	260 (38.7)		1
Yes	277	114 (41.2)		1.1 (0.8 to 1.5)
Hypotension			<0.001
No*	686	160 (23.3)		1	1
Yes	260	214 (82.3)		15.3 (10.6 to 22.0)	8.3 (5.3 to 13.2)
Midline shift			<0.001
No (≤5 mm)*	557	176 (31.6)		1	1
Yes (>5 mm)	389	198 (50.9)		2.2 (1.7 to 2.9)	1.3 (0.8 to 2.0)
SDH
No*	496	163 (32.9)	<0.001	1	1
Yes	450	211 (46.9)		1.8 (1.4 to 2.3)	1.1 (0.7 to 1.6)
EDH
No*	787	325 (41.3)	<0.014	1	1
Yes	159	49 (30.8)		0.6 (0.4 to 0.9)	0.7 (0.4 to 1.2)
BCE			<0.001
No*	775	270 (34.8)		1	1
Yes	171	104 (60.8)		2.9 (2.1 to 4.1)	2.1 (1.3 to 3.4)
tSAH/IVH			<0.001
No*	565	173 (30.6)		1	1
Yes	381	201 (52.8)		2.5 (1.9 to 3.3)	2.2 (1.5 to 3.2)

*Referent category; motor score: 1-no motor response, 2-extension to pain, 3-flexion to pain, 4 withdrawal from pain, 5/6-localising pain/obeys commands.

BCE, basal cistern effaced; EDH, epidural haematoma; IVH, intraventricular haemorrhage; MEI, major extracranial injury; PR, pupillary reactivity; SDH, subdural haematoma; tSAH, traumatic subarachnoid haemorrhage.

Table 3

Association between predictors and unfavourable outcome in development data set

Prognostic variables	Total (n=682)	Unfavourable outcome (n=482) n (row %)	P value	Unadjusted OR (95% CI)	Adjusted OR (95% CI)
Age (years)
≤18*	116	71 (61.2)	<0.001	1	1
19–30	231	145 (62.8)		1.1 (0.7 to 1.7)	1.4 (0.7 to 2.6)
31–40	145	102 (70.3)		1.5 (0.9 to 2.5)	1.8 (0.9 to 3.7)
41–50	95	77 (81.1)		2.7 (1.4 to 5.1)	3.5 (1.6 to 8.3)
51–60	51	46 (90.2)		5.8 (2.1 to 15.8)	5.3 (1.6 to 17.5)
>60	44	41 (93.2)		8.7 (2.5 to 29.6)	12.8 (3.0 to 53.7)
Gender					–
Male*	587	410 (70.0)	<0.238	1
Female	95	72 (75.8)		1.3 (0.8 to 2.2)
Motor score
1	121	117 (96.7)	<0.001	45.6 (16.2 to 127.7)	4.4 (1.3 to 14.9)
2	169	156 (92.3)		18.7 (10.0 to 34.9)	7.6 (3.8 to 15.3)
3	49	42 (85.7)		9.3 (4.0 to 21.7)	8.0 (3.2 to 20.3)
4	105	74 (70.5)		3.7 (2.3 to 6.1)	3.3 (1.9 to 5.8)
5/6*	238	93 (39.0)		1	1
PR
Both reacted*	431	247 (57.3)	<0.001	1	1
One reacted	77	66 (85.7)		4.5 (2.3 to 8.7)	2.8 (1.3 to 6.4)
No reacted	174	169 (97.1)		25.2 (10.1 to 62.5)	5.8 (2.1 to 16.4)
Limb movement
Bilateral well*	463	286 (61.8)	<0.001	1	1
Unequal	108	88 (81.5)		2.7 (1.6 to 4.6)	2.1 (1.1 to 4.0)
Bilateral absent	111	108 (97.3)		22.3 (7.0 to 71.2)	3.5 (0.9 to 13.4)
MEI
No*	473	333 (70.4)	0.814	1
Yes	209	149 (71.3)		1.0 (0.7 to 1.5)	–
Hypotension
No*	446	258 (57.8)	<0.001	1	1
Yes	236	224 (95.0)		13.6 (7.4 to 25.0)	7.2 (3.4 to 15.0)
Midline shift
No (≤5 mm)*	373	238 (63.8)	<0.001	1	1
Yes (>5 mm)	309	244 (79.0)		2.1 (1.5 to 3.0)	1.2 (0.7 to 2.1)
SDH
No*	336	221 (65.8)	0.006	1	1
Yes	346	261 (75.4)		1.6 (1.1 to 2.2)	0.9 (0.6 to 1.5)
EDH
No*	577	418 (72.4)	0.017	1	1
Yes	105	64 (61.0)		0.6 (0.4 to 0.9)	0.6 (0.4 to 1.1)
BCE
No*	538	360 (66.9)	<0.001	1	1
Yes	144	122 (84.7)		2.7 (1.7 to 4.5)	1.6 (0.9 to 3.1)
tSAH/IVH
No*	376	243 (64.6)	<0.001	1	1
Yes	306	239 (78.0)		1.9 (1.4 to 2.7)	1.4 (0.9 to 2.3)

*Referent category; motor score: 1-no motor response, 2-extension to pain, 3-flexion to pain, 4-withdrawal from pain, 5/6-localising pain/Obeys commands.

BCE, basal cistern effaced; EDH, epidural haematoma; IVH, intraventricular haemorrhage; MEI, major extracranial injury; PR, Pupillary reactivity; SDH, subdural haematoma; tSAH, traumatic subarachnoid haemorrhage.