Identification and Validation of a Novel Nomogram Predicting 7-day Death in Patients with Intracerebral Hemorrhage

Background: Intracerebral hemorrhage (ICH) is a serious brain condition with high mortality and disability rates. In recent decades, several risk factors related to death risk have been identified, with several models predicting mortality, but rarely used and accepted in daily clinical practice. Aims: To establish and validate a predictive nomogram of spontaneous ICH death that can be used to predict patient death within 7 days. Study Design: Cohort study.
Methods: A cohort of 449 patients with ICH, diagnosed clinically from January 2015 to December 2017, were identified as the model training cohort. Univariate analysis and least absolute contraction and selection operator (Lasso) regression were used to determine the most powerful predictors of patients with ICH. Discrimination, calibration, and clinical applicability were used to assess the function of the new nomogram. In external validation, we also evaluated the nomogram in another 148 subjects (validation cohort) examined between January and December 2018.
Results: We observed no significant differences in patient baseline characteristics in the training and validation cohorts, including sex, age, Glasgow coma scale (GCS) score, and one-week mortality rates. The model included three predictive variables from univariate and multivariate analysis, including GCS scores, hematoma volume, and brainstem hemorrhage (BSH). Internal validation revealed that the nomogram had a good discrimination, the area under the receiver operating characteristic curve (AUC) was 0.935, and calibration was good (U = -0.004, P = 0.801). Similarly, this nomogram also showed good differentiation ability (AUC = 0.925) and good accuracy (U = -0.007, P = 0.241) in the validation cohort data. Decision curve analysis indicated that the new prediction model was helpful.
Conclusion: At the early stages of the condition, our prediction model accurately predicts the death of patients with ICH.

INTRODUCTION

Intracerebral hemorrhage (ICH) is the most life-threatening stroke condition, accounting for approximately 10–30% of all cerebrovascular accidents. Approximately half of the patients die within one month after the condition is identified, with most deaths occurring within 48 h.[1] Currently, the main treatment methods include conservative medical and surgical treatments; however, surgical treatment clinical trials ICH using different methods have been largely negative or uncertain. Therefore no medical treatments are beneficial to patients with ICH. Previously, several studies proposed promising new treatments, but mortality was not reduced.[2,3,4,5,6] In recent decades, observational studies reported several risk factors influencing ICH patient death, including age, sex, state of consciousness, hematoma volume, hemorrhage location, intraventricular hemorrhage (IVH), blood pressure, and plasma glucose levels, and established several prediction ICH mortality models. While these models play positive roles in clinical decision-making, their accuracy and reliability typically remain unsatisfactory.[7] Due to variable screening methods, study populations, model complexity, unverified and other reasons, ICH model extrapolation is poor, with clinicians having to undergo difficult selections with so many prediction tools. Therefore, the clinical utility of most models is limited. In addition, death prediction by most models is usually 1 month. The time of early death prediction from two studies is 2 and 10 days, respectively.[8,9] However, the clinical value of these predictive tools is poor because models are very complex. Therefore, in this study, we established and verified a simple, accurate, and professional one-week death predictive nomogram for patients with ICH.

MATERIALS AND METHODS

Patient Characteristics

Medical data was collected from patients with ICH at our hospital from January 2017 to December 2015. We identified a training cohort by selecting patients diagnosed with spontaneous ICH. Exclusion criteria were: 1) Clinical symptoms > 1 day; 2) Age < 18 years; and 3) Patients with ICH and primary IVH secondary to traumatic brain injury, hemorrhagic infarction, arteriovenous malformation, brain tumor, and intracranial aneurysm rupture. Finally, the training set included 449 patients. For the validation set, cases were reviewed from January to December 2018, with 148 patients selected using the same criteria as above.

The university ethics committee reviewed and approved this study (number: 20200316036).

Study Data

The medical data collected included past medical history, demographic data, preliminary evaluation (including laboratory test, general physical signs, and imaging examinations), and hospital treatment information. We extracted study variables from patient data at the first evaluation. The diastolic blood pressure, systolic blood pressure, and initial Glasgow coma scale (GCS) recorded at admission were used. Other characteristics were obtained after arrival at the hospital, including sex, age, and history of smoking and drinking. The laboratory test after arrival were also recorded. The results were evaluated for 7-day mortality after ICH.

The abc/2 formula was used to measure hematoma volume; a, b, and, c represented the maximum plane of hematoma on the three vertical axes.[10]

Statistical Analyses

Continuous variables (X ± standart deviation) were assessed using Mann-Whitney U test or two independent samples t-tests. Categorical variables were analyzed using Fisher exact tests or the chi-square test. The least absolute shrinkage and selection operator (Lasso) regression method was used to select the most powerful predictors,[11] and we established a new model combined with the results of the univariate analysis and Lasso regression. Finally, this predictive model was shown in a nomogram.

First, we assessed nomogram discrimination using the area under the receiver operating characteristic (ROC) curve (AUC).[12] Then, a calibration curve evaluating the calibration was generated using the unreliability [U] statistical test.[13] U-test statistics indicated no statistical significance, suggesting that the calibration was good. Then, the model established for the training set was assessed in the verification set; model performance relating to calibration and discrimination was evaluated using the same method as above. Moreover, the clinical practicability of the model was appraised by evaluating the net benefits under various threshold values in the validation set and performing decision curve analysis (DCA).[14]

All data were analyzed in Stata (13.0) and R (3.3.0.).

RESULTS

We compared baseline characteristics between the training and validation cohorts and identified no significant differences with respect to age (60.14 ± 13.33 years, 61.36 ± 14.08 years, P = 0.343), sex [male (64 (70.3), 24 (75.0), P = 0.969)], GCS (10.98 ± 4.12, 11.22 ± 4.16, 0.542), and patient mortality [91 (20.3), 32 (21.7), P = 0.727], indicating the comparability of both cohorts. We also compared baseline characteristics between the survival and death groups in both cohorts (Table 1). From these variables, using Lasso regression analysis, the three most important predictors (Figure 1), including GCS, hematoma volume, and brainstem hemorrhage (BSH), were chosen as the best risk factor combinations for the one-week death prediction of spontaneous ICH. These factors were included in the model, fitted, and represented as a nomogram (Figure 2).While using this nomogram, each patient predictor was located on the corresponding axis. A line was then drawn on the top score axis to generate a score based on each variable. Finally, the scores from all variables were added to calculate the total score. This was located on the “total score” axis, and a straight line was drawn down to generate the one-week mortality of the patient.In the development set, ROC analyses revealed that the nomogram had good discrimination (AUC = 0.935) (Figure 3a). The calibration chart indicated that the observed and predicted values were consistent in the development set, and the U test showed no statistical difference (P = 0.801), suggesting that the model fitted well (Figure 4a). In the validation set, the new model also showed good discrimination (AUC = 0.925), (Figure 3b). U-test statistics also showed good accuracy (P = 0.204) (Figure 4b).

Table 1

Characteristics of Patients in the Training and Validation Cohorts

Figure 1

Variables selection by Lasso regression. (a) Lasso regression graph of the clinical variables. (b) A cross-validation graph of parameter selection was used for the LASSO regression analysis

Figure 2

The 7-day nomogram of patients with ICH. Brainstem hemorrhage (BSH). BSH 1 and 0 indicates the presence or absence of BSH, respectively. While using this nomogram, each patient predictor was located on the corresponding axis. A line was then drawn on the top score axis to generate a score based on each variable. Finally, the scores from all variables were added to calculate the total score. This was located on the “total score” axis, and a straight line was drawn down to generate the one-week mortality of the patient

Figure 3

The ROC curve tests the predictive efficiency of the model using the training (a) and verification (b) sets. The area under the curve (AUC) of a is 0.935. the area under the curve of b is 0.925

Figure 4

The calibration plot of training (a) and validation (b) sets. The calibration chart a indicated that the observed and predicted values were consistent in the development set, and the U test showed no statistical difference (P = 0.801). The calibration chart b showed that U-test statistics also showed good accuracy in validation sets (P = 0.241)

The DCA of the new model is shown in Figure 5. The threshold value was > 4%. Therefore, using this nomogram to discern patients with ICH who may have death events will surpass “treat-all-patients” and “treat-none” plans.

Figure 5

Decision curve analysis showing the value of the new prediction model. The decision curve analysis shows that the threshold value was > 4%. Therefore, using this nomogram to discern patients with ICH who may have death events will surpass “treat all-patients” and “treat-none” plans

DISCUSSION

Our model was useful for determining the prognoses of patients with ICH. When determining appropriate treatments for individual patients with ICH in the early stages, model predictions are more reliable than relying only on the doctor’s experience, thereby avoiding over treatment or lack of treatment.

In 2001, Hemphill proposed a model to evaluate the 30-day mortality of patients, including ICH volume, GCS score, infratentorial/supratentorial origin, IVH, and age.[2] In addition, other studies added new variables into the model, such as hypertension history, NIHSS scores, and subarachnoid dilatation, which improved the prediction performance of the new model.[15,16] While two previous studies determined the predictors of these early deaths, model use was limited due to excessive predictors. In our study, we collected similar parameters to those of other ICH studies. When we screened using Lasso regression and univariate analysis, GCS scores, hematoma volume, and BSH were prominent, whereas other factors were not. Thus, we used these parameters as indicators to predict ICH as a whole.In our study, model reliability was verified in the development and validation sets and showed good discrimination and calibration performance. DCA also demonstrated the clinical practicability of the model. We affirmed that a GCS < 6, BSH, and hematoma volume were the most powerful predictors of 7-day death in patients with ICH, consistent with previous studies. The GCS score is a reliable evaluation tool and has been used in many prediction models. Qureshi et al. reported that patient mortality was higher (84%) for an initial GCS score of < 10. Kimura et al.[17] also showed that hematoma volume was strongly correlated with 14-day mortality. Hemorrhage location is also a common and important predictor of death from ICH;[7] we observed that BSH may be better than other hematoma indices such as infratentorial hemorrhage.

Compared with previous models, our model was simpler and included GCS scores, hematoma volume, BSH, and common predictors such as combined IVH and age, leukocyte levels, D-dimer, and creatinine levels. Some laboratory indices such as hyperglycemia did not show predictive values for predicting patient death. These reasons may be due to: 1) They may be predictive confounding factors of ICH death and interact with other factors; 2) These predictors only existed in the population in this model, which potentially hinders model accuracy and promotion.

This study used the powerful machine learning method (Lasso), which was used to select the most relevant predictors of death due to ICH, and represented them as a nomogram that accurately and rapidly identified the possibility of individual death. When compared with common stepwise regression analysis, we simultaneously included all predictors in the analysis, and selected the minimum non-zero variable, which made our model as accurate as possible and reduced the possibility of including confounding factors.

However, our study has some limitations. Firstly, it was a single-center retrospective cohort study. Secondly, some patients lacked information, which led to a small sample size. However, the sample size of the development and validation sets was large enough to develop and validate the nomogram. In the future, a larger sample size and a multicenter prospective study will be required to ensure prognostic model accuracy and reliability.

We developed a model for 7-day death prediction from ICH. This model has an important value in predicting the prognosis of patients with ICH and allows for personalized survival prediction.