A Dynamic Nomogram to Predict the 3-Month Unfavorable Outcome of Patients with Acute Ischemic Stroke.

Purpose: Despite receiving standard-of-care treatments, a significant proportion of patients with acute ischemic stroke (AIS) are left with long-term functional impairment. Therefore, an easy-to-use tool for predicting of unfavorable outcome following AIS plays an important role in clinical practice. This study was aimed to develop a dynamic nomogram to predict the 3-month unfavorable outcome for AIS patients.
Methods: This was a prospective observational study conducted in consecutive patients with AIS admitted to our stroke center between September 2019 and June 2020. Baseline demographic, clinical, and laboratory information were obtained. The primary outcome was evaluated with modified Rankin Scale (mRS) scores at 3 months. Least absolute shrinkage and selection operator regression was used to select the optimal predictive factors. Multiple logistics regression was performed to establish the nomogram. Decision curve analysis (DCA) was applied to assess the clinical utility of the nomogram. The calibration and discrimination property of the nomogram was validated by calibration plots and concordance index.
Results: A total of 93 eligible patients were enrolled: 28 (30.1%) patients had unfavorable outcome (mRS >2). Glycosylated hemoglobin (OR, 1.541; 95% CI, 1.051-2.261), the Alberta Stroke Program Early Computed Tomography Score (ASPECTS) (OR, 0.635; 95% CI, 0.463-0.871), and National Institute of Health Stroke Scale (NIHSS) (OR 1.484; 95% CI, 1.155-1.907) were significant predictors of the poor outcome of patients with AIS and included into the nomogram model. The nomogram showed good calibration and discrimination. C-index was 0.891 (95% CI, 0.854-0.928). DCA confirmed the clinical usefulness of the model. The dynamic nomogram can be obtained at the website: https://odywong.shinyapps.io/DBT_21/.
Conclusion: The dynamic nomogram, comprised of glycosylated hemoglobin, ASPECTS, and NIHSS score at day 14, may be able to predict the 3-month unfavorable outcome for AIS patients.

Introduction

Ischemic stroke constitutes a dominant cause of disability and mortality globally, and imposes an enormous economic burden especially on developing countries.1,2 Considerable attempts have been made to improve the prognosis of patients with acute ischemic stroke (AIS). In particular, vascular recanalization therapy, such as intravenous thrombolysis (IVT) and endovascular thrombectomy (EVT), has been proven effective in randomized controlled trials and widely adopted in clinical practice.3–7 However, limited benefit of IVT is likely related to the narrow therapeutic time window of 4.5h after stroke and the low recanalization rate of around 20%.8 EVT is mainly applied to large-vessels occlusion stroke patients. And even after successful EVT, more than half of them still die or suffer from a disability.9 Recently, a new therapy concept of multiphase adjuvant neuroprotection was proposed to improve neurologic outcome in patients with AIS after EVT. Nevertheless, current neuroprotective strategies are still limited and need more clinical researches to confirm their efficacy and safety.10,11

Despite receiving standard-of-care treatments, most survivors are still left with long-term disabilities secondary to treatment. Therefore, early prediction of post-stroke functional outcome is of great significance for decision-making in stroke management and aiding patients and their families set realistic expectations. In recent years, based on clinical risk factors confirmed to influence the outcome of patients with ischemic stroke, including age, stroke severity, the pre-stroke functional status, various predictive models have been developed to estimate the prognosis after cerebral infarction in a scoring manner, such as totaled health risks in vascular events (THRIVE) score, stroke prognostication using age and National Institutes of Health Stroke Scale (SPAN)-100, acute stroke registry and analysis of Lausanne (ASTRAL) score and CHA2DS2-VASc score.12–15 The modified Rankin Scale (mRS) scores, as the most frequently used primary outcome measure in clinical stroke trials for its simplicity and ease of interpretation, were commonly applied to evaluate functional outcome of patients with AIS in these scoring systems.16 In addition, a number of blood-based and neuroimaging markers have also been developed to improve the prognostic accuracy, such as copeptin and collateral circulation.17,18 However, a simple and easy-to-use tool for the prognostic prediction of AIS is still lacking.

The advances in computational methods, including machine learning, bring a new avenue for more reliable and accurate prognostic predictions. A nomogram, as a graphical display instrument, is convenient to incorporate all prognostic factors to calculate the precise probability of interest event for each patient, and has been widely applied to prognostic prediction in various diseases, including ischemic stroke. This study was designed to construct a nomogram model to predict the 3-month unfavorable outcome of AIS patients. We conducted a comprehensive evaluation of demographic, clinical, radiological and laboratory parameters that may influence the unfavorable outcome of AIS patients. What is more, the static nomogram was further transformed into a web-based dynamic nomogram owning a user-friendly interface, which became more intelligent and convenient to use.

Materials and Methods

Patient Recruitment

We performed a single-center, prospective observational clinical study of patients with first-ever AIS to create a dynamic nomogram for predicting the individual risk of poor outcome. Between September 2019 and June 2020, we consecutively recruited patients hospitalized for AIS at our stroke center. All enrolled patients were evaluated at baseline and follow-up and totally 93 subjects were involved in the final analyses in this study. Inclusion criteria included: (1) age ≥18 years; (2) patients with focal neurological symptoms related to AIS diagnosed by noncontrast computed tomography (NCCT) or magnetic resonance diffusion-weighted imaging; (3) patients who underwent pre-treatment baseline NCCT and computed tomographic angiography (CTA) within 6 h after onset of symptoms; (4) pre-stroke mRS scores 0–1. Exclusion criteria included: (1) current gestation or lactation; (2) cerebral hemorrhage and brain tumors; (3) severe space-occupying effect (midline shift or even brain herniation).

Data Collection

For all eligible patients, their baseline demographic, clinical, and laboratory information was acquired, including: age, sex, Body Mass Index (BMI), vascular risk factors, medical history, serum biochemistry data, admission National Institute of Health Stroke Scale (NIHSS), NIHSS at day 14, the Alberta Stroke Program Early Computed Tomography Score (ASPECTS), occlusion site (anterior circulation or posterior circulation), types of reperfusion therapy and Trial of Org 10172 in acute stroke treatment (TOAST) classification. BMI was defined as body weight (kilograms) divided by height (meters) squared. All biochemical tests were performed in the department of laboratory medicine of our hospital, which required all participants to remain fasted for at least 10 hours prior to collecting blood at 7:00 am. ASPECTS or posterior circulation (PC) ASPECTS was visually evaluated on the baseline NCCT by deducting one or two points from the total score of 10 for every area with early ischemic signs. One experienced neurologist with 3 years of experience in clinical evaluation and research took on the task, who was blind to all clinical data.

Sample Size

An adequate sample was necessary to ensure predictive power of risk prediction models. According to Harrell’s guidelines, when primary outcome is a binary variable, the minimum sample size of the two response levels should be 10 times greater than the number of predictors. We totally selected 3 predictors through least absolute shrinkage and selection operator (LASSO) in our study, so a sample size of at least 30 participants was required for positive events.19

Outcome

The primary outcome was evaluated with mRS scores at 3 months through telephone interview by a professional neurologist at our stroke center, who was blinded to baseline clinical and imaging information. An unfavorable outcome (UFO) was defined as a mRS >2.

Statistical Analysis

R software (R version 3.5.1, R Foundation for Statistical Computing, Vienna, Austria) was used for all statistical analyses. Patients were dichotomized into two groups based on good prognosis (mRS ≤2) and bad prognosis (mRS >2). We displayed categorical variables as frequency (percentage) and continuous variables as means ± standard deviations (SD) or medians (interquartile ranges, IQR), as appropriate. Differences in baseline characteristics were compared between two groups using Pearson chi-square tests for categorical variables and Kruskal–Wallis tests for continuous variables. P < 0.05 was considered statistically significant with two-sided test.

After descriptive statistics, the following procedures were carried out to construct the nomogram. First, we screened the significant variables related to poor prognosis of AIS patients with the use of LASSO regression, which serves as a dimensionality reduction approach as well as takes into account double-standard error by means of a penalty function. Second, we conducted an analysis of missing data by multiple imputation using chained equations (MICE) generated from the “MICE” package in the statistical program “R” version 3.5.2.20 Fifty imputed datasets were created for use in subsequent regression analyses to determine the factors affecting 3-month unfavorable outcome of AIS patients. The pooled effect estimates were presented as ORs and their 95% CIs. Third, a 3-month unfavorable prognostic nomogram model based on identified predictors was established using the R package “rms” and then transformed into a dynamic nomogram on web page by package “DynNom”. Finally, we plotted a calibration curve to evaluate the calibration of the nomogram by comparing the predicted unfavorable outcome with actual outcome. The clinical usefulness of our nomogram was further assessed through decision curve analysis (DCA), which quantified the net benefit across a range of threshold probabilities.

Results

Patient Characteristics

In total, 93 patients who met the inclusion criteria were entered into the study and evaluated (missing values are depicted in Figure 1). There are 28 (30.1%) patients in the UFO group. Baseline data of the two groups are compared in Tables 1–3. Significant differences between two cohorts are observed in age, glycosylated hemoglobin (GHb), Essen score, admission NIHSS, NIHSS at day 14 and ASPECTS. The patients in UFO cohort were older, and had higher levels of GHb (median 6.8% vs 5.9%; P = 0.004), Essen score (median 3.0 vs 2.0; P = 0.005), admission NIHSS (6.0 vs 3.0; P = 0.004), and NIHSS at day 14 (4.0 vs 1.0; P < 0.001), and had lower ASPECTS (9.0 vs 10.0; P < 0.001).

Figure 1

The aggregation plot of missing values. The first graph describes percentages of missing data, and the second graph shows percentages of different missing combinations. There were no missing values in 79% of participants.

Table 1

Baseline Characteristics of the Two Study Groups

Parameters	mRS >2	mRS ≤2	P value
	n = 28	n = 65
Demographics
Female, n (%)	10 (35.7)	14 (21.5)	0.154
Age, years, mean (SD)	68.5 (12.7)	59.2 (13.7)	0.003
Smoking, n (%)	13 (46.4)	35 (53.8)	0.514
Drinking, n (%)	8 (28.6)	18 (27.7)	0.931
BMI, mean (SD)	24.5 (3.5)	24.0 (3.7)	0.490
Baseline SBP, mmHg, mean (SD)	154.7 (24.4)	152.5 (23.9)	0.683
Baseline DBP, mmHg, mean (SD)	89.9 (15.2)	92.7 (20.9)	0.523
Medical history
Hypertension, n (%)	22 (78.6)	41 (63.1)	0.145
Diabetes mellitus, n (%)	10 (35.7)	12 (18.5)	0.074
Hyperlipidemia, n (%)	11 (39.3)	24 (36.9)	0.830
Atrial fibrillation, n (%)	6 (21.4)	5 (7.7)	0.061
Homocysteinemia, n (%)	2 (7.1)	9 (14.3)	0.337

Abbreviations: mRS, modified Rankin Scale; SD, standard deviation; BMI, body mass index; SBP, systolic blood pressure; DBP, diastolic blood pressure.

Table 2

Comparison of the Serum Biochemistry Data of the Two Study Groups

Parameters	mRS >2	mRS ≤2	P value
	n = 28	n = 65
RDW-SD, mean (SD)	42.5 (4.4)	42.1 (3.3)	0.711
RDW-CV, mean (SD)	13.4 (1.7)	13.2 (1.6)	0.635
PDW, mean (SD)	12.9 (2.8)	11.7 (2.1)	0.057
PCT, mean (SD)	0.23 (0.06)	0.23 (0.05)	0.938
CRP, mg/L, median (IQR)	1.6 (0.2–4.1)	0.9 (0.2–4.0)	0.485
High-density lipoprotein, mmol/L, median (IQR)	1.0 (0.9–1.2)	1.0 (0.9–1.2)	0.410
Total cholesterol, mmol/L, median (IQR)	4.1 (3.3–5.3)	4.7 (4.1–5.5)	0.109
Triglycerides, mmol/L, median (IQR)	1.1 (0.8–2.0)	1.3 (0.8–1.8)	0.929
Low-density lipoprotein, mmol/L, median (IQR)	2.5 (1.9–3.4)	3.1 (2.4–3.7)	0.054
Uric acid, μmol/L, median (IQR)	368.5 (283.0–487.5)	375.0 (304.5–475.5)	0.441
Glycated hemoglobin, median (IQR)	6.8 (5.8–9.7)	5.9 (5.5–6.4)	0.004
D-dimer, mg/L, median (IQR)	0.5 (0.3–1.5)	0.4 (0.2–0.9)	0.156
Fibrinogen, g/L, median (IQR)	3.4 (3.0–3.9)	3.2 (2.7–3.9)	0.490
INR, median (IQR)	1.0 (0.9–1.1)	1.0 (0.9–1.0)	0.497

Abbreviations: mRS, modified Rankin Scale; SD, standard deviation; RDW-SD, red blood cell distribution width-standard deviation; RDW-CV, red blood cell distribution width coefficient of variation; PDW, platelet distribution width; PCT, procalcitonin; CRP, c-reactive protein; INR, international normalized ratio.

Variable Selection and Prognostic Model Establishment

Three valuable variables incorporating GHb, ASPECTS, and NIHSS at day 14 were initially selected by LASSO Cox-regression (Figure 2). With the help of multiple imputation by chained equation, we generated 50 imputed datasets to handle the missing data. Then, we further verified the relationship between the three variables and poor prognosis by applying multiple logistics regression analysis. The pooled results with R package (mice, with, pool) were overall analyzed as following: GHb (OR, 1.541; 95% CI, 1.051–2.261), ASPECTS (OR, 0.635; 95% CI, 0.463–0.871), and NIHSS at day 14 (OR 1.484; 95% CI, 1.155–1.907).

Figure 2

Selection of the optimal prognostic factors by LASSO regression analysis. (A) LASSO coefficient profiles of potential predictors. (B) Screening of the optimal penalization coefficient in the LASSO regression.

Model Verification and Clinical Utility

To evaluate the clinical validity of the model, DCA was employed (Figure 3A and B). We can see a wide and practical range of threshold probabilities in validation cohort, which meant the model performed well to predict the poor outcome of AIS patients in clinic. As shown in Figure 3C, the calibration plot, which compared the predicted effective rate and observed effective rate, revealed accurate predictive ability of the model. What is more, the C-index indicated good discriminative ability with a value of 0.891 (95% CI, 0.854–0.928).

Figure 3

Clinical utility and model verification. (A) Decision curve analysis of the nomogram. The y-axis and x-axis indicate the net benefit and the threshold probability, separately. (B) Clinical impact curve. (C) Calibration plot of the nomogram with a 40 repetition bootstrap and mean absolute error of 0.068. The 45° line in the plot represents an ideal nomogram. The dotted line represents predictive capabilities of the current nomogram, whereas the solid line corrects for any bias in the nomogram.

Nomogram Development

The nomogram was established based on the three significant predictive factors from multivariate logistic regression analysis. Total scores were calculated by adding the corresponding score of each variable to estimate the probability of poor prognosis (Figure 4A). Higher total scores in the nomogram indicated a higher possibility of poor outcome, while lower total scores indicated a lower possibility of poor outcome. In addition to the ordinary nomogram, we also built a dynamic nomogram to facilitate the use for clinicians with an intuitive web-based interface (Figure 4B). People can easily input values of the three predictors followed by a click of the “Predict” button, then the 3-month probability of poor prognosis and 95% confidence interval are exported on the right side of the interface. The probabilities from the website (Dynamic Nomograms: ) are the same as the Nomograms.

Figure 4

The nomogram incorporates GHb, ASPECTS, NIHSS score at day 14 to predict the probability of 3-month unfavorable outcome in patients with AIS. (A) The traditional nomogram. Points were assigned to each prognostic factor by drawing a vertical line from the certain values to the “points” line. The total points were calculated as the sum of them. Corresponding probability of unfavorable outcome can be easily obtained according to the final total points (significance codes: 0 ‘***’ .001 ‘**’ .01 ‘*’ .05 ‘.’ .1 ‘ ’ 1). (B) The intuitive interface of online dynamic nomogram ().

Discussion

This study established a dynamic nomogram to predict the 3-month unfavorable outcome for AIS patients based on GHb, ASPECTS, and NIHSS score at day 14. It is well known that hyperglycemia is correlated with poor outcomes and increased mortality in AIS patients.21–24 The underlying mechanisms include tissue acidosis, blood–brain barrier disruption, reduced fibrinolytic activity and increased production of reactive oxygen species.25 Elevated blood glucose always occurs among patients with poorly controlled diabetes; however, it can also be observed in nondiabetic patients with acute ischemic stroke. The latter is called stress-induced hyperglycemia (SIH) mainly resulting from the activation of the hypothalamic–pituitary–adrenal axis.26 In contrast to SIH, GHb (the production of glucose binding to hemoglobin) reflects the average blood glucose levels over the last two to three months before the onset of ischemic stroke. Our results are consistent with previous prospective studies identifying high GHb to be associated with three-month worse prognosis in AIS patients and as a short-term predictor of prognosis, independent of the transient elevated blood glucose levels induced by SIH.27,28 Therefore, strict pre-stroke glycemic control is important for decreasing the risk of adverse post-stroke outcomes.

ASPECTS is a widely used screening tool to analyze early ischemic changes in AIS patients on NCCT,29 which plays a vital role not only for guiding clinical practice but also for research purposes. In the original trial, ASPECTS value of 7 or below was a cutoff to differentiate whether patients achieved functional dependence after thrombolytic therapy within 3 hours of symptom onset. Subsequent studies recognized baseline ASPECTS ≥6 as a key selection criterion of patients who would benefit from EVT,30,31 which is recommended in updated guidelines of the American Heart Association on acute stroke management.32 In line with previous reports, ASPECTS on baseline NCCT in our nomograms is a significant predictor of unfavorable outcome.33–35

NIHSS score is widely applied to measure the severity and predict the unfavorable outcome of patients with ischemic stroke.36,37 High NIHSS score is associated with a larger infarct size and cerebral edema,38,39 which ultimately lead to poor outcomes. A significant number of predictive models incorporate NIHSS score as a predictor,40 and our model is no exception. Unlike most studies, unfavorable outcome had no significant correlation with the baseline NIHSS score in the present study, but with NIHSS score at day 14. This may be related to the differences in sample size and population characteristics. Our results confirm that baseline severity of ischemic stroke cannot determine actual prognosis of patients.

In our study, acute treatments, such as IVT and EVT, were not found to be associated with the 3-month outcome of patients with AIS. This may be because proportions of the therapeutic modalities (IVT and EVT) were not statistically significantly different in the 2 groups (favorable and unfavorable outcomes) (Table 3).

Table 3

Therapeutic Characteristics of the Two Study Groups

Parameters	mRS >2	mRS ≤2	P value
	n = 28	n = 65
ASPECTS, median (IQR)	9.0 (7.0–9.0)	10.0 (9.0–10.0)	<0.001
Admission NIHSS, median (IQR)	6.0 (4.0–9.5)	3.0 (2.0–5.0)	0.004
NIHSS at day 14, median (IQR)	4.0 (2.0–8.0)	1.0 (0.0–2.0)	<0.001
Occlusion site, n (%)			1.000
Anterior circulation	21 (75.0)	48 (75.0)	–
Posterior circulation	7 (25.0)	16 (25.0)	–
IV thrombolysis, n (%)	26 (92.9)	53 (81.5)	0.164
Endovascular therapy, n (%)	9 (32.1)	13 (20.0)	0.209
TOAST classification, n (%)			0.056
Atherosclerotic	17 (60.7)	25 (39.1)	–
Others	11 (39.3)	39 (60.9)	–

Abbreviations: mRS, modified Rankin Scale; ASPECTS, Alberta Stroke Program Early CT Score; NIHSS, National Institutes of Health Stroke Scale; IV thrombolysis, intravenous thrombolysis; TOAST, Trial of Org 10172 in acute stroke treatment.

Our dynamic nomogram, which includes the three easy-to-obtain variables mentioned above, can calculate the probability of an individual’s unfavorable outcome. For example, the probability of unfavorable 3-month outcome of a patient with a GHb of 7.0%, baseline ASPECTS score of 6 and NHISS score at day 14 of 4 is 76.4%. Prevention is better than treatment, and the dynamic nomogram helps clinicians timely identify high-risk patients with AIS and provides an opportunity for early intervention and close monitoring. Also, it can be used for risk group stratification in clinical researches.

There are some strengths in our dynamic nomogram compared with the current prognostic scores to predict unfavorable outcomes of patients with AIS. In terms of processing variables, some prognostic scores categorize the discrete and continuous predictors like age, NIHSS score and blood glucose level into two or more groups based on artificial cut-off values, which reduces the accuracy of models due to neglect of within-category information. However, our nomogram fully analyzes the effect of specific value of a certain variable, and therefore may serve as a more precisely individualized predictive tool for prognosis. Moreover, because of the result visualization, it is more intuitionistic and convenient in practical application.

Meanwhile, there are several limitations in this study. First, our sample size was relatively limited, and more samples might be needed to increase the accuracy of the research results. Second, this was a single-center study, which led to selection bias. External validation in other stroke centers is necessary due to the differences of race, patient’s wealth, hospital type and ischemic stroke treatments. Third, risk factors excluded in our study, such as collateral circulation and different drug treatments (statins,41 antiplatelets42 and anticoagulants43), are likely to be important for predicting unfavorable outcomes, which may damper the rigor and scientific nature of the prediction model and limit its extensive application in clinic. Despite these limitations, our nomogram based on easily available variables may be widely used.

In summary, the dynamic nomogram developed in this study, comprised of GHb, ASPECTS, and NIHSS score at day 14, may be able to predict the 3-month unfavorable prognosis in AIS patients. Further external validations are needed to evaluate the universality of the model.

Conclusion

The dynamic nomogram, comprised of glycosylated hemoglobin, ASPECTS, and NIHSS score at day 14, may be able to predict the 3-month unfavorable outcome for acute ischemic stroke patients.