Modeling and simulation to predict the degree of disability over time in acute ischemic stroke patients.

Disability in patients with acute stroke varies over time, with the prediction of outcomes being critical for proper management. This study aimed to develop a model to predict the cumulative probability of each modified Rankin Scale (mRS) score over time with inclusion of significant covariates. Longitudinal data obtained from 193 patients, 1-24 months after onset of acute ischemic stroke, were included for a modeling analysis using nonlinear mixed-effect modeling (NONMEM). After selecting a model that best described the time course of the probability of different mRS scores, potential covariates were tested. Visual predicted check plots, parameter estimates, and decreases in minimum objective function values were used for model evaluation. The inclusion of disease progression (DP) in the baseline proportional odds cumulative logit model significantly improved the model compared to the baseline model without DP. An inhibitory maximum effect (Emax ) model was determined to be the best DP model for describing the probability of specific mRS scores over time. In the final model, DP was multiplied with the baseline cumulative logit probability with a baseline adjustment. In addition to differences in lesion volume (DLV), the final model included comorbid diabetes mellitus (DM) and baseline National Institutes of Health Stroke Scale (NIHSS) scores on Emax as statistically significant covariates. This study developed a model including DLV, NIHSS score, and comorbid DM for predicting the disability time course in patients with acute ischemic stroke. This model may help to predict disease outcomes and to develop more appropriate management plans for patients with acute stroke.

WHAT IS THE CURRENT KNOWLEDGE ON THE TOPIC?

The prognosis of acute ischemic stroke varies among patients at different time points. Lesion volume is a known important predictor of neurologic deficits in patients with acute ischemic stroke.

WHAT QUESTION DID THIS STUDY ADDRESS?

There is a lack of longitudinal analysis studies regarding the effects of difference in lesion volume (DLV) on the prognosis of patients with acute stroke. This study developed a model to predict the time course of the degree of disability presented by the modified Rankin Scale (mRS) score.

WHAT DOES THIS STUDY ADD TO OUR KNOWLEDGE?

Inclusion of disease progression in the baseline proportional odds cumulative logit model well‐described the probability of each mRS score over time. The DLV, baseline National Institutes of Health Stroke Scale (NIHSS) score, and comorbid diabetes mellitus (DM) were determined to be significant covariates.

HOW MIGHT THIS CHANGE CLINICAL PHARMACOLOGY OR TRANSLATIONAL SCIENCE?

This study suggests that the disability time course in patients with acute ischemic stroke can be predicted by considering DLV, baseline NIHSS score, and comorbid DM.

INTRODUCTION

Stroke, a leading cause of serious long‐term disability, produces immense health and economic burdens globally, with prognosis varying markedly for different patients at different time points. , For example, by 6 months after stroke, about 20% to 30% of patients have died, 20% to 30% are moderately to severely disabled, and 20% to 25% have mild‐to‐moderate disability, with the remainder having no persistent deficits. , Because the early and accurate prediction of outcome is so critical for proper management of patients with acute stroke, , studies have identified important predictors of functional outcome after stroke, including age, a previous stroke, presence of urinary incontinence, and level of disability on admission. ,

Among various factors associated with stroke outcomes, lesion volume reflects the primary pathological condition, with greater extents associated with subsequent neurologic deficits. Thus, lesion volume may serve as an important predictor of the severity of neurological impairments and of functional outcomes after stroke. A previous longitudinal magnetic resonance imaging (MRI) study has shown the dynamic nature of stroke lesion evolution, with 95% of patients demonstrating increases in lesion volume over the first week after the stroke, potentially due to increasing edema and/or further infarction of tissues at the periphery of lesions. Additionally, acute lesion volumes have been shown to correlate well with 30‐day neurological scores on the National Institutes of Health Stroke Scale (NIHSS). In these contexts, brain imaging techniques, including MRI, have become important prognostic tools for the early prediction of stroke outcomes. ,

To quantify disability, assessments of activities of daily living and categorical disability are frequently used. The modified Rankin Scale (mRS) is a commonly used tool for evaluating daily activities in patients with stroke. , There is a lack of longitudinal analysis studies regarding the effects of differences in lesion volume (DLV) on prognosis in patients with acute stroke. A previous study has performed a model‐based quantitative analysis of the relationship between the mRS scores measured at 3 months after stroke onset and imaging characteristics. Based on that prior study's design, we decided to analyze longitudinal data collected over a period of 24 months after acute stroke onset to generate a model that could predict the time course of the degree of disability in these patients.

Therefore, in this study, we aimed to develop a baseline structural model that could describe the cumulative probability of each mRS score over time and to identify covariates that could significantly improve the model. Using this approach, we hoped to inform the development of more effective management plans based on more realistic assessments of the prognosis in patients after acute ischemic stroke.

METHODS

Study population and data collection

Among data prospectively collected from 405 patients, longitudinal data from 193 patients at 1 to 24 months after onset of acute ischemic stroke were used for this study. The study was conducted in accordance with the Declaration of Helsinki, and the study protocol was approved by the Institutional Review Board of the Asan Medical Center. Written informed consent was obtained from each study participant.

Diagnoses of study participants were confirmed using initial MRI results obtained within 24 h after onset of acute ischemic stroke symptoms. Lesion volumes were measured using both the initial and follow‐up (5 ± 1 days) MRI examinations. The degree of disability was measured using mRS scores obtained at 1, 3, 6, 9, 12, 15, 18, 21, and 24 months after onset of acute stroke. Based on the mRS score, the severity of disability was categorized into six grades as follows: no symptoms at all (grade 0), no significant disability despite symptoms (grade 1), slight disability (grade 2), moderate disability (grade 3), moderately severe disability (grade 4), severe disability (grade 5), and death (grade 6). For data analyses, grades 5 and 6 were combined. In addition, covariate analyses assessed a variety of potentially significant variables, including demographic characteristics; stroke severity assessed by NIHSS scores; risk factors for stroke, such as comorbid diabetes mellitus (DM), smoking status, and stroke history, and stroke subtypes classified according to Trial of Org 10172 in Acute Stroke Treatment (TOAST). The DLV was calculated as the follow‐up lesion volume−initial lesion volume. A transformed DLV (TDLV = DLV + 20 cm3) was used in the analyses to ensure that values remained positive.

Model development

A modeling analysis using longitudinal data was performed using nonlinear mixed‐effect modeling (NONMEM version 7.4.0; Icon Development Solutions). A proportional odds cumulative logit model was used, as has been reported previously. Briefly, the cumulative probability P (mRS ≥ m, [m = 1, 2, 3, 4, and 5]) was estimated after logit transformation. The probability P of each mRS score (mRS = m, [m = 0, 1, 2, 3, 4, and 5]) was then calculated as follows: where m = 0, 1, 2, 3, or 4, with 5 and 6 calculated as P (mRS ≥ 5).

Various structural models were evaluated, including an exponential decay model, a combination of exponential decay and exponential increment models, a Weibull model, and a maximum effect (Emax) model. After selecting the model that best described the time course of the probability of different mRS values, several potential covariates were tested to determine if they significantly improved the model, including age, sex, baseline NIHSS scores, TOAST, history of a stroke, and comorbidities like DM, smoking, hyperlipidemia, and hypertension. A covariate was considered statistically significant if the minimum objective function value (MOFV) decreased by >6.63 (p < 0.01) during forward selection and increased by >7.88 (p < 0.005) during backward elimination.

Model evaluation

We evaluated final candidate models using visual predicted check (VPC) plots, standard errors of parameter estimates, and decreases in MOFV. Based on parameter estimates of the final model, simulation of each mRS value over time was performed with 1000 replicates and it was compared with observed probabilities over time. Three‐dimensional plots were drawn to evaluate the relationship between DLV and a significant covariate, such as the NIHSS score, versus the probability of each mRS score. In addition, randomization tests were performed with 1000 datasets with randomly permutated covariates to confirm the statistical significance of selected covariates. For graphical model diagnosis and statistical analyses, R software (version 3.5.3; R Foundation for Statistical Computing) was used.

RESULTS

Characteristics of patients

The mRS scores at 1 to 24 months after onset of acute stroke in 193 patients were included in this analysis. The number (percentage) of patients with DM, hyperlipidemia, stroke history, and/or hypertension were 54 (28.0%), 83 (43.0%), 45 (23.3%), and 137 (71.0%), respectively (Table 1). The median (range) values for age, DLV, and NIHSS score were 61.0 (31.0–86.0) years, 0.6 (−7.0–152.0) cm3, and 4 (0–21), respectively.

TABLE 1

Demographic characteristics of patients

Characteristic	Value
Number of patients	193
Age (years), median (range)	61.0 (31.0–86.0)
Sex, number (%)
Male	73 (37.8%)
Female	120 (62.2%)
Initial lesion volume (cm3), median (range) a	0.9 (0.0–138.9)
Follow‐up lesion volume (cm3), median (range) b	2.2 (0.0–267.0)
Difference in lesion volume (cm3), median (range) c	0.6 (−7.0–152.0)
Baseline NIH Stroke Scale, median (range)	4 (0–21)
TOAST, number (%)
0	44 (22.8)
1	62 (32.1)
2	50 (25.9)
3	37 (19.2)
Diabetes mellitus, number (%)
(+)	54 (28.0)
(−)	139 (72.0)
Smoking, number (%)
(+)	65 (33.7)
(−)	128 (66.3)
Hyperlipidemia, number (%)
(+)	83 (43.0)
(−)	110 (57.0)
Previous stroke, number (%)
(+)	45 (23.3)
(−)	148 (76.7)
Hypertension, number (%)
(+)	137 (71.0)
(−)	56 (29.0)

Baseline median and range values are presented for age and NIH Stroke Scale.

Abbreviations: NIH, National Institutes of Health; TOAST, Trial of Org 10172 in Acute Stroke Treatment stroke subtype.

Lesion volume on magnetic resonance imaging (MRI) within 24 h after symptom onset.

Lesion volume on follow‐up MRI performed 5 ± 1 days after symptom onset.

Follow‐up lesion volume – initial lesion volume.

The observed probability of an mRS score of 0 tended to increase over time from 1 to 24 months after stroke, with rapid increases until 3 months and more gradually increases thereafter. Meanwhile, the observed probability of an mRS score of 1 gradually decreased from 1 to 24 months after stroke. For the remaining mRS scores, observed probabilities demonstrated either little change or a tendency to decrease over time (Figure 1). Based on the exploratory plot and MOFV, an inhibitory Emax model was determined to best reflect disease progression (DP), with inclusion of DP in the baseline proportional odds cumulative logit model significantly improving the model compared to the baseline model without DP inclusion (∆MOFV = −233.444, p < 0.005; Table 2).

FIGURE 1

Exploratory plot of the observed probability of modified Rankin Scale (mRS) scores over time

TABLE 2

Model selection based on the effects of potential covariates on pharmacodynamic parameters

Model description	MOFV	ΔMOFV
Base model without Emax model for disease progression	5098.725	‐
Final base model a	4865.281	‐
Effect on Emax
Hypertension	4852.849	−12.432
Hyperlipidemia	4864.268	−1.013
Initial NIHSS score	4655.101	−210.180
Previous stroke	4827.994	−37.287
Glucose level	4844.548	−20.733
TOAST	4853.829	−11.452
Smoking	4858.614	−6.667
DM	4822.832	−42.449
Age	4865.305	0.024
Sex	4858.073	−7.208
Effect on ET50
TDLV	4869.717	4.436
Hypertension	4841.334	−23.947
Hyperlipidemia	4863.455	−1.826
Initial NIHSS	4805.449	−59.832
Previous stroke	4846.096	−19.185
Glucose level	4836.192	−29.089
TOAST	4873.842	8.561
Smoking	4836.647	−28.634
DM	4826.796	−38.485
Age	4863.14	−2.141
Sex	4864.234	−1.047
Initial NIHSS and DM b	4608.183	−46.918 c
		−214.649 c

Stepwise addition (p = 0.01, ΔMOFV = –6.63) and elimination (p = 0.005, ΔMOFV = –7.88) methods were applied to the selection of covariates.

Abbreviations: DM, diabetes mellitus; Emax, maximum effect; ET50, time to reach 50% of the maximum effect; MOFV, minimum objective function value; NIHSS, National Institutes of Health Stroke Scale; TDLV, transformed difference in lesion volume; TOAST, Trial of Org 10172 in Acute Stroke Treatment; ΔMOFV, change in MOFV relative to final base model.

Final base model was selected as a model containing TDLV as a covariate affecting maximum change of disease progression in the form of Emax model over time.

Final model selected based on covariate analysis.

The difference was –46.918 in the objective function values between the model incorporating initial NIHSS scores and the final model, with –214.649 as the difference between the model with DM and the final model.

Because the primary purpose of this study was to identify the effect of DLV on stroke progression, we opted to include DLV on the Emax and/or time to reach 50% of the maximum effect (ET50; i.e., TDLV at 50% of Emax) of the baseline model. Based on covariate screening results, we selected the model containing TDLV as a covariate affecting the maximum change in DP in the form of the Emax model over time as a final baseline structural model. We also evaluated the effects of other covariates, including age, sex, and other comorbidities, on the improvement of the model. Consequently, baseline NIHSS scores and comorbid DM were identified as statistically significant covariates. Taken together, the following equation was used to describe the progression of stroke using an Emax model: where, and

All estimates of parameters constituting the final model are presented in Table 3.

TABLE 3

Parameter estimates for the final model

Parameter (unit)	Estimate	RSE (%)	95% CI
SHIFT	6.27	16.4	4.25 to 8.29
β1	1.27	11.7	0.98 to 1.56
β2	−0.868	6.5	−0.98 to −0.76
β3	−1.44	4.9	−1.58 to −1.3
β4	−1.05	7.2	−1.2 to −0.9
β5	−1.45	9.7	−1.72 to −1.18
Emax, DM(‐)	0.445	19.8	0.27 to 0.62
Emax, DM(+)	0.322	22.3	0.18 to 0.46
ΘEmax‐TDLV	−0.102	34.2	−0.17 to −0.03
ΘEmax‐NIHSS	−0.149	17.0	−0.2 to −0.1
ET50	0.713	58.3	−0.1 to 1.53

The final model is presented as follows: where, and (TDLV, transformed difference in lesion volume in cm3 [follow‐up lesion volume – initial lesion volume +20 cm3]).

Abbreviations: CI, confidence interval; DM, diabetes mellitus; Emax, DM(−), maximum effect in patients without diabetes mellitus, which is 0 in patients with DM; Emax, DM(+), Emax in patients with DM, which is 0 in patients without DM; ET50, time to reach 50% of the maximum effect; NIHSS, National Institutes of Health Stroke Scale; RSE, relative standard error (standard error divided by the parameter estimate); SHIFT is the population mean for the degree of y‐axis parallel translation; transformed difference in lesion volume in cm (follow‐up lesion volume – initial lesion volume +20 cm3) at 50% of Emax; TDLV, transformed difference in lesion volume; βk is the population mean baseline logit for the different mRS scores (βk, k = 1, 2, 3, 4, 5); ΘEmax‐NIHSS, covariate effect of the National Institute of Health Stroke Scale score on Emax; ΘEmax‐TDLV, covariate effect of TDLV on Emax.

Model validation

VPC plots demonstrated that the developed model was able to successfully predict observed mRS scores over time, although discrepancies existed at the inflection point where the mRS score was 2 (Figure 2). In randomization tests, the baseline NIHSS score and presence of DM showed statistical significance (p < 0.005), with the MOFV of the final model being lower than the 2.5 percentile of the MOFV distribution obtained from 1000 datasets per covariate in which the covariate was randomly permuted (Figure 3). The probability of each mRS score was simulated based on the DLV and time, whereas the NIHSS value was fixed at the median value. The results of three‐dimensional plots demonstrated similar patterns both in the presence and absence of DM. The probability of an mRS score of 0 decreased with increasing DLV, whereas the probability of an mRS score of 4 increased with increasing DLV. On the other hand, the probability of an mRS score of either 5 or 6 tended to decrease with increasing DLV. These trends persisted when the probability of each mRS score was simulated based on the NIHSS score and time, with the DLV fixed at the median value (Figure S1).

FIGURE 2

Visual predicted check of the observed versus predicted probabilities of modified Rankin Scale (mRS) scores over time. Black dots indicate observed probability, red solid lines indicate model‐predicted median probabilities, and yellow areas indicate the 95% prediction interval of probability

FIGURE 3

Randomization tests for statistical significance of covariates, including the National Institutes of Health Stroke Scale (NIHSS) score and comorbid diabetes mellitus. Dotted lines indicate minimum objective function values (MOFVs) corresponding to the 2.5 percentile of MOFV distributions, with solid lines indicating MOFVs of the final model

DISCUSSION

In this study, we predicted the probability of the degree of disability over time in patients with acute stroke using a model that we developed, which accounted for significant covariates, including DLV, the presence of comorbid DM, and baseline NIHSS scores. We evaluated the developed model using several strategies, including VPC plots, which indicated good predictive performance. The results of this study are meaningful in that we have quantitatively predicted the probability of each mRS score over time and identified significant factors affecting the prognosis in patients with acute stroke.

During construction of the baseline structural model, inclusion of DP in the baseline proportional odds cumulative logit model significantly improved the model. To explore the time course of DP, observed probabilities of each mRS score over time were illustrated. Consequently, different temporal patterns for the probability of different scores were identified. For example, the probability of an mRS score of 0 tended to increase over time, with a more rapid rate of increase until 3 months. In contrast, the probability of an mRS score of 1 gradually decreased during the observed time period. For other mRS scores, the observed probabilities showed either little change or a tendency to decrease over time.

Based on the exploratory plot and other methods used to evaluate the model, including MOFVs, the inhibitory Emax model was determined to be the best DP model to describe the probability of specific mRS scores at different time points. In the final model, we opted to multiply DP with the baseline cumulative logit probability rather than adding it because DP was affected by the baseline cumulative logit probability. After visualization of the observed cumulative logit probabilities over time, we found different tendencies of cumulative logit probabilities over time according to the mRS scores. To describe these empirically observed cumulative logit probabilities over time, the parameter SHIFT, indicating the degree of translation parallel to the y‐axis was applied.

The baseline structural model was significantly improved after the inclusion of DLV as a covariate of Emax. Lesion volume has previously been identified as a significant prognostic factor in patients with stroke. , Furthermore, the early infarct growth in the initial 5 days after stroke onset can be used as a marker of early treatment effects in acute ischemic stroke. In addition, other factors, such as age, glucose levels, and cerebral stroke history, have also been reported to have prognostic implications in patients with acute ischemic stroke. , For example, a previous study reported that risks of early neurological deterioration and poor short‐term outcomes (30‐day mRS score 2–6) were significantly higher in a group of patients with DM than in a group of patients with normal glucose tolerance. Another study reported that baseline NIHSS scores strongly predicted outcomes after stroke, with each additional point on the NIHSS decreasing the likelihood of excellent outcomes by 24% at 7 days and by 17% at 3 months. Similarly, our model demonstrated that comorbid DM and baseline NIHSS scores, in addition to DLV on Emax, had statistically significant effects. Furthermore, in randomization tests with 1000 datasets per covariate, these two covariates also demonstrated statistical significance with a p value of 0.005.

The parameter estimate for Emax in patients without DM (Emax, DM(‐)) was larger than the Emax in patients with DM (Emax, DM(+)). Based on equations used in our model, this finding implies that the probability of having a good outcome (mRS score = 0) is higher in patients without DM than in patients with DM. Additionally, parameter estimates of the covariate effects of TDLV on Emax (ΘEmax‐TDLV) and NIHSS score on Emax (ΘEmax‐NIHSS) were both negative values, suggesting that higher DLVs and baseline NIHSS scores are associated with smaller changes in mRS scores over time.

VPC plots based on our model demonstrated generally good predictive performance, although discrepancies existed at the inflection point for an mRS score of 2. Based on VPC plots, the predicted probability of an mRS score of 0, which signifies an absence of symptoms, rapidly increased after stroke, particularly until 3 months. Because the estimated value of ET50 was 0.713, it can be assumed that radical changes in the degree of disability are more likely to occur within 2 months after onset of acute stroke.

The simulation results illustrated by three‐dimensional plots suggested that the probability of an mRS score of 0, implying excellent clinical outcome decreased as the DLV increased, whereas the probability of an mRS score of 4, implying poor clinical outcome, increased as the DLV increased. Similar patterns were seen in the presence and absence of DM. These findings support a previous study demonstrating that lesion growth after stroke is significantly associated with functional outcomes. These trends persisted when the probability of each mRS score was simulated based on the NIHSS score and time, suggesting that the probability of having an excellent clinical outcome decreases the more impaired a stroke patient is. The probability of an mRS score of 5 or 6, however, tended to decrease as DLV or NIHSS score increased. This finding may be explained in two ways. First, in patients with severe disease (mRS scores ≥ 5), treatment strategies are likely more aggressive, and therefore, the DLV or NIHSS score may not be as strongly associated with the degree of disability. Second, the number of patients with mRS scores of greater than or equal to 5 was lower than the number of patients with mRS scores of 0 to 4 at each time point. Because pooled data from all mRS score groups were used to develop the model, the predictability of mRS scores greater than or equal to 5 may be lower than that of mRS scores 0–4.

The model constructed in this study was based on data obtained from patients over 24 months after onset of acute stroke and likely has important clinical value. For example, this model may increase understanding of the general time course of the degree of disability as determined by mRS scores. Furthermore, this model may allow us to predict the probability of different mRS scores at different time points if some clinical data are available, including DLV, presence of DM, and baseline NIHSS scores. This study has some limitations. First, to apply these modeling results in different clinical situations, it would be better to conduct external validation, the most strict validation method. However, we had to use all available data for the model construction, so there were no additional data for the external validation. Further studies, including external validation, will be helpful to increase the reliability of these modeling results. Second, our model shows different predictability according to time or each mRS score although it generally well captured the trend in the probability over time, particularly for the mRS score of 0, indicating an excellent outcome. Third, uncertainties around the parameter estimate of ET50, as evident by the wide range of 95% confidence interval, which included zero, limited the interpretation of the results. Last, these study results are based on longitudinal data from 193 patients, which is relatively small in number. Further studies with more patients in each category of clinical outcome presented by mRS score are warranted to confirm the results of this study.

CONCLUSIONS

In conclusion, we developed a model to explore the effect of DLV on the prognosis of patients with acute stroke. In addition, we identified significant covariates, including baseline NIHSS scores and comorbid DM, that are also likely to affect the prognosis in patients with acute ischemic stroke. Using this model, we performed a simulation to predict the time course of the degree of disability. This model‐based analysis may help us to prepare appropriate management plans based on more accurate prognostic assessments in patients with acute stroke.

CONFLICTS OF INTEREST

The authors declared no competing interests for this work.

AUTHOR CONTRIBUTIONS

S.‐I.P., D.‐W.K., and H.‐S.L. wrote the manuscript. D.‐W.K., and H.‐S.L. designed the research. S.‐I.P., D.‐W.K., and H.‐S.L. performed the research. S.‐I.P. and H.‐S.L. analyzed the data.

Fig S1

Click here for additional data file.