The distribution of the modified Rankin scale scores change according to eligibility criteria in acute ischemic stroke trials: A consideration for sample size calculations when using ordinal regression analysis.

BACKGROUND: Distribution shifts of the modified Rankin scale (mRs) is used as outcome measure in acute ischemic stroke (AIS) randomized controlled trials (RCT). Distribution across strata of mRs is relevant for sample size calculations and may be affected by eligibility criteria. AIM: We aimed to assess the distribution of mRs scores across its different strata in AIS according to usual eligibility criteria.
METHODS: We computed follow-up mRs strata distribution between an unselected cohort and samples with (a) time from symptom onset < 6 h (b) National Institutes of Health Stroke Scale (NIHSS) scores > 3 and < 25, and (c) both criteria combined. We compared distributions with the Mann-Whitney U Test and calculated sample sizes for each distribution.
RESULTS: We included 5849 AIS patients. The unselected sample had a non-normal distribution with a median of 2. All selection criteria yielded significantly different distributions of mRs (p = 004, 0.02 and 0.02 respectively). This resulted in a significant variation in the calculated sample size when applying different selection criteria, with smaller numbers when RCT selection criteria are used (3616 versus 1553).
CONCLUSIONS: The use of usual RCT eligibility criteria result in significant differences in mRs distribution and smaller sample sizes compared to unselected AIS samples.

Introduction

The modified Rankin scale (mRs) is an ordinal disability score of 7 categories (0 = no symptoms to 6 = dead) and is currently the preferred method of evaluating outcome in patients with acute ischemic stroke (AIS) after intervention trials, particularly if the effect size is modest and distributed across all strata [1], [2], [3]. Thus, sample size calculations are increasingly determined by the distribution of the scores in the mRs at 90 days after symptom onset in modern AIS randomized clinical trials (RCT) [4], [5], [6].

The data are analyzed using the t-test for difference of means or the Mann-Whitney U test or its variants - the Robust Rank Test and the Ordinal Logistic Regression (OLR) [7], [8]. The latter also known as Whitehead's Odds Ratio derives from the proportional odds model of McCullagh [9], and has been shown to produce the lowest sample sizes [10]. The analysis assumes an equal chance of being in any strata and requires that at least the proportion of subjects in each scale category in one of the groups be specified, leading to a same odds ratio independently of the chosen partitions [11]. Thus the distribution across strata is relevant in deciding which statistical method is most suitable for sample size calculations when analyzing mRs as an ordinal scale [12].

Acute ischemic stroke RCTs eligibility criteria may impact the outcome distribution across strata and have the potential of being significantly different when compared to unselected stroke patients and between different trials. This could affect the sample sizes to an unknown extent.

Our aim was to compare the distribution of follow-up mRs scores across its different strata in patients with acute ischemic stroke (AIS) from an unselected consecutive cohort with 3 groups of patients meeting usual RCTs eligibility criteria. We hypothesized that variations in the distribution of mRs would significantly influence the sample sizes, when using OLR.

Methods

In this retrospective cohort study, patients were selected from two prospective hospital stroke registries and one population-based registry (Joinville): two in Brazil (Porto Alegre and Joinville) and one in Chile (Santiago).

For the unselected cohort, we included all adult AIS patients (i.e. ≥ 18 years old) with clinical and imaging diagnosis of AIS, data available on NIHSS at admission, time from symptom onset to admission, mRs at follow up and informed consent given. Patients with Transient Ischemic Attacks and with incomplete data were excluded.

The following eligibility criteria have been used in many acute stroke RCTs and were applied to the cohort to produce 3 samples: (a) time from symptom onset < 6 h (b) National Institutes of Health Stroke Scale (NIHSS) scores > 3 and < 25, and (c) both criteria combined.

The corresponding institutional review board and ethics committee approved the 3 registries were these data were derived from: The RECCA registry in Clínica Alemana, the Joinville Stroke Registry and Hospital das Clínicas, Porto Alegre Stroke Registry.

Analysis

We computed skewness, kurtosis, median values and used Mann-Whitney U Test to compare mRs strata distribution for each set of criteria compared to the unselected sample. We then applied the method proposed by S. Simon to calculate sample sizes for an acute trial using the distribution of mRs according to the different sets of eligibility criteria [13]. We grouped categories mRs 5 and 6 into one, as is usual in the analysis of AIS RCTs. All sample sizes were calculated with a modest effect size of 2% between categories of mRs in each group, alfa was set at 0.05 and power at 80% and all calculation were 2 tailed.

Results

The total unselected cohort consisted of 5849 consecutive patients, the demographic and clinical characteristics of which are described for each individual sample in Table 1.

Table 1

Demographic and clinical characteristics of sample cohorts.

Variables	Clínica Alemana, SantiagoN = 1078	Hospital das Clínicas, Porto AlegreN = 1830	JoinvilleN = 2941
Mean age (SD)	70 (16)	66 (14)	66 (14)
Female (%)	539 (50)	934 (51)	1375 (48)
Hypertension (%)	722 (67)	1492 (82)	2170 (74)
Diabetes Mellitus (%)	226 (21)	510 (28)	957 (33)
Hypercholesterolemia (%)	259 (24)	829 (45)	887 (30)
Current smoking (%)	205 (19)	373 (20)	602 (20)
Atrial fibrillation (%)	172 (16)	338 (19)	394 (10)
Ischemic heart disease (%)	197 (9)	418 (23)	170 (6)
Previous stroke/TIA (%)	259 (24)	653 (36)	988 (34)
Cardioembolic (%)	366 (34)	490 (27)	844 (29)
Aterothrombotic (%)	140 (13)	423 (23)	706 (24)
Lacunar (%)	119 (11)	377 (21)	625 (21)
Undetermined (%)	420 (39)	347 (19)	597 (20)
Other determined (%)	32 (3)	193 (10)	169 (6)
NIHSS Median (IQR)	5 (2–10)	6 (3–13)	4 (2–11)

The selection criteria yielded 3275 (56.0%) patients for NIHSS alone, 2636 (45.1%) for time alone and 1134 (19.4%) for both criteria combined (Table 2).

Table 2

Distribution of modified Rankin scores at follow up in the total cohort according to usual acute ischaemic stroke trials selection criteria.

mRs scores	Unselected cohortN (%)	NIHSS >3 and < 25N (%)	Time from onset <6 hN (%)	Both criteriaN (%)
0	1211 (20.1)	391 (11.9)	531 (20.1)	132 (11.6)
1	1664 (28.4)	694 (21.2)	656 (24.9)	258 (22.6)
2	695 (11.8)	447 (13.6)	283 (10.7)	136 (11.9)
3	620 (10.6)	486 (14.8)	278 (10.5)	142 (12.5
4	628 (10.7)	539 (16.5)	318 (12.1)	167 (14.7)
5–6	1031 (17.6)	2718 (21.9)	570 (21.6)	299 (26.3)
Total	5849	3275	2636	1134

The unselected sample had a non-normal distribution of mRs, skewed to the left, with a median of 2 (less disability). When admission NIHSS was used, the mRs distribution was significantly less skewed, with a median of 3 (p = 0.04). When only time from symptom onset was used, the mRs distribution more skewed to lower values but nevertheless different form the unselected sample (p = 0.02). When both criteria were used, the distribution was less skewed with a median mRs of 3 and also significantly different form the whole sample cohort (p = 0.02) (Fig. 1).

Fig. 1

Percentage distribution of patients modified Rankin scale scores at discharge in the total cohort according to usual acute ischemic stroke trials selection criteria.

A significant variation in the calculated sample size was found when applying different selection criteria to the sample, with need of a smaller recruited number of patients when usual RCT selection criteria are used (Table 3).

Table 3

Descriptive statistics of samples and resulting sample sizes according to the distribution of modified Rankin scale scores at follow up after applying different selection criteria to the cohort and resulting odd ratios.

Criteria	Median	Skewness	Kurtosis	Sample size
None	2	0.56	−0.72	3616
Time from onset <6 h	2	0.53	−0.85	3364
NIHSS > 3 and <25	3	0.09	−0.9	1544
Both criteria	3	0.18	−1.07	1553

Discussion

Our results show sample sizes for ORL are significantly affected by the different distributions of follow-up mRs produced when applying usual eligibility criteria for AIS RCTs, being smaller when the resulting distribution is less skewed. In this sample, the effect was greater when stroke severity, measured with NIHSS, was used as a selection criterion together with time from symptom onset. We also found that the 6-h time window does NOT influence the sample size.

Even though mRs is an ordinal scale, it has been traditionally and still is analyzed as a binary outcome measure with many possible cut-off points [14], [15], [16], [17]. Only recently large stroke trials have used its distribution across all strata as primary end point and thus for sample size calculations [18], [19], [20], [21].

An important advantage of using ordinal or continuous analysis of mRs as primary outcome is the need for smaller sample sizes in effective therapies with small effect size or effects that are distributed across all strata [22]. This advantage may be neglected if the impact of patient selection through eligibility criteria on the distribution of mRs strata is not accounted for. This same analysis could apply to other areas of neurology and medicine that are using ordinal scales as outcome measures. A similar approach to calculate the appropriate sample size, could also be used in case an ordinal scale is dichotomized and the distribution of outcome varies in the subgroups after using different selection criteria.

The main strength of this study is that the populations were large and unselected cohorts of patients with AIS presenting at large academic medical centers that may be eligible for any acute treatment or clinical trial enrolment.

The main weakness is that we assumed an even effect size for all categories in mRs, that is improving 2% for 0–2 and worsening 2% for 3–6. This is not the case in many trials in which there could be in improvement in 0–1 and worsening of 5–6 categories or no change in mortality (mRs = 6).

In conclusion, we found that the application of usual eligibility criteria produced a 50% reduction in the required potential sample sizes for AIS RCT analyzing the mRs scale with OLR. The distribution of follow-up mRs scores are significantly different from the unselected data sample. Sample size calculations should consider this distribution in AIS, if primary outcome is ordinal analysis of the distribution of mRs scores with ordinal logistic regression.

Funding

This work was supported by unconditional grants form from Lundbeck Chile and Clínica Alemana de Santiago. It also received financial support from the Fundacao de Amparo a Pesquisa do Estado de Santa Catarina, University of the Region of Joinville and the Joinville Municipal Health Department. Grant support: Fundacao de Apoio a Pesquisa Cientifica do Estado de Santa Catarina-FAPESC. University of the Region of Joinville Univille. The funding sources had no involvement in study design; collection, analysis and interpretation of data; in the writing of the report; or in the decision to submit the article for publication.