Optimizing in-hospital triage for large vessel occlusion using a novel clinical scale (GAI2AA).

OBJECTIVE: To identify a proximal anterior circulation occlusion for effectively administering immediate mechanical thrombectomy by developing a novel, simple diagnostic scale to predict the occlusion, to compare its validity with available scales, and to assess its utility.
METHODS: To develop a novel clinical scale, we retrospectively analyzed a cohort of 429 patients with acute ischemic stroke from a single center. The novel scale GAI2AA was applied to a prospective cohort of 259 patients from 3 stroke centers for external validation. The utility of the scale as an in-hospital triage was compared for the temporal factors of 158 patients with the occlusion.
RESULTS: In a scale-developmental phase, those with a proximal anterior circulation occlusion had significantly more frequent signs of hemispheric symptoms, including gaze palsy, aphasia, inattention, arm paresis, and atrial fibrillation. The GAI2AA scale was developed using consolidated hemispheric symptoms and was scored as follows: score = 2, arm paresis score = 1, and atrial fibrillation score = 1. A cutoff value ≥3 was optimal for the correlation between sensitivity (88%) and specificity (81%), with a C statistic of 0.90 (95% confidence interval 0.87-0.93). External validation indicated that discrimination was significantly better than or not different from that of available complex scales. Door-to-puncture time was significantly reduced (91 [82-111] vs 52 [32-75] minutes, p < 0.001).
CONCLUSION: The GAI2AA scale showed high sensitivity and specificity when an optimal cutoff score was used and was useful as an in-hospital triage tool.

Stroke is associated with 43.7 million (3.2%) annual disability-adjusted life-years worldwide.[1] Mechanical thrombectomy, particularly initiated within 6 hours from the onset of symptoms, is recommended for patients with ischemic stroke due to a proximal anterior circulation occlusion.[2-7] Rapid recognition of the occlusion is essential for earlier initiation of mechanical thrombectomy.[8,9]

Several clinical scales are available to predict occlusions.[10-12] However, selection according to these scales may miss a candidate or include the majority of patients.[13] The role of these scales as an adequate patient triage to preclude the use of intracranial arterial imaging has been unclear.[9,13]

We aimed to develop a novel, simple diagnostic scale, called GAI2AA, to externally validate its generalizability for predicting a proximal anterior circulation occlusion, to compare its validity with the available scales, and to assess the reduction of door-to-puncture time using the scale for an in-hospital triage.

Methods

Study purpose, design, and eligibility criteria

This prediction model study, conducted in accordance with the guidelines of the Transparent Reporting of a Multivariable Prediction Model for Individual Prognosis or Diagnosis,[14] included 3 phases. The first was a scale-developmental phase, a retrospective nonblinded cohort study, that retrospectively analyzed patients' clinical variables and established a novel diagnostic scale. The second phase, a prospective external validation cohort study, analyzed a new dataset to assess the generalizability of the novel scale and to compare its predictive performance with the available scales. The third phase, a retrospective cohort study, compared the temporal factors associated with mechanical thrombectomy before and after the introduction of the scale.

Acute ischemic stroke was defined as a focal or global neurologic deficit with symptoms thought to be of vascular origin but without evidence of intracranial hemorrhage.[15] We included only patients with acute ischemic stroke who arrived within 4.5 hours from the onset of symptoms because good reperfusion should be achieved as early as possible without detailed clinical/perfusion imaging mismatch if treatment can be initiated within 6 hours of symptom onset.[7] Patients were administered IV alteplase before vascular imaging if deemed appropriate by the attending physician. Patients with suspected TIA with no symptoms during the assessment in the emergency room were excluded from the analysis.[15]

Imaging protocols and definition of a proximal anterior circulation occlusion

We used noncontrast CT, MRI, or both to diagnose ischemic stroke. Patients diagnosed with ischemic stroke were sequentially evaluated for an occlusion with magnetic resonance angiography (MRA), CT angiography, or cerebral angiography within 6 hours from onset. CT angiograms were acquired with a 64- or 320-slice multidetector-row CT scanner. Time-of-flight MRA data of the intracranial arteries were acquired with 1.5T or 3T magnetic resonance scanners.

Proximal anterior circulation occlusion was defined as unilateral acute complete symptomatic occlusion of the intracranial internal carotid artery (ICA) and M1 and proximal M2 segments of the middle cerebral artery. Occlusions of other cerebral vessels such as the anterior cerebral or basilar artery were excluded.

Vascular risk factors and neurologic evaluation

The vascular risk factors of hypertension, hypercholesterolemia, diabetes mellitus, and atrial fibrillation (AF) were diagnosed on admission according to self-reported history or the standard definitions of blood pressure >160/90 mm Hg, low-density lipoprotein cholesterol >4.1 mmol/L, National Glycohemoglobin Standardization Program standards for glycated hemoglobin >6.5%, and ECG findings.[16-19] The presence of AF included a history of paroxysmal and persistent AF and newly diagnosed AF detected on the ECG acquired in the ambulance. Alcohol consumers were defined as those consuming an average of >7 standard drinks weekly.[20] Patients with a history of smoking within 5 years of admission were classified as smokers.[21]

The medical records were retrospectively reviewed for the presence of each subitem of the NIH Stroke Scale (NIHSS). The exact NIHSS was scored on admission by a stroke-trained hospital physician.

Strategies for the development of a novel scale to predict the occlusion

To calculate a valid sample size, we assumed that the frequency of a large artery occlusion as the cause of acute ischemic stroke was 28% to 46%,[22,23] requiring a sample size with a 95% confidence interval of 10% (range 310–382).[24] The retrospective cohort included patients treated from September 2012 to February 2017 from a single center in Kochi, Japan.

We designed a simple scoring system. Grading of severity or a complex flowchart was not applied. To determine an appropriate scale, patients' baseline characteristics, including neurologic findings, were compared between patients with and those without an occlusion. We analyzed the clinical variables to identify those that predicted the occlusion. Univariate analyses were initially used to select variables with the highest discriminatory value in our retrospective cohort of patients with acute ischemic stroke. Multivariable logistic regression analysis was performed a priori on the basis of clinical relevance for differences associated with a value of p < 0.20 on univariate analysis.[25] Predictive models were created to select significant variables on the guiding principle of parsimony. Coefficients derived from the multivariable logistic regression analysis were used to weigh the significance of each factor.

The optimal threshold of the novel scale to discriminate the occlusion was determined from receiver operating characteristic (ROC) curve analysis, i.e., values nearest the upper-left corner of the curve. The significance of the threshold was evaluated for the ability to discriminate between the severity of neurologic deficits on the basis of NIHSS values of <6 or ≥6.[5]

Prospective external validation

We conducted a prospective external validation study using a new cohort of patients to show its generalizability.[14] We intended to include a minimum of 100 events as suggested for externally validating a prognostic model.[26] Thus, we included 250 patients representing ≈40% of the population with an occlusion according to a previous survey at each center. We conducted a 12-month study of the patients from the stroke centers in Kochi, Aichi, and Fukuoka, Japan, based on the results of statistical analyses. Thus, from March 2017 to February 2018, patients with acute ischemic stroke who were admitted within 4.5 hours from the onset of symptoms were assigned GAI2AA scale scores by a stroke-trained hospital physician on arrival before vascular imaging and were diagnosed to detect the occlusion. If the presence of AF was unclear, it was not scored.

The accuracy and reliability of prediction with the GAI2AA scale were compared with simple clinical scales, including the Prehospital Acute Stroke Severity (PASS) scale,[10] the Cincinnati Prehospital Stroke Severity Scale,[12] the 3-Item Stroke Scale (3I-SS),[27] the Field Assessment Stroke Triage for Emergency Destination,[11] and the Rapid Arterial Occlusion Evaluation Scale.[28] The Ambulance Clinical Triage For Acute Stroke Treatment and Los Angeles Motor Scale were not compared because they included various conditions administered by paramedics in the field, which could not be retrieved from our collected data.[29,30] The accuracy of the predictions was evaluated for the sensitivity and specificity of each optimal cutoff value. The C statistic was used to assess discrimination. The optimal cutoff points of the scores were defined according to published data.[10-12,27,28]

Comparison of the temporal factors associated with mechanical thrombectomy before and after the introduction of the GAI2AA scale

The temporal factors were compared for patients with occlusion who were treated endovascularly ≤6 hours at a single center in Kochi, Japan. From January 2015 to February 2017, MRA was usually used to identify the occlusion subsequent to head CT. After the introduction of the GAI2AA scale, from March 2017 to January 2019, the patients with a score ≥3 on arrival were considered candidates for direct transfer to an angiographic suite. The patients were scored with this scale by a stroke-trained hospital physician largely depending on the information provided by the emergency department staff. The exact score of the GAI2AA scale was decided by the physician owing to the trial design, which adopted a stroke-trained physician as the sole scorer and designated the decision regarding the use of the angiographic suite to the physician/neurointerventionalist, thereby ensuring consistency throughout the trial duration. If deemed appropriate by the stroke-trained physician/neurointerventionalist after discussions pertaining to the detailed information obtained from family members or relatives, the rationality of the relationship between the paralyzed side and the hemispheric symptoms, the high NIHSS score, or the hyperdense cerebral artery signs in head CT images, patients with a score ≥3 on arrival were directly transferred to the angiographic suite subsequent to head CT. Patients with a score of 0, 1, or 2 were treated similarly as previously reported. Good reperfusion, door-to-puncture time, and door-to-recanalization time were defined on the basis of the previous reports.[2,31]

Data collection and ethics

Data were anonymously analyzed. Missing data were reviewed and added from a second review of medical records. The requirement for a signed informed consent was waived because the guidelines of the institutional ethics review board of each stroke center that approved this study stated that the waiver presented no more than minimal risk of harm to participants and involved no procedures requiring written consent (approvals KHSC161063, KMH17111503, and HM17-394).

Statistical analysis

Dichotomous categorical data such as sex, vascular risk factors, and prehospital functional independence were analyzed with the Fisher exact test. The Mann-Whitney U test was used to analyze ordinal or nonparametric, continuous data such as the NIHSS score, age, onset-to-admission time, and glucose level. Values for noncategorical data are presented as the median and the 25% to 75% interquartile range. Explanatory variables for multivariable logistic regression models were limited to ≈10% of patients in the less frequent category.[32] The logistic regression equation included the significant variables and their coefficients. The outcome variable was expressed as the logit, a natural log (base e) of the odds ratio (OR). ORs and their 95% confidence intervals were calculated for significantly associated variables.

The statistical significance of the discriminative difference between the scales was tested with the DeLong test for 2 correlated ROC curves.[33] The calibration of he model was assessed for goodness of fit by plotting the estimated probability of each score on the x-axis against the observed probability along the y-axis compared with the diagonal line, representing perfect calibration. The threshold for statistical significance was p < 0.05. All statistical analyses were performed with EZR version 1.27.23 (Saitama Medical Center, Jichi Medical University, Saitama, Japan).[34]

Data availability

Anonymized data will be shared by request from any qualified investigator.

Results

Development of the novel GAI2AA scale predicting occlusion

Figure 1 shows the flow diagram of this retrospective cohort.

Figure 1

Flow diagram of the retrospective cohort

CI = cerebral infarction; ICA = internal carotid artery.

Comparison of patient baseline characteristics and neurologic deficits between patients with and without occlusions

Table 1 shows comparisons of the baseline characteristics of patients with and without occlusions. Univariate analyses revealed that patients with an occlusion were significantly older and had a higher NIHSS score, higher body mass index, and higher prevalence of AF and lower prevalence of hyperlipidemia. Multivariable logistic regression analysis revealed that a higher NIHSS score on admission and presence of AF differed significantly between patients with or without an occlusion (table 1).

Table 1

Baseline characteristics and multivariable logistic regression analysis of included factors of the retrospective cohort

To identify crucial subitems of the NIHSS, neurologic deficits were compared between patients with and those without occlusions. Univariate analysis revealed a significant difference for each factor (p < 0.001). Multivariable analysis revealed that gaze palsy, facial palsy, arm paresis, aphasia, and inattention were significant independent predictors of occlusions (table 2).

Table 2

Multivariable logistic regression analysis of neurologic findings of the retrospective cohort

Multivariable logistic regression analysis of the associations of patients' characteristics and neurologic deficits

We refined the variables to develop a simple scale. Gaze palsy, aphasia, and inattention are hemispheric and complement each other; these symptoms suggest the presence of hemispheric dysfunction. The coexistence of these symptoms does not increase the probability of an occlusion. These symptoms were consolidated and called GAI, indicating the presence of any of the following: gaze palsy, aphasia, or inattention. The factor arm paresis, not facial palsy, was selected because of its associated higher OR in the multivariable analysis of neurologic deficits (OR 2.9 for arm paresis, OR 2.1 for facial palsy) (table 3).

Table 3

Multivariable logistic regression analysis depending on factors of the retrospective cohort

Multivariable logistic regression analysis revealed that a proximal anterior circulation occlusion was associated with the independent factors GAI, arm paresis, and AF, which was calculated as follows: logit (probability of the occlusion) = 3.3 (GAI) + 1.5 (arm paresis) + 1.3 (AF) − 4.4 + e. The points of the estimated vs observed values of the occlusion were close to the diagonal line, which indicated the excellent estimate of the model of outcome (figure 2A).

Figure 2

Goodness of fit of the multivariable logistic regression analysis model and validity of the GAI2AA scale

(A) Estimated (x-axis) vs observed (y-axis) values of the probability of an occlusion associated with each score with a reference to the diagonal dotted line, which represents perfect calibration. The receiver operating characteristic curve analysis of the (B) retrospective and (C) prospective cohort showing the sensitivity and specificity of a score of 3 and the C statistics. CI = confidence interval.

Evaluation of the GAI2AA scale for predicting occlusions

The GAI2AA scale was created from the data presented above. This scale estimates the risk of a proximal anterior circulation occlusion in patients with acute ischemic stroke on hospital arrival. This scheme assigns 2 points if GAI is present and 1 point each for arm paresis and AF. If we considered the interactions between factors, the point would be different between AF and arm paresis. However, we gave priority to simplicity and applied 1 point to each factor.

Scores of the GAI2AA scale = 2 (if any of GAI is present) + 1 (arm paresis) + 1 (AF).

Predictive value and discrimination ability of the GAI2AA scale

The GAI2AA scale score significantly correlated with an occlusion. Specifically, higher scores augmented the probability of the occlusion. Figure 2B shows the ROC curve analysis. The precise NIHSS scores on admission were significantly lower for patients with a GAI2AA scale score of 0, 1, or 2 compared with those with scores of 3 or 4 (3 [1-4] vs 16 [13-21], p < 0.001), indicating that GAI2AA scale scores reflected the severity of NIHSS scores on admission.

Patient characteristics and the GAI2AA scale scores of a prospective cohort

From March 2017 to February 2018, 1,515 patients with stroke with onset within 1 week were admitted to the 3 stroke centers, and 259 patients with ischemic stroke were admitted within 4.5 hours from onset, of whom 100 patients had a proximal anterior circulation occlusion (27 ICAs, 49 M1s, and 24 M2s). The GAI2AA scale significantly correlated with the occlusions. Figure 2C shows the ROC curve analysis of the prospective cohort. The precise NIHSS score on admission was significantly lower for patients with GAI2AA scale scores of 0, 1, or 2 vs 3 or 4 (2 [1-5] vs 15 [11-22], p < 0.001).

Accuracy and reliability of the GAI2AA scale compared with other scales

When applied to the retrospective and prospective cohorts, the ability of the GAI2AA to discriminate (C statistic) was significantly better than the PASS and 3I-SS and not significantly different from the others (table 4).

Table 4

Comparison of the GAI2AA scale applied to the retrospective and prospective cohorts with published scales

Comparison of the temporal factors before and after introduction of the scale

Of 158 patients with proximal anterior circulation occlusion, 115 were treated with mechanical thrombectomy from January 2015 to January 2019. Of 61 patients treated after the introduction of the GAI2AA scale as an in-hospital triage, 37 (61%) were directly transferred to the angiographic suite and subsequently underwent mechanical thrombectomy. Among the 39 patients who were directly transferred to the angiographic suite, 2 did not show significant occlusion. One had mutism due to dementia, which could not be confirmed before the angiography. The other showed incomplete but significant recanalization probably due to IV alteplase infusion. Door-to-puncture time was significantly reduced (91 [82-111] vs 52 [32-75] minutes, p < 0.001) (figure 3A).

Figure 3

Boxplots of temporal factors associated with mechanical thrombectomy

Dark line represents median value. Box encloses interquartile range from the first to third quartile. Short horizontal lines represent the minimum and maximum values with the exception of outliers designated with a circle. (A) Door-to-puncture time, (B) puncture-to-recanalization time, and (C) onset-to-recanalization time.

The frequency of good reperfusion (89% vs 82%, p = 0.43) and groin puncture–to–recanalization time (46 [30-74] vs 39 [29-74] minutes, p = 0.64) (figure 3B) were not significantly different between the 2 groups, resulting in a significant reduction of onset-to-recanalization time (241 [205-298] vs 200 [149-275] minutes, p = 0.0098) (figure 3C).

Discussion

Study implications

The GAI2AA scale, comprising 3 factors, accurately predicts a proximal large anterior circulation occlusion with high sensitivity and specificity using an optimal cutoff value and is useful as an in-hospital triage tool.

Simple clinical scales predicting large vessel occlusions

Rapid identification of the occlusion may shorten the time to start mechanical thrombectomy. High NIHSS scores are used to predict occlusion.[35] However, field assessment of the NIHSS is lengthy and therefore infrequently performed.[9] Several simple scales were designed and validated for rapid identification.[9,12,27,28,36-41]

An ideal scale should have high sensitivity, good specificity, and apparent cutoff values acquired from a limited range of scores. Simple clinical scales combining NIHSS subitems provide a gross estimate of the presence or absence of an occlusion.[9] However, clinically relevant thresholds of each scale are uncertain. For example, a scale with high specificity and low sensitivity such as the 3I-SS would miss a significant number of patients with occlusions. In contrast, a scale with low specificity such as PASS may include most patients.[13] Thus, these scales are invalid without optimal cutoff values.

Clinical scales such as the Field Assessment Stroke Triage for Emergency Destination, Rapid Arterial Occlusion Evaluation scale, and 3I-SS use a complex staged scoring of certain symptoms and require special practice for accurate application; the available simple scales must not omit vessel imaging.[9]

Superiority of the GAI2AA scale

Our novel GAI2AA scale comprises hemispheric symptoms as a block, arm paresis, and the clinical risk factor AF, with double weighting of the hemispheric symptoms, on the basis of the results of our analysis of a retrospective cohort. Consolidation of 3 hemispheric symptoms is a distinct feature of the GAI2AA scale. Acute lesions of the frontal eye fields impair eye movement and cause gaze palsy.[42] Aphasia is a defect in language processing caused by dysfunction of the dominant cerebral hemisphere.[43] Contralateral hemineglect often occurs with acute lesions of the nondominant parietal lobes.[44] These 3 symptoms reflect hemispheric dysfunctions and often coexist and thus may be considered together. Another novel feature of this scale is the double weighting of the presence of GAI. Scores ≥3 always include the GAI symptom.

Scoring, accomplished with a simple model incorporating 4 elements of neurologic deficits, does not grade the clinical elements and is operational in prehospital setting with the information conveyed over the phone.[10] The scale yielded the same results when applied to retrospective and prospective cohorts. The inclusion of AF prevalence in addition to neurologic deficits is specific to the GAI2AA scale. The prevalence of AF in the general population is 0.95%[45]; however, it increases to 27% in patients with acute ischemic stroke with a proximal large anterior circulation occlusion.[46] Compared with available scales, the GAI2AA scale comprises a smaller range of scores with the same efficacy and an apparent cutoff value with high sensitivity (95%) and sufficient specificity (79%). The cutoff values of the GAI2AA scale ≥3 were associated with false-negative and false-positive rates of 5% and 21%, respectively, satisfying the required clinical relevance of false-negative and false-positive rates of ≤10% and ≤30%, respectively.[13] A score of 4 always includes AF and represents a higher probability of an occlusion than does a cutoff score of 3, which does not necessarily include AF. This suggests that AF has an additive predictive value by indicating association with cardioembolism. Thus, we believe that the factor AF is clinically relevant and specific to GAI2AA scale.

The GAI2AA scale is simple and quick to score, so it is useful in time-limited stroke emergencies.

Study limitations

To minimize lengthy exclusion steps, the study included elderly patients and functionally dependent patients who might not benefit from mechanical thrombectomy.[2-6] The frequencies of AF vary among different populations. Advancing age, female sex, white race, high income, and large hospital size are associated with increased odds of AF in patients with acute ischemic stroke.[47] Spontaneous or postalteplase recanalization after scoring may increase the selection of false positives. However, only 21% of patients had such recanalization after IV alteplase, and as few as 4.4% patients have ICA occlusion.[48] To assess the accuracy of a scale for prehospital patient selection, the analysis of all patients suspected to have ischemic stroke, including stroke mimics or hemorrhagic stroke, who were examined in the field might be important. However, reducing the delay in administering mechanical thrombectomy is mandatory, and overestimation of occlusions may be acceptable in a prehospital setting. We included only acute ischemic stroke within 6 hours from onset; however, mechanical thrombectomy is recommended for selected patients within 6 to 16 hours of the last known normal with a proximal large anterior circulation occlusion and meet other DWI or CTP Assessment With Clinical Mismatch in the Triage of Wake-Up and Late Presenting Strokes Undergoing Neurointervention With Trevo (DAWN)[49] or Endovascular Therapy Following Imaging Evaluation for Ischemic Stroke (DEFUSE 3)[50] eligibility criteria.[7] Expanded application of the GAI2AA scale requires further verification.

The other simple scales applied to our cohorts showed higher sensitivity than those previously published.[10-12,27,28] The high detection rate of neurologic deficits in the present study may reflect the expertise of the attending physicians. This scale may apply only to those tested centers in Japan; further replication in other centers and countries is needed to generalize our study results. To obtain similar results in a prehospital setting, active inquiry of the symptom details from the emergency staff by a stroke-trained physician will likely be useful for accurate detection of the factors of the GAI2AA scale, particularly the hemispheric symptoms.

Because of the relatively low specificity, unnecessary use of the angiographic suite and resulting exhaustion of neurointerventionalists might occur. Stroke-trained physicians should check other factors and ensure that the occlusion is reasonable to exclude the unnecessary direct transfer.

Utility of the GAI2AA scale

The GAI2AA scale can be a potential beneficial scoring system to help improve outcomes of patients with ischemic stroke due to a proximal large anterior circulation occlusion. Because the scoring performed by a stroke-trained hospital physician was largely dependent on the information provided by the emergency department staff, the simple scoring system of the GAI2AA scale can be used by ambulance staff without special training in a prehospital setting. We hope that the score obtained by them alone could be used as screening of the candidates for mechanical thrombectomy. The high sensitivity and good specificity of the GAI2AA may justify the omission of cerebral vascular imaging and direct transfer of the patient to an angiographic suite after the exclusion of hemorrhagic strokes if the GAI2AA score is ≥3. At least the GAI2AA scale can be used as an alert system for mechanical thrombectomy. When multiple, simultaneous admissions of stroke victims occur, adequate patient triage can be achieved with the GAI2AA scale scores. Although not perfect, the GAI2AA scale is practical in stroke emergencies.

We developed a novel, simple scale to predict a proximal large anterior circulation occlusion based on an analysis within 4.5 hours from onset in a retrospective cohort of patients with acute ischemic stroke. The GAI2AA scale showed high sensitivity and specificity with adequate cutoff scores in the prospective cohort, including patients from 3 stroke centers. Using the GAI2AA scale as an in-hospital triage significantly reduced the door-to-puncture time by 40 minutes. The GAI2AA scale will likely provide a valuable and highly informative screening tool.