Test characteristics of a 5-element cortical screen for identifying anterior circulation large vessel occlusion ischemic strokes.

OBJECTIVE: Stroke severity screens typically include cortical signs, such as field cut, aphasia, neglect, gaze preference, and dense hemiparesis (FANG-D). The accuracy and reliability of these signs, when assessed by emergency physicians, to identify patients with anterior circulation large vessel occlusion (ACLVO) acute ischemic stroke (AIS) is unknown. We hypothesized that the FANG-D screen applied by emergency physicians would be sensitive and reliable for identifying ACLVO AIS.
METHODS: We conducted a prospective cohort study enrolling consecutive patients with suspected AIS presenting within 4.5 hours of last known well to the emergency department (ED). Emergency physicians performed the FANG-D screen prior to, and blinded to the results of, imaging. The imaging standard was defined as a non-contrast computed tomography (CT) for identifying hemorrhage and CT angiography for identifying large vessel occlusion. ACLVO was defined as an occlusion of the internal carotid artery, the middle cerebral artery, or its first branch. A convenience sample of patients had a duplicate FANG-D screen performed by a second emergency physician to assess interobserver agreement.
RESULTS: We performed 608 FANG-D assessments on 491 patients presenting to the ED, of whom 64 (10%) had an ACLVO. FANG-D had a sensitivity of 91% (confidence interval [CI] = 81%-96%) and a specificity of 35% (CI = 31%-39%) for identifying ACLVO. Interobserver agreement was tested on 133 patients and was found to be substantial, with a Fleiss' kappa of 0.77 (CI = 0.64-0.88).
CONCLUSIONS: The FANG-D screen is a sensitive test for identifying ACLVO when performed by emergency physicians and demonstrates substantial interrater reliability.

INTRODUCTION

Background

The 2019 update to the AHA/ASA Guidelines for the Early Management of Patients With Acute Ischemic Stroke (AIS) recommends mechanical thrombectomy in selected AIS patients within 6–24 hours of last known normal who have large vessel occlusions in the anterior circulation (AC) of the brain and meet other DAWN or DEFUSE 3 trial eligibility criteria. These same guidelines recommend performing a computed tomography angiogram (CTA) with computed tomography perfusion (CTP) for “certain patients” as part of their initial imaging, but fails to specify which patients should undergo such imaging. Similarly, it is unclear which suspected stroke patients should be included in the 6‐ to 24‐hour window for consideration for mechanical thrombectomy or “code stroke” protocol activation by emergency physicians. For these reasons, it would be beneficial to have simple, qualitative criteria that can be quickly assessed by emergency physicians to determine candidacy for extended window stroke activation and advanced imaging in the emergency department (ED).

Importance

Over the last several years, many quantitative stroke scale screens have been derived to predict which patients may have a large vessel occlusion involving the internal carotid artery, or middle cerebral artery (MCA) and its main branches, the so‐called “AC” of the brain. , , , , Nonetheless, a retrospective analysis concluded that at least 20% of patients with large vessel occlusion AIS would be missed when applying published cutoffs for many of these quantitative scales, and prospective data are lacking on the performance and reproducibility of these scales by first‐line clinicians, such as emergency physicians. Furthermore, a recently published meta‐analysis suggested that some large vessel occlusion scoring systems including cortical signs , , may have better accuracy to predict large vessel occlusion stroke than others, but concluded that the accuracy of cortical signs for detecting large vessel occlusion requires further prospective evaluation. Heldner et al have demonstrated that the most predictive cortical signs for large vessel occlusion AIS involve assessments of best gaze, motor arms, aphasia/neglect, visual fields, and motor legs. These cortical signs are consistent with those found in many qualitative and quantitative large vessel occlusion screens and can be combined into the easy to remember FANG‐D acronym (field deficit, aphasia, neglect, gaze preference and dense hemiparesis), which was developed by the St. Luke's stroke program. There are no prospective, adequately powered studies of the accuracy and reliability of these signs, when assessed by emergency physicians, to identify patients with large vessel occlusion AIS.

Goals of this investigation

Our primary aim was to determine the sensitivity of the FANG‐D screen, assessed by emergency physicians, for ACLVOs. As a secondary aim, we analyzed the specificity of the screen for identifying ACLVOs, as well as the sensitivity and specificity of the screen in identifying any large vessel occlusion or intracranial hemorrhage, because we recognize if such a screen was accurate in detecting all of these stroke etiologies, it may be useful in the out‐of‐hospital setting to determine routing to comprehensive stroke centers. Additionally, since all of the published large vessel occlusion screen studies lack measurements of reproducibility, we compared the interobserver agreement of the screen in a convenience sample of patients with duplicate FANG‐D screens performed by a second emergency physician.

The Bottom Line

Early identification of patients suffering from large vessel occlusion acute ischemic strokes is critical to prioritize these patients for potential thrombectomy. This publication reports that the FANG‐D score, that has been designed to identify these patients, has a good sensitivity to identify at risk patients but may miss up to 10% of patients.

METHODS

Study design and setting

We performed a prospective cohort study fulfilling STROBE criteria at an urban academic Comprehensive Stroke Center with an ED volume of >115,000 patients annually. The ED conducts over 750 Code Stroke protocol activations annually, of which ≈60% involve patients presenting to the ED within 4.5 hours of last known well. Our institutional review board (IRB) reviewed our study and determined it met criteria for waiver of authorization and expedited review.

Selection of participants

Eligible patients were ≥18 years of age presenting to our ED within 4.5 hours of last well known with signs or symptoms consistent with a possible acute stroke. Signs and symptoms suggesting a possible stroke included acute onset of unilateral limb weakness, abnormal speech, abnormal vision, unilateral sensory loss, facial drooping, vision changes, abnormal balance, abnormal coordination, or other symptoms deemed consistent with an acute stroke by an emergency physician. We excluded patients with advanced renal dysfunction who were not eligible for CTA. Participating ED physicians (resident and attending physicians), nurse practitioners (NPs), and physician assistants (PAs) recruited eligible patients 24 hours a day and 7 days a week from June 1, 2018 to November 30, 2019.

Interventions

Prior to enrolling patients, all physicians, NPs, and PAs working in the ED during the time of this study received training on the FANG‐D screen, which included a lecture and an online video describing how to correctly apply the screen. Video viewing was confirmed by an electronic audit within an online learning module. One hundred percent of all potentially eligible emergency physicians, NPs, and PAs completed training. Additionally, the FANG‐D screen, with instructions on how to score each individual element, was displayed on paper forms (Figure S1) available to and completed by the emergency physicians, NPs, and PAs during their evaluation of Code Stroke patients. It was our institutional policy at the time of this study to complete a FANG‐D screen on every Code Stroke patient. Our evaluators independently remembered or were reminded by a member of the research team if present to complete the paper form for collection. It typically took up to 1 minute to complete a FANG‐D screen. To determine agreement of different evaluators performing the screen, a convenience sample of patients were screened by a second emergency physician, NP or PA using a separate FANG‐D screen form with blinding to the results of the initial evaluator. Performance of duplicate assessments largely relied on the presence of research personnel, who were available Monday through Friday during normal work hours. However, 2 evaluators could individually assess a patient and complete separate forms without research personnel available. These duplicate, blinded assessments were evaluated as separate screens for diagnostic accuracy analyses. All screens were performed before performance of computed tomography (CT) imaging studies.

All patients initially underwent an unenhanced head CT after FANG‐D screening. If a hemorrhage was identified, based on the immediate interpretation by a board certified neuroradiologist, no additional imaging was performed. All remaining patients underwent performance of a CTA of the neck and intracranial CTA, with most receiving CTP imaging of the brain.

Measurements

The assessing phyisician, NP, or PA recorded the overall screen result (Positive/Negative), along with a binary response (Yes/No) to each of the 5 individual elements in the FANG‐D screen form. If the overall score was positive or any element was positive, the FANG‐D screen was considered positive. If the overall score was negative or all of the individual elements were negative, the FANG‐D screen was considered negative. If the overall FANG‐D score was documented, but one or more FANG‐D elements were not documented on the form, individual FANG‐D elements were considered as missing (not negative).

Completed FANG‐D forms were either immediately collected by a member of our research team or placed within a research document bin in the ED to be collected the following day. We entered all FANG‐D form data, along with other demographic and clinical data into a database (REDCap version 9.2.2). Trained data abstractors used standardized forms and explicitly defined variables to collect the demographic and past medical history data, based on established standard operating procedures for our stroke patient database.

CT reports by board certified radiologists were used to determine the presence of hemorrhage (based on unenhanced CT) or an large vessel occlusion involving any or all of the following vessels (based on CTA): the intracranial portion of the internal carotid artery, the MCA (M1), the first branch of the M1 (M2), the anterior cerebral artery (ACA), the posterior cerebral artery (PCA), and basilar artery (BA). Any large vessel occlusion involving the intracranial portion of the internal carotid artery, M1, or M2 was considered an ACLVO. All patients included in our database had at minimum an unenhanced CT performed. For the purpose of this study, sufficient imaging was defined as either an unenhanced CT with a hemorrhage identified, or a CTA that either identified or excluded a large vessel occlusion. All reports were reviewed by one of the study authors (AWA), who was blinded to the corresponding FANG‐D screen results, to verify the presence of a large vessel occlusion in a qualifying vessel. Adjudication of any large vessel occlusion involving a qualified vessel (eg, extracranial vs intracranial internal carotid artery) was performed by another study author (RRK).

Outcomes

Our primary outcome was the sensitivity of the FANG‐D screen to identify an ACLVO involving any or all of the following vessels: the intracranial portion of the internal carotid artery, the MCA (M1), and the first branch of the M1 (M2). ACLVO locations were chosen because the strongest class of evidence exists to perform mechanical thrombectomy for clots in these locations. We focused on sensitivity and negative predictive value based on the goal of identifying all ACLVO patients and excluding large vessel occlusion in patients with negative screens.

Our secondary outcomes included (1) the specificity of the overall FANG‐D screen for an ACLVO, and (2) the accuracy of the FANG‐D screen for detecting large vessel occlusion involving any of the following vessels: intracranial internal carotid artery, M1, M2, ACA, PCA, and the BA, with or without intracranial hemorrhage. These other large vessel occlusion locations were included as secondary outcome measures, because lower levels of evidence exist for performing thrombectomy for large vessel occlusion AIS in these locations. In addition, we assessed the accuracy of each element of the FANG‐D screen and the combinations of different elements for detecting an ACLVO. A separate test consisting of dense hemiparesis and any positive FANG element was also evaluated for ACLVO.

Analysis

All statistical analyses except for inter‐rater reliability were performed using SAS (version 9.4, SAS Institute Inc., Cary, North Carolina). We described patients’ demographic and clinical characteristics at both patient and assessment levels. We reported mean and SD, and median and interquartile range when appropriate, for continuous variables, and frequency and percentage for categorical variables.

We calculated sensitivity, specificity, positive predictive value, negative predictive value, and the associated 95% confidence intervals (CIs) for the primary and all secondary outcomes. As our data at the assessment level contained 2 assessments per patient for some patients, we reported Wilson score CIs for these measures. Wilson CIs have been reported to perform well in complex sample surveys in which clustering effect may exist. , We also calculated the area under the receiver operating curve (AUC) and the 95% CIs for each outcome. To adjust for correlation between assessments within a patient, we first performed a logistic regression model with R‐side random effects using the GLIMMIX procedure. We then used the predicted probabilities outputted from the GLIMMIX model as a single predictor in a regular logistic regression to obtain AUC. ,

Because there was substantial missingness in the overall FANG‐D measurement, we performed a post hoc sensitivity analysis assuming missing were either false–positive or false–negative depending on ACLVO results (ie, the worst‐case scenario). We also reported diagnostic accuracy of each individual FANG‐D element at predicting ACLVO. A worst‐case scenario analysis was performed for each individual element as well.

To assess the agreement between different raters, we performed inter‐rater reliability analysis for all patients with a second FANG‐D form completed. We used an R‐script K_ALPHA developed by Zapf et al in R (version 3.4.2, The R Foundation for Statistical Computing, Vienna, Austria) to calculate both Fleiss’ kappa and Krippendorff's alpha and their bootstrap CIs. We reported Fleiss’ kappa statistic and its 95% bootstrap CIs as the results from both measures are similar. To handle missingness in ratings in our data, we used multiple imputation using chained equations. We first ordered the items with least amount of missing values to most missing data and then regressed them on demographic and clinical characteristics, large vessel occlusions, hemorrhage, rater types (attending, PGY1, etc), and other items in the FANG‐D screen that have been imputed. We developed 10 imputed datasets as opposed to the more typical 5 imputations to produce optimal results. Fleiss’ kappa and 95% CI based on imputed data were reported as well. We also performed a sensitivity analysis eliminating these duplicate assessments. Duplicate assessments were excluded for the sensitivity analysis based on the order that assessments were entered into our study database.

We performed sample size calculation based on the primary outcome—sensitivity of FANG‐D in screening ACLVO. The predetermined values of sensitivity and prevalence of ACLVO in our patient population were 90% and 12%, respectively. With a maximum marginal error of estimate not exceeding 7%, we estimated the required sample size to be 588.

RESULTS

Main results

From June 2018 to December 2019, a total of 640 adult Code Stroke patients arrived to the ED within 4.5 hours of last known well. The patient flowchart is displayed in Figure 1. A total of 149 patients were excluded due to non‐completion of a FANG‐D form (n = 64), insufficient imaging (n = 95), or both (n = 10). Patients excluded for insufficient imaging had an unenhanced CT that did not identify a hemorrhage, but did not have a CTA performed for vessel imaging. A total of 133 patients had a second FANG‐D screen performed for evaluation of interobserver agreement, 117 of whom were included in the total number of assessments, and 16 were excluded for insufficient imaging. A total of 608 assessments from 491 patients were included in the diagnostic accuracy analysis.

FIGURE 1

Flow chart demonstrating patient inclusion and exclusion criteria for generating the overall number of assessments included in the final analysis. LKW, last known well; ACLVO,  anterior circulation large vessel occlusion; MT, mechanical thrombectomy

Demographic and clinical characteristics of the study population are presented in Table 1 at both patient and assessment levels. Table 1 also includes baseline characteristics of patients who were not enrolled due to non‐completion of a FANG‐D form or insufficient imaging and patients who had a second FANG‐D screen performed for interobserver agreement. Most FANG‐D assessments were performed by postgraduate year (PGY) 3 emergency medicine residents (n = 190, 31%) and attending physicians (n = 202, 33%), with fewer assessments performed by PGY 1 and 2 emergency medicine residents, NPs and PAs.

TABLE 1

Patient characteristics

	Patients within 4.5 h LKW (n = 640)	Patients within 4.5 h LKW with IRR (n = 117)	No FANG‐D form completed (n = 64)	Insufficient imaging (n = 95)	Individual patients in final analysis (n = 491)
Age, mean (SD)	62 (15)	60 (14)	63 (15)	61 (16)	62 (15)
Sex (male)	49% (n = 312)	53% (n = 62)	44% (n = 28)	33% (n = 31)	52% (n = 255)
Race (white)	45% (n = 289)	50% (n = 59)	44% (n = 28)	35% (n = 33)	47% (n = 233)
Comorbidities
Hypertension	69% (n = 69)	73% (n = 85)	69% (n = 44)	71% (n = 67)	69% (n = 340)
Diabetes	29% (n = 184)	30% (n = 35)	19% (n = 12)	36% (n = 34)	29% (n = 142)
Hyperlipidemia	45% (n = 290)	40% (n = 47)	44% (n = 28)	48% (n = 46)	45% (n = 221)
Atrial fibrillation	14% (n = 88)	10% (n = 12)	11% (n = 7)	8% (n = 8)	15% (n = 73)
Clinical characteristics
NIHSS, mean (SD), median (Q1, Q3)*	7.55 (8.26), 4 (1,12)	8.12 (8.5), 5 (1,13)	8.81 (9.00), 6 (1,15)	5.44 (7.14), 3 (1,8)	7.76 (8.3), 4 (1,12)
ACLVO	8% (n = 51)	11% (n = 13)	2% (n = 1)	0	10% (n = 51)
Any LVO	10% (n = 65)	13% (n = 15)	8% (n = 5)	0	11% (n = 54)
ICH	10% (n = 61)	12% (n = 14)	16% (n = 10)	0	10% (n = 51)
IV tPA	19% (n = 120)	23% (n = 27)	13% (n = 8)	4% (n = 4)	22% (n = 109)
Thrombectomy	5% (n = 30)	9% (n = 11)	2% (n = 1)	0	6% (n = 29)
Any stroke dx	37% (n = 239)	48% (n = 56)	33% (n = 21)	10% (n = 10)	43% (n = 211)

This table compares the demographic and clinical characteristics of patients presenting to the ED within 4.5 h LWK, those with IRR form completed, those with no FANG‐D form completed, those with insufficient imaging, and all patients included in our final analyses. For any stroke diagnosis, ICD‐10 codes 160, 161, and 163 were used. The NIHSS was completed by a member of the neurology team. LKW, last known well; ACLVO, anterior circulation large vessel occlusion; LVO, large vessel occlusion; ICH, intracranial hemorrhage; IV tPA, intravenous tissue plasminogen activator; IRR, inter rater reliability.

*For NIHSS, the total n = 569, n = 192 for FANG‐D‐negative and n = 377 for FANG‐D‐positive patients.

Of the 608 assessments, 64 (10%) had an ACLVO, 69 (11%) had any large vessel occlusion, 65 (10%) had any type of intracranial hemorrhage, and 40 (6%) underwent mechanical thrombectomy. Sixty‐eight percent (n = 413/608) of the assessments were FANG‐D positive. A comparison of the clinical characteristics of FANG‐D positive versus negative assessments can be seen in Table 2. An ACLVO was present in 14% (58/413) of the FANG‐D positive assessments versus 3% (6/195) in the FANG‐D negative assessments. Two assessments with ACLVO had isolated field cut, 12 had isolated aphasia, 0 had isolated neglect, and 4 had isolated gaze preference.

TABLE 2

Clinical characteristics of FANG‐D positive versus FANG‐D negative assessments

	Total (n = 608)	FANG‐D‐negative (n = 195)	FANG‐D‐positive (n = 413)	P a
Age, mean (SD)	62.8 (15.2)	60.8 (15.2)	63.8 (15.1)	0.022
Sex (male)	52.1% (n = 317)	52.3% (n = 102)	52.1% (n = 215)	0.954
Race (white)	48% (n = 292)	56.4% (n = 110)	44.1% (n = 182)	0.004
Comorbidities
Hypertension	69.9% (n = 425)	60.5% (n = 118)	74.3% (n = 307)	< 0.001
Diabetes	29.1% (n = 177)	25.1% (n = 49)	31.0% (n = 128)	0.137
Hyperlipidemia	44.1% (n = 268)	45.6% (n = 49)	43.3% (n = 179)	0.594
Atrial fibrillation	14% (n = 85)	13.8% (n = 27)	14.0% (n = 58)	0.948
Clinical characteristics
NIHSS, mean (SD), median (Q1, Q3)*	7.8 (8.3), 5 (1, 12)	2.3 (4.0), 1 (0, 3)	10.6 (8.6), 9 (4, 17)	< 0.001
ACLVO	10.5% (n = 64)	3.1% (n = 6)	14.0% (n = 58)	< 0.001
Any LVO	11.4% (n = 69)	4.1% (n = 8)	14.8% (n = 61)	< 0.001
ICH	10.7% (n = 65)	3.4% (n = 7)	14.0% (n = 58)	< 0.001
IV tPA	22.4% (n = 136)	18.0% (n = 35)	24.5% (n = 101)	0.072
Thrombectomy	6.6% (n = 40)	0.5%  (n = 1)	9.4% (n = 39)	< 0.001
Any stroke dx	43.9% (n = 267)	34.4% (n = 67)	48.4% (n = 200)	0.001

Comparison of clinical characteristics and comorbidities between patients with FANG‐D positive and negative assessments. If an individual patient had 2 screens performed as part of the reliability assessment and the 2 raters disagreed on the overall FANG‐D result, they are accounted for in both the FANG‐D‐positive and FANG‐D‐negative groups. For any stroke diagnosis, ICD‐10 codes 160, 161, and 163 were used. The NIHSS was completed by a member of the neurology team. LKW, last known well; ACLVO, anterior circulation large vessel occlusion; LVO, large vessel occlusion; ICH, intracranial hemorrhage; IV tPA, intravenous tissue plasminogen activator; IRR, inter rater reliability.

*For NIHSS, the total n = 569, n = 192 for FANG‐D‐negative and n = 377 for FANG‐D‐positive patients.

Chi‐square test was used for categorical variables and T test for age and nonparametric test for NIHSS.

The main diagnostic accuracy results for FANG‐D are presented in Table 3. The FANG‐D screen had a sensitivity of 91% (CI = 81%–96%) and a specificity of 35% (CI = 31%–39%) for ACLVO. Results from the worst‐case scenario analysis treating missing as either false–positive or false–negative depending on ACLVO result indicated some change in accuracy measures. Sensitivity and specificity in the worst‐case scenario were 89% (CI = 79%–95%) and 32% (CI = 29%–36%), respectively. For patients who underwent mechanical thrombectomy, FANG‐D had a sensitivity of 98% (CI = 87%–99%). FANG‐D had a sensitivity of 96% (CI = 86%–98%) and specificity of 34% (CI = 30%–38%) for patients with occlusions of the internal carotid artery or M1, excluding M2. The combination of dense hemiparesis plus any other element resulted in a sensitivity of 67% (CI = 55%–78%) and specificity of 72% (CI = 68%–76%) for ACLVO. Two assessments with ACLVO were FANG positive without dense hemiparesis, 2 of whom underwent mechanical thrombectomy. A sensitivity analysis excluding duplicated FANG‐D assessments obtained for evaluating interobserver agreement indicated no significant difference in sensitivity (90%, CI = 82%–98%) and specificity (35%, CI = 31%–40%) of the FANG‐D screen for identifying ACLVO. Diagnostic accuracy results for each FANG‐D element at predicting ACLVO are presented in Table 4.

TABLE 3

Test characteristics of the FANG‐D screen

	Sensitivity	Specificity	PPV	NPV	AUC
FANG‐D for ACLVO (ICA/M1/M2)	91% (CI = 81%–96%)	35% (CI = 31%–39%)	14% (CI = 11%–18%)	97% (CI = 93%–99%)	0.63 (CI = 0.59–0.67)
FANG‐D for ICA/M1	96% (CI = 86%–98%)	34% (CI = 30%–38%)	11% (CI = 8%–15%)	98% (CI = 96%–99%)	0.65 (CI = 0.61–0.68)
FANG‐D for any LVO	84% (CI = 78%–94%)	34% (CI = 30%–38%)	14% (CI = 11%–18%)	95% (CI = 92%–97%)	0.61 (CI = 0.57–0.65)
FANG‐D for ACLVO/ICH	90% (CI = 83%–94%)	38% (CI = 34%–42%)	28% (CI = 24%–32%)	93% (CI = 89%–96%)	0.64 (CI = 0.60–0.67)
FANG‐D for any LVO/ICH	88% (CI = 82%–92%)	37% (CI = 33%–42%)	28% (CI = 24%–32%)	92% (CI = 87%–95%)	0.63 (CI = 0.59–0.66)
FANG‐D for cases that underwent mechanical thrombectomy	98% (CI = 87%–99%)	34% (CI = 30%–38%)	9% (CI = 7%–13%)	99% (CI = 97%–99%)	0.66 (CI = 0.63–0.69)
Dense hemiparesis + any FANG element for ACLVO	66% (CI = 54%–77%)	73% (CI = 69%–77%)	22% (CI = 16%–28%)	95% (CI = 93%–97%)	0.69 (CI = 0.63–0.75)

Comparison of test characteristics of the FANG‐D screen for identifying ACLVO, LVO, ICH, and cases that underwent mechanical thrombectomy. This table also includes for comparison the characteristics of a separate test, dense hemiparesis plus any other FANG element. ACLVO, anterior circulation large vessel occlusion; LVO, large vessel occlusion; ICH, intracranial hemorrhage.

TABLE 4

Test characteristics of FANG‐D elements for ACLVO

	n	Missing	Sensitivity	Specificity	PPV	NPV	AUC
Field cut	551	57	31% (CI = 20%–45%)	82% (CI = 78%–85%)	15% (CI = 9%–23%)	92% (CI = 89%–94%)	0.56 (CI = 0.49–0.63)
Aphasia	600	8	70% (CI = 57%–80%)	63% CI = (59%–67%)	17% CI = (13%–23%)	95% CI = (92%–97%)	0.66 CI = (0.60–0.72)
Neglect	565	43	49% (CI = 37%–62%)	86% (CI = 83%–89%)	28% (CI = 20%–38%)	94% CI = (91%–96%)	0.67 (CI = 0.60–0.74)
Gaze preference	603	5	52% (CI = 40%–64%)	84% (CI = 81%–87%)	28% (CI = 21%–37%)	94% (CI = 91%–96%)	0.68 (CI = 0.62–0.74)
Dense hemiparesis	598	10	74% (CI = 62%–83%)	55% (CI = 51%–60%)	16% (CI = 12%–21%)	95% (CI = 92%–97%)	0.64 (CI = 0.58–0.70)

The FANG‐D screen demonstrated substantial inter‐rater reliability for overall screen results (positive or negative), with a Fleiss’ kappa of 0.77 (CI = 0.64–0.88). The interrater reliability for the individual elements of the screen is listed in Table 5. When analyzing the agreement between reviewers on individual elements, hemiparesis demonstrated the best agreement with a Fleiss’ kappa of 0.78 (CI = 0.67–0.88), and neglect, with a Fleiss’ kappa of 0.63 (CI = 0.44–0.81), demonstrated the worst agreement.

TABLE 5

FANG‐D interrater reliability

		Complete case scenario			Multiple imputation with chained equations
			95% CI			95% CI
	n	Fleiss’ kappa	Lower	Upper	Fleiss' kappa	Lower	Upper
Field cut present vs. not	107	0.731	0.554	0.872	0.635	0.581	0.688
Aphasia present vs. not	126	0.673	0.531	0.804	0.668	0.614	0.722
Neglect present vs. not	116	0.632	0.439	0.816	0.602	0.548	0.656
Gaze preference present vs. not	132	0.765	0.629	0.887	0.753	0.699	0.807
Dense hemiparesis present vs. not	128	0.781	0.666	0.875	0.771	0.718	0.825
Overall FANG‐D result	133	0.772	0.642	0.882

Interrater reliability for each element of the FANG‐D screen, as well as for overall FANG‐D result (positive or negative). To handle missingness in reporting of each individual element, we report multiple imputation using chained equations.

Limitations

There were several limitations in our study. Our CI for the sensitivity of the FANG‐D screen for ACLVO is wider than anticipated, because of a lower ACLVO prevalence in our study population than assumed in our sample size calculation. Although our goal was consecutive enrollment, our patient flow chart demonstrates that some subjects fulfilling our inclusion criterion of presenting to the ED within 4.5 hours of last known well did not have a FANG‐D assessment documented and/or did not undergo sufficient imaging. Nonetheless, those subjects were tracked and described in Table 1. Those patients were more likely to have had an intracranial hemorrhage and less likely to have a large vessel occlusion or undergo mechanical thrombectomy. This likely biases our results for less accuracy of the screen for identifying any group that included intracranial hemorrhage patients. Regarding the duplicate screens used to assess interrater agreement, some of these were completed when a member of our research team was not present; therefore, we cannot verify truly independent performance for all duplicate screens. As demonstrated in Table 4, all of the FANG‐D elements were not included in all screens performed. This demonstrates the challenge for emergency physicians in assessing for some cortical signs in certain patients and represents the real‐world experience of applying a large vessel occlusion screen. In particular, visual field deficit and neglect were the cortical signs least frequently assessed, which suggests emergency physicians find these challenging to assess in certain suspected stroke patients. Indeed, for quantitative large vessel occlusion screens that require a threshold quantitative score to most accurately detect a large vessel occlusion stroke, the impact of not including all scored components may make those scales less accurate. Although all physicians, NPs, and PAs were trained on the proper performance and interpretation of each of the cortical screening elements, the adherence to standardized exam performance and scoring is unknown. We do not know how many unique reviewers completed the assessments; however, we are able to report the number of assessments performed by reviewers of differing levels of experience. Concerning differences in screen performance based on performer experience, our study was not powered to conduct a subgroup analysis for different training levels. We defined hemiparesis at a level of inability to maintain antigravity from the gurney for a count of 5. Although this may not equate to “dense hemiparesis” for all cases, we chose this threshold as one that would be most reliable to assess consistently. We cannot exclude the possibility that some large vessel occlusions included were chronic, but this is a limitation of any study that relies on CTA or digital subtraction angiography for large vessel occlusion confirmation. Finally, we did not include internal carotid artery occlusions limited to the cervical portion of the internal carotid artery in our large vessel occlusion definition, which is consistent with many other studies evaluating large vessel occlusion screen accuracy. We included M2 occlusions in our definition of ACLVOs and did not limit this to the region of the M2 segment for which embolectomy was considered. Nonetheless, there is no established criterion for this designation and any M2 segment occlusion is consistent with the AC vessels now considered eligible for mechanical thrombectomy in the most recent AHA/ASA guidelines.

DISCUSSION

Our study represents the first prospective study of a large vessel occlusion screen performed by emergency physicians. We demonstrate that the FANG‐D cortical screen has good sensitivity for identifying ACLVO; however, based on our findings, 1 out of 10 patients with ACLVO would be missed with this screen. FANG‐D does demonstrate a high negative predictive value for excluding an ACLVO AIS when the screen is negative. While the screen has low specificity and positive predictive value for large vessel occlusion, it must be emphasized that the rationale for our study was to easily determine which ED patients should undergo extended Code Stroke protocol activation out to 24 hours from last known well. The strengths of the FANG‐D screen are its simplicity and high negative predictive value for ruling out both an ACLVO and any patient potentially eligible to undergo thrombectomy. The mnemonic facilitates remembering the components of the screen and it is less cumbersome to apply than any quantitative screen and other qualitative large vessel occlusion screens, since patients are simply FANG‐D positive if any cortical sign is present. Indeed, our study findings are consistent with the paradigm shift described in a recent retrospective analysis, which concluded that assessing for any objective focal neurologic findings may be the best strategy for screening to determine candidacy for CTA and Code stroke activation. This is supported by another retrospective study that found cortical symptoms alone to be a reliable indicator for large vessel occlusion (sensitivity: 91%) and mechanical thrombectomy (sensitivity: 90%) in acute stroke patients. However, a limitation of that study is that cortical signs were retrospectively derived from corresponding NIHSS score elements performed by experienced neurologists.

Before our study, the vision, aphasia, and neglect screen had the best reported sensitivity for any quantitative or qualitative screen to identify a large vessel occlusion amenable to thrombectomy, but that was reported in a small pilot study in which the vision, aphasia, and neglect screen was applied by specially trained nurses. A recently published study attempting to validate the vision, aphasia, and neglect screen found a much lower sensitivity of 79% for large vessel occlusion. Our FANG‐D screen sensitivity also exceeds that reported for the RACE, FAST‐ED, and CPSS screens. The distinguishing difference between the FANG‐D screen and vision, aphasia, and neglect screen is that vision, aphasia, and neglect screen requires some arm weakness along with another cortical sign. Therefore, a limitation of vision, aphasia, and neglect screen and some other large vessel occlusion screens is that they exclude pure aphasia or hemineglect. This was highlighted in a small prospective study that analyzed the performance of multiple large vessel occlusion scales applied in the ED by research study nurses, in which sensitivities ranged from 42% to 64% for detecting large vessel occlusion. The exclusion of patients with pure aphasia or hemineglect in that study may explain the low sensitivity of their findings and supports including those elements alone. As our results indicate, ACLVO patients can present with the isolated cortical signs of a visual field cut, aphasia, and gaze preference. The importance of isolated aphasia is especially notable, because this represents a disabling neurological deficit and can frequently be associated with an M2 branch occlusion amenable to thrombectomy.

We realize that a logical question is how the FANG‐D screen might compare to other stroke severity screens as an out‐of‐hospital routing screen used by emergency medical service personnel to determine transport destination to an endovascular or comprehensive stroke center. Although that comparison is confounded by the retrospective methodology and varying types of raters used in other studies, the specificity of the FANG‐D screen precludes it from use as an out‐of‐hospital screen, because its low specificity, combined with an overall low prevalence of large vessel occlusion in out‐of‐hospital patients screened, would result in excessive rates of over triage. This represents a logistical challenge for virtually all of the qualitative and quantitative large vessel occlusion screens. , ,

In addition to our prospective design, a strength of our study is our measurement of inter‐rater reliability, which is lacking from all other studies of qualitative and quantitative screens. The FANG‐D screen demonstrated substantial agreement between raters for overall FANG‐D score, which further validates its generalizability. As mentioned in limitations, the missingness of field cut and neglect suggest ED physicians may find these cortical signs more difficult to assess. Supporting this notion is that these cortical signs also demonstrated the lowest interrater reliability. On the other hand, because the screen was deemed positive if any 1 cortical element was positive, evaluators may have been less inclined to test for additional cortical signs once 1 positive element was identified. Aphasia also demonstrated low interrater reliability in our study. This may be explained by emergency physicians interpreting dysarthria to be aphasia, especially because the inter‐rater reliability for dysarthria is among the lowest for any NIHSS component. Additionally, some of the components of the NIHSS we included in our aphasia assessment have been demonstrated to have low levels of agreement. Certainly, our results suggest if the FANG‐D screen is to be used, it would be worthwhile to educate users on how to appropriately assess for aphasia and distinguish it from dysarthria.

In summary, our results suggest screening for cortical symptoms alone via the FANG‐D acronym may be a simple and reliable approach to determine candidacy for extended window Code Stroke activation and diagnostic imaging to identify an ACLVO and likely any large vessel occlusion amenable to thrombectomy. Nonetheless, the FANG‐D screen requires external validation before being implemented in EDs to potentially guide decisionmaking regarding Code Stroke activation and advanced imaging in potential stroke patients. Although a quantitative screen or some combination qualitative screens could improve the specificity of the screen, some large vessel occlusions eligible for thrombectomy are likely to be missed with screens that are more challenging to apply.

CONFLICT OF INTEREST

AWA received royalty payments from Wolters Kluwer Health | UpToDate Inc. and provided consulting services to Medtronic, Inc.

AUTHOR CONTRIBUTIONS

AWA conceived the study and obtained research funding. AWA and JH designed the study. JH and CD aided in recruitment of patients. JH, DS, JR, and CD managed the data. BC and HY analyzed the data. JH, AWA, BC, and HY drafted the manuscript. All authors contributed to revision of the manuscript. JH takes final responsibility of the article.

Additional supporting information may be found online in the Supporting Information section at the end of the article

Click here for additional data file.