Streamlining Acute Stroke Care by Introducing National Institutes of Health Stroke Scale in the Emergency Medical Services: A Prospective Cohort Study.

BACKGROUND: National Institutes of Health Stroke Scale (NIHSS) is the most validated clinical scale for stroke recognition, severity grading, and symptom monitoring in acute care and hospital settings. Numerous modified prehospital stroke scales exist, but these scales contain less clinical information and lack compatibility with in-hospital stroke scales. In this real-life study, we aimed to investigate if NIHSS conducted by paramedics in the field is a feasible and accurate prehospital diagnostic tool.
METHODS: This prospective cohort study is part of Treat-NASPP (Treat-Norwegian Acute Stroke Prehospital Project) conducted at a single medical center in Østfold, Norway. Sixty-three paramedics were trained and certified in NIHSS, and the prehospital NIHSS scores were compared with the scores obtained by in-hospital stroke physicians. Interrater agreement was assessed using a Bland-Altman plot with 95% limits of agreement. In secondary analysis, Cohen κ was used for the clinical categories NIHSS score of 0 to 5 and ≥6. As a safety measure, prehospital time was compared between paramedics conducting NIHSS and conventional paramedics.
RESULTS: We included 274 patients. The mean difference in NIHSS scores between the paramedics and the stroke physicians was 0.92 with limits of agreement from -5.74 to 7.59. Interrater agreement for the 2 clinical categories was moderate with a κ of 0.58. The prehospital NIHSS scoring was performed mean (SD) 42 (14) minutes earlier than the in-hospital scoring. Prehospital time was not significantly increased in the NIHSS-trained paramedic group compared with conventional paramedics (median [interquartile range] on-scene-time 18 [13-25] minutes versus 16 [11-23] minutes, P=0.064 and onset-to-hospital time 86 [65-128] minutes versus 84 [56-140] minutes, P=0.535).
CONCLUSIONS: Paramedics can use NIHSS as an accurate and time efficient prehospital stroke severity quantification tool. Introducing NIHSS in the emergency medical services will enable prehospital evaluation of stroke progression and provide a common language for stroke assessment between paramedics and stroke physicians. REGISTRATION: URL: https://www. CLINICALTRIALS: gov; Unique identifier: NCT03158259.

Acute stroke identification and treatment require a diagnostic tool that rapidly and accurately identifies stroke symptoms and severity, since shorter time to revascularization treatment leads to better outcomes.[1-3] Delayed hospital arrival remains the main exclusion criteria for thrombolytic therapy and prehospital delay is strongly correlated with in-hospital delay.[4,5] Optimizing both prehospital and in-hospital systems are required to correctly identify patients with stroke, select the correct level of care, and limit time delays.[6]

The National Institutes of Health Stroke Scale (NIHSS) is the most validated diagnostic tool in acute care and hospital settings for accurate stroke identification, quantification, monitoring, and prediction of outcome.[7] For prehospital use however, several different modified stroke scales are in use, mostly to detect single aspects of acute stroke care like screening for stroke or prediction of large vessel occlusion (LVO).[8-10] These prehospital scales are derived from NIHSS, but the scores must be interpreted differently and they are also nonsuperior to NIHSS regarding accuracy in detecting stroke and LVO.[8,9,11-13] A compatible and accurate stroke scale which includes several aspects of acute stroke care may be the missing link in streamlining the acute stroke care chain. The reason for using modified stroke scales in the emergency medical services (EMS) instead of NIHSS, is the assumption that NIHSS is too complicated and time-consuming to be used by paramedics in the field.[8,11,14] This assumption has not been challenged, and therefore, leaves an open question to how NIHSS would work in the hands of paramedics. This lacking knowledge is crucial and should be explored. Prehospital full-scale NIHSS may increase accuracy in prehospital stroke identification and provide a common language along the stroke care chain, and it is already used successfully in prehospital settings in mobile stroke units by on-site neurologists or via telemedicine.[15] Furthermore, NIHSS was initially adapted as a tool also for non-neurologists,[7] and nurses and helicopter emergency medical service providers have been trained in NIHSS and show good interrater agreement with stroke physicians.[15,16] It is also confirmed that the training effect of NIHSS training programs remains stable over time.[17]

The NASPP (Norwegian Acute Stroke Prehospital Project) showed that anesthesiologists working in a mobile stroke unit could use NIHSS as a prehospital diagnostic tool to identify acute stroke.[18] In this follow-up study, Treat-NASPP,[19] we compared the prehospital NIHSS scores obtained by trained and certified paramedics with scores from in-hospital stroke physicians. Our aim was to investigate if NIHSS can be used as an accurate stroke severity quantification tool by paramedics in the field. As a safety measure, we explored if conducting a prehospital NIHSS scoring influenced the prehospital time.

Methods

The present article is reported according to the STROBE guidelines. Data supporting these analyses are available from the corresponding author upon reasonable request. This prospective, cohort study is a part of Treat-NASPP, a single medical center study conducted in the catchment area of Østfold Hospital Kalnes, Østfold county, Southeast Norway.[19] Treat-NASPP started May 15, 2017 and was finalized March 27, 2020 after fulfillment of the power analysis.[20] The county covers about 4000 km2 (1550 mi2), has ≈300 000 residents and 1 primary stroke center (PSC) located at Østfold Hospital Kalnes, Department of Neurology. The nearest comprehensive stroke center, Oslo University Hospital, is 90 km (55 mi) North of the PSC. The Department of Prehospital Services at Østfold Hospital Kalnes includes all 5 ambulance stations in the county. The ambulance stations are situated from 7 to 50 km (5–31 mi) from the PSC (Figure S1). As part of the Treat-NASPP study, patient baseline characteristics have previously been compared between the conventional ambulance and the mobile stroke unit.[20] The median transportation time for conventional ambulance was 20 minutes in the Treat-NASPP trial,[20] and according to data from Østfold Hospital, the average transportation time from scene to hospital for a total of 35 717 acute ambulance dispatches from 2017 to 2019 was 23 minutes.

Trial Design

This is a post hoc study from the Treat-NASPP trial, and the setting and frames of this trial are previously described.[19] Ambulance dispatch for acute stroke was decided by the emergency medical communication center using the Norwegian Index for Emergency Medical Assistance.[21] Patient inclusion criteria were decided for the Treat-NASPP trial, and were age ≥18 years, nonpregnant and ongoing stroke symptoms lasting ≤4 hours. Inclusion was done consecutively. Due to logistical and economical limitations, patients were initially recruited 8 am to 8 pm 2 weeks per month except weekends and vacations including a 2-months off-period in the summer. Due to a lower inclusion rate than expected the inclusion was extended to: (1) all weekdays 8 am to 8 pm from February 2018, (2) also weekends and vacations 8 am to 8 pm from April 2018, and (3) 24 hours all-day inclusion from January 2019.

Conventional Prehospital Care

The Norwegian EMS is government-funded, and the ambulances are staffed with a 2-person crew.[22] The ambulance crew consist of emergency medical technicians and paramedics where some have additional training as nurses. All ambulance personnel are referred to as paramedics in this article.

When encountering a potential stroke patient, the standard procedure is that the paramedic performs a short patient history, a rapid assessment of vital signs and the Face Arm Speech Time test.[23] The main goal is rapid examination and transportation (load-and-go) to the nearest PSC. The standard procedure also involves the paramedic prenotifying the stroke physician at the PSC before hospital arrival whenever acute stroke is suspected. The stroke physician activates an in-hospital stroke alert gathering the stroke team. The stroke team consists of 1 radiologist, 2 radiographers, 1 biomedical laboratory scientist, 1 stroke nurse‚ 1 emergency nurse‚ and 1 or 2 stroke physicians (a resident of neurology and an experienced neurologist during the 12 hours dayshift). At hospital arrival, the routine is that the paramedics transport the patient directly to the computed tomography (CT) imaging room where the stroke team awaits. If the patient needs advanced life support, a medical acute team (an internist, an anesthesiologist, and a nurse anesthetist) is also alerted to assist. Study folders were accessible in all ambulances in the county, and all conventional paramedics (n=230) could recruit patients with suspected stroke. The study folder was handed over to the stroke physician at hospital arrival.

Cohort

Sixty-three out of 69 volunteering paramedics completed a training program and an NIHSS certification to participate in the study. The training program consisted of a 2-day theoretical and practical course in NIHSS and acute stroke assessment including simulation training (Supplemental Material). The program was 1 day with physical attendance and 1 day with web based NIHSS training and certification.[24] The NIHSS certification was compulsory before participation in the study. Participation was voluntary, and the only criteria was that the participant was an authorized ambulance worker. There was a representative and wide spread in age, education, and work experience in the NIHSS-trained paramedic group and paramedics from all stations were represented (Table S1).

When encountering a suspected acute stroke patient, the trained paramedics performed an NIHSS scoring instead of the Face Arm Speech Time test during the standard procedure. If stroke symptoms were confirmed, rapid transportation (load-and-go) to the PSC was performed. During the standard prenotification, relevant information, including the NIHSS score, was reported to the stroke physician before arrival at the hospital. The study folder including information about symptom onset and the NIHSS scoring was handed over at hospital arrival. The stroke physician repeated the NIHSS scoring immediately after arrival, before CT, and initiation of any treatment.

Safety Measure

We compared the total prehospital time and the on-scene-time between the NIHSS-trained paramedics and the conventional care paramedics. Door-to-CT time was also compared between the groups.

Stroke Severity Quantification Tool

The NIHSS is a stroke severity quantification tool containing 11 parameters and a scoring system ranging from 0 to 40 points (Tables S2 and S3), where higher scores correspond with increased stroke severity.[7] NIHSS is measured on a continuous scale. However, we also dichotomized the NIHSS scorings according to clinical relevance, comparing the prehospital and in-hospital scores according to this dichotomization. As scores ≥6 have a relatively high sensitivity for detecting LVO and is often used as eligibility criteria for endovascular thrombectomy, since endovascular thrombectomy in LVO patients with minor stroke (NIHSS <6) is reported to be nonsuperior to thrombolysis alone,[20,25-28] the scorings were dichotomized into these 2 categories: a low score category 0 to 5 (mild symptoms) and a high score category 6 to 40 (moderate to severe symptoms).

Ethics

The Norwegian regional ethics committee (REK sør-øst) approved the Treat-NASPP study (document-id: 2016/974) and approved for deferred consent, that is, retrospective consent after study inclusion. Written, informed consent was obtained from all patients or from an authorized representative or person responsible if the patient was not able to sign.

Statistical Analysis

Non-normally distributed data were analyzed with Mann Whitney U test or with Wilcoxon signed-rank test and presented as median and interquartile range (IQR). Normally distributed data were presented as mean and SD and analyzed in a paired samples t test. Categorical variables were compared using Pearson χ2 test. The agreement between prehospital and in-hospital NIHSS scores was assessed by a Bland-Altman plot where the difference between the scores are plotted against their mean, together with the limits of agreement (LoA) and their 95% CI (Figure 2).[29] The LoA should contain 95% of the expected differences in future measurement pairs. To decide the acceptable values for LoA and bias, a grading system was developed based on values reported from 3 relevant studies using raters with different education and clinical training.[18,30,31] The grading table spans from grade A to D where grade A is the highest level of agreement (Table S4). If the current study’s results are within grade C, the agreement is deemed acceptable. The differences in NIHSS were not normally distributed due to a few outliers, but the Bland-Altman plot is robust against non-normally distributed data.[29] However, a nonparametric Bland-Altman approach was performed to complement the parametric analysis.

Figure 2.

Bland-Altman plot with a mean difference (bias) in National Institutes of Health Stroke Scale (NIHSS) of 0.92 (red line) with the limits of agreement (LoA) −5.74 to 7.59 (green lines) and 95% CIs (dotted green lines) −6.75 to −4.73 and 6.58 to 8.60. The area between the LoA includes 95% of the differences between the prehospital and in-hospital NIHSS scores.

In both prehospital and in-hospital settings, it is of clinical relevance if the patient is in a low or high NIHSS score category, therefore, secondary analysis with Cohen kappa (κ) was used to test the interrater agreement for the predefined, dichotomized NIHSS categories (0–5 and ≥6). An NIHSS score variability that led to a change in the clinical category was considered clinically relevant, as this may result in altered triage and treatment options. A kappa value ≤0.2 represents poor agreement, 0.21 to 0.40 fair agreement, 0.41 to 0.60 moderate agreement, 0.61 to 0.80 good agreement, and 0.81 to 1.00 very good agreement.[32] As the exact time of NIHSS examination was rarely documented by both the paramedics and the neurologists, arrival time at patient scene and arrival time at the hospital were used as surrogates for prehospital and in-hospital time of examination. Patients with a missing prehospital or in-hospital NIHSS score were excluded from the agreement analyses. IBM SPSS version 27 and R 4.0.3 were used for statistical analyses with a statistical significance level of 0.05.

Results

In total 406 patients were enrolled, and 274 patients were eligible for inclusion (Figure 1). The NIHSS-trained paramedics included 138 patients and the paramedics performing conventional prehospital care included 136 patients (Figure 1). The median (IQR) age in the intervention group was 71 (59–81) and 74 (61–82) years in the controls, P=0.246 (Table 1). The baseline characteristics were balanced between the groups (Table 1). Prehospital and in-hospital NIHSS were available and analyzed in 130 patients (Figure 1). The prehospital NIHSS scoring was performed mean (SD) 42 (14) minutes earlier than the in-hospital scoring (Table 1), and median (IQR) transportation time was 21 (16–28) minutes.

Figure 1.

Flow chart showing enrollment by ordinary ambulance (paramedics not trained in National Institutes of Health Stroke Scale

[NIHSS]) and NIHSS ambulance (paramedics trained in NIHSS). EMS indicates emergency medical services.

Table 1.

Baseline Characteristics of Patients in the NIHSS-Trained Paramedic Group Compared With the Conventional Paramedic Group

The NIHSS score was missing in 5 patients in the NIHSS-trained paramedic group. In 4 of these patients, the paramedic documented that the absent scoring was due to agitation or severe disorientation/dementia. Three in-hospital NIHSS scores were missing due to the patients being unconscious at hospital arrival. These 3 patients had a prehospital NIHSS score of ≥6 but could not be included in the Cohen κ analysis due to the missing in-hospital scores.

The prehospital NIHSS scores were slightly higher than the in-hospital scores (Table 2), and the mean (SD) difference (bias) between the scores was 0.92 (3.40), P=0.002, which is a grade C in the grading table (Table S4). A difference between the prehospital and in-hospital score of ≤2 points was observed in 67.4% of the patients, and a difference of ≤3 points in 77.5% of the patients. The proportion of patients with a difference of ≥3 points was not significantly different between patients with confirmed AIS or ICH compared with transient ischemic attack and mimics (AIS and ICH 29% [n=15] versus transient ischemic attack 28% [n=5] versus mimics 28% [n=17], P=0.996).

Table 2.

Time From Prehospital to In-Hospital NIHSS Scoring and Difference in Prehospital and In-Hospital NIHSS Score

The Bland-Altman plot showed LoA (95% CI) ranging from −5.74 (−6.75 to −4.73) to 7.59 (6.58 to 8.60) with an LoA width of 13.33 (Figure 2). The LoA and the LoA width get a grade B in the grading table (Table S4). The nonparametric Bland-Altman analysis showed LoA ranging from −5 to 8. In secondary analyses, the interrater agreement for the clinical categories 0 to 5 (mild stroke) and 6 to 40 (moderate to severe stroke) was κ 0.58, which indicates moderate agreement (Table 3).

Table 3.

Agreement of NIHSS Clinical Groups Between Prehospital Paramedics and In-Hospital Neurologists

The median (IQR) prehospital time (onset-to-hospital) was not statistically different between the paramedics performing NIHSS and the conventional care paramedics (86 [65-128] minutes versus 84 [56-140] minutes, P=0.535; Table 1). The median (IQR) on-scene-time was not statistically different between the paramedics performing NIHSS and the conventional care paramedics (18 [13-25] minutes versus 16 [11-23] minutes, P=0.064; Table 1). The median (IQR) door-to-CT time was 10 (3–17) minutes in the NIHSS-trained paramedic group and 13 (3–17) minutes in the conventional care paramedic group, but the difference was not statistically significant, P=0.238 (Table 1).

Discussion

Trained paramedics can use NIHSS as an accurate tool for prehospital stroke severity quantification, and the prehospital on-scene NIHSS scoring did not influence the prehospital time. To our knowledge, this is the first real-life study to investigate if NIHSS may be used as an accurate stroke severity quantification tool by paramedics in the field.

The mean prehospital NIHSS score was slightly higher than the in-hospital score, but with relatively wide LoA mainly affected by a minority of the patients who showed a marked difference between the prehospital and in-hospital scores (Figure 2). Using the grading table, the LoA get a grade B which means that the agreement is acceptable and that the raters in the current study are on the same level as prehospital anesthesiologists (Table S4).[18] Importantly >75% of the cases showed only small differences between the prehospital and in-hospital scores (≤3 points). The results from the nonparametric Bland-Altman approach also support the LoA values reported in the parametric analysis. A higher prehospital NIHSS score and wide LoA are also reported in other studies.[15,18,30,31] However, we do stress that for lower NIHSS scores, smaller differences than reported in this article may indeed be of clinical importance, and for patients with higher NIHSS scores even larger differences may not change the clinical handling of the patient. The LoA should be viewed in the light of this. The interrater agreement for the dichotomized NIHSS scores between the paramedics and stroke physicians was moderate (Table 3). In a real-life acute stroke setting, a moderate agreement should be anticipated due to the time difference between the prehospital and in-hospital examinations (Table 1). Stroke symptom fluctuations are common in the first hours of onset where spontaneous improvement is more common than spontaneous worsening.[33] Spontaneous recanalization, clot progression, good collateral vessels, or collateral failure may lead to rapid changes of symptoms, and several hemodynamic, and biochemical factors are potential predictors and mechanisms.[33-35]

It is important to note that the prehospital time did not increase when NIHSS was conducted in the field. This, in combination with the agreement, supports that using NIHSS as a prehospital tool in the EMS, is both feasible and time efficient. However, the time from hospital arrival to CT (door-to-CT, Table 1) was not significantly reduced and a reason for this may be that we did not interfere with any of the in-hospital procedures. Repeating the NIHSS scoring during transportation or close to hospital arrival may reduce the need for an immediate scoring upon patient reception. In future studies, the in-hospital patient reception should be optimized to take advantage of the information provided by prehospital NIHSS.

Training prehospital personnel in NIHSS may improve the competence in identifying stroke symptoms and potentially increase the detection rate of stroke as NIHSS contains more clinical information, especially in posterior circulation strokes which often are missed by EMS personnel.[7,10,36,37] Prehospital NIHSS allows for compatible clinical information and stroke monitoring in the early and prehospital phase of the stroke care chain—a time span previously unavailable for in-hospital stroke physicians. An NIHSS scoring conducted shortly after symptom onset is moreover an important predictor of stroke severity, stroke localization, LVO, and outcome.[7-9,14,38] This information may improve the quality of prenotification information transferred between the paramedics and stroke physicians and lead to a more efficient reception of the patient and reduced time to CT and treatment.

NIHSS has the potential of being a prehospital triage tool for direct transfer to a comprehensive stroke center and a cutoff score of ≥6 has previously been used for decision of direct transfer to a comprehensive stroke center for patients assessed in a mobile stroke unit.[20] In the EMS, a low score category could indicate triage to a PSC with options for thrombolysis, whereas a high score category could indicate triage to a comprehensive stroke center with options for endovascular thrombectomy and neurosurgery. This can reduce delay from interhospital transfers which leads to earlier treatment and better outcomes.[6] We suggested a cutoff of NIHSS ≥6 based on previous reports,[25-27] but the cutoff can be set lower or higher depending on the acceptance of false positive and false negative cases.[39] Even though NIHSS is a promising clinical tool in the prehospital field, it cannot reliably identify LVO due to the relatively high number of misclassified cases.[39] Promising future strategies in improving LVO detection from an NIHSS scoring in the field could include blood biomarkers,[40,41] transcranial ultrasound,[42] or prehospital CT scanners.[43,44]

Further research is needed to explore the impact of NIHSS in prehospital stroke detection and triage. The feedback from the NIHSS-trained paramedics has been positive. They felt empowered and reported improved communication with the in-hospital stroke physicians. We have planned a follow-up study on the paramedics’ and stroke physicians’ perception of prehospital NIHSS. Optimized training systems and digital solutions are needed to implement NIHSS as a routine diagnostic tool in the EMS. An ongoing study derived from the previously described NASPP studies, Paramedic-NASPP, is introducing NIHSS in the EMS in Oslo, Norway, and will explore the effect on stroke detection and prehospital delay in a large prehospital stroke population.[45]

The study has limitations, one being the nonrandomized setup. The neurologists were not blinded for the prehospital NIHSS score which may have influenced the in-hospital scoring. Relying on previous research for grading LoA has limitations: the reported limits are point estimates for different rater categories and thus contain uncertainty, and the studies had different settings and designs.[18,30,31]

Conclusions

Trained paramedics can use NIHSS as an accurate stroke severity quantification tool in the field, and the prehospital NIHSS scoring did not increase prehospital time. Implementing NIHSS in the EMS will improve communication between paramedics and stroke physicians and enable evaluation of stroke progression already in the prehospital part of the stroke care chain. Further large-scale trials are needed to explore if NIHSS improves prehospital stroke detection, prehospital triage, and reduces the time to treatment in patients with acute stroke.

Article Information

Acknowledgments

Special thanks to all contributing ambulance personnel and to all contributing departments at Østfold Hospital. Our sincere thanks to all the participants and their relatives.

Sources of Funding

The study was funded by the Norwegian Air Ambulance Foundation.

Disclosures

None.

Supplemental Material

Expanded Methods

Figure S1

Tables S1–S4

References 18‚24‚30‚31