Clinical Prediction Rules to Classify Types of Stroke at Prehospital Stage.

Background and Purpose- Endovascular therapy is effective against acute cerebral large vessel occlusion (LVO). However, many patients do not receive such interventions because of the lack of timely identification of the type of stroke. If the types of stroke (any stroke, LVO, intracranial hemorrhage [ICH], and subarachnoid hemorrhage [SAH]) were to be predicted at the prehospital stage, better access to appropriate interventions would be possible. Japan Urgent Stroke Triage (JUST) score was clinical prediction rule to classify suspected patients of acute stroke into different types at the prehospital stage. Methods- We obtained information for signs and symptoms and medical history of consecutive suspected patients of acute stroke at prehospital stage from paramedics and final diagnosis from the receiving hospital. We constructed derivation cohort in the historical multicenter cohort study from June 2015 to March 2016 and validation cohort in the prospective multicenter cohort study from August 2016 to July 2017. The derivation and the validation cohorts included 1229 and 1007 patients, respectively. We constructed multivariate logistic regression models with 21 variables to develop clinical prediction rules, which distinguish between different types of stroke: any stroke, LVO, ICH, and SAH. Results- Among the 1229 patients (median age, 72 years; 55% men) in the derivation cohort, 533 stroke, 104 LVO, 169 ICH, and 57 SAH cases were observed. The developed rules showed that the areas under the receiver operating curves were 0.88 for any stroke, 0.92 for LVO, 0.84 for ICH, and 0.89 for SAH. The validation cohort of 1007 patients (median age, 75 years; 56% men) showed that the areas under the curves of any stroke, LVO, ICH, and SAH were 0.80, 0.85, 0.77, and 0.94, respectively. Conclusions- These clinical prediction rules can help paramedics classify the suspected patients of stroke into any stroke, LVO, ICH, and SAH groups with excellent accuracy.

Endovascular therapy (EVT) and intravenous tPA (tissue-type plasminogen activator) have become the standard therapy for acute cerebral infarction[1] and are commonly used in the developed countries.[2,3] However, the time constraints of EVT and tPA, of 8 and 4.5 hours, respectively, were a hindrance to applying these approaches to all patients.[4,5] For example, <5% of the patients with acute cerebral infarction receive tPA,[6] and many patients with acute cerebral large vessel occlusion (LVO) did not receive timely EVT because of time restriction or the lack of capable operators.[7] The importance of timely treatment for intracranial hemorrhage (ICH) or subarachnoid hemorrhage (SAH) is similar because the delay in surgical or other interventions is associated with poor prognosis.[8,9]

Therefore, a system of promptly transferring the patients with suspected stroke, or appropriate placement of capable physicians, is needed to improve the low success rate of the state-of-the-art techniques. Rapid diagnostic procedures after arrival at the hospital and shortening of the treatment duration would undoubtedly increase the access to the treatment. In addition, the prehospital estimation of the likelihood of stroke and prediction of the type of stroke should also improve. Several reports on prehospital prediction rules exist for any stroke,[10-12] LVO,[13-19], and SAH,[20] but no such rules exist for ICH. Moreover, these prediction rules dealt with either only one type of stroke or with any stroke in general, without any classification. From the perspective of patients or paramedics, the clinical prediction rules, which simultaneously apply to all types of stroke, are needed. Therefore, we conducted a historical and prospective cohort study to develop the Japan Urgent Stroke Triage (JUST) score, which predicts any stroke, LVO, ICH, and SAH in patients suspected to have acute stroke by paramedics.

Methods

Study Design and Population

We conducted a historical multicenter registry study to develop clinical prediction rules for stroke, followed by a prospective validation based on prospective multicenter registry from 8 centers in Japan. The institutional review boards of all the participating centers approved the protocol. Written informed consent from each patient was waived for this study because we used information obtained during routine clinical practice. Institutional review boards approved this exemption, in accordance with the Ethical Guidelines for Medical and Health Research Involving Human Subjects in Japan. The data, analytic methods, and study materials will not be made available to other researchers for purposes of reproducing the results or replicating the procedure.

The 2 registries provided 2 cohorts: a derivation cohort and a validation cohort. The derivation cohort consisted of patients examined from June 2015 to March 2016, and the validation cohort consisted of those examined from August 2016 to July 2017. Consecutive patients, whom the paramedics suspected to have stroke were considered for enrollment. We enrolled patients transferred to any of the participating centers and excluded those who did not receive diagnostic investigation based on either computed tomography (CT) or magnetic resonance imaging (MRI).

Measurements and Data Collection

Certified paramedics assessed the 29 predetermined potentially predictive variables consisting of symptoms or signs that indicated stroke, as well as the medical history. The medical history included age, sex, smoking status, history of stroke (cerebral infarction, cerebral hemorrhage, and SAH), and medication (warfarin, direct oral anticoagulant, and antiplatelet). The variables were onset (sudden or not), improvement after the onset, progression after onset, and presence of related symptoms (headache, numbness, dizziness, convulsion, and nausea or vomiting). The signs included systolic and diastolic blood pressure and related signs (arrhythmia, disturbance of consciousness, anisocoria, aphasia, dysarthria, conjugate deviation, unilateral spatial neglect, facial palsy, and paralysis of upper and lower limbs). The uncertain or undetectable (missing) symptoms and signs were considered null for the quick and convenient assessment of predictive variables and clinically plausible prediction rules. These data were recorded on paper or the web and transferred to the electronic data capture for analyses.

At the transferred centers, either neurologists or neurosurgeons conducted standardized care for the patients and confirmed the diagnosis with CT or MRI. If the resulting diagnosis differed from the initial one, the later diagnosis was considered final. If any diagnosis was uncertain, neurosurgeons who were not aware of the potential risk factors scrutinized the imaging data and clinical courses and discussed to reach consensus. These data were recorded as electronic medical records and transferred to electronic data capture.

Definition of Outcomes

Stroke was defined as any acute neurological symptom, such as paralysis of face or limbs, or disturbance of consciousness, in addition to confirmation from abnormal findings from CT, MRI, CT angiography, or magnetic resonance angiography, which were consistent with the symptoms. Transient ischemic attacks were excluded. Once stroke was confirmed, it was classified into the following subcategories of stroke, based on the findings from imaging: LVO was defined as occlusion of the cerebral main artery, detected by CT angiography, magnetic resonance angiography, or cerebral angiography, with a low-density area with CT or a low-intensity area with diffusion-weighted MRI when CT and diffusion-weighted MRI did not show early ischemic lesions in the corresponding area. The physicians in charge determined the LVO based on occlusions of the cerebral main artery; ICH was defined as a high-density area in CT or a high-intensity area in MRI images, indicating bleeding in the brain parenchyma; SAH was defined as a high-density area in CT, or a high-intensity area in MRI with fluid-attenuated inversion recovery, in the subarachnoid space; ICH with SAH accompanied by rupture of the cerebral aneurysm was classified as SAH. When SAH was suspected without apparent MRI findings, cerebrospinal fluid examination was performed to detect xanthochromia or red blood cells. However, such cases were not observed in the derivation and validation cohorts. All patients who were diagnosed to have any stroke received any angiography, including CT angiography, magnetic resonance angiography, or cerebral angiography. However, some patients without apparent vascular lesions, such as ICH or lacunar strokes, did not receive the angiography. These outcomes were established a priori.

Statistical Analysis

We presented the number and percentage for categorical variables and median and interquartile ranges for continuous variables. Comparisons of variables between derivation and validation cohorts were conducted by χ2 test for categorical variables and the Wilcoxon rank-sum test for continuous variables.

We dichotomized age and blood pressure to ensure that the final model did not contain any continuous variable so that clinicians could categorize patients as high- or low-risk without performing any calculations. We dichotomized ages ≥75 years and <75 years based on the criteria for advanced age used by the Japanese health insurance system. We distinguished between hemorrhagic and ischemic strokes because the method of initial treatment is different. We, thus, dichotomized systolic and diastolic blood pressure above and below 165 and 95 mm Hg, respectively, based on a previous report, which showed that systolic blood pressure ≥165 mm Hg and diastolic blood pressure ≥95 mm Hg were significantly associated with ICH.[8]

We constructed univariate logistic models to assess the strength of the association between each of the 29 potentially predictive variables and the 4 outcomes for the derivation cohort. We then constructed multivariate logistic regression models to predict the 4 outcomes in the same cohort. Variables found to be associated with P values <0.20 in the univariate models were included in the multivariate models. Applying the backward model selection procedure to eliminate the variables with higher P values, we constructed multivariate logistic regression models using variables with P values <0.05 for the 4 outcomes.

The results of the multivariate logistic regression models were then used to develop clinical prediction rules.[21] Each β coefficient was divided by the smallest β coefficient and rounded to the nearest integer, for each model. The risk score for each patient was determined by assigning points for each variable present and summing them.

We applied these clinical prediction rules on the validation cohort. The calculated risk score for each patient was compared with the actual outcomes. The discriminatory performances of the rules were assessed by analyzing the receiver-operating characteristic curves obtained for the derivation and validation cohorts.[22] We calculated the areas under the curves (AUCs), for each rule, for the derivation and validation cohorts, and compared the results from both the cohorts. The resulting continuous distribution of each risk score, from all patients in the validation cohort, was then stratified into 5 categories that were grouped according to the level of probability.

To explore the utility of the developed clinical prediction rules, we applied the Cincinnati Prehospital Stroke Scale (CPSS),[10] the Rapid Arterial Occlusion Evaluation (RACE) scale,[15] and the Field Assessment Stroke Triage for Emergency Destination (FAST-ED) scale[17] and calculated the AUCs for LVO. For comparisons with RACE and FAST-ED scales, we assigned both 1 point and 2 points to all variables because we did not differentiate the grade of variables.

All statistical analyses were conducted by a physician (K. Uchida) and study statistician (T. Morimoto) using JMP 13.0 (SAS Institute, Inc, Cary, NC) or SAS 9.4 (SAS Institute, Inc, Cary, NC). The level of statistical significance was 0.05, and all hypothesis tests were 2 sided.

Sample Size

We estimated the sample size based on the objective of developing clinical decision rules. To make these rules developed by multivariate logistic models reliable, we estimated that at least 50 cases for each type of stroke would be necessary. Prevalence of SAH has been reported to be lowest, and comprises of 10% of all strokes,[23] and generally, only half of the patients with neurological symptoms actually experience stroke, based on local data. Assuming 10% of all stroke cases to be SAH and 50% of the patients experiencing stroke, out of all at-risk cases, at least 1000 patients were needed in each of the derivation and validation cohorts. The study periods were thus determined to fulfill the sample size.

Results

Derivation Cohort

The derivation cohort included 1229 patients. The median age was 72 years, and 55% of the patients were men (Table 1). Fifty-eight percent of patients exhibited sudden onset of symptoms, and 1 out of 3 patients had either dysarthria or paralysis of upper or lower limbs. Among the 1229 patients who were suspected to have stroke, 533 patients were confirmed as having stroke, including 104 LVO, 169 ICH, and 57 SAH cases (Figure 1).

Table 1.

Patients Characteristics for the Derivation and Validation Cohorts

Figure 1.

Study flowchart. ICH indicates intracranial hemorrhage; LVO, large vessel occlusion; and SAH, subarachnoid hemorrhage.

The univariate analyses revealed 22 among the 29 potentially predictive variables to be associated with any stroke (Table 2). Similarly, 22, 22, and 23 variables were associated with LVO, ICH, and SAH, respectively (Table 2). Patients exhibiting improvement after onset or dizziness were less likely to have stroke.

Table 2.

Factors Associated With Any Stroke, LVO, ICH, and SAH

Multivariate logistic regression models showed 21 variables to be independently associated with any stroke, LVO, ICH, or SAH. Table 3 shows the final models in which each β coefficient was rounded to obtain the risk score. Headache was significantly associated with presence of SAH but with the absence of LVO. Similarly, arrhythmia was significantly associated with the presence of LVO but the absence of ICH.

Table 3.

Final Prediction Rule

The receiver-operating characteristic analyses showed that the AUCs for any stroke, LVO, ICH, and SAH were 0.88, 0.92, 0.84, and 0.89, respectively (Figure 2).

Figure 2.

Receiver-operating characteristic curve. A, Any stroke, P<0.001. B, Large vessel occlusion, P=0.002. C, Intracranial hemorrhage, P=0.005. D, Subarachnoid hemorrhage, P=0.06. AUC indicates area under the curve.

Validation Cohort

The validation cohort consisted of 1007 patients. The median age was 75 years, and 56% of the patients were men (Table 1). History of cerebral infarction was more common than in the derivation cohort (Table 1). The systolic blood pressure was higher than that observed for the derivation cohort, but diastolic blood pressure was similar, and arrhythmia was twice as prevalent in the validation cohort. Stroke was detected in 617 patients, with 131 LVO, 183 ICH, and 50 SAH cases (Figure 1).

When applying the clinical prediction rules obtained from the derivation cohort to the validation cohort, AUCs in the receiver-operating characteristic analyses were 0.80 for any stroke, 0.85 for LVO, 0.77 for ICH, and 0.94 for SAH (Figure 2). The probabilities of the 4 outcomes, according to the categorized risk scores, showed excellent stratification for patients with different types of stroke, ranging from low probability to high probability (Figure 3). For example, when score for LVO was ≥8, the positive predictive value was 79% (30/38; Figure 3B). However, when score for SAH was ≤−3, the negative predictive value was 100% (0/216; Figure 3D).

Figure 3.

Probability of stroke according to the score. A, Any stroke. B, Large vessel occlusion (LVO). C, Intracranial hemorrhage (ICH). D, Subarachnoid hemorrhage (SAH).

Comparison With Previous Scores

On applying CPSS on the validation cohort, the AUC for LVO was 0.77, whereas that for our clinical prediction rules was 0.85. When applying RACE and FAST-ED scales, the AUCs of RACE and FAST-ED scales were 0.84 to 0.85 and 0.86 to 0.87, respectively.

Discussion

Differences across the types of stroke directly influence the differences in the respective treatment approaches. The urgency for invasive treatment also differs across the types. Hospitals should prepare for the treatment of LVO, ICH, and SAH differently. Our clinical prediction rule is the first tool to simultaneously classify these types of stroke at the prehospital stage, helping to transfer the patients suspected to have stroke to the appropriate hospitals. This tool shows the ability to discriminate between any stroke, LVO, ICH, and SAH, with 21 variables in the validation cohort.

There are several reports on clinical prediction rules for acute strokes at the prehospital stage.[10-12] The CPSS was originally developed for the selection of candidates for tPA, and it predicted acute ischemic stroke with a sensitivity of 0.59 and specificity of 0.88 at the cutoff level of 1, when scored by prehospital providers, in the validation cohort.[10] The CPSS was widely used during prehospital triage because it was convenient to calculate the probability of acute ischemic stroke with 3 items. Indeed, the AUC of CPSS for our validation cohort was good enough of 0.77, compared with 0.85 for our prediction rules. However, the necessity to discriminate LVO from other ischemic strokes became greater in the era of EVT. Therefore, other prehospital clinical prediction rules for LVO were developed.[13-19] For example, the Prehospital Acute Stroke Severity scale predicted acute cerebral LVO with 3 variables.[18] The AUC for Prehospital Acute Stroke Severity scale score was 0.73, with a sensitivity of 0.61 and specificity of 0.83 at the cutoff level of 2 in the validation cohort. The AUC for our prediction rules was 0.85, with a sensitivity of 0.69 and specificity of 0.91 at the cutoff level of 5 or a sensitivity of 0.84 and specificity of 0.69 at the cutoff level of 4 in the validation cohort. The AUC for RACE scale score was 0.82 and that for FAST-ED scale score was 0.81 in their original reports.[15,17] Thus, our clinical prediction rules had the highest discriminative ability among all the prediction rules ever developed for LVO which were validated by a different cohort.

More importantly, patients who were suspected to have acute stroke might have hemorrhagic strokes and thus need different treatment approach immediately.[24] Our prediction rules with 21 variables simultaneously predict ICH and SAH with high discrimination abilities. There were only one prediction rules for SAH in patients with acute headache[20] but none for ICH. The reported clinical prediction rule for SAH had a sensitivity of 0.99 and specificity of 0.28,[20], but our rules showed a sensitivity of 0.98 and specificity of 0.56 at the cutoff level of 0. Therefore, our scale had similar sensitivity values but better specificity in addition to the ability to distinguish other types of stroke.

The biggest drawback of our clinical prediction rules is the need for 21 variables to be assessed at the prehospital stage. However, we have already attested the value of our clinical prediction rules by using it during the daily practice of paramedics, in the phase of validation cohort. It took median time of 37 seconds (interquartile range, 24–58) to input the 21 variables when the paramedics used the application on tablet devices. Most paramedics, in developed countries, nowadays use tablet devices to search for the receiving hospitals or record the status of patients. With the help of our clinical prediction rules, if the paramedics suspect the patients to have acute stroke with neurological symptoms, they can easily estimate the probability of each type of stroke and transfer the patient to an appropriate hospital without unnecessary delay when the clinical prediction rules were installed in their tablet devices (online-only Data Supplement). Even in the setting where only one hospital caters to all types of strokes, the physicians in charge could benefit from preparing for the most likely type of stroke. Such a system, with appropriate clinical prediction rules, would undoubtedly reduce the healthcare cost, including that of unnecessary imaging and the labor of healthcare professionals.

There were other potential limitations in this study. The only patients who were suspected to have stroke by the paramedics were included and transferred to the study centers. Thus, some patients of acute stroke might not have been identified by the paramedics and were excluded from the study. If such cases were prominent, the discrimination abilities might be lower. However, because we had conveyed the study purpose and design to all paramedics and physicians at the study centers in advance, the missed cases would be few in number, if any.

Another limitation might be the reliability of assessment of the predictive variables. We did not assess the reliability of scoring variables across the paramedics because it was not realistic in the setting of prehospital care. However, the variables were easily obtained from the family or witnesses of patients and simple physical examination. We also treated the uncertain or missing variables as null to make the assessment easy to conduct. Therefore, the reliability of assessment of variables should be resistant to individual variability.

Third, the time to assess the 21 variables should be considered. Although we measured the time to input of 21 variables by the application on tablet devices and found it was reasonably short, it was hard to measure the time to assess the findings in the prehospital emergency settings. Because the paramedics always took necessary time to assess the symptoms and signs at the first contact to any patient who called an ambulance, our clinical prediction rules should not delay the decision time.

Fourth, these clinical prediction rules were not perfect, and the misclassifications were not evitable in nature. However, patients suspected to have stroke had substantial chances to be transferred to hospital without appropriate therapy if there were no prehospital triages. We should minimize the misclassified patients as much as possible, and our clinical prediction rules which simultaneously discriminate 4 types of stroke substantially decreased such misclassifications compared with the practice without triage.

Finally, these clinical prediction rules were developed in Japan, and are consequently, of uncertain generalizability. However, the variables used are common findings clinically associated with the pathology of all types of stroke and generally considered as relevant risk factors. In addition, all clinical prediction rules were developed locally and then tested in other settings to establish the standard rules. Thus, our clinical prediction rules should be used to assess their utility in other settings.

Conclusions

The clinical prediction rules for suspected patients of acute stroke at prehospital stage, named Japan Urgent Stroke Triage (JUST) score, could simultaneously predict any stroke, LVO, ICH, and SAH with excellent discriminative abilities. Applying these rules to daily clinical practice should help more patients with acute stroke receive appropriate interventions, such as EVT, tPA, and surgeries, just on time. Such a system would undoubtedly save the lives and decrease the disability of patients with acute stroke.

Acknowledgments

We are indebted to Susumu Seki and HAYASHI Computer, Ltd for the development of the electronic data capture system, and the application for tablet devices. We also thank paramedics at Nishinomiya City Fire Bureau, Amagasaki City Fire Department, Odawara City Fire Department, Kobe City Fire Department, Western City of Hiroshima Fire Department, Itami City Fire Department, Firefly Office Worker Association in Hirosaki, Firefly Office Worker Association in Ajigasawa, Tachikawa Emergency, Nishikicho Emergency, Sunagawa Emergency, Akishima Emergency, Tokura Emergency, Mitsutaki Emergency, Ogami Emergency, Kodaira Emergency, Hino Emergency, Kokubunji Ambulance, Kunitachi Emergency, Kita Tama Western Emergency, Higashimurayama Emergency, Akitsu Emergency, Kumagawa Emergency, and Fuchu-honcho Emergency for their prehospital care and collection of data.

Sources of Funding

This study was supported by the Grant-in-Aid for Graduate Students, Hyogo College of Medicine.

Disclosures

Dr Uchida reports lecturer’s fees from Nihon Medi-Physics. Dr Yoshimura discloses research grants from Medtronic, Medicos Hirata, Termo, Bristol-Myers Squibb, and Otsuka; lecturer’s fees from Boehringer Ingelheim, Otsuka, Bayer, Sanofi, Pfizer, Bristol-Myers Squibb, Stryker, Medtronic, Mitsubishi Tanabe. Dr Takeuchi reports lecturer’s fees from Stryker, Kaneka, Medtronic, and Johnson and Johnson. Dr Shigeta discloses research grants from Johnson and Johnson; lecturer’s fees from Termo, Bristol-Myers Squibb; article fee from Termo, Medtronic. Dr Morimoto reports research grant from Nexis; lecturer’s fees from AbbVie, AstraZeneca, Daiichi Sankyo, Kyorin, Mitsubishi Tanabe, and Pfizer; article fee from Pfizer; advisory boards for Asahi Kasei, Boston Scientific, and Bristol-Myers Squibb. The other authors report no conflicts.