Diagnostic Accuracy of the Explicit Diagnostic Criteria for Transient Ischemic Attack: A Validation Study.

Background and Purpose- The clinical diagnosis of a transient ischemic attack (TIA) can be difficult. Evidence-based criteria hardly exist. We evaluated if the recently proposed Explicit Diagnostic Criteria for TIA (EDCT), an easy to perform clinical tool focusing on type, duration, and mode of onset of clinical features, would facilitate the clinical diagnosis of TIA. Methods- We used data from patients suspected of a TIA by a general practitioner and referred to a TIA service in the region of Utrecht, the Netherlands, who participated in the MIND-TIA (Markers in the Diagnosis of TIA) study. Information about the clinical features was collected with a standardized questionnaire within 72 hours after onset. A panel of 3 experienced neurologists ultimately determined the definite diagnosis based on all available diagnostic information including a 6-month follow-up period. Two researchers scored the EDCT. Sensitivity, specificity, and predictive values of the EDCT were assessed using the panel diagnosis as reference. A secondary analysis was performed with modified subcriteria of the EDCT. Results- Of the 206 patients, 126 (61%) had a TIA (n=104) or minor stroke (n=22), and 80 (39%) an alternative diagnosis. Most common alternative diagnoses were migraine with aura (n=24; 30.0%), stress related or somatoform symptoms (n=16; 20.0%), and syncope (n=9; 11.3%). The original EDCT had a sensitivity of 98.4% (95% CI, 94.4-99.8) and a specificity of 61.3% (49.7-71.9). Negative and positive predictive values were 96.1% (86.0-99.0) and 80.0% (75.2-84.1), respectively. The modified EDCT showed a higher specificity of 73.8% (62.7-83.0) with the same sensitivity and a similar negative predictive value of 96.7%, but a higher positive predictive value of 85.5% (80.3-89.5). Conclusions- The EDCT has excellent sensitivity and negative predictive value and could be a valuable diagnostic tool for the diagnosis of TIA.

The diagnosis of transient ischemic attack (TIA) can be notoriously difficult, mainly because it is often solely based on history taking. Patients suspected of a TIA require an urgent assessment with timely start of antithrombotic therapy to reduce the risk of an early ischemic stroke.[1] However, even after careful evaluation at a TIA service, the final diagnosis made by the neurologist often holds a degree of uncertainty.[2] Both excluding and confirming a TIA can be difficult and, therefore, underdiagnosis as well as overdiagnosis are common.

Strict criteria for the diagnosis of TIA do not exist. In clinical practice and research, the diagnosis is usually at the discretion of the treating physician without specific requirements with respect to the type of neurological symptoms and deficits. Attempts have been made to facilitate the diagnosis of TIA by creating a diagnostic score based on multivariable logistic regression modeling.[3,4] Yet, these scores did not find their way to the clinic, probably because they are not feasible in clinical practice considering the large number of items included in the scores. Most importantly, these scores are not sufficiently accurate to confirm or rule out TIA.[5]

Skeptics state that possible symptoms and signs of a TIA are too heterogeneous to create a useful diagnostic score. Still, recently, 2 of us (E.R. Lebedeva and J. Olesen) developed a set of Explicit Diagnostic Criteria for TIA (EDCT) based on clinical practice and experience instead of statistical methods.[6] The criteria were originally developed with a focus on the discrimination between TIA and migraine with aura, one of its most common mimics. As a first step, the performance of these criteria was evaluated in separate cohorts of patients with TIA or with migraine with aura. EDCT correctly classified 99% of TIAs and 95% of migraine with aura cases (as non-TIA). The criteria have, however, not been validated in the clinically relevant domain of patients suspected of TIA. In case the EDCT correctly classifies those with and without TIA, diagnostic management of these patients could be improved considerably. Thus, we evaluated the diagnostic value of EDCT criteria in patients with suspected TIA.

Methods

The authors declare that all supporting data are available within the article.

The criteria of the EDCT are summarized in Table 1. We used the MIND-TIA (Markers in the Diagnosis of TIA) cohort to validate the accuracy of the EDCT.[7] This cohort consists of 206 patients suspected of a TIA by their general practitioner (GP), who were evaluated between September 2013 and September 2016, in the region of Utrecht, the Netherlands. All participants were referred by their GP to a regional TIA service for evaluation by a neurologist and ancillary investigations, including brain imaging. Signs and symptoms were recorded with a standardized questionnaire filled out by a research nurse within 72 hours after onset (see the online-only Data Supplement). In addition, a taped narrative of the patient was collected. Thus, a predefined set of variables could be obtained per participant.

Table 1.

Original EDCT and the Modified Subcriteria C1, C2, and C3[6]

Assessment of the EDCT

The data gathered in the MIND-TIA study provided all necessary information for classification according to the criteria of the EDCT. For each participant, we double-checked the data retrieved from the standardized questionnaire with the correspondence of the consulting neurologist and the of GP (D. Veluponnar). In case any doubt about the scoring or discrepancy between the results of the research nurse’s interview and the correspondence, a second researcher (Dr Dolmans) also made a judgment. If there was discrepancy between the 2 researchers, a third researcher (Dr Kappelle) was asked for the majority vote. Also, one of us (Dr Dolmans) checked the scoring of (1) all cases in which the EDCT came to another diagnosis than the expert panel standard and (2) a random sample of 20% of all cases.

We also included the cases that had a final diagnosis of minor disabling stroke. These cases of minor strokes had to fulfill the essential criteria A, C, and D but were allowed to not fulfill criteria B (duration <24 hours) and E (absence of infarction on imaging; Table 1).

Panel Diagnosis

An expert panel consisting of 3 experienced stroke neurologists (Dr Nederkoorn, Dr van Dijk, and Dr Kappelle) determined the definite diagnosis, using all information from the following: (1) the standardized questionnaire; (2) a taped patient’s narrative of the event; (3) the correspondence of the GP; (4) the discharge letters from the treating neurologist, and other specialists if attended; (5) the results of the ancillary investigations, including brain imaging (computed tomography, magnetic resonance imaging, or both); and (6) a 6-month follow-up period.

The expert panel determined whether patients had a TIA or minor disabling stroke, or an alternative diagnosis, applying the time-based definition of TIA.[8] The panel members first assessed all cases individually and estimated the probability of a TIA on a visual analogue scale. Consensus on the diagnosis of TIA was assumed if all 3 neurologists similarly scored the probability of TIA ≤20% or ≥80%. All other cases were discussed in a panel meeting, and a final judgement was based on a majority of votes.

Data Analysis

We assessed the diagnostic accuracy of the EDCT (sensitivity, specificity, predictive values and c-statistic, with 95% CIs), with the panel diagnosis as reference standard.

During the process of scoring, we recognized certain patterns in the assessment of the C-criterion that led to cases falsely identified as TIA. Therefore, and also to reduce the chance of misinterpretation by the user, we rephrased the original subcriteria C1-C3 (describing an onset in full intensity [C1], symptoms occurring simultaneously [C2], and the presence of actual neurological deficits), so that these apply to all symptoms instead of one or some of the symptoms (Table 1). As a secondary analysis, we also assessed the performance of this modified EDCT.

Ethical Approval

The MIND-TIA study has been approved by the Medical Research Ethics Committee of the University Medical Center of Utrecht, the Netherlands. All participants gave written informed consent.

Results

Of the 206 patients suspected of TIA by their GP, 126 (61%) participants had a TIA (n=104) or minor disabling stroke (n=22), and 80 (39%) patients had an alternative diagnosis according to the expert panel. Mean (SD) age was 67.7 (13.7) years and was higher among those with TIA/minor stroke than in those with an alternative diagnosis (71.4 [12.0] versus 62.0 [14.2]; Table 2).

Table 2.

Characteristics of 206 Patients Suspected of TIA by the General Practitioner, According to Those With a Final Diagnosis of TIA or Minor Stroke and Those With an Alternative Diagnosis

Migraine with aura was the most common alternative diagnosis (n=24; 30.0%), followed by stress-related or somatoform symptoms (n=16; 20.0%) and syncope (n=9; 11.3%; Table 3).

Table 3.

Definite Diagnoses in 80 Patients With No TIA or Minor Stroke According to the Expert Panel

The EDCT classified 155 (75.2%) as TIA or minor stroke. There were 2 false negative cases (1.6% of TIA/minor stroke), that is, cases that had no TIA/minor stroke according to the EDCT but were classified as a TIA by the panel. Both cases had diplopia as the primary symptom. These 2 participants with a false negative EDCT did not suffer from a recurrent cerebrovascular event (TIA nor stroke) during the 6-month follow-up period. Thirty-one cases (38.8% of those with an alternative diagnosis) were false positive, that is, cases that fulfilled the EDCT criteria but were judged as no TIA (or minor stroke) by the panel. The diagnoses among these 31 false positive patients are shown in Table 4.

Table 4.

Final Diagnosis in Those Patients With a False Positive Test Outcome of the Original EDCT and the Modified EDCT

Table 5 shows an overview of the diagnostic accuracy of the EDCT. The original EDCT had a sensitivity of 98.4% (95% CI, 94.4–99.8) and a specificity of 61.3% (95% CI, 49.7–71.9). Negative and positive predictive values were 96.1% (95% CI, 86.0–99.0) and 80.0% (95% CI, 75.2–84.1), respectively.

Table 5.

Diagnostic Accuracy of the Original EDCT and the Modified EDCT

Reassessment of the EDCT after modification of the C-criterion resulted in 10 less false positive patients (21 instead of 31). These included 9 patients diagnosed with migraine with aura, and 1 with stress-related/somatoform symptoms. The number of 2 false negative patients remained unchanged. The modified EDCT had a specificity of 73.8% (95% CI, 62.7–83.0) and a sensitivity of 98.4% (95% CI, 94.4–99.8). The negative and positive predictive values were 96.7% (95% CI, 86.0–99.0) and 85.5% (95% CI, 80.3–89.5), respectively.

Separate analyses of the 22 patients with a minor disabling stroke did not substantially change the results (data not shown).

To assess interobserver variability, a second researcher (Dr Dolmans) also scored the EDCT for all false positive and negative cases (according to the assessment of the first researcher, D. Veluponnar) and a 20% random sample of all 206 patients. For the modified EDCT, there was agreement on the 2 false negative cases and the random sample and disagreement on 1/22 false positive cases. A third researcher (Dr Kapelle) assessed the modified EDCT of the false positive and negative cases and came to the same results (100% agreement) as the second researcher.

Discussion

This first evaluation of the diagnostic accuracy of the EDCT in patients suspected of TIA demonstrates that the criteria have an excellent sensitivity (98.4%) and negative predictive value (96.1%). Moreover, modification of the EDCT by rephrasing the C-criteria resulted in a similar negative predictive value but in an increase in positive predictive value.

In the primary care setting, it is most valuable if a tool can safely exclude a TIA, which requires a high negative predictive value. If a GP would use the modified EDCT in 100 patients suspected of TIA (with a prior chance of a TIA/minor stroke of 61%), 71 patients would be referred to a TIA service and as a result 60 confirmed as TIA and 11 would receive another final diagnosis after evaluation by the neurologist. Among the 29 patients in whom the GP would make another diagnosis, only 1 patient would wrongly not receive the diagnosis TIA. Both false negative cases in our study had diplopia, which could mean that the EDCT is more reliable to diagnose a TIA in the anterior than in the posterior circulation.

Two previous diagnostic scores that aim to facilitate clinicians were developed based on regression analysis. The Dawson score consists of 9 determinants, including age and history of hypertension, supplemented with 7 symptoms.[3] The Diagnosis of TIA (DOT) score consists of 17 determinants, including age, history of hypertension, history of or actual atrial fibrillation, supplemented with 14 specific symptoms.[4] Both the Dawson and the DOT scores are not widely used and have not been established as a useful tool in clinical practice nor in research. The Dawson score had poor diagnostic value when applied by GPs (c-statistic 0.70).[5] The DOT score performed better in a direct comparison with the Dawson score in a cohort of 525 suspected TIA patients seen at a British TIA service (c-statistic 0.89 [0.85–0.92] versus 0.83 [0.79–0.87]).[4] However, this comparison was performed in the derivation cohort of the DOT score and, therefore, very likely overestimates the performance of the DOT score. Comparing our results with these studies and considering that the EDCT is just based on clinical experience, the overall discriminative ability of the EDCT in this external validation is remarkably high.

One might argue that a purely clinical score is not necessary anymore in the modern era of sensitive neuroradiological methods such as diffusion-weighted magnetic resonance imagine or perfusion computed tomography-scanning.[9,10] This is true in a hospital setting in developed countries, but imaging cannot always help to distinguish between TIA and the most common mimics. In addition, it does not apply at all in a primary care setting or in non-Western countries.

In the current study, we found a lower specificity of the EDCT than in the first study in which he EDCT was tested in separate cohorts of patients with migraine with aura or with a TIA. Testing EDCT in a large cohort of patients with migraine with aura including many who had aura without headache showed a specificity of 95%, whereas in the present study migraine was the most common false positive diagnosis. This difference can be explained by the fact that the patients with migraine in the MIND-TIA study were all initially suspected of a TIA by the GP and could therefore be considered to be more profound mimics of TIA. The quality of the collected information about characteristics of migraine might have been better if the investigator would have had the EDCT at hand during data collection. This should be tested in further prospective studies. In the current form, EDCT is excellent for screening patients for research projects because of a very high sensitivity. Before inclusion in TIA trials, the diagnosis must, however, be refined by expert evaluation.

Our study is the first to assess the EDCT among patients suspected of TIA by their GP. Strong points are the standardized way of collecting the required information and the completeness of data and the standardized way in which we assessed the TIA diagnosis by an expert panel. The use of an expert panel as the reference standard can, however, also be criticized. Although the panel consisted of experienced neurologists, they had to make a diagnosis on the basis of written information and did not speak to the patients themselves. One might also argue that neurologists on a regular basis disagree about the diagnosis of TIA.[2] However, in the absence of better alternatives, we feel that the use of consensus meetings by a panel of experts is the best available option for the reference standard. Initial history taking was performed by a trained nurse and not by a medical specialist. This is different from most clinical practices, but a standardized questionnaire guaranteed objective and straightforward information about the symptoms and signs of the patients. Another limitation of our study is that the modification of EDCT was based on our MIND-TIA data and that we also validated that score in the same dataset. Thus, another external validation in a larger cohort is needed.

Finally, the actual usability in clinical practice and the performance of the score, and the modified score, when applied by GPs or physicians at an emergency department is unknown at this point. There may be differences between a structured nurse interview and everyday history taking by a GP or Emergency physician. We, therefore, recommend to perform an implementation study in primary care and emergency department setting as the final step before use in everyday practice by GPs.

In conclusion, this study showed that the original, and especially the proposed modified EDCT are easy to apply, and have excellent diagnostic properties in patients suspected of TIA in primary care. They could be a valuable diagnostic tool for use in primary care and emergency departments as well as being a valuable supplement in TIA clinics.

Appendix

The MIND-TIA Study Group L.S. Dolmans, MD (Julius Center for Health Sciences and Primary Care, University Medical Center Utrecht, Utrecht University, Utrecht, the Netherlands); M.E.L. Bartelink, MD, PhD (Julius Center for Health Sciences and Primary Care, University Medical Center Utrecht, Utrecht University, Utrecht, the Netherlands); F.H. Rutten, MD, PhD (Julius Center for Health Sciences and Primary Care, University Medical Center Utrecht, Utrecht University, Utrecht, the Netherlands); A.W. Hoes, MD, PhD (Julius Center for Health Sciences and Primary Care, University Medical Center Utrecht, Utrecht University, Utrecht, the Netherlands); L.J. Kappelle, MD, PhD (Department of Neurology, University Medical Center Utrecht, Utrecht University, Utrecht, the Netherlands); E.J. van Dijk, MD, PhD (Department of Neurology, Radboud University Medical Center, Nijmegen, the Netherlands); P.J. Nederkoorn, MD, PhD (Department of Neurology, Amsterdam UMC, University of Amsterdam, Amsterdam, the Netherlands); S. van Delft, PhD (Saltro Diagnostic Center for Primary Care, Utrecht, the Netherlands); G.J. Seppenwoolde (Saltro Diagnostic Center for Primary Care, Utrecht, the Netherlands).

Disclosures

None.