Impact of different methods defining post-stroke neurocognitive disorder: The Nor-COAST study.

INTRODUCTION: Post-stroke neurocognitive disorder (NCD) is common; prevalence varies between studies, partially related to lack of consensus on how to identify cases. The aim was to compare the prevalence of post-stroke NCD using only cognitive assessment (model A), DSM-5 criteria (model B), and the Global Deterioration Scale (model C) and to determine agreement among the three models.
METHODS: In the Norwegian Cognitive Impairment After Stroke study, 599 patients were assessed 3 months after suffering a stroke.
RESULTS: The prevalence of mild NCD varied from 174 (29%) in model B to 83 (14%) in model C; prevalence of major NCD varied from 249 (42%) in model A to 68 (11%) in model C. Cohen's kappa and Cohen's quadratic weighted kappa showed fair to very good agreement among models; the poorest agreement was found for identification of mild NCD. DISCUSSION: The findings indicate a need for international harmonization to classify post-stroke NCD.

INTRODUCTION

Stroke increases the risk of cognitive impairment. However, no consensus exists on how best to measure cognitive function post‐stroke, and the estimated prevalence of mild and major neurocognitive disorder (NCD) varies according to the threshold for defined abnormalities, the diagnostic criteria chosen, and how they are applied. , , , , ,

The National Institute of Neurological Disorders‐Canadian Stroke Networks (NINDS‐CSN) Harmonization Standards made a number of recommendations regarding the choice of cognitive tests, aiming for greater consistency across studies on vascular cognitive impairment (VCI). The more‐recent Stroke and Cognition consortium (STROKOG) highlighted the importance of standardizing measures and methods to improve research quality. Widely accepted definitions of major NCD, such as the 10th version of the International Statistical Classification of Diseases and Related Health Problems (ICD‐10) and the 4th edition of the Diagnostic and Statistical Manual of Mental Disorders (DSM‐IV), include memory impairment as an absolute feature, which is appropriate for Alzheimer's disease (AD) but not necessarily for VCI. , , In contrast, in the 5th edition of the Diagnostic and Statistical Manual of Mental Disorders (DSM‐5), any cognitive impairment—not necessarily memory—is sufficient to meet NCD diagnostic criteria, an approach that may be more appropriate for impairment caused by cerebrovascular disease.

In a systematic review of major NCD after stroke, rates ranged from 7.4% (95% confidence interval [CI] 4.8 to 10.0) in population‐based studies of first‐ever stroke excluding pre‐stroke major NCD, to 53.4% (95% CI 46.9 to 59.8) in hospital‐based studies of recurrent stroke including participants with pre‐stroke major NCD. , , , However, heterogeneity in the case mix explained most of this variance rather than method of dementia diagnosis. The incidence of major NCD in the first year after severe major stroke is 45 times higher than the background major NCD rate, compared to only three times higher after minor stroke. In contrast, different methods of diagnosing mild NCD post‐stroke result in widely varying rates of cognitive impairment, even within a given set of diagnostic criteria in the same set of patients. ,

Therefore, we hypothesized that, within a given patient population, models defining mild NCD would show greater variation in measured NCD rate and lower agreement than models defining major NCD. Diagnosing post‐stroke NCD based on cognitive tests alone is used in research. The recommended DSM‐5 criteria combines a requirement for neuropsychological performance with a requirement for instrumental activities of daily living (I‐ADL) function as part of the diagnosis, but these requirements are not necessarily congruent. The global deterioration scale (GDS) is a tool assessing cognitive function as well as the ability to perform daily life activities. In research settings, it can be considered to be close to a clinical assessment. Thus, this study's primary aim was to assess the prevalence of all post‐stroke NCD and, separately, mild and major NCD in the Norwegian Cognitive Impairment After Stroke (Nor‐COAST) study population using DSM‐5 and to compare that with two other methods used for classification. Further, we aimed to explore agreement among these three methods.

METHODS

Nor‐COAST, a multicenter prospective cohort study, recruited consecutive participants in five Norwegian stroke units (May 2015 to March 2017). Inclusion criteria were hospitalization with acute ischemic or hemorrhagic stroke within 1 week after symptom onset, fluency in a Scandinavian language, and age >18 years. The only exclusion criterion was an expected survival of less than 3 months. Participants unable to complete all tests due to, for example, dysphasia, poor vision or hearing, or inability to use their dominant arm were not excluded. Participants gave informed written consent; if unable to give consent, informed written consent was given by a family proxy. The study was approved by the Norwegian Regional Committee for Medical and Health Research Ethics (REK) North (REC number 2015/171). The protocol for Nor‐COAST has been published previously.

Baseline characteristics and neuropsychological assessment

Demographic characteristics and vascular risk factors were collected from medical records at the first assessment; stroke severity was assessed with the National Institutes of Health Stroke Scale (NIHSS), and ischemic stroke subtype was defined according to the Trial of Org 10172 in Acute Stroke Treatment (TOAST) classification.

Cognitive function was assessed by trained study nurses with a 30‐minute neuropsychological test battery based on NINDS‐CSN Harmonization Standards using broadly similar neuropsychological tests available and validated in Norwegian. The test battery comprised the Word List Memory and Recall Test and Verbal Fluency Test Category (animals) from the Consortium to Establish a Registry for Alzheimer's Disease (CERAD) battery , ; Verbal Fluency Test Letter (FAS) , ; Trail Making Tests A (TMT‐A) and B (TMT‐B) ; and the Montreal Cognitive Assessment (MoCA), version 7.3. In addition, cognitive function was assessed with GDS and the Ascertain Dementia 8‐item Informant Questionnaire (AD8). Activities of daily living (ADL) were assessed with the Barthel Index (BI) and functional outcome with the Modified Rankin Scale (mRS). I‐ADL was defined as the ability to manage finances (from the relevant question in AD8) and a study question to participants regarding their ability to manage their medications.

Baseline assessments were performed during hospital stays. Three‐month follow‐ups were performed at the hospitals’ outpatient clinics. For participants unable to attend, assessments were performed through telephone interviews with the participants, their caregivers, or nursing home staff with assessment of AD8, mRS, GDS, BI, information on drugs, and whether study participants were able to administer their own medications. For telephone assessments, the Telephone MoCA (T‐MoCA) was used.

Classifying cognitive status

Five of six cognitive domains cited in DSM‐5 criteria were assessed; social cognition was not measured. Complex attention was measured by TMT‐A, executive function by TMT‐B and FAS, memory by Word List Recall, language by Verbal Fluency Test Category (animals), and perceptual‐motor function by the visuospatial/executive part of MoCA (Figure 1). ,

FIGURE 1

Stepwise algorithm for evaluation of participants’ performance on the neuropsychological test battery used in models A and B. DSM‐5, Diagnostic and Statistical Manual of Mental Disorders; MoCA, Montreal Cognitive Assessment; TMT‐A, Trail Making Test A; TMT‐B, Trail Making Test B. The tests shown in Step 1 were used to evaluate performance on the neuropsychological test battery for participants with complete testing and those with incomplete testing scoring <−1.5 SD on at least one cognitive domain. Step 2, MoCA total score, was used to evaluate neuropsychological performance of the participants completing MoCA only and for those with incomplete neuropsychological testing but normal scores on completed tests

To classify cognitive status, we created three different models (Figure 2).

FIGURE 2

The three different analytic models for classifying neurocognitive disorder: Model A, based on neuropsychology alone; Model B, based on DSM‐5 and including I‐ADL impairment; and Model C, based on the GDS. GDS, Global Deterioration Scale; I‐ADL, Instrumental activities of daily living; NCD, neurocognitive disorder; SD, standard deviation

Model A was based strictly on neuropsychological test scores meeting the cognitive requirements of the DSM‐5 criteria requiring modest cognitive decline for mild NCD and a score in the range of −1 standard deviation (SD) to −2 SD. Following other studies, , , we chose −1.5 SD as the cut‐off between normal cognition and mild NCD. Participants scoring < −1.5 SD in at least one of the five cognitive domains were defined as having post‐stroke NCD, with mild NCD scoring in the range −1.5 to −2 SD and major NCD scoring ≤ −2 SD. Model A is illustrated in Figure S1 in supporting information. Published international normative data from high‐income Western countries comparable to Norway were used (Table S1 in supporting information).

Model B was based on the DSM‐5 criteria, which base diagnostic workups on both neuropsychological test scores and I‐ADL function. As in model A, participants scoring < −1.5 SD in at least one cognitive domain were defined as having post‐stroke NCD (Table S1). Major NCD was defined as post‐stroke NCD and dependency in I‐ADL; mild NCD was defined as post‐stroke NCD without impairments in I‐ADL.

Model C was based on GDS, a global measure of cognitive function. The assessors were authorized nurses carefully instructed in the use of the scale; they used all available information from cognitive and functional tests and self‐/proxy reporting, making this assessment the closest we could get to a clinical evaluation in our study. GDS was originally designed to measure cognitive decline secondary to AD but has also been shown to be valid for detecting vascular dementia. , Scores 1–2 indicated normal cognition; 3, mild NCD; and 4–7, major NCD. ,

To include participants who did not complete the entire test battery and to minimize bias from missing data, a stepwise algorithm meeting the cognitive requirements of DSM‐5 criteria was developed for use in models A and B when analyzing data (Figure 1).

HIGHLIGHTS

No consensus exists on how to best measure post‐stroke neurocognitive disorder.

In this study we compared three different methods for defining the prevalence of post‐stroke neurocognitive disorder.

The prevalence of post‐stroke neurocognitive disorder varies according to the method used to define cases.

The poorest agreement was found among models defining mild neurocognitive disorder

RESEARCH IN CONTEXT

Systematic review: The authors searched the literature using standard databases (eg, PubMed) for articles on how to measure post‐stroke neurocognitive disorder (PSNCD). The estimated prevalence of mild and major neurocognitive disorder (NCD) seemed to vary according to the threshold for defined abnormalities, the diagnostic criteria chosen, and how they were applied. We recognized that there were higher discrepancy and lower agreement for defining mild than major NCD.

Interpretation: By using three different methods for classifying NCD 3 months post stroke, we demonstrated that the prevalence of mild and major NCD varied depending on diagnostic approach. Overall agreement was better among the methods for identification of major than for mild NCD.

Future directions: Before a final consensus on the definition of PSNCD can be made, more studies assessing the reliability of different diagnostic approach are needed. There is also a need for studies validating the research criteria for PSNCD against clinical diagnosis.

Step 1 (n = 505): neuropsychological performances were based on all completed neuropsychological tests except MoCA. Participants included those with complete testing and those with incomplete testing scoring <−1.5 SD on at least one cognitive domain.

Step 2 (n = 94): neuropsychological performance was based on MoCA scores for participants completing MoCA only and for those with incomplete neuropsychological testing but normal scores on completed tests.

A consensus group of experienced dementia researchers (KE, GS, and ARØ) approved this stepwise algorithm before data were analyzed.

Statistics

Z‐scores normalized by mean and SD of the normative data (Table S1) were derived from the raw scores of the neuropsychological tests as shown in Figure 1. Lower z‐scores indicate poorer outcomes. The executive‐function domain comprised two tests. If z‐scores from both tests were available, the average was taken; otherwise, the single completed test score was used.

Single items missing in MoCA and T‐MoCA were imputed as described in the supporting information. For participants starting but not completing Trail Making Test A or B, the test result was set to 300 seconds. Other missing data were not imputed but treated as missing.

The proportions with normal cognition, mild, and major NCD were calculated, with sensitivity analyses excluding pre‐stroke major NCD, defined as a pre‐stroke GDS score of 4–7 and previous stroke. Agreement between the models was quantified using Cohen's kappa (κ), as well as positive and negative agreement for dichotomous categories. For ordinal categories with more than two categories, agreement between the models was quantified using Cohen's quadratic weighted kappa (κw). (See details in supporting information.) Data were analyzed using SPSS 25, with Extension Hub for analysis with κw.

RESULTS

Of the 815 participants included in the Nor‐COAST study, 700 were assessed at 3 months post‐stroke. Of these, 101 had missing data; 93 had missing data on neuropsychological testing, due almost exclusively to severe illness; and 8 had missing data on I‐ADL, resulting in a study sample of 599 participants (mean/SD age = 72/12 years, 257 (43%) female, mean/SD education = 12/3.8 years, mean/SD NIHSS = 3.7/4.7) assessed at a mean/SD 3.8/0.9 months from the index stroke event (Figure 3, Table 1).

FIGURE 3

Flowchart for inclusion of participants. GDS, Global Deterioration Scale; I‐ADL, Instrumental Activities of Daily Living; NIHSS, National Institutes of Health Stroke Scale; SD, standard deviation

TABLE 1

Baseline characteristics

Demographics	N = 599
Mean age, years (SD)		72	(12)
Female sex, n (%)		257	(43)
Mean education, years (SD)		12	(3.8)
Vascular risk factors, n (%)
Hypertension, n (%)	N = 599	329	(55)
Hypercholesterolemia, n (%)	N = 599	304	(51)
Current cigarette smoking, n (%)	N = 597	112	(19)
Diabetes mellitus, n (%)	N = 599	113	(19)
Mean BMI, kg/m2 (SD)	N = 567	26.1	(4.2)
Vascular disease, n (%)	N = 599
Coronary heart disease, n (%)		104	(17)
Atrial fibrillation, n (%)		140	(23)
Previous stroke, n (%)		106	(18)
Previous TIA, n (%)		27	(4.5)
Stroke subtype, n (%)	N = 599
Cerebral infarction		547	(91)
Cerebral hemorrhage		52	(8.7)
TOAST classification, n (%)	N = 529
Large‐vessel disease		56	(11)
Cardioembolic disease		123	(23)
Small‐vessel disease		119	(23)
Other aetiology		15	(2.8)
Undetermined etiology		216	(41)
Thrombolysis, n (%)	N = 542	143	(26)
Thrombectomy, n (%)	N = 547	11	(2.0)
Pre‐stroke GDS (1‐7), n (%)	N = 594
GDS = 1‐2		536	(90)
GDS = 3		36	(6.1)
GDS = 4‐7		22	(3.7)
Assessments
NIHSS (0‐42) at admittance, mean (SD)	N = 583	3.7	(4.7)
mRS (0‐6) at discharge,a mean (SD)	N = 597	2.1	(1.3)
Barthel Index (0‐100) at discharge,a mean (SD)	N = 597	89	(19)

Abbreviations: BMI, body mass index; GDS, Global Deterioration Scale; mRS, modified Rankin Scale; NIHSS, National Institutes of Health Stroke Scale; SD, standard deviation; TIA, transient ischemic attack; TOAST, Trial of Org 10172 in Acute Stroke Treatment.

At discharge or day 7 if length of stay extends beyond 7 days.

The percentage of participants defined as having normal cognition was highest in model C at 403 (67%) and lowest in models A and B at 267 (45%; Figure 4). The prevalence of mild NCD was highest in model B at 174 (29%) and lowest in model A at 83 (14%); the prevalence of major NCD was highest in model A at 249 (42%) and lowest in model C at 68 (11%).

FIGURE 4

Proportion of participants with normal cognition, mild, and major NCD three months post‐stroke, N = 599. NCD = Neurocognitive disorder. *Model A: normal cognition defined as score ≥ −1.5 SD for all cognitive domains; mild NCD defined as score in the range of −1.5 to −2 SD for at least one cognitive domain; and major NCD defined as a score ≤ −2 SD for at least one cognitive domain. †Model B: normal cognition defined as score ≥−1.5 SD for all cognitive domains; NCD defined as score <−1.5 SD for at least one cognitive domain; major NCD defined as having post‐stroke NCD with dependency in instrumental activities of daily living (I‐ADL), defined as the need for assistance in managing one's finances and/or medications. Mild NCD was post‐stroke NCD without impairments in I‐ADL. ‡Model C: evaluation based on Global Deterioration Scale (GDS); normal cognition defined as a GDS score of 1–2; mild NCD defined as a GDS score of 3; and major NCD defined as a GDS score of 4–7

Comparing the models regarding normal cognition versus all NCD, there was fair agreement among them (A/B and C; κ = 0.40 [95% CI 0.34 to 0.47]; Table 2). As expected, very good agreement was found between models A and B (κw = 0.85 [95% CI 0.83 to 0.88]) because normal cognition was equally defined. However, of 332 participants with post‐stroke NCD in model A, 249 (75%) had major NCD compared to 158 (48%) in model B (Figure 4). There was fair agreement between models A and C (κw = 0.38 [95% CI 0.32 to 0.44]) and moderate agreement between models B and C (κw = 0.52 [95% CI 0.46 to 0.58]; Table 2). The details underlying the counts in Table 2 are provided in Table S2 in supporting information.

TABLE 2

Comparison of the models A, B, and C

Comparison of Model A/B and C
	Model A/B
Model C	Normal cognition, n	Mild and major NCD, n		Total, n (%)
Normal cognition, n	242	161		403 (67)
Mild and major NCD, n	25	171		196 (33)
Total, n (%)	267 (45)	332 (55)		599
κ = 0.40 (95% CI 0.34 to 0.47) Positive agreement 0.65. Negative agreement 0.72.

NCD, neurocognitive disorder; κ, Cohen's kappa; κw, Cohen's quadratic weighted kappa.

Model C was more restrictive in defining cognitive impairment than model B, which was, in turn, more restrictive than model A (Figure 4). Of 403 participants classified with normal cognition in model C, 60% were also classified with normal cognition in models A and B (Table S2). The poorest agreement among models was seen in the classification of participants with mild NCD, as only 15% of the 128 classified with mild NCD in model C were classified with mild NCD in model A and 40% in model B. The greatest agreement was seen for the classification of participants with major NCD, as 85% of the 68 participants classified with major NCD in model C were classified with major NCD in model A and 93% in model B.

The exclusion of participants with pre‐stroke major NCD and previous strokes resulted in a slightly higher proportion of participants having normal cognition and a lower prevalence of major NCD, while the prevalence of mild NCD was stable (Figure S2 and Figure S3 in supporting information).

DISCUSSION

In this descriptive study, we aimed to assess the prevalence of all post‐stroke NCD and subtypes mild and major NCD using three different models. We showed that prevalence varied considerably among these models. Overall agreement was greater among the different methods for identification of major NCD than for mild NCD, supporting the pre‐hoc hypothesis.

To our knowledge, this is one of the few studies using DSM‐5 criteria (model B) to classify post‐stroke NCD and comparing prevalence with other methods used for classifying post‐stroke NCD. The prevalence of all post‐stroke NCD based on neuropsychological testing (models A and B) at 55% is slightly higher than that of other recent studies of post‐stroke NCD. , In these models, we found a higher proportion of major NCD and a lower proportion of mild NCD compared to the most recent review and meta‐analysis, , probably due to the stepwise algorithm developed to avoid bias from missing data, including participants unable to complete the entire neuropsychological test battery. However, the rate of major NCD in model B at 26% aligns with findings for hospital‐based studies on first or recurrent stroke including pre‐stroke dementia in another recent review and meta‐analysis.

In a recent paper comparing the prevalence of NCD classified by different criteria, Sachdev et al. showed very good agreement among DSM‐5, The International Society of Vascular Behavioural and Cognitive Disorders (VAS‐COG), and The Vascular Impairment of Cognition Classification Consensus Study (VICCCS) criteria, which all require impairment in at least one cognitive domain, and lower agreement between these criteria and DSM‐IV criteria, requiring impairment in memory in addition to one other cognitive domain. Use of the updated DSM‐512 and VAS‐COG criteria could, therefore, lead to a higher prevalence of all post‐stroke NCD compared to studies using DSM‐IV or ICD‐108 criteria, but for criteria demanding impairment in the same number of cognitive domains, the prevalence of all post‐stroke NCD is probably more similar.

Furthermore, the prevalence of mild and all post‐stroke NCD will obviously differ considerably based on the choice of cut‐offs. The DSM‐5 criteria define modest cognitive decline as test performance typically in the 1–2 SD range below normative mean, leaving room for interpretation; this will significantly affect prevalence. Therefore, even within DSM‐5 criteria, the prevalence of mild and all post‐stroke NCD will vary with the use of different cut‐offs. , , As we mostly used one test per cognitive domain in the present study, we chose −1.5 SD as the cut‐off, which also aligns with some other studies using DSM‐5 criteria. ,

The GDS, with similarities to clinical evaluation, was performed by experienced nurses after explicit instruction, and it showed the lowest prevalence of all post‐stroke NCD. The prevalence of major NCD based on the GDS (model C) aligns with two other recent studies , ; however, the prevalence of mild NCD is lower, possibly indicating the need for more‐comprehensive testing for classifying mild NCD.

The three models agreed fairly well regarding those with major NCD but showed less agreement regarding those with mild NCD. This supports the hypothesis that, within a given patient population, there will be greater variation between methods used to define mild NCD than in those defining major NCD, in line with the findings of systematic reviews on post‐stroke NCD , and studies of mild NCD methodology. , , Most participants classified with major NCD by the GDS were also classified with major NCD in models A and B, indicating a high specificity of this method. The discrepancy for mild and major NCD between models A and B highlights a problem with applying the DSM‐5 criteria, as the criteria have requirements for both neuropsychological performance and for I‐ADL to decide on the severity of NCD. This could be interpreted differently across different studies and affect prevalence and agreement.

The advantage of classifying NCD using neuropsychological tests alone (model A) is the avoidance of the ceiling effect of commonly used I‐ADL scales that could possibly underestimate the prevalence of major NCD, as subtle changes are difficult to detect. In contrast, using neuropsychological tests alone may also result in overestimating the prevalence of major NCD. In model B, in line with the DSM‐5 criteria, I‐ADL impairment was mandatory for major NCD, which resulted in a shift from major to mild NCD compared to model A and moved the prevalence of mild and major NCD closer to the findings of other studies. , The I‐ADL measures we used were defined only by ability to manage one's medications and finances; more extensive I‐ADL measures may have given different results as I‐ADL impairment was probably underestimated. In contrast, I‐ADL impairments may also be caused by physical rather than cognitive impairment; therefore, I‐ADL measures constructed and validated for stroke survivors should be used. However, most participants in the present study had experienced milder strokes, so this may have been less important. Based on prevalence of all post‐stroke NCD, mild, and major NCD in other studies, our findings support the classification of post‐stroke NCD based on both neuropsychological tests and I‐ADL measures.

Major strengths of the present study were its multicenter design, providing a fairly representative stroke population, and the use of recommended robust tests for stroke patients. Another strength is the stepwise algorithm developed to avoid bias from missing data, allowing inclusion of participants unable to complete the entire test battery.

The study also has several limitations. The lack of a stroke‐free control group made it difficult to evaluate the extent to which the measured post‐stroke NCD was greater than expected in the background population. Additionally, cognitive domains were assessed using a limited number of neuropsychological tests; only one test in most domains that may have overestimated the impairments, but lengthy batteries are often poorly tolerated by frail older patients and may result in selection bias underestimating the impairments. In line with DSM‐5 criteria, we included measures of I‐ADL, but this was defined only by ability to manage one's medications and finances, probably underestimating the I‐ADL impairments.

CONCLUSION

In this study, the prevalence of mild and major NCD varied depending on diagnostic approach. Overall agreement was better between the different methods for identification of major NCD than for mild NCD, supporting our hypothesis. The present study shows that there is need for more research with focus on validating research diagnosis against clinical diagnosis of post‐stroke NCD. Data collected for research are more limited than the information used in clinical diagnostic work‐up on patients’ cognitive status, on the other hand making clinical diagnosis in large research studies not feasible. Issues remain in the interpretation and application of methods for classifying post‐stroke NCD. The DSM‐5 criteria are not specific enough regarding which cut‐off values for impairments in cognitive tests should be applied and to decide on the severity of NCD. Furthermore, I‐ADL measures associated with cognitive impairment in a stroke population need to be better defined.

We recommend using the combination of neuropsychological tests and a valid measure of I‐ADLs when classifying post‐stroke NCD.

CONFLICT OF INTEREST

The authors declare they have no competing interests. ABK and IS have been investigators in the drug trial Boehringer‐Ingelheim 1346.0023, and ABK has also been an investigator for Roche BN29553.

Supplementary information

Click here for additional data file.