Comparative validity of informant tools for assessing pre-stroke cognitive impairment.

OBJECTIVES: Various informant-based questionnaires are used in clinical practice to screen for pre-stroke cognitive problems. However, there is no guidance on which tool should be preferred. We compared the validity of the two most commonly used informant-based tools.
METHODS: We recruited consecutively admitted stroke patients. Patients' informants completed the Informant Questionnaire for Cognitive Decline in the Elderly Short Form (IQCODE-SF, 16-item) and Ascertain Dementia 8 (AD8). We assessed construct validity (accuracy) against a semi-structured clinical interview for dementia or mild cognitive impairment (MCI), describing test accuracy metrics and comparing area under ROC curves (AUROC). We described criterion validity by evaluating associations between test scores and neuroimaging markers of dementia and overall 'brain frailty'. Finally, we described prognostic validity comparing ROC curves for 18-month clinical outcomes of dementia, death, stroke, and disability.
RESULTS: One-hundred-thirty-seven patient-informant dyads were recruited. At usual clinical cut-points, the IQCODE-SF had comparable sensitivity to the AD8 (both = 92%) for pre-stroke dementia, but superior specificity (IQCODE-SF: 82% vs. AD8: 58%). Youden index suggested that the optimal AD8 threshold for diagnosis of dementia is ≥4. The IQCODE-SF demonstrated stronger associations with markers of generalised and medial-temporal lobe atrophy, neurovascular disease, and overall brain frailty. IQCODE-SF also demonstrated greater accuracy for predicting future dementia (IQCODE-SF AUROC = 0.903, 95% CI = 0.798-1.00; AD8 AUROC = 0.821, 95% CI = 0.664-0.977).
CONCLUSIONS: Both IQCODE-SF and AD8 are valid measures of pre-stroke dementia. Higher cut points for AD8 may improve performance in the acute stroke setting. Based on consistent superiority across a range of validity analyses, IQCODE-SF may be preferable to AD8 for pre-stroke dementia screening.

INTRODUCTION

Evaluating the pre‐stroke state is recommended in stroke care guidelines. Assessments for pre‐stroke cognitive impairment can contextualise post‐stroke impairments, aid realistic goal setting, and inform the risk of short‐term (delirium) and long‐term (dementia) outcomes. Pre‐stroke issues with cognition are often undiagnosed before hospital admission with stroke and the acute stroke setting offers a valuable opportunity to identify prevalent cognitive problems.

As stroke survivors may be too unwell to participate in testing or lack insight into their impairments, ascertainment of premorbid conditions in acute stroke usually requires an interview with the patient's close relative or friend i.e., an informant. The Informant Questionnaire for Cognitive Decline in the Elderly (IQCODE) and Ascertain Dementia 8 (AD8) are the most widely used informant‐based cognitive assessment questionnaires and are used in stroke research and practice. However, at present there is a lack of evidence regarding each tools' validity in stroke and few studies that compare the tests. Establishing comparative validity is essential to support evidence‐based decision making regarding optimal tool selection. The absence of guidance on the preferred screening tool is a commonly cited barrier to adoption of routine cognitive screening in clinical practice.

The aim of this study was to compare various aspects of validity of the IQCODE and AD8 as informant‐based screening tests for assessing pre‐stroke cognitive issues. We were interested in three complementary and clinically relevant aspects of validity: the informant‐based screening tests' diagnostic test accuracy for detecting pre‐stroke dementia or mild neurocognitive disorder (construct validity); the informant‐based screening tests' association with objective neuroimaging markers of neurodegeneration, neurovascular disease, and overall ‘brain frailty’ (criterion validity), and the ability of the informant‐based screening tests to predict adverse outcomes (prognostic validity).

METHOD

We followed STARDdem (Standards for Reporting of Diagnostic Accuracy studies in Dementia) guidelines for conduct and reporting in this study.

This is a sub‐study of the ‘APPLE’ (Assessing Psychological Problems in stroke: A longitudinal Evaluation) project—a multicentre, prospective comparative test accuracy study embedded within the UK National Health Service. A summary of our methodology is provided below. Comprehensive details of the APPLE methodology can be seen in the study protocol. Ethical approval was obtained for all participating sites (REC number: 16/SS/0105).

Setting

The APPLE project recruited consecutive stroke and transient ischaemic attack (TIA) patients admitted to 11 participating hospitals across the UK. The study ran from November 2016 to February 2019, with each hospital site contributing participants for a variable period. Participating sites offered hyper‐acute stroke services and admitted all patients with suspected stroke or TIA.

Population

We approached and consented informants of patients recruited into the APPLE project. Stroke and TIA were diagnosed by the treating clinician in accordance with the World Health Organisation definition. We operated no restrictions on age, stroke‐type, stroke‐severity, or comorbidity. While we applied no restrictions on time since stroke, sites were encouraged to recruit within the first week following admission. The only exclusions were if the treating clinical team thought that any form of cognitive assessment was inappropriate, or if the informant was unable to speak English. Where possible, the patient identified their preferred informant. Patients unable to consent (e.g. due to severe aphasia or cognitive impairment) were still included with agreement of a relevant proxy decision maker.

Informant tool assessment

Our choice of two informant‐based screening tests was based on their use in stroke practice and suitability for a stroke setting. Each participant's informant was asked to complete the IQCODE‐SF and AD8. The sequence of tests differed in sequential study case report forms, allowing for pseudo‐randomisation of the order of questionnaire administration.

The IQCODE‐SF is a 16‐item questionnaire that asks an informant to rate how much their friend/relative's cognition has changed over the past 10 years. Each item is scored on a 5‐point ordinal scale and totals are averaged to generate a score ranging from 1 (much improved) to 5 (much worse). Scores closer to 5 suggest greater cognitive impairment. A score of >3.4 is most commonly used in stroke to indicate possible dementia.

The AD8 is an 8‐item cognitive questionnaire that operates on a binary scale (‘yes, a change’ or ‘no change’; a third, unscored, option of ‘don't know’ is also available) and requires the informant to indicate if a change has occurred in their friend/relative's cognition over the past ‘several years’. Scores closer to 8 suggest greater cognitive impairment; a score of ≥2 is recommended by the developers to indicate possible dementia.

Informants were asked to complete the questionnaires in relation to how the patient's cognition was up until the point immediately before their most recent stroke. Informants were asked to complete questionnaires as soon as possible following recruitment and ideally within 1 month from index stroke. Questionnaires could be completed in the presence of the consenting researcher, or at a later point and returned via post.

Cognitive assessment (construct validity)

Our primary condition of interest was pre‐stroke dementia (major neurocognitive disorder), diagnosed according to DSM‐5 criteria. We additionally assessed for mild neurocognitive disorder, also defined according to DSM‐5 criteria.

Assessment of pre‐stroke cognitive impairment was determined via a multicomponent assessment. A trained researcher (MT) conducted a structured clinical interview with the patient and informant. The researcher was masked to results of the informant‐based cognitive screening tests. We used the structured Clinical Dementia Rating Scale (CDR) as a template to evaluate for pre‐stroke dementia and mild neurocognitive disorder. A score of 0.5 on the CDR was classified as indicative of mild neurocognitive disorder, while a score of ≥1 was classified as indicative of dementia.

We complemented the diagnostic interview with other relevant clinical information. We assessed primary and secondary care records for any formal diagnosis of cognitive disorder before the index stroke event. We assessed pre‐stroke medication history for any prescription of cholinesterase inhibitor drugs. We reviewed results of cognitive assessments performed during the acute stroke admission (details of these assessments are provided in the study protocol ). Post‐stroke cognitive performance was used to help establish pre‐stroke cognitive status. For instance, where cognition was unimpaired on post‐stroke cognitive testing, it was assumed pre‐stroke cognition was also unimpaired. Similarly, when there was objective evidence of post‐stroke cognitive impairment that was consistent with any subjective pre‐stroke impairments described during the structured clinical interview, it was assumed that these objective impairments were likely to have existed before the index stroke.

We triangulated the clinical interview, medical records and post‐stroke assessments and used all this information to reach a final diagnostic formulation based on discussion and consensus between the interviewing researcher (MT) and a stroke physician (TQ). We categorised pre‐stroke cognitive status as dementia, mild neurocognitive disorder, or no cognitive impairment.

Imaging markers of neurodegeneration (criterion validity)

All participants had CT brain imaging at time of admission as part of their routine stroke care. We assessed each scan for evidence of pre‐stroke neurodegenerative or neurovascular changes using three visual rating scales: the Fazekas, Schelten's, and Wahlund scales. A trained neurologist (MH) also assessed for presence of old infarcts.

We established an overall ‘brain frailty’ score for each participant, guided by a previously validated method encapsulating white matter hypoattenuation (WMH), atrophy, and old infarcts. We combined ‘modest’ WMH (total score of ≥3 out of six on Fazekas scale), ‘modest’ atrophy (average score of ≥2 on Wahlund scale), and presence of old infarcts. One point was assigned for each component and a score ranging from 0 to 3 was generated (closer to 3 = more severe brain frailty).

Scale‐based assessments were performed by a researcher (LM) trained in the use of the scales and blinded to the informant‐based screening test scores. As tests of internal and external consistency of scoring, the first rater of each measure re‐scored 20 scans and a second trained researcher (MT), blinded to both the informant‐based screening test scores and the original assessor's ratings, evaluated a randomised selection of 20 scans. The 20 scans were selected using a random number generator (https://www.gigacalculator.com/calculators/random‐number‐generator.php).

Clinical outcomes (prognostic validity)

At 18 months following admission to the acute stroke unit, clinical outcome data for each participating patient was established. New dementia, mortality and secondary stroke outcome data was identified via medical records. Post‐stroke functional disability was measured via the modified Rankin scale (mRS) at an in‐person or telephone interview. The mRS is a 7‐level ordinal scale (0 = independent, 6 = dead) designed to measure post‐stroke functioning; a cut‐point of ≥3 was used to define functional disability.

Statistical analysis

Accuracy (construct validity)

Accuracy of each informant‐based screening test was assessed against our diagnostic reference standard. We constructed 2 by 2 tables to examine each informant‐based screening test's performance at commonly used clinical cut‐points (>3.4 IQCODE‐SF; ≥2 AD8). We described sensitivity, specificity, positive predictive values (PPV), negative predictive values (NPV) and likelihood ratios with corresponding 95% confidence intervals (95% CI) for each tool. The Youden index was used to evaluate optimal cut‐points for each informant‐based screening test.

We compared overall accuracy for ‘pre‐stroke dementia’ versus ‘no pre‐stroke dementia’ and ‘any pre‐stroke cognitive impairment’ (i.e. dementia plus minor neurocognitive disorder combined) versus ‘no pre‐stroke cognitive impairment’ using area under empirical ROC (AUROC) curves. We used pairwise comparisons to assess if AUROC differences between tools were statistically significant.

Neuroimaging (criterion validity)

We used Spearman's rank correlation to assess the monotonic relationship between each informant‐based screening test and various components of the neurodegeneration/cerebrovascular disease measurement scales (Fazekas Scale, Schelten's Scale and Wahlund Scale). The correlation coefficients were interpreted using conventional cut‐points. A clinically meaningful correlation was defined as a correlation of at least 0.3. Chi‐square test was used to evaluate correlation of each informant‐based screening test, dichotomised at recommended cut‐points (>3.4 IQCODE‐SF; ≥2 AD8), with presence of old infarcts.

Mann–Whitney U test analyses were used to compare combined total neuroimaging scores for each scale‐based measure as well as overall ‘brain frailty’ for those patients with and without impairment (>3.4 IQCODE‐SF; ≥2 AD8)

Cohen's Kappa analysis of inter and intra‐observer reliability was performed to validate the scoring of CT scans for the Fazekas, Schelten's and Wahlund scales.

Clinical outcomes (prognostic validity)

Prognostic validity of IQCODE‐SF and AD8 was examined using ROC curves. We evaluated association of the IQCODE‐SF and AD8 for outcomes of new dementia, mortality, secondary stroke, and functional impairment at 18 months. Clinical outcomes were analysed both individually and as a composite to improve statistical power. Patients with an existing pre‐stroke dementia diagnosis were excluded from this analysis on the basis that these patients could not receive a ‘new’ diagnosis of dementia and would be unlikely to require cognitive screening data to highlight future risk of the dementia state in clinical practice.

Missing data

If two or fewer questions were missing from the IQCODE‐SF we used total score averages. If two or fewer questions were missing from the AD8, we imputed values. Specifically, missing and ‘don't know’ AD8 responses were scored as ‘no change’ on the basis that no change in cognition had been communicated by the informant.

Sensitivity analyses

We conducted sensitivity analyses re‐scoring missing and ‘don't know’ AD8 scores as ‘yes, a change’. In addition, we revaluated all results after removing any IQCODE‐SF scores that were <3.0, implying improvement in cognition, on the basis that these were likely incorrectly completed questionnaires. We also evaluated informant‐based screening tests at their optimal cut‐point according to the Youden index, wherever this differed from the currently recommended clinical cut‐point, and evaluated test performance after removing assessments that were completed >1 month post‐stroke.

Sample size calculation

We performed a sample size calculation to determine numbers needed to establish if a clinically ‘useful’ AUROC (i.e. 0.8) was statistically significantly above chance (AUROC = 0.5). With 80% power and assuming 10% condition prevalence, 88 participants were required to determine if an informant‐based screening test performing at AUROC 0.8 was statistically significantly different to chance performance at p < 0.05. Eighty‐four patients were required to detect a correlation with markers of neurodegeneration/neurovascular disease as low as 0.3.

MedCalc version 18.11 and SPSS version 27 were used for all analyses.

RESULTS

A total of 357 participants were recruited into APPLE. Of these, a total of 137 (38%) patient/informant dyads agreed to participate in this sub‐study. A breakdown of study numbers for each validation assessment and reasons for exclusion from analysis can be seen in Figure 1.

FIGURE 1

Flow chart depicting study attrition and reasons for exclusion for each validity analysis

Median time following stroke unit admittance to completion of informant‐based screening tests was 5 days (IQR = 8). Median time following stroke unit admission to completion of the cognitive impairment diagnostic reference standard assessment was 7 days (IQR = 9).

According to our reference standard diagnostic criteria, 12/102 (11.7%) patients had dementia before their stroke/TIA, of whom 7 (58.3%) had a formal clinical diagnosis of dementia. Thirty‐one out of 102 (30.3%) patients were cognitively impaired before their stroke/TIA, including 19 with minor neurocognitive disorder. Descriptive statistics of the included study population can be seen in Table 1.

TABLE 1

Baseline study population characteristics

Variable	Overall (N = 137) a
Age (median; 25th–75th percentile)	71 (59–79)
Sex male (%)	83/134 (61.9%)
Stroke‐type (%)
Total anterior circulation stroke	12/135 (8.8%)
Partial anterior circulation stroke	47/135 (34.8%)
Lacunar stroke	22/135 (16.3%)
Posterior circulation stroke	30/135 (22.2%)
Transient ischaemic attack	24/135 (17.8%)
NIHSS (Mean; SD)	3.43 (4.78)
Pre‐stroke modified ranking scale (nn; %)
0–2	104/133 (78.2%)
3–5	29/133 (21.8%)
Vascular disease (nn; %)	46/134 (34.3%)
Heart failure (nn; %)	11/134 (8.2%)
Post‐stroke delirium (nn; %)	3/134 (2.2%)
Post‐stroke aphasia (nn; %)	19/134 (14.2%)
Previous stroke/TIA (nn; %)	66/134 (49.3%)
Diabetes mellitus (nn; %)	25/134 (18.7%)
Atrial fibrillation (nn; %)	19/134 (14.2%)
Alcohol dependency (nn; %)	12/134 (8.9%)
Education years (mean; SD)	11.9 (3.06)

Denominators ≠ 137 due to missing data.

Pre‐stroke dementia versus no pre‐stroke dementia

Sensitivity and specificity values for the IQCODE‐SF at cut‐point >3.4 were 91.7% (95% CI = 61.5%–99.8%) and 82.2% (95% CI = 72.7%–89.5%); AD8 at cut‐point ≥2 had sensitivity 92.3% (95% CI = 64.0.7%–99.8%) and 58.2% (95% CI = 47.4%–68.5%) specificity.

AUROC for IQCODE‐SF was 0.91 (95% CI = 0.83–0.96), p < 0.01 and 0.87 (95% CI = 0.79–0.93), p < 0.01 for AD8. There was no statistically significant difference between informant‐based screening tests' AUROC (p = 0.35) according to pairwise comparisons.

The Youden index suggested the AD8 performs best at a cut‐point ≥4 for pre‐stroke dementia screening (sensitivity = 90.9%, 95% CI = 58.7%–99.8%; specificity= 80.7%, 95% CI = 71.1%–88.1%) but was still less accurate than IQCODE‐SF in absolute terms. The Youden index for the IQCODE‐SF did not differ from the recommended cut‐point of >3.4. Figure 2.

FIGURE 2

Comparative empirical area under receiver operating characteristics curves of AD8 and IQCODE‐SF. Curves depict ability of IQCODE‐SF and AD8 to discriminate between people with and without pre‐stroke dementia. A larger area under the curve suggests greater accuracy

Any pre‐stroke cognitive impairment versus no pre‐stroke cognitive impairment

Sensitivity and specificity values for the IQCODE‐SF at cut‐point >3.4 were 63.3% (95% CI = 43.9%–80.1%) and 88.9% (95% CI = 79.3%–95.1%), respectively. For AD8 at cut‐point ≥2, sensitivity was 87.1% (95% CI = 70.2%–96.4%) and specificity 68.5% (95% CI = 56.6%–78.9%).

AUROC curves were 0.86 (95% CI = 0.78–0.92), p < 0.01, for IQCODE‐SF and 0.84 (95% CI = 0.76–0.90), p < 0.01, for AD8. Difference in informant‐based screening tests' AUROC was not statistically significant (p = 0.41).

The Youden index suggested AD8 performed best at a cut‐point of ≥3 (sensitivity = 73.3%, 95% CI = 54.1%–87.7%; specificity = 81.8%, 95% CI = 70.3%–89.3%), and IQCODE‐SF performed best at a lower cut‐point of >3.1 (sensitivity = 93.1%, 95% CI = 77.2%–99.2%; specificity = 68.5%, 95% CI = 56.6%–78.9%). Figure 3.

FIGURE 3

Comparative empirical area under receiver operating characteristics curves of AD8 and IQCODE‐SF. Curves depict ability of IQCODE‐SF and AD8 to discriminate between people with and without any degree of pre‐stroke cognitive impairment. A larger area under the curve suggests greater accuracy

Neuroimaging (criterion validity) analysis

Spearman's rank correlation revealed consistently stronger correlations for the IQCODE‐SF over the AD8 for all scale‐based measures of neurodegeneration/neurovascular disease. The IQCODE‐SF demonstrated low positive correlations with measures of white matter disease, medial temporal lobe atrophy, and general atrophy. By contrast, the AD8 demonstrated negligible correlations for all scale‐based neuroimaging markers apart from two components of the Wahlund scale (Table 2). Neither informant‐based screening test significantly correlated with presence of old infarcts (IQCODE‐SF X 2 = 0.811, p = 0.368; AD8 X 2 = 1.362, p = 0.243)

TABLE 2

Spearman's rank correlations

			Fazekas scale		Schelten's scale		Wahlund scale
			PVH	DWMH	Left	Right	LV	IFACC	LRSF	OS	FS	PS
Spearman's rho	AD8	Correlation coefficient	0.20 b	0.14	0.27 a	0.21 b	0.19 b	0.33 a	0.18 b	0.26 a	0.32 a	0.18 b
		Sig. (2‐Tailed)	0.03	0.12	0.00	0.02	0.04	0.00	0.04	0.00	0.00	0.04
		N	120	120	120	120	120	120	120	120	120	120
	IQCODE‐SF	Correlation coefficient	0.30 a	0.19 b	0.34 a	0.34 a	0.24 a	0.39 a	0.33 a	0.29 a	0.35 a	0.30 a
		Sig. (2‐Tailed)	0.00	0.04	0.00	0.00	0.00	0.00	0.00	0.00	0.00	0.00
		N	120	120	120	120	120	120	120	120	120	120

Note: Bold, Clinically meaningful correlation (0.3).

Abbreviations: AD8, Ascertain Dementia 8; DWMH, Deep White Matter Hypoattenuation; Elderly‐Short Form; FS, Frontal Sulci; IFACC, Interhemispheric Fissure Anterior to the Corpus Collosum; IQCODE‐SF, Informant Questionnaire on Cognitive Decline in the PVH, LRSF, Left and Right Sylvian Fissures; LV, Lateral Ventricles; Periventricular Hypoattenuation; OS, Occipital Sulci; PS, Parietal Sulci.

Correlation is significant at the 0.01 level (2‐tailed).

Correlation is significant at the 0.05 level (2‐tailed).

In Mann–Whitney U analysis, IQCODE‐SF dichotomised at >3.4 demonstrated significantly different combined periventricular hypoattenuation (PVH) and deep white matter hypoattenuation (DWMH) Fazekas scale scores, combined left and right Schelten's scale scores, combined Wahlund scale scores, and overall ‘brain frailty’ scores. AD8 dichotomised at ≥2 demonstrated significantly different combined left and right Schelten's scale scores, combined Wahlund scale scores, and overall ‘brain frailty’, but no significantly different combined PVH and DWMH Fazekas scale scores (Table 3).

TABLE 3

Comparison of neuroimaging scale scores in ‘impaired’ and ‘unimpaired’ patients according to informant tool cut‐point

	AD8				IQCODE‐SF
	Mean rank (AD8 <2) N = 66	Mean rank (AD8 ≥2) N = 57	U	Sig. (2‐tailed)	Mean rank (IQCODE‐SF ≤3.4) N = 93	Mean rank (IQCODE‐SF >3.4) N = 27	U	Sig. (2‐Tailed)
Fazekas scale (combined score)	56.9	67.8	1548.5	0.087	55.6	77.3	801.5	0.004 a
Scheltens scale (combined score)	55.2	69.9	1432.0	0.021 b	54.1	82.7	655.5	<0.001 a
Wahlund scale (combined score)	52.9	72.5	1285.5	0.002 a	54.3	81.9	678.5	<0.001 a
Brain frailty	53.6	70.6	1336.5	0.006 a	53.7	81.7	657.5	<0.001 a

Note: Bold, statistically significant association at p < 0.05. Fazekas Scale score was evaluated on a scale of 0–6 (combined PVH and DWMH), Schelten's Scale score was evaluated on a score of 0–8 (combined total of left and right), Wahlund Scale score was evaluated on a score of 6–18 (all components of Wahlund combined).

Abbreviations: AD8, Ascertain Dementia 8; IQCODE‐SF, Informant Questionnaire on Cognitive Decline in the Elderly Short Form.

Significant at the 0.01 level (2‐tailed).

Significant at the 0.05 level (2‐tailed).

In the subset of 20 randomly selected participant scans, intra‐observer agreement was acceptable for the Fazekas and Wahlund scale scores (k = 0.69, k = 0.73) and strong for the Schelten's scale (k = 0.88). Overall inter‐observer agreement was acceptable at 0.73.

Prognostic validity

At 18 months, 13 participants had died, 5 participants received a new diagnosis of dementia, there were 10 recurrent strokes, and 56 participants were deemed to be functionally disabled (mRS ≥3).

The IQCODE‐SF demonstrated stronger prognostic accuracy than the AD8 for predicting future dementia (IQCODE‐SF AUROC = 0.90, 95% CI = 0.80–1.00; AD8 AUROC = 0.82, 95% CI = 0.66–0.98). This AUROC performance difference was significant according to pairwise comparisons (p = 0.039).

Both informant‐based screening tests demonstrated weak overall accuracy for prediction of a ‘poor outcome’ (composite of new dementia, mortality, recurrent stroke, and functional impairment) and all non‐dementia outcomes. There were no statistically significant differences between informant‐based screening tests for any of these measures (all p > 0.05) apart from mortality, which was significantly stronger for the IQCODE‐SF (IQCODE‐SF AUROC = 0.65, 95% CI = 0.50–0.79; AD8 AUROC = 0.54, 95% CI = 0.38–0.71; pairwise comparison of AUROC difference: p = 0.01) (Figures S1–S5).

Sensitivity analysis

We conducted a sensitivity analysis re‐scoring ‘don't know’ or missing responses with the ‘yes, a change’ response for each question on the AD8. In addition, we removed six instances in which the IQCODE‐SF score was <3.0, implying improvement in cognition. Method for scoring ‘don't know’ or missing responses and presence of ‘incorrect’ IQCODE‐SF scores did not have a meaningful impact upon the results (Table S1).

Sensitivity analysis evaluating AD8 dichotomised at the higher cut‐point of ≥4 demonstrated significantly different total scores between ‘impaired’ and ‘unimpaired’ patients on all 3 scale‐based neuroimaging measures as well as on overall brain frailty, although strength of association was still generally weaker than those seen for IQCODE‐SF (Table S2). At this cut point, the AD8 also showed a significant correlation with presence of old infarcts (X 2 = 5.55, p = 0.02).

We removed 16 cases where the informant‐based screening tests were completed >1 month following stroke and repeated our primary construct validity analyses. Results were consistent with our primary analyses (Figures S6 and S7).

DISCUSSION

Both IQCODE‐SF and AD8 are valid informant‐based screening tests for pre‐stroke dementia assessment; however, our data reveal potentially important differences regarding respective tool properties and associated neurological damage. While differences between measures were modest overall, IQCODE‐SF demonstrated consistently better validity metrics across almost all analyses.

Our findings are consistent with those of Nieuwkerk et al. who found an IQCODE‐SF AUROC of 0.94 for pre‐stroke dementia assessment. Similarly, IQCODE‐SF scores have previously been associated with periventricular WMH, medial temporal atrophy, and global atrophy in an atrial fibrillation population. Conversely, the AD8 has never previously been validated as an informant‐based pre‐stroke cognitive screening test and has only previously been shown to correlate with biomarkers of Alzheimer's disease.

Clinical recommendations

Although there is evidence that the IQCODE‐SF may have some validity advantages over the AD8, it is important to highlight that arguably the most clinically useful measure of validity (accuracy for diagnosis of pre‐stroke dementia) did not show significant differences between the informant‐based screening tests. While this implies either test can be used for informant‐based pre‐stroke cognitive screening, there are notable performance differences between each questionnaire when used at traditional clinical cut‐points. When screening for pre‐stroke dementia, the IQCODE‐SF appears to be the superior tool, demonstrating greater specificity than the AD8 while retaining a comparable sensitivity. By contrast, when screening for ‘any cognitive impairment’ each tool has divergent strengths: the AD8 is the more sensitive informant‐based screening test, while the IQCODE‐SF is the more specific. However, for informing post‐stroke treatment and care pathways, it is dementia, rather than more subtle levels of cognitive impairment, that are important. Moreover, the IQCODE‐SF shows stronger associations with cerebrovascular small vessel disease that may be predominant in this population, has a stronger association with overall pre‐stroke ‘brain frailty’, and also has greater accuracy for predicting future dementia. We therefore suggest that IQCODE‐SF may be the preferred informant‐based screening test for routine pre‐stroke assessment.

The superior performance of the IQCODE‐SF may be a reflection of the comparative comprehensiveness of the tool. The IQCODE‐SF has twice as many items as the AD8 and involves a more extensive evaluation of executive impairments. Executive impairments are amongst the most commonly observed in a stroke population and previous studies have demonstrated that the executive components of the IQCODE‐SF are the most discriminating in a stroke population.

There may still be instances to prefer use of AD8 over the IQCODE‐SF. The AD8 is shorter than the IQCODE‐SF and easier to score. Thus, for situations where clinicians or researchers are not able to help informants score the questionnaire, for example, a postal assessment, AD8 may be a better choice. Indeed, there were 6 (4%) instances in which informants apparently misunderstood how to complete the IQCODE‐SF in our study, while a prior study suggested an even higher error rate of almost 11%.

Where the AD8 is used in stroke, it may perform better if the threshold for a positive test is changed to a higher cut‐off. It is not uncommon for screening tools developed in a non‐stroke setting to need adjustment for use in stroke. For example, the Montreal Cognitive Assessment is often used in stroke care, but for diagnosis of important post‐stroke cognitive issues, there is evidence that the threshold for test positivity should be lowered.

Strengths and limitations

Our study provides the first direct comparison on the validity of IQCODE‐SF and AD8 in stroke. Distinct from other studies, we did not exclude on the basis of physical, cognitive or communication difficulties. We followed best practice guidelines for conducting diagnostic test accuracy research and present results that have ‘real world’ clinical value. Despite this, our study is limited by our sample size and as such there is uncertainty around our comparative analyses meaning we can't be sure of any true difference between the tools. Larger scale evaluations would be beneficial to corroborate our results; however, for the minor difference in overall AUROC suggested by many of our analyses, comparisons would require sample sizes of several thousands to achieve sufficient power to definitively prove a difference.

Due to challenges of recruitment, we did not operate strict timeframe restrictions for completion of the IQCODE/AD8 or structured clinical interview. This is liable to increase recall bias and may impair the accuracy of tests. However, informants were encouraged to complete questionnaires and partake in the structured clinical interview no later than 1 month following the stroke‐survivor's admission to hospital and there were minimal instances where assessments were completed beyond this timeframe. Indeed, the median duration for completion of the questionnaires and the structured clinical interview was 5 and 7 days, and our sensitivity analysis suggests the lack of timeframe restrictions did not significantly impact our results.

CONCLUSIONS

For assessing pre‐stroke dementia both IQCODE‐SF and AD8 are valid informant‐based screening tests, and we would encourage clinicians and researchers to formally screen for pre‐stroke dementia using either questionnaire. Based on our data, when the informant‐based screening tests are used at conventional cut‐points, the IQCODE‐SF may be preferable to the AD8 for informant‐based pre‐stroke dementia screening.

CONFLICT OF INTEREST

Dr. Taylor‐Rowan reports no disclosures. Ms. McGuire reports no disclosures. Dr. Hafdi reports no disclosures. Dr. Quinn reports no disclosures. Professor Evans reports no disclosures. Professor Stott reports no disclosures. Ms. Wetherall reports no disclosures. Dr. Elliott reports no disclosures. Dr. Drozdowska reports no disclosures.

Supporting Information S1

Click here for additional data file.