Subjective memory impairment: No suitable criteria for case-finding of dementia in primary care.

INTRODUCTION: Subjective memory impairment (SMI) might be used for the case-finding of dementia. Present analyses aim to determine the diagnostic value and the predictive ability of SMI and related worries for the discrimination of patients screened positive or negative for dementia.
METHODS: The analyses are based on data derived from the ongoing German general practioner (GP)-based, randomized controlled trial DelpHi-MV. A total of 5106 patients (age ≥ 70, living at home) were first asked for SMI and related worries and then screened for dementia in 110 participating GP practices (November 2011 to August 2014; preliminary data) using the DemTect.
RESULTS: A total number of 2556 patients (50%) stated that they experience SMI and 892 patients (17%) screened positive for dementia. The sensitivity of SMI for the correct classification of positively screened patients was 54%, the positive predictive value (PPV) 19%. The specificity of SMI was 51%; the negative predictive value (NPV) 84%. Among 2480 patients with SMI, 45% reported SMI-related worries (sensitivity 52%; specificity 57%; PPV 22%; NPV 84%). Receiver operating characteristics analyses showed no statistically significant improvement in the area under the curves when using SMI or related worries as predictors (additional to age and sex) for the discrimination between positively and negatively screened patients. DISCUSSION: The analyses showed that the risk of overlooking cognitive impairment in the subgroup of patients who state that they do not experience SMI would be unreasonable high. Thus, the results provide clear evidence that neither SMI nor related worries can be used as a valid criteria to decide whether an elderly primary care patient should be tested for dementia. TRIALS REGISTRATION: ClinicalTrials.gov Identifier: NCT01401582.

Introduction

The timely recognition of dementia is the prerequisite for adequate information, treatment, and care. Nevertheless, dementia is known to be considerably underdiagnosed; even in high-income countries with advanced medical care systems about 50% to 80% of people with dementia (PWD) are not formally diagnosed [1], [2]. There are approaches to improve the recognition of dementia, such as the proactive “dementia case finding scheme” that was initiated by the government of the United Kingdom [3] or the “Annual Wellness Visit” for Medicare enrollees in the United States which includes the detection of any cognitive impairment [4]. However, the best practice for the identification of dementia in primary care has not yet been established. Previous studies showed that the use of structured screening instruments improves the identification of cognitive impairment in primary care and that the screening for dementia increases diagnosis rates [5], [6], [7]. Nevertheless, routine screening is controversially discussed and not recommended in respective dementia guidelines because there is still a lack of evidence that patients benefit from it [8], [9], [10], [11], [12]. Arguments against routine screening include the risk of receiving a false-positive diagnosis of dementia after a positive screening outcome; the cause of anxiety or depression among positively screened subjects; unnecessary examinations and treatments; the diversion of resources that would better be used to care for real dementia cases; or the danger that older patients will avoid visiting their general practitioner (GP) because they fear to be diagnosed with dementia [13], [14], [15], [16]. Therefore, routine screening of asymptomatic patients is not seen as the favorable solution to improve the recognition of patients with dementia in primary care. It has been suggested that the case-finding of dementia should focus on patients presenting with cognitive complaints [14], [15].

Subjective memory impairment (SMI) may represent the first symptomatic manifestation of Alzheimer's disease and SMI and related worries have been identified as risk factors for the incidence of dementia in people without objective cognitive impairment [17], [18], [19]. However, the diagnostic value of self-reported cognitive impairment for prevalent dementia seems to be limited for several reasons: SMI is associated with depression [20], [21], [22]; cognitively healthy older persons frequently complain about memory impairment [23], [24]; and PWD are often not aware of their memory problems [25], [26].

Mitchell [27] conducted a meta-analysis of the diagnostic value of subjective memory complaints for manifest dementia in community samples with a low prevalence of dementia and found a positive predictive value (PPV) of 19% and a negative predictive value (NPV) of 94%. Mitchell concludes that the absence of subjective memory complaints may be a reasonable method of excluding dementia and could be incorporated into short screening programs in settings with low prevalence of dementia. However, to our knowledge this assumption has not yet been validated in clinical settings. This study aims to determine whether self-reported SMI or SMI-related worries could be used as a valid criteria to decide if an elderly primary care patient should be screened for dementia.

Therefore, we want to determine (1) the diagnostic value and (2) the predictive ability of self-reported SMI and related worries for the discrimination of patients with and without cognitive impairment (i.e. patients screened positive and negative for dementia).

Methods

Study design

The present cross-sectional analyses are based on data derived from the ongoing German GP-based, randomized, controlled intervention trial DelpHi-MV (dementia: life- and person-centered help in Mecklenburg, Western Pomerania). The details of the study are described elsewhere [28], [29], [30]. The eligible patients (≥70 years, living at home) in participating GP practices are asked by the GP or the assistant whether they experience SMI and if so, whether they worry about their SMI. After the patients answered these questions they are screened for dementia using DemTect [31], which is a widely used dementia screening test in GP practices in Germany [32]. The DemTect score<9 is the inclusion criteria for the DelpHi-trial. Present analyses are based on the larger pool of patients tested for eligibility of the clinical trial. The patients who meet the inclusion criteria for DelpHi-MV are informed by their GPs about the study, invited to participate, and asked to provide written informed consent. When the patient is unable to give a written informed consent, his or her legal representative is asked to sign the consent form on his or her behalf (as approved by the Ethical Committee of the Chamber of Physicians of Mecklenburg, Western Pomerania, registry number BB 20/11). To compensate for their additional effort, study physicians receive an allowance for each screening (10€ per patient) and an additional allowance for the inclusion of patients in the trial (100€ per patient).

Study population

Of 5511 eligible patients (age ≥70 years, living at home) screened for dementia in 110 participating GP practices (November 2011 to August 2014) we included 5106 patients with complete data regarding age, sex, DemTect-score, and self-reported SMI into the analyses. A total of 406 patients were excluded of the analyses because of missing data in the variables sex (n = 16) and SMI (n = 389). For the analysis of SMI-related worries we included 2480 patients who reported the presence of SMI and responded to the question for SMI-related worries.

Procedures and instruments

For sample description we analyzed age, sex, DemTect-score, self-reported SMI, and related worries. DemTect [31] is a personal interview-based instrument that includes five tasks (recall of word list, number transcoding task, word fluency task, digit span reverse, delayed recall of word list). It is a highly sensitive screening test to identify people with dementia in the early stage of the disease (sensitivity was 80% for detecting MCI and 100% for Alzheimer's disease). The total DemTect-score ranges from 0 to18, a total score<9 indicates dementia. SMI (“Do you have memory problems?”) and SMI-related worries (“Do you worry about your memory problems?”) were analyzed as dichotomized variables (yes/no).

Statistical analyses

We summarized the variables that describe the sample using descriptive statistics. To test for differences between positively and negatively screened patients we used nonparametric tests (two-sample Wilcoxon rank-sum test for continuous and Pearson's chi2-test for categorized variables with α = 0.05). To determine the diagnostic value of SMI and SMI-related worries we calculated the sensitivity (percent of positively screened patients with dementia who reported SMI/SMI-related worries); the specificity (percent of negatively screened patients who did not report SMI/SMI-related worries); the PPV (percent of patients reporting SMI with positive screening outcome); and the NPV (percent of patients not reporting SMI with negative screening outcome). To compare our results with the outcomes of the meta-analyses of Mitchell [27], we calculated the clinical utility index (UI). The predictive value is heavily influenced by the prevalence, whereas the utility index combines discriminatory ability (sensitivity/specificity) and occurrence (PPV/NPV). The positive utility index (UI+) as a measure of “rule-in accuracy” is a product of sensitivity and PPV; the negative utility index (UI−) as a measure of “rule-out accuracy” is a product of specificity and NPV. The UI can be considered a measure of the clinical value of a diagnostic test and is interpreted as follows: <0.2 poor; >0.2 ≤ 0.4 fair; >0.4 ≤ 0.6 moderate; >0.6 ≤ 0.8 good; and >0.8 ≤ 1.0 very good.

We provide subgroup analyses by age (5-year intervals) and sex as supplemental material.

To determine the predictive ability of SMI and related worries we performed receiver operating characteristics (ROC) analyses which test whether the addition of a term for SMI/SMI-related worries increases the AUC enough to justify its inclusion in the models with age and sex. Age and female sex are important risk factors for dementia [10] that are easy to observe in a GP practice. To test for possible interaction between SMI/SMI-related worries and sex or age we fitted additionally logistic models including corresponding multiplicative interaction terms, but neither interaction term was statistically significant. Because of the large sample we conducted stratified ROC analyses to examine the efficiency of SMI and related worries for both sexes. Statistical analyses were performed by STATA/IC [33].

Results

Sociodemographic and clinical characteristics

Table 1 presents the sociodemographic and clinical characteristics of the study sample.

Table 1

Sociodemographic and clinical characteristics of the study sample

	Total sample (N = 5106)	Patients screened negative (n = 4214)	Patients screened positive (n = 892)	P value
Sex (female), n (%)	3059 (60)	2543 (60)	516 (57)	.167∗
Age (yrs), mean (SD)	77.80 (0.73)	77.22 (0.08)	80.51 (0.19)	<.001†
DemTect, mean (SD)	12.35 (0.06)	13.72 (0.43)	5.91 (0.08)	–

Abbreviation: SD, standard deviation.

Pearson's chi2-test.

Wilcoxon rank-sum test.

Diagnostic value of SMI and SMI-related worries

A total of 5106 patients (≤70 years) were screened for dementia; and 50% of these patients reported the presence of SMI. Seventeen percent of all patients were screened positive for dementia (n = 892).

Before the screening, 46% of the positively screened patients had stated that they had not experienced SMI. On the other hand, 49% of the negatively screened patients had stated that they had experienced SMI. The sensitivity, specificity, the predictive value, and the UI for SMI are shown in Table 2.

Table 2

Subjective memory impairment and related worries among patients screened negative and positive for dementia: sensitivity, specificity, predictive value, and clinical utility index

	Total sample	Patients screened negative	Patients screened positive	Sensitivity	Specificity	PPV	NPV	UI+	UI−
Total, n	5106	4214	892
No SMI	2550 (50%)	2142 (51%)	408 (46%)	54%	51%	19%	84%	0.10 (poor)	0.43 (moderate)
SMI	2556 (50%)	2072 (49%)	484 (54%)
Total, n	2480	2011	469
No worries	1362 (55%)	1138 (57%)	224 (48%)	52%	57%	22%	84%	0.11 (poor)	0.47 (moderate)
Worries	1118 (45%)	873 (43%)	245 (52%)

Abbreviations: SMI, subjective memory impairment; PPV, positive predictive value; NPV, negative predictive value; UI+, positive utility index; UI−, negative utility index.

Subgroup analyses for SMI (by age and sex) are provided in Supplemental Table 1. Across the different age groups the sensitivity of SMI ranged between 54% and 58% (female) and 47% and 65% (male); the specificity ranged between 39% and 52% (female) and 41% and 53% (male). The PPV (range: 6%–38%) as well as the UI+ (range 0.03–0.23) were lowest in the youngest age group and increased with age. The NPV (range: 60%–95%) and the UI− (range: 0.23–0.49) were highest in the youngest age group and decreased with age.

Among 2480 patients that reported SMI, 45% stated that they worried about their SMI. SMI-related worries were indicated by 52% of the patients screened positive and 43% of the patients screened negative for dementia. Table 2 presents the sensitivity, specificity, the predictive value, and the UI for SMI-related worries.

Subgroup analyses for SMI-related worries (by age and sex) are provided in Supplemental Table 2. Across the different age groups the sensitivity ranged between 39% and 70% (female) and 30% and 46% (male); the specificity ranged between 54% and 64% (female) and 54% and 80% (male). The PPV (range: 11%–39%) and the UI+ (range 0.04–0.20) were lowest in the youngest age group and increased with age. The NPV (range: 68%–97%) and the UI− (range: 0.37–0.73) were highest in the youngest age group and decreased with age.

Predictive ability of SMI and SMI-related worries

The results of the ROC analyses are shown in Table 3. Regarding the discrimination of positively and negatively screened patients there were no statistically significant differences between the AUCs when using SMI or SMI-related worries as predictors additional to age and sex (see Fig. 1A and B).

Table 3

ROC analyses for predictive models with and without subjective memory impairment and related worries

	Observations	AUC	SE	95% confidence interval		Chi2	P value
Total
Age, sex	5016	0.678	0.010	0.660	0.697	0.18	.669
Age, sex, SMI	5016	0.678	0.010	0.659	0.697
Female patients
Age	3059	0.703	0.012	0.679	0.727	0.22	.638
Age, SMI	3059	0.705	0.012	0.681	0.728
Male patients
Age	2047	0.637	0.016	0.606	0.668	0.06	.804
Age, SMI	2047	0.637	0.016	0.606	0.668
Total
Age, sex	2480	0.678	0.013	0.652	0.705	1.60	.207
Age, sex, worries	2480	0.684	0.013	0.658	0.710
Female patients
Age	1491	0.697	0.017	0.664	0.730	0.22	.638
Age, worries	1491	0.700	0.016	0.668	0.732
Male patients
Age	989	0.648	0.022	0.604	0.709	3.45	.063
Age, worries	989	0.666	0.022	0.623	0.709

Abbreviations: ROC, receiver operator curve; AUC, area under the curve; SE, standard error; SMI, subjective memory impairment.

Fig. 1

(A) Receiver operating characteristics (ROCs) for models with and without subjective memory impairment. (B) ROCs for models with and without subjective memory impairment-related worries.

Stratified analyses revealed that AUCs did not improve statistically significantly when SMI or SMI-related worries were included as predictors rather than just age, neither for female nor for male patients (see Table 3).

Discussion

This study aimed to determine whether self-reported subjective memory impairment or related worries could be used as a suitable criteria for case-finding of dementia in elderly primary care patients.

Our analyses showed that the sensitivity of SMI was just 54%. This means that 46% of the primary care patients screened positive for dementia did not report SMI before the screening. Thus almost half of the patients with cognitive impairment would have been overlooked if the presence of SMI would have been the precondition for performing the structured cognitive test. In addition, both the PPV (19%) and the UI+ (0.10; interpreted as “poor” rule-in-accuracy) of SMI were very low. Mitchell [27] reported similar results with a meta-analytic pooled sensitivity of 43%, a PPV of 19%, and an UI+ of 0.08 and concluded that “subjective memory complaints should not be relied on for case-finding”. These findings are in line with the results of previous studies that showed that people with dementia are often not aware of their memory problems [25], [26].

On the other hand, 49% of all patients who had screened negative for dementia in the present sample stated that they experience SMI. The specificity of SMI was just 51%, the NPV was 84%, and the UI− was 0.43 (moderate). These results agree with the outcomes of previous studies showing that cognitively healthy older persons often complain about memory impairment [23], [24]. Because the present cross-sectional analyses are based on prevalent cases, our results are unrelated to the use of SMI as a predictor for the risk of future development of dementia in patients without objective cognitive impairment [17], [18], [19]. We are not able to determine whether SMI-positive, but DemTect-negative patients in our sample are at an increased risk to develop dementia in the future.

For SMI-related worries, we found similar results: the sensitivity (53%), the specificity (57%), the PPV (22%), and the UI+ (0.11) were very low. If the presence of SMI in conjunction with SMI-related worries would have been the preselecting criteria for further cognitive testing, even more than 70% of the positively screened patients would not have been tested for cognitive impairment.

Subgroup analysis by age and sex showed some inconsistent differences in the sensitivity and specificity of SMI and SMI-related worries across the age groups and between male and female patients. The “rule-in accuracy” (PPV/UI+) increased with age, whereas the “rule-out accuracy” (NPV/UI−) decreased, because the predictive value and (to a lesser extend) the UI are influenced by the age-related increase of the prevalence of dementia. Although the diagnostic value of SMI or SMI-related worries was for some subgroups better than for the total sample, the subgroup differences were rather inconsistent. Therefore our results do not allow identifying specific subgroups for which the presence of SMI or related worries would be suitable as preselection criteria for the case finding of dementia.

ROC analyses showed that the AUCs of the models that included just age and sex as predictors for a positive screening outcome differed only marginally from the models that included SMI or related worries as additional predictors. Despite the large sample size, the analyses showed no statistically significant improvement in the AUCs. If interpreting the statistics one-sided, there was a significant improvement when using SMI-related worries as predictor in addition to age in male patients. The statistical approach of comparing AUCs of nested models (as done in the present study) is known to lead to overly conservative P-values [34]; therefore P-values have to be treated with care.

Overall, the predictive ability cannot be considered satisfactory. Our results indicate that neither SMI nor related worries discriminated adequately between patients screened positive or negative for dementia. These results are consistent with the findings of previous studies that showed that self-reported memory problems correlate poorly with the present psychometric impairment [20], [22], [24].

Mitchell [27] reported a NPV of 94% and a UI− of 0.77 (interpreted as good) in community samples. He assumed that the absence of subjective memory complaints may be a reasonable method of excluding dementia and could be used as a preselection criteria in screening programs for dementia in settings with low prevalence of dementia. To our knowledge, this is the first study that has examined this hypothesis in a large sample of elderly primary care patients. In our sample we found lower values, the NPV was 84%; and the UI− 0.43 (interpreted as “moderate” rule-out-accuracy). A reason for these differences might be the higher prevalence of dementia in primary care practices than in community samples. In our sample, the prevalence of positively screened patients was 17%. This is similar to the findings of a study in Seattle (USA), where 18% of 524 primary care patients (aged 65+) screened positive for dementia [6].

To conclude, our results provide clear evidence that in neither SMI nor related worries can be used as valid criteria to decide whether an elderly primary care patient should be tested for dementia or not. The risk of overlooking cognitive impairment in the subgroup of patients who state that they do not experience SMI or related worries would be unreasonable high.

For the current clinical practice our results indicate that it is not sufficient to examine only those patients for dementia who are complaining about memory problems. Physicians should apply more appropriate methods to ensure the timely recognition of dementia.

In contrast to self-reported SMI, the evaluation of cognitive impairment by informants may be a more reliable predictor for cognitive impairment. Carr et al. [24] showed that informants were able to identify 92% of those subjects judged by the clinician to have dementia. However, many older people live alone and visit their GP unaccompanied, thus an informant is often not available.

Recently, the Alzheimer's Association International proposed an algorithm to detect cognitive impairment during the Medicare “Annual Wellness Visit” in a primary care setting that addresses the problem of the unreliability of self-reported SMI and the unavailability of informants: whenever signs of cognitive impairment are self-reported by the patient or noted by the clinician, a brief structured assessment should be performed. In case there is no informant present to provide confirmatory information, the brief structured assessment should be performed anyway. A positive screening outcome triggers further full dementia evaluation [4]. Further studies should evaluate whether this algorithm would permit a more reliable case-finding of dementia in primary care, because the timely identification of people with dementia is a major unsolved problem in dementia care [1], [2].

Systematic review: We searched PubMed for articles with the following search terms: dementia and subjective memory impairment; subjective memory complaints; subjective cognitive decline; related worries; screening, case-finding; and primary care. In addition we reviewed the reference lists of articles identified.

Interpretation: It has been assumed that the absence of subjective memory impairment (SMI) may be a reasonable method of excluding dementia and could be incorporated as a preselection criterion into short screening programs. Present results provide evidence that SMI cannot be used as a valid criteria to decide whether an elderly primary care patient should be screened for dementia. The risk of overlooking cognitive impairment in the subgroup of patients who state that they do not experience SMI would be unreasonable high.

Future directions: Further studies should evaluate whether the additional assessment of cognitive impairment by informants would permit a more reliable case-finding of dementia in primary care.