A Brief Dementia Test with Subjective and Objective Measures.

BACKGROUND: The development of an effective brief dementia test will help in the early identification of dementia. AIM: This study investigates the diagnostic utility of a brief cognitive test for dementia which combines a short subjective informant-rated questionnaire (AD8) with an objective cognitive measure (Mini-Mental State Examination, MMSE) or its subcomponents.
METHODS: Subjects with mild dementia (Clinical Dementia Rating Scale score ≤1) were matched with community-dwelling, cognitively intact controls. MMSE and Clinical Dementia Rating Scale were administered to all subjects, while AD8 was completed by a reliable informant. Receiver operating characteristics analysis determined the diagnostic accuracies of AD8, MMSE, and AD8 combined with MMSE (AD8+MMSE). Stepwise logistic regression identified the subcomponents of MMSE which, combined with AD8, best discriminated dementia patients from controls.
RESULTS: The AD8 (area under the curve [AUC] = 0.92, 95% confidence interval [CI] 0.89-0.95) was superior to the MMSE (AUC = 0.87, 95% CI 0.83-0.92) in discriminating mild dementia patients from controls, and AD8+MMSE (AUC = 0.95, 0.92-0.98) increased its superior discrimination over MMSE alone. AD8 combined with three-item recall and intersecting pentagon copy (AUC = 0.95, 95% CI 0.92-0.97) performed as well as AD8 combined with full MMSE.
CONCLUSION: AD8 combined with the MMSE subcomponents three-item recall and intersecting pentagon copy has excellent diagnostic utility and is a promising brief cognitive test for early dementia.

Introduction

The push for early detection of dementia, particularly in primary care settings, has intensified in recent years [1]. Diagnosing dementia early enables patients to plan for their future before cognitive decline impairs mental capacity. The availability of effective symptomatic therapies and interventions also supports the cause for a timely diagnosis [2]. However, despite its growing numbers, dementia has remained underdiagnosed and undertreated [3]. Even in those who are symptomatic, a delay in diagnosis of several years from initial manifestations is not uncommon [4].

A major challenge lies in the lack of an effective test for dementia [5]. The ideal test needs to possess high sensitivity and specificity, be brief and acceptable to both health care providers and test subjects. Many tests have been developed for this purpose, however each with its own strengths and limitations. Continued research is needed to refine dementia assessment instruments to better fit the requirements of the ideal test.

The current instruments are generally classified into either objective tests of patients' cognition or subjective assessments of patients' function by informants or the patients themselves. Using objective cognitive tests, such as the Mini-Mental State Examination (MMSE) [6] and the Abbreviated Mental Test, to detect dementia is intuitive and necessary given that cognitive impairment is a core manifestation of dementia. Such tests, however, suffer from several limitations. First, they only assess cognition at one point in time and do not reflect its decline over time. While it may be possible to detect changes in cognitive function with serial cognitive assessments, it may not be practical to perform repeat testing for large numbers of patients in busy clinical settings. Second, they do not assess the subject's functional status. Both of the above are essential for the clinical diagnosis of dementia. Third, several tests have also been shown to be susceptible to education, language and cultural influences, further limiting their utility as screening tests in general and multi-ethnic populations [7]. Thus, it is not surprising that the diagnostic accuracies of these cognitive tests in the primary care settings vary much in different studies. MMSE, the most commonly utilized cognitive instrument, performed only modestly in an unselected population. Depending on the cut-offs used, its sensitivity ranged from 71 to 92% and its specificity from 56 to 96% [5]. A study of 1,092 community-dwelling elderly in Singapore found that at a cut-off ≥24, MMSE had a sensitivity of 97.5% and a specificity of 75.6%. There was, however, a significant difference in performance according to educational level. The specificity in those with >6 years of education was 85.2%, but only 57.3% in the group with ≤6 years of education [7].

The other category of dementia tests involves subjective reports of functional decline, such as the Functional Activities Questionnaire [8] and AD8 [9]. These tests provide a relevance to daily functioning with good cross-cultural portability as well as a longitudinal perspective. AD8 comprises an eight-question brief informant interview focusing on intra-individual functional changes due to cognitive problems. Using the cut-off ≥2, it has been shown to detect dementia with a sensitivity of 74% and a specificity of 86% [9]. The main limitation of such instruments is that they rely per se on subjective recall of functional decline. In the absence of objective evidence of cognitive impairment, they are liable to subjective biases in reporting and carry the risk of under- or overreporting. There are also relatively fewer studies on the clinical utility of these questionnaires as compared to objective cognitive tests.

Given that dementia involves cognitive decline resulting in functional limitation, we propose that cognitive and functional assessments be combined within an assessment protocol. Mindful of the strengths and limitations of stand-alone cognitive and functional assessment instruments, we hypothesize that such a combination will improve the clinical relevance of the protocol and in turn result in better diagnostic performance. MMSE was chosen as the cognitive test of choice as it is the most commonly utilized test in both clinical and research settings, thereby enabling better standardization and acceptability. AD8 was chosen for the functional assessment component because of its excellent acceptability and ease of administration. It can generally be completed independently by the informant in ≤3 min [9,10].

Time constraint is a perennial problem in community and primary care settings. The MMSE, though brief, still requires about 15 min to administer. As such, we also explored the combination of only the subcomponents of MMSE, rather than the full test, with the AD8 in achieving the dual goals of good diagnostic accuracy and brevity. Therefore, the primary objective of our study was to examine the performance of the combination of MMSE and AD8 in differentiating early dementia patients from normal subjects. The secondary objective was to examine the performance of subcomponents of MMSE combined with AD8 in discriminating early dementia patients from normal subjects.

Methods

Setting

The subjects with dementia were recruited from consecutive assessments at the memory clinic of a tertiary hospital over a 15-month period. The relevant data were drawn from the memory clinic dementia registry and had prior approval from our institutional ethics review board. The normal controls were voluntary participants in a public health education and screening initiative held in conjunction with the opening of a new hospital in Singapore. They were community-dwelling elders with no cognitive impairment who were age-matched (≥60 years) to our subjects with dementia.

Participants

All subjects were aged ≥60 years, with adequate literacy in English or Mandarin to complete the assessment. They were required to have satisfactory vision and hearing. Satisfactory vision was defined as being able to read newsprint, while satisfactory hearing was defined as being able to carry out a normal conversation, with or without the use of hearing aids. In addition, the subjects had to have a reliable and literate informant who was able to complete AD8 in English or Mandarin.

For the dementia group, only patients with mild dementia, defined by a Clinical Dementia Rating Scale (CDR) [11] score ≤1, were included. Patients with mild cognitive impairment by the International Working Group criteria [12] were excluded. Patients with concomitant depression or psychiatric illness were excluded. For the normal controls, subjects with a known diagnosis of dementia, mild cognitive impairment, depression or other psychiatric disorders, any recent hospitalization in the past 3 months, a score ≥2 on the seven-item Geriatric Depression Scale [13] or a CDR score >0 were excluded.

Diagnosis and Assessments

All dementia subjects underwent a clinical assessment by consultant geriatricians specialized in dementia care which comprised a semi-structured interview of the subject and his informant as well as a physical examination. MMSE and CDR were also administered. Relevant blood investigations and magnetic resonance brain imaging were performed. A standardized neuropsychological test battery was performed in subjects with questionable dementia by a clinical psychologist. The final diagnosis was made via the consensus of a panel comprising geriatricians, a clinical psychologist and a dementia nurse clinician. Diagnosis of dementia and its subtype was based on the published Diagnostic and Statistical Manual of Mental Disorders, 4th edition (DSM-IV) criteria [14] and the National Institute of Neurological and Communicative Disorders and the Alzheimer's Disease and Related Disorders Association criteria [15] for Alzheimer's disease (AD) and AD with cerebrovascular disease, and the National Institute of Neurological Disorders and Stroke-Association Internationale pour la Recherche et l'Enseignement en Neurosciences [16] for probable vascular dementia. There was no subject with dementia with Lewy bodies, frontotemporal or Parkinson's disease dementia in this study population.

All dementia subjects had an accompanying informant during the memory clinic assessment. Each informant completed the AD8 questionnaire, which comprised eight questions pertaining to intra-individual change observed in the subject. The consensus panel for dementia diagnosis was blinded to the AD8 scores.

The healthy controls completed a questionnaire that included their demographics, general health, functional and cognitive status. MMSE and CDR were likewise administered. Clinical assessments of the controls were performed to ensure that they did not meet the DSM-IV criteria for dementia and delirium [14], as well as the International Working Group criteria for mild cognitive impairment [12]. AD8 questionnaires were completed by reliable informants, usually spouses or children, who had accompanied the subjects for health screening.

Statistical Analysis

All analyses were performed using SPSS version 20. Descriptive statistics were used to report the demographic characteristics of the patients as well as their CDR, MMSE and AD8 scores.

The items in AD8 are rated according to the number of functional deficits present in the subject (score range 0-8), whereas for the MMSE the total score reflects the number of correct responses (score range 0-30). For this reason, in the receiver operating characteristics (ROC) analysis to determine the optimal cut-offs, the AD8 score was reversed when combined with the MMSE total score or its subcomponents to align the scales in the same direction in order to score the total number of ‘positive’ responses.

Logistic regression models were generated with dementia as the dependent variable (CDR score 0 = 0, CDR score 0.5-1 = 1). MMSE and AD8 were entered into the logistic regression analyses to determine their ability to correctly classify subjects as having no dementia (CDR = 0) or mild dementia (CDR = 0.5-1). Next, a stepwise forward selection approach with AD8 incorporated into the model was undertaken to determine the contribution of individual subcomponents of MMSE in predicting dementia. The selected subcomponents of MMSE that were identified to be significant predictors of dementia in the presence of AD8 were then combined with AD8 to derive a parsimonious measurement combination of AD8 and MMSE. The odds ratio (OR) and confidence interval (CI) of each component was reported. ROC analysis was performed for the selected combination of AD8 and MMSE subcomponents to determine its diagnostic performance and optimal cut-off.

Results

Subject Characteristics

The sample comprised 245 subjects; 159 (64.9%) had mild dementia while 86 (35.1%) had no dementia. The demographic characteristics, CDR, MMSE and AD8 scores of the two groups are shown in table 1. The dementia group was older and comprised a higher proportion of male subjects compared to the control group. More than half of the subjects in both groups had <6 years of education. The dementia group had lower MMSE scores (20.8 vs. 26.8, p < 0.001) and higher AD8 scores (5.13 vs. 0.77, p < 0.001) than the normal control group.

Table 1

Demographics, CDR, MMSE and AD8 scores of the sample population

Characteristics	Dementia (n= 159)	No dementia (n = 86)
Age, years	76.6 ± 8.1	67.9 ± 7.0
Gender
Female	102 (64.1)	62 (72.1)
Male	57 (35.9)	24 (27.9)
Education
<6 years	103 (64.8)	45 (52.3)
≥6 years	56 (35.2)	41 (47.7)
CDR
0	0 (0)	86 (100)
0.5	52 (32.7)	0 (0)
1.0	107 (67.3)	0 (0)
Dementia subtype
AD	101 (63.5)
AD with CVD	42 (26.4)
Vascular dementia	16 (10.1)
MMSE	20.83 ± 4.310*	26.76 ± 2.687*
AD8	5.13 ± 2.395*	0.77 ± 1.205*
Three-word recall	1.31 ± 1.134	2.40 ± 0.674*
Intersecting pentagon copy	0.69 ± 0.462	0.95 ± 0.223*

Figures are mean ± SD or n (%).

CVD = Cerebrovascular disease.

p < 0.001.

Modeling AD8 and MMSE to Detect Dementia

The optimal cut-off identified for AD8 was 1/2 with the corresponding sensitivity of 89.9% and specificity of 80.2%. For MMSE, the optimal cut-off was 25/26, which provided a sensitivity of 77.9% and a specificity of 83.0%. AD8 (Wald χ2 = 61.62, OR = 2.55, 95% CI 2.03-3.22, p < 0.001) and MMSE (Wald χ2 = 56.76, OR = 1.57, 95% CI 1.39-1.76, p < 0.001) were both significant predictors of dementia. Logistic regression modelling showed that the combination (AD8+MMSE) contributed more to the observed variance in dementia (R2 = 0.742) when compared to either AD8 (R2 = 0.646) or MMSE (R2 = 0.506) alone (table 2).

Table 2

Models of AD8, MMSE and combinations of AD8 with MMSE individual components (Recall and Copy)

Model	Variable	Wald	p value	OR (95% CI)	Nagelkerke R2
Model 1	AD8	61.62	<0.001	2.55 (2.03 – 3.22)	0.654
Model 2	MMSE	56.76	<0.001	1.57 (1.39 – 1.76)	0.506
Model 3	AD8	40.21	<0.001	2.26 (1.75 – 2.90)	0.742
	MMSE	23.03	<0.001	1.41 (1.23 – 1.63)
Model 4	AD8	48.03	<0.001	2.43 (1.89 – 3.13)	0.708
	Recall	17.37	<0.001	2.77 (1.72 – 4.48)
Model 5	AD8	45.30	0.001	2.50 (1.91 – 3.26)
	Recall	13.86	0.001	2.54 (1.56 – 4.15)	0.727
	Copy	6.353	0.012	4.90 (1.42 – 16.9)

A forward selection approach in logistic regression combining AD8 with subcomponents of MMSE identified the three-item recall (Recall) (score range 0-3) and the intersecting pentagon copy (Copy) (score range 0-1) tests as the only two significant contributors. As depicted in table 2, the Recall and the Copy test each contributed independently as well as additionally. The coefficient of determination of AD8+Recall+Copy (model 5) was only marginally lower (R2 = 0.727) than the combination of AD8 with the full MMSE (model 3) (R2 = 0.742). The addition of the rest of the individual components of MMSE did not significantly contribute to the prediction of dementia.

Discriminative Performance of Models of AD8 and MMSE in Predicting Dementia

ROC were generated for AD8 alone, MMSE alone, AD8+MMSE, AD8+Recall and AD8+Recall+Copy (fig. 1). The area under the curve (AUC) for each model is reported in table 3. AD8 showed excellent diagnostic performance in differentiating mild dementia from no dementia with an AUC of 0.92 (95% CI 0.89-0.95) and was superior to MMSE (AUC = 0.87, 95% CI 0.83-0.92). The combined AD8+MMSE, with an AUC of 0.95 (95% CI 0.92-0.98) increased its superior discrimination over MMSE alone. The briefer versions of AD8+Recall (AUC = 0.94, 95% CI 0.91-0.97) and AD8+Recall+Copy (AUC = 0.95, 95% CI 0.92-0.97) performed comparably with the combination of AD8 and full MMSE.

Fig. 1

ROC comparing the utility of AD8, MMSE, AD8+MMSE, AD8+Recall and AD8+Recall+Copy in discriminating mild dementia from no dementia.

Table 3

AUCs, optimal cut-off points and corresponding sensitivities, specificities and diagnostic accuracies of the models

Models	AUC (95% CI)	Cut-off	Sensitivity	Specificity	Diagnostic accuracy
AD8 (total = 8)	0.92 (0.89 – 0.95)	5/6	90.7%	84.3%	86.5%
MMSE (total = 30)	0.87 (0.83 – 0.92)	25/26	77.9%	83.0%	81.2%
AD8+MMSE (total = 38)	0.95 (0.92 – 0.98)	29/30	91.9%	84.3%	86.9%
AD8+Recall (total = 11)	0.94 (0.91 – 0.97)	7/8	88.4%	84.9%	86.1%
AD8+Recall+Copy (total = 12)	0.95 (0.92 – 0.97)	8/9	87.2%	88.1%	87.8%

Discussion

The results support our hypothesis that the combination of AD8 and MMSE improves detection of mild dementia. The MMSE, one of the most commonly used cognitive tests [5], performed modestly in our study population. In comparison, the addition of a functional assessment component in the form of AD8 to MMSE resulted in a significantly improved diagnostic performance (sensitivity 91.9%, specificity 84.3%). The combination of AD8 and MMSE is intuitive and meaningful as it assesses both objective cognitive deficits and longitudinal intra-individual functional decline. The addition of a subjective component can improve the credibility and acceptability of the test results by subjects and their families compared to a test that assesses only cognition. Previous studies have also explored the use of informant interviews in conjunction with objective cognitive measures. For example, one study found that the AD8 combined with Word List Recall improved dementia detection [17]. Nevertheless, given the time and manpower constraints in primary care settings, we recognize brevity to be a key priority. The AD8+MMSE combination would require at least 15 min to administer, rendering it less practical for use in primary care. For this reason, it is pertinent to derive even briefer versions of the test.

We found that AD8 in combination with only three-item recall and intersecting pentagon tests (AD8+Recall+Copy) achieved diagnostic performance comparable to the AD8 with the full MMSE. At a cut-off of 8/9 for AD8+Recall+Copy (total score = 12) and 29/30 for AD8+MMSE (total score = 38), both combinations have comparable sensitivities (87.2 and 91.9%, respectively) and specificities (88.1 and 84.3%, respectively). Therefore, in settings where limited manpower and time are essential, the relatively shorter test (AD8+Recall+Copy) appears to be an appropriate substitute for the AD8+MMSE protocol.

The two subcomponents, namely three-item recall and intersecting pentagon copy, of the MMSE test the important cognitive domains of memory and spatial constructional praxis. They are also relatively less likely to suffer from language or educational bias compared to the other subcomponents of MMSE. To operationalize the brief (AD8+Recall+Copy) test in practice, we recommend that the Recall test be initiated first by presenting the three items to the subject. This is then followed by administering the AD8 to the informant and subsequently the Copy test to the subject. The test is concluded by asking the subject to recall the three items. The final score is obtained by summing the reverse AD8 score (maximum 8 points) with the Recall (maximum 3 points) and Copy (maximum 1 point) to give a maximum possible score of 12.

This study has its limitations. It is not a population-based study and the use of subjects in a tertiary memory clinic limits its generalizability to the general population. The use of a volunteer sample from a health screening program may pose the risk of selection bias, and likewise a larger proportion of subjects with <6 years of education in the sample. Ideally, the two groups should also be matched in factors other than age, such as language and education. However, the modest size of our sample limits meaningful stratification into further subgroups for analysis. Our study included only subjects with AD, vascular dementia and AD with cerebrovascular disease, hence the conclusions may not be extrapolated to other dementia subtypes. An important point of note is that this study included only subjects with mild dementia, with the goal of being able to detect early cognitive decline. As such, the diagnostic performance of the proposed assessment protocol may be even better when applied to settings that include subjects beyond the early stages of dementia.

Further studies should be performed in the primary care setting to examine the performance (sensitivity and specificity), applicability and acceptability of this proposed method for detecting early dementia. A potential impediment to the application of this diagnostic process is that a knowledgeable informant is not always available. Limited studies thus far have shown varying degrees of accuracy of self-rated AD8 in detecting dementia [18,19]. Based on our study procedures, we recommend informant rating instead of self-rating of the AD8.

Conclusion

This study confirms that AD8, combined with three-item recall and intersecting pentagon copy of the MMSE, is a valid and convenient tool of good diagnostic utility for mild dementia. It is brief and easily administered, supporting its potential as a practical and inexpensive means to detect dementia, especially in resource-constrained primary care and community settings.

Disclosure Statement

The authors have no conflict of interest to declare.