Comparison of the diagnostic accuracy of neuropsychological tests in differentiating Alzheimer's disease from mild cognitive impairment: can the montreal cognitive assessment be better than the cambridge cognitive examination?

OBJECTIVE: Considering the lack of studies on measures that increase the diagnostic distinction between Alzheimer's disease (AD) and mild cognitive impairment (MCI) and on the role of the Cambridge Cognitive Examination (CAMCOG) in this, our study aims to compare the utility of the CAMCOG, Mini-Mental State Examination (MMSE) and Montreal Cognitive Assessment (MoCA) in helping to differentiate AD from MCI in elderly people with >4 years of schooling.
METHOD: A total of 136 elderly subjects - 39 normal controls as well as 52 AD patients and 45 MCI patients treated at the Institute of Geriatrics and Gerontology, Porto Alegre, Brazil - were assessed using the MMSE, CAMCOG, clock drawing test (CDT), verbal fluency test (VF), Geriatric Depression Scale and Pfeffer Functional Activities Questionnaire.
RESULTS: The results obtained by means of a receiver operating characteristic curve showed that the MoCA is a better screening test for differentiating elderly subjects with AD from those with MCI than the CAMCOG and MMSE as well as other tests such as the CDT and VF.
CONCLUSION: The MoCA, more than the CAMCOG and the other tests, was shown to be able to differentiate AD from MCI, although, as Roalf et al. [Alzheimers Dement 2013;9:529-537] pointed out, further studies might lead to measures that will improve this differentiation.

Introduction

According to the new diagnostic criteria for dementia, the use of cognitive tests assumes great importance during the process of diagnostic investigation [1,2]. Several tests are used in gerontology, and those with higher accuracy are the most important [3]. The Cambridge Mental Disorders of the Elderly Examination (CAMDEX) is a tool that provides a complete frame for neuropsychological aspects involved in dementia diagnosis. But several problems can be found in this diagnostic process, such as the assessment of years of schooling, so that patients with dementia and a high educational level can score high in cognitive assessment and cognitive decline is not identified. In addition, the CAMDEX was not developed for the diagnosis of mild cognitive impairment (MCI) [4]. The Cambridge Cognitive Examination (CAMCOG) is part of the CAMDEX and has demonstrated good accuracy in predicting Alzheimer's disease (AD) in patients with a lower educational level [5]. This test appears to be affected by several variables such as age, gender and social class and has been vulnerable to educational bias [6].

Other instruments have demonstrated better predictive power for AD, such as the Mini-Mental State Examination (MMSE) [7] and Montreal Cognitive Assessment (MoCA) [8]. The MMSE was developed in 1975 and since then has undergone changes regarding the cutoff points among elderly with different schooling levels. The MoCA is a more recent test developed in 2005 and is more useful in screening cases of mild dementia and MCI. When compared with the MMSE, the MoCA has more memory and executive function items as well as a more structured language subitem [8]. These qualities, together with a greater diagnostic accuracy when compared with the MMSE, made the MoCA an important screening test for all educational levels [8].

Education plays an important role in modulating age-related changes in cognition. Lower education implies a higher risk of dementia and cognitive impairment in the elderly, and this fact becomes more worrying when considering developing countries with a large portion of the elderly having fewer years of schooling [9,10]. Hence, a better understanding of the effects of education on cognition is needed, as well as good screening tests for dementia and MCI as these can lead to AD on all educational levels [11].

The study of Roalf et al. [11] aimed to compare the utility of the MoCA with that of the MMSE in helping in AD and MCI diagnosis. The authors administered the tests to 321 AD patients, 126 MCI patients and 140 controls. The Consortium to Establish a Registry of Alzheimer's Disease (CERAB-NB) standardized neuropsychological battery was also administered to be compared with other cognitive tests regarding the differentiation of groups. The results pointed out a good diagnostic accuracy of the clinical tools, and the MoCA was the best test, with the highest sensitivity and specificity in differentiating MCI patients from normal controls (NC). The authors concluded that even though the MoCA is superior to other tools in discerning earlier stages of cognitive decline, the overall diagnostic accuracy is improved when its information is combined with an informant-based functional measure, and they suggest that new measures are necessary to increase diagnostic specificity in distinguishing between AD and MCI.

The authors did not employ the CAMCOG test in their comparison, which is a good assessment tool for differentiating MCI from AD patients [4] and AD patients from normal elderly [5]. Hence, considering the lack of studies with measures that increase diagnostic specificity in differentiating between AD and MCI and the role of the CAMCOG in this, our study aimed to compare the utility of the CAMCOG, MMSE and MoCA in helping in AD and MCI diagnosis in elderly subjects with >4 years of schooling. We also included the verbal fluency test (VF) and the clock drawing test (CDT) in our research as these tests have also proven their efficiency in differentiating between MCI, AD and control groups [12].

Methods

This cross-sectional study was carried out at the Institute of Geriatrics and Gerontology in Porto Alegre, Brazil, beginning in April 2010 and ending in December 2012. A total of 136 elderly subjects aged over 60 years – both genders, with an educational level of >4 years – were studied. This sample was composed of three diagnostic groups, which are described more closely below.

Patients with other dementia syndromes, such as vascular dementia, frontotemporal dementia, dementia with Lewy bodies or any other clinical symptoms that would suggest non-Alzheimer's dementia, were excluded from the sample. We also excluded patients with severe dementia (Clinical Dementia Rating scale score ≥3), a history of stroke, paralysis in both hands, major depressive disorder, Parkinson's disease, tremors, severe hearing and visual impairments, and <4 years of schooling.

The diagnostic criteria for dementia were based on those of the DSM-IV [13] and NINCDS-ADRDA [2]. For the diagnosis of MCI, the inclusion criteria comprised not having problems with activities of daily living, not fulfilling dementia syndrome criteria by cognitive assessment and not having cognitive complaints. The study included only patients with amnestic MCI, which includes only elderly with memory impairment.

As cognitive test scores were considered dependent variables in the present study, they were not used as diagnostic criteria. All AD and MCI diagnostics as well as neuropsychological assessment were made by an experienced gerontologist. Neuroimaging and other neurological examinations were conducted to investigate whether the participants fulfilled the inclusion criteria, i.e. that they did not have any other neurological problems that justified their symptoms. The inclusion criteria for the NC were scoring above the cutoff points on the neuropsychological tests, having no dementia symptoms, scoring <7 on the Geriatric Depression Scale (GDS) and not having problems performing activities of daily living.

Of the 136 participants, 52 elderly (38.2%) received a diagnosis of AD, 45 elderly (33.1%) received a diagnosis of MCI and 39 elderly (28.7%) comprised the NC. The majority of the sample were women (n = 89; 65.4%), and the mean age of the sample was 75.74 years (±7.38 years; range: 60-92 years). The sample had a higher number of elderly people with >9 years of schooling (58.8%).

The mean age of the control group was 71.82 years (±6.89 years; range: 60-89 years) and 74.4% were women; the control group's mean educational level was 7 years (±1.75 years). The AD group had a mean age of 77.92 years (±6.97 years; range: 64-91 years), with women comprising 63.5%; this group had a mean educational level of 7 years (±1.80 years). The MCI participants had a mean age of 76.60 years (±7.06 years; range: 63-92 years), with 60% women; this group had a mean of 6 years of schooling (±1.94 years). The variables age, sex and education show the considerable participation of women in the three diagnostic groups and that the AD group had the highest mean age.

In spite of the different educational levels (χ2 = 9.15; p = 0.517) and age ranges (χ2 = 7.35; p = 0.356) of the three groups, they did not provide significant differences. The sex ratios also did not differ significantly between the groups, with all of them having a predominance of women. This research was approved by the Research Ethics Committee (process No. 54/11).

Instruments and Procedures

All patients were submitted to detailed clinical and neuropsychological screening. The neuropsychological tests applied were the CAMCOG [14], MMSE [7], CDT [15,16], Brazilian version of the MoCA [17], GDS with 15 short items [18], Pfeffer Functional Activities Questionnaire (PFAQ) [19] and VF [20].

The CAMCOG is a cognitive battery that is integrated in the CAMDEX, developed by Roth et al. [14]. The maximum score is 107 and the cutoff point for dementia is 80 points. Its subtests assess memory, digit span, concentration, sentence, praxis, abstract thought and perception. The MMSE was developed by Folstein et al. [7], having a maximum score of 30 points and with cutoff points for dementia varying with patients' educational levels. The VF is one of the simplest tests for assessing cognitive impairment in initial phases of a dementia syndrome. Brucki et al. [20] suggest cutoff points according to years of schooling, varying from 9 to 13 points. The semantic type was employed in the fruit category, and in words with M, the phonemic type was used. The CDT criteria were based on Mendez et al. [15] and Shulman et al. [16].

After the clinical and neuropsychological evaluation, we performed an interview with an informant or a relative in daily contact with the respective elderly participant in order to report possible symptoms and add other relevant information such as behavioral changes. All patients were submitted to the CAMCOG after having been diagnosed.

Statistical Analysis

Considering the sample calculation, we used a 5% significance level and sample error (confidence interval). The minimum number of participants had to be 101. The collected sample from April 2010 to December 2012 totaled 136 participants, reaching the minimum sample size required.

All collected data were analyzed by SPSS version 15.0 (2007). Descriptive statistics concerning the sample distribution (age, gender and educational level) were prepared, setting a significance level of 5%. The variables were described by using frequencies and means with standard deviations.

We also calculated the accuracy of the MoCA, MMSE, CAMCOG and CDT by the receiver operating characteristic (ROC) curve methodology. Most of the measures, with exception of numbers and pointers from the MoCA, had a nonnormal distribution assessed by means of the Kolmogorov-Smirnov normality test; hence, we chose to employ nonparametric tests to evaluate group differences. In addition, we used the χ2 and Kruskal-Wallis tests to compare MoCA scores between the NC, MCI and AD groups in order to check which subitems presented significant differences between the groups.

Results

All test scores were above the cutoff points for the NC, and they were higher than in the other groups. The mean scores for the MCI group were also above the mean in the CAMCOG (88.4 points; cutoff: ≥80 points), in the CDT according to the criteria by Mendez et al. [15] (18.5 points; cutoff: ≥18 points) and by Shulman et al. [16] (4.2 points; cutoff: ≥3 points) and in the animal version of the VF (13.9 points; cutoff: ≥13 points). The scores for the fruit version (mean: 11.0 points) and words with the letter M (mean: 10.4 points) of the VF were below the cutoff expected (12 and13 points, respectively). In the MCI group, the GDS scores were <5 points and the average PFAQ score was compatible with independence in activities of daily living (<5 points).

In the AD group, only the CDT measure by the Shulman scale had a mean value above the cutoff point (3.6 points; cutoff: ≥3 points). The PFAQ score indicated greater dependence in activities of daily living (table 1). The cutoff score for the MMSE for schooling from 5 to 8 years was 26 points. According to this cutoff value, the group diagnosed with MCI and the NC scored above. Only the group with AD had a lower average (22.9 points).

Table 1

Comparison of test scores according to study groups

	MCI (n = 45)		AD (n = 52)		NC (n = 39)		p	MCI × NC	AD × NC	MCI × AD
	mean	SD	mean	SD	mean	SD
MoCA	22.1	2.5	16.7	3.8	26.9	1.8	<0.0001	<0.0001	<0.0001	<0.0001
CAMCOG	88.4	6.6	77.2	11.5	97.7	5.4	<0.0001	<0.0001	<0.0001	<0.0001
MMSE	26.9	2.0	23.4	3.9	29.1	1.2	<0.0001	<0.0001	<0.0001	<0.0001
Mendez	18.5	2.8	16.6	3.9	19.5	0.9	<0.0001	0.0034	<0.0001	0.0017
Shulman	4.2	0.7	3.6	0.9	4.6	0.5	<0.0001	0.0045	<0.0001	0.0005
VF, animals	13.9	4.2	10.9	3.2	17.5	5.0	<0.0001	0.0012	<0.0001	0.0008
VF, fruits	11.0	3.1	8.8	2.9	14.9	3.8	<0.0001	<0.0001	<0.0001	0.0014
VF, ‘M’	10.4	4.5	8.4	3.4	14.1	4.6	<0.0001	0.0006	<0.0001	0.0532
GDS	2.8	2.8	2.6	2.3	2.1	2.1	0.3887	–	–	–
PFAQ	3.7	4.2	12.6	9.2	0.8	1.9	<0.0001	<0.0001	<0.0001	<0.0001
VF points	10.0	3.9	8.1	3.3	12.3	4.0	<0.0001	0.0053	<0.0001	0.0197

VF, ‘M’ = VF words with letter M. Kruskal-Wallis test.

In the sample with >9 years of schooling, only the NC had a higher mean score, and the expected cutoff was 28 points. However, considering the standard deviation (±1.902 points) in the group with MCI, it can be inferred that the fact that this group scored just 1 point below the cutoff point could be due to measurement error, i.e. one could find patients with MCI scoring normally on the MMSE.

The ROC curve analysis showed that the highest value for the area under the curve was found for the MoCA, i.e. the MoCA was most effective in distinguishing between the MCI and NC groups, with a sensitivity and specificity of 82.2 and 92.3%, respectively. The same test was also important for distinguishing between the NC and AD groups, with 98.1% sensitivity and 100% specificity, and between the MCI and AD groups, with 95.8% sensitivity and 88.1% specificity. The cutoff values presented for the MoCA to differentiate between the diagnostic groups were 21 points for AD versus NC and 24 points for MCI versus AD. The cutoff value that distinguished the MCI from the NC group was 25 points. The respective ROC curves are presented in figures 1, 2, 3.

Fig. 1

ROC curve for NC and AD. Sunderland: study by Sunderland et al. [28].

Fig. 2

ROC curve for NC and MCI. Sunderland: study by Sunderland et al. [28].

Fig. 3

ROC curve for MCI and AD. Sunderland: study by Sunderland et al. [28].

Discussion

The main objective of this study was to compare the scores of cognitive tests between MCI, AD and NC groups in community-dwelling elderly patients with >4 years of schooling. With regard to the educational level, we observed that most (69.2%) of the healthy controls had >9 years of schooling. One hypothesis to explain the significant differences in this variable refers to a cognitive reserve that the extra years of study provide [21]. Other studies found in the literature corroborate these findings regarding the influence of education on cognitive test results. Diniz et al. [22] discuss the influence of education on MMSE scores, as do Matallana et al. [23], who analyzed a larger number of participants (2,861 elderly) from native Hispanic populations and found elevated scores on tests among older people with high education. Agreeing with the present results, Brito-Marques and Cabral-Filho [24] studied 253 participants divided into groups according to age (60-65, 66-70, 71-75, 76-80, 81-85 and 86-90 years) and educational level (1-4, 5-8 and >8 years of schooling), and the MMSE results showed differences between the age groups and the schooling groups.

Age and years of schooling are important data that should be considered during any neuropsychological interview with elderly subjects. In a survey, Valls-Pedret et al. [25] used test figures with the aim of evaluating visual episodic memory in patients with and elderly without cognitive impairment (137 normal elderly and 126 patients with AD). This study suggested that the youngest group (50-59 years) achieved higher average test scores than individuals >80 years of age and that there was a significant difference within the control group, i.e. differences in test scores were found even among the elderly who presented no evidence of dementia, demonstrating the influence of age on memory tests. This result differs from the one found in this study because there were no differences in age and we demonstrated that even at the same ages, these tests can discriminate between MCI, AD and NC.

Regarding years of education, several studies [22,23,24,25] have corroborated the results of this research, which shows that older people with more years of schooling have higher average scores in cognitive tests. Diniz et al. [22] went further and referred to high educational level as a protective factor against dementia, and according to their hypothesis, brain stimulation caused by years of study produces a cognitive reserve that lowers the risk of developing dementia. In this sense, the research conducted by Yassuda et al. [6] on the effects of memory training on elderly subjects corroborates the ideas raised by Diniz et al. [22]. Age and education are variables considered to be risk factors for dementia, i.e. the higher the age and the lower the educational level of an individual, the more likely there is a decline in cognitive functions.

The MoCA displayed the best diagnostic accuracy when compared with the MMSE, CAMCOG, VF and CDT. Other studies have also indicated the effectiveness of the MoCA as a tool aiding in differential diagnosis. Dong et al. [26] analyzed 230 elderly patients with MCI, and their results showed that the MoCA had a significantly greater area under the curve than the MMSE (0.92 vs. 0.84). Furthermore, Roalf et al. [11] suggested that the MoCA was the best test to differentiate AD patients from controls. Their findings corroborate the data of this study, where the three diagnostic groups were better distinguished by the MoCA than by the other instruments. Another study supporting the hypothesis that the MoCA is an effective tool for the diagnosis of cognitive decline was performed by Markwick et al. [27], in which the authors found that even in cognitively impaired patients the MMSE scores were normal, while the scores of the MoCA pointed to a decline, which led them to the conclusion that the MoCA appeared to be a more sensitive screening test for the early detection of cognitive decline. This finding is consistent with the findings of this study, where the MoCA was the test with the highest sensitivity and specificity in differentiating MCI from AD cases and NC.

One of the main criteria of the Brazilian Ministry of Health for dispensing costly drugs for AD is an MMSE score between 12 and 24 points for elderly subjects with >4 years of schooling. This is also one of the requirements in Brazil in order to qualify for free medication. We observed that the average scores for AD patients were 23.57 points (5-8 years) and 27 points (>9 years), and that the standard deviations were 3.9 and 1.4 points, respectively. This indicates that many patients with dementia may not be able to get their medicine as they do not fulfill the inclusion criterion of the Ministry of Health. Therefore, it is suggested that the MoCA be included in any assessment of suspected cases of dementia as it has a higher predictive value for patients with >4 years of schooling.

Conclusion

The MoCA proved to be an appropriate tool when screening for MCI among elderly subjects in Brazil with >4 years of schooling. It was also shown that the MoCA displayed a higher accuracy than the CAMCOG, VF and CDT in differentiating AD and MCI from healthy aging. Studies with larger numbers of participants are needed to further validate the test also for elderly with low education. Importantly, studies of this type tend to provide a greater diagnostic accuracy and thus improve response procedures and rehabilitation, as well as favoring the understanding of the symptomatology. The CAMCOG and other tests were not better than the MoCA in differentiating the groups as we hypothesized at the beginning. Hence, the MoCA, more than the CAMCOG and the other tests, was shown to be able to differentiate AD from MCI, although, as Roalf et al. [11] pointed out, further studies might lead to better measures that will improve this differentiation.