Diagnostic performance of a combination of Mini-Mental State Examination and Clock Drawing Test in detecting Alzheimer's disease.

OBJECTIVE: Because of the growing need for quick cognitive screening tests to distinguish Alzheimer's disease (AD) from mild cognitive impairment (MCI), we compare the diagnostic performance of a combination of the Mini-Mental State Examination (MMSE) and a Clock Drawing Test (CDT) to the Japanese version of the Alzheimer's Disease Assessment Scale-cognitive subscale (ADAS-J cog) in differentiating between patients with AD, patients with MCI, and healthy controls (HC).
METHODS: Data from 146 subjects with AD and 60 subjects with MCI, as well as 49 HC, was retrospectively analyzed. We used logistic regression analysis with diagnosis as dependent variables and scores of the MMSE, the CDT-command, and the CDT-copy as independent variables, and receiver operating characteristic analysis to distinguish patients with AD from patients with MCI or HC.
RESULTS: When patients with AD were compared to HC, the independent predictors of AD were scores on the MMSE and the CDT-command. This combination was more sensitive than the MMSE alone and has nearly the same sensitivity and specificity as the ADAS-J cog. When patients with AD were compared to patients with MCI, the independent predictors were the MMSE and the CDT-copy. This combination was more sensitive and specific than the MMSE alone and was almost as sensitive and specific as the ADAS-J cog.
CONCLUSION: The combination of the MMSE and the CDT could be a powerful screening tool for differentiating between patients with AD, patients with MCI, and HC. Its sensitivity and specificity are comparable to ADAS-J cog, which takes more time.

Introduction

Following the introduction of the concept of mild cognitive impairment (MCI),1 general practitioners are frequently required to follow patients diagnosed with MCI and decide whether patients convert from MCI to Alzheimer’s disease (AD). This distinction is essential to starting treatment, including the prescription of cholinesterase inhibitors, in a timely fashion. Considering that MCI has been defined as presence of cognitive impairment among intact daily functions, accurate assessment of functional status is important to detect the conversion from MCI to AD. Although performance-based assessments are ideal measures for functional status, they are difficult to administer in busy primary care settings. Instead, cognitive tests that are quick and easy to administer are feasible for general practitioners.

Although several screening instruments are available for detecting dementia, the Mini-Mental State Examination (MMSE) and the Clock Drawing Test (CDT) have been widely used, particularly as quick cognitive screens for dementia in the primary care setting.2 Both the MMSE and the CDT are reported to be sensitive in identifying cognitive impairments. A recent meta-analysis of 34 dementia studies and five MCI studies suggested MMSE had a modest sensitivity of 77% and a specificity of 90% for application in high-prevalence specialist settings and a sensitivity of 81% and a specificity of 87%.2 Although CDT varies in scoring, a comprehensive literature review suggested that all CDT scorings are remarkably consistent, and both sensitivity and specificity levels have a mean of 85%.3 However, the MMSE, in combination with the CDT, has an improved diagnostic performance (sensitivity, 82%–100%; specificity, 83.9%–95.4%), and it is recommended as an easily administered and accurate instrument.4–7 Together, these two assessments take about 10 minutes to perform.

In contrast, the Japanese version of the AD Assessment Scale cognitive subscale (ADAS-J cog) is predominantly used in randomized controlled trials that examine the symptomatic benefits of therapy for AD.8,9 The ADAS-J cog consists of eleven tasks measuring the disturbances of memory, language, praxis, attention, and other cognitive abilities, which are often referred to as the core symptoms of AD. It is useful as an instrument for evaluating dementia in a specialized psychogeriatric outpatient setting. However, it takes more than 30 minutes to perform and is thus difficult to use when time is limited.

In this study, we sought to compare diagnostic performance of a combination of the MMSE and the CDT with the ADAS-J cog differentiating between patients with AD, patients with MCI, and healthy controls (HC). The goal was to clarify whether this combination would lead to higher or comparable accuracy than the ADAS-J cog.

Methods

Subjects

Subjects with MCI (n = 60) and AD (n = 146) were recruited from patients in the Aino Hospital. The diagnosis of MCI was made according to Petersen et al’s criteria,1 which corresponds to amnestic MCI by reported memory concerns, memory impairment on standard tests, absence of significant impairment in activities of daily living, and the absence of dementia. The diagnosis of AD was made according to the National Institute of Neurological and Communicative Disease and Stroke-Alzheimer’s Disease and Related Disorders Association’s (NINCDS-ADRDA) criteria for probable AD,10 respectively. All subjects had been assessed comprehensively by a geriatric psychiatrist, received physical and neurological examinations, and reported their medical histories. Clinical data including functional status was confirmed by family members or other available caregivers. All pertinent laboratory tests and magnetic resonance imagings (MRIs) were reviewed.

HC (n = 49) were individuals living in the community who participated in the dementia prevention program provided by Aino Hospital. Exclusion criteria were a score of less than 25 on the MMSE,11 and a score of less than 6 on the ADAS-J cog.9 An interviewer had available access to the social workers in charge of each community before starting the dementia prevention program provided by Aino Hospital to confirm that HC led an independent life, with both basic and instrumental intact daily living activities.

This study was conducted at Aino Hospital and approved by the hospital’s institutional review board. Informed consent was obtained from all subjects or from their substitute decision makers after receiving a complete description of the study.

Instruments

Subjects were interviewed by trained clinical psychologists with the CDT, the MMSE, and the ADAS-J cog.

CDT

For the CDT, subjects were presented with a blank sheet of paper and given the following instructions: “I would like you to draw a clock, put in all numbers, and set the hands for 10 past 11” (CDT-command). Next, they were presented with a drawing of a clock and asked to copy it (CDT-copy). The creators of the instrument, Rouleau et al,12 scored subjects on a 10-point scale that is designed to assess the accuracy of the clock face representation (maximum 2 points), the layout of the numbers (maximum 4 points), and the position of the hands (maximum 4 points) independently.

MMSE

The MMSE11 is a frequently used screening instrument in the evaluation of cognitive impairment. It consists of 30 points grouped into seven categories: orientation, registration, attention, calculation, recall, language, and visual construction.

ADAS-J cog

The ADAS-J cog9 is generally considered the gold standard by regulatory authorities for assessing cognitive function in AD drug trials. It consists of 70 points, with higher scores indicating greater cognitive impairment, which are grouped into eleven categories: word recall, naming, commands, constructional praxis, ideational praxis, orientation, word recognition, spoken language, comprehension, word finding, and remembering instructions.

Statistical analysis

Gender was compared using the Chi-square test. Age and scores of the MMSE, the CDT-command, the CDT-copy, and ADAS-J cog between patients with AD, patients with MCI, and HC were compared using one-way analysis of variance.

An analysis of covariance was used to compare scores from the MMSE, the CDT-command, the CDT-copy, and ADAS-J cog between patients with AD, patients with MCI, and HC, with age and gender as the covariates. Post hoc testing was performed using Tukey’s honestly significant difference tests. Logistic regression analysis was used to optimize the classification performance of MMSE, CDT-command, and CDT-copy scores, namely, with diagnosis as the dependent variable and scores of the MMSE, the CDT-command, and the CDT-copy as the independent variables. Receiver operating characteristic analysis was used to compare diagnostic performance of the optimized combination with ADAS-J cog scores. Data was analyzed using the Statistical Package for the Social Sciences 17.0 (SPSS; IBM Corporation, Armonk, NY, USA), and resultant statistics where P < 0.05 were considered significant.

Results

Subjects’ characteristics

The demographic characteristics with respect to age and gender for each group, along with MMSE, CDT-command, CDT-copy, and ADAS-J cog mean scores, are shown in Table 1. Mean age was significantly lower in HC as compared to patients with MCI and AD.

Table 1

Mean (SD) values for selected clinical variables by group

	HC(n = 49)	MCI(n = 60)	AD(n = 146)
Gender (male/female)	17/32	16/44	39/107
Age (years)	66.1 (8.0)	76.1 (9.2)a	79.1 (8.0)a,d
MMSE (score out of 30)	29.1 (1.2)	24.6 (4.1)a	19.1 (5.1)a,b
ADAS-J cog (score out of 70)	3.5 (1.6)	10.7 (6.1)a	19.2 (8.1)a,b
CDT-command (score out of 10)	9.5 (0.9)	7.6 (2.7)a	5.6 (2.7)a,b
CDT-copy (score out of 10)	10.0 (0.2)	9.7 (0.7)	8.2 (2.3)a,b

Notes:

P < 0.001 versus HC;

P < 0.001 versus MCI;

P < 0.05 versus HC;

P < 0.05 versus MCI.

Abbreviations: SD, standard deviation; HC, healthy controls; n, number; MCI, mild cognitive impairment; AD, Alzheimer’s disease; MMSE, Mini-Mental State Exam; ADAS-J cog, Japanese version of the Alzheimer’s Disease Assessment Scale-cognitive subscale; CDT, clock drawing test.

Diagnostic performance

The mean scores in all four neuropsychological tests (MMSE, CDT-command, CDT-copy, and ADAS-J cog) were significantly worse in the AD group as compared to the MCI group and HC (Table 1). After adjusting for the confounding variables of age and gender, AD patients had significantly worse performance in all neuropsychological tests (Table 2).

Table 2

Mean (SD) ANCOVA values

	HC(n = 49)	MCI(n = 60)	AD(n = 146)
Adjusted for age and gender
MMSE	28.1 (0.7)	24.6 (0.5)	19.4 (0.4)a,b
ADAS-J cog	4.8 (1.0)	10.7 (0.9)a	18.7 (0.6)a,b
CDT-command	9.0 (0.4)	7.6 (0.3)c	5.8 (0.2)a,b
CDT-copy	10.0 (0.3)	9.7 (0.2)	8.2 (0.2)a,b

Notes:

P < 0.001 versus HC;

P < 0.001 versus MCI;

P < 0.05 versus HC.

Abbreviations: SD, standard deviation; ANCOVA, analysis of covariance; HC, healthy controls; n, number; MCI, mild cognitive impairment; AD, Alzheimer’s disease; MMSE, Mini-Mental State Exam; ADAS-J cog, Japanese version of the Alzheimer’s Disease Assessment Scale-cognitive subscale; CDT, clock drawing test.

When the AD group was compared to the HC group, the areas under the curve (AUC) were 0.988 (confidence interval [CI] = 0.976−0.999) for MMSE (P < 0.001), 0.912 (CI = 0.871−0.952) for CDT-command (P < 0.001), 0.764 (CI = 0.700−0.829) for CDT-copy (P < 0.001), and 0.989 (CI = 0.979−1.000) for ADAS-J cog (P < 0.001). MMSE and CDT-command as identified by logistic regression analysis led to slightly higher values than MMSE alone, and values were nearly the same as those obtained with ADAS-J cog alone (0.997 [CI] = 0.987−1.000, P < 0.001), yielding a sensitivity of 91.1% and a specificity of 100.0%. The positive predictive value was 100.0% with the prevalence of AD at 74.9% in the sample (Table 3 and Figure 1). The equation used in logistic regression for differentiating between patients with AD and HC was:

Table 3

Diagnostic performance

Variables	AUC(with 95% CI in parentheses)	Cut-off	Accuracy
			SE	SP	PPV	LR+
AD versus HC
MMSE	0.988 (0.976–0.999)***	24/25	85.6	100.0	100.0	–
CDT-command	0.912 (0.871–0.952)***	8/9	82.9	87.8	95.3	6.8
CDT-copy	0.764 (0.700–0.829)***	10	55.5	95.9	97.6	13.5
ADAS-J cog	0.989 (0.979–1.000)***	7.5/7.6	95.2	100.0	100.0	–
MMSE + CDT-command	0.997 (0.987–1.000)***	0.39	91.1	100.0	100.0	–
AD versus MCI
MMSE	0.795 (0.730–0.860)***	23/24	74.7	68.3	85.2	2.4
CDT-command	0.711 (0.630–0.792)***	8/9	74.0	58.3	81.2	1.8
CDT-copy	0.686 (0.614–0.759)***	10	55.5	75.0	84.4	2.2
ADAS-J cog	0.806 (0.741–0.871)***	13.5/13.6	75.3	71.7	86.6	2.7
MMSE + CDT-copy	0.811 (0.747–0.875)***	0.67	75.3	70.0	85.9	2.5
MCI versus HC
MMSE	0.837 (0.761–0.912)***	26/27	63.3	95.9	95.0	15.4
CDT-command	0.717 (0.622–0.812)***	8/9	50.0	87.8	83.4	4.1
CDT-copy	0.606 (0.501–0.711)	–	–	–	–	–
ADAS-J cog	0.893 (0.831–0.955)***	13.5/13.6	80.0	87.8	88.9	6.6

Notes: Values are AUC (with 95% CI in parentheses).

P < 0.001.

Abbreviations: AUC, area under the curve; CI, confidence interval; SE, sensitivity; SP, specificity; PPV, positive predictive value; LR+, likelihood positive ratio; AD, Alzheimer’s disease; HC, healthy controls; MMSE, Mini-Mental State Exam; CDT, Clock Drawing Test; ADAS-J cog, Japanese version of the Alzheimer’s Disease Assessment Scale-cognitive subscale; MCI, mild cognitive impairment.

Figure 1

ROC curves. (A) Subjects with Alzheimer’s disease versus healthy controls; (B) subjects with Alzheimer’s disease versus mild cognitive impairment.

Abbreviation: ROC, receiver operating characteristic.

The AUC against the AD group and the MCI group were 0.795 (CI = 0.730−0.860) for MMSE (P < 0.001), 0.711 (CI = 0.630−0.792) for CDT-command (P < 0.001), 0.686 (CI = 0.614−0.759) for CDT-copy (P < 0.001), and 0.806 (CI = 0.741−0.871) for ADAS-J cog (P < 0.001). The best predictors of AD identified by logistic regression analysis were MMSE and CDT-copy. This combination led to slightly higher AUC than MMSE and was nearly the same as that obtained with ADAS-J cog (0.811 [CI] = 0.747−0.875, P < 0.001), yielding a sensitivity of 75.3% and a specificity of 70.0%. The likelihood ratio of a positive test was 2.5. The positive predictive value was 85.9% with the prevalence of AD at 70.9% in the sample, according to the statistical coefficients shown (Table 3 and Figure 1). The equation used in logistic regression for differentiating between patients with AD and patients with MCI was:

The AUC in differentiating MCI group from HC group were 0.837 (CI = 0.761−0.912) for MMSE (P < 0.001), 0.717 (CI = 0.622−0.812) for CDT-command (P < 0.001), and 0.893 (CI = 0.831−0.955) for ADAS-J cog (P < 0.001) (Table 3). However, neither CDT-copy nor the combination of MMSE and CDT was identified by logistic regression analysis.

Discussion

We examined the abilities of a combination of the MMSE and the CDT administered according to different procedures against the ADAS-J cog alone to differentiate between patients with AD or MCI and HC. In differentiating patients with AD from HC, the combination of the CDT-command and the MMSE had better sensitivity and specificity, while a combination of the CDT-copy and the MMSE had better sensitivity and specificity in differentiating AD from MCI. The screening properties of these combinations were comparable to the ADAS-J cog.

When AD was compared to HC, the combination of the MMSE and the CDT-command was more sensitive than the MMSE alone, and has nearly the same sensitivity and specificity as the ADAS-J cog. These findings are consistent with previous studies that found enhanced accuracy in the detection of AD with the combined MMSE and CDT, as opposed to the MMSE or the CDT alone.4–7 Heinik et al6 showed that combining the MMSE and the CDT improved sensitivity to 100% and specificity to 91%, as compared with the MMSE alone (sensitivity, 96%; specificity, 81%). Further, this combination performed better than the Cognitive and self-contained part of the Cambridge Examination for Mental Disorders of the Elderly, a 30-minute multicognitive task.13 We found that the MMSE combined with the CDT-command showed enhanced sensitivity as a first-stage dementia screen, as CDT-command requires multiple cognitive domains, such as receptive language, memory, abstraction, and executive functions.3,13–15

When patients with AD were compared to patients with MCI, the combination of the MMSE and the CDT-copy had better sensitivity and specificity than the MMSE or the CDT alone, and was nearly the same as that obtained with the ADAS-J cog. The CDT-copy mainly requires visuospatial and visuoconstructional function,3,15 as with the doublepentagon copy of the MMSE. However, the CDT-copy might be more sensitive for measuring visuospatial and visuoconstructional functions than the double-pentagon copy, because the CDT-copy requires the spatial layout of the complex component features of a clock. In addition, the double-pentagon copy is only worth 1 point on the MMSE, and disability might be underestimated as measured by the total MMSE score. As a screening test, CDT-command is more popular than CDT-copy. However, our results suggest that CDT-copy is useful to differentiate AD from MCI if it is administered with MMSE.

The combination of the MMSE and the CDT was found to have a higher sensitivity than (89.4%) and the identical specificity of (83.9%) the MMSE alone (sensitivity, 86.4%; specificity, 83.9%) in detecting mild AD and MCI.5 From our data, we found that the MMSE has good sensitivity (74.7%), but it lacks good specificity (68.3%); conversely, the CDT-copy lacks sensitivity (55.5%) but has high specificity (75.0%) in differentiating MCI from AD. Thus, this combination enhanced accuracy complementarily.

Hence, the combination of MMSE and CDT would be useful as one of the assessments when general practitioners evaluate dementia and decide timing to prescribe a cholinesterase inhibitor in a primary care or community setting.

On the other hand, the combination of the MMSE and the CDT was not able to accurately differentiate patients with MCI from HC in this study. Cacho et al5 studied 66 individuals with mild AD, 21 individuals with MCI, and 66 HC individuals in a memory clinic, and found that the AUC for this combination (mini-clock) were higher than what was obtained using the MMSE or the CDT alone in differentiating patients with MCI from HC. Cacho et al5 concluded that mini-clock assessment is reasonably accurate in distinguishing patients with MCI from HC; however, this study had small sample sizes, especially for patients with MCI. Only a few studies have tried to test the screening properties of this combination in differentiating between patients with MCI and HC. Hence, further research is necessary to conclude whether this combination can be used to differentiate patients with MCI from HC.

This study has several limitations. First, the MCI construct has still been controversial, and many types and stages of MCI have been proposed. We diagnosed subjects with MCI solely based on their informant-based functional status. However, some cases might possibly have met criteria for dementia if their functional status had been more strictly assessed. First, we should have defined stages of MCI, early or late MCI, and used the activities of daily living and instrumental activities of daily living to evaluate the functional autonomy of AD, not only MCI. Second, HC were significantly younger as compared to patients with MCI and AD. This may have enhanced sensitivity too much. However, it is noteworthy that cognitive scores for the MMSE, the CDT-command, and the ADAS-J cog were significantly worse in individuals with AD and MCI compared to HC, even after adjusting for the confounding variable of age. Third, we did not collect data on the level of education of our subjects. Since HC were sampled from a dementia prevention program, they might be more likely to have higher education levels than the patients. However, all subjects are known to have at least received the Japanese compulsory education of 6 years.

In summary, these results are consistent with previous studies that reported high sensitivity and specificity with the combination of the MMSE and the CDT when AD was compared with HC and with MCI. Our results also suggest that the combination of the MMSE and the CDT may be a similarly accurate replacement for the ADAS-J cog, which is to be used when screening time is limited.