Usefulness of the Clock Drawing Test as a Cognitive Screening Instrument for Mild Cognitive Impairment and Mild Dementia: an Evaluation Using Three Scoring Systems.

BACKGROUND AND
PURPOSE: Although the clock drawing test (CDT) is a widely used cognitive screening instrument, there have been inconsistent findings regarding its utility with various scoring systems in patients with mild cognitive impairment (MCI) or dementia. The present study aimed to identify whether patients with MCI or dementia exhibited impairment on the CDT using three different scoring systems, and to determine which scoring system is more useful for detecting MCI and mild dementia.
METHODS: Patients with amnestic mild cognitive impairment (aMCI), vascular mild cognitive impairment (VaMCI), mild Alzheimer's disease (AD), mild vascular dementia (VaD), and cognitively normal older adults (CN) were included. All participants were administered the CDT, the Korean-Mini Mental State Examination (K-MMSE), and the Clinical Dementia Rating scale. The CDT was scored using the 3-, 5-, and 15-point scoring systems.
RESULTS: On all three scoring systems, all patient groups demonstrated significantly lower scores than the CN. However, while there were no significant differences among patients with aMCI, VaMCI, and AD, those with VaD exhibited the lowest scores. Area under the Receiver Operating Characteristic curves revealed that the three CDT scoring systems were comparable with the K-MMSE in differentiating aMCI, VaMCI, and VaD from CN. In differentiating AD from CN, however, the CDT using the 15-point scoring system demonstrated the most comparable discriminability with K-MMSE.
CONCLUSIONS: The results demonstrated that the CDT is a useful cognitive screening tool that is comparable with the Mini-Mental State Examination, and that simple CDT scoring systems are sufficient for differentiating patients with MCI and mild dementia from CN.

INTRODUCTION

Cognitive screening tests have long been used as an initial step in the assessment of dementia. Ideally, a cognitive screening test should be brief, acceptable to patients, easy to score, independent of educational/cultural/language confounders, psychometrically robust, and broad in its coverage of cognitive domains.12 In this regard, the value of the clock drawing test (CDT) as a screening instrument for global cognitive deficits has been recognized by many studies due to its ease of use and brief administration time for patients with mild cognitive impairment (MCI) and/or various types of dementia.34 Proficiency in the CDT requires a wide range of cognitive domains, such as auditory comprehension, sustained attention, visuospatial ability, memory, abstract thinking, planning, motor execution, and executive function.1234

Researchers have used slightly different instructions and methodologies to administer the CDT to cognitively impaired patients. These include using a pre-drawn circle, additional copying or time-reading commands, as well as free-drawing, which is the most commonly used method.45678 In addition, several different scoring systems have been used, such as the two-point system in the Saint Louis University Mental Status9; the 3-point system in the Montreal Cognitive Assessment (MoCA)10; the five-point system in the Alzheimer's Disease Neuroimaging Initiative (ADNI)'s cognitive assessment; the 15-point system in the Behavioural Neurological Assessment (BNA)11; and the 20-point system in the Mendez's scoring system.12 Although there is no consensus on which CDT scoring system is the most effective, recent reviews suggest that simpler scoring systems are better because of their ease of use and their strong correlations with more complex systems.23

The utility of the CDT for screening patients with dementia as compared to normal participants has been widely accepted, and many studies have found that the positive and negative predictive values of the CDT are good.131415 Additionally, the value of the CDT for differentiating patients with MCI from those with dementia has been recognized in many studies.1617 However, there have been inconsistent findings regarding the utility of the CDT in discriminating between MCI and healthy normal controls. Some researchers have reported that the CDT is not useful for differentiating MCI from normal participants,181920 while others have found that the CDT is a valuable screening tool for MCI patients.1617

Previous studies examining whether the CDT could differentiate types of dementia have yielded conflicting results.21222324 For example, one study found that patients with Alzheimer's disease (AD) demonstrated poorer performance on the CDT than those with vascular dementia (VaD),22 whereas other studies reported that VaD patients scored lower than AD patients.2324 Considering the subtypes of MCI, no difference was found in the performances on the CDT between amnestic mild cognitive impairment (aMCI) and vascular mild cognitive impairment (VaMCI) patients.2526 However, to date, no studies have examined the validity of the CDT in differentiating these two MCI groups.

Collectively, we assumed that these inconsistent findings regarding the utility of the CDT may be attributed to differences in the scoring systems used, the severity of cognitive impairment, and the subtypes of MCI and dementia among study participants. Therefore, the present investigation aimed to identify whether an impairment in the CDT exists for MCI as well as dementia using three different scoring systems. In addition, we examined which scoring system is most useful for detection in patients with MCI (aMCI and VaMCI) and mild dementia (AD and VaD).

METHODS

Participants

The subjects were selected from a group of patients who were diagnosed by neurologists in the Department of Neurology at Hallym University Sacred Heart Hospital (Anyang, Korea). There were 42 patients with aMCI who were diagnosed based on Petersen's criteria.27 A total of 40 patients with mild AD were diagnosed based on the criteria described by the National Institute on Aging-Alzheimer's Association (NIA-AA) workgroups.28 Forty-one patients with VaMCI were diagnosed with criteria for probable VaMCI from the American Heart Association-American Stroke Association (AHA-ASA).29 Forty patients with VaD were diagnosed based on criteria from the AHA-ASA.29 All patients underwent a comprehensive neuropsychological assessment and brain magnetic resonance imaging (MRI). Forty-five community-dwelling older adults were recruited through community outreach. They were screened based on Christensen's health screening criteria30 and on having a total score higher than the 16th percentile on the Korean-Mini Mental State Examination (K-MMSE)31 (cognitively normal older adults; CN). Clinical psychology graduate students who had been trained by the author (YK) visited welfare centers for older adults in the community, then administered the CDT to older adults. The present study is a retrospective study using the data already collected for clinical use and other purposes. Thus, informed consents from the participants could not be obtained.

Administration of the CDT

The CDT was conducted using the free-drawn command. The instruction was as follows: “Draw a face of the clock. Put in all the numbers and set the time to 10 past 11.” Participants were not permitted to look at a clock or wristwatch when drawing this clock. The instructions were repeated for individuals who could not understand them, and there was no time limitation.

Three CDT scoring systems

The three most popular scoring systems used in the clinical field were selected. The 3-point scoring system adopted for the MoCA10 assigns one point each for drawing a closed circle, placing all expected numbers in their correct positions, and correctly placing the clock hands to reflect the requested time. The 5-point scoring system was adopted from the ADNI's cognitive assessments, and was developed and modified by Goodglass and Kaplan.32 One point each is assigned for drawing an approximately circular face, the symmetry of number placement, the correctness of the numbers, the presence of two hands, and the hands showing correct the lengths and time. The 15-point scoring system, developed by Freedman et al.,33 was used in the BNA.11 Two points are given for contour, 6 points for numbers, 6 points for clock hands, and 1 point for the center. These three scoring systems are summarized in Fig. 1. The scoring for the CDT was performed by a clinical neuropsychologist (SK).

Fig. 1

Scoring systems of the clock drawing test.

*“O” all of the scoring criteria in the corresponding column must be satisfied in order to assign points in each scoring system.

Other measures

Several studies have reported that the Mini-Mental State Examination (MMSE) is insufficiently sensitive to detecting MCI, and is particularly insensitive to deficits in executive functioning and visual constructive ability3435; nevertheless, the MMSE remains the most widely used cognitive screening instrument and is considered to be the “gold standard” for general cognitive function.1 Therefore, the K-MMSE31 was administered in comparing the utility of the CDT as a cognitive screening tool. The Clinical Dementia Rating (CDR)36 scale was also rated in all patients to obtain information regarding the severity of dementia.

Statistical analysis

After examining whether there were significant differences in demographic characteristics, multivariate analysis of covariance was performed to compare group differences in total K-MMSE scores and the three CDT scores after controlling for demographic variables. Receiver operating characteristic (ROC) curve analyses were performed in order to examine the ability of the three CDT scoring systems to differentiate aMCI, VaMCI, AD, and VaD from CN, while demographic variables were controlled as covariates. Areas under the receiver operating characteristic curves (AUCs) for the three CDT scoring systems were compared with those for the K-MMSE to ensure that they were comparable with the K-MMSE as a cognitive screening instrument. Additionally, multiple comparison tests of the AUCs were performed to identify which scoring system was the most useful for each patient group.

RESULTS

Characteristics of demographic variables, K-MMSE, and CDR

Demographic variables as well as K-MMSE and CDR scores for CN, aMCI, VaMCI, AD, and VaD are presented in Table 1. There were no significant differences among the groups in levels of education (F[4, 203]=1.26, ns). However, there were significant group differences in age, with the VaMCI group being the youngest (F[4, 203]=3.55, p<0.05). There were also significant group differences in sex (X[4, 203]=10.64, p<0.05). The average scores of the K-MMSE in AD and VaD groups were significantly lower than those in the other three groups, while the CN group exhibited higher scores than the aMCI and VaMCI groups (F[4, 203]=34.30, p<0.001). There were significant group differences in the CDR-Global Score (CDR-GS) and CDR-Sum of Boxes (CDR-SB), with the AD and VaD groups exhibiting higher scores than the aMCI and VaMCI groups (CDR-GS: F[3, 160]=212.93, p<0.001; CDR-SB: F[3, 160]=115.92, p<0.001).

Table 1

Demographic characteristics of the CN, mild cognitive impairment, and mild dementia patient groups

Variable	CN (n=45)	aMCI (n=42)	VaMCI (n=41)	AD (n=40)	VaD (n=40)	F or Χ2	Post-hoc* (Tukey)
Age (yr)	70.67 (8.39)	71.19 (8.60)	67.20 (10.46)	73.85 (8.96)	73.07 (9.20)	3.55†	a=b=d=e, a=b=c, c<d=e
Sex (M/F)	24/21	24/18	29/12	14/26	21/19	Χ2=10.6†	-
Education (yr)	8.09 (3.56)	8.88 (4.82)	8.07 (4.46)	9.69 (4.31)	9.43 (4.17)	1.26	ns
K-MMSE	28.40 (1.40)	25.19 (3.03)	25.63 (3.14)	21.50 (3.84)	22.10 (3.79)	34.30‡	a>b=c>d=e
CDR-GS	-	0.50 (0.00)	0.46 (0.13)	0.95 (0.15)	0.96 (0.13)	212.93‡	b=c<d=e
CDR-SB	-	1.92 (0.88)	1.43 (1.10)	5.29 (1.46)	5.26 (1.41)	115.92‡	b=c<d=e

CN: cognitively normal older adults, aMCI: amnestic mild cognitive impairment, VaMCI: vascular mild cognitive impairment, AD: Alzheimer's disease, VaD: vascular dementia, K-MMSE: Korean Mini-Mental State Examination, CDR-GS: clinical dementia rating-global score, CDR-SB: clinical dementia rating-sum of boxes.

*a: CN, b: aMCI, c: VaMCI, d: AD, e: VaD; †p<0.05; ‡p<0.001.

CDT results scored using three CDT scoring systems

The scoring results for the CDT based on the 3-, 5-, and 15-point scoring systems are summarized in Table 2. The patients in the aMCI, VaMCI, AD, and VaD groups exhibited significantly lower scores than the CN group in all of the CDT scores recorded using the 3 scoring systems (F[4, 203]=9.53, p<0.001; F[4, 203]=17.45, p<0.001; F[4, 203]=11.60, p<0.001, respectively). Post-hoc analyses for the three-point scoring system revealed that the VaMCI, AD, and VaD groups performed worse than the CN group, although there was no significant difference between the aMCI and CN groups. There were no significant differences between patients with VaMCI and those with mild dementia. With the 5-point scoring system, the aMCI, VaMCI, AD, and VaD groups each exhibited poorer performance than the CN group. The VaD group had the lowest score, while there were no significant differences among the aMCI, VaMCI, and AD groups. Using the 15-point scoring system, the aMCI, VaMCI, AD, and VaD groups demonstrated significantly worse scores than the CN group. There were no significant differences between the aMCI and VaMCI, or the AD and VaD groups. The aMCI group exhibited significantly higher scores than the VaD, although the VaMCI did not demonstrate any differences with the AD or VaD groups.

Table 2

Clock drawing test performances of the CN, mild cognitive impairment, and mild dementia patient groups

Variable	CN (n=45)	aMCI (n=42)	VaMCI (n=41)	AD (n=40)	VaD (n=40)	F	Post-hoc* (Bonferroni)
3-point	2.87 (0.40)	2.55 (0.67)	2.38 (0.71)	2.15 (0.83)	1.93 (0.83)	9.53†	a=b>e, a>c=d=e, b=c=d
5-point	4.89 (0.32)	3.76 (0.98)	4.27 (0.95)	3.70 (1.07)	3.20 (1.29)	17.45†	a>c=b=d, a>c=d>e, e=b
15-point	14.87 (0.34)	13.40 (2.21)	13.68 (2.07)	12.40 (2.53)	11.93 (2.81)	11.60†	a>c=d=e, a>b>e, b=c=d

CN: cognitively normal older adults, aMCI: amnestic mild cognitive impairment, VaMCI: vascular mild cognitive impairment, AD: Alzheimer's disease, VaD: vascular dementia.

*a: CN, b: aMCI, c: VaMCI, d: AD, e: VaD; †p<0.001.

Comparisons of AUCs

The AUCs for the three scoring systems and the K-MMSE total score are shown in Fig. 2 and Supplementary Table 1. ROC curve analysis revealed that all three CDT scores derived from the three scoring systems were comparable with the K-MMSE in differentiating patients with aMCI, VaMCI, and VaD from those were CN. Multiple comparison tests revealed that the AUCs of the 3 CDT scores in each comparison (aMCI vs. CN, VaMCI vs. CN, and VaD vs. CN) did not significantly differ from one another. However, it was found that the CDT scored using the 15-point scoring system differentiated AD from CN significantly better than that scored using the three-point scoring system (AUC difference, 0.037; 95% confidence interval [CI], 0.002–0.071; p<0.05). Moreover, the K-MMSE discriminated AD from CN significantly better than the CDT scored using the three-point scoring system (AUC difference, 0.088; 95% CI, 0.016–0.159; p<0.05) and the five-point scoring system (AUC difference, 0.083; 95% CI, 0.015–0.151; p<0.05), whereas there was no significant difference between the CDT scored using the 15-point scoring system and the K-MMSE.

Fig. 2

Receiver operating characteristic curves of CDT and K-MMSE for mild cognitive impairment and dementia patient groups.

CDT: clock drawing test, K-MMSE: Korean Mini-Mental State Examination, aMCI: amnestic mild cognitive impairment, VaMCI: vascular mild cognitive impairment, AD: Alzheimer's disease, VaD: vascular dementia.

DISCUSSION

The present study aimed to examine whether an impairment in the CDT exists for MCI and/or dementia with the three different scoring systems commonly used for the CDT. Our results demonstrated that CDT scores in the aMCI and VaMCI groups were significantly lower than those of the CN group when the CDT was scored using the 5- or 15-point scoring systems. However, when using the three-point scoring system, no significant differences were found between the aMCI and CN groups. In addition, there were no significant differences in CDT scores between the aMCI and VaMCI groups between all of the three scoring systems. Some recent studies have reported that there is no significant difference in CDT performance between MCI and CN subjects,3738 while others have found that MCI patients demonstrate worse scores than cognitively unimpaired (i.e., CN) participants.39 The results of the present study suggest that deficits do exist in the performance of the CDT, even for aMCI and VaMCI patients, although a simple scoring system, such as the three-point system, was not able to detect subtle deficits in patients with aMCI.

The results also demonstrated that the CDT scored using any of the three scoring systems could not provide any information regarding the severity of dementia (i.e., MCI vs. dementia). There were no significant differences among the aMCI, VaMCI, and AD groups in CDT scores as scored using all three scoring systems. However, the VaD group always demonstrated the lowest CDT scores, and a significantly poorer performance than AD was found for the VaD group using the 5-point scoring system. The CDT has been known to assess various cognitive functions with more focus on executive control and visuospatial functions.40 Although deficits in executive functions and episodic memory occur in the very early stages of AD,41 many studies have demonstrated that VaD exhibits more impaired executive function and visuoconstructional skills.42 A recent study performed a qualitative analysis of error types in CDT and found that, compared to AD patients, VaD patients exhibited more stimulus-bound responses, difficulty in planning, and perseveration errors that reflected frontal lobe dysfunction.43 In this regard, our results demonstrating that the VaD group exhibited the lowest CDT scores in all three scoring systems re-confirms that the CDT is a good measure of frontal/executive functions.

Our results also demonstrated that all CDTs scored using the three different scoring systems yielded satisfactory AUCs comparable with the K-MMSE in the MCI and VaD groups, although we did not observe significant differences between the aMCI and CN groups in the total mean scores of the CDT using the 3-point scoring system. This result indicates that the CDT is a useful cognitive screening instrument for aMCI, VaMCI, and VaD, regardless of the scoring system used. Recently, review studies have suggested that elaborating the detail and complexity of CDT scoring systems does little to improve the test's ability to identify significant cognitive impairment.23 Our results were also consistent with these findings. Thus, we conclude that simple scoring systems (i.e., 3- and 5-point) for the CDT can be as good as complex scoring systems (i.e., 15-point) for differentiating MCIs and VaD from CN.

However, the 15-point scoring system was found to have significantly higher discriminatory power than the 3-point scoring system for differentiating AD from CN, although the 3- and 5-point scoring systems also had significant discriminability. The 15-point scoring system includes more items relating to visuospatial skills, such as “contour is circular with the ratio of the shortest diameter no greater than 1:1.5,” “paper not rotated,” “hands overlap or join within 12 mm,” and “center drawn or inferred”, which are not scored in the 3- or 5-point systems. Visuospatial function relies on the parietal lobe, which is affected in the early stages of AD.444546 Therefore, it appears that the better discriminability of the 15-point scoring system for AD was due to the fact that it evaluated these items relating to visuospatial functions, while the other systems did not. Moreover, only the CDT scored using the 15-point scoring system demonstrated comparable discriminability with the K-MMSE for differentiating AD from CN. Therefore, these results support the use of more complex scoring systems, including more items relating to visuospatial functions, for differentiating AD from CN. However, in the case that there is some reason that a simple scoring system must be used, we recommend using a combination of the CDT and MMSE, similar to what some previous studies have suggested.4748

The present study had several limitations. First, it was a retrospective investigation with a relatively small sample size. Second, we could not control the effects of medication on CDT performance in the patient groups. Third, we did not distinguish aMCI into two groups, such as single-domain or multiple-domain; therefore, further research classifying the different subtypes of aMCI is necessary. Fourth, we only applied the “free-drawing to verbal command” method and quantitative scoring, as they are the ones most commonly used in clinical settings. Thus, further research using other conditions, such as copying and qualitative analysis (e.g., error type analysis), is needed in order to further discern the utility of the CDT.

In summary, our results demonstrated that deficits in CDT performance are present even in patients with aMCI and VaMCI as well as individuals with mild dementia, indicating that the CDT is a useful cognitive screening tool with results comparable to those of the MMSE. The clinical implications of the results suggest that simple scoring systems for the CDT, such as a three-point system, are sufficient for differentiating VaMCI and VaD from CN, although more detailed scoring systems, such as a 15-point system, are better for aMCI and AD.