Validation of the Malay Version of Addenbrooke's Cognitive Examination III in Detecting Mild Cognitive Impairment and Dementia.

BACKGROUND/AIMS: This study aimed to investigate the validity and reliability of the Malay version of Addenbrooke's Cognitive Examination III (ACE-III) for detecting mild cognitive impairment (MCI) and dementia.
METHODS: A total of 152 participants (dementia = 53, MCI = 38, controls = 61) were recruited from two teaching hospitals. The Malay version of ACE-III was translated following the standard guidelines for cross-cultural adaptation of measure. All the participants were assessed with the Malay version of ACE-III and Mini-Mental State Examination (MMSE).
RESULTS: The reliability of the Malay version of ACE-III was good with Cronbach's α coefficient of 0.829 and intraclass correlation coefficient of 0.959. There was a strong positive correlation between the Malay version of ACE-III and MMSE (r = 0.806). Age (r = -0.335) and years of education (r = 0.536) exerted a significant correlation with total score performance. The cutoff score to discriminate dementia from healthy controls was 74/75 (sensitivity = 90.6%, specificity = 82.0%) whereas to discriminate MCI, the cutoff score was 77/78 (sensitivity = 63.2%, specificity = 63.9%). The diagnostic accuracy of ACE-III was higher than that of MMSE in the detection of dementia (area under the curve: ACE-III = 0.929 vs. MMSE = 0.915).
CONCLUSIONS: The Malay version of ACE-III demonstrated to be a reliable and valid screening tool for dementia.

Introduction

With an increasing population of older adults globally, both the incidence and prevalence of dementia is set to increase substantially worldwide [1]. The need for early screening and detection of dementia is paramount in providing timely intervention to delay the disease progression, prolong independence, and enhance the quality of life of the individual with dementia. Therefore, the availability of sensitive and specific cognitive assessment tools to detect cognitive impairment is vital.

The Mini-Mental State Examination (MMSE) [2] is the most commonly used brief cognitive screening test in the clinical, research and community setting. However, psychometric limitations of the MMSE, such as sensitivity to practice effects [3] and large floor and ceiling effects [4], have been reported. Moreover, systematic reviews and meta-analysis concluded that the MMSE has limitations in distinguishing people with mild cognitive impairment (MCI) or mild Alzheimer's disease (AD) from healthy individuals [5, 6]. To overcome these limitations, several other cognitive rating scales have been developed [7]. In 2000, Addenbrooke's Cognitive Examination (ACE) was initially developed as a modified form of the MMSE and reported as a brief and reliable screening tool for mild dementia as well as for differentiating AD from frontotemporal dementia (FTD) [8]. In 2006, the revised version of Addenbrooke's Cognitive Examination-Revised (ACE-R) [9] was published. Content modifications on the naming component, visuospatial component, and the creation of three different alternative versions with different stimuli for the name and address recall were made on ACE-R in order to facilitate cross-cultural usage and to increase sensitivity. Finally, the 26 components of ACE-R were combined to produce 5 subscores with each subscore representing a specific cognitive domain [9]. The ACE-R further exhibits its usefulness for both screening and diagnostic properties and hence has been widely used in both clinical and research settings and adopted in different languages globally [10, 11, 12, 13, 14, 15].

In 2013, Addenbrooke's Cognitive Examination III (ACE-III) was developed after rectifying the relative weakness of its predecessor ACE-R [16]. Main changes were made on the language and visuospatial domains. For example, the three-stage command is replaced by a short graded test of grammatical comprehension and the overlapping pentagons are replaced by the intersecting infinity loops. The validation of ACE-III domains against standard neuropsychological tests has exhibited a high correlation between domain scores and the targeted tests [16]. Besides, the ACE-III demonstrates a higher diagnostic accuracy over the MMSE [17] and Montreal Cognitive Assessment [18] in screening dementia. Likewise, the ACE-III has also been validated in different languages globally [17, 19, 20]. Hence this study aims to validate the Malay version of ACE-III for detecting people with MCI and dementia.

Methods

Study Design

This is a cross-sectional study in which participants were recruited from the Memory Clinic and Psychogeriatric Clinic, Department of Psychiatry, University Kebangsaan Malaysia Medical Centre (UKMMC) and the Memory Clinic of the University Malaya Medical Centre (UMMC). Convenient sampling was used in this study. Participants were informed about the purpose and procedure of the study and written informed consent was obtained before the assessments. The recruited participants were categorized into three groups: dementia group, MCA group, and healthy control group. This study was approved by the UKM Research Ethics Committee, University Kebangsaan Malaysia and the UMMC Research Ethics Committee.

Participants

All participants were 60 years old and above and able to speak, read and write in the Malay language. Participants must have sufficient hearing, sight and physical ability to complete the assessments. Individuals diagnosed with amnestic MCI or dementia according to DSM-5 by a geriatrician were recruited into the respective groups. The geriatrician established the diagnosis based on the Clinical Practice Guidelines: Management of Dementia by the Ministry of Health Malaysia [21]. Healthy controls were recruited among patients' family members attending the clinics. Healthy controls were independent in their activities of daily living and did not have memory complaints. The participants were excluded if they had a history of psychosis, major mental health problem, severe physical condition or uncorrected sensory problem, e.g. severe uncorrected impairment of vision or hearing, which might interfere with their ability to complete the assessment.

All the participants took the MMSE (Malay version) and ACE-III (Malay version) assessments. In order to prevent bias, the MMSE (Malay version) assessment was conducted by a trained nurse or occupational therapist of the respective clinics, whereas the investigator performed the ACE-III (Malay version) assessment.

Instruments

MMSE (Malay Version)

The MMSE is a tool commonly used to screen cognitive impairment including dementia [2]. It measures 5 domains of cognitive function: orientation, registration, attention and calculation, recall, and language. The score ranges from 0 to 30 where higher scores indicate better cognition. The Malay version of the MMSE was validated in a local setting with modifications on the test of attention and repetition of questions. The Malay version of the MMSE is a valid and reliable screening tool for dementia in the Malaysian population [22, 23].

Addenbrooke's Cognitive Examination III

The original English version of the ACE-III was developed and validated after improving the weakness of its predecessor ACE-R [16]. It consists of 5 domains, each representing a specific cognitive function: attention (18 points), memory (26 points), fluency (14 points), language (26 points), and visuospatial component (16 points). The total score of the ACE-III is 100 points with higher scores indicating better cognitive function. The ACE-III and ACE-R highly correlated (r = 0.99, p < 0.01). Two cutoff scores which were similar to its predecessor ACE-R were recommended: (1) 88 (sensitivity = 1.0; specificity = 0.96) and (2) 82 (sensitivity = 0.93; specificity = 1.0) to discriminate dementia. In addition, the ACE-III has good internal consistency with Cronbach's α coefficient of 0.88. The cognitive domains within ACE-III also showed high levels of correlation between the domain scores and its targeted standardized neuropsychological tests [16].

Translation and Adaptation of the ACE-III

The original English version of the ACE-III was translated into the Malay language following the MAPI translation guidelines [24]. First, the original version of ACE-III was translated into the Malay language by two local professional independent translators. These two forward translations were then compared and discussed by both the translators and researchers and a reconciled version was produced. In the event of discrepancy, the original author was contacted for advice and clarification on the reconciled version and adaptation was made accordingly.

The reconciled version was adapted culturally and linguistically after seeking the advice of the author. In the orientation part of the attention domain, we replaced season by time as Malaysia is located near the equator and without change of seasons. For verbal letter fluency, we replaced the letter “P” by the letter “B” during the 1-min word generation beginning with the specific alphabet. In the memory domain, the names and addresses in anterograde memory, recall and recognition were replaced by common Malay names and addresses. As for retrograde memory, the adaptations were that the subject was asked to name the first Prime Minister or commonly known as Father of Independence rather than the woman who was Prime Minister and Singapore's first Prime Minister rather than the US president who was assassinated in the 1960s. Major adaptations were made on the language domain. As Malaysia is a multiracial country, we included the major festivals (Christmas, Deepavali, Chinese New Year or Hari Raya) in the instruction to facilitate the subject sentence writing part. As regards the single-word repetition items, we replaced “caterpillar, eccentricity, unintelligible, statistician” by “jururawat, keselamatan, penyelenggaraan, keberkesanan,” which were multisyllabic words with a decreased familiarity as per the original version. Similarly, the proverb repetition was replaced by common Malay proverbs. In the object naming section, we replaced the majority of the pictures which might not be familiar with the older generation, e.g., “kangaroo,” “penguin,” “harp,” “rhinoceros,” “barrel,” “crown,” “alligator,” and “accordion” by “tiger,” “mouse deer,” “guitar,” “elephant,” “pail,” “keris” (Malay traditional sword), “crocodile,” and “badminton.” Therefore, all the instructions in the comprehension section were also adapted. The adapted instructions were: “point to the one which is associated with a traditional Malay weapon,” “point to the one which is only found in the Borneo and Sumatra rainforest,” “point to the one which is a Malaysian favorite sport,” and lastly “point to the one which is related to the desert.” Finally, in the reading section, all the words “sew, pint, soot, dough, and height” were replaced by irregular Malay words of “mengelip,” “nyala,” “masyhur,” “sengal,” and “engsel,” which break the normal phoneme-grapheme rules as per the original version to test on subjects with semantic dementia.

Following this, the adapted reconciled Malay version was back-translated by a bilingual professional translator who was blinded to the original version of the questionnaire. The backward translated version was revised by a committee panel comprising of a psychogeriatrician, clinical psychologist, geriatrician, and neurologist to fine-tune the translation and ensure the content validity and cultural appropriateness. A pilot study involved 15 Malay-speaking participants (5 from each category). The final version of the Malay ACE-III was developed after further improvement from the pilot study suggestions.

Statistical Analysis

Statistical analyses were computed using the SPSS 23.0. The χ2 test was used for categorical demographic data. The one-way analysis of variance (ANOVA) was used to determine the statistical significance in demographic variables (age and education), MMSE score, total and subdomain scores of ACE-III between groups, followed by post hoc Bonferroni analyses. Internal consistency was measured by Cronbach's α coefficient. Test-retest reliability was evaluated by the intraclass correlation coefficient. Concurrent validity was calculated using a two-tailed Pearson correlation between ACE-III total score and MMSE score. Normative data at different age groups were compared using one-way ANOVA with post hoc Bonferroni analyses. Pearson correlation and Spearman's rho correlation were used to gauge the effect of age, years of education, and gender and the coefficient of determination was generated. The receiver operating characteristic (ROC) was graphed to compare the areas under the curve (AUC) and define the sensitivity and specificity at different cutoff points.

Results

A total of 152 participants with an age range from 61 to 90 years were recruited in this study: 61 were healthy controls, 38 displayed amnestic MCI, and 53 were diagnosed with dementia. The majority of the patients in the dementia group were diagnosed with AD (60.4%), followed by vascular dementia (32.1%) and mixed dementia (7.5%).

Demographic Characteristics of the Dementia, MCI and Control Groups

Table 1 illustrates the comparison of demographic parameters, MMSE scores, ACE-III total scores, and its subdomain scores among these three groups. The majority of the participants of each group were Malay, followed by Chinese. There were no significant differences in gender (χ2 = 1.50, df = 1, p = 0.472), age [F(2, 149) = 0.60, p = 0.55] and years of education [F(2, 149) = 1.94, p = 0.47] among these groups. Contrarily, significant differences were observed in the mean scores of the MMSE [F(2, 149) = 52.20, p < 0.001], mean scores of ACE-III total [F(2, 149) = 60.91, p < 0.001] and its subdomains of attention [F(2, 149) = 37.17, p < 0.001], memory [F(2, 149) = 62.48, p < 0.001], fluency [F(2, 149) = 54.28, p < 0.001], language [F(92, 149) = 12.85, p < 0.001], and visuospatial component [F(2, 149) = 11.62, p < 0.001] between these groups. Post hoc analysis revealed that the dementia group had significantly lower mean scores for the MMSE, ACE-III, and all 5 subdomains when compared with the MCI and control groups. Comparison between the MCI and control groups showed that the MCI group only had significantly lower mean scores in the MMSE, ACE-III total and its subdomains of memory and fluency.

Table 1

Comparison of demographic characteristics, MMSE scores, ACE-III total and subdomain scores between the dementia, MCI, and control groups (mean ± SD)

	Dementia (n = 53)	MCI(n = 38)	Control (n = 61)	p value
				dementia vs. MCI	dementia vs. control	MCI vs. control
Racea
Malay	29 (54.7%)	21 (55.3%)	38 (62.3%)
Chinese	19 (35.8%)	13 (34.2%)	19 (31.1%)
Indian	5 (9.4%)	4 (10.5%)	4 (6.6%)
Gendera
Male	25 (47.2%)	21 (55.3%)	26 (42.6%)
Female	28 (52.8%)	17 (44.7%)	35 (57.4%)
Age, yearsb	75.13±6.38	74.53±6.55	73.77±7.00
Education, yearsb	7.92±3.50	8.63±3.18	9.08±2.76
MMSEb	21.32±4.01	25.05±2.80	27.05±1.92	***	***	**
ACE-IIIb	59.62±11.64	72.18±11.08	81.64±9.35	***	***	***
Attentionb	12.17±2.96	15.05±2.01	15.87±1.91	***	***	0.286
Memoryb	11.79±3.81	15.15±4.08	19.72±3.61	***	***	***
Fluencyb	4.92±2.18	7.16±2.58	9.56±2.39	***	***	***
Languageb	18.51±4.67	21.32±4.25	22.20±3.02	**	***	0.855
Visuospatialb	12.23±2.81	13.50±2.38	14.30±1.66	*	***	0.286

ns, not significant.

p < 0.01,

p < 0.001,

p < 0.0001.

Pearson x2.

One-way analysis of variance with post hoc Bonferroni correction.

Reliability and Validity

The Cronbach's α coefficient for the Malay version of the ACE-III was 0.829, which indicates good reliability. The test-retest analysis on a subset of 15 participants (5 from each group) at a 2-week interval also yielded good reliability with an intraclass correlation coefficient of 0.959 (95% CI: 0.880–0.987). Furthermore, a strong positive correlation was observed between the Malay version of ACE-III and MMSE (Pearson's r = 0.806, two-tailed, p < 0.001) indicating that ACE-III total score will increase when MMSE score increases.

Normative Data

In order to investigate the effect of age on score performance, the healthy control participants were divided into three groups according to age (60–69, 70–79, and above 80 years old). Table 2 shows the normative mean scores generated from healthy controls for ACE-III total and its subdomain scores according to different age groups. Among these groups, no significant differences in years of education was observed [F(2, 58) = 1.84, p = 0.168]. On the other hand, the mean score of ACE-III total [F(2, 58) = 11.37, p = 0.000], attention [F(2, 58) = 7.84, p = 0.001], memory [F(2, 58) = 4.33, p = 0.018], fluency [F(2, 58) = 6.22, p = 0.004], language [F(2, 58) = 6.19, p = 0.004], and visuospatial subscores [F(2, 58) = 4.59, p = 0.014] showed significant differences among the groups. Post hoc analysis showed that the age group above 80 had a significantly lower mean score than the 60–69 age group for all variables. When comparing the 60–69 and 70–79 age groups, significant differences were observed in ACE-III total, attention and language scores. Those in the 60–69 age group had significantly higher mean scores in these three variables compared to those in the 70–79 age group. Contrarily, there were no significant differences among all the variable mean scores between the age group of 70–79 and above 80 years.

Table 2

Normative data for the ACE-III total and subdomain scores for the healthy control group according to age group (mean ± SD)

	Age group			p value
	60–69 (1)(n =21)	70–79 (2) (n = 26)	>80 (3) (n = 14)	(1) vs. (2)	(1) vs. (3)	(2) vs. (3)
Education, years	9.43±2.69	9.46±2.61	7.86±2.96
ACE-III total	87.81±5.97	80.27±9.17	74.93±8.58	**	***	0.15
Attention	16.90±1.22	15.73±2.05	14.57±1.70	0.07	**	0.14
Memory	21.43±2.68	19.15±4.10	18.21±2.99	0.08	*	1.00
Fluency	10.52±2.14	9.69±2.26	7.86±2.21	0.61	**	0.05
Language	23.86±2.15	21.65±3.12	20.71±2.97	*	**	0.94
Visuospatial	15.10±1.00	14.04±1.66	13.57±2.03	0.08	*	1.00

p < 0.01,

p < 0.001,

p < 0.0001. Pairwise comparison was performed using Bonferroni's test.

Table 3 shows that age and years of education exerted a significant association on the assessment scores. Age was inversely associated with the scores of ACE-III total and all of its subdomains and attributed for 4.0–11.2% of the variability in test scores. The score performance decreases when age increases. In contrast, score performance except the visuospatial subdomain was positively associated with the years of education. The higher the education level a participant achieves, the better the score performance observed, and it attributes to 8.1–38.7% of the score variability. There was no significant association between gender and test scores observed in this study.

Table 3

Correlation coefficients and coefficients of determination of scores by age, education, and sex

	Age (years)a		Education (years)a		Sexb
	r	r2	r	r2	r	r2
ACE-III total	−0.335**	0.112	0.536**	0.287	−0.112	0.013
Attention	−0.200*	0.040	0.438**	0.192	−0.126	0.016
Memory	−0.258**	0.067	0.282**	0.080	−0.038	0.001
Fluency	−0.328**	0.108	0.432**	0.187	−0.050	0.003
Language	−0.283**	0.080	0.622**	0.387	−0.135	0.018
Visuospatial	−0.266**	0.071	0.385**	0.148	−0.143	0.022

p < 0.05 (two-tailed),

p < 0.01 (two-tailed).

Pearson correlation.

Spearman rho correlation.

Diagnostic Interpretation

Table 4 reveals the cutoff scores and sensitivity and specificity of ACE-III and MMSE for identifying dementia and MCI through ROC analyses. The optimal cutoff score of ACE-III to discriminate healthy controls from the dementia group was 74/75, which yielded a sensitivity of 90.6% and a specificity of 82.0%. The AUC of ACE-III (0.929) was slightly larger than the MMSE (0.915) (Fig. 1). For discriminating between the healthy controls and the MCI group, the optimal cutoff score of ACE-III observed was 77/78 (sensitivity = 63.2%, specificity = 63.9%). The AUC for both the ACE-III and MMSE were 0.727 and 0.705, respectively (Fig. 2).

Table 4

Cutoff scores, sensitivity, and specificity for ACE-III and MMSE for identifying dementia and MCI

ACE-III cutoff score	Dementia		ACE-III	MCI
	sensitivity	specificity	cutoff score	sensitivity	specificity
70.5	0.849	0.885	73.0	0.474	0.836
71.5	0.849	0.852	74.5	0.500	0.820
72.5	0.849	0.836	75.5	0.526	0.754
73.5	0.906	0.836	76.5	0.579	0.721
74.5	0.906	0.820	77.5	0.632	0.639
75.5	0.943	0.754	78.5	0.632	0.607

Fig. 1

ROC curve of the ACE-III and MMSE for discriminating healthy controls from dementia patients.

Fig. 2

ROC curve of the ACE-III and MMSE for discriminating healthy controls from MCI patients.

Discussion

To our knowledge, these are the first findings to report on the reliability and validity of ACE-III in the Malay language. This study shows that the Malay version of ACE-III had a good internal consistency, test-retest reliability, and concurrent validity, suggesting that the Malay version of the ACE-III is a valid and reliable cognitive screening tool for dementia. This finding is consistent with the validation result of ACE-III in other languages [17, 20].

The AUC of 0.929 revealed that the Malay version of the ACE-III had a high diagnostic accuracy in discriminating participants with dementia from healthy controls. The optimal cutoff point for identifying dementia was 74.5 (74/75), which is lower than that of the original English version of ACE-III [16] and ACE-R [9] but a similarly low cutoff point had been reported in the Spanish [17] and Thai [19] version of ACE-III and its predecessor ACE-R in the Chinese [11], Cantonese [25] and Korean [14] languages. These differences in the cutoff point could be attributed to the effects of age and years of education, which had been widely reported in cognitive test batteries [9, 26, 27]. This was concurrent with our study analyses on the demographic characteristics association with ACE-III score performance. Our finding showed that age was inversely associated with score performance whereas years of education exerted a significantly positive correlation in score performance, which was concurrent in other studies [13, 17]. When scrutinized closely the population demographic characteristics of those studies on ACE-III and ACE-R with a cutoff point above 80 [9, 12, 13, 16, 20, 28], it was not difficult to find that these study populations had either a younger mean age (around 65–70 years old) or higher mean of education years (above 10 years) or both as compared to our study populations. The reverse demographic characteristics on age (higher mean age) and/or low education years (mean below 10) was observed in studies which yielded a cutoff point below 80 [11, 14, 17, 19, 20] in distinguishing healthy controls from dementia participants.

When comparing the AUC of the Malay version of the ACE-III and MMSE, the diagnostic accuracy of ACE-III in discriminating dementia from healthy controls was higher than that of MMSE for which similar findings had been reported for other languages [17, 20]. Moreover, ACE-III encompasses a broader range of cognitive abilities assessment, including executive function, memory, language, and visuospatial components, than MMSE, which renders it a more useful and precise assessment tool for the detection of dementia and its subtypes like FTD. The high sensitivity and specificity obtained in this and other studies [16, 17, 20] further reinforced its usefulness.

Similar to the Korean version of ACE-R [14], this study did not manage to show sufficient diagnostic accuracies of ACE-III in discriminating MCI from healthy control as compared to other validation studies of ACE-R [11, 13]. Again, this discrepancy could be explained by the effect of age and education on different studies. The mean age of both the Japanese (control: 66.3 ± 10.0 and MCI: 71.4 ± 9.2) and Chinese studies (control: 68.16 ± 8.18 and MCI: 69.52 ± 9.01) was younger than the current study population. Moreover, in both the Japanese and Chinese studies, the population had higher mean years of education (> 10 years) as compared to this study. When compared with the Korean study which had closer demographic characteristics (mean years of education < 10 years and mean age > 70 years), the AUC (0.697), sensitivity (67%), and specificity (65%) of ACE-R were much lower and closer to our study finding. This once again reiterates the effects of age and years of education on the performance of ACE-III. Nonetheless, the AUC value of ACE-III was still larger than that of the MMSE indicating relatively better diagnostic accuracy.

There are a few limitations in this study. First, this study only recruited participants from two urban teaching hospitals. Thus, the result is subject to referral bias and may not be applicable for community-based populations. Second, this study did not classify the subtypes of dementia (e.g., AD, vascular dementia, and FTD), which ACE has been shown to have the capacity to discriminate [12, 16, 29]. Third, although normative data for different age groups had been performed in this study, the sample size is relatively small. As suggested by Matías-Guiu et al. [30], future studies with larger populations of healthy controls should be conducted in order to examine the detailed correlation of the Malay version of ACE-III and obtain reference values adjusted by age and years of education. Finally, this study only compared the Malay version of ACE-III with the MMSE; other cognitive screening tools such as the Montreal Cognitive Assessment [31] and the Rowland Universal Dementia Assessment Scale [32] should also be taken into consideration in the future.

In summary, this study demonstrates that the Malay version of the ACE-III is a valid and reliable screening tool for dementia. Future research on a large-scale community-based setting are needed to provide normative data, and the discriminatory ability of the Malay version of ACE-III across the different subtypes of dementia and validation of the Malay version of ACE-III with other neuropsychological tests shall be conducted.

Statement of Ethics

The study was approved by the Research Ethics Committee UKM (RECUKM) and University of Malaya Medical Research Ethics Committee (MREC). All participants provided written informed consent before the assessments.

Disclosure Statement

The authors have no conflicts of interest to disclose. The sponsors did not play any role in the research process of this study.

Funding Sources

Kwai Ching Kan's doctoral studies are sponsored by “MyBrain 15 Program” under Ministry of Higher Education Malaysia. This study was jointly funded by Young Researcher Encouragement Grant (GGPM-2013-046), University Kebangsaan Malaysia and Long-Term Research Grant Scheme (LRGS) provided by the Ministry of Education Malaysia (LRGS/BU/2012/UKM-UKM/K/01).