Mini-Addenbrooke's Cognitive Examination (MACE): a Useful Cognitive Screening Instrument in Older People?

BACKGROUND: The Mini-Addenbrooke's Cognitive Examination (MACE) is a recently described brief cognitive screening instrument.
OBJECTIVE: To examine the test accuracy of MACE for the identification of dementia and mild cognitive impairment (MCI) in a cohort of older patients assessed in a neurology-led dedicated cognitive disorders clinic.
METHODS: Cross-sectional assessment of consecutive patients with MACE was performed independent of the reference standard diagnosis based on clinical interview of patient and, where possible, informant and structural brain imaging, and applying standard clinical diagnostic criteria for dementia and MCI. Various test accuracy metrics were examined at two MACE cut-offs ( ≤ 25/30 and ≤ 21/30), comparing the whole patient cohort with those aged ≥ 65 or ≥ 75 years, hence at different disease prevalences.
RESULTS: Dependent upon the chosen cut-off, MACE was either very sensitive or very specific for the identification of any cognitive impairment in the older patient cohorts with increased disease prevalence. However, at both cut-offs the positive predictive values and post-test odds increased in the older patient cohorts. At the more sensitive cut-off, improvements in some new unitary test metrics were also seen.
CONCLUSION: MACE is a valid instrument for identification of cognitive impairment in older people. Test accuracy metrics may differ with disease prevalence.

INTRODUCTION

Age is the most important risk factor for the development of cognitive decline and dementia. Various guidelines and recommendations addressing the use of cognitive screening instruments (CSIs) in older adults have been published. The Canadian Task Force on Preventive Health Care strongly recommended against screening asymptomatic older adults (≥ 65 years), but indicated that consideration should be given to cognitive assessment if patients had signs and symptoms of impairment or if family members of patients expressed concerns about possible cognitive decline.( As to which CSI(s) might be used for this purpose, the US Preventive Services Task Force reported that several brief instruments can adequately detect dementia, but found no empirical evidence that such screening improved outcomes.( The Alzheimer Association mentioned 15 different tools which might be used, but recognized that no optimal CSI was suitable for all patient populations and settings.( In the United Kingdom, the Alzheimer’s Society produced a “practical toolkit” to assess cognition in older people, recommending different CSIs in different settings: for memory clinics, the specific recommendations were the Montreal Cognitive Assessment (MoCA) and the third iteration of the Addenbrooke’s Cognitive Examination, ACE-III.(

The Mini-Addenbrooke’s Cognitive Examination (MACE) is a relatively recently described screening instrument, derived from ACE-III by Mokken scaling.( MACE comprises tests of attention, memory (7-item name and address), verbal fluency, clock drawing, and memory recall (score range 0–30). The index study identified two cut-offs, one with high sensitivity (≤ 25/30) and one with high specificity (≤ 21/30).( Independent studies of MACE have reported its utility for identification of cognitive impairment in various clinical settings.(

Pragmatic screening test accuracy studies (DTAS) are required to inform the choice of suitable CSIs.( The aim of this study was to examine the screening utility of MACE for dementia and mild cognitive impairment (MCI) versus subjective memory complaint in older people (≥ 65 and ≥ 75 years) (i.e., in samples enriched for those at greatest risk of cognitive impairment and dementia). This affords an opportunity to observe how the examined metrics vary (i.e., how the test performs) with changing disease prevalence, for although within a given population sensitivity and specificity are properties of the test, their values change in different populations.

Some preliminary (two-year) data from this study have appeared as part of a broader study examining the utility of various CSIs in older patients (aged ≥ 65 years) seen in a dedicated cognitive disorders clinic.(

METHODS

Subjects

The dataset from a pragmatic prospective screening test accuracy study examining MACE in consecutive new patients referred over the period June 2014 to December 2018 to a neurology-led dedicated cognitive function clinic based in a regional neuroscience centre( was re-interrogated. The clinic operates no age-related exclusion criteria. Data from patients aged ≥ 65 and ≥ 75 years, as well as the whole cohort, were examined. Subjects gave informed consent and the study protocol was approved by the institute’s committee on human research.

Procedure

Cross-sectional assessment of all patients comprised semi-structured patient history enquiring about cognitive symptoms and functional performance, with collateral history where possible; neuroradiological examination (brain CT in all patients; interval MR imaging in some cases); and formal neuropsychological assessment in some cases. Administration of MACE occurred on the same day as, but separate from, the cross-sectional assessment. Standard diagnostic criteria for dementia (DSM-IV)( and MCI (Petersen et al.)( were used; absence of dementia or MCI was categorized as subjective memory complaint (SMC). Criterion diagnosis (reference standard) was by judgment of an experienced clinician based on diagnostic criteria, but blinded to MACE scores in order to avoid review bias. STARDdem guidelines for reporting diagnostic test accuracy studies in dementia were observed.(

Analyses

Using the two MACE cut-offs (≤ 25/30, more sensitive, and ≤ 21/30, more specific) from the index study,( in order to avoid any possible introduction of bias,( standard summary measures of discrimination( were calculated in each of the three cohorts (whole, ≥ 65 years, and ≥ 75 years) for any cognitive impairment (dementia plus MCI) versus SMC. These measures were sensitivity, specificity, positive and negative predictive values (PPV and NPV), correct classification accuracy (CCA), positive and negative likelihood ratios (LR+, LR-) classified as per Jaeschke et al.,( and positive and negative clinical utility indexes (CUI+, CUI-) classified as per Mitchell.(

In addition to these standard summary measures, “number needed” metrics were also calculated: to diagnose (NND) = 1/Y, where Y = Youden index (Y = sensitivity + specificity − 1);( to predict (NNP) = 1/PSI, where PSI = predictive summary index (PSI = PPV + NPV − 1);( and to misdiagnose (NNM) = 1/(1 - CCA).( New unitary metrics for DTAS,( as used in the main study,( were also calculated, namely the likelihood to diagnose or misdiagnose (LDM = NNM/NND or NNM/NNP), the summary utility index (SUI = CUI+ + CUI-), and the number needed for screening utility (NNSU = 1/SUI), and classified as previously.(

RESULTS

Over one-third of the patients were aged ≥ 65 years (n = 287; 38%), whereas less than one-fifth were aged ≥ 75 years (n = 119; 16%). As anticipated, the prevalence of any cognitive impairment (dementia plus MCI) was higher in the older age groups compared to the whole cohort (Table 1).

TABLE 1

Demographics

	Dementia plus MCI vs. no cognitive impairment (SMC)
Whole Cohort
N (dementia plus MCI vs. SMC)	755 (336 vs. 419)
F:M (%F)	352:403 (46.6%)
Prevalence (P = pre-test probability)	Dementia plus MCI 0.445
Pre-test odds (= P/1 − P)	Dementia plus MCI 0.802
Cohort Aged ≥ 65 Years
N (dementia plus MCI vs. SMC)	287 (215 vs. 72)
F:M (%F)	135:152 (47.0%)
Prevalence (P = pre-test probability)	Dementia plus MCI 0.749
Pre-test odds (= P/1 − P)	Dementia plus MCI 2.986
Cohort Aged ≥ 75 Years
N (dementia plus MCI vs. SMC)	119 (111 vs. 8)
F:M (%F)	69:50 (58.0%)
Prevalence (P = pre-test probability)	Dementia plus MCI 0.933
Pre-test odds (= P/1 − P)	Dementia plus MCI 13.875

Standard summary measures of discrimination for the identification of dementia plus MCI versus SMC at the MACE ≤ 25/30 cut-off (Table 2) showed the test was very sensitive (>0.95) in all three patient cohorts (whole vs. ≥ 65 years vs. ≥ 75 years cohorts) with very large negative likelihood ratios. The positive predictive value (0.589 vs. 0.859 vs. 0.957), correct classification accuracy (0.685 vs. 0.854 vs. 0.950), post-test odds (1.432 vs. 6.088 vs. 22.0), and positive clinical utility index (0.570 vs. 0.827 vs. 0.948) all increased with increasing prevalence of cognitive impairment, whilst negative predictive value and negative clinical utility index both decreased.

TABLE 2

Diagnosis of dementia plus MCI vs. no cognitive impairment (SMC): comparison of standard summary measures of discrimination (with 95% CI) using MACE cut-off ≤ 25/30 in whole cohort vs. cohorts of older patients (aged ≥ 65 and ≥ 75 yrs)

	Whole Cohort	Older Patients Aged ≥ 65 Yrs	Older Patients Aged ≥ 75 Yrs
N	755	287	119
Sensitivity (Sens)	0.967 (0.948–0.986)	0.963 (0.937–0.988)	0.991 (0.973–1.00)
Specificity (Spec)	0.458 (0.411–0.506)	0.528 (0.412–0.643)	0.375 (0.040–0.710)
Positive Predictive Value (PPV = post-test probability)	0.589 (0. 548–0.630)	0.859 (0. 815–0.903)	0.957 (0.919–0.994)
Negative Predictive Value (NPV)	0.946 (0.915–0.977)	0.826 (0.717–0.936)	0.750 (0.326–1.00)
Correct classification accuracy (Acc)	0.685 (0.652–0.718)	0.854 (0.813–0.895)	0.950 (0.910–0.989)
Positive Likelihood Ratio (LR+)	1.785 (1.63–1.95) (slight)	2.039 (1.60–2.61) (slight)	1.586 (0.93–2.71) (slight)
Negative Likelihood Ratio (LR−)	0.071 (0.065–0.078) (very large)	0.071 (0.055–0.090) (very large)	0.024 (0.014–0.041) (very large)
Post-test odds (= pre-test odds × LR+)	1.432	6.088	22.00
Positive Clinical Utility Index (CUI+ = Sens × PPV)	0.570 (adequate)	0.827 (excellent)	0.948 (excellent)
Negative Clinical Utility Index (CUI− = Spec × NPV)	0.433 (poor)	0.436 (very poor)	0.281 (very poor)

Examining the “number needed” and unitary metrics (Table 3), there was an increase in the number needed to misdiagnose with increasing prevalence of cognitive impairment (3.17 vs. 6.85 vs. 20.0), and this was reflected in the increasing values of the likelihood to be diagnosed or misdiagnosed (LDM) measure. SUI and NNSU remained relatively constant, and adequate, throughout.

TABLE 3

Diagnosis of dementia plus MCI vs. no cognitive impairment (SMC): comparison of “number needed” and unitary measures of discrimination using MACE cut-off ≤ 25/30 in whole cohort vs. cohorts of older patients (aged ≥ 65 and ≥ 75 yrs)

	Whole Cohort	Older Patients Aged ≥ 65 Years	Older Patients Aged ≥ 75 Years
N	755	287	119
Youden index (= Sens + Spec − 1)	0.425	0.491	0.366
Number needed to diagnose (NND = 1/Y)	2.35	2.04	2.73
Predictive Summary Index (= PPV + NPV − 1)	0.535	0.685	0.707
Number needed to predict (NNP = 1/PSI)	1.87	1.46	1.41
Number needed to misdiagnose (NNM = 1/(1 − Acc))	3.17	6.85	20.0
Likelihood to be diagnosed or misdiagnosed (LDM = NNM/NND, NNM/NNP)	1.35, 1.70	3.36, 4.69	7.33, 14.2
Summary Utility Index (SUI = CUI+ + CUI−)	1.003 (adequate)	1.263 (adequate)	1.229 (adequate)
Number needed for screening utility (NNSU = 1/SUI)	0.997 (adequate)	0.792 (adequate)	0.814 (adequate)

Standard summary measures of discrimination for the identification of dementia plus MCI versus SMC at the MACE ≤ 21/30 cut-off (Table 4) showed the test was more specific than sensitive in all three patient cohorts (whole vs. ≥ 65 years vs. ≥ 75 years cohorts) with increasing positive likelihood ratios. As observed with the ≤ 25/30 cut-off, once again the positive predictive value (0.737 vs. 0.975 vs. 0.989), correct classification accuracy (0.767 vs. 0.780 vs. 0.807), post-test odds (2.798 vs. 39.0 vs. 89.0), and positive clinical utility index (0.546 vs. 0.708 vs. 0.793) all increased with increasing prevalence of cognitive impairment, whilst negative predictive value and negative clinical utility index both decreased.

TABLE 4

Diagnosis of dementia plus MCI vs. no cognitive impairment (SMC): comparison of standard summary measures of discrimination (with 95% CI) using MACE cut-off ≤21/30 in whole cohort vs. cohorts of older patients (aged ≥65 and ≥75 yrs)

	Whole Cohort	Older Patients Aged ≥ 65 Yrs	Older Patients Aged ≥ 75 Yrs
N	755	287	119
Sensitivity (Sens)	0.741 (0.694–0.788)	0.726 (0.666–0.785)	0.802 (0.728–0.876)
Specificity (Spec)	0.788 (0.748–0.827)	0.944 (0.892–0.997)	0.875 (0.646–1.00)
Positive Predictive Value (PPV = post-test probability)	0.737 (0. 690–0.784)	0.975 (0.951–0.999)	0.989 (0.967–1.00)
Negative Predictive Value (NPV)	0.791 (0.752–0.830)	0.535 (0.449–0.622)	0.241 (0.086–0.397)
Correct classification accuracy (Acc)	0.767 (0.737–0.797)	0.780 (0.733–0.828)	0.807 (0.736–0.878)
Positive Likelihood Ratio (LR+)	3.489 (2.87–4.24) (moderate)	13.06 (5.03–33.9) (very large)	6.414 (1.02–40.2) (large)
Negative Likelihood Ratio (LR−)	0.329 (0.27–0.40) (moderate)	0.291 (0.11–0.75) (moderate)	0.227 (0.04–1.42) (moderate)
Post-test odds (= pre-test odds × LR+)	2.798	39.0	89.0
Positive Clinical Utility Index (CUI+ = Sens × PPV)	0.546 (adequate)	0.708 (good)	0.793 (good)
Negative Clinical Utility Index (CUI− = Spec × NPV)	0.623 (adequate)	0.505 (adequate)	0.211 (very poor)

Examining the “number needed” and unitary metrics (Table 5), there was discrepancy in NND (decreased) and NNP (increased) with increasing prevalence of cognitive impairment and, hence, no consistent pattern in values of the likelihood to be diagnosed or misdiagnosed (LDM) measure. SUI and NNSU remained relatively constant, and adequate, throughout.

TABLE 5

Diagnosis of dementia plus MCI vs. no cognitive impairment (SMC): comparison of “number needed” and unitary measures of discrimination using MACE cut-off ≤21/30 in whole cohort vs. cohorts of older patients (aged ≥65 and ≥75 yrs)

	Whole Cohort	Older Patients Aged ≥ 65 Yrs	Older Patients Aged ≥ 75 Yrs
N	755	287	119
Youden index (= Sens + Spec − 1)	0.425	0.670	0.677
Number needed to diagnose (NND = 1/Y)	2.35	1.49	1.48
Predictive Summary Index (= PPV + NPV − 1)	0.535	0.510	0.230
Number needed to predict (NNP = 1/PSI)	1.87	1.96	4.35
Number needed to misdiagnose (NNM = 1/(1 − Acc))	4.29	4.55	5.18
Likelihood to be diagnosed or misdiagnosed (LDM = NNM/NND, NNM/NNP)	1.83, 2.30	3.05, 2.32	3.50, 1.19
Summary Utility Index (SUI = CUI+ + CUI−)	1.169 (adequate)	1.213 (adequate)	1.004 (adequate)
Number needed for screening utility (NNSU = 1/SUI)	0.855 (adequate)	0.824 (adequate)	0.996 (adequate)

DISCUSSION

MACE proved acceptable to patients in the clinic setting, and was quick and easy to administer and score, as previously reported.( The data indicate that, at both MACE cut-offs examined, test PPV, CCA, post-test odds, and CUI+ improved in the older patient cohorts with increased prevalence of cognitive impairment, with some decline in NPV and CUI-. At the more sensitive MACE cut-off there was increase in the values of LDM with increasing prevalence of cognitive impairment, and the values of SUI and NNSU remained adequate at both cut-offs.

Potential limitations of the study include those related to all clinic-based studies, such as the inevitable selection bias. Most of the patients seen in this clinic are referred direct from primary care settings, have not been previously seen by any specialist (general physician, geriatrician, or neurologist), and have not been administered any cognitive screening instrument prior to referral.( Older patients seen in a neurology-led clinic may differ from those seen in geriatric or old age psychiatry memory clinics, for example in terms of comorbidities. Clearly further studies of MACE in these settings are required, and no definite comment can be made about test utility for the oldest old patients. The chosen age cut-offs (≥ 65 and ≥ 75 years) were arbitrary. One consequence of this choice was the relatively small number of SMC patients in the ≥ 75 years cohort (n = 8), a limitation reflected in the wide confidence intervals of some of the standard test metrics. Another possible limitation was examining only two MACE cut-offs, as per the index paper,( in order to avoid any possible introduction of bias,( although a previous analysis looked systematically at a range of cut-offs.(

Notwithstanding these limitations, the data presented suggest MACE may be a valid instrument, meaning that it does identify what it claims to identify, namely individuals with any cognitive impairment (dementia plus MCI), and it may do this effectively in cohorts of older people in whom the prevalence of cognitive impairment is higher. Whereas the content-related (construct) validity of MACE was established in the index study,( the current study examines criterion-related (concurrent) validity using standard diagnostic criteria (DSM-IV for dementia, Petersen for MCI).

The findings confirmed those of the preliminary report,( and greatly extended these, by examining two MACE cut-offs, two age cut-offs, and more summary measures in a larger patient cohort. The results suggest that in clinical practice either MACE cut-off may be chosen, dependent upon exact clinician requirements, since both increase PPV. The ≤ 25/30 cut-off has greater sensitivity, inevitably with more false positives, and ≤ 21/30 has greater specificity, inevitably with more false negatives. Avoidance of the latter (missed cases) is usually priority for clinicians.

The combination of this patient performance measurement with an informant interview, as per the recommendations of the Alzheimer Association( and the International Association of Gerontology and Geriatrics,( may be worth further investigation.