Concordance of the Montreal cognitive assessment with standard neuropsychological measures.

INTRODUCTION: The concordance of the Montreal cognitive assessment (MoCA) with more comprehensive neuropsychological measures remains unclear. This study examined the individual MoCA domains with more comprehensive and commonly used neuropsychological measures to determine the degree of overlap.
METHODS: Data included individuals seen in an outpatient neurology clinic specializing in neurodegenerative disease who were administered the MoCA and also underwent neuropsychological assessment (n = 471). A principal component analysis with varimax rotation was completed using the MoCA domain scores and comprehensive neuropsychological evaluation measures.
RESULTS: Four factors emerged accounting for 55.6% of the variance: (1) visuospatial/executive functioning; (2) memory; (3) attention; and (4) language. The individual MoCA domain scores demonstrated high factor loadings with standard neuropsychological measures purported to measure similar cognitive constructs. DISCUSSION: These findings provide empirical validation for the MoCA domain classifications, lending further support for the use of the MoCA as a cognitive screen that reflects similar constructs as those measured by a comprehensive battery.

Introduction

The Montreal cognitive assessment (MoCA) [1] is a brief cognitive screening measure commonly used for both clinical and research purposes. It is often completed by primary care doctors and neurologists to screen for cognitive decline, including Alzheimer's disease. Given the large number of individuals with Alzheimer's disease, which is expanding exponentially, it is important to understand the relationship between this brief screening measure and the current gold standard of cognitive assessment: a full neuropsychological evaluation. Current validated use of the MoCA is restricted to interpretation of the total score, using a cutoff of less than 26 to signify impairment, which has demonstrated adequate sensitivity to cognitive impairment in a number of clinical populations, including mild cognitive impairment [1], [2], [3], [4], Alzheimer's disease [1], [2], stroke [5], [6], Parkinson's disease [7], [8], and Huntington's disease [9], [10].

The extent to which performance on the MoCA relates to general cognitive functioning as assessed by more detailed neuropsychological tests has been explored, providing evidence of convergent validity for the overall total score [11], [12]. Prior research has also compared the sensitivity and specificity of the MoCA to detecting cognitive impairment, as defined by impaired performance on standard neuropsychological measures, and again found adequate concurrent validity for the overall score [1], [3], [5], [6], [7], [13], [14]. Comparatively few studies, however, have examined the construct validity of the individual domain scores. The primary aim of the present study was to explore the construct validity of the individual MoCA domain scores in an effort to determine the extent to which those scores reflect similar cognitive constructs as those measured by more traditional and comprehensive neuropsychological measures.

Similar research conducted by Moafmashhadi and Koski (2013) examined the factor structure of commonly used neuropsychological measures and correlated the calculated factor scores with the individual items of the MoCA in a sample of geriatric clinical outpatients and found significant, albeit modest, correlations between their calculated factor scores and the MoCA items, suggesting construct overlap. In their examination of the sensitivity and specificity of the individual MoCA domain scores at predicting impaired cognitive performance on similar neuropsychological measures, the visuospatial/executive score demonstrated the best predictive accuracy; however, the MoCA domain scores were generally poor predictors of impairment on standard neuropsychological measures and the authors caution against clinical interpretation of domain scores [13]. Lam et al. (2013) found significant correlations among MoCA domain scores and respective neuropsychological domain scores in patients with mild Alzheimer's disease or mild cognitive impairment. They found significant cross-correlations among different domains with the highest correlation between memory domains and the lowest between language domains. For both the neuropsychological measures and the MoCA items, they grouped subtests/items based on the construct purportedly being measured, rather than using a statistical method of combining items based on shared variance.

The following study addresses a gap in our current understanding: how the MoCA domain scores relate to more comprehensive neuropsychological testing, without using an a priori categorization of the neuropsychological measures. Here, we examine the construct validity of the MoCA domain scores by using a factor analytic approach to objectively explore the construct validity. Our goal was to determine the extent to which the individual domain scores load onto similar factors with comparable indices of cognitive functioning taken from standard neuropsychological measures.

Methods

Participants

Data were drawn from a sample of individuals seen in a subspecialty outpatient memory disorders clinic, specializing in diagnosis and treatment of neurodegenerative disease (e.g., mild cognitive impairment, Alzheimer's disease, dementia with Lewy-bodies, frontotemporal dementia, Parkinson's disease). All patients were administered the MoCA during their intake appointment with neurology and were subsequently referred for neuropsychological assessment as part of routine clinical care. The analyzed sample consisted of 471 complete cases and was 49.9% women and predominantly Caucasian (91.3%) with an average age of 68.0 years (standard deviation [SD] = 9.3; range 25–92 years), average education of 14.7 years (SD = 2.8; range 7–20 years), and average MoCA score of 22.3 (SD = 4.0; range = 8–30). Neuropsychological testing was completed within 180 days of MoCA screening for all patients with an average interval of 47.8 days (SD = 45.8). This study was reviewed and approved by the Institutional Review Board at the Cleveland Clinic (14-565), and all patients gave written informed consent for the use of their data for research purposes.

Measures

Montreal cognitive assessment

The MoCA is a manually administered paper-and-pencil cognitive screening that takes approximately 10 minutes to administer and with appropriate training can be administered by multiple levels of health care providers (e.g., medical assistants, nursing staff, physician assistants, psychometrists, and so forth). It consists of 12 individual tasks, most of which are binary, that are scored and summed with a 6-item orientation screening and an educational correction (i.e., one point added for individuals with 12 years of education or less) to generate a total score representing global cognitive functioning. The individual MoCA items have been grouped into cognitive domains, including (1) visuospatial and executive functioning, (2) naming, (3) attention (e.g., simple attention, working memory, vigilance), (4) language (e.g., repetition, phonemic fluency), (5) abstraction, (6) delayed memory recall, and (7) orientation. Multiple domain classifications have been suggested [1], [2], although none are currently validated for clinical interpretation. The current analysis uses the original domain organization established by the test authors.

Neuropsychological battery

All patients were evaluated using a comprehensive neuropsychological battery as part of routine clinical care. Measures included the brief visuospatial memory test, revised (BVMT-R) [15] delayed recall score and copy score, Hopkins verbal learning test, revised (HVLT-R) [16] delayed recall score, Wechsler memory scale, fourth edition (WMS-IV) [17] logical memory II, all five trails of the Delis-Kaplan executive function system (DKEFS) [18] trail making test (scanning, number sequencing, letter sequencing, switching, and speed), Boston naming test (BNT) [19], phonemic (FAS) and category (animal) fluencies, and the block design, similarities and digit span subtests of the Wechsler adult intelligence scale, fourth edition (WAIS-IV) [20]. Digit span scores were further subdivided into forward, backward, and sequencing sections. Table 1 provides an overview of the measures and descriptions of the scores included in the analyses. Table 2 provides the mean and SDs for each variable.

Table 1

Neuropsychological and MoCA variables used in factor analysis

Measure	Scores used
BVMT copy	Copy of simple figures
WAIS-IV block design	Raw score
DKEFS trail making test	Time to complete each of the five trails
WMS-IV logical memory II	Sum of delayed recall of two stories
HVLT-R	Sum of delayed recall of a word list
BVMT-R	Sum of a delayed recall of simple figures
WAIS-IV digit span	Raw score of forward, backward, and sequencing
Phonemic fluency	Sum of words produced beginning with letters F, A, and S
Category fluency	Sum of words produced in the semantic category animals
WAIS-IV similarities	Raw score
Boston naming test	Number of pictures correctly named
MoCA visuospatial/executive	Sum of trails switching, cube copy, and clock drawing
MoCA delayed memory	Sum of uncued delayed recall of five words
MoCA orientation	Sum of orientation for date and location
MoCA attention	Sum of digit span forward, backward, vigilance, and serial 7
MoCA naming	Sum of naming three pictures
MoCA language	Sum of repetition ability and phonemic fluency (F)
MoCA abstraction	Sum of similarities between words

Abbreviations: MoCA, Montreal cognitive assessment; BVMT, brief visuospatial memory test; WAIS-IV, Wechsler adult intelligence scale, fourth edition; DKEFS, Delis-Kaplan executive function system; WMS-IV, Wechsler memory scale, fourth edition; HVLT, Hopkins verbal learning test.

Table 2

Neuropsychological and MoCA variable scores

Variable	Mean	SD
BVMT copy	11.01	1.24
Block design	29.23	9.69
DKEFS scanning	28.72	10.87
Numbers	53.07	25.48
Letters	57.33	28.90
Switching	140.50	61.67
Speed	38.21	21.40
Logical memory II	11.63	8.24
HVLT delayed	4.69	3.82
BVMT delayed	4.83	3.25
Digit span forward	9.43	2.20
Backward	7.44	2.17
Sequencing	6.67	2.52
Phonemic fluency (FAS)	31.51	11.51
Category fluency (animals)	15.30	5.08
Similarities	22.95	5.63
Boston naming test	50.94	8.62
MoCA visuospatial/executive	3.74	1.18
MoCA delayed memory	1.76	1.67
MoCA orientation	5.31	1.05
MoCA attention	5.03	1.18
MoCA naming	2.77	0.50
MoCA language	1.99	0.92
MoCA abstraction	1.39	0.78

Abbreviations: MoCA, Montreal cognitive assessment; SD, standard deviation; BVMT, brief visuospatial memory test; DKEFS, Delis-Kaplan executive function system; HVLT, Hopkins verbal learning test.

Data analysis

A principal component analysis (PCA) with varimax rotation was completed using the 6 MoCA domain scores, the orientation score, and analogous cognitive measures completed on the same individuals as part of a comprehensive neuropsychological evaluation. An overall test battery mean (OTBM) [21] was also calculated for the neuropsychological battery by converting all age-adjusted standardized scores to a common metric (i.e., T scores) and calculating the resulting mean in an effort to create a value analogous to the MoCA total score. The average OTBM for the sample was 45.7 (SD = 6.9; range = 24.5–62.5). Although not included in the PCA, the relationship between the MoCA total and OTBM was evaluated using the Pearson product-moment correlation coefficient.

Results

The OTBM significantly correlated with the MoCA total score, r = 0.66, P < .001, indicating a significant relationship among overall MoCA performance and overall performance on a larger, more comprehensive neuropsychological battery.

Four factors emerged in the PCA. All four components had eigenvalues >1 and in total they accounted for 55.6% of the variance (Table 3). The number of components was chosen based on eigenvalues <1, the scree test, and minimal partial average test, which all converged on extraction of four components. Loading on the first component was the MoCA visuospatial/executive score, the BVMT copy score, block design, and all five trails of the trail making test. This component was labeled the visuospatial/executive factor and accounted for the greatest amount of variance, with component loadings ranging from 0.45 (BVMT copy) to −0.79 (DKEFS number sequencing). Given that the scores on the DKEFS trail making test reflect time to completion, such that lower times reflect better performance, the negative component loadings reflect the expected relationship.

Table 3

Factor loadings

Variable	Factor
1	2	3	4
MoCA visuospatial/executive	0.533
BVMT copy	0.451
Block design	0.698
DKEFS scanning	−0.678
Numbers	−0.792
Letters	−0.732
Switching	−0.650		−0.383
Speed	−0.740
MoCA delayed		0.781
MoCA orientation		0.576
Logical memory II		0.821
HVLT delayed		0.790
BVMT delayed		0.768
MoCA attention			0.583
MoCA language			0.587	0.364
Digit span forward			0.738
Backward			0.747
Sequencing	0.362		0.511
Phonemic fluency (FAS)			0.604
MoCA naming				0.727
MoCA abstraction				0.434
Similarities			0.352	0.632
Category fluency (animals)				0.550
Boston naming test				0.785
Eigenvalue	7.77	2.36	1.82	1.39
Proportion of variance (rotated), %	17.56	14.19	12.75	11.15

Abbreviations: MoCA, Montreal cognitive assessment; BVMT, brief visuospatial memory test; DKEFS, Delis-Kaplan executive function system; HVLT, Hopkins verbal learning test.

NOTE. Loadings <0.350 were suppressed from the Table.

The second component, memory, consisted of the MoCA delayed recall and orientation scores, as well as the delayed recall scores from logical memory, HVLT, and BVMT and represented the clearest component loading patterns. The orientation score generated the lowest loading (0.58), which is not surprising as this measure is the most discrepant from the other measures within this component. The remaining variables loading on the memory component exceeded 0.75, with the highest noted for the HVLT delayed recall score, which most closely parallels the memory component of the MoCA (i.e., word list recall).

The third component, labeled attention, was the most variable and consisted of the MoCA attention and language scores, as well as the digit span scores (i.e., forward, backward, and sequencing), and phonemic fluency. The switching trial of DKEFS trails and similarities subtest of the WAIS-IV also loaded on this component, although to a lesser degree, and both of these variables showed stronger loadings with other factors. Although not all tasks within this component were clearly related to attention, the highest loadings were noted for the attentional measures. The loadings of the DKEFS switching and phonemic fluency may reflect the attentional components of these measures but the relationship between similarities and the remaining measures is less clear. Had the similarities subtest showed the highest loading on this component, labeling it as measuring attention would have been more difficult; however, this was not the case and the loading was modest by comparison.

The final component, language, consisted of the MoCA naming, language, and abstraction scores, as well as DKEFS category fluency, BNT, and similarities. Given that the preponderance of these indices relate to language functions, the chosen label was felt most appropriate. Interestingly, the MoCA language domain did not show the highest loading with this factor, which is discussed in the following.

Discussion

The individual MoCA domain scores demonstrated high factor loadings with standard neuropsychological measures purported to measure similar cognitive constructs, providing empirical validation for the construct validity of the MoCA domain classifications. These findings lend further support for the use of the MoCA as a brief screen of cognition that reflects similar constructs as those gleaned from a more comprehensive battery. Most notably, the memory domain from the MoCA demonstrated a strong association with standard neuropsychological measures of memory and of the individual MoCA domain scores demonstrated the strongest loading. This is of particular relevance to practitioners working with neurodegenerative disease populations, as individuals demonstrating poor memory performance on the MoCA are likely to also demonstrate memory impairment on more comprehensive neuropsychological testing, validating the utility of the MoCA as a memory screening tool. What remains unclear is whether similar impairment profiles emerge on the MoCA as those on more standard neuropsychological measures (e.g., retrieval vs. encoding deficits), which could further aid in refining differential diagnoses; this reflects an empirical question of interest and target of future study.

A significant advantage of the MoCA over other cognitive screening measures is the breadth of cognitive domain coverage beyond memory, which increases its clinical utility, particularly for detection of individuals presenting with nonamnestic cognitive changes. As shown by the present findings, performance on the MoCA is also sensitive to visuospatial ability and executive functioning, attention, as well as language and that the standard domain organization developed by the authors of the MoCA parallels that which is measured by more comprehensive assessment. One limitation of the present study is the long wait times between the administration of the MoCA and the full neuropsychological battery (e.g., up to 180 days). The time between MoCA and full neuropsychology testing is an organic factor of wait times for a neuropsychological evaluation at the time of data collection. Although there may be some progression of symptoms in the interim, we would expect the progression to manifest similarly within domains, so would likely not have an impact on our results.

Although most measures were clearly associated with a single construct, several measures demonstrated multiple associations. The most evenly distributed was the digit span sequencing score, a measure of verbal working memory that loaded on both the visuospatial/executive component (0.36) and the attention component (0.51), which is consistent with previous literature suggesting that working memory, particularly mental manipulation, requires both attention, as well as executive demands [22], [23]. The MoCA language score was strongly associated with the attention component, as well as the language component, albeit to a lesser degree. Reviewing the items comprising the MoCA language score finds that 2 of 3 possible points are derived from a repetition task, which requires basic attention to complete and may account for the higher loading on attention. Similarly, the WAIS-IV similarities subtest, which requires verbal abstract reasoning and concept formation, showed a split-loading, associating strongly with the language component but also with the attention component. The switching trial of DKEFS trails was also associated with two components, including the visuospatial/executive component and the attention component, which may be attributable to the executive demands required to switch effectively, as well as the working memory demands required to maintain set.

Although the present study supports the construct validity of the MoCA when compared with more comprehensive neuropsychological measures, it does not evaluate the diagnostic sensitivity and specificity of the domain scores. In previous research, when the diagnostic sensitivity and specificity of the MoCA was compared with a full neuropsychological evaluation, the total MoCA score has been found to have high sensitivity but low specificity [5], [7], which is to be expected given the broad sampling of cognitive constructs. Although poor specificity precludes use of the MoCA as a diagnostic tool, the high sensitivity reinforces the notion that the MoCA is a valuable screening tool that can be used to guide clinical decision making to help determine when more comprehensive neuropsychological testing would be beneficial.

As suggested in previous research, a score below the cutpoint warrants a full neuropsychological evaluation to better characterize the extent of cognitive impairment and more thoroughly evaluate an individual's cognitive pattern of performance. Because of the high correspondence between MoCA domain scores and standard neuropsychological measures, our results also suggest the MoCA can provide a qualitative understanding of an individual's performance on subdomains, even though the deficit may be highly specific. Furthermore, although our findings are suggestive of the utility of the MoCA to assess the overall cognitive profile of patients and future research may find that the cognitive profiles produced with the MoCA are diagnostically useful, clinical use of scores other than the total score has not been validated. Until such time, the MoCA remains an excellent tool for cognitive screenings, which have their place in many contexts. Comprehensive neuropsychological assessment, however, when and where available remains the preferred method for generating a more refined profile regarding cognitive functioning.

Systematic review: The authors used traditional sources (e.g., PubMed) to review the literature. It was found that the Montreal cognitive assessment (MoCA) has been evaluated in comparison with standard neuropsychological batteries using a priori cognitive domain classifications. These studies are discussed and appropriately cited.

Interpretation: Our findings lend further support for the use of the MoCA as a brief screen of cognition, with the additional finding that the MoCA domains appear to reflect similar constructs as those gleaned from a more comprehensive neuropsychological battery.

Future directions: Although the present study supports the construct validity of the MoCA domain scores when compared with more comprehensive neuropsychological measures, it does not evaluate the diagnostic sensitivity and specificity of the MoCA domain scores, which would be a valuable future direction. Future research may also consider using more heterogeneous samples with regard to clinical diagnosis.