Using the Montreal Cognitive Assessment Scale to screen for dementia in Chinese patients with Parkinson's Disease.

BACKGROUND: Dementia is one of the most distressing and burdensome health problems associated with Parkinson's Disease (PD). The Montreal Cognitive Assessment scale (MoCA) is widely used to screen for dementia in PD patients, but the appropriate diagnostic cutoff score when used with Chinese PD patients is not known. AIM: Determine a diagnostic cutoff value of the Chinese version of the MoCA (MoCA-C) for Chinese PD patients and describe the characteristics of PD patients screened positive for dementia using the MoCA-C.
METHODS: The presence of dementia in 616 PD patients and 85 community controls was determined using the Movement Disorder Society Task Force criteria (the gold standard diagnosis). We administered the MoCA-C to these individuals and used a receiver operating characteristic (ROC) curve to identify the cutoff score of the MoCA-C that most efficiently identified dementia in both PD patients and community controls. Demographic and clinical characteristics of PD patients who were screened positive or negative for dementia using the MoCA-C were compared.
RESULTS: A MoCA-C score of 23 was the optimal cutoff score for dementia in both patients and controls. Using this cutoff score, the sensitivity and specificity of the MoCA-C in PD patients were 0.70 and 0.77, respectively; the positive and negative predictive values were 0.59 and 0.85, respectively; and the overall concordance (kappa [95% confidence interval]) was 0.45 (0.39-0.52). The corresponding kappa value (concordance) in community controls was only 0.25 (0.05-0.45). Compared to PD patients who screened negative for dementia, those who screened positive for dementia were significantly impaired in all cognitive domains, including visuospatial and executive functioning, naming, attention, language, abstraction, delayed recall and orientation (all p<0.001). Among the PD patients, screening positive for dementia was independently associated with old age, low educational attainment, female gender and more severe motor impairment.
CONCLUSIONS: The commonly recommended cutoff screening score for dementia of 26 on the MoCA it too high for PD patients in China; a cutoff score of 23 is more appropriate. Potential risk factors for dementia in Chinese PD patients include older age, less education, and more severe motor symptoms of PD.

Introduction

Parkinson's disease (PD) was once primarily described as a movement disorder, but there is now increasing recognition that the clinical features of PD also include non-motor symptoms, particularly cognitive impairment and dementia.[1] The estimated prevalence of dementia in PD patients ranges from 24 to 31%.[2] The cognitive profile of PD dementia (PD-D) includes severe and extensive deficits in executive functions, attention, memory, and visuospatial abilities.[1] Compared to PD patients without dementia, those with dementia have a shorter life expectancy and a lower quality of life, so early recognition is key for the prevention and treatment of PD-D.[3]

Diagnosis of dementia in PD using the fourth edition of the Diagnostic and Statistical Manual of Mental Disorders (DSM-IV)[4] is often difficult due to the old age and poor motor functioning of most PD patients,[5] so in 2007 the Movement Disorder Society Task Force (MDS-TF)[6] developed a separate diagnostic procedure for PD-D. This algorithm – which has proven more sensitive than the DSM-IV diagnosis[5] – includes the following criteria: (a) meets diagnostic criteria for PD proposed by the Queen Square Brain Bank;[7] (b) PD developed prior to the onset of dementia; (c) PD is associated with a decreased global cognitive efficiency demonstrated by a Mini-Mental State Examination (MMSE)[8] score of under 26; (d) cognitive deficiency is severe enough to impair daily life; and (e) impairment is evident in more than one cognitive domain.[6] The advantage of this algorithm is that it can be used by clinicians in an office setting or at the bedside; it doesn't require expertise in neuropsychological methods.

There are, however, problems with the MDS-TF algorithm. The MMSE is part of the MDS-TF algorithm and is the most commonly used instrument in diagnosing dementia, but the scale by itself has poor sensitivity and specificity in detecting dementia in PD patients. A comparative study has shown that the Montreal Cognitive Assessment scale (MoCA) is a better tool than the MMSE in this population.[9] When both tools were applied in Asian populations under various conditions, the reliability and validity of MoCA was superior to that of the MMSE for detecting cognitive impairment.[10]-[13] However, the diagnostic cutoff score for the MoCA used in these studies varies. Rossetti and colleagues[14] used a MoCA score of under 26 to identify individuals with cognitive impairment in a population-based sample, but it is unclear whether or not this cutoff score of 26 is appropriate for determining the presence of dementia in different populations, such as in individuals with PD.

The psychometric properties of the Chinese version of the MoCA (MoCA-C) have been evaluated by Hu and colleagues in a sample of elder Chinese;[15] they recommended a cutoff value of 26 for the diagnosis of mild cognitive impairment (MCI) and Alzheimer's disease (AD). However, the MoCA-C hasn't yet been validated in PD patients in China and an appropriate cutoff score for screening for dementia in Chinese patients with PD hasn't been determined. The purpose of the current study is to determine the cutoff score of MoCA-C when screening for dementia in PD patients in China and to assess the validity of this classification by comparing the cognitive functioning and other characteristics of PD patients who are screened positive or negative for dementia using this cutoff score.

Methods

Participants

As shown in Figure 1, enrolled patients were identified at the Parkinson's Disease Clinic at the Nanjing Brain Hospital from 2009 to 2012. Among the 671 patients treated at the clinic, 55 (8.2%) were excluded because of other neurological conditions (including cerebral infarction, cerebral hemorrhage, brain tumor, brain trauma), sensory impairments, depression (Hamilton Depression Scale[16] score>20), atypical clinical symptoms of PD or poor drug treatment effect (suggesting a questionable diagnosis of PD), or having less than 10 years of education. The recommended MMSE cutoff score of 26 (used when making the gold standard MDS-TF diagnosis of dementia) is only appropriate for respondents under 80 years of age with at least 10 years of formal education, so PD patients who didn't meet these criteria were excluded.

Figure 1

Flowchart of the study

The control group were a convenience sample of community members living in three neighborhoods in Nanjing who met the following criteria: (a) 50 to 80 years of age; (b) at least 10 years of formal education; (c) did not have PD or other neurological or mental disorders, and (d) not using any psychotropic medications. Among the 91 potential community control subjects identified, six were excluded due to a history of mental disorder, neurological disease or drug abuse.

All participants were able to speak Mandarin. All participants signed informed consent before their basic demographics information was obtained. The study was approved by the Ethics Committee of the Nanjing Medical University.

Assessments

Enrolled PD patients and community controls were classified into those with and without dementia using the MDS-TF criteria by a movement disorder specialist (author XF). As a part of making the MDS-TF diagnosis of dementia, all subjects were tested with the MMSE. Both MMSE and MoCA-C assess a range of cognitive skills on a scale of 0 to 30 points with higher scores indicating better performance. Believing that it would be best to make the assessments with the two scales at the same time, and considering the substantial overlap of several questions in the two scales, we decided to re-group individual items of the scales into four widely used cognitive domains (visuospatial, language, memory, and orientation) based on previous research.[17] The visuospatial items include design copy (both tests) and figure drawing to command (only MoCA-C). The language items include object naming (both tests), phrase/sentence repetition (both tests), verbal commands (only MMSE), and reading comprehension (only MMSE). The verbal memory items include recall of either five (MoCA-C) or three (MMSE) previously presented words. The MoCA-C also includes a fifth domain about executive function and attention that has items about phonemic fluency and visuospatial sequencing and alternation.

Chinese versions of the following instruments were also administrated to PD patients (but not to community controls) to assess PD motor symptoms, PD non-motor symptoms, depression, and anxiety: part III of the United Parkinson's Disease Rating Scale (UPDRS),[18] part V of the UPDRS (the modified Hoehn and Yahr staging of Parkinson's Disease),[18] Parkinson's Disease Non-Motor Symptoms questionnaire (PD NMS),[19] Parkinson's Disease Sleep Scale (PDSS) (higher scores represent better sleep quality),[20] Hamilton Depression Rating Scale (HAMD),[16] Hamilton Anxiety Rating Scale (HAMA),[21] Zung Self-rating Depression Scale (SDS),[22] and Zung Self-rating Anxiety Scale (SAS).[23]

All scales were administered by two trained neurologists using standardized survey language. A week after the initial assessment ten community controls were randomly selected to retest MMSE and MoCA-C; the test-retest reliability (interclass correlation coefficient [ICC]) was 0.63 and 0.71, respectively. In addition, test-retest reliability (ICC) values for the full battery of scales among 40 patients who were retested 2 to 4 weeks after the initial evaluation were as follows: MMSE=0.72; MoCA-C=0.69; PD NMS=0.76; UPDRS=0.71; PDSS=0.55; HAMA=0.56; HAMD=0.66.

Statistical analysis

All statistical analyses were performed using SPSS version 17.0 (SPSS Inc., Chicago IL). Chi-squared tests, Mann-Whitney rank tests (Z-tests), and t-tests were used to compare scores of MoCA-C and MMSE and other variables between groups. The significance level was set at 0.05.

To determine the screening cutoff score of MoCA-C, the MoCA-C scores of PD patients and of community controls with and without dementia (classified using the MDS-TF critera) were compared. The predictive accuracy of different potential cutoff scores was assessed using the area under the receiver operating curve (AUC). For each possible cutoff score, the sensitivity and specificity were computed and differences in sensitivity and specificity were tested with McNemar's tests. The score with the highest Youden Index score[24] (which maximizes sensitivity and specificity, giving them equal weighting) was identified as the optimal cutoff score.

The newly determined cutoff score of MoCA-C was then used to divide PD patients into two groups, those who screen positive or negative for dementia using the MoCA-C. Standard Cohen's kappa values were used to assess overall concordance of the MoCA-C screening results for dementia with the ‘gold standard’ diagnosis of dementia (i.e., based on MDS-TF criteria) and the 95% confidence intervals for kappa were assessed using the method proposed by Fleiss. [25] The demographic and clinical characteristics of PD patients classified into these two groups (i.e., those screened positive or negative for dementia using the MoCA-C) were compared. Variables that were statistically significant in the univariate analysis were subsequently entered into a multivariate logistic regression analysis that used a stepwise method to identify variables independently associated with screening positive for dementia using the MoCA-C.

Results

Diagnosis of dementia

As shown in Figure 1, using the MDS-TF criteria as the gold standard, 196 (31.8%) of the 616 PD patients met the criteria for dementia and 42 (49.4%) of the 85 community controls met the criteria for dementia. The demographic characteristics of the PD patients with dementia (PD-D) were not significantly different from those of the PD patients without dementia (PD-ND): 57.7% (113/196) of PD-D patients versus 64.5% (271/420) of PD-ND patients were male (χ2=2.69, p=0.101); the mean (sd) age of PD-D patients was 68 (10) years versus 66 (11) years for PD-ND patients (t=1.72, p=0.087); and the mean age of onset of PD was 60 (10) years in the PD-D group versus 59 (11) years in the PD-ND group (t=0.95, p=0.340). Similarly, there were no significant differences in the characteristics of community controls who did and did not met MDS-TF criteria of dementia: 57.1% (24/42) of community members with dementia were male versus 58.1% (25/43) of community members without dementia (χ2=0.01, p=0.926); and the mean age of those with dementia was 67 (11) years versus 63 (10) years in those without dementia (t=1.74, p=0.086).

Optimal cutoff score for dementia of the MoCA-C

The discriminatory validity of MoCA-C for detecting PD-D patients (defined using MDS-TF criteria) was examined using receiver operating characteristics (ROC) curves. As shown in Figure 2, the area under the curve (AUC) was 0.83 (95%CI=0.80-0.86) for PD patients. The biggest Youden Index was 0.48, and the optimal cutoff value was 22.5 (22/23). The sensitivity and specificity of this cutoff were 0.78 and 0.70, respectively.

Figure 2

Receiver operating characteristics (ROC) curve of Chinese version of Montreal Cognitive Assessment (MoCA-C) for identifying dementia (defined by MDS-TF criteria) in Chinese patients with Parkinson's disease

The ROC analysis of the community control group is shown in Figure 3. Among the controls, the AUC was 0.87 (95%CI=0.79-0.95), the biggest Youden Index was 0.62, and the optimal cutoff value was 23.5 (23/24). The sensitivity and specificity of this cutoff were 0.81 and 0.81, respectively.

Figure 3

Receiver operating characteristics (ROC) curve of Chinese version of Montreal Cognitive Assessment (MoCA-C) for identifying dementia (defined by MDS-TF criteria) in Chinese community control subjects

Since the cutoff value was 22/23 among PD patients and 23/24 among controls, we took 23 as the threshold to identify dementia in both the patient group and the community control group. Using a score of 23 or lower on the MoCA-C to identify PD patients at risk for dementia and comparing these results to the gold standard diagnosis using MDS-TF criteria, the sensitivity and specificity of the MoCA-C were 0.70 and 0.77, respectively; the positive and negative predictive values were 0.59 and 0.85, respectively; and the overall concordance (kappa [95% confidence interval]) was 0.45 (0.38-0.52). In the community control group the sensitivity and specificity of MoCA-C for identifying those with dementia were 0.76 and 0.49, respectively; the positive and negative predictive values were 0.59 and 0.68, respectively; and the overall concordance (kappa) was 0.25 (0.05-0.45). In the PD patients, the concordance of the MoCA-C screening result for dementia with the MMSE screening result for dementia (i.e., scores <26 screen positive and scores≥26 screen negative) was high (Kappa=0.80, 95% CI=0.75-0.91); the corresponding concordance of the dementia screening result of the MoCA-C and MSSE in community controls was fair (Kappa=0.67, 95% CI=0.52-0.82).

As shown in Table 1, for both the PD patient group and the community control group, individuals who screened positive for dementia using the MoCA-C total score of 23 had significantly poorer functioning than those who screened negative for dementia on all seven MoCA-C subscales (all p<0.001): visuospatial and executive functioning, naming, attention, language, abstraction, delayed recall, and orientation.

Table 1

Comparison of the total score and subscale scores of the Chinese version of the Montreal Cognitive Assessment (MoCA-C) scale between Parkinson's Disease patients who do and do not screen positive for dementia based on the total MoCA-C score (cutoff at 23) and between community controls who do and do not screen positive for dementia

	Patients with Parkinson's Disease				Community controls
	Screen positive for dementia(n=235)median (IQR)	Screen negative for dementia(n=381)median (IQR)	Mann-WhitneyZ-testa		Screen positive for dementia(n=54)median (IQR)	Screen negative for dementia(n=31)median (IQR)	Mann-WhitneyZ-testa
Total MoAC-C score	18 (15-21)	26 (24-28)	20.4		17 (12-19)	28 (26-30)	6.8
visuospatial and executive functioning	1 (0-3)	4 (3-5)	16.2		0 (0-2)	5 (4-5)	7.2
naming	3 (2-3)	3 (3-3)	11.3		3 (2-3)	3 (3-3)	4.5
attention	4 (3-5)	6 (5-6)	12.5		3 (2-5)	6 (6-6)	7.6
language	2 (1-3)	3 (3-3)	11.0		2 (0-3)	3 (3-3)	5.4
abstraction	1 (0-1)	2 (2-2)	13.0		1 (0-1)	2 (1.75-2)	5.9
delayed recall	1 (0-2)	4 (2-5)	14.2		2 (1-3)	5 (3-5)	6.4
orientation	6 (5-6)	6 (6-6)	9.2		6 (4-6)	6 (6-6)	4.8

IQR, interquartile range

a p-values for all Z-tests are <0.001

Comparison of PD patients who do and do not screen positive for dementia using the MoCA-C

The demographic and clinical results for the 235 PD patients who screened positive for dementia using the MoCA-C versus those of the 381 patients who screened negative for dementia are shown in Table 2. All variables assessed were significantly different between the two groups. Compared to those who screened negative for dementia using the MoCA-C, those who screened positive were more likely to be female, to be older, to have a lower level of education, and to have a later age of onset of PD; they also had more severe depressive and anxiety symptoms (both as assessed by the clinician and as self-reported), more severe non-motor and motor PD symptoms, a higher Hoehn and Yahr stage of PD, and more severe sleep disturbance (lower scores on the PDSS represent worse sleep quality).

Table 2

Demographic characteristics and clinical symptoms of Parkinson's Disease (PD) patients who do and do not screen positive for dementia on the Chinese version of the Montreal Cognitive Assessment (MoCA-C) scale

	Screen positive for dementia(n=235)	Screen negative for dementia(n=381)	statistic	p-value
Female (n [%])	123 (53.0%)	118 (34.3%)	χ2=19.94	<0.001
Age (mean [sd])	69.3 (10.0)	64.8 (10.6)	t=5.04	<0.001
Age of onset (mean [sd])	60.8 (11.4)	58.4 (10.9)	t=2.54	0.012
Years of education (median [IQR])	9 (6-12)	12 (9-15)	Z=10.60	<0.001
Depression (HAMD score) (median[IQR])	13 (7-21)	10 (6-16)	Z=3.67	<0.001
Depression (SDS score) (median [IQR])	31 (26-37)	29 (25-35)	Z=2.90	0.004
Anxiety (HAMA score) (median [IQR])	11 (7-17)	9 (4-14)	Z=3.83	<0.001
Anxiety (SAS score) (median [IQR])	30 (26-34)	28 (25-34)	Z=2.40	0.017
Sleep (PDSS score) (median [IQR])	116 (98-131)	121 (108-135)	Z=-3.80	<0.001
PD Non-motor symptoms (median [IQR])	12 (9-16)	10 (6-14)	Z=4.58	<0.001
PD motor symptoms (score of Part III of UPDRS)(median[IQR])	34 (22-43)	26 (24-28)	Z=7.57	<0.001
Hoehn and Yahr staging of PD (Part V of UPDRS) (median[IQR])	2.5 (2-3)	2.0 (1.5-2.5)	Z=5.15	<0.001

HAMD, Hamilton Depression Rating Scale

HAMA, Hamilton Anxiety Rating Scale

SDS, Zung self-rating Depression Scale

SAS, Zung self-rating Anxiety Scale

PDSS, Parkinson's disease Sleep Scale

UPDRS, United Parkinson's Disease Rating Scale

The logistic regression results, which identify the demographic and clinical variables of PD patients independently associated with being screened positive for dementia (i.e., a score of less than 23 on the MoCA-C) are shown in Table 3. Similar to the univariate analysis, greater age, female gender and lower educational attainment were associated with being screened positive for dementia. Among the measures of the severity of PD, only more severe motor symptoms (i.e., higher scores on Part III of the UPDRS) was independently associated with screening positive for dementia. None of the five psychological measures that were significant in the univariate analysis remained in the final multivariate model, primarily because they were all strongly correlated with the severity of PD motor symptoms (correlation of the five scale scores with the score on Part III of UPDRS ranged from 0.25 to 0.37).

Table 3

Demographic and clinical characteristics of 616 patients with Parkinson's Disease (PD) that are independently associated with screening positive for dementia on the Montreal Cognitive Assessment (MoCA-C) scale

Characteristic	p-value	Odds ratio (95%CI)
Age	<0.001	1.06 (1.03-1.08)
Male	0.045	0.60 (0.36-0.99)
Years of education	<0.001	0.80 (0.75-0.85)
Score on the motor symptoms section of the UPDRS	<0.001	1.06 (1.04-1.08)

UPDRS, United Parkinson's Disease Rating Scale

aConsidering all variables that were significant in the univariate analysis (see Table 2), the multivariate logistic regression analysis first forced age, gender, and years of education into the model and then used a forward stepwise method to identify other significant variables. The Nagalkerke R-squared for the final model was 0.406.

Discussion

Main findings

In this study, the proportion of PD patients with dementia was 38.2%, which is in accordance with previous findings. [2],[26] The optimal diagnostic cutoff scores for the MoCA-C determined in our study were 22/23 for PD-D patients and 23/24 for controls. Both were lower than the previously published normative data for the MoCA. [14] In fact, many researchers have recommended that the optimal cutoff value of MoCA for diagnosing dementia should be lower than 26. [9],[27] In a recent study, Wang and colleagues[28] examined the sensitivity, specificity, and threshold scores of the MoCA-C in identifying dementia with Lewy bodies (DLB), and the cutoff score they found was 22/23, with a sensitivity of 91.7% and a specificity of 80.6% (AUC=0.932). When using a score of 26 as the cutoff, Nazem and colleagues[29] found that PD patients who screened positive for dementia performed worse on only five out of seven MoCA subtests; the subscale scores for attention and abstraction were not significantly different between PD patients who screened positive and negative for dementia. Using a lower cutoff score of 23, we found that PD patients who screened positive for dementia differed significantly in all seven subtests of MoCA-C than those who screened negative for dementia, a difference that was also observed among controls. Therefore, using a MoCA-C score of 26 as the threshold to screen for dementia might falsely include PD patients (and, possibly, other types of individuals) whose global cognition is not sufficiently impaired to justify classification as dementia.

PD-D has been called a ‘dysexecutive dementia’, a condition that leads to a broader range of cognitive deficits than in Alzheimer's Dementia. [6],[30] Previous studies reported a higher prevalence of PD-D in patients with the postural instability and gait disorder (PIGD) motor subtype of PD. [31],[32] Mild cognitive impairment, older age and severe motor symptoms were also found to be risk factors for PD-D. [33] Another study conducted by Levy and colleagues[34] suggested that the combined effects of increasing age and more extrapyramidal signs is the primary factor that contributes to the elevated risk of dementia in patients with PD. Our study, which compared PD patients screened positive or negative for dementia using the MoCA-C cutoff score of 23, found that current age, lower educational attainment and the overall severity of motor impairment were independently associated with the development of dementia in PD patients. As measured by UPDRS III, PD patients who screened positive for dementia had much more difficulty in specific motor tasks than PD patients who screened negative for dementia. We also found that despite the lack of a significant difference in the prevalence of dementia (based on the MDS-TF criteria) between female patients with PD and male patients with PD, female patients with PD were more likely than male patients with PD to screen positive for dementia using the MoCA-C; further prospective studies using the MoCA-C are needed to clarify the reasons for this difference. Overall, our results both confirm the validity of the MoCA-C as a screening tool for dementia in PD patients and suggest that progressive motor disability has a synergistic effect with increasing age that elevates the risk of dementia among patients with PD. [1]

Limitations

Several issues need to be considered when assessing these results. It is unclear how representative PD patients recruited from the Parkinson's Disease clinic of the Nanjing Brain Hospital are of all patients with PD; we expect that this sample is representative of urban patients in China who are receiving treatment for PD, but the characteristics of rural PD patients and of PD patients not receiving regular treatment may be quite different.

The relatively small number of community control subjects (n=85) and the very high prevalence of dementia in these subjects (49.4%) indicates that they were not representative of older community members as a whole. Most of these individuals were elderly residents who were available at home during the day, so many had physical disabilities and cognitive difficulties. The MoCA-C cutoff score identified for dementia in community members may be different if a more representative group of elder urban community residents were chosen.

In order to simplify the administration of the survey, we integrated the items for MMSE and MoCA-C. This may have influenced respondents' results so it is not certain that the MMSE scores and MoCA-C scores are the same as would occur if the two scales were administered independently.

We determined the optimal cutoff score for the MoCA-C using the Youden Index, which gives equal weight to sensitivity and specificity and, thus, assumes that false positive results are of equal importance as false negative results. If, however, the relative importance of false positive and false negative results when screening for a condition are different – which may be the case in dementia because of the stigma associated with the label and the current lack of effective treatments – there would be a need to adjust the cutoff score accordingly.

Implications

To our knowledge, this is a first report to determine the appropriate cutoff score of the Montreal Cognitive Assessment scale for identifying dementia in Chinese patients with Parkinson's Disease. We recommend a substantially lower cutoff score than has previously been recommended in other countries (23 v. 26) so the results have substantial clinical implications for the screening of Chinese PD patients, and, potentially, for the general population screening for dementia in China. Similar to other studies we found that PD patients who screen positive for dementia using the MoCA-C cutoff score were older, had a lower educational level, and had more severe PD motor symptoms; this suggests that further research is needed to clarify the interaction between motor impairment and cognitive impairment among patients with PD.