Validity of the Mini-Mental State Examination-2 in Diagnosing Mild Cognitive Impairment and Dementia in Patients Visiting an Outpatient Clinic in the Netherlands.

This study examined the utility of the recently published MMSE-2:SV in detecting cognitive impairment. We used receiver operating characteristics to test the discriminative power of the MMSE-2:SV for distinguishing between older adults without mild cognitive impairment (MCI) or dementia (n=67) and patients with MCI (n=76) or dementia (n=79). The results show that the MMSE-2:SV had excellent discriminative ability in distinguishing older controls from patients with dementia, with cut-off scores of 26 and 27 (max=30) yielding appropriate sensitivity (0.810 and 0.924, respectively) and specificity (0.940 and 0.806). Discriminative power was close to good in distinguishing between older controls and patients with MCI. Here, however, no optimal cut-off point could be determined. Even though this study shows good sensitivity and adequate specificity for the MMSE-2:SV in discriminating individuals without MCI or dementia from those with dementia, its validity is limited for identifying patients with MCI.

Cognitive decline is the primary criterion for dementia and its predementia stage, mild cognitive impairment (MCI). Consequently, valid, reliable, and sensitive assessment of cognitive (dys)function in patients referred to a memory clinic is crucial. A widely used screening instrument to assess cognitive decline in patients with suspected dementia is the Mini-Mental State Examination (MMSE).1 Although widely used, the MMSE has several shortcomings. These include the lack of sensitivity for detecting MCI, the failure to adequately discriminate between patients with mild dementia from older controls, especially in individuals with higher education levels, the lack of parallel versions and poor standardization and harmonization across countries.2

In 2010, a revised version of the MMSE, the Mini Mental State Examination—Second Edition (MMSE-2) has been introduced to improve the sensitivity and validity of the test (problematic items have been removed and several tasks have been changed to further standardize the instrument for translations into other languages and cultures, and to adjust the difficulty level). At the same time, the structure and scoring of the original 30-point MMSE remains equivalent in the standard version of the MMSE-2 (MMSE-2:SV) to enable upgrading from the MMSE to the MMSE-2 in longitudinal data collection.3 The MMSE-2 has been released in 3 versions that are available in several languages; the standard MMSE-2:SV, the brief version (MMSE-2:BV), and the extended version (MMSE-2:EV), with 2 parallel versions.

To date only limited evidence on the validity of the MMSE-2 is available.4,5 The primary objective of this study is to determine the validity of the MMSE-2:SV in clinically classifying cognitive impairments (MCI and dementia) and to determine cut-off scores for the MMSE-2:SV for discriminating between older adults without a diagnosis of MCI or dementia, MCI patients and dementia patients, by determining the sensitivity, specificity, and areas under the curve (AUCs).

METHODS

A sample of 199 geriatric outpatients was recruited, who were all referred for a neuropsychological examination and who were recruited through their treating psychologists via the geriatric outpatient clinic of the Jeroen Bosch Hospital, ‘s-Hertogenbosch, the Netherlands. Only patients with a clinical diagnosis of MCI or dementia were eligible for this study. Other inclusion criteria were aged 50 or older, being fluent in Dutch, and having completed at least primary school. Patients were excluded if they had impairments in vision, a history of or current primary psychiatric disorders, alcohol, and/or drug abuse or severe fatigue.

The patients’ clinical diagnoses were made in a multidisciplinary way based on the clinical interview, neuropsychological examination (NPE, see Supplemental Digital Content 1, http://links.lww.com/WAD/A288), physical examination, neurological assessment and—if available—neuroimaging findings (magnetic resonance imaging or computed tomography). The clinical criteria of MCI were met when the patient reported a subjective decline in cognitive function, the test scores on the NPE showed cognitive impairment in the absence of daily-life interference.6 The clinical criteria for dementia were met when the NPE suggested cognitive impairments that interfered with daily life.7

Sixty-nine participants without a diagnosis of MCI or dementia (older controls) were recruited from the waiting room of the same outpatient clinic. Participants were included if they had no subjective cognitive complaints based on an in-depth interview regarding potential cognitive and psychiatric complaints, were living independently at home, were aged 50 or older, fluent in Dutch, and had completed at least primary school. Exclusion criteria were a (history of) psychiatric disorders or the use of psychotropic medication that could affect cognitive function.

Ethical approval was obtained for this study from the institutional review board of Jeroen Bosch Hospital and written informed consent was obtained from all participants. Education was coded using an ordinal rating scale ranging from score 1 (primary education) to score 8 (university degree). A total of 268 participants completed the authorized Dutch version of the MMSE-2:SV3 and were assessed between June 16, 2015 and November 23, 2017.

To examine the validity and to calculate the cut-off scores for distinguishing the 3 diagnostic groups, receiver operating characteristics curves were determined using IBM SPSS version 26.0. A cut-off score was found adequate if its sensitivity and specificity were higher than 0.8. The AUC was calculated as a measurement of the discriminating power. AUC values between 0.9 and 1.0 were considered excellent, values between 0.8 and 0.9 as good, values between 0.7 and 0.8 as acceptable, and values between 0.6 and 0.7 as poor.8

RESULTS

Two control participants (lack of time to finish the examination) and 44 patients (diagnosed with a primary psychiatric disorder, failing performance validity testing, or not meeting the MCI or dementia criteria) were removed from the analyses. A total of 222 participants were included in this study, 67 of whom were older controls without MCI or dementia and 155 were patients. Seventy-six patients were clinically classified as having MCI and 79 as having dementia. The mean age of the total group of participants was 72.2 (SD=9.0). Table 1 shows the relevant demographic characteristics of the 3 groups of participants and the total scores on the MMSE-2.

TABLE 1

Demographic Variables and Performance on the MMSE-2:SV for the Older Controls Without an MCI or Dementia Diagnosis, the Patients With MCI, and the Patients With Dementia

			Dementia (n=79)
	Older Controls	MCI	AD	VaD	FTD	Mixed	Other
n	67	76	35	9	8	18	9
Age (y)
Mean (SD)	66.9 (10.4)	73.2 (7.3)	75.9 (7.7)	74.6 (4.6)	73.1 (7.7)	76.7 (6.4)	75.2 (7.6)
Range	50-86	54-93	56-87	67-81	63-83	67-91	62-87
Sex
Men	25	40	14	5	3	13	6
Women	42	36	21	4	5	5	3
Education
Low	18	29	12	3	3	5	6
Primary school	5	8	3	1	0	3	2
Lower vocational	13	21	9	2	3	2	4
Middle	25	23	12	1	4	6	6
Middle school	11	8	5	0	2	4	2
Secondary vocational	14	15	7	1	2	2	4
High	24	24	11	3	1	6	0
Selective secondary education	3	4	1	0	0	2	0
Higher vocational	13	16	7	3	1	2	0
Higher secondary education	0	1	0	0	0	0	0
University	8	3	3	0	0	2	0
MMSE-2:SV	28.7 (1.3)	26.6 (2.3)	23.3 (4.1)	25.7 (2.3)	24.1 (4.4)	23.5 (2.5)	25.3 (1.5)

Mixed dementia indicates combined etiology, e.g. AD with VaD.

The “other” category includes Parkinson spectrum dementia (n=2) or unclear etiology (n=7).

AD indicates Alzheimer’s Dementia; FTD, Frontotemporal Dementia; MCI, Mild Cognitive Impairment; VaD, Vascular Dementia.

Figure 1 shows the AUCs of the receiver operating characteristics analyses. The discriminating power of the MMSE-2 for distinguishing controls from dementia patients was excellent (AUC=0.946; P<0.001) and was good (AUC=0.790; P<0.001) for distinguishing control participants from MCI patients. Next, our aim was to determine valid cut-off scores of the MMSE-2 based on adequate sensitivity (>0.8) and the accompanying specificity (>0.8) for discriminating individuals without MCI or dementia from MCI and dementia patients. Cut-off scores of 26 and 27 both had a good sensitivity (0.810 and 0.924, respectively) and good specificity (0.940 and 0.806) in distinguishing between older controls and patients with dementia. However, no optimal cut-off point could be determined for distinguishing between individuals without MCI or dementia and patients with MCI. The optimal cut-off score for discriminating older controls from MCI patients was <27, which had a poor specificity (0.642) and an acceptable sensitivity (0.776), not meeting our criteria for an adequate cut-off.

FIGURE 1

Areas under the curve (AUC) of the 2 receiver operating characteristics analyses. MCI indicates mild cognitive impairment; OC, older controls.

DISCUSSION

It was the aim of this study to determine the validity of the MMSE-2:SV in clinically classifying cognitive impairments (MCI and dementia) and determine sensitive and specific cut-off scores for the MMSE-2:SV to discriminate between older adults without an MCI or dementia diagnosis, MCI patients, and dementia patients based on clinical diagnoses. A cut-off score of 27, with both a good sensitivity (0.924) and specificity (0.806), could be determined for discriminating older individuals without an MCI or dementia diagnosis from dementia patients. No sensitive and specific cut-off scores could be determined for discriminating individuals without an MCI or dementia diagnosis from MCI patients. When a cut-off score ≤28 was selected for discriminating older adults without MCI/dementia from MCI-patients, sensitivity was acceptable (0.776), but insufficiently specific (0.642). Therefore, an extended NPE would still be necessary to correctly classify patients. These results indicate that the MMSE-2:SV is not an improvement in discriminating adults without MCI or dementia from MCI patients, compared with the original MMSE. For this reason, we conclude in line with previous evidence2,9 that the psychometric characteristics and validity of the MMSE-2 are limited.

One finding that should be discussed is the relatively high cut-off score of 27 that was found to best discriminate between dementia patients and the older control group without MCI/dementia, compared with the more traditional score of 24 that is commonly used for the MMSE.1 First of all, cut-off scores for dementia may be significantly higher in highly educated patients.10 While some in our sample indeed were highly educated, the majority (75%) had a low to average education. Next, most of our patients were in the early stage of dementia with relatively mild cognitive deficits. Furthermore, it is possible that we included a high functioning control group with consequently very high MMSE scores. Nonetheless, these explanations all reflect the limitations of the MMSE(-2), an instrument that has been shown repeatedly to have poor psychometric properties, notably a low sensitivity in average to highly educated patients with only mild cognitive deficits, as is the case in our memory clinic patients.

This study also has some limitation. Even though the sample used in the present study is representative for the geriatric population in the Netherlands, this study has a bias in the examined population. The patients in our study, who were all referred for an extensive neuropsychological examination, only had minimal to mild cognitive problems. More severely impaired individuals (who were referred for a shorter neuropsychological screening) were not included. Possibly, the current study underestimates the discriminative power of the MMSE-2, at least for those analyses that included the dementia patients. In addition, including all-cause dementia instead of only Alzheimer may have underestimated the discriminative power of the MMSE-2. However, in this study the discriminative power of the MMSE-2 to distinguishing older adults without MCI or dementia from dementia patients was already excellent. Finally, the older controls without MCI or dementia did not complete an extensive neuropsychological assessment. One could thus argue that as a result, this sample may include individuals with “undetected” cognitive impairments. However, none of these older controls reported any cognitive problems in an in-depth interview regarding potential cognitive and psychiatric complaints. As a result, none of the older controls fulfilled the Petersen6 clinical criteria for MCI as a clinical syndrome, as none of our controls experienced subjective cognitive complaints which is a prerequisite for this diagnostic label,6 in line with their near-ceiling MMSE-2:SV score. Our finding that the MMSE-2:SV cannot distinguish validly between older controls and MCI patients is also in line with a study examining the psychometric properties of the Korean version of the MMSE-2:SV,5 which reported a lower control mean MMSE-2:SV score of 27.3 (SD=2.7) in a group with a similar mean age as ours, as well as a “best” cut-off score of 26 that even had a worse sensitivity (0.74) and specificity (0.59) than the one reported in our paper.

In sum, even though this study showed good sensitivity and adequate specificity for the MMSE-2:SV in discriminating older adults without an MCI or dementia diagnosis from those with dementia, no cut-off score with good sensitivity and adequate specificity could be determined for discriminating MCI patients from dementia patients or for discriminating older controls without MCI or dementia from patients with MCI. We therefore do not recommend using the MMSE-2:SV to screen for cognitive impairments in a memory-clinic population.

Supplemental Digital Content is available for this article. Direct URL citations appear in the printed text and are provided in the HTML and PDF versions of this article on the journal's website, www.alzheimerjournal.com.