Validation of Four Methods for Converting Scores on the Montreal Cognitive Assessment to Scores on the Mini-Mental State Examination-2.

BACKGROUND AND
PURPOSE: There are many methods for converting scores from the Montreal Cognitive Assessment (MoCA) to those on the Mini-Mental State Examination (MMSE). In this study we aimed to validate 4 methods that convert the full score range (0-30 points) of the MoCA to an equivalent range for the MMSE.
METHODS: We examined the medical records of 506 subjects who completed the MoCA and MMSE-second edition (MMSE-2) on the same day. For the validation index, we calculated mean, median, and root-mean-squared error (RMSE) of the difference between true and equivalent MMSE-2 scores. We also calculated intraclass correlation coefficients (ICCs), the Bland-Altman plot, and the generalizability coefficient between true and equivalent MMSE-2 scores for reliability. We compared the ICCs according to age, sex, education, MMSE, and cognitive-status subgroups. For accuracy, we evaluated a ±2 point difference between the true and equivalent MMSE-2 scores.
RESULTS: The 4 conversion methods had a mean of -0.79 to -0.05, a median of -1 to 0, and an RMSE of 2.61-2.94 between true and equivalent MMSE-2 scores. All conversion methods had excellent reliability, with an ICC greater than 0.75 between true and equivalent MMSE-2 scores. These results were almost maintained in the subgroup analyses. These conversion methods provided more than 65% accuracy within ±2 points of the true MMSE-2 scores.
CONCLUSIONS: We suggest that these 4 conversion methods are applicable for converting MoCA scores to MMSE-2 scores. They will greatly enhance the usefulness of existing cognitive data in clinical and research settings.

INTRODUCTION

The Mini-Mental State Examination (MMSE) is the most widely used instrument for screening cognitive dysfunction. It has been translated and validated in many countries and languages.12 To date, the MMSE has been used as a standardized cognitive screening tool, and its score or score changes are important in clinical practice.34 In 2010, a revised version, the MMSE-second edition (MMSE-2), was introduced. The original 30-point MMSE (MMSE-1) structure and scoring are equivalent to the standard version of the MMSE-2 to enable upgrading from the MMSE-1 to the MMSE-2 in longitudinal data collection.56 However, under copyright restrictions, the MMSE-1 can no longer be used, and the MMSE-2 must be purchased, potentially limiting its routine use in clinical and research settings.7

The Montreal Cognitive Assessment (MoCA) is another screening tool that can discriminate normal cognitive function, mild cognitive impairment (MCI), and early-stage dementia.8 Because it can be used free of charge and its psychometric properties are superior to those of the MMSE in various aspects, MoCA use is increasing.

Clinical practice guidelines for dementia include both the MMSE and the MoCA as dementia screening tools.9 However, real clinical and research settings differ in their use of these 2 tools. This difference makes comparing the patient's cognitive status during longitudinal follow-up and meta-analyses difficult, because a direct comparison of MMSE and MoCA scores is complicated. The MMSE, but not the MoCA, has been required as a special examination for prescription and reimbursement for Alzheimer's disease treatments in many countries, including South Korea.1011 Many clinicians want to predict an MMSE score based on an MoCA score.

Previous studies have attempted to develop methods for converting MoCA scores into MMSE scores.121314151617181920 For the conversion algorithms, most methods use the equipercentile equating algorithm.1213141516171819 Equipercentile equating allows for a direct crossover from the score on one test to a score on a different test based on percentile rankings of scores that always fall within the range of possible scores. Another method used is the regression equation,20 which is a simple way to calculate approximate conversion scores. Still, these scores may be out of the range of possible scores, or the equation formula may be complex because of the relationship between the 2 scores. For coverage of score ranges, 5 methods can convert the full possible score range of MoCA (from 0 to 30 points);1213141520 other methods cannot convert all possible scores.16171819 For the MMSE version, only Yang's method evaluated the conversion of MoCA scores into MMSE-2 scores.12 Other methods evaluated the conversion of MoCA scores to MMSE-1 scores.1314151617181920 However, it is possible to switch from the MMSE-1 to the MMSE-2 with no change in the scores.5 We also evaluated converting to MMSE-2 by using MoCA to MMSE-1 conversion methods. Considering these various aspects including the simplicity of the conversion algorithm, coverage of the full score range, and switching of the MMSE version, we evaluated 4 methods for converting MoCA scores to MMSE-2 scores.12131415

METHODS

Subjects

This was a retrospective observational study of subjects referred for neuropsychological screening to a memory clinic at a university hospital in the Republic of Korea. Overall, we evaluated 506 participants who visited the hospital from May 2020 to April 2021. The participants included 182 subjects with dementia, 84 with MCI, and 240 who were cognitively unimpaired (CU).

Subjects with dementia met the criteria for a major neurocognitive disorder proposed by the Diagnostic and Statistical Manual of Mental Disorders 5 of the American Psychiatric Association.21 Subjects with MCI were diagnosed by the criteria proposed by the International Working Group on MCI.22 In this study, CU subjects did not meet the criteria for MCI or dementia, but were recruited and assessed in the same way.23 CU subjects were functionally independent. Our purpose was to validate conversion scores in clinical practice. Therefore, we categorized the subjects as having a wide range of cognitive disorders, not specific types of dementia or MCI, as occurs with cognitive screening in real clinical practice.

We collected demographic data, including age, sex, and years of education. All participants underwent a comprehensive evaluation consisting of a detailed medical history, neurological examination, and a neuropsychological evaluation. In addition, we excluded the participants with a history of major neurological or psychiatric illness, a history of medication, major medical problems, or visual or hearing impairment that could affect cognitive function or interfere with cognitive testing. All patients underwent brain magnetic resonance imaging and underwent the MMSE-2 followed by the MoCA on the same day. The results of the MoCA and MMSE-2 were not available during the consensus diagnosis.

For subgroups analysis, we divided the subjects in terms of age (<60 [n=57], 60−70 [n=115], and ≥70 [n=334] years of age), sex (male [n=207] and female [n=299]), education (≤6 years [n=250] and ≥7 years [n=256] of education), cognitive status (dementia [n=182], MCI [n=84], and CU [n=240]), and MMSE level (0–10 points [n= 41], 11–20 points [n=134], 21–25 point [n=149], and 26–30 points [n=182] of the true MMSE-2 score).

A previous study reported that the true and equivalent scores of the MMSE-2 had an intraclass correlation coefficient (ICC) of 0.849–0.961.12 We calculated the needed sample size as being 31 subjects by using Bonett's method24 at a confidence level of 95%, a precision of 0.1, 2 raters per subject, and a given ICC of 0.849–0.961. This study met and exceeded the minimal requirement of the sample size for total participants and for each subgroup.

The Institutional Review Board of Korea University Guro Hospital reviewed and approved the study protocol (2021GR0236). Informed consent was not necessary, because the study was retrospective and used de-identified data.

Neuropsychological assessments

The MoCA and MMSE-2 are simple, widely used cognitive screening tools with scores ranging from 0 to 30, where higher scores indicate better cognitive function. The MoCA evaluates visuospatial, naming, attention, language, abstract, memory, and orientation. The MMSE-2, like the MMSE-1, evaluates orientation in time, orientation in place, memory registration, attention and calculation, memory recall, and language and other functions. We used the Korean versions of the MoCA25 and the MMSE-2: Standard Version, Blue Form.6

Statistical analyses

Data are expressed as means±standard deviations (SDs) for continuous variables and as percentages for categorical variables. We evaluated these variables with chi-squared tests for differences between proportions, and used the Kruskal-Wallis test for differences between continuous variables after doing Levene's test for equality of variance. We used Bonferroni correction for post hoc comparisons. We used 4 methods to convert MoCA scores to MMSE-2 scores: Yang's method,12 Roalf's method,13 Trzepacz's method,14 and Saczynski's method15 (Table 1).

Table 1

Four methods for converting MoCA scores to MMSE-2 scores

MoCA score	Equivalent MMSE-2 score
	Yang's method12	*Roalf's method13	*Trzepacz's method14	*Saczynski's method15
0	0	3	6	0
1	2	6	9	2
2	4	8	10	5
3	7	9	11	8
4	10	10	12	11
5	13	11	12	12
6	14	12	13	14
7	16	13	14	15
8	17	14	14	16
9	18	15	15	17
10	19	16	15	18
11	20	17	16	18
12	20	19	17	19
13	21	20	18	20
14	22	21	19	21
15	22	22	20	21
16	23	22	21	22
17	24	23	22	23
18	25	24	24	24
19	26	25	25	24
20	26	26	26	25
21	27	26	27	26
22	27	27	27	27
23	28	28	28	27
24	28	28	29	28
25	29	29	29	29
26	29	29	30	29
27	30	29	30	30
28	30	30	30	30
29	30	30	30	30
30	30	30	30	30

MoCA: Montreal Cognitive Assessment, MMSE: Mini-Mental State Examination.

*These methods were originally developed for converting MoCA scores to MMSE-1 scores.

We calculated various validation index values using the mean with SD, median with interquartile range (IQR), and root-mean-squared error (RMSE) based on the difference between true and equivalent MMSE-2 scores. Smaller absolute values of the mean, median, and RMSE indicated a more accurate conversion from MoCA to MMSE-2 scores.

We evaluated the reliability using the ICC and confidence interval between the true and equivalent MMSE-2 scores. The ICC expresses concordance between 2 measures in a score ranging from 0 (no agreement) to 1 (perfect agreement). The ICC values were interpreted as poor (ICC < 0.40), fair (ICC = 0.40–0.59), good (ICC = 0.60–0.74), or excellent (ICC = 0.75–1.0).26 In this study, we selected a single-measure 2-way mixed absolute model with a type of absolute agreement. The ICC outperforms Pearson's correlation coefficient, because it contains both information on the correlation and the systematic difference between measurements. For reliability comparisons between the subgroups, we converted coefficients and compared them with Fisher's Z-transformation by Donner's method.27

The Bland-Altman plot28 evaluated the reliability between the true and equivalent MMSE-2 scores with limits of agreement (LOA) at ±1.96 SDs from the mean. This plot indicated that a range of differences between measurements is expected for any individual participant, unlike the ICC, which depends on sample heterogeneity. The 95% LOA between the true MMSE-2 score and the equivalent MMSE-2 score expressed the degree of error proportional to the mean of the measurement units. If the differences between the measurements tended to agree, the results were close to zero. These plots showed the difference between each pair of measurements on the y axis against the mean of each pair of measurements on the x-axis. We assessed bias using linear regression analysis.

Additionally, generalizability theory can analyze the variance to decompose various error types generated while measuring and calculate the test's reliability and validity.29 In this study, one facet was under investigation: form (the equivalent score of the MMSE-2). We estimated the variance of the measurement facets and their proportion with univariate generalizability. We also calculated the generalizability coefficient (G coefficient): a value greater than 0.8 is generally accepted as sufficient for high-stakes decisions. A value greater than 0.6 is sufficient for formative evaluations.29

We calculated accuracy as the proportion of participants whose score difference was within an acceptable range between the true and equivalent MMSE. In previous studies,1930 accuracy was evaluated with a ±2-point difference between the true and equivalent MMSE scores. Additionally, the reliable change index, a psychometric criterion used to evaluate whether the score difference between 2 measurements is statistically significant, was reported as 3 points6 or 4 points5 for the MMSE-2. We decided to evaluate accuracy with a percentage of converted scores within ±2 points of error. An error was the difference between the true score and the equivalent score of the MMSE-2.

We did analyses using SPSS for Windows, version 20.0 (IBM Corp., Armonk, NY, USA) and R 4.0.2 software with its appropriate packages (The R Foundation for Statistical Computing, Vienna, Austria).

RESULTS

The mean age of all participants was 72.4±9.6 years, and the mean length of education was 7.8±5.1 years. About 59.1% were women. The mean MMSE-2 and MoCA scores were 21.7±6.4 (range, 0–30) and 16.4±7.5 (range, 0–30), respectively. Clinical characteristics of participants are summarized in Table 2.

Table 2

Clinical characteristics of participants

Characteristics	Total (n=506)	Cognitive status			p-value	Post hoc
		Dementiad (n=182)	MCIm (n=84)	CUc (n=240)
Age	72.4±9.6	75.8±8.3	75.1±8.0	68.9±9.8	<0.001	d=m>c
Sex (female)	299 (59.1)	107 (58.8)	51 (60.7)	141 (58.8)	0.947	-
Education	7.8±5.1	6.7±4.9	6.5±5.5	9.2±4.7	<0.001	d=m<c
MMSE-2	21.7±6.4	15.4±5.9	21.5±3.1	26.5±2.3	<0.001	d<m<c
MoCA	16.4±7.5	9.0±5.2	15.5±3.9	22.3±3.8	<0.001	d<m<c

Values are presented as the means±standard deviations or numbers (%).

The p-values were compared between cognitive statuses by the Kruskal-Wallis test or the χ2 test.

MCI: mild cognitive impairment, CU: cognitively unimpaired, MMSE: Mini-Mental State Examination, MoCA: Montreal Cognitive Assessment.

Table 3 shows the results of validating the conversion from MoCA to MMSE-2 by 4 methods. For the validation index, the mean difference between true and equivalent MMSE-2 scores was −0.79, the median difference was −1.00, and the RMSE was 2.79 using Yang's method. Table 2 also summarizes these validation indexes using the other methods.

Table 3

Validation of MoCA to MMSE-2 conversion by various methods

Measures		Yang's method14	Roalf's method15	Trzepacz's method16	Saczynski's method17
Validation index
	Mean±SD	−0.79±2.37	−0.09±2.30	−0.05±2.65	−0.08±2.31
	Median (IQR)	−1 (−2, 1)	0 (−2, 0)	0 (−2, 0)	0 (−2, 0)
	RMSE	2.79	2.61	2.94	2.73
Reliability
	ICC(2,1)	0.964 (0.949–0.973)	0.968 (0.962–0.973)	0.957 (0.949–0.964)	0.968 (0.962–0.973)
G coefficient		0.906	0.904	0.905	0.903
Accuracy		71.3%	74.7%	65.4%	74.5%

ICCs are presented as 95% confidence intervals.

MoCA: Montreal Cognitive Assessment, MMSE: Mini-Mental State Examination, SD: standard deviation, IQR: interquartile range, RMSE: root-mean-squared error, ICC: intraclass correlation coefficient, G coefficient: generalizability coefficient.

For reliability, all 4 methods had excellent intra-rater reliability, with ICCs greater than 0.75 (Table 3). These ICCs of the 4 methods had no statistically significant differences when compared to each other after Bonferroni correction (z = −0.950 between Yang's method and Roalf's method; z=1.437 between Yang's methods and Trzepacz's method; z=−0.950 between Yang's method and Saczynski's method; z=2.387 between Roalf's method and Trzepacz's method; z=0.000 between Roalf's method and Saczynski's method; z=−2.387 between Trzepacz's method and Saczynski's method; for all, p>0.05).

Because there were differences in psychometric characteristics between the MoCA and the MMSE and because cognitive function might be influenced by clinical and demographic characteristics,89 we compared the reliability of the 4 methods according to clinical subgroups of age, sex, education, cognitive status, and MMSE level (Fig. 1).

Fig. 1

Comparison of reliability of 4 methods for MoCA to MMSE-2 conversion. ICCs (represented as dots) and corresponding confidence intervals at α=0.05 (represented as error bars) for subgroups (A) age, (B) sex, (C) education, (D) cognitive subgroups, and (E) MMSE. Gray dashed lines plus asterisks indicate statistically significant differences between subgroups after Bonferroni's correction. Horizontal fine dotted lines indicate the lower limit of good reliability (ICCs=0.6); thus, the ICCs above these horizontal lines have good reliability.

ICC: intraclass correlation coefficient, MoCA: Montreal Cognitive Assessment, MMSE: Mini-Mental State Examination.

For the age subgroups, the subgroup ≥70 years of age had an ICC different from that of the subgroup of 60–70 years by Yang's method (z=2.580, p<0.05), Roalf's method (z=2.916, p<0.05), and Saczynski's method (z=2.504, p<0.05), and the subgroup <60 years of age was different from the subgroup 60–70 years of age by Yang's method (z=2.523, p<0.05). The other comparisons between age subgroups had no statistically significant differences. For the sex and education subgroups, there was no statistically significant difference between the 4 methods. For cognitive status, subjects with dementia had ICCs different from those of subjects who were CU (z=−6.090, p<0.05 by Yang's method; z=−5.832, p<0.05 by Roalf's method; z=−4.129, p<0.05 by Trzepacz's method; z=−6.166, p<0.05 by Saczynski's method), and subjects with MCI (z=−4.287, p<0.05 by Yang's method; z=−3.770, p<0.05 by Roalf's method; z=−2.579, p<0.05 by Trzepacz's method; z=−4.040, p<0.05 by Saczynski's method). For the MMSE-level subgroups, most comparisons between MMSE subgroups with the 4 methods found no statistically significant differences except for the MMSE subgroup of 11–20 points, which had a difference from the subgroup of 21–25 points by Trzepacz's method (z=3.388, p<0.05). All 4 methods had more than good intra-rater reliability between the various subgroups, with an ICC greater than 0.60,26 except for Trzepacz's method in the MMSE 0–10 points and 21–25 points (Fig. 1).

Bland–Altman plots showed that the mean differences between the true and equivalent MMSE-2 scores were −0.84, 0.14, 0.10, and 0.78 in Yang's, Roalf's, Trzepacz's, and Saczynski's methods, respectively (Fig. 2). There was no indication of systemic bias between the 2 scores according to the regression coefficient in Roalf's method (y=−0.025×x+0.4, p=0.161) and Trzepacz's method (y=−0.0026×x−0.043, p=0.898). However, there was an indication of systemic bias in Yang's method (y=−0.055×x+0.38, p=0.002) and Saczynski's method (y=0.13×x−3.6, p<0.001).

Fig. 2

A Bland-Altman plot of the difference in the raw scores and the converted scores of the MMSE-2. The solid line indicates the reference (no mean difference), the middle-dotted line is the mean difference, and the upper and lower dotted lines are the LOA representing ±1.96 standard deviations from the mean difference in which 95% of the differences between the 2 scores are expected to fall. The dash-dotted lines are the fitted regression line, which indicates statistically significant linear trends on (A) Yang's method and (D) Saczynski's method, but there were no statistically significant trends on (B) Roalf's method or (C) Trzepacz's method.

LOA: limits of agreement, MMSE: Mini-Mental State Examination.

When we applied generalizability theory, the reliability estimates compared with the true MMSE-2 scores were as follows: G coefficient of the equivalent MMSE-2 score in Yang's method=0.906, in Roalf's method=0.904, in Trzepacz's method=0.905, and in Saczynski's method=0.903, all of which indicated they were sufficient for high-stakes decisions.

For the accuracy of the 4 conversion methods, we found that 65.4%–74.5% of the 4 equivalent scores were within ±2 points of the true MMSE-2 scores (Table 3).

DISCUSSION

In this study we retrospectively evaluated the 4 methods for converting MoCA scores to MMSE-2 scores. We evaluated the validation index, including mean, median, and RMSE differences and the reliability and accuracy between the true and equivalent MMSE-2 scores. The 4 conversion methods had excellent reliability between true and equivalent MMSE-2 scores. These 4 conversion methods provided more than 65% accuracy for estimated scores within ±2 points of the true MMSE-2 scores. To the best of our knowledge, this is the first study to evaluate the validity of the various methods for converting from the MoCA to the MMSE-2.

For the validation index, we evaluated mean±SD, median (IQR), and RMSE differences between true and equivalent MMSE-2 scores by the 4 methods. Although there were no absolute criteria for these indexes, we thought all 4 methods had acceptable index values. We found that these values were close to a small difference between true and equivalent MMSE-2 scores.

We evaluated ICCs, the Bland-Altman plot, and the G coefficient with the 4 conversion methods for reliability. All 4 methods had excellent ICCs and acceptable G-coefficient values for high-stakes decisions between true and equivalent MMSE-2 scores. However, the Bland-Altman plot analysis showed results different from those from the 4 methods. Although the mean differences between the true and equivalent MMSE-2 scores were close to zero (from −0.84 to 0.78) for the 4 methods, there were some systemic biases in the agreement between the true and equivalent MMSE-2 score measurements. This finding showed that Yang's method had a negative correlation and Saczynski's method a positive correlation between the mean and the difference of the true and equivalent scores of the MMSE-2. Roalf's and Trzepacz's methods had no systemic bias in the agreement between the mean and the difference of the true and equivalent scores of the MMSE-2. As in this study, a previous study reported a bias of distribution across the mean and difference in Yang's method.12 Therefore, taking these characteristics into account, reliability should be carefully considered when using these 4 methods.

In the subgroup analysis, although some methods had differences in the age subgroup analysis, none of the 4 methods had a statistically significant difference in ICCs according to age, sex, and education. However, in the subgroup analysis for cognitive status, the dementia subgroup presented a statistically significant difference in ICCs when the MCI and CU subgroups were compared by means of all 4 methods. The dementia subgroup had higher ICCs between the true and equivalent MMSE-2 scores. We suppose that the reliability in the CU and MCI subgroups might be lower than in the dementia subgroup, because the MMSE was less sensitive than was the MoCA for screening higher cognitive function, and the MMSE has a ceiling effect in higher cognitive function.831 However, all 4 methods had more than good reliability regardless of the subgroup analysis according to age, sex, education, and cognitive status. In the subgroup analysis, according to the true MMSE-2 scores, most methods except for Trzepacz's had more than good reliability, with no statistically significant difference.

For the accuracy of the estimated MMSE-2 scores, we found that the 4 methods had 65.4%–74.5% accuracy within ±2 points of the true MMSE-2 scores. We thought that these results were not grossly different from those of the previous reports that 59.1%–70.0% of the converted MMSE-2 scores were within ±2 points of the raw scores of the MMSE-2.14 Together, we examined the equivalency of the true and equivalent scores of MMSE-2 using G coefficients. The result of this test was greater than 0.8 for all 4 methods, which is sufficient for high-stakes decisions; all 4 methods could have acceptable conversion and equivalent ability with sufficient accuracy and reliability.

This study had some limitations. First, it was subject to all the limitations inherent in a retrospective study, and there may have been some degree of selection bias. Second, because we collected the data with the Korean versions of neuropsychological tools from participants in the Korean population and memory clinic, their generalizability in other language versions of the tools and other demographic or clinical conditions needs to be explored further. Participants with subjective cognitive decline might have been recruited as subjects with normal cognition. No dementia subtypes or MCI subtypes were specifically examined. Third, the order of administration of the MMSE and MoCA was not randomized to minimize how learning from one test might have affected the other. Fourth, there may have been undisclosed minor effect from different items between the MMSE-1 and MMSE-2, although the MMSE-1 and MMSE-2 are known to be interchangeable.56

In conclusion, we validated 4 methods for converting MoCA scores to MMSE-2 scores. These will serve as a useful reference for continuing assessment with the MMSE-2 in subjects who were previously given the MoCA. It provides a straightforward comparison of the MoCA to the MMSE-2, allowing for the continuity of cognitive tracking in clinical settings and the comparability of data between longitudinal studies.