Development and Validation of the Full Version of Story Memory in the Korean-Mini Mental State Examination, 2nd Edition: Expanded Version (K-MMSE-2: EV).

BACKGROUND AND
PURPOSE: The Korean version of Story Memory (SM) in the Korean-Mini Mental State Examination, 2nd Edition: Expanded Version (K-MMSE-2: EV) was developed. Based on the SM, we additionally developed a full version of SM including delayed recall (DR) and recognition adding to immediate recall (IR). This study aimed to examine the reliabilities and validities of the newly developed SM in the K-MMSE-2: EV and its full version.
METHODS: Ninety-five healthy elderly individuals (HE), 90 patients with amnestic mild cognitive impairment (aMCI), and 53 patients with dementia of the Alzheimer's type (DAT) participated in the study. They were administered the full version of SM with the Seoul Verbal Learning Test-Elderly's version (SVLT-E) and Rey Complex Figure Test (RCFT). In addition, the SM was re-administered to 51 participants after a 5-week interval. Two clinical neuropsychologists independently rated the performance of 50 participants.
RESULTS: The test-retest reliabilities of the IR, DR, and recognition of the SM were statistically significant. The inter-rater reliabilities (Cohen's kappa) were high (0.87-1.00) for all the measures. The IR, DR, and recognition of SM had significant positive correlations with those of the SVLT-E and RCFT. Significant group differences in IR and DR of SM were found among the HE, aMCI, and DAT groups. The recognition scores were significantly different between the aMCI and DAT groups, but not between the HE and aMCI groups.
CONCLUSIONS: The newly developed full version of SM in the K-MMSE-2: EV was proven to be a reliable and valid memory measure for clinical use.

INTRODUCTION

Episodic memory is an integral cognitive function in the assessment of dementia, since it is the most affected cognitive function in dementia, and also the key early marker in prodromal stages such as amnestic mild cognitive impairment (aMCI). The common methods for assessing episodic memory in the verbal modality are list learning and story recall. However, previous studies have compared these two methods and reported inconsistent findings. Several studies suggest that list learning has better sensitivity in distinguishing between healthy controls, MCI, and Alzheimer's disease (AD), and for predicting the rate of conversion from mild cognitive impairment (MCI) to AD.123 Rabin et al. suggested that using a combination of the word list recall and story memory methods would provide the highest diagnostic sensitivity and specificity for differentiating MCI from normal aging.4 Wicklund et al.5 reported that story recall was more sensitive in differentiating patients with AD from frontotemporal lobe dementia, compared to word list recall. Tremont et al.6 suggested that story recall is the memory associated with more semantically-related information and may be more directly related to temporal lobe integrity compared to list recall.

Despite its clinical utility, story recall has been used only selectively as a verbal memory measure, while word list recalls are more commonly used in clinical practice in Korea. The 2 most commonly used neuropsychological batteries, the Seoul Neuropsychological Screening Battery, 2nd edition (SNSB-II)7 and the Korean version of the Consortium to Establish a Registry for Alzheimer's Disease (CERAD-K), 2nd edition,8 do not include story recall but include list recall as a verbal memory measure. Although the Literacy Independent Cognitive Assessment (LICA)9 published recently includes a story recall test, it has limitations of use such as the ceiling effect on highly-educated people since the LICA was developed specifically for the illiterate and less-educated people.

Poor performances on free recall tests are common in many disorders.10 Different types of memory deficits seemed to be caused by different mechanisms. For example, impaired registration may be caused by attentional deficits due to depression,1112 while impaired consolidation and storage are associated with lesions of the hippocampus and related structures commonly seen from patients with AD.1314 Executive dysfunction is a common cause of deficits in retrieval of stored information.1516 Memory impairments are manifested in a variety of forms, encoding, storage, and retrieval problems. Thus, at least three different stages of memory should be assessed separately such as immediate recall, delayed recall, and recognition trials.17

The newly-developed Mini-Mental State Examination, 2nd Edition (MMSE-2) provides 3 versions: Brief Version (MMSE-2: BV), Standard Version (MMSE-2: SV), and Expanded Version (MMSE-2: EV).18 The MMSE-2: EV includes 2 new subtests in addition to the Standard Version, Story Memory (SM), and Processing Speed (Symbol-Digit Coding). These 2 subtests were selected to make the MMSE more sensitive to changes associated with aging and subcortical dementia, and sufficiently difficult so that it would not have a ceiling effect.18 The original MMSE-2: EV includes only an immediate recall (IR) trial of SM. The addition of delayed recall (DR) and recognition trials of SM were required to fully understand the nature of memory impairment. Thus, we added the DR trial and developed a new recognition test to increase its clinical utility as a complete story recall test. This study was conducted; 1) To validate the IR of the Korean version SM in the K-MMSE-2: EV and 2) To validate its newly added DR and recognition test as memory measures.

METHODS

Participants

The participants in this study included 95 healthy elderly individuals (HE), 90 patients with aMCI, and 53 patients with dementia of the Alzheimer's type (DAT). The HE participants were recruited through community outreach. Those who fulfilled Christensen's health screening criteria19 and showed normal performance in the Korean-Mini Mental State Examination (K-MMSE)20 were selected. Patients with aMCI or DAT were selected from those who visited the Department of Neurology at university hospitals. They all underwent clinical diagnostic dementia work-up, including a comprehensive neuropsychological evaluation and brain imaging. Cognitive impairment was defined as at least 1.5 standard deviation (SD) below the norm.21 Petersen's criteria for MCI were used.22 The clinical diagnosis of DAT was based on the National Institute of Neurological and Communicative Disorders and Stroke-Alzheimer's Disease and Related Disorders Association (NINCDS-ADRDA) criteria.23

Test instruments

SM

For developing the Korean version of SM in MMSE-2: EV, two clinical neuropsychologists (YK and MS) translated the original story of the MMSE-2: EV (Blue form) into Korean. The back-translation was performed by a bilingual translator to confirm that the meanings of the original story were well translated.

The administration and scoring method for the IR of SM were identical to the original version. A brief story composed of four sentences was verbally narrated to an examinee, and the examinee was tested for the ability to recall the story immediately after the narration. The immediate free-recall output was assessed by scoring 0 or 1 for 27-word units with a maximum score of 25. In addition to the original IR trial, the examinee was asked to recall the story again in 15 minutes of delay (DR) and the response was scored in the same way as IR.

We developed a recognition test that comprised 15 items to which the examinee could respond with yes or no on each item. The IR of original SM were composed of 27-word units. Repeated contents (e.g., the name of main character) were excluded and 1–3 word units were combined to make the final 15 items that covered all story contents.

Neuropsychological and other measures

The K-MMSE, Seoul Verbal Learning Test-Elderly's version (SVLT-E),8 and Rey Complex Figure Test (RCFT)24 were administered to all the participants. For the aMCI and DAT groups, the Clinical Dementia Rating (CDR)25 and the Global Deterioration Scale (GDS)26 were additionally administered to measure the severity of dementia. The Short form of the Geriatric Depression Scale (SGDS)27 was given to control the depression level for all groups.

Procedures

Non-verbal tasks that did not impact verbal memory were used as filler tests for a 15-minute delay. The filler tests were the Visual Discrimination Test (VDT)28 and Paper Folding Test (PFT).29 The participants were not informed that there will be later DR and Recognition trials at the end of the IR. All participants performed the tests in the same sequences.

Clinical psychology graduate students familiar with psychometric scales, were trained by the authors (YK and MS) and collected the data from the HE. They visited the houses or welfare centers for older adults in the community and then administered the tests and scales. Clinical neuropsychologists working at the Department of Neurology of university hospitals collected the data from the patient groups.

For assessing the test-retest reliability of SM, 51 participants from the HE group were selected and re-administered SM after a 5-week interval. The inter-rater reliability of SM was assessed by two clinical neuropsychologists (MS and SHL) by independently measuring 50 randomly chosen data sets (25 HE and 25 patients).

Statistical analysis

One-way analysis of variance (ANOVA), Pearson's χ2 test, and Student's t-test were used for examining the differences of demographic and other variables among the groups. The significant results of ANOVA were followed by post hoc comparisons using Bonferroni adjustments. The test-retest reliability was assessed with Pearson's correlation coefficient (r). The inter-rater reliability was assessed with the Cohen's weighted Kappa coefficients. Convergent validity was evaluated by calculating the partial correlation coefficients among the IR, DR, and recognition test of the SM, SVLT-E, and RCFT controlled for age, education, and depression level. Multivariate analysis of covariance (MANCOVA) was conducted to evaluate the differences of the SM among the 3 groups (HE, aMCI, and DAT) with age, education, and depression level controlled as covariates.

Ethics statement

The study protocol was reviewed and approved by the Institutional Review Board (IRB) of Hallym University (HIRB-2019-44).

RESULTS

Characteristics of demographic and other variables

The demographic characteristics and K-MMSE, SGDS, CDR, and GDS scores of the participants are shown in Tables 1 and 2. There was no significant group difference in either sex ratio or education level. However, significant group differences were observed in age, general cognitive function (K-MMSE), depression level (SGDS), and severity of dementia (CDR and GDS).

Table 1

Demographic characteristics and the K-MMSE and SGDS scores of the participants

Variables	HE (n=95)	aMCI (n=90)	DAT (n=53)	F or χ2 or t	p value	Post hoc (Bonferroni)
Age (yr)	76.05±6.54	71.39±10.12	75.94±8.05	F=8.52	<0.001	HE=DAT>aMCI
Sex (male/female)	27/68	38/52	21/32	χ2=4.17	0.124	ns
Education (yr)	8.42±4.49	9.94±4.11	8.58±5.25	F=2.97	0.053	ns
K-MMSE	27.15±2.98	25.58±2.83	21.26±3.76	F=61.57	<0.001	HE>aMCI>DAT
SGDS	2.68±3.03	4.87±3.76	4.87±4.02	F=10.80	<0.001	HE<aMCI=DAT

K-MMSE: Korean-Mini Mental State Examination, SGDS: Short form of the Geriatric Depression Scale, HE: healthy elderly individuals, aMCI: amnestic mild cognitive impairment, DAT: dementia of the Alzheimer's type, ns: not significant.

Table 2

CDR and GDS scores of the participants with aMCI or DAT

Variables	aMCI (n=90)		DAT (n=53)		t	p value
	Mean±SD	Median (min–max)	Mean±SD	Median (min–max)
CDR-GS	0.50±0.00	0.50 (0.00–3.50)	0.85±0.23	1.00 (0.50–1.00)	−10.97	<0.001
CDR-SB	1.97±0.97	2.00 (1.00–4.50)	4.68±1.60	4.50 (2.00–9.00)	−11.18	<0.001
GDS	3.14±0.35	3.00 (3.00–4.00)	4.00±0.62	4.00 (3.00–5.00)	−9.20	<0.001

CDR: Clinical Dementia Rating, GDS: Global Deterioration Scale, aMCI: amnestic mild cognitive impairment, DAT: dementia of the Alzheimer's type, SD: standard deviation, CDR-GS: Clinical Dementia Rating-Global Score, CDR-SB: Clinical Dementia Rating-Sum of Boxes.

Reliability

The Pearson's correlation coefficient for test-retest reliability (average interval, 38.94 ± 10.41 days) of the IR, DR, and recognition of SM were 0.54, 0.63, and 0.55, respectively (p<0.001) (Table 3). Cohen's weighted Kappa coefficients for the inter-rater reliability of the IR, DR, and recognition of SM were 0.87, 0.91, and 1.00, respectively.

Table 3

Test-retest reliability of the SM

SM	1st test	2nd test	Pearson's r	p value
Immediate Recall	11.43±5.10	14.78±5.14	0.54	<0.001
Delayed Recall	10.69±5.19	13.67±5.49	0.63	<0.001
Recognition	11.67±1.93	12.33±1.73	0.55	<0.001

SM: Story Memory.

Validity

All of the three SM measures exhibited significant correlations with all the measures in total participants on the SVLT-E (IR: r=0.48, p<0.001; DR: r=0.65, p<0.001; recognition: r=0.43, p<0.001) and RCFT (IR: r=0.32, p<0.001; DR: r=0.56, p<0.001; recognition: r=0.32, p<0.001). In the HE group, 3 SM measures exhibited significant correlations with all the measures of the SVLT-E (IR: r=0.44, p<0.001; DR: r=0.47, p<0.001; recognition: r=0.34, p<0.01), and with DR (r=0.28, p<0.01) and recognition (r=0.24, p<0.05) of the RCFT. In the aMCI group, there were statistically significant correlations between IR (r=0.25, p<0.05) and DR (r=0.45, p<0.001) of the SVLT-E, while it had a significant correlational relationship only with DR (r=0.38, p<0.01) of the RCFT. In the DAT group, SM had a significant correlation only with DRs of the SVLT-E (DR: r=0.64, p<0.001) and the RCFT (DR: r=0.39, p<0.01). (Table 4). The SM was more strongly correlated with the SVLT-E rather than the RCFT.

Table 4

Partial correlations among the SM, SVLT-E, and RCFT

Groups	SM	SVLT-E: IR	SVLT-E: DR	SVLT-E: Recognition	RCFT: IR	RCFT: DR	RCFT: Recognition
Total	IR	0.48‡			0.32‡
	DR		0.65‡			0.56‡
	Recognition			0.43‡			0.32‡
HE	IR	0.44‡			0.18
	DR		0.47‡			0.28†
	Recognition			0.34†			0.24*
aMCI	IR	0.25*			0.14
	DR		0.45‡			0.38‡
	Recognition			0.18			0.01
DAT	IR	0.24			0.06
	DR		0.64‡			0.39†
	Recognition			0.12			0.22

SM: Story Memory, SVLT-E: Seoul Verbal Learning Test-Elderly's version, RCFT: Rey Complex Figure Test, HE: healthy elderly individuals, aMCI: amnestic mild cognitive impairment, DAT: dementia of the Alzheimer's type, IR: immediate recall, DR: delayed recall.

*p<0.05, † p<0.01, ‡ p<0.001.

The MANCOVA revealed significant group differences in the IR (F[2, 232]=18.45, p<0.001), DR (F[2, 232]=41.08, p<0.001), and recognition (F[2, 232]=33.98, p<0.001) of SM (λ=0.66, F[2, 232]=17.67, p<0.001). Follow-up post hoc analysis with Bonferroni adjustments showed that the IR and DR of SM were significantly different among the HE, aMCI, and DAT groups. The recognition scores were not significantly different between the HE and aMCI groups, but exhibited significant difference between the aMCI and DAT groups (Table 5).

Table 5

Group differences of the SM

SM	HE (n=95)	aMCI (n=90)	DAT (n=53)	F	p value	Post hoc (Bonferroni)
IR	7.35±3.89	6.77±3.09	4.02±2.46	18.45	<0.001	HE>aMCI>DAT
DR	6.09±4.34	4.50±3.85	0.91±2.00	41.08	<0.001	HE>aMCI>DAT
Recognition	10.65±2.01	10.23±2.29	7.36±3.01	33.98	<0.001	HE=aMCI>DAT

SM: Story Memory, HE: healthy elderly individuals, aMCI: amnestic mild cognitive impairment, DAT: dementia of the Alzheimer's type, IR: immediate recall, DR: delayed recall.

DISCUSSION

The results revealed a good test-retest reliability of the Korean version of SM in the K-MMSE-2: EV and its full version including DR and recognition test. This indicates that the measures, the original IR as well as newly-adopted DR and recognition of SM, are stable over time. Also, the inter-rater reliability of SM was markedly high. This suggests that the scoring criteria of SM are clear and easy to follow, such that they can produce reliable scores.

The relationships of SM with other established measures of memory were also evaluated. In total participants, the IR, DR and recognition of SM had reasonable relationships with the IR, DR and recognition of SVLT-E and RCFT, respectively. It was not surprising that SM had strong positive relationships with the word list recall test, SVLT-E.30 Story recall and word list recall require rapid processing of a constant stream of verbal information to facilitate encoding and consolidation via mechanisms primarily based on medial temporal lobe.31 SM also showed small to moderate positive relationships with visual memory measures of RCFT, although the strength of correlations was not as high as the word list recall. These results proved good convergent validity of SM as a memory test.

Analysis of each group revealed that the relationships of SM with other established measures of memory observed in the total participants were similar in the HE group. However, these correlational relationships were found only with DR in aMCI and DAT groups. The IR and recognition of SM were not significantly correlated to those of the SVLT-E and RCFT in the clinical groups. This suggests that the DR of SM is a particularly crucial memory measure for patients with cognitive impairment, and that if only the IR of SM in the K-MMSE-2: EV was administered, it is difficult to obtain accurate information on memory.

We examined the discriminant validity of SM by comparing the performance among HE, aMCI, and DAT groups. Significant group differences were revealed in all the measures of SM. The performance of IR and DR were highest in the HE group, moderate in the aMCI group, and lowest in the DAT group. These significant group differences implied that the IR and DR of SM are sensitive in detecting subtle memory impairment at prodromal stages of dementia as well as in differentiating the degree of memory impairment.3233 Rabin et al. found that learning across multiple trials provided the most sensitive index for distinguishing MCI from normal aging, but the inclusion of DR of a story recall test enhanced the overall accuracy of classification.4 Thus, if the full version of SM in K-MMSE-2: EV is used with a list learning test such as SVLT-E, it would provide a better diagnostic tool in clinical settings.

The recognition test of SM showed group differences between IR and DR. The DAT group showed significantly lower performance in the recognition test than the other two groups, while no significant difference was found between the HE and aMCI groups. These patterns are consistent with previous findings that aMCI had a retrieval deficit while DAT had deficits in encoding and storage.3435 As suggested by Batchelder and Riefer,36 to identify the different types of memory deficits, the additional trials of DR and recognition are necessary. Thus, the full version of SM consisting of IR, DR, and recognition test would be a more effective tool for evaluating the degenerative progression of memory deficits from normal aging to aMCI and DAT.

Another advantage of story recall test is that it is relatively easier to administer and obtain responses from the examinee compared to the word list recall test. At least three repetitive trials of IR should be applied to evaluate the learning process using word list recall. Some words in the list may not be familiar to the examinee, especially the elderly, and could cause them to be un-interested and bored. However, the story memory test consists of a single trial for IR. The story consists of simple sentences describing the events that can occur in real-life situations. Thus, the examinee may feel more interested in it than remembering a list of unrelated words.

In conclusion, the reliability and validity of the newly-developed SM in the K-MMSE-2 and its full version as a memory test were confirmed. Also, since the full version involves DR and recognition, it can provide more information about the nature of memory impairment than the original SM in MMSE-2: EV that consists of only IR. If the normative data for IR, DR, and recognition of SM are published in the near future, it is expected that the full version of SM will be used more widely for assessing memory impairment in the community and clinical fields.