Literacy Independent Cognitive Assessment: Assessing Mild Cognitive Impairment in Older Adults with Low Literacy Skills.

OBJECTIVE: Comprehensive neuropsychological tests are important in the diagnosis and follow-up of patients with MCI; however, most were developed without consideration of illiteracy. We developed the Literacy Independent Cognitive Assessment (LICA) as a comprehensive neuropsychological assessment battery applicable to older adults who are either literate or illiterate. This study aimed to assess the reliability and validity of the LICA for diagnosis of MCI.
METHODS: Normal controls (n=634) and patients with MCI (n=128) were recruited from 13 centers were included in this study. Participants were divided into illiterate or literate groups, based on their performance on a brief reading and writing test. The LICA, Korean Mini-Mental State Examination (K-MMSE), and Seoul Neuropsychological Screening Battery (SNSB) were administered.
RESULTS: Total LICA scores distinguished MCI patients from controls (p<0.001). They were closely and positively correlated to the K-MMSE scores (r=0.632, p<0.001) but negatively correlated to clinical dementia rating (CDR) (r=-0.358, p<0.001) and CDR sum of boxes (r=-0.339, p<0.001). Area under the receiver operating characteristic curve for patients with MCI by total LICA score was 0.827 (0.783-0.870), superior to that presented by the K-MMSE. For the classification of MCI subtypes, inter-method reliability of LICA with the SNSB was good (κ 0.773; 0.679-0.867, p<0.001).
CONCLUSION: The results of this study show that the LICA may be reliably used to distinguish MCI patients from cognitively intact adults, to identify MCI subtypes and monitor progression toward dementia, regardless of illiteracy.

INTRODUCTION

Mild cognitive impairment (MCI) refers to a clinical transitional state between normal cognitive aging and the earliest pathological features of dementia.1 Patients with MCI may progress to dementia due to several diseases such as Alzheimer's disease (AD), vascular dementia (VaD), dementia with Lewy body (DLB), and frontotemporal lobar degeneration (FTLD).23 As an evolving concept, MCI is heterogeneous in clinical characteristics, etiology, and prognosis.1 MCI is diagnostically classified as amnestic MCI (aMCI) or nonamnestic MCI (naMCI), and further sub-classified as affecting one or multiple cognitive domains. It is known that the aMCI-multiple domain type tends to convert to AD, and naMCI tends to convert to VaD or FTLD.45 However, some patients continue to have MCI without further cognitive deterioration.1 Due to the heterogeneous nature and prognosis of MCI, neuropsychological evaluation in the diagnosis and follow-up of patients with MCI is important.67 Patients with severe memory impairment tend to show more rapid cognitive decline than those with less memory impairment.8 Similarly, a recent study indicated that patients with the aMCI-multiple domain type actually had poorer survival and more rapid progression than patients with the aMCI-single domain type;5 such a finding can be predicted by detailed neuropsychological testing. In the future, drugs may be developed to effectively modulate progression from MCI to AD910 and comprehensive neuropsychological tests may be used more frequently to monitor the effects of interventions on MCI.

However, comprehensive neuropsychological assessments may not be available to a vast number of patients. They are time-consuming and patients who are illiterate may have difficulties completing comprehensive neuropsychological tests due to poor reading and writing skills. Globally, one out of five people is illiterate, and one out of four people is illiterate in developing countries, indicating that seven hundred million people in the world are illiterate.11 In South Korea, the illiteracy percentage was about 1.7% in 2008.12 Due to the Korean War, the percentage increased to 70% for older adults aged above 70 years old, in particular.12 Most of the currently used neuropsychological tests were developed without considering illiteracy. Thus, when these tests are applied to older adults who are illiterate, unfamiliarity with the test content may make these older adults appear to have low-level cognition, resulting in diagnosis of dementia.13141516 As of today, commonly used neuropsychological assessments such as the Alzheimer's Disease Assessment Scale-Cognitive subscale (ADAS-Cog) and the Consortium to Establish a Registry for Alzheimer's Disease (CERAD) include many items available to those who can write and read, and thus it is questionable whether such assessments are valid and applicable for individuals who are illiterate and undereducated. To address this problem, the Literacy Independent Cognitive Assessment (LICA) was developed. The LICA was previously reported to be a valid and reliable instrument for screening dementia among older adults who are literate or illiterate.17 The battery may also be able to distinguish MCI from normal aging in adults who are illiterate. This study assessed the validity of the LICA for distinguishing MCI from normal aging, as well as the optimal cut-off scores for diagnosis of MCI among older adults with minimum literacy skills.

METHODS

Participants

From May to December 2008 a total of 762 older adults aged 60 years old and older were recruited from six university hospitals and seven senior welfare centers. The sample included 634 normal control participants [literate group (n=106), illiterate group (n=528)] and 128 patients with MCI [literate group (n=39), illiterate group (n=94)]. Patients with MCI were recruited consecutively among outpatients from the memory clinics of university hospitals. The normal controls were spouses of patients or were recruited from the senior welfare centers. Table 1 provides demographic characteristics for participants. The Institutional Review Board approved the study protocol. Participants in the literate group provided written informed consent to participate in the study, and participants in the illiterate group provided verbal consent.

Table 1

Comparisons between control and MCI

Values are presented as mean±standard deviation and numbers. Statistical analysis was performed by Student t-test and χ2 test, adjusted for age and illiteracy. MCI: mild cognitive impairment, K-MMSE: Korean Mini-Mental Status Examination, CDR: Clinical Dementia Rating, CDRSOB: Clinical Dementia Rating, sum of boxes, LICA: Literacy Independent Cognitive Assessment

Participants were included in the normal control group if they denied having any memory or cognitive complaints, neurological disease, or psychiatric disease. The controls were aged ≥60 years, did not have any of 28 diseases or a history suggestive of a decrease in cognitive function (stroke or transient ischemic attack, seizures, Parkinson's disease, multiple sclerosis, cerebral palsy, Huntington's disease, encephalitis, meningitis, brain surgery, vascular surgery of the brain, diabetes requiring insulin control, improperly managed hypertension, cancer diagnosed within the previous 3 years excluding skin cancer, shortness of breath while sitting still, use of oxygen at home, heart attack with changes in memory, walking, or solving problems lasting at least 24 hours afterwards, kidney dialysis, liver disease, hospitalization for mental or emotional problems in the previous 5 years, current use of medications for mental or emotional problems, alcohol consumption greater than 3 drinks daily, drug abuse in the previous 5 years, treatment for alcohol abuse in the previous 5 years, unconsciousness for more than 1 hour other than during surgery, overnight hospitalization due to head injury, illness causing a permanent decrease in memory or other mental functions, trouble with vision that prevents reading ordinary print even with glasses, or difficulty understanding conversations because of hearing even with a hearing aid).18 The controls also had scores that were at least one standard deviation above the mean scores of the respective age- and education-matched population from the Korean Mini-Mental State Examination (K-MMSE)19 and had an average score of 0.42 or less from the Korean Instrumental Activities of Daily Living (K-IADL).20

In this study, the diagnosis of MCI was provided following a clinical evaluation by a trained neurologist or psychiatrist. MCI was defined in consideration of the criteria set by the International Working Group on MCI.21 First, persons were judged as not normal besides not fulfilling diagnostic criteria for dementia. Secondly, functional activities of the person were mainly preserved, or at least that impairment is minimal. Furthermore, the person had evidence of cognitive decline, measured either by self and/or informant report in conjunction with deficits on objective cognitive tasks, and/or evidence of decline over time on objective neuropsychological tests. To fall in the MCI group, participants needed to report a cognitive complaint that was preferably corroborated by an informant. Despite intact general cognitive functioning (K-MMSE ≥ mean -2SD for age and education level), participants in the MCI group had evidence of cognitive impairment (below 1.0 SD relative to age- and education-matched healthy individuals), as demonstrated by their performance on more comprehensive cognitive tests. Although the cognitive tests administered were not consistent across centers, the Seoul Neuropsy-chological Screening Battery (SNSB)22 or CERAD were usually performed for the diagnosis of MCI. Among the MCI participants, only 65 participants with MCI completed all the sub-tests of SNSB. This subset of participants was further divided into aMCI (n=50) and naMCI (n=15) groups, based on their performance on the SNSB. Twelve participants with MCI who completed the full subtests of CERAD were also divided into 3 aMCI and 9 naMCI, based on their CERAD performance.

All participants with MCI had preserved to mild impairment in instrumental activities of daily living. Participants were excluded from the group if there were obvious medical, neurological, or psychiatric explanations for the memory loss. In addition, participants underwent brain MRI or CT, and routine biochemical and serological tests that assessed thyroid functioning, serum vitamin B12 and folate levels, and presence of venereal diseases, were conducted.

Measures

Literacy screen

Participants were divided into illiterate or literate groups, by a brief reading and writing test. The participants were asked to read aloud the following two sentences, "Young-Hee was thirsty due to physical exercise. She opened a refrigerator." They were then asked to write about what would happen next. A participant who was able to read the sentences and write an appropriate response was judged to be literate.

Literacy Independent Cognitive Assessment

The final version of the LICA is a 300-point test, consisting of 13 subtests assessing memory, visuospatial ability, executive function, attention, language, and calculation.17 The story recall task involves immediate, delayed recall after approximately 20 minutes, and recognition. The word recall task involves three learning trials of ten nouns, an immediate recall trial, a 20-minute delayed recall trial, and recognition. Visuospatial ability and memory are assessed using copy construction and recognition of 10 pictures of stick formations that were modified from the Stick Pattern Reversal Test.23 Executive functions are assessed using the Digit Stroop test and the animal fluency task. Attention, concentration, and working memory are evaluated using forward and backward repetition tapping tasks of nine Corsi blocks. The order of the nine blocks on the board and the tapping sequences were modified from Corsi's block-tapping task.24 Language is assessed using a fifteen-item confrontation naming task with animals, fruits, and vegetables and the aforementioned fluency task. The Color and Object Recognition Test (CORT) was modified from semantic knowledge tests on the visual form and color of objects.25 Finally, calculation is evaluated.

Seoul Neuropsychological Screening Battery

The SNSB assesses episodic memory, visuospatial function, language, calculation, attention, and frontal executive function. Subtests include the Seoul Verbal Learning Test (12-word list), the Short Form of the Korean-Boston Naming Test (black and white line drawings), Rey-Osterrieth Complex Figure, digit span forward and backward tests, semantic and phonemic verbal fluency, and the Color-Word Stroop test.

Statistical analysis

The SPSS 13.0 package was used for data analysis, and p values below 0.05 were considered statistically significant. First, to compare differences in age, gender, education level, K-MMSE, clinical dementia rating (CDR), total LICA score, and LICA subtest scores between controls and patients with MCI, Student t-tests and chi-square (χ2) tests were used. This analysis was also repeated to detect differences between literate and illiterate participant groups. Analyses were adjusted for age, education level, and literacy skill level.

Second, Pearson's correlations were used to evaluate the LICA's concurrent validity with other measures such as K-MMSE and CDR.

Third, receiver operating characteristic (ROC) analysis was used to assess the ability of the LICA to differentiate participants with MCI from normal participants, when compared with the K-MMSE. By using SPSS' random sampling software, controls and patients with MCI were matched 2:1 by age, sex, and education level. Larger area under curves (AUCs) indicated improved diagnostic accuracy. Sensitivity and specificity of LICA total score using optimal cut-off scores indicated by the ROC curve were assessed. This analysis was also repeated for both literate and illiterate participant groups.

Fourth, comparisons of the LICA total score and subtest scores among three groups (controls, aMCI, and naMCI) were performed with analysis of covariance (ANCOVA) and the Sheffe post hoc test. Participants were classified into four MCI subtypes, aMCI-single, aMCI-multiple, naMCI-single, and naMCI-multiple, by their performance on the SNSB and also by the norms from the validation study of the LICA.17

Lastly, by using Cohen's kappa coefficient, inter-method reliability of the LICA with the SNSB was assessed among the 50 aMCI patients who were diagnosed with the SNSB. ROC analysis was used to compare the AUR, sensitivity, and specificity between the LICA total score and the K-MMSE for diagnosis of aMCI.

RESULTS

Descriptive characteristics of participant groups

The demographic characteristics of the participants are shown in Table 1. The distributions of age, gender, and education years were not significantly different between subjects with MCI and normal controls. However, the proportion of illiteracy was significantly higher in patients with MCI (26.56%) than in normal controls (16.72%).

LICA score comparisons between participant groups

A series of Student's t-tests revealed that the LICA total and subtest scores significantly differed between the control and MCI groups, as well as between literate and illiterate groups, with the exception of two subtests, forward visuospatial span and CORT. Within the illiterate group, the mean forward visuospatial span score was not significantly different between controls and MCI patients. In addition, the mean CORT score was not significantly different between the illiterate control group and the illiterate MCI group. However, the LICA total and subtest scores were significantly different between literate and illiterate MCI groups. Table 1 and 2 present LICA score means, standard deviations, and Student t-test findings across study groups.

Table 2

Comparisons between control and MCI according to illiteracy

Values are presented as mean±standard deviation and numbers. Statistical analysis was performed by Student t-test and χ2 test, adjusted for age and education years. MCI: mild cognitive impairment, K-MMSE: Korean Mini-mental Status Examination, CDR: Clinical Dementia Rating, CDRSOB: Clinical Dementia Rating, sum of boxes, LICA: Literacy Independent Cognitive Assessment, CORT: Color and Object Recognition Test

LICA associations with the K-MMSE and the CDR

Correlations of the LICA with other validation measures (K-MMSE, CDR, and CDR sum of boxes) were moderate to strong. The LICA total scores were found to have a strong positive association with the K-MMSE scores [r(760)=0.632, p< 0.001] and negative moderate associations with CDR [r(760)= -0.358, p<0.001] and the CDR sum of boxes [r(760)=-0.339, p<0.001].

Sensitivity and specificity of the LICA in MCI, compared to the K-MMSE

The ROC curves were drawn for all patients with MCI versus the 256 age-, gender-, and education-matched controls to determine the discriminatory validity of the total LICA score (Figure 1A). The area under the ROC curve (AUR) for the LICA total score within the MCI group was 0.827 (0.783-0.870). At the cut-off value of 202/203, the sensitivity was 76.0% and the specificity was 72.7%. In the illiteracy group (Figure 1B), the AURs for the MCI patients by the total LICA was 0.835 (0.752-0.917). At the cut-off value of 187/188, the sensitivity was 76.0% and the specificity was 70.3%. In the literacy group (Figure 1C), the AURs for the MCI patients by the total LICA was 0.824 (0.773-0.875). At the cut-off value of 209/210, the sensitivity was 75.5% and the specificity was 71.4%. These results show that the LICA may differentiate these groups well each other; on the contrary, the AUR values for the K-MMSE were 0.698 (0.644-0.753), 0.624 (0.508-0.740), and 0.765 (0.703-0.828), within the whole sample, illiterate group, and literate group, respectively.

Figure 1

The receiver operating characteristic (ROC) curves of the literacy independent cognitive assessment (LICA), compared to the Korean version of Mini-Mental State Examination (K-MMSE), used to make a diagnosis of mild cognitive impairment (MCI) in the whole (A), illiterate (B), and literate (C) participants. The areas under the ROC curve (AURs) for all MCI patients (A) by the total LICA was 0.827 (0.783-0.870). Also, the AURs by the total LICA were 0.835 (0.752-0.917) in the illiteracy group (B) and 0.824 (0.773-0.875) in the literacy group (C). On the contrary, the AUR values for the K-MMSE were 0.698 (0.644-0.753) in the whole participants, 0.624 (0.508-0.740) in the illiteracy participants, and 0.765 (0.703-0.828) in the literacy participants, respectively.

Classification into MCI subtypes

Based on their SNSB scores, 50 patients were classified into the aMCI group and 15 patients were classified into the naMCI group. Between the control, aMCI, and naMCI groups, the LICA total score and all LICA subtest scores, with the exception of the CORT and stick construction subtest scores, were significantly different. Table 3 presents means and standard deviations for LICA total and subtest scores, by control, aMCI, and naMCI groups. Performance on story recall (immediate and delayed), story recognition, word recall (immediate and delayed), and word recognition was lowest in the aMCI group. Performance on visuospatial span backward and animal fluency was lowest in the naMCI group.

Table 3

Comparisons among three groups, control, aMCI and naMCI

Values are presented as mean±standard deviation and numbers. Statistical analysis was performed by analysis of variances and Scheffe method for the multiple comparisons. *significant differences after multiple comparisons by Scheffe method, compared to control, †significant differences after multiple comparisons by Scheffe method, compared to aMCI. aMCI: amnestic type of mild cognitive impairment, naMCI: non-amnestic type of mild cognitive impairment, K-MMSE: Korean Mini-Mental Status Examination, CDR: Clinical Dementia Rating, CDRSOB: Clinical Dementia Rating, sum of boxes, LICA: Literacy Independent Cognitive Assessment, CORT: Color and Object Recognition Test

The 65 MCI patients who were classified into aMCI and naMCI subtypes by their performance on the SNSB, were further classified into aMCI-single, aMCI-multiple, naMCI-single, and naMCI-multiple groups, with the LICA normative data. Inter-method reliability of the LICA with the SNSB, by using Cohen's kappa coefficient, was good (κ=0.773; 0.679-0.867, p<0.001).

For the 50 aMCI patients and the 100 age-, gender-, and education-matched controls, the AURs for the MCI patients by the total LICA was 0.808 (0.739-0.877). At the cut-off value of 209/210, the sensitivity was 71.0% and the specificity was 68.0%. The AUR value for the K-MMSE was 0.771 (0.691-0.851).

DISCUSSION

In this study, the total LICA score and most of the LICA subtests successfully differentiated participants of the MCI group from the control group. The test's ability to differentiate groups was also present after adjusting for age and education level. Moreover, when the groups were subdivided into literate and illiterate groups, differences between controls and participants with MCI were significant. The LICA was moderately to strongly correlated to concurrent measures, including the K-MMSE and the CDR. This study presents LICA cut-off total scores for patients with MCI, literate patients with MCI, and illiterate patients with MCI, respectively. The LICA also had considerably high inter-method reliability when compared to the SNSB for classifying participants into MCI subtypes.

The authors developed the LICA and reported its validation results elsewhere.17 The LICA was found to be a valid and reliable instrument for detection of dementia in older adults with and without literacy difficulties.17 It performed well in discriminating participants across CDR stages in both literacy groups.17 These previous results, in addition to the current findings, indicate that the LICA may be an effective tool for monitoring disease progression. The LICA appears to be a psychometrically robust tool for assessing cognitive functions in older adults with difficulties with literacy, as the inter-rater and the test-retest reliabilities were considerably high.

In addition to the timely diagnosis of dementia, early detection of prodromal phase, or MCI, is becoming more important for early interventions such as disease-modifying therapies.26 MCI is heterogeneous; some patients with MCI may progress to AD, while others may remain stable or develop other dementing disorders such as VaD, FTLB, and DLB. Therefore, comprehensive neuropsychological assessments are essential for identifying specific cognitive impairments in patients with MCI and tracking cognitive declines indicative of dementia. However, many neuropsychological tests were not developed with consideration of illiteracy, or specifically early detection of MCI among older adults who are illiterate. Because individuals who are illiterate might have difficulty completing complex neuropsychological tests that include reading and writing, they could be considered to have cognitive impairments indicative of dementia, in spite of having intact cognitive functioning. The LICA was useful for identifying cognitive impairment as well as screening MCI, among individuals with minimal literacy skills.

Low education is known to be one of the risk factors of dementia.27 People with higher education tend to have greater cognitive reserve, and as such, greater pathological changes are needed for dementia symptoms to manifest.28 Illiteracy has also been associated with a higher risk of AD after adjustment for education level.29 However, there have been few clinical studies focused on individuals who are illiterate, and such adults are usually excluded from dementia drug trials.30 We suggest that future clinical trials include participants who are illiterate by using the LICA.

There are some limitations in generalizing the results of this study. The study's relatively small sample sizes for patients with MCI may not provide enough statistical power to detect all significant group differences. Larger samples of such patients may be needed to validate statistical differences between groups in this study. In addition, the diagnosis of MCI was performed respectively by the individual centers. Although the K-MMSE, the CDR and the K-IADL were commonly used in the individual centers, the clinical cognitive tests were not the same across all centers. Some centers used only parts of the SNSB, while others used alternative tests, such as the CERAD. Only 65 MCI patients completed the full SNSB. One of the other limitations is that the same diagnostic process was applied to participants in both illiterate and literate groups. Some of the neuropsychological tests, such as the SNSB and CERAD, might not have been applicable to administer to the illiterate participants. Therefore, cognitively intact participants from the illiterate group might have been misclassified into the MCI group. In addition, it is postulated that participants in the illiterate group tended to have lower socioeconomic status than participants in the literate group; however, LICA scores were not adjusted for socioeconomic status.

In conclusion, the LICA was developed with considering the effect of illiteracy on cognitive performance. The LICA was considered a valid and reliable instrument for older adults with and without substantial literacy skills. The current study findings indicate that the LICA is a useful comprehensive neuropsychological battery to distinguish the MCI patients from the normal controls, differentiate MCI subtypes, and monitor progression to dementia, regardless of illiteracy.