Cognitive Screening Instruments for Older Adults with Low Educational and Literacy Levels: A Systematic Review.

This study presents a systematic review on existing cognitive screening tools for mild cognitive impairment and dementia in populations with low education and literacy levels. Cochrane Library, PubMed and LILACS databases were examined for studies including adults aged 50 years old or older with low educational level. 61 articles were included. Despite its frequent use, studies on Mini-Mental State Examination (MMSE) revealed that educational level biased the score obtained, regardless of other factors. Separately, the Informant Questionnaire on Cognitive Decline in the Elderly, the Fototest, or the Eurotest, appear to minimize the effect of education and literacy. MMSE is unreliable for individuals with low literacy. Tasks involving reading, writing, arithmetics, drawing, praxis, visuospatial, and visuoconstructive skills have a greater educational bias than naming, orientation, or memory. An adequate determination of educational level and validation of instruments in populations with heterogeneous levels of literacy requires further research.

Introduction

Major Neurocognitive Disorder (also known as dementia) is a decline in mental ability severe enough to interfere with independence and daily life (American Psychiatric Association, 2013). To meet diagnostic criteria, there must be a substantial cognitive decline from a previous level of higher performance in one or more cognitive domains (attention, executive function, learning, memory, language, visuoperceptive and visuoconstructive functions, and social cognition), and must interfere with the individual’s independence (López-Álvarez & Agüera-Ortiz, 2015). Mild Cognitive Impairment (MCI), separately, is an intermediate state between normal cognition and dementia, more specifically, an early stage of loss of memory or other cognitive abilities (such as language or visual/spatial perception) in individuals who otherwise show minimal impairment of most instrumental activities of daily living (IADL) (Petersen et al., 2018).

Appropriate diagnosis of MCI and dementia is important in order to assess for reversible causes of cognitive impairment, to help patients and families understand the cause of their cognitive concerns, and to discuss prognostic possibilities (Petersen et al., 2018). This diagnosis is especially difficult for individuals raised in environments where low education and literacy are more prevalent, for two reasons: (1) difficulties for accessing healthcare resources, (2) an unacceptably high rate of false positive diagnoses among low-educated individuals, derived from cognitive tests with high verbal and educational demands that lack adequate sensitivity (Klekociuk et al., 2014), as it will be described further below.

In order to intervene as early as possible at the initial stages of decline, it is necessary to have instruments for the detection of both MCI and major neurocognitive disorder. Brief cognitive screening tests (i.e., administered in less than 20 minutes—Olazaran et al., 2016) are often the initial step in the evaluation of patients with suspected cognitive impairment, and can be used by both specialists and Primary Care professionals, but are not intended to confirm diagnosis by themselves (which would require a combination of tools, included a further in-depth neuropsychological assessment). These brief tests may lead to erroneous clinical interpretations, as many rely on a high demand of reading and writing abilities (Rosetti et al., 1990). Subsequently, high-educated individuals with actual cognitive decline may perform well in these tasks (as they match their premorbid skills) and thus mask their decline (false negatives), while healthy individuals with low educational background may perform poorly (due to reduced comprehension of the instructions or items/questions with a high literacy demand) and be mistakenly identified as cases of suspected cognitive decline (false positives). This “educational bias” would imply that a considerable proportion of the population falling in a range of a lower education level would be excluded from most of the instruments currently available. Ostrosky-Solis et al. (1998) and Manly et al. (1999) showed that performance of non-demented subjects (of a wide age range and compared by educational level) was directly affected by literacy in tasks such as naming, comprehension, verbal abstraction, orientation, figure matching, and recognition; and that schooling represented a stronger variable than age.

Some of the most widely used brief cognitive screening tests on a routine basis, according to an early survey of the International Psychogeriatrics Association (Shulman et al., 2006) are the 30 points standard Mini-Mental State Examination (MMSE; 77.1% of the times) and Clock Drawing Test (CDT; 45.8%). The Montreal Cognitive Assessment (MoCA) (Nasreddine et al., 2005) appeared later as an alternative to the MMSE, with comparable performance on detection of mild cognitive impairment (Tsoi et al., 2015). However, it was originally validated with highly educated healthy controls (13.3 ± 3.3 years of education) (Davis et al., 2015) and the adjustment of 1 point for those with fewer years of education is highly questionable. Moreover, tests like the MoCA and the MMSE share a problematic underestimation of the importance of considering demographic factors such as education given the high failure rate on certain items in the general population (Rosetti et al., 1990).

Some studies have investigated the adequacy of currently available cognitive screening instruments in samples with low educational levels, with controversial results. Gagnon et al. (2013) noted that adjusting the MoCA total score by lowering the cut-off point for education had a detrimental effect on sensitivity with only a slight increase in specificity. Moreover, adjusting the cut-off points of the screening tests by educational level assumes that patients’ premorbid intellectual function can be represented by merely counting years of formal education.

Some attempts have been preliminarily identified in the literature as potential solutions for individuals with low literacy levels. First, the MoCA-Basic (Julayanont et al., 2015), which assesses similar cognitive domains as the original MoCA, could facilitate the detection of mild cognitive impairment in illiterate and lower educated subjects (reportedly without any age, education, or literacy effect) and with a 81% sensitivity, 86% specificity, and 84% overall accuracy when screening for MCI at the optimal cut-off score (<25/30), although much more research with this test is required. Second, Mini-COG (Borson et al., 2000), whose application time is about 3 minutes, has no significant influence from language, culture or literacy, but the range of cognitive functions assessed is limited and far for providing an overall picture of global cognitive function in a screening context (Diaz-Orueta et al., 2018).

Following all the previous considerations on the need to determine appropriate assessment tasks for individuals with low educational level, the goals of this work are (1) to perform a review of the literature on cognitive evaluation tools commonly used when screening for MCI and dementia in older populations with low educational levels, (2) to define which tasks within these tests present greater difficulties for these populations, and (3) to determine which cognitive tasks are most suitable for individuals with low literacy.

Methods

We performed a systematic review on articles published in English or Spanish, comprising primary studies, with a cross-sectional or prospective design, whose study population were adults with an age equal to or older than 50 years old with low educational level (tentatively defined as 6 years or less of formal education, or a group average of a maximum of 8 schooling years), whose goal was the evaluation of the use of cognitive screening tools for the detection of MCI or dementia.

Other systematic reviews and meta-analyses, studies that excluded illiterate individuals, or that selected a sample with a concurrent pathology (either psychiatric, neurological—cerebrovascular disease, and neurodegenerative disorders—or systemic) were excluded.

Searching Strategy

Studies were identified by searching electronic databases (Cochrane Library, PubMed and LILACS). The article search and screening process was carried out between February 25th and April 22nd, 2020. Table 1 describes the syntax used for the search across different databases.

Table 1.

Description of the search strategy in the three electronic databases consulted: The Cochrane Library, PubMed, and LILACS (in Spanish).

The Cochrane Library

(Aged [MeSH] OR Elderly OR Seniors) AND (Dementia [MeSH] OR Cognitive Dysfunction [MeSH]) AND (Mental Status and Dementia Tests [MeSH] OR Screening) AND (Literacy [MeSH] OR Illiteracy) AND (Educational Status [MeSH] OR Low Educational level OR Low Education)

PubMed

(((((“Aged"[Mesh]) OR “aged, 80 and over”[Mesh]) AND “Literacy”[Mesh]) OR “Educational Status”[Mesh]) AND “Dementia”[Mesh]) OR “Cognitive Dysfunction”[Mesh]

LILACS

(Edad avanzada OR Ancianos) AND (Bajo nivel educativo OR Analfabet*) AND (Demencia OR Deterioro cognitivo)

Abstract Screening Strategy

Due to the large number of articles obtained according to the previously described search parameters (85,507 results in the preliminary search, 67,978 in The Cochrane Library, 17,485 in PubMed, and 44 in LILACS), the Cochrane Library and PubMed articles were sorted by best matching the terms of the research, limiting the abstract screening to a more manageable number in both cases (16,000 in the first one, 3000 in the second one). Article screening was independently performed by two researchers, and conflicts were solved by consensus between both of them.

Results

Figure 1 represents the flow diagram, following the PRISMA model for the article selection process.

Figure 1.

PRISMA Flow Diagram for the identification and selection of screening instruments for early detection of MCI and dementia in elder subjects with low educational level.

Sixty-one articles were finally included, compiled in Supplementary Table 1, belonging to the three databases: The Cochrane Library, PubMed and LILACS. There was a majority of cross-sectional studies (93.4%).

The sample sizes ranged between 40 and 39,451 participants, and were carried out in 22 countries, mostly (42.6%) in Brazil, China, and Spain.

Across the studies, a wide range of cognitive assessment tools (n = 64) were used to detect MCI and dementia, with a higher frequency for the MMSE on its different versions (57.3%) and the Clock Drawing Test (11.47%).

The samples of the selected studies were stratified according to the educational level of the participants, although the classification method varies. We aimed to contact authors of the selected studies via the corresponding author email or platforms like ResearchGate, with a positive response rate of 30% (18 out 60, plus 1 set of authors unavailable), in order to collect further information about the means used to collect information about the educational background of their respective samples. Supplementary Table 2 represents an exemplar of the vast heterogeneity of estimations of educational level across those studies from which further information could be gathered, showing the challenges to reach a common metric or scale. Some distinguish between illiterate and literate; others describe average years of education of each group; and others subdivide them by the number of years of schooling, or educational attainment (primary, secondary, and higher education…).

In terms of cognitive assessment tools, 21 of them showed a higher accuracy for the detection of suspected cognitive decline in older adults with low levels of formal education, with no reported statistically significant differences based on the educational level of participants. These tests are listed in Table 2.

Table 2.

Screening tools included in this review that showed higher accuracy for the detection of suspected cognitive decline in older adults with low levels of formal education.

Authors	Tests (Single or Combined) Showing Higher Accuracy for the Detection of Suspected Cognitive Decline in low literacy Groups
Pérez-Leguizamón et al. (2017)	Modified Community screening instrument for Dementia (CSID)
Fuh et al. (1995), Phung et al. (2015)	Informant Questionnaire on Cognitive Decline in the Elderly (IQCODE)
Nielsen et al. (2016)	IQCODE combined with RUDAS
De Paula et al. (2017)	IQCODE combined with MMSE
Brito-Marques and Cabral-Filho (2005)	Modified MMSE (3MS)
Sales et al. (2011)	Severe MMSE (SMMSE)
Paddick et al. (2017)	Alzheimer’s disease assessment scale—cognitive (ADAS-Cog)
Maillet et al. (2017)	Test of associative memory (TMA)-93
De Paula et al. (2017)	Five digit test
Carnero-Pardo and Montoro-Ríos (2004); Carnero-Pardo et al. (2011)	Eurotest
Carnero-Pardo et al. (2011); Zegarra-Valdivia, Solis & Chino-Vilca (2019)	Fototest
Carnero-Pardo et al. (2011)	Memory alteration test (M@T)
Appollonio et al. (2005)	Bedford Alzheimer Nursing Severity Scale (BANS-S) and the modified test for severe impairment (mTSI)
Takada et al., 2006)	Delayed recall in the brief cognitive screening Battery (DR-BCSB)
Chan et al. (2006)	C-Executive interview (EXIT)-25
Iavarone et al. (2007)	Activities of daily life (ADL)
Iavarone et al. (2007)	Instrumental activities of daily life (IADL)
Caldas et al. (2012)	Leganés cognitive test
Calero et al. (2002)	Boston naming test in its reduced version of 15 items (BNT-15)
Phung et al. (2014)	10/66 Dementia research group (DRG)
De Paula et al. (2013)	Combined battery that includes Rey auditory verbal learning test, frontal evaluation, phonemic and semantic fluency, Stick design test, Clock drawing test, Digits test, Token test, Naming and activities of daily living

In addition, there are two tests that turned out to have acceptable diagnostic accuracy for screening MCI and dementia in samples with low educational level, but that present a “ceiling effect” in groups with higher education. The first one is the Alzheimer’s Disease (AD)-8, a test based on the interview with the caregiver about the degree of autonomy of the patients, which reportedly overestimates abilities of the subjects with a higher educational level (Chen et al., 2018). The second one is the Abbreviated Mental Test (AMT) (Sahadevan et al., 2000), which includes orientation, calculus, deferred memory, and naming tasks, and reportedly more suitable for patients with less than 6 years of formal education, but too simple for those with more education years.

On the other hand, 25 studies include the MMSE in its different versions (standard 30-item and modified, extended, reduced, and/or culture-specific versions), sometimes with the goal of validating the test, while others try to define its usefulness and detect possible biases in low educational level population. Some studies combine the use of MMSE with the IQCODE questionnaire using the statistical method “weighted sum,” thus mitigating the effect of educational level.

Results reflect that educational level correlates directly and proportionally with the MMSE score, regardless of the age of the individuals, thus proposing different cut-off points for subjects with low literacy in order to obtain acceptable levels of sensitivity and specificity (ranging in a wide range of values, from 14 points for dementia and 15 for MCI for illiterate subjects, and 17/18 when they present at least 1 year of schooling in a Brazilian population studied by Scazufca et al. (2009), to 24/25 points for people with studies lower than primary education and 26/27 for primary and higher education in a Dutch population in the Schmand et al. (1995)).

Among the tasks for which individuals with low literacy have greater difficulties, those involving reading and writing, calculation and drawing are prominent. In contrast, memory (both immediate and delayed), three-steps command, and object naming are the items that show less impact from educational level. Raina et al. (2015) focus on the copy of the intersection of pentagons, and although the percentage of illiterate subjects who try to carry out the task is greater than the literate (56.11 vs. 42.08%), the success in the task is only 4.12% in the group without formal education, while in the literate sample it is 13.46%.

Thus, in the modified version of the MMSE (3MS) (Brito-Marques & Cabral-Filho, 2005) (designed to sample a broader variety of cognitive functions, cover a wider range of difficulty levels, and enhance the reliability and the validity of the scores), drawing of the intersection of pentagons is replaced by two triangles, and instead of subtracting seven from 100, it starts from 25 and it is subtracted one by one; and in the Severe version (SMMSE) (Sales et al., 2011) the visuospatial function is rated with the copy of a square, and the executive function is rated with the command of drawing a circle, while the calculation task is removed. These variations (among others) allow both tests to be suitable for both illiterate and low-educated individuals.

Discussion

Education and cognitive test performance are closely related, and subsequently it is essential to distinguish between the loss of skills and functions that dementia entails, and the normal aging of a person who never acquired these skills. One difficulty is that the level of literacy is significantly associated with almost all neuropsychological measures, although the relationship between education and test scores will depend on the specific test administered and the content of the tasks (Ardila et al., 2010). Literacy influences verbal and visual memory, phonological knowledge, visuospatial and visuomotor skills, and functional neuroimaging studies confirm that literacy modifies neural networks, such as those used for problem solving (Dehaene et al., 2015). This review confirms that illiterate or low educational-level patients present specific difficulties in tasks involving reading, writing, calculating, drawing, praxis (i.e., the ability to conceptualize, plan, and organize movements in order to complete unfamiliar motor tasks), visuospatial, and visuoconstructive skills, while showing a better performance in naming, orientation, and memory.

Remarkably, despite many studies focused on its limitations when assessing the cognitive status of older adults with low educational and literacy levels, the MMSE continues to be the most widely used test both in Primary Care and by professionals working with dementia patients. Among the MMSE tasks, only three of its 30 points assess memory, and four of them are dependent on literacy: reading and obeying the command “close your eyes,” writing a sentence, copying the figure of the pentagons and the calculation task, making it inappropriate for illiterate subjects.

As potential alternatives showing advantages for Primary Care settings, there are many instruments showing high accuracy in the detection of suspected cognitive decline in older adults with low levels of formal education. Those tests are the Eurotest, the M@T (in which most of the items evaluate memory), and the Fototest, which examines memory together with naming and verbal fluency. The TMA-93 comprises naming semantically paired everyday objects and memory tasks (free recall and cued recall based on semantic pairs) (Maillet et al., 2017). The Leganés Cognitive Test, in addition to naming and memory, explores temporal, spatial, and personal orientation (Caldas et al., 2012). They all focus on tasks that explore memory domain in its different variants, an area on which educational level seems to have a lower influence. The Five Digit Test (FDT), a modification of the classic Stroop Test, aims to minimize the bias of literacy by using numbers and quantities limited between one and five (De Paula et al., 2017). Moreover, questionnaires such as the IQCODE, the BANS-S, or ADL/IADL (on the basic and instrumental activities of daily life), in which the information is obtained through an informant, effectively avoid the educational bias. Use of informants is envisaged as a possible solution to the problem of estimating cognitive decline, as informants have knowledge of both the subject’s current and premorbid behavior, and may inform based on observations of performance in everyday life rather than on artificial cognitive tasks (Jorm & Jacomb, 1989)

On the other end, instruments like the AMT and AD8, with a “ceiling effect” and thus little sensitivity for populations with higher educational level (i.e., six or more years of education), may be suitable for subjects with lower education level.

Therefore, and taking these results into account, a brief cognitive screening test proposal for individuals with low literacy could comprise (1) the Leganés Cognitive Test, with the orientation items, naming (Rosselli et al., 1990, described how the educational bias is minimized if, in the naming tasks, real objects are used instead of drawings or photographs) and immediate, delayed and logical memory (a short story is told with six key ideas that the patient must remember), (2) a verbal fluency task such as in the Fototest (all names of the same sex, and then of the opposite sex, that the patient is capable to verbalize in 1 minute), and (3) a brief questionnaire to the closest relative or caregiver, to determine the current level of autonomy of the patient.

Among the limitations of this work, it should be noted that there is no consensus on the definition of low educational level, which partially explains the heterogeneity of results for the cognitive tools evaluated. Efforts to collect information from authors of the identified studies was only partially successful, and evidences the challenge ahead to achieve a common metric and methodology for educational background that facilitates cross-cultural studies. It is likely that a more comprehensive in-depth interview on educational background could better articulate the differences between low education, literacy, and cultural features. Also, the ways in which individuals with low educational levels differ from individuals with higher levels of education (i.e., potential higher exposure to worse environmental conditions and higher risks to health, lower access to healthcare, poorer nutrition, etc.) (Mortimer et al. 2007) have not been considered here as confounding variables that may affect cognitive performance. Furthermore, the included studies applied different diagnostic criteria for dementia, which also prevents comparability of results. However, it is noteworthy that, for the study identification, no instrument was excluded, since those administered to both patients and informants were considered with the aim to gather as much information as possible.

In summary, alternatives are required to the MMSE for the cognitive screening of individuals with low educational attainment, which do not involve a high demand of reading and writing skills when exploring cognitive abilities. Tasks involving reading, writing, calculating, drawing, praxis, visuospatial, and visuoconstructive skills imply particular difficulties for these individuals; while naming, orientation, and memory are domains less affected by educational level. However, there are still scarce validation studies of cognitive assessment tools in older adults with low educational level, making it an area that requires further attention from the research community.

Click here for additional data file.

Supplemental Material, sj-pdf-1-jag-10.1177_07334648211056230 for Cognitive Screening Instruments for Older Adults with Low Educational and Literacy Levels: A Systematic Review by Isabel Pellicer-Espinosa and Unai Díaz-Orueta in Journal of Applied Gerontology

Click here for additional data file.

Supplemental Material, sj-pdf-2-jag-10.1177_07334648211056230 for Cognitive Screening Instruments for Older Adults with Low Educational and Literacy Levels: A Systematic Review by Isabel Pellicer-Espinosa and Unai Díaz-Orueta in Journal of Applied Gerontology