Concurrent Validity of the Cognitive Assessment of Minnesota in Older Adults with and without Depressive Symptoms.

Cognitive impairment represents a common mental health problem in community-dwelling and institutionalized older adults, and the prevalence increases with age. Multidisciplinary teams are often asked to assess cognitive and functional impairment in this population. The Cognitive Assessment of Minnesota was created by occupational therapists for this purpose and is frequently used, but has not been extensively validated. This study examined the performance of the CAM and compared it to the MMSE with 113 outpatient clinic patients over the age of 60. Subgroups were established based on scores on a depression inventory to determine if the presence of depressed mood altered the relationship between the measures. Both measures demonstrated good internal consistency. The overall correlation between the two measures was high, statistically significant and remained high regardless of depression status. We offer recommendations about the utility of each measure in screening cognitive functioning for older adults.

1. Introduction

Cognitive impairment represents a common mental health problem in community dwelling and institutionalized older adults, and the prevalence increases with age [1]. Several measures have been developed to assess cognitive functioning in this population with some measures having a greater emphasis on memory and language functioning in analog conditions and others focusing more on functional adaptive skill use. Occupational therapists often work within multi-disciplinary settings and are frequently asked to assess the functional and cognitive status of their patients. The Cognitive Assessment of Minnesota (CAM) [2] was created for this purpose. The CAM is a standardized assessment of cognitive functioning developed by occupational therapists. The advantage of using a measure like the CAM is that it covers a greater range of cognitive impairment compared to most cognitive screens, making it potentially more useful as a screening measure. In addition to providing a global measure of cognition, the CAM also assesses practical skill domains. For example, the CAM allows for assessments of working memory and simple mathematical ability (e.g., can the person make change) that would allow for specific recommendations (e.g., providing assistance to the affected person in managing finances). The CAM also allows for hierarchical grading of cognitive skills, which may be useful in monitoring change over time. Thus, the CAM may be a promising measure to be used in geriatric settings. Although the CAM is frequently used internationally by OTs in practice and in training settings [3-5], the psychometric properties of the CAM have not been extensively validated.

One of the most common screening measures for cognitive impairment is the Mini-Mental State Exam (MMSE) [6], a scale developed in neurology to be used as a screener in a number of settings, including medicine, neurology, and psychiatry. The MMSE has proven to be a useful tool because its simplicity allows providers from any discipline to screen for cognitive impairments.

Depression and cognitive impairment are commonly co-occurring disorders in late life. The National Comorbidity Study Replication (NCS-R) reports lifetime prevalence rates of major depression for community dwelling adults over the age of 60 to be as high as 10.6% and prevalence rates for dysthymia at 1.3% [7]. It is likely that this is a conservative figure as older adults with depression often present with less sadness and report experiencing more physical or cognitive symptoms, and thus the depressive symptoms may go unrecognized [8]. Depression can complicate differential diagnosis as it can exacerbate or mimic symptoms of cognitive impairment. Research has revealed depression is linked to impaired cognition in the areas of short-term memory [9], attention [10-12], information processing [10], recognition [9], and executive functioning [10, 12]. However, Monsch et al. have suggested that skills such as psychomotor speed that decline with aging may be responsible for more of the variance in the decline in scores on cognitive measures, rather than the depression itself [13]. Given that cognitive screening measures have varying demands on different types of cognitive abilities and that older adults may differ from younger adults on the extent that depression affects each area of cognitive functioning, the validity of screening measures for use among older adults with depression may be reduced. However, despite the widespread use of cognitive screening measures among individuals with depression, few studies have compared the psychometric properties of these instruments for use with older adults with depression. If the presence of depression negatively impacts cognitive assessment measures, one would expect to see impaired scores across similar cognitive screening measures. However, if the cognitive measures are differentially affected, then perhaps some other factor may be responsible for the different levels of agreement between those measures. For example, it may be that individuals with depression show variability in response over time due to the effects of fatigue, effortful processing, a perception of task difficulty, and so forth. Clinically, this would be important to take into account before choosing a screening measure to use with individuals with depression. The purposes of this study were to (1) investigate the relationship between two measures of cognitive status (the MMSE and the CAM), (2) examine if depressive symptoms moderate the relationship between these measures, and (3) examine group differences on areas similarly measured in both screening instruments.

2. Materials and Methods

2.1. Participants and Setting

Participants included 113 adults (25 men and 88 women) over the age of 60 (M = 81.00, SD = 8.57, range 61–95), predominantly European American, with 43.8% of the sample having an education level of 12 or more years. Eligible participants were those who completed an evaluation at the Geriatric Assessment Center (GAC) which is affiliated with three university systems in the Midwest. The GAC is a collaborative multi-disciplinary assessment team consisting of a nurse, occupational therapist, pharmacist, geriatrician (physician), psychologist, and a social worker. Typical reasons that individuals would contact the GAC include: self-referral for concern over memory loss or depression; concern for partner's worsening memory, depression/isolation, anxiety, or overmedication; and functional capacity issues (e.g., adult child concern for the capacity for parent(s) to live independently/manage finances/maintain driving, mobility concerns). A comprehensive evaluation by this team includes a 2-3 hour home visit and a 4-hour clinic visit.

Because some sections of the MMSE (e.g., following a written command, writing a sentence, copying a design) and the CAM (e.g., subsections requiring the examinee to look at a flash card and perform the arithmetic problem) are not appropriate for participants with sensory impairment, they would not be given to individuals who are visually impaired or illiterate, and because a section might be refused by a participant, some participants were dropped from the final analyses, resulting in a sample of 103 older adults (23 men and 80 women).

2.2. Assessment Instruments and Administration Procedures

Three measures were administered to each participant as part of a standard comprehensive assessment: The Mini-Mental State Exam (MMSE) [6], the Cognitive Assessment of Minnesota (CAM) [2], and the Geriatric depression Screen, short form (GDS-SF) [14, 15].

The MMSE is a 19-item assessment of cognitive status commonly used in psychiatric settings [16]. The MMSE usually takes approximately 10–15 minutes to administer and has a maximum score of 30. The examination includes measures of orientation, recall and recent memory, abstract thinking, attention and calculation, and object identification. Research on this measure indicates good reliability with interrater correlations of r = .82 and test-retest reliability ranging from r = .89 to r = .98 [6, 17]. The MMSE has been extensively researched and validated. Validity studies on this measure vary depending on the population measured and the cut score utilized. For the commonly used cut score of ≤24, research using the MMSE and comparing it to dementia status as a criterion indicates sensitivity rates of <85% and specificity rates of <98% in community samples when compared to dementia status as diagnosed by trained neuropsychological examiners [18, 19], 89% sensitivity rates with corresponding specificity rates of 81% in general practice samples [20], pooled sensitivity and specificity rates of 71% and 95.6% in hospitalized patients [21], and 84.9% and 60.8% in memory clinics [22]. For this study, a member of the multi-disciplinary team who was trained in its administration and scoring administered the MMSE in the home at intake (Day 1).

The CAM is a 29-item standardized assessment of cognitive functioning developed primarily for use with individuals with brain injury, but also has data on a nonaffected sample. This assessment usually takes 35–45 minutes to administer. The administration time takes substantially longer than a typical screen to administer (approximately 45 min compared to 10–15 min), yet it is more manageable in integrated settings than a complete neuropsychological battery which could take a minimum of 6 hours to complete. The CAM provides information on a range of cognitive skills including: attention span (distractibility), orientation, recent and remote memory, basic language (register, process, and respond to information), mathematical and practical (monetary) skills. It also assesses temporal awareness, visual memory and sequencing, auditory memory and sequencing, discrimination skills (object identification and matching), mental flexibility, and safety and judgment. The maximum total score that can be obtained is 80. Very little data exits on the psychometric properties of the CAM outside from what is found within the examiner's manual. Rustad and DeGroot indicate an interrater alpha of 0.90, adequate test-retest reliability, and good concurrent validity for level of impairment when compared against professional clinical judgment by occupational therapists [23]. This measure was administered on Day 2 in the clinic by an occupational therapist and an occupational therapy student trained in CAM administration and scoring.

The GDS-SF is a 15-item depression inventory specifically designed for use with older adults [15]. High scores indicate greater severity of depressive symptoms. This depression inventory has a maximum score of 15, with scores higher than five considered to be indicative of depression. The items assess overall mood, changes in energy and activity level, and attitudes about the future. The GDS-SF, also called the GDS-15, was found to be as good as the longer version of the GDS for assessing depression among older adults with dementia [24]. This measure was administered on Day 1 in the home setting by one member of a three to four person team trained in administration and scoring. The team consisted of an occupational therapist, nurse, pharmacist, or psychologist and students from one of those disciplines (i.e., 1-2 professionals and 2 students per team).

2.3. Analyses

Pearson Product Moment Correlations were calculated for total scores of the two cognitive measures (CAM and MMSE) to determine the overall level of agreement of these screening measures for a group of older adults.

Subgroups were established based on the presence or absence of depressive symptoms to determine whether the presence of depressed mood, as measured by the GDS-SF, differentially altered the relationship between the two measures. t-tests were conducted comparing participants with depressed mood to participants without depressed mood using the scores on the GDS-SF. The nondepressive symptom group (NON) consisted of those with GDS scores of 0–4. The depressive symptom group (DEP) consisted of GDS scores of 5–15. The Pearson Product Moment correlation described above was computed for each group and coefficients were then compared using Fisher's Z transformation and examined for significant group differences.

Sub analyses were conducted to examine areas similarly measured in both screening instruments. The two cognitive measures were examined for items that sampled the same basic skills (i.e., recall and orientation) (refer to Table 1). The strength of the relationship between these subscales for both measures was examined using Pearson Correlation Coefficients and a Fisher's Z was then conducted to analyze for significant differences. Then, the two groups identified earlier as DEP and NON were compared for the association of depressive symptomology through examinations of these two groups on each of these subscales using the same statistical procedures.

Table 1

Specific items used for sub analyses.

MMSE	CAM
Orientation	Orientation
What is the date?	What is today's date, including month, date, and year?
What is the month?
What is the year?
What day is it?	Is it morning, afternoon, or evening right now?
What season are we in?	What season are we in now?
What state are we in?	Where are you now?
What city?
What county?
What is the name of this place?	What happened to bring you into the hospital?
What floor are we on?
Recall	Recall
A while ago I asked you about 3 items, do you recall what they were?	Tell me the three words I told you earlier?
Apple, penny, table	Plant, clock, rug

Additionally, a reliability coefficient was run on each of the two cognitive measures to provide an index of each measure's reliability in this population. A Cronbach's alpha was conducted on both the total scores of each measure and on the orientation subsection to provide an index of internal consistency. It was not possible to conduct a Cronbach's alpha on the recall subsection due to the fact that both measures only have one score/subscore available for the recall section. According to convention, α of .70 or higher is considered to be acceptable/good within the social sciences.

3. Results

A Cronbach's alpha was calculated for total scores of the cognitive measures and for the orientation subsection. The MMSE was found to have good internal consistency α = .79 for total score and .82 for orientation. The CAM also performed well, yielding an α = .87 for the total score and α = .76 for orientation.

Descriptive statistics for both measures are depicted in Table 2. Pearson Product Moment Correlation was calculated for total scores for 103 participants on the two cognitive measures with r = .74, P < .01, indicating a strong correlation. The depressive symptom group (DEP) consisted of 36 older adults (11 men and 25 women; Mage = 76.9, SD = 9.6, and 52.2% with 12+ years of education). The nondepressive symptom (NON) group consisted of 67 older adults (12 men and 55 women, Mage = 83.1, SD = 6.7, and 45.6% with 12+ years of education). Analyses on the two measures among each subgroup (i.e., DEP and NON) yielded an r = .72, P < .01 for NON and r = .79, P < .01 for DEP. A Fisher Z was then conducted, indicating no significant group difference (z = 0.67, P = .50). When comparing DEP to NON groups, an independent samples t-test indicated no significant group differences on sex, t = 1.47, P = .145, but did indicate a significant group difference on age, t = 3.62, P < .001. Therefore, partial regression correlation coefficients, holding age constant, were calculated for each subanalysis and are described below. Secondary analyses indicated the effects of depressed mood on the overall relation yielded an r = .72, P < .01 for NON and r = .79, P < .01 for DEP. A Fisher Z was then conducted, indicating no significant group difference (z = 0.67, P = .50).

Table 2

Descriptive statistics for total and subgroup CAM and MMSE scores.

Assessment	N	M	SD
MMSE
Totala	103	22.5	5.9
DEPb	36	23.0	5.2
NONc	67	22.2	6.2
CAM
Totala	103	19.9	2.7
DEPb	36	20.4	2.3
NONc	67	19.6	2.8

Note.. aGDS scores for all participants. bGDS scores of 5–15 only. cGDS scores of 0–4 only.

3.1. Subanalyses

Analyses were conducted to examine the agreement between the two cognitive measures on specific subsections for orientation and recall (see Table 3). For orientation, the correlation for all participants was r = .65, P < .01, following standard conventions, a correlation in this range suggests that the two measures have a moderate agreement on this subtest. When comparing DEP and NON groups using partial correlation coefficients (holding age constant), a Fisher Z indicated no significant group differences, z = −1.08, P = .28. For recall, the correlation for all participants yielded an r = .67, P < .01, suggesting that the two measures have moderate agreement on this subtest, as well. When comparing DEP and NON partial correlation coefficients (holding age constant), a Fisher Z indicated no significant group differences, z = −1.42, P = .16.

Table 3

Correlations between groups and subgroups of CAM and MMSE scores

Group	r	P	pr	P
Totala
CAM versus MMSE	.741	.01
Recall	.666	.01
Orientation	.649	.01
DEPb
CAM versus MMSE	.786	.01	.724d	.01
Recall	.462	.01	.462d	.07
Orientation	.801	.01	.802d	.01
NONc
CAM versus MMSE	.724	.01	.746d	.01
Recall	.755	.01	.732d	.07
Orientation	.583	.01	.570d	.01

Note. aGDS scores of all participants. bGDS scores of 6–15 only. cGDS scores of 0–4 only. dSignifies the use of partial regression correlation coefficient where age is held constant.

4. Discussion

Cognitive impairment is a common problem in older adults and has been linked to several negative outcomes including poor treatment adherence [25-28] and increased disability, suggesting a need for early recognition in medical and mental health treatment. Multi- disciplinary teams are often asked to provide more specific and detailed answers and recommendations than can be obtained through brief cognitive screening measures. For example, functional recommendations are often sought by rehabilitation professionals (can Ms. Smith resume daily activities?), social security disabilities services (e.g., can Mr. Jones perform work-related activities?), and by families seeking help with planning (e.g., is my aging parent able to manage his/her finances independently?). Cognitive screening measures like the MMSE, though frequently used, are primarily gross cognitive screening tools and may have several limitations when utilized for purposes beyond detecting global cognitive impairment (e.g., MMSE has poor utility for use with post-stroke medical patients [29]).

The CAM is a promising measure for use in geriatric populations because of the wide range of abilities assessed and its utility with brain-injured populations. Previously only one small study has examined the psychometric properties of the CAM using the MMSE as a comparison. Rustad and colleagues found a moderate correlation (r = .44, P = .05), however the sample size reported in this study was small (n = 16), and additional examinations are needed to determine whether this relationship will exist in other samples [23]. This study examined the utility of the CAM as a screening measure and examined its relationship to the MMSE in a relatively large sample of older adults in an outpatient clinic setting.

Both measures evidenced adequate/good reliability when evaluated using a Cronbach's alpha. When examining the orientation subsection alone, reliability estimates remain adequate/good. The overall relation suggests good agreement between the two measures of cognitive status on orientation. The data suggest that when depressive symptoms are taken into consideration, overall scores were not affected; however there was a tendency for depression to affect cognitive functions such as recall and orientation. For example people in the DEP group scored lower on the recall subsection on the MMSE than they did on the CAM. Interestingly they DEP group scored higher on the orientation subsection on the MMSE than they did on the CAM. These findings were consistent across the NON group, as well. Higher scores on recall on the CAM could be affected by the difference of the type of words used in recall, however both measures use commonly found objects. Attention could clearly be affected by the fact that the CAM takes longer than the MMSE to administer, and this could in part account for differences in scores on orientation. Alternatively, the differences in orientation may be an artifact of the sample, since there is no evidence in the research literature to suggest that differences in orientation should exist between cognitive screens. Replications of this study using larger or more diverse samples may elucidate these findings.

Unfortunately, neither measure has an alternate form available for use with those individuals with visual impairment or those with illiteracy. These same challenges exist for the majority of cognitive screens and many neuropsychological test instruments that are currently available (i.e., many rely heavily on visual and verbal abilities; e.g., Boston Naming Test) constituting a problem for the field of neuropsychology as a whole. In a recent survey of neuropsychologists in the USA and Canada, Rabin and colleagues reported on the most common assessment used. Of the 20 neuropsychological assessment instruments used, 80% require adequate visual skills [30]. The authors are not aware of any published cognitive screening instruments that have been designed specifically for individuals with visual impairment or illiteracy.

While this study is the first large sample study to examine the CAM and the MMSE, the drawback to utilizing clinical data collected from an assessment center is that we cannot control for ethnic diversity and the clinicians may not have always been able to administer the assessments on the days planned. Patient cancellations and rescheduling of appointments may have led to a longer delay in administration (e.g., one week in between administration of assessment measures rather than one day). Since participants with vision or reading disabilities were excluded from the study, results may only generalize to healthy and literate older adults. In addition, because the sample was predominantly European-American, we cannot assume generalization to more diverse samples. In addition, because these data are cross-sectional, we cannot assume that the relationship between the two measures will remain stable over time, particularly in the presence of cognitive impairment. Future research should include follow up data to assess whether the utility of these measures changes over time and with advanced impairment. In addition, in order to investigate the prognostic utility of the CAM, longitudinal studies are required to evaluate its ability to predict both mild cognitive impairment and dementia as compared to assessment by a trained neuropsychologist.

5. Conclusions

This study finds the two measures to have good reliability and agreement in assessing cognitive impairment, but that each measure has a distinct advantage. The MMSE offers a shorter administration time, which may be beneficial in primary care settings where time constraints preclude a longer assessment for cognitive impairment but detailed function recommendations may not be needed. Whereas the direct clinical usefulness of some of the information from specific tasks of the CAM test (i.e., the problem solving task that involves completing multiple errands in a day—assesses executive functions that include planning, organization, and sequencing ability) along with the opportunity to obtain a larger sample of behavior in all tasks assessed, are some of the strengths of the CAM. From an evidence-based perspective, the decision to select an assessment tool should be tied to the goal or purpose of the assessment [31]. If the purpose is only to obtain a quick simple screen (obtain a brief snapshot of the person's cognition to evaluate for risk of cognitive impairment or to establish a baseline), then a clinician could choose to use the MMSE. If the purpose is to make clinically useful recommendations (e.g., focus on functional adaptive skill use), the longer assessment will be more helpful. The CAM is useful for geriatric teams where the goal is to obtain a more comprehensive measure of cognitive impairment. Given that the CAM allows for assessment of more practical aspects of cognitive impairment and offers more extensive assessment information than the MMSE, including hierarchical grading of cognitive skills, the CAM may be more ideal for settings such as home health care or rehabilitation settings where the goal would be to have more information about practical skills and activities of daily living. However, the longer duration of testing may increase issues associated with fatigue during assessment. Thus, examiners would need to incorporate breaks as necessary or administer over sessions to minimize test fatigue.

Therefore, our finding that the two measures are so closely related means that clinicians can adjust or tailor their screening based on the information they need (goal driven assessment). In addition, having strongly correlated measure means that those who work mainly with the MMSE can look at the CAM and have a good idea about the individual's cognitive capacity and those who work with the CAM can look at the MMSE and have a good idea about the individual's cognitive capacity (i.e., aids in communication within the scientific community). As Gerontology is an interdisciplinary field, it is often useful to have tools that would aid in communication about a patient between disciplines.

In sum, the CAM appears to be a promising measure for use with older adults with cognitive impairment and when examiners are interested in functional capacity recommendations. More research is necessary exploring the psychometric properties of the CAM to explore its full utility with diverse populations. In addition, studies comparing the CAM to other measures of functional capacity may be warranted.