Screening for Cognitive Impairment among Community-Dwelling Older Adults: A Comparison of 2 Screening Instruments.

INTRODUCTION: Community aged care services provide support to older adults living in their own homes. Cognitive impairment may increase the complexity of the support required. There is a need to ensure suitable brief screening tools are available to community aged care providers to assess possible cognitive impairment. This study aimed to examine the agreement between 2 validated cognitive impairment screening tools, the Mini-Cog, and Abbreviated Mental Test Score (AMTS), and the perceptions the individuals case manager of Case Manager's.
METHODS: A cross-sectional survey study was undertaken with clients of a community aged care provider. Clients were administered both the screening tools via an electronic survey by their Case Manager.
RESULTS: In total, 158 (54%) eligible participants consented to participate. There was a 70% agreement between the Mini-Cog and AMTS measures, indicating a moderate agreement which was not statistically different from chance (Kappa 0.08, 95% CI -0.04-0.19). Case Managers identified 37% (n = 48/130) of participants as possibly having cognitive impairment, of which, 15% (n = 20) were also identified via a screening tool.
CONCLUSIONS: The findings indicate poor agreement across the 3 measures. To ensure adequate supports are offered to those with cognitive impairment, the use of validated tools that can be administered by non-medical staff in a community setting is a priority. This study highlights a need for further work to determine the most suitable tool for use by community-based aged care services.

Introduction

Internationally, considerable resources have been invested in supporting older adults to retain their independence for as long as possible. Community aged care services provide a range of services to older adults to support them to continue living in their homes. These supports can include the provision of health services, home assistance, social activities, and disability support. Typically, the provision of such support is guided by tailored case management plans developed following the administration of person-centered assessments.

Older adults with cognitive impairment (CI) are at greater risk for malnutrition and dehydration, falls, medication issues, and incontinence, which are in turn associated with increased hospitalization rates. A diagnosis of CI, including dementia, may therefore increase the complexity and level of support required for an older persons’ care. It is critical that CI is identified as early as possible so that can be appropriately modified, and the required services can be provided.

Some community aged care delivery organizations may rely on their healthcare workers incidentally noticing signs like memory loss or confusion to identify possible CI in their clients. However, previous research has demonstrated high rates of undiagnosed CI when standardized screening tools are not used. For example, meta-analytic data indicates that, compared to diagnostic tools, unassisted general practitioner recognition rates for identifying people with dementia was reasonable (73.4%), however this decreases for CI (62.8%) and further if CI is mild (44.7%). There is a need for easily administered screening tools to assess possible CI among community-dwelling older adults to ensure appropriate follow-up care.

Several brief tools have been validated in screening for CI and/or dementia. Two such tools include the Mini-Cog test and the Abbreviated Mental Test Score (AMTS). Both of these tools may be useful as an initial screener of CI for community aged care services as they only take 3 to 4 min to deliver, can be administered after brief training, and do not require clinical expertise. While both tools are commonly used within primary and specialist care settings and have sound psychometric properties, they assess different cognitive domains which may impact screening outcomes. For instance, the AMTS assesses orientation, recall, and concentration, while the Mini-Cog examines memory/recent learning, visual-spatial components, global attention, long-term memory, and executive functions. As initial screening of CI will likely impact which clients receive further assessment with appropriate follow-up care, it is important to ensure there is agreement across different CI measures that may be used in community aged care settings.

This study examined, among community dwelling older people, the: (i) agreement in identifying possible CI between the Mini-Cog and AMTS; and (ii) the perceptions of CI by CM’s.

Method

Design and Setting

A cross-sectional study was undertaken with clients of a not-for-profit Australian government funded aged care provider delivering services to over 8000 older adults across rural, regional, and remote Australia.

CM Recruitment

Eligible CMs provided home visits to clients in 3 Australian states (New South Wales, Tasmania, or Queensland). CMs were identified via staff lists and invited via email to participate in a 1-h study protocol training session. Group training sessions were provided via videoconference or face-to-face by a member of the research team (BH).

Client Recruitment

Eligible clients: resided in the community; were receiving home care services from the participating aged care provider; considered by their CM to be physically and cognitively capable of providing informed consent; and able to complete an English language survey. Clients were excluded if they were too sick to participate, recently bereaved, or on a waiting list for a place in a residential aged care facility. A list of potentially eligible clients was generated by the aged care provider. Of those deemed eligible by their CM, a random computer generator was used to select 400 clients to participate in the study. Randomly selected clients were mailed an information statement and consent form from the aged care provider. Clients were then followed up by their CM with a telephone call or discussion at the next scheduled appointment. Written informed consent was obtained. CMs provided consenting clients with the survey via computer tablet or pen-and-paper and provided assistance when needed. The measures of cognitive impairment were administered by the CMs after participants completed the self-report sections of the survey.

Measures of Cognitive Impairment

Mini-Cog

The Mini-cog contains 2 components, a clock drawing test, and a 3-item recall test. Meta-analytic data indicated a sensitivity of 91% and specificity of 86% for classifying possible dementia using a cut-score of less than 3.

AMTS

The AMTS is a 10-item measure for rapid assessment of possible dementia and other CIs. Meta-analysis of 3 studies and more than 3000 hospital inpatients demonstrated a cut score of less than 7 to have a sensitivity of 81% and specificity of 84% for a possible diagnosis of dementia.

Cognitive impairment identified by CMs

At the conclusion of the patient survey, CMs were asked: “In your opinion how would you rate the level of cognitive impairment of this client?.” Response options were: no CI; low level of CI; moderate level of CI; or severe level of CI.

Demographic variables

Clients self-reported their age, gender, education, Aboriginal, or Torres Strait Islander status, marital status, who they lived with and whether they had private health insurance. CMs self-reported their gender, highest level of education and time working in aged care.

Statistical analysis

The sample used in this study was restricted to non-missing cases of ATMS and Mini-cog score. Participant demographics were summarized with numbers and percentages of non-missing observations. The Mini-Cog score was the sum of 2 components (word recall task [0-3 points] and clock drawing task [0 or 2 points]), with a maximum possible score of 5. The clock drawing task was scored as follows: Yes, no errors (2 points); No, minor spacing errors (0 points); No, other errors (0 points). The AMTS was scored 0 for an incorrect answer and 1 for a correct answer, with a maximum possible score of 10. A Mini-Cog score of 0 to 2; an ATMS score of 0 to 6; and an indication of low, moderate or severe CI by Care Managers were considered to have “possible CI.” Those who did not meet these criteria were considered to have “no CI.” Agreement between Mini-Cog and AMTS score was assessed with percent agreement and Cohen’s Kappa with 95% confidence intervals. The study sample size with a moderate agreement of 0.6 would allow for an estimated precision of +/− 0.16. Statistical analyses were programmed using R (R Foundation for Statistical Computing, Vienna, Austria).

Ethics

Ethics approval was granted by the University of Newcastle Human Research Ethics Committee (H-2017-0356).

Results

CM Sample

A total of 30 CMs were invited to participate, all of whom consented. The majority of CMs were female (n = 27, 90%), did not have a university level education (54%) and had worked in aged care for an average of 13.1 years (±11.8).

Client Sample

Of 400 clients randomly selected to participate, 357 clients were approached to participate by CMs. In total, 295 (83%) were eligible and of these 158 (54%) consented to participate. A total of 146 participants with complete data for both the AMTS and Mini-Cog were included in analysis. Participant demographics can be found in Table 1.

Table 1.

Client Participant Demographics (N = 146).

	Category	N (%)
Age	Mean (SD)	78.4 (8.5)
Gender	Male	53 (36.3)
	Female	92 (63.0)
	Other	1 (0.7)
Education	High school or below	103 (71.0)
	Trade or vocational education	33 (22.8)
	University or postgraduate degree	9 (6.2)
Aboriginal or Torres Strait Islander	No	140 (95.9)
	Yes	6 (4.1)
Marital status	Married or living with partner	56 (38.4)
	Divorced or separated	20 (13.7)
	Widowed	59 (40.4)
	Never married	11 (7.5)
Lives with	Spouse/partner/child(ren)	55 (37.7)
	Other family members	11 (7.5)
	On my own	80 (54.8)
Private health insurance	No	90 (62.1)
	Yes	55 (37.9)

Agreement between the AMTS and Mini-Cog

Possible CI was indicated for 5% (n = 7) of the sample using the AMTS, and 31% (n = 45) of the sample using the Mini-Cog. When comparing CI classification across the tools, 67% (n = 98) of the sample were classified as having no CI by both tools, while only 3% (n = 4) of participants were classed as possibly having CI by both tools (Table 2). About 28% (n = 41) were identified as possible CI by the Mini-Cog only and 3% (n = 4) were identified as possible CI by the AMTS only. Overall, there was a 70% agreement between these measures (ie, 67% classified as no CI and 3% classified as possible CI by both the Mini-Cog and AMTS), indicating moderate agreement, though this was not statistically different from chance (Kappa 0.08, 95% CI −0.04-0.19). There was no clear cut-point that increased agreement between the tools.

Table 2.

Agreement Rates across the Mini-Cog and the Abbreviated Mental Test Score (AMTS).

Mini-Cog	AMTS		Total
	No CI	Possible CI
No CI	98 (67%)*	3 (2%)	101 (69%)
Possible CI	41 (28%)	4 (3%)*	45 (31%)
Total	139 (95%)	7 (5%)	146 (100%)

Abbreviation: CI, cognitive impairment.

Overall agreement rate derived from these cells (70%).

CM Agreement with the AMTS and Mini-Cog

CM perceived CI data was missing for 16 participants. In total, CMs identified 37% (n = 48/130) of participants as possibly having CI. CMs identified possible CI among 51% (n = 22) of participants who were also identified as possible cases by the Mini-Cog, AMTS, or both (n = 43/130). When examining those with complete data across the 3 measures (ie, Mini-Cog, AMTS, and CM; N = 130), 36% (n = 47) of clients screened positive for possible CI on 1 measure, 15% (n = 20) screened positive on 2 measures and only 1.5% (n = 2) screened positive on all 3 measures.

Discussion

To the authors’ knowledge, this study is the first to examine agreement of 2 validated tools, the Mini-Cog and the AMTS, as well as CM identification of suspected CI among a community-dwelling sample of older adults. The findings indicate poor agreement across the 3 examined measures.

The Mini-Cog identified a greater number of individuals with possible CI compared to the AMTS. The lower rate of possible CI identified through the AMTS in this study may have occurred due to those with milder impairment not being detected, whereas the Mini-Cog may have been more sensitive in capturing this group. The Mini-Cog has been found to have greater sensitivity for detecting mild CI when compared to the Mini-Mental State Exam, while the AMTS has been associated with “ceiling effects” in identifying mild CI when compared to the Montreal Cognitive Assessment. This may be due to the different domains of cognition measured by each tool.[4,6]

CMs identified almost half of those clients who were also identified as having possible CI by the 2 standardized measures. This finding aligns with previous research that has found attempts to detect CI without screening tools results in under-identification, even among medically trained professionals. CM unassisted identification is, therefore, unlikely to be a reliable method for identifying of suspected CI.

Limitations

The lack of the inclusion a gold standard measure to compare the findings of these tools limit the interpretation of the study results. The research is also limited by a relatively small sample size, which may impact the generalizability of the findings. Requesting CMs to administer the screening tools before providing their perceptions of CI may have influenced their responses. However, CMs were not trained in scoring the administered tools, nor was this data available to them. Further, given the low rates of agreement between CMs perceptions and the standardized tools, it is not likely that their perceptions were influenced. This study did not examine inter-rater reliability between CMs to determine accuracy of the tool’s administration.

Clinical Implications

The study findings suggest that CMs, even after observing individuals undertake cognitive testing, are not well-positioned to identify possible CI. Further, given the lack of concordance between the 2 standardized measures, index of suspicion from either measure should be followed up by a diagnostic process with a skilled provider. Clear information should be provided to those with possible CI indicating that screening tests results are approximations only and further testing is required for sound clinical judgment.

Future Research

Further examination of standardized measures for CI could include comparison with a gold standard measure to determine the sensitivity and specificity of these measures within community-dwelling older adults. There is a need for continuing development and adaption of tools to ensure reliable and valid methods are available for measuring CI that can be administered by non-medical health care staff.

Conclusion

CI can impact older people’s day-to-day functioning, quality of life and can develop into more severe impairment or dementia. Identifying CI in a timely manner is important for community aged care services to ensure adequate supports are offered. The use of validated tools that can be administered by non-medical staff in a community setting is therefore a priority. The findings from this study highlight a lack of congruence in outcomes in 2 standardized tools for CI screening when administered to community-dwelling older adults. Given the findings of this study, further work is needed to determine the most suitable tool for use by community aged care services.