Ecological, convergent, and discriminative validities of the cognitive abilities screening instrument in people with dementia.

BACKGROUND: The Cognitive Abilities Screening Instrument (CASI) assesses global cognitive function in people with dementia with 9 domains (i.e., long-term memory, short-term memory, concentration, orientation, attention, abstraction and judgment, language abilities, visual construction, and category fluency). However, the ecological, convergent, and discriminant validities of the CASI have not yet been examined.
PURPOSE: This study designed to investigate these 3 validities of the CASI in people with dementia.
METHODS: Fifty-eight participants underwent assessments with the CASI, 3 functional measures, and 3 cognitive measures. Pearson's r was used to estimate correlations among the CASI and 3 functional measures for examining ecological validity. We computed correlations (r) among the CASI and 3 functional measures for examining convergent validity. An independent t-test was applied to compare the levels of disability, and ceiling/floor effects were analyzed for examining discriminative validity.
RESULTS: The CASI total score and domains had moderate to high correlations with 3 functional measures (r = 0.42-0.80), except in 2 CASI domains (i.e., attention and language). The CASI total score and domains showed moderate to high correlations with 3 cognitive measures (r = 0.45-0.93). The t-test results revealed significant differences (P < .05) in the CASI total score and other domains except for the short-term memory domains. Four domains of the CASI showed noticeable ceiling effects (22.4-39.7%).
CONCLUSIONS: The CASI has adequate ecological validity, good convergent validity, and acceptable discriminative validity in people with dementia. The 5 domains with nonsignificant differences or ceiling effects should only be used with caution to distinguish people with dementia.

Introduction

Dementia is a neuropsychiatric syndrome characterized by cognitive decline with a progressive loss of several cognitive and intellectual abilities such as judgment, abstract thinking, and memory. Cognitive disabilities are those that impact an individual's ability to access, process, or remember information.[ Symptoms of people with dementia include loss of memory, difficulty finding the right words to speak, misunderstanding what people are saying, difficulty performing previously routine tasks, and dysfunctional personality or mood changes.[ Therefore, both researchers and practicing clinicians need to address the global cognitive function of people with suitable treatment plans and assessments of those treatment outcomes.

The Cognitive Abilities Screening Instrument (CASI) was designed for cross-cultural studies to measure global cognitive function in people with dementia.[ The development of the CASI was based on the Mini-Mental State Examination (MMSE), the Modified Mini-Mental State test, and the Hasegawa Dementia Screening Scale.[ The CASI consists of 9 domains: long-term memory, short-term memory, concentration, orientation, attention, abstraction and judgment, language abilities, visual construction, and category fluency.[ The CASI has 3 characteristics. First, it includes 9 domains that comprehensively measure cognitive function and provide cognitive profiles of subjects to capture their abilities and disabilities in multidimensional cognitive functions. Second, it has a wide score range (0–100), which can better monitor treatment effects and disease progression. Third, the MMSE score can be transformed from the CASI.[ Thus, the CASI is a measure that can be applied for general use for people with dementia.

Regarding the psychometric properties of the CASI, test–retest reliability and construct validity using confirmatory factor analysis have been evaluated for people with dementia.[ However, the ecological validity, convergent validity, and discriminative validity of the CASI have not been tested in people with dementia. Real-world performance tests the extent of ecological validity.[ The performance of daily functions is a good indicator of cognition in people with dementia. Convergent validity is the extent to which constructs that should be theoretically related are associated in reality.[ Discriminative validity refers to the ability of a measure to discriminate among different functional levels or disability levels in people with dementia.[ Examining various types of validity provides vigorous support for the validation of a measure.[ To enhance the utility of test results of the CASI, it is important to provide empirical evidence of ecological validity, convergent validity, and discriminative validity for clinicians and researchers. The purpose of this study was to examine the CASI for its ecological, discriminative, and convergent validities in people with dementia.

Methods

Participants

We recruited people with dementia in Northern Taiwan between June 2017 and January 2018. The inclusion criteria were as follows: diagnosis of dementia according to the Diagnostic and Statistical Manual of Mental Disorders, fifth edition; age greater than 50 years; and willingness to participate in the study (written informed consent provided by the people with dementia or their caregivers). The exclusion criteria were: diagnosis of mental retardation; and history of severe brain injury. This study received approval from the hospital's institutional review board (CTH-105-2-5-024).

Procedure

All assessments were performed by a single examiner in 2 sessions after consent was provided. Study subjects were accepted if they met the inclusion and had no exclusion criteria. The CASI, the Alzheimer's Disease Assessment Scale-Cognitive subscale (ADAS-Cog), the Mini-Mental State Examination (MMSE), and the Clinical Dementia Rating (CDR) were administered to each person, while the Barthel Index (BI) and the Lawton Instrumental Activities of Daily Living (Lawton IADL) were filled out by their caregivers. All assessments were conducted in a quiet place to avoid interference and prevent participants’ performance from being affected. Demographic data were collected from their medical records.

Measures

The CASI assesses global cognitive function using 9 cognitive domains (score ranges): long-term memory (0–10), short-term memory (0–12), attention (0–8), mental manipulation (0–10), orientation (0–18), abstraction and judgment (0–12), language (0–10), visual construction (0–10), and list-generating fluency (0–10). General knowledge recall ability of the subject was used to measure the long-term memory domain. The individual's ability to recall information provided for a brief time was used to measure the short-term memory domain. The individual's ability to first hear and then echo those words was used to measure the attention domain. The individual's arithmetic abilities were used to assess the mental manipulation domain. The orientation domain measures one's orientation in place, time, and age. The abstraction and judgment domain measures one's abilities to solve problems. The language domain measures one's abilities to read, name, write, and pursue instructions. The individual's ability to copy figures was used to measure the visual construction domain. The individual's ability to list 4-legged animals tested the list-generating fluency domain. The sum of the 9 domains’ scores is the total CASI score, which ranges from 0 to 100. A greater score demonstrates superior global cognitive function.[

The ADAS-Cog assesses cognitive status using 11 items: orientation, word recall, ideational praxis, constructional praxis, following commands, naming objects and fingers, word recognition, remembering test instructions, spoken language ability, language comprehension, and word-finding difficulty.[ The range of the total score is 0 to 70.[ A higher score indicates the worse cognitive performance.[ The reliability and validity of the ADAS-Cog has been examined for people with dementia.[

Five areas of global cognitive function are measured in the MMSE: language, calculation and attention, orientation, recall, and registration. The MMSE score is 0 to 30. The higher the MMSE score, the better the cognitive function.[ The reliability and validity of the MMSE has been demonstrated in people with dementia.[

Cognitive and functional impairments of people with dementia are measured by CDR using 6 domains: community affairs, orientation, memory, hobbies and home, judgment and problem solving, and personal care.[ The 5 domains (i.e., orientation, memory, judgment and problem solving, community affairs, and home and hobbies) include 5 grades (0, 0.5, 1, 2, and 3), while the personal care domain is divided into 4 grades (0, 1, 2, and 3). The total score is derived from 6 domains to define dementia severity: 0 (healthy), 0.5 (questionable dementia), 1 (mild dementia), 2 (moderate dementia), and 3 (severe dementia).[ The CDR's reliability and validity is sufficient in people with dementia.[

The BI measures basic activities of daily living using 10 items (e.g., feeding and bathing). The 10 items are measured by using 2-point (0 and 1), 3-point (0, 1, and 2), or 4-point (0, 1, 2, and 3) Likert scales. The score ranges from 0 to 20 in total. A higher score demonstrates better function of basic activities of daily living. Participants can be categorized into 3 levels of disability according to the BI scores (0–10, severe disability; 11–18, mild to moderate disability; 19–20, independent).[ Participants with severe disability were few in number in this study (participants who had a BI score ≤ 10 amounted to 8.6%), so the participants were divided into 2 groups: 0–18, who had a disability; and 19–20, who did not have a disability.[

The Lawton IADL assesses instrumental activities of daily living using eight 2-point items (0 and 1): using the phone, shopping, cooking, doing housework, laundry, using transportation, medication management, and financial management.[ The total score range is 0 to 8. The higher scores indicate better function of instrumental activities of daily living. The Lawton IADL has proven reliability and validity in people with dementia.[

Data analysis

The hypothesis for examining ecological validity is that the CASI scores (i.e., total and domain scores) have moderate to high correlations with the BI, Lawton IADL, and the 3 functional domains of the CDR (i.e., community affairs, home and hobbies, and personal care). We measured ecological validity by the degree of correlations (using Pearson's r) among the CASI scores with Lawton IADL, and 3 functional domains of the CDR. The criteria for ecological validity were as follows: r < 0.30, poor ecological validity; 0.30 ≤ r < 0.60, adequate ecological validity; and r ≥ 0.60, good ecological validity.[

The hypothesis for examining convergent validity is that the CASI scores are moderately to highly correlated with scores on the ADAS-Cog, MMSE, and 3 cognitive domains of the CDR (i.e., orientation, memory, judgment, and problem solving). Convergent validity was examined using Pearson's r to calculate correlations among the CASI scores and the ADAS-Cog, MMSE, and 3 cognitive domains of the CDR. The criteria of convergent validity were as follows: r < 0.40, low correlation; 0.40 ≤ r < 0.70, moderate correlation; and r ≥ 0.70, high correlation.[

The hypothesis for examining discriminative validity is that the CASI scores can distinguish among different disability levels. We used an independent t-test (α = 0.05) to look for discriminative validity to determine if there were statistically significant differences in the CASI scores between subjects with no disability from those who had a disability.[ Moreover, ceiling and floor effects were estimated by the percentage of participants with the maximum and minimum scores in each domain of the CASI, respectively. A high percentage (≥ 20%) represented the ceiling effect or floor effect.[

Results

A total of 58 participants (29 men and 29 women) were included. Their ages were 59 to 97 years. Most participants had vascular dementia (84.5%). The mean of participants’ CDR score was 1.15. The participants’ demographic information is presented in Table 1.

Table 1

Demographics and clinical characteristics of participants (n = 58).

Characteristic
Age (years), mean (SD)	80.54 (7.90)
Gender, n (%)
Male	29 (50)
Female	29 (50)
Marital status, n (%)
Unmarried	1 (1.7)
Married	23 (39.7)
Divorced	3 (5.2)
Widowed	31 (53.4)
Education, n (%)∗
Below elementary school	23 (41)
Junior to senior high school	17 (30.4)
College and above	16 (28.6)
Diagnosis, n (%)
Alzheimer disease	5 (8.6)
Vascular dementia	49 (84.5)
Parkinson disease	4 (6.9)
Place of residence, n (%)
Home	37 (63.8)
Residential Care Homes	21 (36.2)
CDR, mean (SD)	1.15 (0.76)

CDR = Clinical Dementia Rating, SD = standard deviation.

There are missing data in education (2).

Table 2 shows the ecological validity results. The total CASI score had moderate to high correlations with scores of the BI, Lawton IADL, and 3 functional domains of the CDR (r = 0.46–0.80). Except for the attention and language domains, the other domains of the CASI were moderately correlated with the BI (r = 0.30–0.53). The 9 domains exhibited moderate to high correlations with the Lawton IADL and 3 functional domains of the CDR (r = 0.42–0.75).

Table 2

Results of ecological validity.

				CDR
Total score and domain score	Mean (SD)	BI	Lawton IADL	Community affairs	Home and hobbies	Personal care
CASI total score	51.72 (23.98)	0.46	0.64	0.80	0.75	0.70
Long-term memory	7.45 (3.16)	0.37	0.48	0.75	0.70	0.63
Short-term memory	3.16 (3.03)	0.36	0.64	0.54	0.50	0.53
Attention	5.59 (2.19)	0.20	0.43	0.62	0.52	0.49
Mental manipulation	5.05 (3.15)	0.44	0.51	0.67	0.67	0.63
Orientation	7.36 (5.82)	0.53	0.63	0.66	0.65	0.62
Abstraction and judgement	5.62 (3.32)	0.39	0.61	0.69	0.66	0.65
Language	7.42 (2.88)	0.28	0.42	0.72	0.66	0.58
Visual construction	6.84 (3.69)	0.37	0.52	0.65	0.60	0.49
List-generating fluency	3.22 (1.86)	0.30	0.47	0.59	0.55	0.54

BI = Barthel Index, CDR = Clinical Dementia Rating, Lawton IADL = Lawton Instrumental Activities of Daily Living.

For convergent validity, the total CASI score revealed high correlation with the ADAS-Cog, MMSE, and 3 cognitive domains of the CDR (r > 0.70) (Table 3). Three domains of the CASI (i.e., long-term memory, abstraction and judgment, and language) showed high correlations with the ADAS-Cog, MMSE, and 3 cognitive domains of the CDR (r = 0.71–0.89). The short-term memory and mental manipulation domains revealed moderate correlations with the ADAS-Cog, MMSE, and 3 cognitive domains of the CDR (r = 0.45–0.69). The other 4 CASI domains demonstrated moderate to high correlations with the ADAS-Cog, MMSE, and 3 cognitive domains of the CDR (r = 0.58–0.75).

Table 3

Results of convergent validity and discriminative validity.

			CDR
Total score and domain score	ADAS-Cog	MMSE	Orientation	Memory	Judgement and problem solving	t value (P value)	Ceiling effect (%)	Floor effect (%)
CASI total score	0.93	0.85	0.80	0.85	0.87	−3.51 (.001)∗	0	0
Long-term memory	0.84	0.71	0.75	0.83	0.89	−2.98 (.005)∗	39.7	5.2
Short-term memory	0.68	0.60	0.45	0.57	0.57	−2.01 (.051)	0	10.3
Attention	0.73	0.59	0.61	0.58	0.66	−2.08 (.043)∗	22.4	1.7
Mental manipulation	0.69	0.65	0.65	0.65	0.67	−3.42 (.001)∗	5.2	13.8
Orientation	0.77	0.75	0.68	0.75	0.67	−3.42 (.001)∗	5.2	10.3
Abstraction and judgement	0.82	0.72	0.73	0.77	0.82	−3.87 (.000)∗	1.7	0.3
Language	0.82	0.78	0.72	0.74	0.80	−2.13 (.039)∗	24.1	3.4
Visual construction	0.73	0.69	0.60	0.68	0.69	−2.62 (.012)∗	34.5	17.2
List-generating fluency	0.71	0.59	0.72	0.67	0.73	−2.54 (.015)∗	0	13.8

ADAS-Cog = Alzheimer's disease assessment scale-Cognitive subscale, CDR = Clinical Dementia Rating, MMSE = Mini-Mental State Examination.

Significant at P < .05.

Statistically significant differences were noted in the t-test results (P < .05) in the total CASI score and domains for discriminative validity, with the exception of the short-term memory domain (Table 3). The total CASI score did not show ceiling effect and floor effect. The 4 domains of the CASI demonstrated obvious ceiling effect (22.4–39.7%), including the long-term memory, attention, language and visual construction domains.

Discussion

For ecological validity, the total CASI score and domain scores had moderate to high correlations with the BI, Lawton IADL, and 3 functional domains of the CDR (i.e., community affairs, home and hobbies, and personal care), but not 2 domains (i.e., attention and language) with low correlations with the BI. That is, the CASI can illustrate the degree of independence of people with dementia in the real world, especially instrumental activities of daily living. The CASI scores (i.e., total and domain scores) generally have higher correlations with instrumental activities of daily living (the Lawton IADL community affairs, and home and hobbies) than basic activities of daily living (the BI and personal care domain of the CDR). A possible reason for this is that instrumental activities of daily living are more complex than basic activities of daily living and require more interactions with the environment, which involves more cognitive ability.[ People with dementia who can perform basic activities of daily living may not need to interact with others.[ Clinicians and researchers may provide intervention of functional ability training for people with dementia who have lower CASI scores to improve their ability to perform activities daily living.

The convergent validity analysis showed moderate to high correlations among the CASI scores and the MMSE, ADAS-Cog, and 3 cognitive domains of the CDR (i.e., orientation, memory, judgment, and problem solving). These results indicate that the CASI assesses the same cognitive constructs as other cognitive measures. The short-term memory and mental manipulation domains of the CASI showed relatively lower correlations with the ADAS-Cog, MMSE, and 3 cognitive domains of the CDR. The items of the short-term memory domain of the CASI are assessed by giving hints, whereas the other 3 cognitive measures are tested without hints. The mental manipulation domain of the CASI and MMSE assess serial subtraction of one number, whereas the ADAS-Cog and CDR do not assess digital calculations. The mental manipulation domain of the CASI also assesses reading a series of numbers backward, whereas the MMSE does not include those. Reading a series of numbers backward may spark auditory memory.[ Therefore, CASI's mental manipulation and short-term memory domains have moderate correlations with the 3 cognitive measures. In large part, good convergent validity is observed with the CASI.

The t-test results demonstrated that the CASI total score and domains (except for short-term memory) are able to discriminate between 2 groups of people with dementia who have varying levels of disability. In 4 domains of the CASI, 4 domains did show ceiling effects (i.e., long-term memory, attention, language, visual construction), which implies that these 4 domains were not able to distinguish people with dementia who had the highest scores of those specific domain-cognitive functions. The ceiling effects may result from the high percentage of our sample having less severe dementia (72.4% of the participants had a CDR ≤ 1). To reduce the ceiling effects, future studies may revise and add more difficult items to these 4 domains. Overall, the CASI demonstrates acceptable discriminative validity.

There were several limitations to this study. First, the participants were recruited from only northern Taiwan, which may limit the generalized ability of our results to other regions of Taiwan or other countries. Future studies could expand the research to different cities in Taiwan and other countries to cross-validate our findings. Second, 2 levels of disability were used based on participants’ BI scores. Future studies are warranted to recruit people with dementia with a variety of disabilities to examine discriminative validity. Third, this study included only 58 participants and may not be representative of the larger population. Smaller sample size studies risk results not truly reflecting accurate descriptions of the entire population. A future study with a larger sample group that includes individuals who are relevant to this survey's topic will increase data integrity.

In summary, the CASI has adequate ecological validity, good convergent validity, and acceptable discriminative validity in people with dementia. Assessment of global cognitive function in people with dementia using the CASI is appropriate. The CASI does reflect ability of people with dementia to perform activities of their daily life. For the short-term memory domain with nonsignificant difference and 4 domains with ceiling effects (i.e., long-term memory, attention, language, and visual construction), clinicians and researchers should cautiously use the CASI to distinguish people with dementia who have different levels of disability.

Author contributions

Conceptualization: En-Chi Chiu.

Data curation: Zi-Hua Jian.

Formal analysis: Zi-Hua Jian.

Investigation: En-Chi Chiu.

Project administration: En-Chi Chiu.

Resources: En-Chi Chiu.

Validation: Chih-Ping Li.

Writing – original draft: Zi-Hua Jian, En-Chi Chiu.

Writing – review & editing: Chih-Ping Li.