The Korean Version of the Cognitive Assessment Scale for Stroke Patients (K-CASP): A Reliability and Validity Study.

OBJECTIVE: To develop the Korean version of the Cognitive Assessment Scale for Stroke Patients (K-CASP) and to evaluate the test reliability and validity of the K-CASP in stroke patients.
METHODS: The original CASP was translated into Korean, back-translated into English, then reviewed and compared with the original version. Thirty-three stroke patients were assessed independently by two examiners using the K-CASP twice, with a one-day interval, for a total of four test results. To evaluate the reliability of the K-CASP, intra-class correlation coefficients were used. Pearson correlations were calculated and simple regression analyses performed with the Korean version of Mini-Mental State Examination (K-MMSE) and the aphasia quotient (AQ) to assess the validity.
RESULTS: The mean score was 24.42±9.47 (total score 36) for the K-CASP and 21.50±7.01 (total score 30) for the K-MMSE. The inter-rater correlation coefficients of the K-CASP were 0.992 on the first day and 0.995 on the second day. The intra-rater correlation coefficients of the K-CASP were 0.997 for examiner 1 and 0.996 for examiner 2. In the Pearson correlation analysis, the K-CASP score significantly correlated with the K-MMSE score (r=0.825, p<0.001). The coefficients of determination (r2) of the AQ were 0.586 for the K-MMSE and 0.513 for the K-CASP in the simple regression analysis.
CONCLUSION: The K-CASP is a reliable and valid instrument for cognitive dysfunction screening in post-stroke patients. It is more applicable than other cognitive assessment tools in stroke patients with aphasia.

INTRODUCTION

The reported incidence of cognitive impairment after stroke ranges from 10% to 30% [1]. Post-stroke cognitive impairment is closely associated with deteriorating functions and increased mortality [2]. According to Pasquini et al. [3], the possibility that stroke patients will stay in a shelter after three years is strongly associated with cognitive impairment, regardless of physical disability. Thus, they emphasized the importance of cognitive assessments during the acute phase of stroke and the follow-up.

Various screening tests for cognitive function have been used, such as the Mini-Mental State Examination (MMSE), the Montreal Cognitive Assessment (MoCA), Addenbrooke's Cognitive Examination-Revised (ACER), and the Abbreviated Mental Test (AMT). According to Lees et al. [4], these tests exhibited similar accuracy levels in screening the cognitive decline of stroke patients and none of the tests was clearly superior.

The MMSE, the most widely used screening test for cognitive function, is easy and simple to use in clinical practice. However, there are limitations in detecting mild cognitive dysfunction due to the omission of executive functions and attention, and an overly simple assessment of recall and repetition functions [56]. On the other hand, the MoCA is effective in detecting mild cognitive dysfunction, but it omits many items that assess executive function. Moreover, since both tests require verbal responses, if a patient has a language disorder, assessment of his or her orientation, recall, and memory will be difficult [789].

Thus, in 2014, Benaim et al. developed the Cognitive Assessment Scale for Stroke Patients (CASP), which can assess cognitive function of patients using a visual format that does not require verbal answers. The CASP is more practicable for aphasic patients than other existing tests like the MMSE and the MoCA. With expressive aphasic patients, the total score and mean administration time were less influenced by aphasia severity compared to other tests [810]. In addition, the CASP includes multiple cognitive domains such as executive function, praxis, and neglect. However, a standardized Korean version of the CASP does not yet exist, and neither have its validity and reliability been reported.

In this study, a Korean version of the CASP (K-CASP) was developed, and its reliability and validity were verified.

MATERIALS AND METHODS

CASP

The CASP consists of six domains, namely language, spatial neglect/visual construction, executive function, short-term memory, praxis, and temporal orientation. Each subscale is assigned 6 points with a total CASP score of 36 points.

Language is divided into expression (0–3 points) and comprehension (0–3 points). Expression is assessed by naming six images [11]. Comprehension is tested by the performance of simple tasks, such as ‘touch the pen’ or ‘show me your nose.’ Spatial neglect (0–2 points) is assessed via horizontal line bisection which involves looking at a vertical line on the right side of a paper, and judging whether a horizontal line bisection can be made by the examinees [12]. Visual construction (0–4 points) is measured by copying a cube with the three sides and angles intact.

Executive function consists of the graphic series (0–2 points) and inhibition/flexibility (0–4 points). For the assessment of the graphic series, the examinees are asked to continuously draw repeated characters according to a certain pattern. The inhibition test assesses whether the examinees can follow a fixed rule. When the examiner taps the desk once, the examinees tap twice. When examiner does it twice, the examinees should respond by tapping once. The flexibility test is assessed by a set of similar rules [13]. Short-term memory (0–6 points) is measured by having the examinees identify the images of the previous language expression task among 18 images shown [11].

Praxis (0–6 points) is assessed by the ability to carry out six tasks. When the examiner makes a gesture of ‘horn with fingers’ or verbally orders to ‘blow a kiss,’ the examinees are assessed whether they can imitate the gesture or follow the order correctly. The examiner makes a military salute and observes whether the examinees can associate this gesture with one of three presented drawings. In addition, temporal orientation (0–6 points) is assessed by showing a calendar and checking whether the examinee can show the correct day of the week (e.g., Sunday), date (e.g., the 9th), month, and year.

Korean translation of the CASP

Permission for use of the K-CASP was obtained from the author of the original scale, Charles Benaim via e-mail. The original CASP was translated into Korean, taking considerable care not to change any meaning with an internationally acceptable consistency, using the cross-cultural translation method of Sousa and Rojjanasrirat [14]. It was forward translated into Korean and independently adapted in the Korean context by two bilingual physiatrists. The two forward-translated versions were reviewed and a preliminary initial version was constructed. This version was blindly back-translated into English by other two bilingual-bicultural translators, then reviewed and compared with the original version. In the next step, the pre-final version was made via committee discussions until discrepancies were corrected. This pre-final version was tested on three individuals to assess comprehensibility. Subsequently, the final Korean version of the CASP was constructed (K-CASP) (Appendix 1).

Subjects

This study was conducted with 33 patients admitted to the rehabilitation unit of Seoul St. Mary's Hospital within the period from April 2015 to February 2016. The inclusion criteria were an acute or sub-acute stroke within three months from the onset and an age of 18 years or older. The exclusion criteria included a history of cognitive dysfunction before the stroke, visual disorders not compatible with viewing pictures, psychiatric disorders, or an etiology involving the brain other than stroke. The subjects were assessed by using the Korean version of MMSE (K-MMSE) prior to the assessment with the K-CASP. Approval from the Institutional Review Board of the Catholic University Medical Center was obtained for this study.

Assessment of the K-CASP

All patients were assessed four times with a one-day interval by two physiatrists. One subject was assessed twice by each physiatrist on the same day and re-assessed twice in the same way one day after the previous test to check for consistency. The examiner and the subject sat across each other and the test was performed with a test paper, a pen, a phone, and a sheet of paper. The images were upside-down and the patients did not have to read instructions. Therefore, the test paper was placed on the table with the images facing the patient and the instructions facing the examiner. To maintain sustained attention, the environment was kept quiet and comfortable. With regard to the order of the tests, simple random sampling was carried out to prevent the occurrence of error depending on when a test was administered. No significant differences was found between the two tests in terms of the rehabilitation treatment or drugs that were taken by the subjects.

Reliability and validity test

The consistency between and within the examiners was analyzed based on the intra-class correlation coefficient (ICC), an equivalence assessment method, and through a non-parametric test. To determine the inter-rater reliability of the K-CASP, the assessment results of examiners 1 and 2 were compared both on the first and the second day. To assess the intra-rater reliability, the first performance result was compared with the second one, separately for the examiners 1 and 2.

Concurrent validity exists when an assessment correlates well with a previously validated measure. It was measured by comparing the K-CASP mean score with the total K-MMSE score calculating Pearson correlation coefficients. The patients were divided into a cognitive and a non-cognitive dysfunction group according to K-MMSE score (cut-off value, 24) [15], with 17 subjects in the latter and 16 in the former group. For the validity assessment, an independent t-test with the K-CASP score was conducted between the cognitive and non-cognitive dysfunction groups.

Statistical analyses

The inter-rater reliability and intra-rater reliability were measured using the ICC, which indicates stability of the instrument if ICC≥0.75 [16]. The internal consistency of the K-CASP was evaluated using Cronbach's coefficient alpha for each domain. To assess the validity, Pearson correlation coefficients with the K-MMSE were calculated: values over 0.5 were considered strong, and those between 0.3 and 0.5 moderate. To compare two groups according to MMSE scores, an independent t-test was conducted. Concurrent validity of the K-CASP and the K-MMSE was evaluated by a simple regression analyses that considered the K-CASP and K-MMSE scores as the dependent variables and age, gender, location of the lesion, its type, and the aphasia quotients (AQ) as independent variables. For the statistical analysis, SPSS ver. 24.0 K for Windows (IBM, Armonk, NY, USA) was used, and a p-value of <0.05 was considered statistically significant.

RESULTS

The subjects consisted of 21 men and 12 women. The mean age of the patients was 67.67±12.95 years. The mean period after stroke was 31 days. Twenty-three cases were caused by cerebral ischemia and 10 cases by cerebral hemorrhage. Regarding the locations, 14 patients had lesions in the right hemisphere, 15 in the left hemisphere, and 4 in the bilateral hemisphere. The mean K-MMSE score was 21.50±7.01 and the mean K-CASP score was 24.42±9.47. The Western Aphasia Battery (WAB) was used to determine the type and severity of aphasia [17]. Based on the Korean version of the Western Aphasia Battery (K-WAB), 8 aphasia cases were identified. In addition, aphasia severity was quantified using AQ. The type of aphasia and the mean value of the AQ score for the K-WAB as well as the K-MBI are presented in Table 1.

Table 1

Clinical and demographic data of the patients (n=33)

Values are presented as mean±standard deviation or number (%).

K-CASP, Korean version of the Cognitive Assessment Scale; K-MMSE, Korean-Mini-Mental State Examination; AQ, quotient score for the Korean version-Western Aphasia Battery published by the Paradise Welfare Foundation; K-MBI, Korean version of the Modified Barthel index.

a)n=29.

Inter-rater reliability

There were high inter-rater correlations between the examiners in the first assessment (ICC=0.992, p<0.001) and in the second assessment (ICC=0.995, p<0.001) (Table 2, Fig. 1A, 1B).

Table 2

Inter-rater and intra-rater reliabilities of the K-CASP

K-CASP, Korean version of the Cognitive Assessment Scale; ICC, intra-class correlation coefficient.

*p<0.001.

Fig. 1

Scatter plot of reliability with the simple regression adjusted to fit the data. All scores from the K-CASP are expressed in logits. (A) Inter-rater reliability assessed at the first day, (B) inter-rater reliability assessed at the second day, (C) intra-rater reliability assessed by examiner 1, (D) intra-rater reliability assessed by examiner 2. K-CASP, Korean version of the Cognitive Assessment Scale; ICC, intra-class correlation coefficient.

Intra-rater reliability

The intra-rater correlations were high: the ICC of the intra-rater reliability of examiner 1 was 0.997 (p<0.001), and that of examiner 2 was 0.996 (p<0.001) (Table 2, Fig. 1C, 1D). The internal consistency of the K-CASP subscales are displayed in Table 3. Cronbach's alpha for each domain was >0.900.

Table 3

Internal consistency of the K-CASP by Cronbach's alpha

K-CASP, Korean version of the Cognitive Assessment Scale.

*p<0.05.

Validity

The Pearson correlation coefficient between the total K-MMSE score and the mean scores in the four K-CASP tests was 0.825 (p<0.001) (Fig. 2).

Fig. 2

Correlation between the K-MMSE score and the mean K-CASP score. K-MMSE, Korean version of Mini-Mental State Examination; K-CASP, Korean version of the Cognitive Assessment Scale; r, Pearson correlation coefficient.

As described earlier, subjects were classified into cognitive dysfunction and non-cognitive dysfunction groups based on K-MMSE scores. The validity and inter-rater reliability of the K-CASP according to cognitive level were assessed by comparing these two groups.

In the cognitive dysfunction group, the mean K-CASP scores obtained by the two examiners were 17.26±7.88 and 18.59±8.45, respectively. No significant examinerrelated difference in the score was found (p=0.640). In the non-cognitive dysfunction group, the mean K-CASP scores were 30.78±5.11 and 31.09±5.28, respectively. No significant examiner-related score differences were observed (p=0.866) (Table 4).

Table 4

K-CASP scores according to cognitive level

Values are presented as mean±standard deviation.

K-CASP, Korean version of the Cognitive Assessment Scale; CI, confidence interval.

The patients were divided into aphasia and non-aphasia groups according to the AQ score. In the aphasia group, the mean K-CASP scores obtained by two examiners were 13.81±9.28 and 14.28±9.63, respectively. The mean K-MMSE score was 12.88±6.38. In the non-aphasia group, the mean K-CASP scores obtained by two examiners were 27.38±6.97 and 28.10±6.71, respectively, and the mean K-MMSE score was 24.26±4.57 (Table 5).

Table 5

K-MMSE and K-CASP scores according to aphasia

Values are presented as mean±standard deviation.

K-CASP, Korean version of the Cognitive Assessment Scale; K-MMSE, Korean version of Mini-Mental State Examination; CI, confidence interval.

To determine that the K-CASP assessment is less vulnerable to aphasia severity than the K-MMSE, simple regression analyses was performed. AQ correlated with both the K-MMSE (r2=0.586, p<0.001) and the K-CASP (r2=0.513, p<0.001) (Fig. 3).

Fig. 3

Correlations of the AQ, K-MMSE, and K-CASP. Simple regression analysis was performed to determine the relations. K-CASP, Korean version of the Cognitive Assessment Scale; K-MMSE, Korean version of Mini-Mental State Examination; AQ, quotient score for the Korean version-Western Aphasia Battery published by the Paradise Welfare Foundation. *p<0.001.

DISCUSSION

Post-stroke dementia occurs in up to 30% of stroke patients [18]. According to Babulal et al. [19], cognitive impairment in an acute stroke stage is a predictor of performance in activities of daily living and should be assessed appropriately in stroke rehabilitation. Thus, domain-specific neuropsychological assessments, including visuospatial construction and memory, should be considered in stroke patients at the acute phase [20]. The accurate assessment of cognitive functions is essential for planning the management of stroke patients. Although several assessment tools are available for cognitive impairment, some screening tests such as the MMSE are not suitable for aphasic patients because they require language-based answers. The MMSE is also insensitive to detecting multiple-domain and complex cognitive impairments [21].

Post-stroke aphasia makes assessment of cognitive deficiencies difficult. In language, a process related to cognition is included, and language affects other non-verbal domains; thus, it inevitably interacts with cognition. In stroke patients, aphasia itself affects working memory and attention, which can be explained by comprehension or processing [2223]. The study of Lee and Pyun [24] exhibited a significant correlation between aphasia severity and attention.

In this respect, Benaim et al. [10] designed the CASP by using visual items for stroke patients that can be performed at the patient's bedside with a relatively short duration. This new cognitive assessment tool included multiple domains and could be administered to patients with language problems. According to Barnay et al. [8], the CASP was more feasible than the MMSE in evaluating aphasic stroke patients. In subsequent research with non-aphasic patients the CASP was distinguished from the MMSE or the MoCA by its ability to evaluate spatial neglect [10].

In our study, the CASP was first translated into Korean and then tested with patients. The CASP was introduced in 2014; hence, not many studies have been conducted, particularly related to validity and reliability. In contrast to the original study, the reliability of the CASP was analyzed for the first time in this study. It provided evidence that supports the usefulness of the K-CASP as a valid and reliable tool to evaluate cognitive impairment in stroke patients. According to the results, the K-CASP displayed both a high within-examiner and between-examiner reliability and revealed a quite strong correlation with the K-MMSE score. Since the subjects had acute or sub-acute stroke, cognitive recovery was possible due to neuroplasticity over time. To minimize the effect of natural improvement, the K-CASP was administered to the subjects twice with a one-day interval. All subjects completed the test and the mean administration time was about 10 minutes, even if bed-ridden in a semi-Fowler's position. Four K-CASP tests were conducted within a short 2-day period and there were no significant differences among the four tests' total scores. Thus, a learning effect could be ruled out.

As stated before, Barnay et al. [8] suggested that the CASP is more practicable than the MMSE in stroke patients with aphasia. In the original author's study, the percentage of non-achievable CASP administrations was significant lower compared to the MMSE. In this study, to evaluate that the K-CASP was more useful than the K-MMSE for the patients with language problems, the patients were divided into aphasia and non-aphasia groups. However, both the mean K-MMSE and the mean K-CASP scores were significantly lower in the aphasia group than in the non-aphasia group. Furthermore, in the simple regression analyses, the AQ significantly accounted for both the K-CASP score and the K-MMSE score. These results are consistent with the previous study and showed that severe aphasic patients displayed great cognitive impairment [25].

However, in the simple regression analyses, the coefficient of determination (r2) of the AQ was lower for the K-CASP than for the K-MMSE. Therefore, it is possible that the K-MMSE score is more strongly associated with the AQ than the K-CASP score. Unlike in the original study, the CASP score was influenced by aphasia severity, but it must be considered that aphasia severity was previously determined by the Boston Diagnostic Aphasia Examination (BDAE), which differs from the WAB in classifying aphasia. In this study, the aphasia group consisted of only 8 patients and due to the small sample size, the analysis of each subtype was limited. The K-CASP scores of the patients with motor aphasia were expected to be higher than their K-MMSE scores, since the CASP can evaluate executive function and attention non-verbally [8]. However, the assessment results actually found those abilities to be lower than those of the K-MMSE in two motor aphasic patients. These patients had worsened overall conditions when the K-CASP was administered. On the other hand, a Broca's aphasia patient scored 0/3 in the recall domain in K-MMSE, but was able to answer the ‘short-term memory’ item from the K-CASP with a score of 4/6. In this regard, further research about aphasic patients is necessary.

The limitations of this study are as follows: first, the CASP original version was not used for the general population because of its recent development, and a cut-off value could not be suggested since the data needed to establish it were lacking. Thus, the use of the CASP solely in clinical practice is limited. However, the K-CASP can supplement other cognitive assessment tools, such as the MMSE. Second, this study was conducted with only 33 persons. Nevertheless, the small sample size revealed a normal distribution as determined by the Kolmogorov-Smirnov test. Furthermore, findings from our research were similar to those of the original study. Third, this study tried to retain the original CASP's nature as much as possible by using the images in the original version. However, these pictures, including those of a pineapple, cheetah, and the figure of a judge were not familiar to Koreans; thus, it was presumed that they would yield less accurate results in terms of naming or recall. In this study, among the 10 subjects who were assessed to be normal based on the K-MMSE memory and recall items, seven obtained lower scores in the memory items assessed by image recall with the K-CASP. To exclude the influence of the unfamiliar pictures, a future modification of the standardized pictures is necessary.

In conclusion, the K-CASP is a reliable and valid instrument for the screening of cognitive dysfunction in post-stroke patients. As far as the evaluation of cognitive dysfunction in patients with aphasia is concerned, it is recommended to use the K-CASP along with preexisting evaluation tools for more efficient assessments. This study performed a different type of multi-domain cognitive assessment and its significance is the first-time introduction of the CASP in Korea. In the future, the CASP can be made more accurate through large-scale standardized studies with Korean samples.