The Korean Version of the Schizophrenia Cognition Rating Scale: Reliability and Validity.

OBJECTIVE: This study's aim was to develop and standardize a Korean version (SCoRS-K) of the Schizophrenia Cognition Rating Scale (SCoRS), which is used to evaluate the degree of cognitive dysfunction affecting the everyday functioning of people with schizophrenia.
METHODS: Eighty-four schizophrenia patients with stable symptoms who were receiving outpatient treatment and rehabilitation therapy, and 29 demographically matched non-patient controls, participated in the study. Demographic data were collected, and clinical symptoms, cognitive function, and social function were evaluated to verify SCoRS-K's reliability and validity. Clinical symptoms were evaluated using the Positive and Negative Syndrome Scale and the Clinical Global Impression-Schizophrenia Scale. Cognitive function was evaluated using a short form of the Korean Wechsler Adult Intelligence Scale and the Wisconsin Card Sorting Test (WCST). Social function was evaluated using the Social and Occupational Functioning Assessment Scale, the Schizophrenia Quality of Life Scale, and the Social Functioning Scale.
RESULTS: Data analysis demonstrated SCoRS-K's statistically significant reliability and validity. SCoRS-K has high internal consistency (Cronbach's alpha; patient 0.941, informant 0.905, interviewer 0.964); test-retest reliability [patient 0.428 (p=0.003), informant 0.502 (p<0.001), interviewer 0.602 (p<0.001); and global rating 0.642 (p<0.001)]. The mean scores of subjects were significantly higher than those of the controls (p<0.001), demonstrating SCoRS-K's discriminant validity. Significant correlations between the total scores and global rating score of SCoRS-K and those of the scales and tests listed above (except WCST) support SCoRS-K's concurrent validity.
CONCLUSION: SCoRS-K is a useful instrument for evaluating the degree of cognitive dysfunction in Korean schizophrenia patients.

INTRODUCTION

Cognitive dysfunction is a major symptom of schizophrenia,1 and it manifests in the areas of attention and executive function as well as linguistic and non-linguistic memory.2 A substantial number of schizophrenia patients present cognitive dysfunction from the prodromal stage, and cognitive dysfunction can be found in most patients who have experienced initial and recurrent episodes.3 Cognitive dysfunction is associated with both positive and negative symptoms of schizophrenia4 and patients' prognoses.5 Further, cognitive dysfunction can affect the everyday functioning of people with schizophrenia.6 Therefore, improving cognitive function is a significant goal in the treatment of schizophrenia, and there is much interest in the development of drugs to improve the cognitive function of people with schizophrenia.

In 2008, the expert panel of the National Institute of Mental Health (NIMH) reported the results of the National Institute of Mental Health's Measurement and Treatment Research to Improve Cognition in Schizophrenia (MATRICS) Consensus Cognitive Battery (MCCB). The goal of the MCCB was to develop a consensus cognitive battery for clinical trials of cognition-enhancing treatments for schizophrenia through a broadly based scientific evaluation of measures to assess basic cognitive function in individuals with schizophrenia.7 Many recent studies of the cognitive function of people with schizophrenia have used the MCCB.

However, a series of cognitive function tests in clinical settings is costly in terms of time and money, and training is needed to interpret the test results accurately. Moreover, cognitive function tests are limited in their ability to explain everyday functioning directly. For example, supports for a patient or assistance with their financial situation, and the absence or presence of occupational or psychosocial rehabilitation therapy are intertwined with cognitive function, and they can affect the everyday life of a patient. A cognitive function test needs to be readily applicable in actual clinical settings and useful in explaining clearly the connections between cognitive function and everyday functioning.

Studies of various tools to evaluate cognitive function in everyday life (and their validity) fall into two major categories. One category evaluates cognitive function through interview, and the other evaluates functional capacity through tests. Green et al.8 verified the reliability, validity, and appropriateness in clinical settings of two cognitive function evaluation scales through interviews with schizophrenia patients, and two skill capacity evaluation tools. The cognitive function evaluation scales were the Schizophrenia Cognition Rating Scale (SCoRS),9 which also interviewed informants, and the Clinical Global Impression of Cognition in Schizophrenia (CGI-SCH),10 which also interviewed carers. The skill capacity evaluation tools were the University of California, San Diego (UCSD) Performance-Based Skills Assessment (UPSA)11 and the Maryland Assessment of Social Competence (MASC).12

Of the four tools listed above, SCoRS has been assessed as having high reliability and validity, particularly in the evaluation of cognitive dysfunction affecting the daily life of patients with schizophrenia, although its validity is reduced when the patient's self-report is the sole information source.13 SCoRS assesses the patient's attention, memory, inference and problem solving skills, working memory, language production, and motor skills.14 In contrast with previous interview-based cognitive function tests that found it difficult to relate results to daily life as they interviewed subjects only, SCoRS overcame this limitation by interviewing informants as well as patients.9 SCoRS was verified as a useful evaluation tool for Asian patients with schizophrenia in a Singaporean English-language study,14 and a Japanese version is widely used in Japan to evaluate the cognitive function of patients with schizophrenia.15

This study was designed to develop a Korean version of SCoRS in order to evaluate appropriately the degree of cognitive dysfunction affecting the everyday functioning of Korean patients with schizophrenia, and to standardize the Korean version through tests of reliability and validity.

METHODS

Subjects

Based on the diagnosis standard of the Diagnostic and Statistical Manual of Mental Disorders, Fifth Edition (DSM-5),16 the study subjects were patients diagnosed with schizophrenia aged between 18 and 60 years, with stable symptoms, who had been taking the same dose of antipsychotics for the past three months, and who had been receiving rehabilitation therapy as outpatients. Patients with mental disorders other than schizophrenia (bipolar disorder, schizoaffective disorder, and organic mental disorder); patients with neurological disorders (head injuries or a seizure disorder); patients with a history of alcohol or drug abuse; and patients whose intelligence level was not sufficient to undertake the cognitive function tests were excluded from the study. People who matched the subjects for age, sex, and educational level and who had no history of mental disorder were selected as the control group.

Bonett's method was used to determine the optimum sample size.17 Eighty-four patients with schizophrenia and 29 non-patient controls participated in the study.

This study was approved by the Institutional Review Board for Human Research at Inje University Busan Paik Hospital.

Measures

The duration of the subjects' illness and the types and doses of current medications were identified through medical record review, and the doses of antipsychotics were converted into a chlorpromazine equivalent dose.1819 Experienced psychiatrists conducted face-to-face interviews with the subjects and the members of the control group. Cognitive function was evaluated using a short form of the Korean Wechsler Adult Intelligence Scale (K-WAIS) and the Wisconsin Card Sorting Test (WCST). Clinical symptoms were evaluated using the Positive and Negative Syndrome Scale (PANSS) and the Clinical Global Impression-Schizophrenia Scale (CGI-SCH). Social function was evaluated using the Social and Occupational Functioning Assessment Scale (SOFAS), the Schizophrenia Quality of Life Scale (SQLS), and the Social Functioning Scale (SFS).

The Korean version of the Schizophrenia Cognition Rating Scale

The Schizophrenia Cognition Rating Scale (SCoRS) developed by Keefe et al.9 is an assessment tool that focuses on cognitive dysfunction in people with schizophrenia and its relationship to their real-world functioning and functional capacity. It was developed to assess difficulties in attention, memory, motor skills, language production, and problem solving, and the cognitive difficulty experienced by subjects during the two weeks prior to interview. People with schizophrenia and informants respond to 20 items, and their responses are scored on a four-point scale, with higher scores denoting more severe dysfunction. The test takes approximately 30 minutes to administer.

Written pre-approval to develop a Korean version, SCoRS-K, was obtained from the primary original author, R S Keefe, and a professional translator translated SCoRS into Korean. A research committee of four psychiatrists, including the principal investigator, finalized the first edition after thorough discussion and revision. The text was back-translated into English by a professional translator and reviewed by the original developer to confirm that the intentions of the original author were accurately reflected in the back-translated version.

In the original study by Keefe et al.9 global ratings were identified for three groups-patient, informant, and interviewer- and the scores were compared. In a recent study,14 the sum of the 20 SCoRS items (the SCoRS total) was found to be meaningful, and the SCoRS total was included in addition to global ratings in the analysis in this study.

The short form of the Korean Wechsler Adult Intelligence Scale

K-WAIS is a standardized cognitive test, a Korean version of the Wechsler Adult Intelligence Scale-Revised (WAIS-R), which is used to evaluate verbal and performance intelligence. This study used a short form of K-WAIS, consisting of picture completion, arithmetic, digit span, and similarities. In a previous study comparing short-form tests of cognitive function in patients with schizophrenia, the short-form tests were more readily administered than full tests, and there was a high correlation between estimated intelligence and WAIS-R-measured intelligence.20

The Wisconsin Card Sorting Test Computer Version 4

The WCST developed by Heaton21 was used in the interviews. WCST is a representative test that measures executive function, with an emphasis on perseverance and achievement. The test uses 128 cards that have combinations of four colors, shapes, and numbers. The interviewee must deduce the classifications of the cards first, and then find the changing pattern of one category among color, shape, and number, without any explanation from the interviewer during the test.

The Korean version of the Positive and Negative Syndrome Scale

PANSS is a psychopathology evaluation tool developed by Kay et al.22 that provides “a balanced representation of positive and negative symptoms and gauges their relationship to one another,” and to global psychopathology. It is a 30-item, seven-point rating instrument; seven items assess positive symptoms, seven items assess negative symptoms, and 16 items assess general psychopathology. Higher scores denote more severe psychopathologic conditions. In the current study, a psychiatrist evaluated the patient group using K-PANSS, the Korean version of the test.23

The scores in each category of the five-factor model of PANSS proposed by van der Gaag et al.24 were also obtained. The five-factor model consists of positive symptoms, negative symptoms, cognitive/disorganization symptoms, excitement, and emotional distress.

The Clinical Global Impression-Schizophrenia Scale

The Clinical Global Impression-Schizophrenia scale (CGI-SCH) has been developed as a short, standardized method of assessing the symptoms of schizophrenia in a clinical setting. It consists of two categories; severity of illness and degree of change, each of which rate five dimensions (positive, negative, depressive, cognitive and global) that are evaluated using a seven-point scale.25

The Korean Version of the Social Functioning Scale

The Social Functioning Scale (SFS) was developed by Birchwood et al.26 to assess the self-reported ability of people with schizophrenia to manage social interactions, relationships, and activities of daily life. The scale investigates seven areas including social withdrawal, relationships, social activities, recreational activities, independence (competence), independence (performance), and employment. This study used KSRS, the Korean version of the Social Functioning Scale.27

The Korean language version of the Schizophrenia Quality of Life Scale-fourth revision

The Schizophrenia Quality of Life Scale-fourth revision (SQLS-R4) is a short, self-report quality of life questionnaire, that is generally completed within five to ten minutes.28 It consists of disease-specific items reflecting the daily lives of people with schizophrenia. A higher score denotes a lower quality of life. It comprises 33 items, and the maximum score is 132 points, which is converted to a 100-point scale. This study used the Korean language version of SQLS-R4 (SQLS-R4K).29

The Social and Occupational Functioning Assessment Scale

SOFAS30 assesses the individual's level of social and occupational functioning at the time of the interview, without being directly influenced by the individual's psychological symptoms. However, impairment from general medical conditions is considered in the assessment, which ranges along a continuum from “very outstanding” to “remarkably impaired.” In this study, a psychiatrist evaluated functional level after interview.

Statistical analysis

All continuous variables are reported as means, standard deviations, and ranges, and discrete variables are given as frequencies and percentages. Comparisons between patients with schizophrenia and controls were performed using t-tests, chi-square tests, and Wilcoxon rank sum tests depending on the characteristics of each variable.

Validity was tested through the following methods. For discriminant validity, the SCoRS scores of the subjects and the control group were compared using Wilcoxon rank sum tests. For concurrent validity, the correlations between SCoRS-K and K-WAIS, WCST, PANSS, CGI-SCH, SFS, SQLS, and SOFAS were verified using Pearson's correlation coefficient. Test-retest reliability was evaluated using the Intraclass Correlation Coefficient (ICC), and the internal consistency in each area was tested using Cronbach's alpha coefficient. The test-retest reliability was performed twice for the subjects, with a one-month interval between them.

All analyses were carried out using Statistical Analysis System 9.3 package (SAS Institute Inc., Cary, NC, USA), and the significance level was set at 0.05 or below.

RESULTS

Demographic data

The comparison between the demographic data of the patient group and the control group is presented in Table 1.

Table 1

Demographic and clinical characteristics of patients with schizophrenia and controls

CGI-SCH: Clinical Global Impression-Schizophrenia, CPZE: chlorpromazine equivalent, PANSS: Positive and Negative Syndrome Scale, SD: standard deviation, SFS: Social Functioning Scale, SOFAS: Social and Occupational Functioning Assessment Scale, SQLS: Schizophrenia Quality of Life Scale

There was a higher ratio of males to females in the patient group (57.14% to 42.86%) compared with the control group (48.28% to 51.72%). The mean age of the two groups was closely comparable, at 37.42±9.99 years for the patient group, and 37.34±9.15 years for the control group. The two groups' years of education were also closely comparable, at 13.08±1.62 years for the patient group and 13.12±1.39 years for the control group. The patient group had experienced schizophrenia over the long term; the mean duration of illness was 174.43±112.32 months, and the dosage of antipsychotics, converted to the chlorpromazine equivalent dose, was 610.19±338.86 mg/day.

Clinical characteristics and cognitive function

Patient-only test

The results from the Positive and Negative Syndrome Scale (PANSS), the Clinical Global Impression-Schizophrenia Scale (CGI-SCH), the Social Functioning Scale (SFS), the Schizophrenia Quality of Life Scale (SQLS), and the Social and Occupational Functioning Assessment Scale (SOFAS) are also presented in Table 1 above.

Patient and control group tests

The results for the short-form Korean version of the Wechsler Adult Intelligence Scale (K-WAIS) and the Wisconsin Card Sorting Test (WCST) are presented in Table 2. The patient and control group results for K-WAIS were significantly different, at 92.28±12.32 for patients and 108.79±11.76 for controls (p<0.001). For the three categories in the WCST (perseverative responses, perseverative errors and categories achieved) only the latter demonstrated a statistically significant difference between the patient group and the control group, at 1.46±1.87 for the patient group and 3.88±2.33 for the control group (p<0.001).

Table 2

Comparison of patients with schizophrenia and controls for cognitive function

CA: numbers of categories achieved, K-SCoRS: Korean version of Schizophrenia Cognitive Rating Scale, K-WAIS: Short-form of Korean-Wechsler Adult Intelligence Scale, PE: perseverative errors, PR: perseverative responses, SD: standard deviation, WCST: Wisconsin Card Sorting Test

Discriminant validity

There was a statistically significant difference for cognitive function between the patient group and the control group in all the SCoRS-K results (p<0.001), as shown in Table 2 above. This included interviewees' responses (34.65±11.07 for patients and 22.76±4.09 for the control group), interviewers' responses (36.87±11.76 for patients and 21.31±1.93 for the control group), and global rating (4.33±2.08 for patients and 1.14±0.35 for the control group).

Concurrent validity

Correlation between SCoRS-K and clinical characteristics

Correlations between SCoRS-K and clinical characteristics are presented in Table 3. Patients' SCoRS-K scores correlated significantly with global ratings on the PANSS' positive symptoms, negative symptoms, and general psychopathology subscales (p<0.001); however, SCoRS-K scores were not significantly correlated with informants' responses. Regarding the five-factor PANSS model, patients' SCoRS-K scores correlated significantly with scores on the cognitive/disorganization symptoms and emotional distress factors; additionally, interviewers' responses and global ratings correlated significantly with scores on negative symptoms, cognitive/disorganization symptoms, excitement, and emotional distress (ps<0.001). Scores on the SCoRS-K correlated significantly with scores on all of the CGI-SCH's global rating categories except positive symptoms; however, numerous other categories under patients' and informants' responses were positively correlated with patients' SCoRS-K scores (p<0.001).

Table 3

Correlation between K-SCoRS and demographic/clinical characteristics of patients with schizophrenia

CGI-SCH: Clinical Global Impression-Schizophrenia, CPZE: chlorpromazine equivalent, K-SCoRS: Korean version of Schizophrenia Cognitive Rating Scale, PANSS: Positive and Negative Syndrome Scale, SOFAS: Social and Occupational Functioning Assessment Scale, SQLS: Schizophrenia Quality of Life Scale

Correlation between SCoRS-K and cognitive function

Table 4 presents the results for K-WAIS and WCST correlations with SCoRS-K. Patients' and interviewers' responses and global rating scores correlated significantly with patients' K-WAIS scores (p<0.001). No significant correlations were detected regarding the WCST.

Table 4

Correlation between K-SCoRS and cognitive function in patients with schizophrenia

CA: numbers of categories achieved, K-SCoRS: Korean version of Schizophrenia Cognitive Rating Scale, K-WAIS: Short-form of Korean-Wechsler Adult Intelligence Scale, PE: perseverative errors, PR: perseverative responses, WCST: Wisconsin Card Sorting Test

Correlation between SCoRS-K and social function

Correlations between SCoRS-K and social function are presented in Table 5. Patients' SCoRS-K scores correlated significantly with patients and interviewers' SQLS scores and with patients' global SQLS rating. Regarding the SOFAS, SCoRS-K scores also correlated significantly with patients and interviewers' scores and with patients' global rating (ps<0.001). Regarding the SFS, significant correlations were detected for patients, informants, and interviewers' total ratings, but not for global rating (ps<0.001).

Table 5

Correlation between K-SCoRS and social functioning in patients with schizophrenia

K-SCoRS: Korean version of Schizophrenia Cognitive Rating Scale, SOFAS: Social and Occupational Functioning Assessment Scale, SQLS: Schizophrenia Quality of Life Scale

Inter-rater reliability

Two interviewers (S.S. Jung and D.U. Jung) participated in the same interviews of 13 patients. Intra-class correlations of the relationship between the ratings generated by the two interviewers were calculated to assess interrater reliability. The intra-class correlation coefficients for items were greater than 0.931.

Test-retest reliability

Table 6 records high test-retest reliability at the one-month follow-up for intra-class correlation coefficients calculated for the patient, informant, interviewer, and SCoRS-K global rating scores.

Table 6

Test-retest reliability

ICC: intra-class correlation coefficient, CI: confidence interval

Internal consistency

Cronbach's alpha is recorded in Table 7, and demonstrates high internal consistency for the three evaluation components of patient, informant, and interviewer. There was no item for which Cronbach's alpha improved when any one of 20 questions was excluded. There were no items whose deletion improved the overall internal consistency of the scale by more than 0.01.

Table 7

Cronbach's alpha

DISCUSSION

This study's aim was to develop and standardize a Korean version (SCoRS-K) of the Schizophrenia Cognition Rating Scale (SCoRS), which is used to evaluate the degree of cognitive dysfunction affecting the everyday functioning of patients with schizophrenia.

There were many differences between the scores for patients' responses and informants' responses in the original development and validation of SCoRS by Keefe et al.9 and in those cases interviewers tended to give greater weight to informants' responses than patients'. However, in this study, patients' response scores were similar to interviewers' scores, while informants' scores tended to be different. A number of patients in Korea live with their family unless they are hospitalized and all participants in this study lived with their parents to an advanced age. Most informants in this study were older parents, and the interviewers, who were the patients' primary care doctors or managers in rehabilitation therapy centers, were more knowledgeable about the patients' conditions than the informants were that contrary to expectations. The person with schizophrenia lives with their family in this study, therefore, we might expect them to have a close knowledge of the person's day-to-day functioning.

In order to examine the validity of SCoRS-K in measuring clinical symptoms, the study investigated if there was any correlation between the scores of patients', informants', and interviewers' responses and global ratings in SCoRS-K, and the scales for clinical symptoms in PANSS, CGI-SCH, SQLS and SOFAS. When PANSS was classified into positive symptoms, negative symptoms, and general psychopathology, there were significant correlations with interviewers' scores for all items. When PANSS was evaluated by the five-factor model for a more precise classification, there were significant correlations with interviewers' scores for all items except positive symptoms. For CGI-SCH also, there were significant correlations with the interviewers' scores for all items except positive symptoms. As in this study, the correlation between the clinical symptoms of patients with schizophrenia and cognitive function has been verified in a number of previous studies.3132 A substantial number of studies have reported a lack of correlation between positive symptoms and cognitive function.233

The correlation between SCoRS-K results and cognitive function varied depending on the focus of the different tests used. Significant correlations were found between SCoRS-K and K-WAIS in all cases except for informants' responses; therefore, it may be inferred that SCoRS-K appropriately reflects cognitive function. However, a significant correlation was not identified between SCoRS-K and any items in WCST. WCST is a representative of tests used to measure executive function, while SCoRS is a test to evaluate overall cognitive function (including but not exclusively executive function) in the everyday lives of people with schizophrenia, which may be the reason why there was no significant correlation. Keefe et al.13 conducted a study about the validity of the SCoRS. In that study, the SCoRS interviewers' scores had a significant relationship with cognitive function as measured by the MCCB. Another study found that the SCoRS global rating scores are related with the Brief Assessment of Cognition in Schizophrenia (BACS) composite scores and most of the BACS sub-test scores.14

The correlation between SCoRS-K results and social function, as measured by SQLS and SOFAS, showed significant correlations for interviewers' responses. The KSFS demonstrated correlations with SCoRS-K for a number of items, and significant correlations for the responses of patients, informants, and interviewers to the item about social activity (Table 5). This suggests that SCoRS-K is a test whose evaluation range includes measuring social functioning in daily life rather than merely a cognitive function test.

Both inter-rater reliability and internal consistency were sufficient, and reflected the results obtained by the original authors.9 Test-retest reliability at one month was high (p<0.001) except for patients' responses, which were relatively low (p=0.003). However, patients' self-reports are often not consistent, and informants' and interviewers' responses and global rating all had high test-retest reliability, indicating SCoRS-K's high stability.

This study verifies SCoRS-K's high internal consistency, test-retest reliability, and discriminant and concurrent validity. SCoRS-K will be a verified and useful tool for future studies of the degree of cognitive dysfunction affecting the everyday functioning of Korean patients with schizophrenia.