Validation of the Chinese Version of the Schizophrenia Cognition Rating Scale.

OBJECTIVE: The Schizophrenia Cognition Rating Scale (SCoRS) is an interview-based assessment tool for evaluating the cognitive deficit and daily functioning of patients with schizophrenia.
METHODS: Sixty-eight patients with schizophrenia and 68 age- and sex-matched healthy individuals were recruited to validate the Chinese version of SCoRS in this study. All participants underwent cognitive assessment using the SCoRS, which was verified by the Brief Assessment of Cognition in Schizophrenia (BACS), and the UCSD Performance-based Skills Assessment, Brief Version (UPSA-B). Patients with schizophrenia were additionally assessed using the Positive and Negative Syndrome Scale (PANSS).
RESULTS: SCoRS ratings reported by patients (SCoRS-S), those reported by the interviewer (SCoRS-I), and SCoRS global scores (SCoRS-G) showed significant correlation with all subscales of the BACS and the UPSA-B. On receiver operating characteristic curve analysis, SCoRS-S, SCoRS-I, and SCoRS-G significantly differentiated patients with schizophrenia from healthy controls. Moreover, SCoRS-S and SCoRS-I ratings showed positive correlation with the negative symptoms and general symptoms of PANSS.
CONCLUSION: The Chinese version of SCoRS showed good discriminant, concurrent, and external validity, suggesting that it is a useful and convenient tool for assessment of cognitive function among Mandarin-speaking patients with schizophrenia in clinical practice.

INTRODUCTION

Approximately 1% of the world’s population is affected by schizophrenia, a chronic and severe psychiatric disorder [1]. Cognitive impairment is one of the core features of schizophrenia that affects the life of these patients. Cognitive impairment is associated with impairment of daily functioning, including occupational life, routine activities, interpersonal relationships, and overall quality of life [2,3]. Therefore, assessment of cognitive impairment in schizophrenia is a key imperative to study of impact of the disease and to assess the outcomes of therapeutic intervention.

The National Institute of Mental Health Measurement and Treatment Research to Improve Cognition in Schizophrenia experts recommended that the MATRICS Consensus Cognitive Battery could apply in clinical treatment trials for assessing cognitive function [4]. However, this is difficult to implement in routine clinical practice due to time and resource constraints. The test battery requires much time to administer and is unavailable in most clinical settings. Substantial training is required to interpret the results of the test battery and the results may not reflect the opinion of the patient and/or the patient’s family. Interview-based assessment is needed as a co-primary measure to complement the traditional neuropsychological testing [5]. Interview-based assessment may offer several advantages over performance-based assessment, including a more accurate perception of the difficulties encountered by the patient in daily life and increase in the motivation of clinicians to manage cognitive deficits more effectively. Moreover, it facilitates a better appreciation of the eventual improvement in cognitive function by the patients and their family.

The Schizophrenia Cognition Rating Scale (SCoRS) was developed to address these needs. It is an interview-based cognitive assessment tool administered to patients and their informants [6]. It assesses the cognitive processes which are impaired in patients with schizophrenia, including attention, memory, reasoning and problem solving skills, working memory, language, and motor skills [7]. There are several advantages of interview-based methods including short administration time (approximately 15 min per interview) [6], low burden on patients, interviewers, and informants, good internal consistency, good inter-rater reliability, utility as a repeated measure, good correlation with performance-based measures of cognition and real-world functioning [8-12]. SCoRS measures global concepts, which makes it suitable for use in diverse cultural contexts. The tool has been translated into more than 20 languages. Studies have demonstrated the validity of the versions developed for use in Singapore, Japan, Iran, France, Brazil, Italy, and Korea [10,13-15].

Performance-based assessment tools, such as the Brief Assessment of Cognition in Schizophrenia (BACS) and the UCSD Performance-based Skills Assessment, Brief Version (UPSA-B), are reliable tools for delineating the cognitive deficits and functional impairment in patients with schizophrenia [10,16]. Thus these are suitable validity criteria for the interview-based assessments. BACS is a brief test battery which measures impairment of verbal memory, working memory, motor speed, verbal fluency, attention and processing speed, and executive function. It has been shown to strongly correlate with the real-world functioning of patients with schizophrenia [6,17]. UPSA-B is a performance-based measure of daily function with equivalently weighted measures of real-world activities in the financial and communication domains. UCSD Performance-based Skills Assessment (UPSA) performance was shown to significantly correlate with the severity of cognitive impairment and negative symptoms among patients with severe mental illness [18]. Both measures were adapted into Chinese version and have been validated in Mandarin-speaking populations [19-22].

Chinese is the 2nd most commonly spoken language in the world. More than 1 billion people across the world speak the Chinese language. Therefore, adaptation and validation of SCoRS into Chinese language would be useful for clinical trials in Chinese-speaking patients with schizophrenia. The current study aimed to develop a Chinese version of SCoRS and to evaluate its validity for illustrating cognitive dysfunction affecting the everyday functioning of patients with schizophrenia. In addition, we analyzed the correlation between SCoRS items (interview-based) and performance-based cognitive function assessments (BACS and UPSA-B).

METHODS

Study participants

This was a cross-sectional study conducted at the Chang Gung Memorial Hospital. The study protocol was approved by the Institutional Review Board (IRB No: 103-6849C). Written informed consent was obtained from all participants. Eligible patients with schizophrenia were enrolled at the outpatient department or the psychiatry ward of the Kaohsiung Chang Gung Memorial Hospital if they qualified the following inclusion criteria: 1) age range, 18–65 years; 2) patients who were assessed using the Chinese version of the Mini International Neuropsychiatric Interview (MINI) [translated from English by the Taiwan Society of Psychiatry [23]], and were diagnosed with schizophrenia as defined by the Diagnostic and Statistical Manual of Mental Disorders [24]; 3) patients with relatively stable psychotic symptoms after receiving a stable dose of antipsychotic drugs; 4) no history of major physical illnesses that can affect cognitive function; and 5) ability to speak Mandarin and read Chinese. A total of 68 patients with schizophrenia were recruited (mean age: 41.5 years, 57.4% male).

The control group consisted of 68 healthy individuals who were age- and sex-matched to the patients with schizophrenia. The healthy individuals were recruited from among the staff members of Kaohsiung Chang Gung Memorial Hospital and Keelung Chang Gung Memorial Hospital, as well as community volunteers in Kaohsiung City and Keelung City, Taiwan. The inclusion criteria were: 1) no history of major psychiatric disorders (e.g., psychotic disorders, bipolar disorder, major depressive disorder, organic mental disorders, and substance user disorder) confirmed by using Diagnostic and Statistical Manual of Mental Disorders, Fourth Edition criteria and the MINI; 2) no known systemic or neurological diseases that can affect cognitive function; and 3) ability to speak Mandarin and read Chinese.

Cognitive assessment

All participants (both patients with schizophrenia and healthy controls) were assessed using SCoRS, BACS, and UPSA-B by a well-trained research assistant.

The Schizophrenia Cognition Rating Scale

SCoRS is a 20-item interview-based tool for assessment of cognitive deficits and the degree to which these deficits affect the day-to-day functioning [6]. Interviewers were tasked to read out the questions stated in the scale accordingly. Information was drawn from three separate sources namely, the subject, an informant who had regular contact with subject, and the interviewer. Informant’s rating was obtained within 7 days of patient’s rating. The interviewer’s rating was drawn from the interviews with subject and informant, together with his or her own observation of the subject. Each item was rated on a 4-point scale; a rating of “not applicable” was allowed. Higher ratings reflect a greater degree of impairment. Following the three individual ratings, a global rating score, on a scale of 1 to 10 was generated by the interviewer. Ratings provided by each source (subject, informant, interviewer) were also computed by summing up the scores for all 20 items. The final component of SCoRS, the global score, represents the interviewer’s overall impression of the level of difficulty encountered by the subject in carrying out the day-to-day activities as a result of the attendant cognitive deficits (on a 10-point scale, 1=no impairment; 10=extreme impairment).

SCoRS has been adapted into Chinese by this manuscript’s corresponding author (LJW) and the second author (CKC) using the traditional forward–backward method to translate the SCoRS from English into Chinese. First, SCoRS was translated into Chinese by two bilingual senior psychiatrists (LJW and CKC). The Chinese version was then back-translated into English by an independent professional translator who was blinded to the original English version.

The Brief Assessment of Cognition in Schizophrenia

The cognitive functions of all participants were assessed using BACS [25]. BACS consists of a battery of tests that measure the most commonly impaired cognitive aspects which are closely related to the real-world functioning of patients with schizophrenia [6]. The test session, which lasts approximately 30 to 35 minutes, has a high completion rate among patients, and has high test–retest reliability. The sub-tests of BACS include the List Learning Test, Digit Sequencing Task, Token Motor Task, Category Instances Test, Controlled Oral Word Association Test, Symbol Coding, and Tower of London Test, which assess verbal memory, working memory, motor speed, verbal fluency, attention, processing speed, and executive function, respectively. We calculated a composite score after all of the sub-tests by comparing the performance of each patient on each measurement with the performance of healthy controls, which was defined as the z-score or t-score of the sum [17]. Previous studies have established the reliability and validity of the Chinese version of the BACS [20,21].

The UCSD Performance-based Skills Assessment, Brief Version

UPSA-B is modified version of the UPSA which is designed to assess the daily functioning of patients with severe mental illness [18]. UPSA-B comprises two subscales: the financial and communication aspects. In the financial part, the participants were asked to calculate a certain amount of real currency, make change, and write a bank cheque to pay a bill. The communication domain requires the participant to reschedule an appointment in a hospital by calling the directory service correctly to obtain a phone number. The raw scores of UBSA-B were standardized to a 100-point scale [26]. The Chinese version of UPSA-B has been developed and validated among mentally ill patients in Chinese populations [19].

Psychopathology assessments

The psychopathology of patients with schizophrenia was assessed using the Positive and Negative Syndrome Scale (PANSS) score [27] by senior psychiatrists (LJW, CFH, and YCH). The PANSS consisted of 30 items, each of which is rated on a 7-point Likert-scale, with higher scores indicating greater severity. Subscale scores were calculated using small sets of variables based on three domains: positive, negative, and general psychopathological symptoms [28]. Antipsychotic-related movement symptoms were assessed using the Abnormal Involuntary Movement Scale (AIMS) [29], Simpson–Angus Scale (SAS) [30], and Barnes Akathisia Rating Scale (BARS) [31].

Statistical analyses

Data analyses were performed using the statistical software package SPSS (Version 21.0; IBM Corp., Armonk, NY, USA). Data are presented as mean±standard deviation or frequency (%). Between-group differences with respect to categorical variables were assessed using the chi-squared test, while those with respect to continuous variables were assessed using the independent t test. Two-tailed p values <0.05 were considered indicative of statistical significance.

UPSA-B and BACS served as the validity criterion for SCoRS in this study. Spearman’s correlation was performed to analyze the correlation of SCoRS ratings with each BACS subscore and UPSA-B scale scores. In addition, Spearman’s correlation was also applied to analyze the relationship between SCoRS ratings and psychopathology in the schizophrenia group.

Receiver operating characteristic (ROC) curve analysis was performed to examine the ability of SCoRS ratings, UPSA-B scale scores, and BACS composite scores to differentiate between patients with schizophrenia and healthy controls. Sensitivity and specificity of the optimal cut-off SCoRS ratings were calculated.

RESULTS

A total of 68 patients with schizophrenia (schizophrenia group) and 68 age- and sex-matched healthy control subjects (control group) were recruited in this study. The demographic characteristics of the study population and the results of cognitive assessment are listed in Table 1. Compared with the control group, the schizophrenia group had lower education level, and worse cognitive function (all dimensions) measured using SCoRS, BACS, and UPSA-B.

Table 1.

Demographic characteristics and cognitive assessments of patients with schizophrenia and healthy control subjects

Variables	Patients (N=68)	Controls (N=68)	Statistics value
			χ2 or t	p-value
Age (yr)	41.54±10.16	41.56±10.11	-0.008	0.993
Sex			0.000	>0.999
Male	39 (57.4)	39 (57.4)
Female	29 (42.6)	29 (42.6)
Education (yr)	12.50±3.26	14.88±2.68	-4.654	<0.001[*]
SCoRS
Subjects total score	33.41±10.53	23.28±3.09	7.617	<0.001[*]
Interviewer total score	37.53±12.56	25.18±3.81	7.759	<0.001[*]
Global score	4.49±1.96	1.87±0.71	10.389	<0.001[*]
BACS performance
Verbal memory	25.63±11.43	41.57±9.44	-8.869	<0.001[*]
Working memory	15.22±5.09	23.09±9.59	-5.977	<0.001[*]
Motor speed	56.41±14.68	81.13±12.85	-10.407	<0.001[*]
Verbal fluency	23.34±8.17	34.10±7.70	-7.907	<0.001[*]
Attention & processing speed	37.66±14.23	61.03±12.63	-10.129	<0.001[*]
Executive function	11.96±6.13	17.04±3.33	-6.010	<0.001[*]
Composite score (Z-score)	-3.74±2.49	-0.03±0.95	-11.474	<0.001[*]
UPSA-B
UPSA-B financial	7.13±2.49	9.57±0.72	-7.736	<0.001[*]
UPSA-B communication	5.67±2.29	7.70±1.18	-6.503	<0.001[*]
PANSS
PANSS positive symptoms	17.18±4.92	NA	NA	NA
PANSS negative symptoms	19.52±7.30	NA	NA	NA
PANSS general symptoms	39.18±9.51	NA	NA	NA
AIMS	0.82±1.93	NA	NA	NA
SAS	2.03±2.25	NA	NA	NA
BARS	0.69±1.41	NA	NA	NA

Values are presented as mean±standard deviation or number (%).

p<0.05.

SCoR, the Schizophrenia Cognition Rating Scale; BACS, the Brief Assessment of Cognition in Schizophrenia; UPSA-B, the UCSD Performance-based Skills Assessment, Brief Version; PANSS, Positive and Negative Syndrome Scale; AIMS, the Abnormal Involuntary Movement Scale; SAS, Simpson–Angus Scale; BARS, Barnes Akathisia Rating Scale; NA, not available

Table 2 shows the correlation between SCoRS ratings, each BACS subtest, and UPSA-B scores. SCoRS ratings reported by subjects (SCoRS subject) showed a significant correlation with all dimensions of BACS (verbal memory, working memory, motor speed, verbal fluency, attention, processing speed, executive function, and BACS composite score) and the financial and communication domains of UPSA-B. Similarly, SCoRS scores rated by the interviewer (SCoRS interviewer) and SCoRS global scores showed a significant correlation with all dimensions of BACS and the financial and communication domains of UPSA-B.

Table 2.

Correlation of SCoRS scores, scores of BACS and UPSA-B among patients with schizophrenia

	SCoRS subjects		SCoRS interviewer		SCoRS global score
	r	p-value	r	p-value	r	p-value
BACS performance
Verbal memory	-0.253	0.038[*]	-0.395	0.001[*]	-0.291	0.017[*]
Working memory	-0.310	0.010[*]	-0.410	0.001[*]	-0.374	0.002[*]
Motor speed	-0.297	0.014[*]	-0.260	0.033[*]	-0.345	0.004[*]
Verbal fluency	-0.330	0.006[*]	-0.381	0.001[*]	-0.372	0.002[*]
Attention & processing speed	-0.302	0.012[*]	-0.371	0.002[*]	-0.328	0.007[*]
Executive function	-0.282	0.020[*]	-0.389	0.001[*]	-0.309	0.011[*]
Composite score (Z-score)	-0.392	0.001[*]	-0.485	<0.001[*]	-0.404	0.001[*]
UPSA-B
UPSA-B financial	-0.285	0.018[*]	-0.416	<0.001[*]	-0.498	<0.001[*]
UPSA-B communication	-0.265	0.019[*]	-0.340	0.005[*]	-0.387	0.001[*]

p<0.05.

SCoR, the Schizophrenia Cognition Rating Scale; BACS, the Brief Assessment of Cognition in Schizophrenia; UPSA-B, the UCSD Performance-based Skills Assessment, Brief Version

Figure 1 illustrates the ROC curve analysis using SCoRS ratings, UPSA-B scale scores, and BACS composite scores to distinguish patients with schizophrenia from healthy controls. The SCoRS ratings reported by subjects [area under the curve (AUC): 0.892, p<0.001], SCoRS scores rated by the interviewer (AUC: 0.890, p<0.001), and SCoRS global score (AUC: 0.893, p<0.001) significantly differentiated patients with schizophrenia from healthy controls. The ROC output indicated that SCoRS subject ratings of 26.5 (sensitivity: 79.1%; specificity: 83.6%), SCoRS interviewer ratings of 28.5 (sensitivity: 80.6%; specificity: 82.1%), and SCoRS global score of 2.5 (sensitivity: 77.6%; specificity: 86.6%) had the best discriminant validity, respectively. Moreover, the UPSA-B financial scale (AUC: 0.836, p<0.001) and UPSA-B communication scale (AUC: 0.801, p<0.001) and BACS composite scores (AUC: 0.967, p<0.001) significantly differentiated patients with schizophrenia from healthy controls.

Figure 1.

The ROC analysis for subtests of SCoRS, UPSA-B scales and BACS composite score in discriminating patients with schizophrenia and healthy controls. SCoRS ratings reported by subjects (SCoRS subjects, AUC: 0.892, p<0.001); SCoRS ratings rated by the interviewer (SCoRS interviewer, AUC: 0.890, p<0.001); SCoRS global score (AUC: 0.893, p<0.001); UPSA-B financial scale (AUC: 0.836, p<0.001); UPSA-B communication scale (AUC: 0.801, p< 0.001); BACS composite scores (AUC: 0.967, p<0.001). ROC, receiver operating characteristic; SCoR, the Schizophrenia Cognition Rating Scale; BACS, the Brief Assessment of Cognition in Schizophrenia; UPSA-B, the UCSD Performance-based Skills Assessment, Brief Version; AUC, area under the curve.

Table 3 shows the correlation between SCoRS ratings and psychopathology assessments in the schizophrenia group. We found a positive correlation of the SCoRS subject ratings with the negative symptoms and general symptoms of PANSS, the severity of AIMS and SAS. The SCoRS interviewer ratings showed a positive correlation with the negative and general symptoms of PANSS, and the scores of SAS. Finally, the SCoRS global scores showed a positive correlation with the general symptoms of PANSS.

Table 3.

Correlation SCoRS scores rated by subjects and interviewer, and psychopathology assessments among patients with schizophrenia

	SCoRS subjects		SCoRS interviewer		SCoRS global score
	r	p-value	r	p-value	r	p-value
PANSS
PANSS positive symptoms	0.078	0.529	0.149	0.229	0.077	0.534
PANSS negative symptoms	0.339	0.005[*]	0.361	0.003[*]	0.231	0.060
PANSS general symptoms	0.385	0.001[*]	0.386	0.001[*]	0.286	0.019[*]
AIMS	0.241	0.049[*]	0.067	0.588	0.060	0.631
SAS	0.406	0.001[*]	0.369	0.002[*]	0.234	0.057
BARS	-0.137	0.270	-0.122	0.326	-0.174	0.158

p<0.05.

SCoR, the Schizophrenia Cognition Rating Scale; PANSS, Positive and Negative Syndrome Scale; AIMS, the Abnormal Involuntary Movement Scale; SAS, Simpson–Angus Scale; BARS, Barnes Akathisia Rating Scale

DISCUSSION

This study showed a significant correlation of the Chinese version of SCoRS ratings with all dimensions of BACS and UPSA-B. SCoRS showed good concurrent validity with standard neurocognitive batteries, and good discriminating validity for differentiating patients with schizophrenia from their age- and sex-matched healthy subjects. Our findings suggest that SCoRS is a useful interview-based measure of cognition related to functioning among patients with schizophrenia.

SCoRS showed satisfactory concurrent validity using BACS and UPSA-B as the validity criteria. SCoRS ratings by patients themselves (SCoRS-subjects), those reported by the interviewer (SCoRS interviewer), and global scores all showed a significant correlation with all dimensions of BACS. These results are generally consistent with previous studies, which showed a strong correlation of SCoRS ratings (interview-based measurements) with performance-based measurements [7,10,12,13]. UPSA-B is the major battery used for functional assessment and to distinguish patients with major psychiatry disorders from healthy subjects [22,32]. We found a significant correlation of SCoRS ratings with both the financial and communication domains of UPSA-B. This indicates that SCoRS effectively evaluates cognitive function that are assessed using UPSA-B.

On ROC curve analysis, SCoRS subject, SCoRS interviewer, and SCoRS global score effectively distinguished between patients with schizophrenia and healthy subjects (approximate AUC: 0.89). Relative to patients’ self-report, the interviewers’ ratings appear to have greater validity and reliability, and are less influenced by placebo effects [10]. However, in the current study, SCoRS ratings by the patients themselves as well as those by the interviewers showed similar sensitivity to delineate cognitive impairment in schizophrenia and significant correlations with BACS and UPSA-B. It is noteworthy that SCoRS ratings reported by an informant (someone who is in regular contact with the patient) were lacking in this study. Previous works regarding to SCoRS have found that without an informant, the correlations with BACS performance are low and there is less sensitivity to treatment response [6,11]. However, the results in the current study suggest that although the ratings of informants were absent, SCoRS ratings by interviewers still effectively assess patients’ cognitive function. SCoRS showed better discriminating validity than UPSA-B (Figure 1); however, BACS composite scores (AUC: 0.967) were better than SCoRS in differentiating patients with schizophrenia from healthy controls. Relative to BACS, SCoRS is convenient to administer and is time-saving tool for assessing cognitive function of patients with schizophrenia.

Finally, our results showed a moderate correlation between the Chinese version of SCoRS and the PANSS negative and general subscale. Patients who presented greater negative and general psychopathological symptoms were more likely to experience higher levels of disability in day-to-day functioning. This finding is consistent with a previous study [33] that showed cross-sectional correlations between the severity of negative and cognitive symptoms. Severity of extrapyramidal syndrome (EPS) is associated with poorer scores in tests of cognition and patients with Parkinsonism symptoms may have worse cognitive performance than non-parkinsonian patients [34,35]. The correlation between the SCoRS and the scores of SAS observed in our study is consistent with the result of the previous study in which Parkinsonism symptoms were found to impede the test taking ability. This finding suggested that the presence of EPS should be considered during assessment of cognitive impairment.

Some limitations of this study should be considered. First, the sample size in this study was relatively small. Second, we did not examine the test–retest or inter-rater reliability. A previous study which examined the test–retest reliability of different source of informant of SCoRS11 reported 0.77 for patients, 0.81 for informant, and 0.82 for interviewer, respectively. Further studies are required to examine test–retest reliability of Chinese version of SCoRS. Third, complete administration of SCoRS should involve an interview with an informant, and SCoRS that requires informant input appears to yield good reliability and validity. However, because the caregivers were not available for all patients with schizophrenia, informant data was not obtained in this study.

SCoRS effectively characterizes the level of difficulties encountered by subjects in conducting day-to-day activities as a result of the attendant cognitive deficits. In this study, the Chinese version of SCoRS showed good discriminant, concurrent, and external validity, suggesting that it is a useful and convenient tool for assessment of cognitive function among Mandarin-speaking patients with schizophrenia in clinical practice.