Reliability and validity of the Thai version of the Calgary Depression Scale for Schizophrenia.

AIM: The purpose of this study was to assess the reliability and validity of the Thai version of the Calgary Depression Scale for Schizophrenia (CDSS) for the evaluation of depression in patients with schizophrenia.
METHODS: Sixty patients with schizophrenia according to Diagnostic and Statistical Manual of Mental Disorders: Fourth Edition; Text Revision (DSM-IV-TR) criteria were recruited to the study The Thai version of the CDSS, the Montgomery-Åsberg Depression Rating Scale (MADRS), the Hamilton Depression Rating Scale, 17-item version (HDRS-17), and the Positive and Negative Syndrome Scale (PANSS) were administered. A major depressive episode diagnosed by a psychiatrist according to the DSM-IV-TR was used as a gold standard.
RESULTS: The internal consistency of the Thai version of the CDSS was very good (Cronbach's alpha = 0.869). The inter-rater reliability was found to be in substantial agreement with the intra-class correlation coefficient of 0.979. The test-retest reliability over a period of 3 days was high, with an intra-class correlation coefficient of 0.861. The Thai version of the CDSS showed significant correlations with the MADRS (r = 0.887), the HDRS-17 (r = 0.865), and the depression item of the Positive and Negative Syndrome Scale (PANSS-G6) (r = 0.833). The areas under the receiver operating characteristic curve of the CDSS, MADRS, HDRS-17, and PANSS-G6 against the DSM-IV-TR criteria for major depressive episode were 0.993, 0.954, 0.966, and 0.933, respectively. The optimal cut-off score to discriminate between depressed and non-depressed patients was 6/7, with a sensitivity of 92.31% and specificity of 97.87%.
CONCLUSION: The Thai version of the CDSS is a reliable and valid measure for the evaluation of depression in Thai patients with schizophrenia.

Introduction

Depression is common in the course of schizophrenia. The reported prevalence rate of depression in schizophrenia ranges widely from 10% to 83% depending on the clinical setting, assessment, and duration of study.1,2 Although it leads to impairment, decreased functioning, re-hospitalization, and suicide, depression is frequently under-recognized and untreated.3–5

Depression in schizophrenia can be categorized into three subtypes: (1) depressive symptoms secondary to organic factors (caused by medications, substance abuse, or underlying medical problems); (2) nonorganic depressive symptoms occurring with an acute psychotic episode; and (3) nonorganic depressive symptoms occurring without acute psychotic symptoms (such as prodromal, post-psychotic, and negative symptoms).6 There is difficulty in distinguishing between depression and negative symptoms or extrapyramidal side-effects. Therefore, assessment of depression in schizophrenia is a challenge.

The Calgary Depression Scale for Schizophrenia (CDSS) is a clinician-rated scale specifically developed for the assessment of depression in schizophrenia. It initially consisted of eleven items, but was reduced to nine items after validity studies.7,8 The nine assessment items are (1) depression, (2) hopelessness, (3) self-depreciation, (4) guilty ideas of reference, (5) pathological guilt, (6) morning depression, (7) early wakening, (8) suicide, and (9) observed depression. The CDSS is derived from the Hamilton Depression Rating Scale (HDRS) and the Present State Examination. A study of the English version of the CDSS in 150 patients with schizophrenia showed a high level of inter-rater reliability (intra-class correlation coefficient [ICC] = 0.89) and significant correlations with HDRS as well as the Brief Psychiatric Rating Scale. The internal reliability was good (Cronbach’s alpha = 0.79).10 Furthermore, the CDSS appeared to measure depression, rather than negative or extrapyramidal side-effects.10

Although there are a number of instruments that can be used for the assessment of depression in schizophrenia, including the Montgomery-Åsberg Depression Rating Scale (MADRS), the HDRS, and the depression item of the Positive and Negative Syndrome Scale (PANSS-G6), a systematic review found that the CDSS is superior than other depressive instruments in terms of reliability and validity in patients with schizophrenia.11 It was found to be the best depressive instrument to differentiate depressive symptoms from other symptoms of schizophrenia (divergent validity) and was the least likely to misdiagnose or overlook cases of depression (predictive validity).11

The CDSS is currently available in 35 languages. It has been used in research and in clinical settings with patients with schizophrenia. The purpose of this study was to assess the reliability and validity of the Thai version of the CDSS for the evaluation of depression in Thai patients with schizophrenia.

Methods

The translation of the Thai version of the CDSS

The Thai version of the CDSS was translated from the original CDSS by a bilingual psychiatrist. Each of the nine items were carefully checked and adjusted to retain the purpose of the original version and ensure they communicated effectively and were suitable for the Thai culture. The final version was then translated back into English by another experienced bilingual psychiatrist. The re-translated English version was then examined by the author of the original CDSS, Professor Donald Addington, to determine whether the meaning was congruent with the original version. Consensus was achieved by three experienced Thai psychiatrists and a psychiatric nurse. Pilot testing was conducted in 15 patients with schizophrenia.

Subjects and procedures

This study was conducted with 60 participants at an out-and in-patient clinic of a university hospital in Chiang Mai, Thailand. Inclusion criteria were: Thai-speaking, aged 18 years or over, and diagnosis of schizophrenia according to the Diagnostic and Statistical Manual of Mental Disorders: Fourth Edition; Text Revision (DSM-IV-TR).12 Those with neurological disorders and mental retardation were excluded from the study. The study was approved by the Ethics Committee of the Faculty of Medicine, Chiang Mai University, Thailand. After the details of the study were fully explained, written informed consent was obtained from all patients before the study.

A major depressive episode diagnosed by a psychiatrist according to DSM-IV-TR criteria was used as a gold standard. Each patient was assessed with the following scales: the Thai version of the CDSS, the MADRS,13 the Hamilton Depression Rating Scale, 17-item version (HDRS-17),14 and the Positive and Negative Syndrome Scale (PANSS).15

The joint inter-rater reliability test was performed in ten patients by two raters in a room, with each rater taking turns either interviewing or observing. The test-retest validity of the Thai version of the CDSS was conducted by one rater making two ratings at intervals of 3 days or more.

Statistical analysis

SPSS software (v 16.0; IBM, Armonk, NY, USA) was used for analysis. Internal consistency was tested with Cronbach’s alpha coefficient. Inter-rater and test-retest reliability were analyzed by determining the ICC. Construct validity was investigated using the Pearson product-moment correlation coefficient between the Thai version of the CDSS scores and the scores obtained with the MADRS, the HDRS-17, the PANSS-G6, the positive and negative subscales of the PANSS, the general psychopathology subscale of the PANSS, and the total PANSS score. To examine the diagnostic validity of the four depressive scales (the Thai version of the CDSS, the MADRS, the HDRS-17, and the PANSS-G6), the areas under the receiver operating characteristic (ROC) curve were obtained.

Results

Sample characteristics

Of the 60 participants, 33 (55%) were female, ten (16.7%) were married, and 25 (41.6%) had completed at least secondary education. The mean age of participants was 40.4 ±13.6 years old. The prevalence of a major depressive episode according to DSM-IV-TR criteria was 21.7%. Five patients (8.3%) were taking antidepressants. The mean scores ± standard deviation were: 3.88 + 4.05 (minimum = 0, maximum = 22) for the Thai version of the CDSS, 6.87 ± 7.73 for the MADRS, and 7.07 ± 5.93 for the HDRS-17.

For the PANS S scores, the mean scores were 45.12 ± 12.74 for the PANSS total, 10.92 ± 4.06 for the PANSS positive subscale, 11.32 ± 4.06 for the PANSS negative subscale, the mean score for PANSS-G6 was 2.07±1.27 and 22.88 ± 6.05 for the PANSS general psychopathology subscale.

Reliability

Internal consistency

The internal consistency of the Thai version of the CDSS was very good (Cronbach’s alpha = 0.869). All of the items appeared to correlate well with the scale overall, although the item-total correlation was low for early wakening (C7). Moreover, all items except early wakening (C7) appeared to be essential components of the scale (Table 1).

Table 1

The internal reliability analysis and the inter-rater reliability of the Thai version of the Calgary Depression Scale for Schizophrenia

CDSS-T item	Mean value ± SD	Mean value of item deleted	Item-total correlation	Cronbach’s alpha coefficient with item deleted	ICC	P value
CI depression	0.65 ± 0.78	3.23	0.726	0.843	0.949	<0.001
C2 hopelessness	0.32 ± 0.65	3.57	0.662	0.850	0.904	<0.001
C3 self-depreciation	0.33 ± 0.57	3.55	0.675	0.850	0.923	<0.001
C4 guilty ideas of reference	0.50 ± 0.70	3.38	0.580	0.858	1.000	<0.001
C5 pathological guilt	0.57 ± 0.59	3.32	0.602	0.856	1.000	<0.001
C6 morning depression	0.50 ± 0.70	3.38	0.758	0.840	1.000	<0.001
C7 early wakening	0.52 ± 0.77	3.37	0.303	0.888	1.000	<0.001
C8 suicide	0.12 ± 0.32	3.77	0.522	0.867	0.800	0.002
C9 observed depression	0.38 ± 0.59	3.50	0.773	0.841	1.000	<0.001
Total	3.88 ± 4.05			0.869	0.979	<0.001

Abbreviations: CDSS-T, Thai version of the Calgary Depression Scale for Schizophrenia; ICC, intra-class correlation coefficient; SD, standard deviation.

Inter-rater reliability

Inter-rater reliability was tested on a sample of ten patients with schizophrenia. The overall inter-rater reliability was found to be in substantial agreement with the ICC of 0.979 (P < 0.001). The ICC for inter-rater reliability for each item was also high, ranging from 0.800 to 1.000 (Table 1). The test-retest reliability showed almost perfect agreement, with an ICC of 0.861 (P < 0.001) for the total score of the Thai version of the CDSS.

Validity

The Thai version of the CDSS showed significant correlations with the MADRS (r = 0.887, P < 0.01), the HDRS (r = 0.865, P < 0.01), and the PANSS-G6 (r = 0.833, P < 0.01). There were also significant correlations between the Thai version of the CDSS and the PANSS scores (positive, negative, general psychopathology, and total score) (Table 2). The areas under the ROC curve of the CDSS, MADRS, HDRS-17, and PANSS-G6 against the DSM-IV-TR criteria for major depressive episode were 0.993, 0.954, 0.966, and 0.933, respectively (Figure 1). The optimal cut-off scores to discriminate between depressed and non-depressed patients was 6/7, with a sensitivity of 92.31% and specificity of 97.87% (Table 3).

Table 2

Correlations between depression rating scales and the Positive and Negative Syndrome Scale (PANSS) (Pearson product-moment correlation)

	CDSS-T	MADRS	HDRS-17	PANSS-G6	PANSS-P	PANSS-N	PANSS-GP	PANSS-T
CDSS-T	1.00
MADRS	0.887*	1.00
HDRS-17	0.865*	0.901*	1.00
PANSS-G6	0.833*	0.831*	0.847*	1.00
PANSS-P	0.670*	0.657*	0.669*	0.670*	1.00
PANSS-N	0.621*	0.698*	0.658*	0.635*	0.559*	1.00
PANSS-GP	0.709*	0.756*	0.803*	0.835*	0.732*	0.777*	1.00
PANSS-T	0.748*	0.791*	0.810*	0.812*	0.845*	0.865*	0.956*	1.00

Note:*Correlation is significant at the 0.01 level (two-tailed).

Abbreviations: CDSS-T, Thai version of the Calgary Depression Scale for Schizophrenia; HDRS-17, Hamilton Depression Rating Scale - 17 item version; MADRS, Montgomery-Åsberg Depression Rating Scale; PANSS-G6, depression item of the Positive and Negative Syndrome Scale; PANSS-P, Positive and Negative Syndrome Scale positive subscale; PANSS-N, Positive and Negative Syndrome Scale negative subscale; PANSS-GP, Positive and Negative Syndrome Scale general psychopathology subscale; PANSS-T, Positive and Negative Syndrome Scale total score.

Figure 1

The receiver operating characteristic (ROC) curve of the depression rating scales and the Diagnostic and Statistical Manual of Mental Disorders: Fourth Edition; Text Revision12 (DSM-IV-TR) for major depressive episode diagnosis.

Notes: The ROC curves of the CDSS-T, MADRS, HDRS-17, and PANSS-G6 against the DSM-IV-TR criteria for major a depressive episode were 0.993, 0.954, 0.966, and 0.933, respectively.

Abbreviations: CDSS-T, Thai version of the Calgary Depression Scale for Schizophrenia; HDRS-17, Hamilton Depression Rating Scale-17 item version; MADRS, the Montgomery-Åsberg Depression Rating Scale; PANSS-G6, depression item of the Positive and Negative Syndrome Scale.

Table 3

Performance of the Thai version of the Calgary Depression Scale for Schizophrenia in screening for major depressive episodes in patients with schizophrenia

Cut-off score	Major depressive episode
	Cases, n	Sensitivity	Specificity	Positive predictive value	Negative predictive value
4/5	0/13	100.00	80.85	59.09	100.00
5/6	0/13	100.00	87.50	68.42	100.00
6/7	1/12	92.31	97.87	92.31	97.87
7/8	4/9	69.23	100.00	100.00	92.16

Discussion

This study demonstrates that the Thai version of the CDSS is a reliable and valid measure for the evaluation of depression in Thai patients with schizophrenia. The internal consistency of the Thai version of the CDSS, which indicates the agreement among the individual components of the measure, was very good (Cronbach’s alpha = 0.869). This result supports the high internal consistency of the CDSS, which was also found in other versions, including the Japanese version (Cronbach’s alpha = 0.82), the Greek version (Cronbach’s alpha = 0.87), and the French version (Cronbach’s alpha = 0.82).16–18 All the items appeared to correlate well with the scale overall, although the item-total correlation was low in early wakening (C7). This finding was similar to that reported in the French version.16 The small item-total correlation in C7 might be due to use of benzodiazepines, which were highly prescribed in this population, as well as the low severity of depressive symptoms and psychotic symptoms found in the study participants.

The overall inter-rater reliability of the Thai version of the CDSS was found to be in substantial agreement (ICC = 0.979) and the ICC for each item was also high. These findings are similar to those for other versions of the CDSS.16,18,19 The test-retest was done at intervals of 3 days or more. Although depression in patients with schizophrenia might change over time, the test-retest reliability for the total score of the Thai version of the CDSS in this study was high, with an ICC of 0.861.

The mean total score of Thai version of the CDSS in this study significantly correlates with those of the MADRS, the HDRS-17, and the PANSS-G6. These results are similar to those of Addington et al, who reported significant relationships between the CDSS, HDRS, and Beck Depression Inventory7 Our study also found significant correlations between the Thai version of the CDSS and the PANSS score (positive, negative, general psychopathology, and total score). The relationship between the CDSS and PANSS scores identified in previous studies has been controversial. Lançon et al found significant correlations between the CDSS, the PANSS positive subscale, the PANSS general psychopathology subscale, and PANSS total score,20 while Addington et al reported a significant correlation between the CDSS and the PANSS negative subscale.7 Additionally, some items of the CDSS, including guilty ideas of reference (C4) and early wakening (C7) were found to be significantly correlated with the PANSS positive subscale when the patients were in remission.21 These correlations might represent the contamination of the CDSS by the psychotic symptoms in patients with schizophrenia. Nevertheless, there was no significant correlation between the CDSS and the PANSS positive and negative subscale, and the Scale for the Assessment of Negative Symptoms (SANS) score on admission.21

The DSM-IV-TR criteria for major depressive episode were utilized as a gold standard following the original study of Addington et al, which used DSM-III-R criteria.10 It was found that the area under the ROC curve of the CDSS against the DSM-IV-TR criteria for major depressive episode was higher than that of the other depressive measures. This indicates that the Thai version of the CDSS is more effective than the MADRS, HDRS-17, and PANSS-G6 in evaluating depressive symptoms in patients with schizophrenia. The cut-off point of 6/7 for detecting major depressive episodes provided a high sensitivity (92.31%) and specificity (97.87%) as well as a high positive predictive value (92.31%) and negative predictive value (97.87%).

Some limitations should be taken into account when interpreting the findings of this study. First, the depression and psychotic symptoms of the patients with schizophrenia in this study were moderately low. This problem has been reported in studies looking at other versions of the CDSS and may limit the use of the scale in patients with severe depressive and psychotic symptoms.16 However, it may have an advantage in detecting borderline cases of depression. Second, the particular sample characteristics from a university hospital in Thailand might reduce generalization to other patients with schizophrenia in different settings. Third, some of the patients were being treated with antidepressants and benzodiazepines during the evaluation, which may have interfered with some items in the CDSS. As such, further study of the reliability and validity with a wide range of depressive and psychotic symptoms in different settings, such as in a psychiatric hospital, should be conducted to confirm the results of this study.

Conclusion

The Thai version of the CDSS is a reliable and valid measure for the evaluation of depression in Thai patients with schizophrenia with the optimal cut-off score of seven or greater. The scale is recommended for the assessment of depression in patients with schizophrenia in research and clinical practice. Further study on the reliability and validity of the version with a wide range of depressive and psychotic symptoms in different settings should be conducted.