Clinical Validation of the Psychotic Depression Assessment Scale, Hamilton Depression Rating Scale-6, and Brief Psychiatric Rating Scale-5: Results from the Clinical Research Center for Depression Study.

OBJECTIVE: The aim of this study was to validate the psychotic depression assessment scale (PDAS), which includes the six-item melancholia subscale from the Hamilton depression rating scale (HAMD-6) and the five-item psychosis subscale from the brief psychiatric rating scale (BPRS-5). Data from the Clinical Research Center for Depression (CRESCEND) study, which is a 52-week naturalistic trial, were analyzed.
METHODS: Fifty-two patients with psychotic depression from the CRESCEND study met our inclusion criteria. The patients underwent the following psychometric assessments: the PDAS, including HAMD-6 and BPRS-5, the clinical global impression scales, the HAMD, the positive symptom subscale, and the negative symptom subscale. Assessments were performed at the baseline and then at weeks 1, 2, 4, 8, 12, 24, and 52. Spearman correlation analyses were used to assess the clinical validity and responsiveness of the PDAS.
RESULTS: The clinical validity and responsiveness of the PDAS, including HAMD-6 and BPRS-5, were acceptable, with the exception of the clinical responsiveness of the PDAS for positive symptoms and the clinical responsiveness of BPRS-5 for negative symptoms.
CONCLUSION: The clinical relevance of the PDAS has been confirmed and this clinical validation will enhance its clinical utility and availability.

INTRODUCTION

Psychotic depression (PD) is clinically distinct from non-psychotic depression (non-PD).12 PD have an increased vulnerability to bipolar disorder,3 a higher rate of parental history of bipolar disorder,4 greater risk of death,5 higher recurrence rate,6 higher rates of suicide, suicidal attempt, and suicidal ideation,7 poorer cognitive performance, including verbal learning, visual learning, and processing speed,8 and lower rate of co-morbidity with physical disease9 in comparison to patients with non-PD. In terms of neurobiological cause, the dysregulation of the hypothalamo-pituitary-adrenal axis,1011 a reduction in bilateral hippocampal volume due to the high cortisol states,12 and decreased activity of dopamine β-hydroxylase1314 have all been associated with PD rather than non-PD. Previously, the psychotic features of PD were regarded as secondary to the greater severity of depressive symptoms.1516 However, a recent study showed that greater severity of depressive symptoms is a sufficient but not a necessary condition for the presence of psychotic features in PD.17 Furthermore, the significant association between psychotic mania, PD and mixed affective episodes has been demonstrated in a large study of 14,529 patients with bipolar disorder.18 Consistent with these findings, PD has been redefined for the 11th revision of the International Classification of Diseases (ICD-11) as the “meta-syndrome” across unipolar psychotic depression and bipolar psychotic depression.1 In addition, according to the 5th edition of the Diagnostic and Statistical Manual of Mental Disorders (DSM-5), the statement “with psychotic features” can be used in the diagnosis and documentation of major depression and dysthymia regardless of severity, and the previous version that mentioned the predominance of “mood-congruence” over “mood-incongruence” with regard to psychotic features has been omitted.19

Although PD symptoms include both depressive and psychotic domains, the severity of PD is usually evaluated using an assessment scale restricted to only one particular domain. The psychotic depression assessment scale (PDAS), however, is a unique assessment tool as it covers both the depressive and psychotic symptoms of PD.202122 The PDAS includes both the six-item melancholia subscale from the Hamilton depression rating scale (HAMD-6; depressive mood, guilt feelings, work and activities, psychomotor retardation, psychic anxiety and general somatic symptoms),23 and the five-item psychosis subscale from the brief psychiatric rating scale (BPRS-5; hallucinatory behavior, unusual thought content, suspiciousness, emotional withdrawal and blunted affect).24 The PDAS can be used to evaluate the outcome measures from randomized clinical trials, and identify the patients with PD from those with non-PD.25262728 The reasons for employing the composite scale of the HAMD-6 and BPRS-5 were as follows: since the HAMD-6 is superior to the HAMD in terms of clinical validity and unidimensionality, it was used as a subscale in the PDAS to measure the severity of depressive symptoms. In addition, the combination of the HAMD-6 and BPRS-5 has shown the greatest clinical validity and responsiveness of the various composite scales of the HAMD-6 and BPRS-X (X=1-7).20 A previous study showed that, when assessed by the PDAS, patients with PD (n=53) scored higher on the items of depressed mood, hallucinatory behavior, unusual thought content, suspiciousness, blunted affect, and emotional withdrawal than those with non-PD (n=441).26 Moreover, in the same depressed patients, use of a cut-off score of 1, based on a receiver operating characteristics (ROC) curve, showed that the BPRS-5 detected PD with sensitivity of 71.2%, and specificity of 87.2%. Hence, in terms of the PDAS, the BPRS-5 has been proposed as a potential screening tool for detecting PD.27

However, to our knowledge, the clinical responsiveness of the PDAS has not been evaluated over a long period. Hence, the current study aims to validate the PDAS for a follow-up period of 52 weeks, using data from the Clinical Research Center for Depression (CRESCEND) study, which is the first large, prospective, naturalistic study of patients with depressive disorders in South Korea.26272829303132 Thus, for the current study, we identified patients with PD from the CRESCEND study, and then evaluated the clinical validity and responsiveness of the PDAS, including HAMD-6 and BPRS-5, by comparing the results against several assessment scales including the clinical global impression (CGI) scales,33 HAMD,23 the positive symptom subscale (PSS),34 and the negative symptom subscale (NSS).34

METHODS

Study overview

The CRESCEND study is described elsewhere;24252627 briefly, 1,183 patients with first-onset or recurrent depressive disorder (defined as major depression, dysthymia, or other non-specified depressive disorders) and who were beginning treatment were enrolled at 18 clinical centers (16 university-affiliated hospitals and two general hospitals) across South Korea, during the period January 2006 to August 2008. In terms of the use of pharmacotherapy including antidepressants and other psychotropic medication, the interventions took the form of a naturalistic study. At baseline and then at weeks 1, 2, 4, 8, 12, 24, and 52, the patients were evaluated on the assessment scales.

The institutional review board of the Catholic Medical Center (receipt number: CUMC07U001) approved the CRESCEND study. Patients volunteered for the study, and written informed consent was obtained. Under the supervision of clinical psychiatrists at each of the clinical centers, trained and certified research coordinators collected and evaluated the clinical data and assessment scale scores.

Study subjects

In this study, PD was defined as a depressive disorder accompanied with psychotic symptoms, regardless of the severity of the depressive disorder, as proposed Keller et al.35 and Østergaard et al.12 Hence, our inclusion criteria were as follows: 1) age ≥18 and ≤65 years; 2) diagnosis of major depressive disorder (MDD), dysthymic disorder, or non-specified depressive disorder based on DSM-IV36 and confirmation of the diagnosis with the structured clinical interview for DSM-IV axis I disorders (SCID-I)37; 3) presence of definite delusions and/or hallucinations; 4) availability of fully completed HAMD23 and BPRS assessments24; and 5) total score on the PDAS assessment ≥8.25 Fifty-two patients with PD from the CRESCEND study met our inclusion criteria and were analyzed in the current study.

PDAS, HAMD-6, and BPRS-5

The scores for most items in the HAMD ranged from 0 to 4 (Likert scale), and the scores for all items in the BPRS ranged from 1 to 7 (Likert scale). Therefore, the BPRS-5 scores were converted according to the following formula: 1=0; 2−3=1; 4−5=2; 6=3; and 7=4.1921 Østergaard et al.38 previously reported the clinical validity, responsiveness and unidimensionality of the PDAS, including the HAMD-6 and BPRS-5 components, in patients with PD. The HAMD-6 is concerned with the severity of depressive symptoms, whereas the BPRS-5 is concerned with the severity of psychotic symptoms.21 In addition, the PDAS has been proposed as a promising diagnostic tool for PD.2728 Østergaard et al.25 proposed the following severity classification: mild PD (a PDAS score of 8–15), moderate PD (a PDAS score of 16–23), and severe PD (a PDAS score >23).

Other assessment scales

The CGI-severity (CGI-S) scale,33 CGI-improvement (CGI-I) scale,33 HAMD,23 PSS,34 and NSS34 were used to measure global severity, global improvement, depressive symptoms, positive symptoms, and negative symptoms, respectively, at the baseline and again at weeks 1, 2, 4, 8, 12, 24, and 52. As proposed by Lachar et al.34 the PSS score was defined by the summed score of conceptual disorganization, suspiciousness, hallucinatory behavior, unusual thought content, and disorientation, and the NSS score was defined by the summed score of emotional withdrawal, motor retardation, and blunted affect. Higher scores on the CGI-S, CGI-I, HAMD, PSS, and NSS were consistent with greater severity, less improvement or more exacerbation, and poorer social functions. All the assessment scales were formally translated into Korean and their Korean versions were formally standardized previously.3940 Twice per year the research coordinators from the CRESCEND study met to discuss the CGI-S, CGI-I, HAMD, PSS, and NSS to improve the reliability of the assessment scales.

Statistical analyses

The clinical validity of the PDAS was evaluated using the Spearman correlation analyses of PDAS versus CGI-S, HAMD, PSS, and NSS at the baseline and again at weeks 1, 2, 4, 8, 12, 24, and 52, and the Spearman correlation analyses of endpoint-baseline change in the PDAS score versus the CGI-I score and endpoint-baseline change in the CGI-S, HAMD, PSS, and NSS scores at weeks 1, 2, 4, 8, 12, 24, and 52. The clinical validities of the HAMD5 and BPRS-5 were also evaluated using Spearman correlation analyses. Spearman's ρ range was classified as reflecting very high (±0.9 and above), high (±0.7 to <0.9), moderate (±0.5 to <0.7), low (±0.3 and <0.5) and little (±<0.3) association.41

Statistical significance was set at p<0.05 (two-tailed) for all tests. All statistical analyses were performed with SPSS 21 for Windows (SPSS Inc., Chicago, IL, USA).

RESULTS

Baseline characteristics of the included patients

As shown in Table 1, the mean age of the 52 patients with PD included in the study was 41.1 [standard deviation (SD)=15.1] years at baseline, the mean scores on the assessment scales were as follows: PDAS, 14.0 (SD=4.7); HAMD-6, 11.4 (SD=3.2); BPRS-5, 2.6 (SD=2.6); CGI-S, 5.1 (SD=0.8); HAMD, 22.2 (SD=5.6); PSS, 6.8 (SD=2.6); and NSS, 7.1 (SD=4.2).

Table 1

Baseline characteristics of the study subjects (N=52)

BPRS-5: 5-item psychosis subscale, CGI-I: Clinical Global Impression-Improvement, CGI-S: Clinical Global Impression-Severity, HAMA: Hamilton Anxiety Rating Scale, HAMD: Hamilton Depression Rating Scale, HAMD-6: 6-item melancholia subscale, NSS: Negative Symptom Subscale, PDAS: Psychotic Depression Assessment Scale, PSS: Positive Symptom Subscale, SOFAS: Social and Occupational Functioning Assessment Scale, SSI-Beck: Beck Scale for Suicidal Ideation

Most of the patients were female (69.2%) and were also outpatients (67.3%), and most patients reported delusions (67.3%) and suicidal ideation (65.4%). The percentages of mild, moderate and severe PD were 59.6, 36.6, and 3.8%, respectively.

Correlation between the PDAS assessment and the CGI/HAMD/PSS/NSS assessments

As shown in Table 2, the correlation between PDAS and CGI-S assessments was significant at all time points, and Spearman's ρ ranged between 0.47 and 0.81, suggesting a low-to-high association. In addition, the correlation between the endpoint-baseline change in the PDAS and CGI-I scores was significant at all time points except for weeks 12 and 52, and Spearman's ρ ranged between 0.40 and 0.74, suggesting a low-to-high association. The correlation between the endpoint-baseline change in the PDAS score and the endpoint-baseline change in the CGI-S score was significant at all time points except for weeks 24 and 52, and Spearman's ρ ranged between 0.34 and 0.61 suggesting a low-to-moderate association.

Table 2

Correlation between PDAS score and CGI/HAMD/PSS/NSS scores

*p<0.05, **p<0.01. CGI-I: Clinical Global Impression-Improvement Scale, CGI-S: Clinical Global Impression-Severity Scale, HAMA: Hamilton Anxiety Rating Scale, HAMD: Hamilton Depression Rating Scale, NSS: Negative Symptom Subscale, PDAS: Psychotic Depression Assessment Scale, PSS: Positive Symptom Subscale, SOFAS: Social and Occupational Functioning Assessment Scale, SSI-Beck: Beck Scale for Suicidal Ideation

The correlations between PDAS and the HAMD assessments were significant at all time points, and Spearman's ρ ranged between 0.63 and 0.92, suggesting a moderate-to-very-high association. The correlations between the endpoint-baseline change in the PDAS score and the endpoint-baseline changes in the HAMD scores were also significant at all time points, and Spearman's ρ ranged between 0.65 and 0.82, suggesting a moderate-to-very-high association.

The correlations between PDAS and PSS assessments were significant at all time points except for week 52, and Spearman's ρ ranged between 0.42 and 0.77, suggesting a low-to-high association. However, the correlations between the endpoint-baseline change in the PDAS score and the endpoint-baseline changes in the PSS scores were not significant at any time point, and Spearman's ρ ranged between 0.08 and 0.44, suggesting little-to-low association. The correlations between the PDAS and NSS assessments were significant at all time points except for week 52, and Spearman's ρ ranged between 0.42 and 0.82, suggesting a low-to-high association. The correlations between the endpoint-baseline change in the PDAS score and the endpoint-baseline changes in the NSS score at weeks 1, 4, 8, and 12 were significant, and Spearman's ρ ranged between 0.21 and 0.60 overall, suggesting a small-to-moderate association.

Correlations between the HAMD-6 assessment and the CGI/HAMD/PSS/NSS assessments

As shown in Table 3, the correlations between HAMD-6 and CGI-S assessments were significant at all time points, and Spearman's ρ ranged between 0.49 and 0.93, suggesting a low-to-very-high association. In addition, the correlations between the endpoint-baseline change in the HAMD-6 score and the CGI-I scores were significant at all time points except for weeks 12 and 52, and Spearman's ρ ranged between 0.32 and 0.69, suggesting a low-to-moderate association. The correlations between the endpoint-baseline change in the HAMD-6 score and the endpoint-baseline changes in the CGI-S score were significant at all time points except week 52, and Spearman's ρ ranged between 0.43 and 0.58, suggesting a low-to-moderate association.

Table 3

Correlation between HAMD-6 score and CGI/HAMD/PSS/NSS scores

*p<0.05, **p<0.01. CGI-I: Clinical Global Impression-Improvement Scale, CGI-S: Clinical Global Impression-Severity Scale, HAMA: Hamilton Anxiety Rating Scale, HAMD: Hamilton Depression Rating Scale, HAMD-6: 6-item melancholia subscale, NSS: Negative Symptom Subscale, PSS: Positive Symptom Subscale, SOFAS: Social and Occupational Functioning Assessment Scale, SSI-Beck: Beck Scale for Suicidal Ideation

The correlations between HAMD-6 and HAMD assessments were significant at all time points, and Spearman's ρ ranged between 0.78 and 0.95, suggesting a high-to-very-high association. The correlations between the endpoint-baseline change in the HAMD-6 score and the endpoint-baseline changes in the HAMD scores were significant at all time points, and Spearman's ρ ranged between 0.71 and 0.93, suggesting a high-to-very-high association.

The correlations between HAMD-6 and PSS assessments were significant at all time points except for baseline and at weeks 24 and 52, and Spearman's ρ ranged between 0.18 and 0.64, suggesting a low-to-moderate association. However, the correlations between the endpoint-baseline change in the HAMD-6 score and the endpoint-baseline changes in the PSS score were not significant at any time point, and Spearman's ρ ranged between −0.08 and 0.19, suggesting little association. The correlations between HAMD-6 and NSS assessments were significant at all time points except for week 52, and Spearman's ρ ranged between 0.34 and 0.70, suggesting a low-to-high association. However, the correlation between the endpoint-baseline change in the HAMD-6 score and the endpoint-baseline change in the NSS score were not significant at any time point, and Spearman's ρ ranged between −0.07 and 0.25 suggesting little association.

Correlation between the BPRS-5 assessment and the CGI/HAMD/PSS/NSS assessments

As shown in Table 4, the correlations between the BPRS-5 and CGI-S assessments were significant at all time points except for weeks 12, 24, and 52, and Spearman's ρ ranged between 0.14 and 0.74, suggesting a low-to-moderate association. In addition, the correlations between the endpoint-baseline change in the BPRS-5 score and the CGI-I scores were significant at weeks 1 and 4, and Spearman's ρ ranged between −0.15 and 0.64, suggesting a low-to-moderate association. The correlations between the endpoint-baseline change in the BPRS-5 score and the endpoint-baseline changes in the CGI-S scores were significant at week 4, and Spearman's ρ ranged between −0.12 and 0.42, suggesting a little-to-low association.

Table 4

Correlation between BPRS-5 score and CGI/HAMD/PSS/NSS scores

*p<0.05, **p<0.01. BPRS-5: 5-item psychosis subscale, CGI-I: Clinical Global Impression-Improvement Scale, CGI-S: Clinical Global Impression-Severity Scale, HAMA: Hamilton Anxiety Rating Scale, HAMD: Hamilton Depression Rating Scale, NSS: Negative Symptom Subscale, PSS: Positive Symptom Subscale, SOFAS: Social and Occupational Functioning Assessment Scale, SSI-Beck: Beck Scale for Suicidal Ideation

The correlations between the BPRS-5 and HAMD assessments were significant at weeks 1, 2, 4, and 8, and Spearman's ρ ranged between 0.20 and 0.95, suggesting a little-to-very high association. The correlations between the endpoint-baseline changes in the BPRS-5 score and the endpoint-baseline changes in the HAMD score were significant at week 4, and Spearman's ρ ranged between −0.04 and 0.54, suggesting a little-to-moderate association.

The correlations between the BPRS-5 and PSS assessments were significant at all time points and Spearman's ρ ranged between 0.59 and 0.99, suggesting a moderate-to-very high association. However, the correlations between the endpoint-baseline change in the BPRS-5 score and the endpoint-baseline change in the PSS score were significant at all time points except for week 24, and Spearman's ρ ranged between 0.33 and 0.96, suggesting little association. The correlation between the BPRS-5 and NSS assessments was significant at all time points and Spearman's ρ ranged between 0.63 and 0.99, suggesting a moderate-to-very-high association. However, the correlations between the endpoint-baseline change in the BPRS-5 score and the endpoint-baseline changes in the NSS score were significant at weeks 1 and 8, and Spearman's ρ ranged between 0.23 and 0.61, suggesting little-to-moderate association.

DISCUSSION

Firstly, we have shown that the PDAS has acceptable concurrent validity, as the PDAS is moderately-to-highly associated with the CGI-S, HAMD, PSS, and NSS at most time points studied. In addition, the PDAS has acceptable clinical responsiveness, as the endpoint-baseline change in the PDAS score is moderately-to-highly associated with the CGI-I score and also with the endpoint-baseline change in the CGI-S, HAMD, and NSS scores at most time points studied. In terms of positive symptoms, however, the PDAS showed unacceptable clinical responsiveness, as the endpoint-baseline change in the PDAS score is not significantly associated with the endpoint-baseline change in the PSS score at any time point.

Secondly, we have shown the concurrent and divergent validity of the HAMD-6 assessment, as the HAMD-6 score is moderately-to-highly associated with the CGI-S and HAMD scores, and has low-to-moderate association with the PSS and NSS scores at most time points. The HAMD-6 has acceptable clinical responsiveness, as the endpoint-baseline change in the HAMD-6 scores is associated with the CGI-I score, and also with the endpoint-baseline change in the CGI-S and HAMD scores at most time points. Furthermore, the HAMD-6 assessment is not significantly associated with the PSS or NSS assessments.

Thirdly, we have shown the concurrent and divergent validity of the BPRS-5 assessment, as the BPRS-5 score is moderately-to-highly associated with PSS and NSS scores at all time points and has low-to-moderate association with CGI-S and HAMD scores at relatively early time points. In terms of positive symptoms, the BPRS-5 has acceptable clinical responsiveness, as the endpoint-baseline change in the BPRS-5 score has low-to-moderate association with the endpoint-baseline change in the PSS score. Conversely, in terms of global severity and negative symptoms, the BPRS-5 assessment has unacceptable clinical responsiveness, as the endpoint-baseline change in the BPRS-5 score is only associated a little with the CGI-I score and with the endpoint-baseline change in the CGI-S and NSS scores.

In the current study, the mean age (41.1 years) of the patients is younger, and the proportion of female patients (69.2%) is higher than patients included in a previous validation study (53.8 years and 52%, respectively).38 This is significant because a previous study revealed that age of onset of PD is differentially associated with severity of depressive, anxious, and gastrointestinal symptoms.42 Although one study has shown that gender is not significantly associated with treatment outcome in 259 patients with MDD with psychotic features,43 another study has shown that men with PD have more severe anxiety symptoms, more prevalent suicidal ideation, and more hallucinatory behavior than women with PD.28 Hence, age, gender, and other baseline characteristics might be associated with a greater proportion of mild PD at baseline in the present study. In the current study, the baseline mean scores of the PDAS, HAMD-6, and BPRS-5 assessments were 14.0 (SD=3.2), 11.4 (SD=3.2), and 2.6 (SD=2.6), respectively, whereas in a previous validation study38 the scores were 29.5 (SD=11.3), 18.9 (SD= 6.0), and 10.6 (SD=5.8), respectively. The sequenced treatment alternatives to relieve depression (STAR*D) study44 showed that patients with MDD with auditory-visual hallucinations have poorer remission rates than those without auditory-visual hallucinations in Whites and Blacks, but not Latinos, and patients with paranoid ideation have poorer remission rates than those without paranoid ideation in Whites only. Moreover, a 12-week, double-blind, randomized controlled trial for 259 patients with MDD with psychotic features [the study of pharmacotherapy for psychotic depression (STOP-PD)]45 proposed that impaired insight into delusions is a predictor for poorer treatment outcome. Consistent with these findings, it can be speculated that psychotic symptoms are associated with the clinical course of PD. In comparison to the previous validation study,38 we report a remarkably small BPRS-5 score in the current study; this corresponds to the changes in severity of overall and depressive symptoms and to the small change in severity of psychotic symptoms. Hence, the unacceptable clinical responsiveness of the PDAS assessment for positive symptoms and the BPRS-5 assessment for negative symptoms might be a consequence of less severe psychotic symptoms at the baseline in the present study. Thus, further studies that include patients with PD who have more severe psychotic symptoms are needed.

There are a number of limitations of the current study. Firstly, there is potential variability in the way that the assessments were scored by different people. However, the inter-rater reliability of the PDAS, HAMD-6, and BPRS-5 assessments was not evaluated Secondly, the severity of psychosis symptoms in patients with PD is relatively low in our study. Hence, less severe psychosis symptoms might be associated with the unacceptable clinical responsiveness of the PDAS and BPRS-5 assessments. Thirdly, the CRESCEND study is not a randomized controlled trial but rather a naturalistic trial, and pharmacotherapy for PD was not standardized. Fourthly, the sample size is relatively small.

Despite the limitations, this study describes the clinical validity and responsiveness of the PDAS, HAMD-6, and BPRS-5 assessments, with the exception of the clinical responsiveness of the PDAS for positive symptoms and the BPRS-5 for negative symptoms. More specifically, the PDAS assessment quantifies global, depressive, and negative symptom severity, the HAMD-6 quantifies global and depressive symptom severity, and the BPRS-5 quantifies positive symptom severity. Our findings will enhance the clinical utility and availability of the PDAS, HAMD-6, and BPRS-5 assessments. Further studies of patients with PD and a wider range of psychosis symptoms will further confirm the clinical validity of the PDAS assessment.