Reliability and Validity of the Korean Version of Children's Depression Inventory 2 Short Version as a Screening Tool: A Comparison With the Standard 28-Item Version.

OBJECTIVE: To investigate the reliability and validity of the Korean version of Children's Depression Inventory 2 Short Version (CDI 2:S) in comparison with its full-length version (CDI 2) as a screening tool for depressive youth.
METHODS: A total of 714 children from the community and 62 psychiatric patients were enrolled in this study. The Korean version of the Kiddie Schedule for Affective Disorders and Schizophrenia Present and Lifetime Version (K-SADS-PL-K) served as the reference standard for computing receiver operating characteristic (ROC) curves. To evaluate the ability of the CDI 2 and CDI 2:S to discriminate major depressive disorders, areas under the curves (AUCs) were compared. To investigate psychometric properties of the CDI 2:S, internal consistency was calculated and confirmatory factor analysis was conducted.
RESULTS: For the CDI 2, the cutoff at 20 yielded the best balance between sensitivity (83%) and specificity (91%). For the CDI 2:S, the cutoff point of 10 resulted in high sensitivity (82%) and high specificity (93%). The short form was proven to be as sensitive and specific as the CDI 2. Further analyses confirmed that the CDI 2:S also had good reliability and validity.
CONCLUSION: The CDI 2:S, a sensitive and brief form of the CDI 2, may serve as a better option in time-constrained psychiatric settings.

INTRODUCTION

Depression is a worldwide burden [1]. Depression developed at younger ages has more debilitating effects on emotional, functional, and interpersonal domains than that emerged in adulthood [2-5]. Depressed youths tend to socially withdraw themselves, which consequently make them vulnerable to severe depression [6-9]. The prevalence of child and adolescent depression has increased after the COVID-19 pandemic. According to a systematic review by Loades et al. [10], children and adolescents are more likely to feel loneliness and have experienced dramatic disruptions to their daily lives after enforced social isolation. Considering that this early development stage is a critical period that shapes one’s social competence and even long-term socioeconomic status [6], the prevention and management of child depression are necessary. However, research on depression assessment has mostly focused on the stage of adulthood [11,12], hindering the identification of depressed youth.

Self-report questionnaire is the most widely used method to detect a depressive disorder [13]. Self-report inventories help clinicians to screen for depression and improve the recognition rates. The past studies have reported that several screening tools for child depression have high reliability, validity, and diagnostic utility [5]. However, most of these tools (e.g., CES-D, BDI-II) were originally developed for use in adults, whereas the clinical profile is different between youth and adult. Although Reynolds Adolescent Depression Scale (RADS) was originally designed for detecting adolescent depression, it consists of 30 items and takes at least 10 minutes to complete, which decreases time efficiency in a time-constraint clinical setting. Thus, we propose a need for the evidence-based screening tool with two critical points: 1) originally developed for child and adolescent population, 2) short administration time.

The Children’s Depression Inventory (CDI), which has been developed by Kovacs in 1992 is one of the most widely used scales for assessing depressive symptoms in children and adolescents [14-16]. The CDI has been translated into more than 20 languages and currently in use worldwide. For instance, it has been validated in Arabic [17], Spanish [18], Dutch [19], German [20], Greek [21], Italian [22], Malaysian [23], Chinese [24], Japanese [25] version. Also, the standardization studies have been conducted among diverse populations.

Since it was developed in 1992, Kovacs developed CDI 2 with updated normative sample for controlling birth cohort effect. In the CDI 2, some items were changed to better capture core symptoms of child depression and delivering clearer questions [26].

To date, the CDI:S (Kovacs 1992) has been relatively widely used in clinical and research settings [20,27]. However, the CDI 2:S has not been yet validated as it is relatively new. It was originally intended to screen youth at risk for depression. However, cut-off scores have not been proposed yet. Therefore, we conducted an initial yet unique study of the CDI 2:S and compared it with its full-length version (CDI 2) using a representative sample of Korean children and adolescents and clinical sample from a tertiary care hospital.

METHODS

Participants

We used 2 cohorts: community-based general population and clinical sample. The community sample were recruited from schools in 3 different cities (Seoul, Cheongju, and Jinju) in South Korea (for a detailed description) [28]. Participants were between 7 and 17 years old, possessing Korean reading and comprehension proficiency. They completed an informed consent and parental consent, and a sociodemographic questionnaire. Cases with missing demographic data or item responses were excluded in the analysis. In our study, we excluded the cases aged under 11 in community sample in order to match the age range of the clinical group. Since developmental stage is critical to emergence of depressive symptoms, we attempted to reduce selection bias.

Our clinical sample was recruited from the psychiatry at a tertiary care outpatient clinic from January 2021 to May 2021. Outpatient children and adolescents (11–17 years) meeting the DSM-IV diagnostic criteria for major depressive disorder (MDD) and/or dysthymic disorder (DD) as determined by the Korean version of the Kiddie Schedule for Affective Disorders and Schizophrenia Present and Lifetime Version (K-SADS-PL-K) were enrolled in the study (n=62). Informed consent was obtained from parents/children under protocols approved by the Institutional Review Board of Samsung Medical Center (IRB No. 2021-05-144-001). Patients were given the CDI 2 and asked to complete it on the day they visited the clinic. Total of 62 participants were included in the analyses. Thirty cases had single diagnosis of MDD (82%), or DD (18%). However, 20 participants had comorbid psychiatric disorders: attention-deficit/hyperactivity disorder (60%), generalized anxiety disorder (20%), and social phobia (20%).

Whereas comorbid disorders with MDD or DD were allowed in order to increase statistical power with large sample size, MDD or DD was the primary disorder. Exclusion criteria for clinical samples were as follows: 1) full-scale intelligence quotient (FSIQ) lower than 50, 2) patients with psychotic symptoms, 3) primarily diagnosed with bipolar I or II disorders. Eight participants were excluded from the study: 4 cases with psychotic symptoms, 3 cases diagnosed with bipolar ii disorders, 1 case with intellectual disability. Two cases were missing demographic information, thus were not included in the analyses.

Measures

Children Depression Inventory 2 (CDI 2) and its short form (CDI 2:S)

The CDI 2 consists of 28 items measuring depressive symptoms. Each item is scored on a scale of 0 (no symptom) to 2 (clinically significant), with higher score indicating more severe depressive symptoms (0–56). Participants choose the statement that best describes how they have felt for the past 2 weeks. The average time to complete the CDI 2 is within 15 minutes. The abbreviated version of the CDI 2 (CDI 2:S) is developed for a screening purpose. Kovacs omitted items with lower inter-items correlation, and item ‘suicidal idea’. The CDI 2:S consists of 12 items, and total scores range from 0 to 24. The administration time for the CDI 2:S is less than 5 minutes.

Korean version of the Kiddie Schedule for Affective Disorders and Schizophrenia Present and Lifetime Version (K-SADS-PL-K)

K-SADS-PL-K has been used as a gold standard measure to confirm diagnosis of mood disorders according to DSM-IV-R criteria [29]. The K-SADS- PL-K has good reliability and validity for diagnosing psychiatric disorders in children and adolescents [30]. We used the K-SADS-PL-K to confirm diagnoses. Eleven clinical psychologist residents interviewed the child and primary caregiver, and later confirmed by the three child/adolescent psychiatrists interviewed the child and primary caregiver.

Statistical analysis

Descriptive statistics, reliability, concurrent validity analyses were performed using IBM SPSS 23 (IBM Corp., Armonk, NY, USA). To analyze all demographic data, we conducted descriptive statistics and frequency analysis. Diagnosis of major depressive disorder was performed according to the K-SADS-PL-K which served as an external criterion. Diagnostic utilities of the CDI 2 and the CDI 2:S was assessed by the area under the receiver operating characteristic (ROC) curve (AUC). If the value of the AUC was between 0.7 and 0.8, it was considered acceptable. If AUC ≥0.8, it was considered excellent [31]. Then we used sensitivity, specificity, and predictive values to calculate the optimal cut-off score that could discriminate between the group with depression and the group without depression group. Optimal cut-off values were those that demonstrated a maximal Youden Index (Y) [32] indicating the highest combination of sensitivity and specificity. Additionally, assuming that the CDI 2:S could perform as well as the full-length version, we tested factorial validity of the CDI 2:S. The lavaan package in R statistical software (R Studio, Boston, MA, USA) [33] was used to conduct confirmatory factor analysis (CFA). The factorial structure was investigated using confirmatory factor analysis with weighted least squares estimator having a mean and variance (WLSMV). WLSMV is known to perform better than maximum likelihood method with fewer than five response categories as it does not assume normally distributed variables [34]. Fit indices including comparative fit index (CFI) [35], tucker lewis index (TLI) [36], and root mean square error of approximation (RMSEA) [37] were used to evaluate the overall model fit. CFI and TLI ≥0.90 [38] and RMSEA value of ≤0.08 [39] indicated a reasonably good fit.

RESULTS

Demographic characteristics and diagnostic results

Detailed information about the descriptive results of demographic characteristics and the scores of the CDI 2 and the CDI 2:S are presented in Table 1. Gender difference in depression is a common phenomenon [40]. The results showed that regardless gender, depression group significantly scored high in the CDI 2 and the CDI 2:S (all p<0.001). We additionally conducted independent sample t-test analysis between genders in each group separately. Girls scored higher than boys for both the CDI 2 and the CDI 2:S in a community sample (t(709.19)=-3.41, p=0.001, t(710.21)=-3.20, p=0.002, respectively). However, in the depression group, a significant difference between gender is only observed in the CDI 2 (t(52)=-2.11, p=0.04). There was no significant difference in the CDI 2:S across gender (t(52)=-1.60, p=0.12).

Table 1.

Descriptive statistics of demographic characteristics and scores of Children Depression Inventory 2 (CDI 2) and its short-form (CDI 2:S)

	Normal (N=714)	Depression group (N=54)	p value
Age	13.85 (1.87)	15.37 (1.71)	0.001
Sex (male/female) (%)	330 (46.2)/384 (53.8)	18 (33.3)/36 (66.7)	0.067
CDI 2	10.43 (6.75)	28.74 (10.08)	<0.001
Male	9.52 (5.91)	24.78 (11.32)	<0.001
Female	11.21 (7.32)	30.72 (8.91)	<0.001
CDI 2:S	4.78 (3.29)	13.43 (4.54)	<0.001
Male	4.36 (2.91)	12.06 (5.30)	<0.001
Female	5.14 (3.56)	14.11 (4.01)	<0.001

Standard deviation is shown in parentheses. p value for sex was calculated by chi-square test, p values for age, CDI 2 and CDI 2:S scores were calculated by independent sample t-test

Diagnostic validity

We conducted ROC analyses for the clinical group vs. community sample. Results of the ROC analysis are presented in Figure 1. For the CDI 2, the AUC value was 91.6 (95% CI: 86.5–96.7) and the optimal screening cut-off score was 20 (Y=0.739) (Table 2). The positive predictive value (PPV) was 35.2% and the negative predictive value (NPV) was 98.6% in our sample. When we recalculated PPV and NPV according to disease prevalence in Korean child/adolescent sample (2.3%; National Mental Health Survey, 2009), PPV and NPV were 8.8%, 96.1%, respectively. For the CDI 2:S, the AUC value was 0.918 (95% CI: 0.866–0.971) and the optimal screening cut-off score was 10 (Y=0.749) (Table 3). In our clinical sample, PPV was 39.1% and NPV was 98.6%. Considering a disease prevalence of 2.3%, PPV and NPV were 9.6% and 95.8%, respectively.

Figure 1.

Receiver operating characteristic (ROC) curve for the Children Depression Inventory (CDI) 2 short form and Children Depression Inventory 2. AUC, area under the receiver operating characteristic curve; CI, confidence interval.

Table 2.

Psychometric investigation and clinical validity of the Children Depression Inventory 2

	Sensitivity	Specificity	PPV	NPV	LR+	LR-	Youden’s index	Accuracy
14	0.91	0.77	22.5	95.7	3.96	0.12	0.62	77
15	0.91	0.80	19.5	95.7	4.55	0.11	0.67	80
16	0.89	0.82	17.6	95.7	4.94	0.13	0;70	82
17	0.85	0.84	15.6	95.8	5.31	0.18	0.71	84
18	0.85	0.87	12.7	95.8	6.38	0.20	0.70	87
19	0.83	0.88	11.7	95.8	6.92	0.19	0.72	88
20[*]	0.83[*]	0.91[*]	8.8[*]	96.1[*]	8.11[*]	0.30[*]	0.72[*]	91[*]
21	0.82	0.92	7.8	95.9	10.25	0.20	0.74	92
22	0.79	0.94	5.9	95.9	13.17	0.22	0.74	94
23	0.76	0.95	4.9	96.0	15.20	0.25	0.73	95
24	0.74	0.96	3.0	96.1	18.50	0.27	0.71	95
25	0.72	0.97	2.9	96.1	24.00	0.29	0.70	96

the point that maximizes Youden’s index.

PPV, positive predictive value; NPV, negative predictive value; LR+, positive likelihood ratio; LR-, negative likelihood ratio

Table 3.

Psychometric investigation and clinical validity of the Children Depression Inventory 2 short form

	Sensitivity	Specificity	PPV	NPV	LR+	LR-	Youden’s index	Accuracy
6	0.89	0.74	34.8	95.7	2.55	0.14	0.55	65
7	0.89	0.80	25.5	95.7	3.41	0.15	0.63	74
8	0.83	0.86	19.6	95.7	4.44	0.14	0.69	80
9	0.83	0.90	14.0	95.8	5.83	0.19	0.69	86
10[*]	0.82[*]	0.93[*]	9.6[*]	95.8[*]	8.50[*]	0.18[*]	0.74[*]	90[*]
11	0.69	0.95	6.4	95.9	12.38	0.20	0.75	93
12	0.63	0.98	4.7	96.2	14.39	0.33	0.64	95
13	0.56	0.99	2.5	96.3	24.98	0.38	0.60	97
14	0.52	0.99	1.1	96.5	49.58	0.45	0.54	98

the point that maximizes Youden’s index.

PPV, positive predictive value; NPV, negative predictive value; LR+, positive likelihood ratio; LR-, negative likelihood ratio

Internal consistency and factorial validity of the CDI 2:S

Cronbach’s alpha was 0.90 for the CDI 2 and 0.84 for the CDI 2:S. Correlation between the 28-item CDI 2 and the 12-item CDI 2:S was significant (r=0.248, p<0.001). The factorial validity of the CDI 2 was discussed in Kim et al. [28]. Since no previous factorial analyses has been done on the short form, we tested the factorial validity of this short form. Results of CFA analysis indicated that the CDI 2:S was a unidimensional measure. Test statistics were all over the acceptable fit (RMSEA= 0.04, CFI=0.97, TLI=0.98). Factor loadings were higher than 0.4 for all items in the CDI 2:S. The descriptive statistics and the measurement invariance results are presented in Supplementary Tables 1 and 2 (in the online-only Data Supplement).

DISCUSSION

The aim of this study was to investigate clinical utility of the Children Depression Inventory 2 short form and determine if the CDI 2:S could perform as well as the CDI 2 to discriminate depressive youth in Korea. The CDI 2:S yielded a good diagnostic accuracy. The cut-off score ≥10 showed an excellent specificity (93%) and maintained a satisfactory sensitivity (82%) for the CDI 2:S. The CDI 2 also showed a good diagnostic accuracy with balancing values for sensitivity (83%) and specificity (91%) at an optimal cut-off at 20.

This is the first study to investigate the reliability and validity of the CDI 2:S. Although the notion of depression is multifaceted, the author of the CDI and the CDI 2 intended its use as a unidimensional measure. The CFA analysis showed that the CDI 2:S reliably measured ‘depression’ construct. Factor loadings for most items were greater than 0.4, suggesting that these items reliably measured the unidimensional construct of depression. This is consistent with previous findings showing that the CDI short version [41] and the CDI [42] should be interpreted as an univocal measure. Ahlen and Ghaderi [41] tested rigorously tested psychometric properties of the CDI short form (CDI-S) with one-factor, two-factor, and the bi-factor models, and concluded that the CDI-S is a unidimensional measure. They noted that the new short version CDI 2:S would have a stronger predictive validity than does the CDI-S. The investigation of the CDI2:S in an independent sample adds further evidence of its validity of scale scores.

Cut-off scores to detect depression in Korean population are known to be somewhat higher than those in Western countries [43]. Since no previous study has reported the cut-off score for the CDI 2, we reviewed findings of the original version. In a German sample of medically ill children, a 10-item CDI short form was as sensitive as a 26-item CDI, but was less specific than the full version [20]. They suggested a cutoff value of 12 for the CDI and a cutoff value of 3 for the CDI:S. On the other hand, in a Korean sample, a score of 20 for the CDI was chosen as the optimal screening cutoff score, with a sensitivity of 0.83 and a specificity of 0.89 [43]. Cultural differences may help explain this phenomenon. Academic achievements seem to put more burden on Asian students than on Western students [44]. Confucianism places value on education and family dynamics, which may foster academic achievements [45]. However, it simultaneously creates performance pressure, which leads to subjective feelings of excessive stress and low self-esteem [46]. A recent network analysis study of the CDI in a large sample of school-aged children in Korea [47] has shown that schoolwork difficulty may cause depression when these children are frustrated with personal achievement in school. Therefore, we postulate that even though not diagnosed with depressive disorders, Korean youths commonly score high in the CDI 2.

As discussed earlier, early detection of childhood depression may improve an individual’s life functioning [6]. Primary health care may play a critical role in opening a door for entry to psychiatric care for people with possible depression [48]. One empirical study has demonstrated that implementation of an evidence-based depression screening in primary care can improve the screening rate from 0% to 75% over a 2-month project period [49]. Among those subjects, 6.1% were diagnosed with depression and started pharmacotherapy. Such a finding calls for an action for primary care providers to adopt evidence-based screening tools. Patients with unknown somatic complaints, sleep disorder, and appetite problem may be at an increased risk of depression [50,51]. In fact, the prevalence rate before adolescence period is generally low. Many patients with a major depressive disorder start with anxiety symptoms. Those diagnosed with anxiety disorder during childhood, and such anxiety symptoms can develop into depressive symptoms which later meet the criteria for major depressive episode. By screening, primary physicians may refer them to mental health care. We suggest the use of the CDI 2 for a thorough evaluation of emotional symptoms after screening as the CDI 2 has four subscales: negative mood/physical symptoms, negative self-esteem, interpersonal problems, and ineffectiveness. Likewise, clinicians may benefit from a multidimensional assessment when specifying immediate intervention targets.

This study has several limitations. First, since the sampling period was limited to five months, the number of clinical sample with depression was relatively small. This could have reduced statistical power. Second, the clinical sample was recruited from a single tertiary care outpatient clinic, limiting generalizability. Third, inter-rater reliability and test-retest assessment were not analyzed, which may reduce accuracy and coherent validity of the data. Moreover, there was a time gap between the CDI 2 and the K-SADS-PL administration. The administration date was different for each individual due to an outpatient psychiatric setting. Most participants completed the CDI 2, and were interviewed with K-SADS-PL almost one month later. For this short period, possible environmental changes (i.e. increased social support from peers and families) helping them overcome depressive symptoms had occurred. At the same time, stressful events could have worsened depressive symptoms. Furthermore, while we attempted to increase a statistical power by including patients with comorbid psychiatric disorders, sample could be heterogeneous. Depression has high comorbidity in nature, but discriminant validity was not explored in the study. Therefore, the results may fail to provide an evidence that the CDI 2:S identify distinct areas of psychopathology in depression. Further validation is required using other self-report, behavioural measures and reports from other respondents such as parents or teachers. Multitrait-multimethod (MTMM) analysis would help increase validity [52].

In previous studies that investigated the diagnostic utility of the CDI [43], participants were recruited from schools or communities. Although they used specific structured interview tools to diagnose depression, participants were originally sampled from non-clinical settings. This sampling bias might contribute to false positives, with possibility of subjects being misclassified as MDD. However, in our research, a large sample of children and adolescents was used and every subject in the clinical group underwent rigorous structured interview by clinical psychologists. The power of psychometric properties of the CDI 2:S in diagnostic validity was based on a structured diagnostic interview, which increases methodological power.

To conclude, this is an initial study reporting sensitivity, specificity, and AUC parameters of the CDI 2 cut-off scores. Our study confirmed the use of both the CDI 2 and the CDI 2:S in a psychiatric setting in Korea. Although the short form was suggested to use as a screening tool in primary care settings, only a few studies have examined the utility of the CDI short form [41]. We expect the use of the CDI 2 and the CDI 2:S in various settings. It might be used as a screening tool for a group of youth to determine what proportion of students may require further psychological evaluations. Also, teachers may use the CDI 2:S to detect depressive students to further refer them to clinics and allow them to benefit from further support. In clinical settings, CDI 2:S can be used as a 5-minute quick screening tool. Our results suggest that the 12-item CDI 2:S is an attractive alternative to the 28-item CDI 2 in a time-limited psychiatric outpatient setting. As a screening tool, those with positive screening results should further take clinical interviews to rule out false positives and verify depression diagnoses.