Preliminary Validation Study of the Korean Version of the DSM-5 Level 2 Cross-Cutting Symptom Measure: Depression and Irritability for Parents of Children Aged 6-17 Years.

Objectives: This study investigated the reliability and validity of the Korean version of the Diagnostic and Statistical Manual of Mental Disorders, Fifth Edition (DSM-5) Level 2 Cross-Cutting Symptom Measure-Patient-Reported Outcomes Measurement Information System (PROMIS)-Depression and the Irritability for parents of children aged 6-17 years.
Methods: Participants were 190 children diagnosed with depressive disorder (n=14), anxiety disorder (n=21), attention-deficit/hyperactivity disorder (ADHD; n=111), ADHD with anxious depression (n=13), and tic disorder with somatic symptoms (n=31). Patients were 8-15 years of age. The participants' mothers completed the Korean versions of the DSM-5 Level 2 Cross-Cutting Symptom Measure- PROMIS Depression and Irritability (Affective Reactivity Index, ARI), and the Korean Child Behavior Checklist (K-CBCL). Using these data, we calculated the reliability coefficient and examined the concurrent and discriminant validity of the PROMIS Depression and the Irritability (ARI) scales for assessing depression and irritability in children.
Results: The reliability coefficient of the PROMIS Depression scale (Cronbach's α) was 0.93. The correlation coefficient with the KCBCL DSM emotional problem score was 0.71. The PROMIS Depression scale significantly discriminated children with depressive disorders from those with other conditions. The reliability coefficient of the Irritability (ARI) scale was 0.91, suggesting its high reliability.
Conclusion: Our results suggest that the Korean version of the DSM-5 Level 2 Cross-Cutting Symptom Measure for Depression and Irritability Scales for parents of children aged 6-17 years is reliable and valid and may be an efficient alternative to the K-CBCL.

INTRODUCTION

Childhood and adolescence are periods marked by a multitude of biological, cognitive-emotional, and psychosocial changes that occur during growth and maturation. Many children and adolescents experience internal stress during normal development. However, because the primary focus for children and adolescents in Korean society is high school achievement to ensure enrollment in good universities, parents often do not pay close attention to their children’s emotional problems. Therefore, they may not accurately recognize their children’s emotional disorders.

Emotions such as depression, anxiety, and irritation are natural reactions to frustration, failure, and adverse life events in life; however, failure to relieve these emotions (and, rather, exacerbate them) due to social and environmental stress may elicit an array of emotional and behavioral problems in children and adolescents [1,2]. Thus, emotional problems in children and adolescents require prompt therapeutic intervention because of the high probability of their progression to psychiatric disorders or persistence into adulthood. The appropriate assessment of emotional and behavioral problems is required to ensure prompt intervention [3].

Depression and anxiety are the most prevalent emotional problems experienced by children and adolescents [4]. However, previous studies have reported that, in contrast to depressive disorder in adults characterized by depressive mood, hopelessness, diminished interest, psychomotor retardation, insomnia, and suicidal ideation, depressive disorder in children and adolescents manifests as irritability rather than depression. Affected individuals are easily agitated and hypersensitive [5,6]. Furthermore, they often show externalized problems such as hyperactivity and delinquent behaviors rather than displaying internalized symptoms such as helplessness [7,8]. In other words, emotional problems in children and adolescents occur concurrently with different behavioral issues. Thus, an accurate diagnostic assessment requires a reliable and valid test that encompasses the core symptoms of each disorder.

In the actual clinical setting, various types of assessment tools, including clinical interviews, psychological tests, and self-reported/parent-rated scales, are used to systematically and objectively evaluate emotional problems in children and adolescents. Scales rated by parents and caregivers are also utilized to address the fact that children and adolescents are not fully capable of rating themselves accurately. Such scales have been widely used in clinical settings owing to their ease of administration, scoring, and interpretation [9]. The Diagnostic and Statistical Manual of Mental Disorders, Fifth Edition (DSM-5) Level 2 Cross-Cutting Symptom Measure—Patient- Reported Outcomes Measurement Information System (PROMIS) Depression and Irritability (Affective Reactivity Index, ARI) for parents, provided in DSM-5 Section III, is used in countries outside Korea [10]. This short scale can be used to efficiently assess emotional problems among children and adolescents in clinical settings and is valuable for post-treatment follow-up and evaluating treatment effects. The scale was translated into Korean by Shin et al. [11], but no study has evaluated its reliability and validity. Therefore, this study aimed to evaluate the reliability and validity of the Korean version of the scale to establish standard outcome measures for clinical diagnosis and research to enable a more accurate diagnosis of emotional problems, such as depression and irritability, and assess treatment outcomes in children and adolescents aged 6–17 years.

The specific objective of this study was to examine the reliability and discriminant validity of the Korean versions of the DSM-5 Level 2 Cross-Cutting Symptom Measures PROMIS Depression and the Irritability (ARI) for parents of children aged 6–17 years in various clinical groups. This study also examined the concurrent validity of the PROMIS Depression and the Irritability (ARI) by analyzing their correlations with the previously validated Korean Child Behavior Checklist (K-CBCL) to determine their possible use as complementary instruments [12,13].

METHODS

Subjects

Children and adolescents aged 8–15 years who visited the Department of Child and Adolescent Psychiatry of a university hospital in Seoul between January 2020 and December 2021 were enrolled in this study. Child psychiatrists diagnosed 190 children and adolescents according to DSM-5 criteria and psycho-diagnostic tests. Their parents participated in this study. The children were divided into five groups based on the DSM-5 diagnostic criteria: depressive disorder (n=14), anxiety disorder (n=21), attention-deficit/hyperactivity disorder (ADHD; n=111), ADHD with anxious depression (n= 13), and tic disorder with somatic symptoms (n=31). There was a large difference in the number of subjects between the ADHD group and the other groups. Therefore, 30 children with ADHD were randomly selected for the comparison of group means. There were no differences in any variables between the entire ADHD cohort and the 30 randomly selected ADHD patients (Table 1). The participants’ demographic characteristics are presented in Table 2.

Table 1

Comparison of the entire ADHD group and the 30 randomly selected ADHD group

	Entire ADHD (n=111)	Random ADHD (n=30)	t-value (sig)
Sex (M:F)	90:21	22:8	0.24 (NS)*
Age (yr)	9.88±1.89	9.97±1.99	0.21 (NS)
PROMIS Depression	15.99±7.25	14.63±8.54	-0.88 (NS)
Irritability (ARI)	3.16±3.54	2.23±2.81	-1.33 (NS)

Data are presented as mean±standard deviation. *chi-square test. NS, not significant; PROMIS, Patient-Reported Outcomes Measurement Information System; ARI, Affective Reactivity Index; ADHD, attention-deficit/hyperactivity disorder; M, male; F, female

Table 2

Subjects’ demographic variables (n=190)

Variable	n (%)
Sex
Male	144 (75.8)
Female	46 (24.2)
Age (yr)
8	49 (25.8)
9	33 (17.4)
10	30 (15.8)
11	24 (12.6)
12	20 (10.5)
13	7 (3.7)
14	18 (9.5)
15	9 (4.7)
Diagnosis
Depressive disorder	14 (7.4)
Anxiety disorder	21 (11.1)
ADHD	111 (58.4)
ADHD with anxious depression	13 (6.8)
Tic disorder with somatic symptoms	31 (16.3)

ADHD, attention-deficit/hyperactivity disorder

The subjects were 8–15 years old. We included children with a full-scale intelligent quotient of 70 or higher and a language comprehension score of 80 or higher on the Korean Wechsler Intelligence Scale for Children–Fourth Edition, a measure of children’s intellectual ability. The analysis used data from 190 children to calculate the internal consistency (Cronbach’s α reliability coefficient) and concurrent validity. This study was conducted after approval was obtained from the Institutional Review Board of Seoul National University Hospital (no. 1905-145-1035). The study details were explained to the parents and children, who provided consent prior to their participation.

Measures

Patient-Reported Outcomes Measurement Information System Depression scale (short form)

The National Institutes of Health developed the PROMIS. Shin et al. [11] translated the PROMIS Depression Measure into Korean to create a Korean version. This parent- or guardian- rated measure evaluates the severity of symptoms such as loss of interest or joy, depressive mood, and hopelessness in the past two weeks in children and adolescents. Eleven items are rated on a five-point Likert scale for a total score of 11–55. A higher score indicated more severe depression. The raw scores for the 11 items are summed to calculate the total raw score, which is converted to a T-score. A T-score of less than 55 is considered normal, while T-scores of 55–60, 60–70, and 70 or higher indicate mild, moderate, and severe depression, respectively. The Cronbach’s α of the original scale was 0.92, while that of the K-PROMIS translated by Cho et al. [14] was 0.93.

Irritability scale: Affective Reactivity Index

Stringaris [15] developed the ARI to measure irritability in 2011. We used the Korean version translated by Shin et al. [11]. The original version assesses symptoms in the past six months. In contrast, the revised version used in the DSM-5 field study measured symptoms over the past two weeks. This scale is a parent- or guardian-rated measure for assessing the severity of irritability in children and adolescents who appear to be easily irritated, be quick to anger, or have difficulty controlling their anger in the past two weeks. A higher score indicated more severe irritability. Clinicians can compute the mean total score based on a three-point scale, where 0 indicates no symptoms, 1 indicates mild to moderate symptoms, and 2 indicates moderate to severe symptoms.

Korean Child Behavior Checklist for ages 6–18

The CBCL was developed in 1983 by Achenbach and Edelbrock [3], and the K-CBCL was standardized in 1997 [16]. This is a parent- or guardian-rated measure for assessing behavioral problems and social adaptation in children and adolescents aged 6–18 [17]. The scale comprises 120 items and syndrome scales, DSM-oriented scales, and problem behavior scales to measure emotional and behavioral problems in children and adolescents [3]. The DSM-oriented scales include DSM emotional problems, DSM anxiety problems, DSM somatic problems, DSM ADHD, DSM oppositional defiant problems, and DSM conduct problems [18,19]. We used the DSM emotional problem and DSM anxiety problems scales in this study because the former comprises items about depression- related symptoms, including “feels worthless or subpar to others” and “feels overly guilty,” while the latter comprises items describing anxiety symptoms, including “afraid of/worried about going to school.”

Statistical analysis

All statistical analyses were performed using SPSS 20.0 software (IBM Corp., Armonk, NY, USA). First, Cronbach’s α reliability coefficients of the Korean versions of the PROMIS Depression and the Irritability (ARI) were calculated. Further, we examined the concurrent validity of these scales based on their correlation with the DSM emotional problem and DSM anxiety problem scales of the K-CBCL. Finally, we evaluated the discriminant validities of the scales of PROMIS Depression and the Irritability (ARI) by comparing the mean scores of the five groups (depressive disorder, anxiety disorder, ADHD, ADHD with anxious depression, and tic disorder with somatic symptoms).

RESULTS

Reliability

The internal consistency (Cronbach’s α) of the Korean version of the PROMIS Depression (0.93) and the Irritability (ARI, 0.91) were relatively high. In other words, the items in the Korean versions of the PROMIS Depression and the Irritability (ARI) are highly homogeneous, showing that they reliably assess depression and irritability in children and adolescents (Table 3).

Table 3

Internal consistency coefficients of the PROMIS Depression and the Irritability (ARI) scales

	Cronbach’s α
PROMIS Depression	0.93***
Irritability (ARI)	0.91***

***p<0.001. PROMIS, Patient-Reported Outcomes Measurement Information System; ARI, Affective Reactivity Index

Concurrent validity

Concurrent validity of the Patient-Reported Outcomes Measurement Information System Depression

The Korean version of the PROMIS Depression had a relatively high correlation with the CBCL DSM emotional problems scale (0.71) and a moderate correlation with the CBCL DSM anxiety problems scale (0.53). Table 4 shows that the Korean version of the PROMIS Depression has comparable validity for assessing depressive symptoms in children and adolescents.

Table 4

Correlation coefficients of the PROMIS Depression, Irritability (ARI), K-CBCL DSM emotional problem, and K-CBCL DSM anxiety problem scores (n=109)

	PROMIS Depression	Irritability ARI	K-CBCL DSM emotional problem	K-CBCL DSM anxiety problem
PROMIS Depression	1.00
Irritability (ARI)	0.58**	1.00
K-CBCL DSM emotional problem	0.71**	0.52**	1.00
K-CBCL DSM anxiety problem	0.53**	0.46**	0.73**	1.00

**p<0.01. PROMIS, Patient-Reported Outcomes Measurement Information System; ARI, Affective Reactivity Index; K-CBCL, Korean Child Behavior Checklist; DSM, Diagnostic and Statistical Manual of Mental Disorders

Concurrent validity of the Irritability (Affective Reactivity Index)

The Korean version of the Irritability (ARI) showed a moderate correlation with the CBCL DSM emotional problems scale (0.52) and the CBCL DSM anxiety problems scale (0.46) (Table 4).

Discriminant validity

The discriminant validities of the PROMIS Depression and the Irritability (ARI) were examined by comparing the mean scores among the five groups (depressive disorder, anxiety disorder, ADHD, ADHD with anxious depression, and tic disorder with somatic symptoms). Prior to the analysis, 30 children were randomly selected from the ADHD group to examine the discriminant validities of the scales due to the significant intergroup variation in sample sizes.

The mean differences of the PROMIS Depression and the Irritability (ARI) were analyzed using one-way analysis of variance. The results showed significant intergroup differences in mean PROMIS Depression scores (F=4.75, p<0.01). A Bonferroni post-hoc comparison demonstrated that the depressive disorder group (a) had a significantly higher mean score on the PROMIS Depression than the ADHD (c) and tic disorder with somatic symptoms groups (e) (a>c, e). There were significant intergroup differences in mean the Irritability scores (Table 5; F=2.54, p<0.05). Participants with ADHD with anxious depression showed the highest mean Irritability (ARI) score, suggesting its discriminant validity for diagnosing ADHD in children with anxious depression.

Table 5

Comparisons of mean PROMIS Depression Measure and Irritability (ARI) scores (n=109)

	Depressiona (n=14)	Anxietyb (n=21)	ADHDc (n=30)	ADHD+anxious depressiond (n=13)	Tic+somatic symptomse (n=31)	F	post hoc
PROMIS Depression	24.36±8.47	18.52±7.39	14.63±8.54	21.00±8.80	16.06±6.24	4.75**	a>c, e
Irritability (ARI)	3.57±3.88	4.19±4.01	2.23±2.81	5.00±3.81	2.42±2.72	2.54*

Data are presented as mean±standard deviation. *p<0.05; **p<0.01. PROMIS, Patient-Reported Outcomes Measurement Information System; ARI, Affective Reactivity Index; ADHD, attention-deficit/hyperactivity disorder

DISCUSSION

This study examined the reliability and validity of the Korean versions of the PROMIS Depression and the Irritability for parents of children aged 6–17 years. The results confirmed that these measures have high internal consistency and significant discriminant and concurrent validity. In addition, the Korean version of the PROMIS Depression had a high reliability (Cronbach’s α) of 0.93. In other words, the PROMIS Depression is a homogeneous and reliable scale for assessing depressive symptoms. Furthermore, its results were relatively highly correlated with those of the CBCL DSM emotional problems scale assessing depression (r=0.71) and moderately correlated with those of the CBCL DSM anxiety problems scale (r=0.53). The significant correlations of the PROMIS Depression score with the scores of CBCL DSM emotional problem and DSM anxiety problems scales appear to be attributable to the fact that children and adolescents often concurrently show depression and anxiety symptoms. However, considering that the correlation with the CBCL DSM emotional problems scale was relatively higher than that with the CBCL DSM anxiety problems scale, the PROMIS Depression more accurately differentiated and assessed depressive symptoms in children and adolescents. This finding is similar to the CBCL DSM emotional problems scale.

The discriminant validity of the PROMIS Depression was examined by comparison of the mean scores of the five study groups (depressive disorder, anxiety disorder, ADHD, ADHD with anxious depression, and tic disorder with somatic symptoms groups). We found that the depressive disorder group had a significantly higher mean PROMIS Depression score than the other four groups. These results suggest that the PROMIS Depression can significantly differentiate children and adolescents with depressive symptoms from those with other psychiatric disorders such as ADHD and tic disorder with somatic symptoms.

The Irritability ARI had a high reliability (Cronbach’s α) of 0.91. The Irritability ARI is a homogeneous and reliable scale for assessing irritability. Furthermore, it had a moderate correlation with the results of the CBCL DSM emotional problems scale (r=0.52) and the CBCL DSM anxiety problems scale (r=0.46). These results may be pertinent to the high prevalence of irritability problems in children and adolescents with depressive or anxiety disorders. Examination of the discrimination validity of the Irritability scale (ARI) revealed statistically significant differences in the mean scores of the five groups of depression disorder, anxiety disorder, ADHD, ADHD with anxious depression, and tic disorder with somatic symptoms. Although the ADHD with anxious depression group showed the highest mean Irritability (ARI) score, the difference was not statistically significant. Children and adolescents with ADHD as well as emotional problems, such as depression and anxiety, may be the most irritable, consistent with previous findings that irritability is common among children with ADHD [20]. Therefore, these results suggest that the Irritability ARI is useful and valid for diagnosing children with ADHD and anxious depression.

This study’s findings are significant for several reasons. First, the clinically useful PROMIS Depression and the Irritability (ARI) scales , based on the DSM-5 and translated into Korean, were validated. This validation enables the use of these scales in the diagnosis and assessment of treatment outcomes for psychiatric disorders in children and adolescents in Korean clinical settings as well as in research. Thus, the Korean version of the PROMIS Depression and the Irritability (ARI) can be widely utilized to assess depression and irritability in children and adolescents. In particular, the scales do not require much time to complete; thus, they are cost-effective for assessing depression, anxiety, and irritability in children and adolescents in terms of little time, low cost, and administration ease. Furthermore, scales with existing psycho-diagnostic tests may enhance assessment accuracy and monitoring of depression and irritability in children and adolescents. The accurate diagnosis and rapid therapeutic intervention for emotional and behavioral problems can help promote the well-being of children and their parents.

This study has several limitations. First, the participants were limited to children and adolescents who visited the Department of Child and Adolescent Psychiatry at a university hospital in Seoul. Therefore, the scale was not validated in a control group. Further studies in various clinical and normal groups are required to address this issue. Second, test-retest reliability should be evaluated. Finally, as the scales are intended for use by parents and guardians, the subjective emotional discomfort and severity of children’s and adolescents’ experiences may not be adequately reflected. Therefore, the use of these scales with the DSM-5 Level 2 self-report scales could augment their diagnostic and therapeutic value.

CONCLUSION

This study examined the reliability and validity of the Korean version of the DSM-5 Level 2 Cross-Cutting Symptom Measure—the PROMIS Depression and the Irritability (ARI) for parents of 190 children and adolescents aged 6–17 years diagnosed with depressive disorder, anxiety disorder, ADHD, ADHD with anxious depression, or tic disorder with somatic symptoms. The results demonstrated that these scales have high reliability and validity, suggesting that the Korean versions of the PROMIS Depression and the Irritability (ARI) for parents are simple assessment tools with broad application in clinical, mental health service, and research settings.