Development and Validation of a Japanese Version of the Emotion Regulation Questionnaire for Children and Adolescents.

PURPOSE: We developed a Japanese version of the Emotion Regulation Questionnaire for Children and Adolescents (ERQ-CA) and examined its reliability and validity across three studies. PATIENTS AND METHODS: In Study 1, the Japanese version of ERQ-CA was developed and administered to 389 children aged 8-12 years. In Study 2, the questionnaire was administered to 1738 adolescents aged 12-18 years. In Study 3, utilizing a sample of 1300 children and adolescents, the test was administered twice over a period of four weeks in order to assess test-retest reliability.
RESULTS: In Study 1, the Japanese version of ERQ-CA showed the same factor structure as the original version, along with good internal consistency reliability and acceptable construct validity. In Study 2, the questionnaire's factor structure, internal consistency reliability, and construct validity were again confirmed. Finally, in Study 3, measurement invariance was tested across distinct age groups (8-11, 12-15, and 16-18 years), and the questionnaire had good test-retest reliability over a period of four weeks.
CONCLUSION: The Japanese version of the ERQ-CA had good reliability and validity.

Introduction

The number of studies on emotion regulation in children and adolescents has dramatically increased over the past two decades, mainly due to the association between emotion regulation and symptoms of psychopathology in children and adolescents.1 For children, emotional dysregulation can lead to an array of social problems in schools,2 while emotion regulation is a central correlate of behavioral and emotional problems in adolescents.3

Among studies on emotion regulation, the process model of emotion regulation proposed by Gross4 has had a profound influence. This focuses on two principal strategies: cognitive reappraisal (CR), which refers to a cognitive change that occurs prior to the generation of an emotion and modifies the emotional impact of a situation; and expressive suppression (ES), which involves the inhibition of emotion-expression behavior after an emotional response has been generated.5 CR is conducted early in the emotion-generative process and decreases the intensity of an emotion while having no impact on memory. Contrastingly, ES occurs later in the emotion-generative process and fails to decrease emotion intensity.6

To assess these two emotion strategies, Gross and John7 developed an emotion regulation scale: the Emotion Regulation Questionnaire (ERQ). Then, Gullone and Taffe8 created children and adolescents’ version of the ERQ (ERQ-CA), in which item expressions were modified to make them easier for children and adolescents to understand. The ERQ-CA consists of 10 items: six evaluate CR (“I control my feelings about things by changing the way I think about them”) and four evaluate ES (e.g., “I keep my feelings to myself”).

The reliability and validity of the ERQ-CA were investigated by administering it to a population of children and adolescents aged 10–18 years.8 Consequently, good alpha coefficients and stability (intraclass correlation) coefficients were shown for both CR and ES over a 12-month period, indicating good reliability.8 Regarding validity, construct validity using confirmatory factor analysis (CFA) was investigated, and a two-factor model was a suitable fit.8 In addition, adequate convergent validity was shown regarding the correlation between the ERQ-CA and depression and the “Big Five” personality traits (i.e., neuroticism, extraversion, agreeableness, conscientiousness, and openness to experience).8 Assessing CR and ES using ERQ-CA was determined to be a means of predicting mental manifestation in children and adolescents. Specifically, CR was significantly negatively correlated with both depression and anxiety, while ES was significantly positively correlated with these.9 Therefore, the ERQ-CA is a key tool for understanding the psychopathology of children and adolescents.

Although a Japanese version of the ERQ has been developed,10 there is, as of yet, no Japanese version of the ERQ-CA. Emotion regulation is essential for Asian children and adolescents, and some studies have suggested that emotion regulation strategies differ between Western and Eastern societies. Suppression occurs more frequently, and is less associated with negative emotion in Asian cultures compared to Western ones.11 To compare Western and Eastern emotion regulation strategies, it is necessary to develop and examine the reliability and validity of Asian language version of ERQ-CA. Clarifications of psychometric properties of ERQ-CA amongst this population are needed for facilitating further research to understanding the nature of emotion regulation in this population. In the current study, we consequently developed a Japanese version of the ERQ-CA (ERQ-CA-J) and examined its reliability and validity. This paper describes three studies, each of which examined various dimensions of the psychometric properties of the ERQ-CA-J.

Study 1

The purpose of Study 1 was to develop the ERQ-CA-J and investigate its factor structure, internal consistency reliability, and construct validity among children aged 8–12 years.

Methods

Participants and Procedures

The institutional review board (IRB) at the University of Tsukuba approved this study’s ethical and scientific protocols (approval number: 筑29-117). Participants were recruited from a primary school in Japan after obtaining informed consent from the school’s principal. After obtaining passive parental consent, whereby parents could choose to exclude their child from the research, the children’s verbal assent was obtained by the parents, as they asked their children to participate. Additionally, in order to ensure that all children were informed and chose whether to participate in the study, the teachers also informed the children about the study and verbally asked the children to participate. The children were informed that the survey would take approximately 15–20 mins to complete, that their participation was voluntary, that they were free to withdraw at any time without giving a reason, and that all information collected would be anonymous and kept strictly confidential.

Four hundred twelve children completed the survey in the classroom with the support of a teacher; in other words, if they could not understand a word’s meaning, they could ask the teacher. The researcher was not in the room when the surveys were completed. Twenty-three participants were omitted because their submissions contained significant missing data (i.e., leaving > 20% of the items blank in at least one measure). This resulted in a sample of 389 participants (aged 8–12 years, Mage = 10.30 years SDage = 1.23 years; 202 boys, 186 girls, and one who did not specify sex).

Measures

ERQ-CA-J: The original ERQ-CA8 comprises 10 items assessing the emotion-regulation strategies of CR (six items) and ES (four items). Items are rated using a five-point Likert-type response scale (1 = “strongly disagree,” 2 = “disagree,” 3 = “half-and-half,” 4 = “agree,” 5 = “strongly agree”). The original ERQ-CA possesses a two-factor structure (ES and CR), with a correlated error added between item 1 (“when I want to feel happier, I think about something else”) and item 3 (“when I want to feel better [e.g., less sad, angry, or worried], I think about something else”).8 Regarding reliability, the alpha coefficients were previously calculated to be 0.83 for CR and 0.75 for ES.8 In addition, previous work found the stability (intraclass correlation) coefficients over a 12-month period to be moderate for both CR and ES. For CR, the intraclass correlation coefficients ranged in size from 0.37 (for 13–15-year-olds) to 0.47 (for 16–18-year-olds) in earlier studies, while for ES, the coefficients ranged in size between 0.40 (for 10–12-year-olds) and 0.63 (for 16–18-year-olds).8 Finally, the correlation between the original ERQ-CA and depression and the Big Five personality traits showed adequate convergent validity in a previous study; specifically, the correlation between depression was −0.26 for CR and 0.37 for ES.8

The COnsensus-based Standards for the selection of health Measurement INstruments (COSMIN),12 which are guidelines for assessing the methodological quality of studies on measurement properties of self-reported outcomes, were applied in the development of the ERQ-CA-J. After obtaining permission from the original developer to create the ERQ-CA-J, two authors, Japanese clinical psychologists, independently translated the original scale from English into Japanese. Then, two additional clinical psychologists discussed the clarity, language expression, and conceptual equivalence of the combined forward-translated version. Next, two bilingual individuals, unaware of the goal of the present study, independently back-translated the scale from Japanese to English. The research team conducted an examination of the conceptual equivalence of the two translated versions with the original version and highlighted problematic expressions. This process was repeated several times. The original developer of the ERQ-CA then confirmed the conceptual equivalence of the original and back-translated versions.

Children’s Emotional Regulation Scale: The Children’s Emotional Regulation Scale13 is an 18-item measure of emotion regulation for children originally developed in Japan. The scale has a three-factor structure: emotion inhibition (seven items; e.g., “I often suppress my anger”), difficulties concerning emotion regulation (six items; e.g., “I tend to lose my temper”), and coping with emotion regulation (five items; e.g., “When I am scared, I try to be with someone else”). Each item is rated using a five-point scale (1 = “strongly disagree” to 5 = “strongly agree”).

Spence Children’s Anxiety Scale: The Spence Children’s Anxiety Scale (SCAS)14,15 is a 38-item measure of anxiety symptoms designed for children and adolescents. The SCAS has a six-factor structure: separation anxiety disorder, social phobia, generalized anxiety disorder, panic attacks and agoraphobia, obsessive-compulsive disorder, and physical injury fears.

The original version of the SCAS14 was determined through CFA and exploratory factor analysis to support six factors, and in the model the first-order factors loaded significantly onto a single second-order actor, generalized anxiety disorder. For internal consistency, alpha coefficients were 0.82 (panic agoraphobia), 0.70 (separation anxiety), 0.70 (social phobia), 0.60 (physical injury fears), 0.73 (obsessive-compulsive disorder), and 0.73 (generalized anxiety).14 Further, performing test–retest reliability over a six-month period showed a correlation coefficient of 0.60 for total score.14 Thus, the convergent and discriminant validity was supported.14

The Japanese version of the SCAS15 reported sufficient reliability coefficients (0.94 and 0.92) for the full-scale scores among children and adolescents aged 9–15 years. In addition, the scale had sufficient test–retest reliability over 2–4 weeks: r = 0.76 for children and r = 0.86 for adolescents (ps < 0.001). Each item was rated using a four-point scale (0 = “never” to 3 = “always”).

Depression Self-Rating Scale: The Depression Self-rating Scale (DSRS)16,17 is an 18-item measure of depressive symptoms for children. The DSRS has a single-factor structure. The test–retest reliability of the scale was 0.80 which was sufficient.16 Each item had a reliability coefficient of 0.65–0.95.16 The split-half reliability was 0.86, which implied satisfactory internal consistency.16 The original version of the DSRS had adequate face and factorial validity.16 For clinical validity, 155 children aged 8–14 years who were psychiatry outpatients were grouped into a depressed population and others.18 Children who scored 15 and over on the DSRS were significantly more likely to have a depressive diagnosis.18 The DSRS total score could predict as well as the global score of the history of depression and appearance of depression rated at interview by Child Psychiatrists.18

The Japanese version of the DSRS showed good test–retest reliability.18 The Cronbach’s α was 0.77, which was sufficient.17 For validity, the Japanese version of DSRS was significantly correlated (r = 0.71) with the Japanese version of the Children’s Depression Inventory (CDI)17 and with teachers’ evaluations.17 Each item is rated using a three-point scale (0 = “never” to 2 = “always”).

Statistical Analyses

Statistical analyses were conducted by IBM SPSS Statistics 25.0 and Amos 25.0. To examine the structural validity, of Japanese version of the ERQ-CA, we randomly split the whole sample into two subsamples. Then, the first sample (subsample 1) was used for exploratory factor analysis (EFA), and the second sample (subsample 2) for CFA, respectively. For EFA, the number of factors was estimated through the visual scree test.19 Maximum likelihood estimations using promax rotation were performed to explore factor loadings. An item was retained if it had (a) an item-factor loading of at least 0.35 on a primary factor, (b) cross-loadings at least 0.10 difference between factor loadings as guidelines for meaningful factor loadings generally range from 0.30 to 0.40.20 For CFA, the scale parameter was estimated using the maximum likelihood method rather than Mean- and Variance-adjusted Weighted Least Squares (WLSMV), as the data were not categorical but quantitative.21 The model fit was determined based on the recommendations of Hu and Bentler:22 using the Comparative Fit Index (CFI) and the root mean square error of approximation (RMSEA). Specifically, CFI values of ≥0.95 and RMSEA values of ≤0.06 suggest a good fit of the model to the data, whereas CFI values of 0.90–0.94 and RMSEA values of 0.07–0.10 suggest an acceptable fit.

For correlation analyses, we assumed that even if Pearson’s correlation values were very weak, they could show significance because of the sample size. Therefore, we adopted the criteria applied by Cohen,23 in which Pearson correlation values of r = ± 0.50 are considered strong, r = ± 0.30 are considered moderate, and r = ± 0.10 are considered weak. Regarding the correlation between ES and other validity measures, it was hypothesized that ES would show a strong positive relationship with emotion inhibition, a moderate positive relationship with difficulties concerning emotion regulation, a weak positive relationship with coping with emotion regulation, a moderate positive relationship with anxiety, and a weak to moderate relationship with depression. For CR, it was hypothesized that it would show a strong negative relationship with emotion inhibition, a moderate negative relationship with difficulties concerning emotion regulation, a weak positive correlation with coping with emotion regulation, a moderate negative relationship with anxiety, and a weak to moderate relationship with depression.

Results and Discussion

Descriptive Statistics

The means and standard deviations for the two emotion-regulation strategies (ES and CR) are shown in Table 1. The means were almost the same as those in the original version; however, the standard deviation of CR was larger.8

Table 1

Means (and Standard Deviations) for the Two Emotion Regulation Strategies (ES and CR)

Sample		Emotion Regulation Strategy
ES	CR
Study 1	Overall	11.83 (3.59)	19.51 (5.47)
Study 2	Overall	11.51 (3.27)	19.33 (4.57)
	Age 12–15 years	11.22 (3.35)	18.75 (4.74)
	Age 16–18 years	11.85 (3.15)	20.02 (4.27)
Study 3	Overall	11.70 (3.01)	19.77 (4.45)
	Age 8–11 years	12.47 (3.26)	20.40 (4.92)
	Age 12–15 years	11.33 (2.91)	19.24 (4.34)
Age 16–18 years	11.94 (2.87)	20.52 (4.11)

Note: Means and standard deviations for Study 3 were obtained the first time.

Abbreviations: CR, cognitive reappraisal; ES, expressive suppression.

Structural Validity

We conducted EFA on subsample 1, to examine the unique factorial structure for the Japanese sample. The visual scree test indicated two-factor structure same as the original version as most appropriate. Table 2 shows the rotated factor pattern matrix and extraction communalities. Two factors explained 52.76% of the variance of ERQ-CA scores. Factor loadings on the first factor (CR) ranged 0.42–0.81 and the second factor (ES), 0.42–0.80. The two factors were correlated (r = 0.57). Thus, the EFA indicated the Japanese version of the ERQ-CA has a two-factor structure same as the original version.

Table 2

Exploratory Factor Analysis Factor Pattern Matrix

Item	Study1			Study2
F1: CR	F2: ES	h2	F1: CR	F2: ES	h2
1. When I want to feel happier, I think about something different.	0.42	0.14	0.26	0.47	0.02	0.23
3. When I want to feel less bad (e.g., sad, angry or worried), I think about something different.	0.67	−0.02	0.44	0.57	−0.01	0.32
5. When I am worried about something, I make myself think about it in a way that helps me feel better.	0.66	−0.01	0.42	0.64	−0.03	39
7. When I want to feel happier about something, I change the way I am thinking about it.	0.51	0.15	0.36	0.67	0.02	46
8. I control my feelings about things by changing the way I think about them.	0.66	0.03	0.45	0.69	0.06	51
10. When I want to feel less bad (e.g., sad, angry, or worried) about something, I change the way I am thinking about it.	0.81	−0.10	0.57	0.75	−0.05	53
2. I keep my feelings to myself.	−0.13	0.71	0.41	−0.10	0.78	0.56
4. When I am feeling happy, I am careful not to show it.	0.11	0.49	0.31	−0.02	0.52	0.26
6. I control my feelings by not showing them.	0.03	0.80	0.66	0.06	0.66	0.48
9. When I am feeling bad (e.g., sad, angry, or worried), I am careful not to show it.	0.18	0.42	0.30	0.12	0.54	0.37

We conducted CFA on subsample 2, to confirm the results provided by EFA. Using CFA, a two-factor model based on the findings of Gross and John7 showed excellent fit (CFI = 0.954, RMSEA = 0.057). Following the recommendations of Gullone and Taffe,8 a correlated error was added between item 1 (“when I want to feel happier, I think about something else”) and item 3 (“when I want to feel better [e.g., less sad, angry, or worried], I think about something else”), which have similarly worded phrases, and this produced a better fit (CFI = 0.959, RMSEA = 0.055, see Table 3). Standardized item-factor loadings for CR ranged from 0.50 to 0.68 and ES ranged from 0.52 to 0.78 (Table 4), which were considered sufficient. Therefore, the soundness of the ERQ-CA-J’s two-factor structure was indicated, similar to that of Gullone and Taffe.8

Table 3

Goodness of Fit for ERQ-CA-J

Model		CFI	RMSEA	df
Study 1	Overall	0.959	0.055	33
Study 2	Overall	0.923	0.077	33
	Age 12–15 years	0.930	0.075	33
	Age 16–18 years	0.910	0.080	33
Study 3	Overall	0.908	0.075	33
	Age 8–11 years	0.897	0.079	33
	Age 12–15 years	0.876	0.086	33
	Age 16–18 years	0.912	0.074	33
	Configural invariance model	0.889	0.047	99
	Factor loading invariance model	0.890	0.044	115
	Intercept invariance model	0.863	0.045	135

Note: The results shown in this table represent the model where a correlated error was added between Item 1 and Item 3.Abbreviations: CFI, Comparative Fit Index; df, degrees of freedom; RMSEA, Root Mean Square Error of Approximation.

Table 4

Confirmatory Factor Analysis Standardized Item-Factor Loadings

Items	Study 1		Study 2			Study 3
Overall	Overall		Age 12–15	Age 16–18		Overall	Age 8–11	Age 12–15	Age 16–18
Cognitive reappraisal
1. When I want to feel happier, I think about something different.	0.49	0.42		0.44	0.37		0.45	0.51	0.41	0.45
3. When I want to feel less bad (eg, sad, angry or worried), I think about something different.	0.61	0.48		0.48	0.45		0.58	0.59	0.56	0.65
5. When I am worried about something, I make myself think about it in a way that helps me feel better.	0.59	0.50		0.52	0.46		0.52	0.46	0.53	0.54
7. When I want to feel happier about something, I change the way I am thinking about it.	0.65	0.70		0.72	0.66		0.61	0.62	0.60	0.58
8. I control my feelings about things by changing the way I think about them.	0.63	0.74		0.75	0.72		0.66	0.66	0.65	0.62
10. When I want to feel less bad (e.g., sad, angry, or worried) about something, I change the way I am thinking about it.	0.65	0.67		0.65	0.72		0.69	0.68	0.69	0.72
Emotion suppression
2. I keep my feelings to myself.	0.61	0.69		0.68	0.68		0.59	0.59	0.57	0.54
4. When I am feeling happy, I am careful not to show it.	0.52	0.51		0.50	0.50		0.48	0.45	0.50	0.49
6. I control my feelings by not showing them.	0.73	0.78		0.77	0.79		0.77	0.69	0.78	0.88
9. When I am feeling bad (eg, sad, angry, or worried), I am careful not to show it.	0.50	0.58		0.60	0.56		0.49	0.56	0.44	0.51

Note: The results shown in this table represent the model where a correlated error was added between Item 1 and Item 3.

Internal Consistency Reliability

Cronbach’s alpha coefficient was 0.70 for the ES items (0.71 for boys; 0.69 for girls), and 0.80 for the CR items (0.79 for boys; 0.78 for girls). Note that the result of full data set was higher than that of boys and girls, as the data set included data for one sex non-specified individual. Thus, the ERQ-CA-J demonstrated acceptable-to-moderate internal consistency.

Construct Validity

We conducted correlation analyses for two ERQ-CA-J factors, with other validity scales; the subscales of Children’s Emotional Regulation Scale (emotion inhibition, difficulties concerning emotion regulation and coping with emotion regulation), SCAS, and DSRS (Table 5). The correlations between the ES score and the other validity scales were as expected.

Table 5

Study 1 Correlations

		Children’s Emotional Regulation Scale			SCAS	DSRS
Emotion Inhibition	Difficulty in Emotion Regulation	Coping with Emotion Regulation
Overall Sample	ES	0.61**	0.32**	0.15*	0.49**	0.15**
CR	0.48**	0.31**	0.27**	0.37**	0.01
Girls	ES	0.62**	0.20*	−0.03	0.49**	0.20*
CR	0.39**	0.22**	0.21*	0.38**	0.01
Boys	ES	0.58**	0.38**	0.27**	0.46**	0.08
CR	0.49**	0.29**	0.23**	0.27**	−0.06

Notes: *p < 0.05, **p < 0.01.

Abbreviations: CR, cognitive reappraisal; DSRS, Depression Self-Rating Scale; ES, expressive suppression; SCAS, Spence Children’s Anxiety Scale.

On the other hand, the correlations between the CR score and the other validity scales differed from our expectations. A moderate positive correlation was shown between emotion inhibition and CR score, while a weak positive association was shown between CR score and difficulties concerning emotion regulation, coping with emotion regulation, and SCAS. There was almost no relation between DSRS and CR score. A possible explanation for this is that the CR and ES may not have been clearly differentiated among the children. For Japanese children, emotion-inhibition-related items similar to CR have the same factors.13

Study 2

The purpose of Study 2 was to develop the ERQ-CA-J and investigate its factor structure, internal consistency, and construct validity among adolescents aged 12–18 years.

The institutional review board (IRB) at the University of Tsukuba approved this study’s ethical and scientific protocols (approval number: 筑29-117). Participants were recruited from a middle school and three high schools in Japan. Written informed consent was acquired from both the school’s principal and the participants after the study was described to them. Of the 1779 participants, 41 were omitted from analyses because of significant missing data (leaving >20% of the items blank for at least one measure). This resulted in a sample of 1738 participants (Mage = 15.10 years, SDage = 1.63 years; 822 boys, 908 girls, eight did not specify their sex). For analysis purposes, the sample was divided into two age groups: 935 adolescents aged 12–15 years (Mage = 13.86 years, SDage =1.07 years; 454 boys, 480 girls, one non-specified) and 795 aged 16–18 years (Mage = 16.56 years, SDage = 0.67 years; 367 boys, 427 girls, one non-specified). Eight participants who did not provide their ages were not included in the age-based categorization.

ERQ-CA-J: The ERQ-CA-J used in Study 2 was the same scale as that developed in Study 1.

Behavior for Regulating Emotions Scale: The Behavior for Regulating Emotions Scale24 is an 11-item measure of behavior regarding regulating emotions. The scale has a three-factor structure: positive reappraisal (four items; e.g., “to calm negative emotions, I try to think of the event as a good experience”), distraction (three items; e.g., “to calm negative emotions, I engage in recreation”), and emotional expression (four items; e.g., “to feel positive emotions, I do something to release my emotions”). Each item is rated using a six-point scale (1 = “strongly disagree” to 6 = “strongly agree”).

Trait Subscale of State-Trait Anxiety Inventory: The Trait subscale of State-Trait Anxiety Inventory (STAI)25,26 is a 40-item measure of anxiety symptoms. The STAI has a two-factor structure: a state anxiety subscale and trait anxiety subscale. In the present research, participants rated the trait subscale (T-STAI), which comprises 20-items, with each item rated using a four-point scale (1 = “almost never” to 4 = “almost always”).

Center for Epidemiologic Studies Depression Scale: The Center for Epidemiologic Studies Depression Scale (CES-D)27,28 is a 20-item measure of depressive symptoms. The CES-D has a single-factor structure and each item is rated using a four-point scale (0 = “not at all” to 3 = “over five days a week”).

The software program and estimator used for analysis, criterion of model fit, and the Pearson’s correlations were the same as that in Study 1. For construct validity, regarding the correlation between ES and other validity measures, we hypothesized that it would show no relationship with positive reappraisal, a weak positive relationship with distraction, a weak negative relationship with emotional expression, a weak positive relationship with anxiety, and a weak positive relationship with depression. Further, for CR, it was hypothesized that it would show a moderate positive relationship with positive reappraisal, a moderate positive relationship with distraction, a moderate positive correlation with emotional expression, a moderate negative relationship with anxiety, and a moderate negative relationship with depression.

The means and standard deviations for the two emotion regulation strategies (ES and CR) are shown in Table 1. The means and standard deviations were almost the same as those of the original version.8

We conducted EFA on subsample 1, to examine the unique factorial structure for the Japanese sample. The visual scree test indicated two-factor structure same as the original version as most appropriate. Table 2 shows the rotated factor pattern matrix and extraction communalities. Two factors explained 52.44% of the variance of ERQ-CA scores. Factor loadings on the first factor (CR) ranged 0.47–0.75 and the second factor (ES), 0.52–0.78. The two factors were correlated (r = 0.43). Thus, the EFA indicated the Japanese version of the ERQ-CA has a two-factor structure same as the original version.

We conducted CFA on subsample 2, to confirm the results provided by EFA. Using CFA, a two-factor model, based on the findings of Gross and John,7 showed acceptable fit (CFI = 0.901, RMSEA = 0.086). Following the recommendations of Gullone and Taffe,8 a correlated error was added between item 1 (“when I want to feel happier, I think about something else”) and item 3 (“when I want to feel better (e.g., less sad, angry, or worried), I think about something else”), which have similarly worded phrases, and this produced a better fit (CFI = 0.923, RMSEA = 0.077, see Table 3). In this model, both groups aged 12–15 years (CFI = 0.930, RMSEA = 0.075) and 16–18 years (CFI = 0.910, RMSEA = 0.080) showed acceptable model fit. Item-factor loadings of CR ranged from 0.44 to 0.73 (0.41–0.73 for ages 12–15 years; 0.45–0.71 for ages 16–18 years), and ES ranged from 0.50 to 0.74 (0.51–0.73 for ages 12–15 years, 0.48–0.74 for ages 16–18 years, see Table 4), which was considered sufficient. Therefore, the soundness of the ERQ-CA-J’s two-factor structure was the same as that of Gullone and Taffe.8

Cronbach’s alpha was 0.73 for the ES items (0.71 for boys, 0.75 for girls, 0.73 for age 12–15 years, and 0.71 for age 16–18 years), and was 0.79 for CR items (0.80 for boys, 0.76 for girls, 0.79 for ages 12–15 years, and 0.77 for ages 16–18 years). Thus, the ERQ-CA-J was internally consistent.

We conducted correlation analyses for two ERQ-CA-J factors with each validity scales: the subscales of Behavior for Regulating Emotions Scale (positive reappraisal, distraction, and emotional expression), T-STAI, and CES-D (Table 6). The correlations between the ES score and other validity scales were as expected, and the correlations between the CR score and behaviors for regulating emotions subscales were also as expected.

Table 6

Study 2 Correlations

		Behavior for Regulating Emotions Scale			T-STAI	CES-D
Positive Reappraisal	Distraction	Emotional Expression
Overall Sample	ES	0.08**	0.13**	−0.11**	0.24**	0.22**
CR	0.46**	0.49**	0.36**	0.00	−0.08**
Girls	ES	0.00	0.06	−0.21**	0.25**	0.24**
CR	0.44**	0.48**	0.32**	−0.08*	−0.16**
Boys	ES	0.19**	0.22**	0.02	0.24**	0.20**
CR	0.48**	0.50**	0.39**	0.09*	0.01
Age 12–15 years	ES	0.12**	0.16**	−0.09*	0.27**	0.23**
CR	0.46**	0.53**	0.38**	0.01	−0.07
Age 16–18 years	ES	0.02	0.07	−0.15**	0.20**	0.20**
CR	0.47**	0.43**	0.33**	−0.05	−0.12**

Notes: *p < 0.05, **p < 0.01.

Abbreviations: CES-D, Center for Epidemiologic Studies Depression Scale; CR, cognitive reappraisal; ES, expressive suppression; T-STAI, Trait subscale of State-Trait Anxiety Inventory.

However, the associations between the CR score and some measures for validity investigation differed from expectations. There was almost no correlation between the T-STAI and CR scores and CES-D. This may be because the CR items were misinterpreted as a form of avoidance; it is possible that the participants mistakenly believed that the CR items did not concern changing their ways of thinking but related to thinking about something completely different.

Study 3

The purpose of Study 3 was to investigate the measurement invariance across the children and adolescent age groups and the scale’s test–retest reliability over four weeks.

The institutional review board (IRB) at the University of Tsukuba approved this study’s ethical and scientific protocols (approval number: 筑29-117). Participants were recruited from two elementary schools, three middle schools, and two high schools in Japan. The procedure was the same as in Study 1 for elementary school children and the same as in Study 2 for middle school and high school adolescents.

Overall, 1329 participants completed a second questionnaire to assess test–retest reliability for the ERQ-CA-J with a four-week gap between the two surveys, which was based on Hasani’s29 study of the ERQ’s test–retest reliability over a five-week period. Of these participants, 29 were omitted because they had significant missing data (leaving >20% of the items blank for at least one measure). This resulted in a sample of 1300 participants (Mage = 13.53 years, SDage =2.48 years; 656 boys, 644 girls). In the CFA, the data for the first period were used. For analysis purposes, the sample was divided into three age groups, 265 children aged 8–11 years (Mage = 9.75 years, SDage = 0.98 years; 134 boys, 131 girls), 736 adolescents aged 12–15 years (Mage = 13.60 years, SDage = 1.05 years; 389 boys, 347 girls), and 299 adolescents aged 16–18 years (Mage = 16.70 years, SDage = 0.70 years; 133 boys, 166 girls). Participants were volunteers and received no reward for their participation.

ERQ-CA-J: The ERQ-CA-J used in Study 3 was the same scale as that developed in Study 1 and used in Study 2. The participants completed the measure twice, four weeks apart.

The software program and estimator used for analysis and criterion of model fit were the same as those in Study 1.

The means and standard deviations for the two emotion regulation strategies (ES and CR) are shown in Table 1. These means and standard deviations were almost the same as those of the original version.8

Confirming Structural Validity

Through CFA, the same model as that used in Study 1 and 2 showed acceptable fit (CFI = 0.908, RMSEA = 0.075, see Table 3). In this model, the fitness was calculated for each age group (8–11 years (CFI = 0.897, RMSEA = 0.079), 12–15 years (CFI = 0.876, RMSEA = 0.086), and 16–18 years (CFI = 0.912, RMSEA = 0.074). Although the CFIs of the 8–11 years and 12–15 years groups were slightly lower than the criteria recommended by Hu and Bentler,22 RMSEA showed acceptable fit; thus, the model can be considered to have acceptable fit. Item-factor loadings of CR ranged from 0.45 to 0.69 (0.46–0.68 for ages 8–11 years, 0.41–0.69 for ages 12–15 years, and 0.45–0.72 for ages 16–18 years), and ES ranged from 0.48 to 0.77 (0.45–0.69 for 8–11 years, 0.44–0.78 for 12–15 years, and 0.49–0.88 for ages 16–18 years; see Table 4), which were considered to be sufficient. Therefore, the soundness of the ERQ-CA-J’s two-factor structure was the same as that of Gullone and Taffe.8

Test of Measurement Invariance

As a further investigation, we tested measurement invariance, determining whether the ERQ-CA-J was invariant among the participants’ age groups. Measurement invariance was tested on three distinct levels: (a) configural invariance (i.e., whether similar factors are measured in each age group), (b) factor-loading invariance (i.e., whether the magnitude of factor loadings is the same across each age group), and (c) intercept invariance (i.e., whether the intercept of the regression relating each item to its factor is the same). Chen30 recommended exploring practical model fit changes: if ΔCFI ≤ 0.010, ΔRMSEA ≤ 0.015 for factor-loading invariance, then measurement invariance is evidenced. The results of the investigation of measurement invariance are shown in Table 3. Although the configural invariance model had poor fit with regard to CFI (CFI = 0.889), the fit based on the RMSEA was good (RMSEA = 0.047). The factor-loading invariance model also showed poor fit for CFI (CFI = 0.890), good fit for RMSEA (RMSEA = 0.044). The intercept invariance model showed a poor fit for CFI (CFI = 0.863) and good fit for RMSEA (RMSEA = 0.045). The model fit changes between the configural invariance model and factor-loading invariance model (ΔCFI = 0.001, ΔRMSEA = 0.003) met the criterion by Chen.30 The model fit changes between the factor-loading invariance model and intercept invariance model met the criterion for RMSEA (ΔRMSEA = 0.001), but not for CFI (ΔCFI = 0.027). Therefore, the measurement invariance between age groups was not clearly demonstrated.

Test–Retest Reliability

The intraclass correlation coefficients (ICCs) were used to estimate the stability of the score of the ERQ-CA-J over the four weeks. The ICCs should be 0.75 or higher to show adequate time stability.31 For the CR subscale, the ICCs were 0.76 (0.74 for ages 8–11 years, 0.75 for ages 12–15 years, and 0.79 for ages 16–18 years); for the ES subscale, the ICCs were 0.76 (0.73 for ages 8–11 years, 0.76 for ages 12–15 years, and 0.75 for ages 16–18 years). These findings indicate the ERQ-CA-J had adequate test–retest reliability over the four-week period.

General Discussion

In the present research, we developed and investigated the psychometric evaluation of the ERQ-CA-J using three samples. Overall, the ERQ-CA-J’s factor structure was the same as the original version, and internal consistency, test–retest reliability, and construct validity were supported for those aged 8–18 years.

It is noteworthy that there were some results that differed from expectations; in both Study 1 and 2, the correlations between the CR scores and some other validity scales were unexpected. In Study 1, CR and ES showed almost the same correlation patterns between other validity measures; this may be because CR and ES were not clearly differentiated by the children. Children, and to some extent adolescents, do not always have the neurobiological or cognitive capacities to regulate their emotions.32 As they get older, thinning of lateral prefrontal cortical areas during development facilitates improvement of adolescents’ cognitive emotion regulation abilities.33 In addition, it can be said for both Study 1 and 2 that the CR items could easily have been misunderstood as relating to forms of avoidance. It is possible that the participants mistakenly understood the CR items as not relating to changing their ways of thinking, but to thinking about something completely different. For example, when a child is anxious that he/she may be hated by a peer, the child can, using CR strategies, think of another possibility, such as “the peer does not necessarily hate me.” On the other hand, the child can also think about something completely different, such as “what is for dinner today?”—which constitutes a form of avoidance.

In Study 3, the measurement invariance between age groups was not clearly demonstrated. This finding could be because emotion regulation develops during childhood and adolescence, with better regulation and therefore higher intercepts as they grow older. The ability to utilize cognitively complex processes (e.g., cognitive reappraisal) increases through childhood to adolescence, along with the capacity to flexibly use a wider range of strategies in response to emotions.34,35

This study had some limitations. First, only self-report measures were used in the present study, relying on individuals’ accuracy and honesty. In future, observational or semi-structured interview instruments could be used to eliminate response bias. Second, the internal consistency, test–retest reliability, factor structure, and validity of the ERQ-CA were all determined using classic psychometrics (e.g., using correlation analyses). An emerging model, the clinimetric approach,36 an innovative, clinically based measurement method, has recently been introduced to evaluate not only the psychometric properties but also the clinical validity of rating scales. While classic psychometrics mainly focuses on assessing the frequency of psychometric properties, using self-rating scales where all items have the same weight, clinimetric analysis allows for the treatment of syndromes as heterogeneous constructs with potential differences in weight.37 In future studies, the clinimetric approach may serve as an innovative method for examining the clinical validity of the ERQ-CA. Third, our sample did not contain diverse cultures, and future research should include children and adolescents from diverse cultural backgrounds to enable intercultural comparison, especially in relation to emotion regulation differences between Western and Eastern participants.11 Such a project would clarify the causal factors of differences found between the original and Japanese versions in the current study.

With the present research’s development of the ERQ-CA-J, further studies concerning emotion regulation can be conducted in Japan. Emotion dysregulation is considered both a risk and maintenance factor for psychiatric disorders, particularly emotional disorders such as those relating to depression and anxiety.38 It is desirable to conduct psycho-education regarding emotion regulation for not only clinical populations, but also non-clinical children and adolescents as a means of preventing psychopathology, such as depression and anxiety.