Validation of the Korean Version of the Children's Revised Impact of Event Scale.

OBJECTIVE: This study examined the psychometric properties of the Korean version of the Children's Revised Impact of Event Scale (CRIES) and its validity as a screening instrument for the post-traumatic stress disorder (PTSD).
METHODS: The study population consisted of two samples. The clinical sample consisted of 60 child and adolescent patients from the Department of Neuropsychiatry, Ilsan Paik Hospital, Inje University College of Medicine. The normal sample consisted of 291 students from four schools (primary, middle, and high schools). We administered four self-report questionnaires (the CRIES, Child Reports of Post-traumatic Symptoms [CROPS], State-Trait Anxiety Inventory for Children [STAI-C], and Children's Depression Inventory [CDI]) to 351 children and adolescents after obtaining informed consent from all participants and their parents.
RESULTS: The CRIES showed good reliability (Cronbach's α for the full scale and subscales ranged from 0.85 to 0.93). The total CRIES score was positively correlated with CROPS, STAI-C, and CDI. Confirmatory factor analysis indicated that a three-factor structure for the CRIES (intrusion, avoidance, and hyper-arousal) had a significantly better fit than a two-factor model (intrusion/hyper-arousal and avoidance). Receiver operating characteristic curve analysis indicated that a cutoff of 26 offered the optimum predictive point. That is, this cutoff maximized the balance between sensitivity (0.88) and specificity (0.85). Using this cutoff, the positive predictive value was 0.86, and the negative predictive value was 0.99.
CONCLUSION: These findings imply that the CRIES is a highly accurate diagnostic test in clinical settings.

INTRODUCTION

Traumatic events during childhood and adolescence - which are sensitive periods for neuro-physiological development - have more serious effects than do traumatic events experienced during adulthood. Short-term neuro-physiological responses include hyper-arousal and hyper-sensitization, which can lead to problems with sleeping, eating, peer relationships, school life, and in severe cases, psychotic symptoms such as hallucinations and delusions.1,2) In the long term, exposure to additional or repeated trauma can cause complex psychological problems, including changes in personality that in turn affect emotions, cognition, behaviors, and personal relations.1,3,4)

Most previous studies have been conducted on adults, and thus, little research has investigated the severity of the long- and short-term effects of childhood traumatic experiences. To understand the long- and short-term effects of childhood traumatic experiences on children's growth and development, many factors must be considered in order to carry out effective interventions to help children and adolescents recover from the negative effects of traumatic experiences. For this purpose, effective screening and diagnosis of post-traumatic stress disorder (PTSD) symptoms in children and adolescents exposed to traumatic events is very important for mental health care providers.

Several self-report tools have been developed to diagnose PTSD in children, such as the 15-item Child/Adolescent Impact of Event Scale, the 13-item Children's Revised Impact of Event Scale (CRIES), the Child PTSD Checklist, the Child Stress Reaction Checklist, the Child and Parent Reports of Post-traumatic Symptoms (CROPS & PROPS), and the Child's Reaction to Traumatic Events Scale (CRTES). However, all of these scales have been developed in non-Korean settings.

Of these, the CROPS & PROPS and CRTES are the only survey tools that have been translated into Korean. Neither has been tested for reliability and validity yet, nor are they fully suitable for clinical use. In the present study, therefore, we examined the reliability and validity of the CRIES,5) which is known to overcome the limitations of other questionnaires and is capable of estimating post-traumatic symptoms in children and adolescents. A modified version of Horowitz's Impact of Event Scale (IES) for use with children, the CRIES has been recognized as a valid and stable tool for measuring the impact of traumatic events on children and adolescents worldwide, including the United States and China.

The CRIES was developed as a screening instrument for children at risk of developing PTSD after experiencing a traumatic event, adapted from the IES-8.6) It includes four items measuring intrusion, four items measuring avoidance, and five items measuring arousal, thereby assessing the Diagnostic and Statistical Manual of Mental Disorders, 4th edition (DSM-IV) cluster B, C, D symptoms of PTSD, respectively (Table 1). It demonstrates satisfactory internal consistency and good convergent validity across sex and age groups.7,8,9) However, there remain controversies concerning the stability of the factor structure and cross-cultural validity of PTSD assessment frameworks in children and adolescents.5) Previous research on the cross-cultural validity of CRIES has found inconsistent factor structures in children across different cultures. Two studies reported a two-factor solution (intrusion and avoidance)10,11) using exploratory factor analysis (EFA), and three studies reported a three-factor solution (intrusion, avoidance, arousal)7,12) using confirmatory factor analysis (CFA). These studies demonstrated that intrusion and avoidance are prominent factors of CRIES, while the stability and consistency of the arousal factor has varied across studies.

Table 1

Children's Revised Impact of Event Scale (CRIES)

The CRIES has been used to screen children at risk of developing PTSD for a broad range of traumatic events, but empirical studies investigating the specificity and sensitivity of the self-reported CRIES with a clinical diagnosis of PTSD are rare. One study in China recommended a cutoff score of 32 on the CRIES to obtain the highest consistency between self-reported ratings and clinical diagnosis for PTSD.13) However, in another study, a score of 30 and above was found to be the most effective cutoff.9)

Thus, the aim of the present study was to examine the validity of the Korean version of the CRIES in identifying children at risk of developing PTSD after various traumatic events. Another aim of the study was to verify cross-cultural validity and the factor structure of the CRIES in a Korean sample in order to further understand factor structures across different cultures.

METHODS

Subjects

Children and adolescents aged 10-19 with sufficient reading comprehension to complete self-report questionnaires were recruited for the study. The study population consisted of two samples: the clinical sample was collected from the Department of Psychiatry, Ilsan Paik Hospital, Inje University College of Medicine. Individuals with head injuries and/or intellectual disability with an intelligence quotient (IQ) of 70 or below were excluded. The final participants were 60 child and adolescent hospital patients. The normal sample consisted of 400 students randomly selected from one primary school, two middle schools, and one high school in Jeollabuk-do. One class per level (from higher elementary school grades to higher high school grades) was randomly selected in each school.

A total of 460 children and adolescents participated in this study, with 109 excluded because they did not provide demographic information or did not complete the full scale. Thus, the final study sample consisted of 351 participants (186 boys and 165 girls; 60 clinical and 291 normal). The mean age of this sample was 14.05 years (SD=1.64), and average time elapsed since trauma was 19.7 months.

Participants were divided into three groups. The PTSD group (n=34) included clinical participants who met the full DSM-IV PTSD criteria. The non-PTSD clinical group (n=26) included clinical participants who did not meet full or met sub-syndromal PTSD criteria. Thus, non-PTSD clinical group has no connection with the DSM-IV cluster B, C, D symptoms of PTSD. The non-PTSD clinical group consisted of one participant with bipolar disorder, five with major depressive disorder, nine with attention deficit/hyperactivity disorder, one with schizophrenia, one with obsessive-compulsive disorder, two with conduct disorder, one with panic disorder, one with conversion disorder, one with sleep disorder, two with anxiety disorder, one with impulse-control disorder, and one with adjustment disorder. The normal group (n=291) included all students recruited from the four schools.

Translation of the Children's Revised Impact of Event Scale

First, approval for using CRIES for research and a sample scale of the English version of CRIES were obtained from the Children and War Foundation (http://www.childrenandwar.org/measures/).14) The researchers translated the original text into Korean, which was in turn back-translated into English by a bilingual Korean/English speaker who has lived in English-speaking countries for 7 years. After the translation process, three researchers, each with an M.A. in clinical psychology, checked the internal consistency between both versions based on a 7-point likert scale. They adopted items with five points or higher, and revised those with less than five points for further testing. They repeated this process until consistency was reached. We then conducted a pilot study with 30 individuals aged 10-19 years to determine how comprehensible the questionnaire was for the study population. Any items found to be too difficult or too vague were revised. Finally a specialist psychiatrist with a PhD in clinical psychology reviewed a final version of the questionnaire.

Materials

Children's Revised Impact of Event Scale (CRIES)5)

The CRIES is developed 13-item scale as a screening instrument for children at risk of developing PTSD after experiencing a traumatic event, adapted from the IES-8.6) It includes four items measuring intrusion, four items measuring avoidance, and five new items measuring arousal. Each item is rated on a 4-point scale (0=not at all, 1=rarely, 3=sometimes, and 5=often) with no reversed items. The total score indicates the severity of a child's post-traumatic stress reactions (range of possible scores: 0-65). Earlier studies have demonstrated that the revised scale has reasonably good psychometric properties.7,9) For instance, Smith et al.11) found Cronbach's alpha (α) to be 0.80 for the total scale, and 0.70, 0.73, and 0.60 for intrusion, avoidance, and arousal subscales, respectively. A score of 30 or above has been confirmed as the most effective cutoff score for screening cases of PTSD.9)

The Child and Parent Reports of Post-traumatic Symptoms (CROPS & PROPS)15)

For testing the concurrent validity of the CRIES, we used the CROPS & PROPS, which is a self-report measure for children and adolescents that assesses a broad range of post-traumatic symptoms. It consists of two scales, 26-item Child Report of Post-traumatic Symptoms (CROPS) and 32-item Parent Report of Post-traumatic Symptoms (PROPS). Each item on the CROPS is rated on a 3-point scale, scored 0, 1, or 2 with no reversed items. The total score indicates the severity of the child's post-traumatic stress reactions (range of possible scores: 0-52). Each item on the PROPS is rated on a 3-point scale, scored 0, 1, or 2 with no reversed items. The total score indicates the severity of child's post-traumatic stress reactions from the parents' perspective (range of possible scores: 0-64). The Korean version has been shown to be reliable and valid.16) In a sample of Korean high school students, the CROPS revealed Cronbach's αvalues from 0.60 to 0.73 and the PROPS revealed Cronbach's αvalues from 0.65 to 0.75. In this study, only the CROPS was used (Cronbach's α=0.95).

Children's Depression Inventory (CDI)17)

For testing the convergent validity of the CRIES, we used the CDI to measure the severity of childhood depression. Each item has three categories (e.g., "I have fun in many things," "I have fun in some things," and "Nothing is fun at all"), which are answered based on the respondents' experiences within the last two weeks. Each category is rated on a 3-point scale, scored 0, 1, or 2. The Korean version has been shown to be reliable and valid.18) In a sample of Korean elementary school children, a Cronbach's α of 0.88 was reported. In the present study, Cronbach's α was 0.93.

State-Trait Anxiety Inventory for Children (STAI-C)19)

For testing the convergent validity of the CRIES, we selected the State-Trait Anxiety Inventory for Children (STAI-C), which is a self-report measure for assessing childhood anxiety. It consists of two scales, namely, the 20-item state anxiety scale and the 20-item trait anxiety scale. Each item is rated on a 3-point scale (1, 2, 3). The total score indicates the severity of children's anxiety (range of possible scores: 20-60). The Korean version has been shown to be reliable and valid.20) In this study, Cronbach's αs were 0.94 and 0.92 for state and trait anxiety, respectively.

Procedure

Ethical approval was obtained from the Institutional Review Board (IRB) at Ilsan Paik Hospital, Inje University College of Medicine. All researchers attended an ethics education session hosted by the Korea Association of Institutional Review Board.

Data were collected between February 2012 and February 2013. For the clinical sample, research staff from the Department of Psychiatry, Ilsan Paik Hospital, informed all patients about the study. Written informed consent was obtained from the patients and their parents. The patients were assured that their information would be kept confidential and that their participation was voluntary. The patients completed a structured and anonymous questionnaire. Four clinical psychologists and one psychiatrist conducted interviews with all patients in accordance with the Structured Clinical Interview for DSM-IV Axis I Disorders (SCID-I). Patients who fully met the DSM-IV PTSD criteria were assigned to the PTSD group, whereas those who did not meet all the criteria or had sub-syndromal PTSD were assigned to the non-PTSD clinical group.

For the normal sample, teachers were trained to administer the CRIES in class. Oral informed consent was obtained from both the students and their parents before they completed the questionnaire. The IRB at Ilsan Paik Hospital, Inje University College of Medicine approved the verbal consent procedure. The students completed the questionnaire anonymously in their classrooms. The students were assured that data processing and presentation would be completely anonymous.

Statistical Analysis

SPSS software ver. 15.0 (SPSS Inc., Chicago, IL, USA) was used for assessing the reliability and validity of CRIES. Descriptive statistics (means and standard deviations [SDs]) are presented. Cronbach's αs and item-scale correlation coefficients were also computed. We estimated validity using Pearson correlation coefficients between the CRIES and other psychological measures (i.e., depression, anxiety, and post-traumatic stress symptoms). We also compared mean scores between the three groups (i.e., PTSD group, non-PTSD clinical group, and normal group) on socio demographic characteristics and psychometric measures.

CFA of CRIES was conducted using AMOS 7.0 (IBM, Armonk, NY, USA). Two models were tested across the whole sample. Model 1 tested two inter-correlated latent factors, namely, avoidance and intrusion/arousal. Model 2 tested three inter-correlated latent factors, namely, intrusion, avoidance, and arousal.

The robust maximum-likelihood estimator was used for data analysis. Indices used to assess the fitness of each model included the comparative fix index (CFI),21) the Tucker-Lewis index (TLI)22) and the root mean square error of approximation (RMSEA). 23) The CFI and TLI compare the fit of the hypothesized model to the independent model, and higher values are preferred (>0.90, acceptable; >0.95, good).24) The RMSEA is an index of the fit between the hypothesized model and the saturated model; lower values are preferred (>0.10, bad; 0.08-0.10, ordinary; 0.06-0.08, favorable; <0.05, good).24) Receiver operating characteristic (ROC) analysis using SPSS software ver. 15.0 was conducted to estimate sensitivity, specificity, and the predictive value of the CRIES as well as the most effective cutoff score for screening cases of PTSD.

RESULTS

Description of the Traumatic Event

For traumatic events, 35.3% experienced the death of someone close (e.g., family, friend, relative), 15.1% witnessed that individual suffering from serious maladies (e.g., cancer, stroke, epilepsy), 13.4% were physically or verbally abused by someone close, 6.6% directly experienced traumatic events (e.g., traffic accident, crime, explosion), 6.3% suffered natural disasters (e.g., flood, earthquake, typhoon), 6.3% suffered serious maladies (e.g., cancer, stroke, epilepsy), 2.8% suffered sexual abuse (either from someone close or strangers), 2.6% witnessed someone close being attacked or injured, 1.7% were physically or verbally abused by strangers, and 10% experienced some variety of other traumatic events. For the time elapsed since the traumatic event, 33.6% indicated more than three years; 31.3%, six months to three years; 17.9%, three months to six months; 7.4%, a month to three months; and 9.7%, within the last month.

Reliability

For the CRIES, Cronbach's α for all items was 0.93 and was 0.88, 0.85, and 0.85 for the intrusion, avoidance, and arousal subscales, respectively. The item-total correlations were acceptable (range: 0.58-0.75). The item-total correlation for item 10 ("Do you try not to think about it?") was the highest (0.75), and that of item 13 ("Do you have sleep problems?") was the lowest (0.58) (Table 2).

Table 2

Pearson correlation among the CRIES, CROPS, STAI-C, CDI

CRIES, Children's Revised Impact of Event scale; it includes four items measuring intrusion (CRIES-intrusion), four items measuring avoidance (CRIES-avoidance), five items measuring arousal (CRIES-hyper arousal).

CROPS, The Child and Parent Reports of Post-traumatic Symptoms.

STAI-C, State-Trait Anxiety Inventory for Children; it includes 20 items measuring state-anxiety, 20 items measuring trait-anxiety.

CDI, Children's Depression Inventory.

*p<0.01.

Validity

Total scores on CRIES and CROPS scales were moderately correlated (r=0.68, p<0.01) across the whole sample, thus providing evidence of concurrent validity. Furthermore, CROPS total scores were moderately correlated with the CRIES hyper arousal subscale (r=0.72, p <0.01), intrusion subscale (r=0.56, p<0.01), and avoidance subscale (r=0.57, p<0.01). In the assessment of convergent validity, the CRIES total scores were moderately associated with state anxiety (r=0.62, p<0.01) and trait anxiety (r=0.57, p<0.01) as measured by the STAI-C, as well as the CDI (r=0.56, p<0.01) (Table 3).

Table 3

Comparisons of CRIES, CROPS, STAI-C, CDI among three groups

Values are presented as mean (standard deviation).

CRIES, Children's Revised Impact of Event Scale; CROPS, The Child and Parent Reports of Post-traumatic Symptoms; CDI, Children's Depression Inventory.

STAI-C, State-Trait Anxiety Inventory for Children; it includes 20 items measuring state-anxiety, 20 items measuring trait-anxiety.

Post-traumatic stress disorder (PTSD) group, meet full PTSD criteria in clinical sample; non-PTSD clinical group, do not meet full or subsyndromal PTSD criteria in clinical sample normal group, non-clinical sample.

A one-way analysis of variance (ANOVA) was used to investigate between-groups differences. For socio-demographic characteristics, there were no significant differences among the three groups for gender (F=2.299, p>0.05) or age (F=0.040, p>0.05). There were significant differences among the three groups for other psychometric measures. Specifically, Scheffé's post-hoc comparisons revealed that the CRIES mean score for the PTSD group was 23.23 points higher than that of the non-PTSD clinical group (p<0.05) and 27.40 points higher than that of the normal group (p<0.05), while there was no difference between the non-PTSD clinical group and the normal group (Table 3).

Factor Structure of the Korean Version of the CRIES

In order to determine the appropriate number of factors (two or three), the following two models were tested (Table 4). Model 1 tested for two inter-correlated latent factors (avoidance and intrusion/hyper-arousal). Model 2 tested for three inter-correlated latent factors (avoidance, intrusion, and arousal). CFA revealed satisfactory fit for Model 2 (χ2=125.23, degree of freedom [df]=59, p<0.001; CFI=0.97; goodness of fix index [GFI]=0.95; RMSEA=0.06; TLI= 0.97). Model 2 had significantly better fit than did Model 1 (χ2=319.22, df=64, p<0.001; CFI=0.91; GFI=0.87; RMSEA=0.11; TLI=0.89). Therefore, we adopted Model 2 (intrusion, avoidance, and hyper arousal) as our final model (Fig. 1).

Table 4

Goodness-of-fit indices for the specified models across the whole sample (n=351)

df, degree of freedom; CFI, comparative fix index; GFI, goodness of fit index; TLI, Tucker-Lewis index; RMSEA, root mean square error of approximation.

Fig. 1

Confirmatory factor analysis for the Korean version of the CRIES.

Fit indices: χ2 (59)=125.23, p<0.001, CFI=0.97, GFI=0.95, RMSEA=0.06, TLI=0.97.

Standardized path coefficients are shown.

CFI, comparative fix index; GFI, goodness of fit index; RMSEA, root mean square error of approximation; TLI, Tucker-Lewis index.

Predictive Performance

In the ROC curve analyses for the Korean version of CRIES, area under the curve (AUC) was excellent, at 0.91 (p<0.001, standard error=0.03) (Fig. 2). Inspection of the various cut-off scores on the CRIES revealed that a score of ≥26 was the optimum predictive point. That is, the cutoff maximized the balance between sensitivity (0.88) and specificity (0.85). Using this cut-off, the positive predictive value (PPV) was 0.86 and negative predictive value (NPV) was 0.99. These results imply that the CRIES is a highly accurate diagnostic test in clinical settings.

Fig. 2

Receiver operating characteristic (ROC) curve: total (n=351).

Area under the curve=0.91, standard error=0.03, p<0.001.

DISCUSSION

The current study evaluated the psychometric properties of the Korean version of the CRIES and verified its clinical diagnostic value by using it to assess traumatic stress reactions of child patients at the department of neuropsychiatry in a hospital.

The internal consistency of the total PTSD score was excellent (Cronbach's α=0.93), with acceptable values for the three subscales (Cronbach's α=0.88, for intrusion, 0.85 for avoidance, and 0.85 for arousal). These were higher than were those reported by Smith et al.5) and Giannopoulou et al.7) Item-scale correlations were also satisfactory, ranging from 0.58 to 0.75.

The CROPS is a self-report measure for children and adolescents that assesses a broad range of post-traumatic symptoms in accordance with the DSM-IV PTSD symptom clusters. The strong correlation between the total CROPS and CRIES scores (r=0.68) demonstrates the concurrent validity of the scale. The significant correlations between the total CRIES scores and state anxiety (r=0.62) and trait anxiety (r=0.57) STAI-C scores, as well as total CDI scores (r=0.56), provide evidence for the convergent validity of the scale. A one-way ANOVA revealed significant differences between the three groups on mean CRIES scores. Especially, the CRIES mean score for the PTSD group was 23.23 points higher than that of the non-PTSD clinical group (p<0.05). It appears CRIES has good discrimination capacity to distinguish not only between the PTSD group and the normal group but also between the PTSD group non-PTSD clinical groups in clinical settings.

Previous studies have disagreed on the CRIES factor structure. In a factor analytic study of the CRIES-13, Smith et al.11) found that the arousal items loaded strongly on the four-item intrusion scale, and suggested a two-factor structure (intrusion and avoidance/arousal). However, in our study, the three-factor structure (intrusion, avoidance, and arousal) had a significantly better fit than did the two-factor structure. This is in line with Giannopoulou et al.,7) Perrin et al.9) and Lau et al.8) and implies that Korean adolescents' symptom patterns in response to trauma are best reflected by the three-factor structure, in accordance with the DSM-IV PTSD symptom clusters. Future research should examine the stability of the arousal factor of CRIES following different traumatic events. The present study confirms earlier findings that intrusion and avoidance are robust and independent factors of CRIES across cultures and different types of traumatic experiences. This suggests that post-traumatic stress reactions transcend cultural barriers, and that the CRIES factor structure is robust across different types of trauma.

ROC curve analyses were conducted to examine the validity of the Korean version of CRIES for identifying children at risk of developing PTSD. Previous research9,25) suggested a CRIES cutoff point of 30. In our study, a cutoff of 26 was the optimum predictive point. That is, the cutoff maximized the balance between sensitivity (0.88) and specificity (0.85). Using this cutoff, the PPV was 0.86 and NPV was 0.99. These results imply that the Korean version of the CRIES is a useful instrument for diagnosing children's and adolescents' PTSD in clinical settings.

The study had several limitations. First, our sample was recruited from only four schools in Jeollabuk-do and one hospital in Gyeonggi-do. Therefore, caution must be exercised when generalizing the results of this study to the general Korean population. Second, in our study, participants' traumatic events and the elapsed time from the occurrence of the traumatic events varied considerably. Post-traumatic symptoms and their clinical features could be affected by trauma type, number, and time elapsed since trauma. Furthermore, the Jeollabuk-do area is different from the Gyeonggi-do area in terms of cultural background. Thus, future studies should seek to maintain homogeneity between samples.

In our study, there was a significant but weak correlation between type of trauma and CRIES total score (r=0.13, p<0.05), and between time elapsed and total CRIES score (r=-0.14, p<0.01). Therefore, we conducted an analysis of covariance to verify whether these variables (types of trauma and time elapsed) affected group differences in total CRIES score. The significant differences between the three groups (PTSD group, non-PTSD clinical group, normal group) remained even when controlling for trauma type and time elapsed.

In conclusion, the Korean version of the CRIES is a valid and reliable self-report instrument that assesses the symptoms of PTSD among Korean children and adolescents who have experienced various traumatic events. This instrument can be used in future PTSD research and in hospital settings to aid in the prevention and treatment of childhood PTSD.