Validation of the Impact of Event Scale-Revised for adolescents experiencing the floods and mudslides.

The purpose of this study was to validate the Impact of Event Scale-Revised (IES-R) for adolescents who had experienced the floods and mudslides caused by Typhoon Morakot in Taiwan. The internal consistency, construct validity, and criteria validity of the instrument were examined. Principal component analysis followed by an oblique rotation was used to derive a three-factor solution. These factors were labeled intrusion, hyperarousal, and avoidance; all three factors together accounted for 58.1% of the variance. The total Cronbach's alpha of 0.94 reflected the good internal consistency of the instrument. With reference to diagnosis of posttraumatic stress disorder, the IES-R cutoff point for posttraumatic stress disorder was 19 of 20 with a sensitivity of 85.7% and specificity of 84.1%. In conclusion, the IES-R can be used as a reliable and valid instrument when evaluating psychological distress among adolescents who have experienced a natural disaster, such as flooding and mudslides.

Introduction

Typhoon Morakot brought unprecedented amounts of rainfall in Taiwan peaking at 2,777 mm (109.3 in.) and caused severe flooding and enormous mudslides on August 8, 2009. The result was catastrophic damage in southern Taiwan, which left more than 700 people dead and caused roughly $3.3 billion USD in damages. Most of deaths in this disaster were as a result of terrifying mudslides, and nearly all of the inhabitants living in the affected mountainous areas experienced the threat of mudslides. Typhoon Morakot and the associated floods and mudslides had a great psychological impact on these inhabitants (http://en.wikipedia.org/wiki/Typhoon_Morakot).

Adolescents as a population are vulnerable to posttraumatic stress disorder (PTSD) when they are exposed to life-threatening stressors. Epidemiological studies for PTSD in Taiwan have shown higher prevalence of PTSD among the people experiencing the earthquake disasters and infection outbreak [1], [2], including adolescents [3], [4]. PTSD symptomatology in adolescents resembles that in adults but also includes some aspects of the childhood presentation with modifications [5]. Adolescents might present more frequently with repetitive and intrusive thoughts and flashbacks, sleep disturbance, anger, separation anxiety, and cognitive problems [6]. Therefore, it is not always suitable to detect PTSD in adolescents using an instrument that has not been specifically validated for adolescents.

There has been several instruments developed for screening of PTSD, such as Clinician-Administrated PTSD Scale [7], PTSD Checklist [8], Post-traumatic Stress Diagnostic Scale [9], and the Impact of Event Scale (IES). The latter three instruments were self-administered with similar numbers of items. Each of these assessment tools have their own merits. As our study team has used IES to screen stress reaction among the hospital adult workers during the severe acute respiratory syndrome outbreak [2], we would like to examine its use for the adolescents in the natural disaster. IES was developed by Horowitz et al. [10] as a measure of psychological stress reactions after trauma. IES is a short, easily administered, self-report questionnaire of 15 questions. It has been widely adopted and used in related research for more than 30 years. The revised version of the IES (IES-R) contains seven additional items related to the hyperarousal symptoms of PTSD [11]. The IES-R has been used to measure mental reactions in response to stressors, such as injury [12], [13] and illness [14], [15]. Despite its wide use, this tool has not been specifically and formerly validated for posttraumatic responses to natural disaster among adolescents or children.

The aim of this study was, therefore, to examine the internal consistency, factor structure, and criteria validity of the IES-R in a sample of adolescents who had experienced the threat of floods and mudslides caused by Typhoon Morakot. Additionally, the implications of these findings, in terms of the psychometric properties of the IES-R for use with natural disasters, are discussed.

Methods

This study was conducted in the heavy damaged area 3 months following the floods and mudslides caused by Typhoon Morakot. A total of 277 adolescents who were in Grade 7–9 from three junior high schools were recruited into this study. They had been separated from their families and allocated to three boarding areas where they could continue to receive formal education in a group setting. Most of them were living in group dormitories and visits involving family members were only possible at weekends. The Institutional Review Board of Kaohsiung Medical University approved the study, which allowed the use of passive consent from the parents of 277 displaced adolescents. Before conducting the study, we prepared a leaflet explaining the purpose and procedure of this study. Students were told to take the leaflet with them at a weekend for their parents or main caretakers to inspect. If parents disagreed about taking part in the study, they could tell their children to refuse to participate. Written informed consent was also obtained from each student themselves before the study started.

The participants were invited to complete the self-administered 22-item Chinese version of IES-R [11], [16]. Participants were asked to report the frequency of symptoms over the past week on a 5-point scale ranging from 0 (not at all) to 4 (extremely). The IES-R yields total scores that range from 0 to 88. The sum of all individual scores was taken to indicate the severity of the psychological reactions to the previous stress.

Blind to the results of the IES-R, diagnostic interviews were conducted using Mini-International Neuropsychiatric Interview for Children and Adolescents (M.I.N.I.-Kid) schedule [17]. Based on the Diagnostic and Statistical Manual of Mental Disorders, 4th Edition (American Psychiatric Association, 1994), the M.I.N.I.-Kid assesses 22 common psychiatric disorders and suicide risk in children and adolescents. It has been used previously among early adolescents in Taiwan [18]. Because of time limitations and specific purpose of this study, only the PTSD module of the M.I.N.I.-Kid was used for the interview. Fifteen psychiatric staff, made up of two senior board-certified child psychiatrists, five psychiatrists, five clinical psychologists, and three psychiatric nurses participated as interviewers. All of them had many years of clinical experience in both adolescent and adult psychiatry. Before conducting the diagnostic interviews with the adolescents who had experienced Typhoon Morakot, all interviewers received comprehensive training in the use of the M.I.N.I.-Kid. First, all interviewers attended a workshop on the M.I.N.I. held by the Taiwanese Society of Psychiatry. The workshop introduced the M.I.N.I.’s concept, its content, and the principle of use; it also included a video-recorded case interview using the M.I.N.I. that allowed the attendees to rate a diagnosis. Second, two senior board-certified child psychiatrists (P. Y. and C. F. Y.) delivered a seminar to the other interviewers to introduce the M.I.N.I.-Kid modules. Third, each interviewer conducted a diagnostic interview with an adolescent using the PTSD module of the M.I.N.I.-Kid under the supervision of a child psychiatrist (C. F. Y.) and then received feedback about the interview. Lastly, the inter-rater diagnostic reliability among the 15 interviewers was examined using six tape-recorded M.I.N.I.-Kid case interviews. The kappa for the PTSD ratings ranged from 0.67 to 1, which indicated good inter-rater diagnostic reliability.

Cronbach’s alpha coefficients were calculated to determine the internal consistency of the IES-R. Exploratory factor analysis, using principle component analysis followed by oblique rotation, was used. Factor numbers were determined using the scree test and needed to explain at least 10% of the variance. In the scree test, the eigenvalues of all the factors were examined after each factor was extracted until a large jump or discontinuity was observed, after which the factors that remained were retained. Items with loadings greater than 0.3 were entered into a factor. If the factor loading was greater than 0.4 for more than two items, the author would judge the appropriate factor for the item. The criteria validity of IES-R against the diagnosis of PTSD as generated by the M.I.N.I.-Kid was determined via relative operating characteristic (ROC) analysis [19]. Accordingly, the optimal cutoff point was estimated with respect to sensitivity, specificity, positive predictive value (PPV), and negative predictive value (NPV).

Results

Six students were excluded because of absence on two consecutive visits. There was no difference in the gender ratio of these six students (n  = 6, 4 boys and 2 girls) with that of the groups who participated. However, these six students were a little older than their counterparts (Z of Mann Whitney U test = −2.645, p  < 0.01). In total, 271 students (97.8%) completed all questionnaires and the interview. Among them, 124 (45.8%) were boys and 147 (54.2%) were girls. Their mean age was 13.4 years (standard deviation = 1.0 years, range: 12–15 years). Of the 271 students, 70 (25.8%) met the PTSD diagnostic criteria in relation to Typhoon Morakot. The mean of the total scores of the IES-R was 16.9 (standard deviation = 16.2) with a range from 0 to 82.

Factor analysis yielded three factors with eigenvalues exceeding unity, and these accounted for 58.1% of the total variance. Six items (Item 1, 3, 6, 9, 14, and 20) with loadings greater than 0.3 made up the Factor I subscale accounting for 20.1% of the explained variance. Factor II consisted of nine items (Item 5, 7, 8, 11, 12, 13, 14, 17, 21, and 22) and accounted for 18.4% of explained variance. Factor III consisted of seven items (Item 2, 4, 10, 15, 16, 18, and 19) and accounted for 19.6% of the explained variance. According to the meanings of the items making up each factor, the three factors were labeled as intrusion, hyperarousal, and avoidance, respectively (Table 1 ).

Table 1

Factor loading of the Impact of Event Scale-Revised items

Original factor and items [20]		Factor IIntrusion	Factor IIAvoidance	Factor IIIHyperarousal
Intrusion factor
1	Any reminder brought back feelings about it	0.883	−0.087	0.023
2	I had trouble staying asleep	0.143	−0.212	0.884
3	Other things kept making me think about it	0.946	−0.218	0.103
6	I thought about it when I did not mean to	0.492	0.414	−0.101
9	Pictures about it popped into my mind	0.494	0.370	−0.023
16	I had waves of strong feelings about it	0.278	0.251	0.379
20	I had dreams about it	0.341	0.238	0.182
Avoidance factor
5	I avoided letting myself get upset when I thought about it or was reminded of it	0.153	0.347	0.151
7	I felt as if it had not happened or was not real	0.113	0.598	−0.176
8	I stayed away from reminders about it	−0.004	0.781	−0.038
11	I tried not to think about it	−0.126	0.457	0.389
12	I was aware that I still had a lot of feelings about it, but I did not deal with them	−0.057	0.731	−0.055
13	My feelings about it were kind of numb	−0.070	0.508	0.212
17	I tried to remove it from my memory	−0.142	0.646	0.112
22	I tried not to talk about it	0.303	0.466	−0.032
Hyperarousal factor
4	I felt irritable and angry	0.060	0.113	0.512
10	I was jumpy and easily startled	0.150	0.201	0.444
14	I found myself acting or feeling like I was back at that time	0.502	0.334	0.024
15	I had trouble falling asleep	−0.026	−0.121	0.917
18	I had trouble concentrating	−0.122	0.208	0.608
19	Reminders of it caused me to have physical reactions, such as sweating, trouble breathing, nausea, or a pounding heart	0.195	0.086	0.457
21	I felt watchful and on guard	−0.045	0.570	0.127
		0.883	−0.087	0.023
% Of explained variance		20.1	18.4	19.6
Reliability (Cronbach’s alpha)		0.90	0.85	0.88

Based on the items making up the newly extracted factors, the analyses of internal consistency revealed Cronbach’s alpha values of 0.89, 0.85, and 0.88 for the intrusion, avoidance, and hyperarousal subscales, respectively. The overall Cronbach’s alpha for the total IES-R was 0.94.

The criteria validity of the IES-R for PTSD was examined using ROC analyses (Fig. 1 ). The area under the ROC curve was 0.92 (95% confidence interval = 0.88–0.96), which indicated excellent criteria validity for the IES-R in relation to a diagnosis of PTSD. Table 2 shows the criteria validity of the IES-R in terms of sensitivity, specificity, PPV, NPV, and overall misclassification rate (OMR) at various different cutoff points. As the IES-R is treated to be used as a screening tool, which suggests a requirement for greater sensitivity, we suggest that the optimal cutoff point for the IES-R should be 19 of 20, which gives a sensitivity of 85.7%, a specificity of 84.1%, a PPV of 65.2%, a NPV of 94.4%, and an OMR of 15.5%.

Figure 1

The relative operating characteristic curve of the revised version of the Impact of Event Scale at different cutoff points.

Table 2

Validity indices (%) of the Impact of Event Scale-Revised at different cutoff scores using the diagnosis of posttraumatic stress disorder as the criterion for “case” definition

Cutoff point	Sensitivity (%)	Specificity (%)	PPV (%)	NPV (%)	OMR (%)
18/19	85.7	83.1	63.8	94.4	16.2
a19/20	85.7	84.1	65.2	94.4	15.5
20/21	82.9	86.6	68.2	93.5	14.4
21/22	78.6	87.6	68.8	92.1	14.7

NPV = negative predictive value; OMR = overall misclassification rate; PPV = positive predictive value.

optimal cutoff point.

Discussion

In this study, we found that the IES-R as an instrument for evaluation of adolescent psychological stress associated with exposure to nature disasters, such as flooding and mudslides, has a three-factor structure. These factors assess intrusion, hyperarousal, and avoidance symptomatology. The high Cronbach’s alpha values for all subscales as well as the total scale indicates very good internal consistency. The optimal cutoff point for the IES-R in this study, when detecting PTSD, was determined to be 19 of 20, which gives both good sensitivity and good specificity.

The IES-R factor structure in this sample was identical to that in the original studies at the component level with minor differences at the item level [20], although a validation study did not support three factors model for a sample of accident participants or emergency room patients in Hong Kong [16]. As the latter article explained, difference in validation of IES-R may come from different clinical samples and age groups studied. Similarities in the IES-R factor structure when the present study and the original study are compared imply that the same psychological distress psychopathologies occurred with the present flooding stressor. Compared with the original factor analysis [20], four items in our study did not load on their originally proposed factors. These were Item 2 (I had trouble staying asleep) and Item 16 (I had waves of strong feelings about it), which was reclassified as hyperarousal rather than intrusion; Item 14 (I found myself acting or feeling like I was back at that time), which was reclassified as intrusion rather than hyperarousal; and Item 21 (I felt watchful and on guard) from hyperarousal to avoidance. These inconsistencies at item level may be attributed to differences in the presentation of PTSD between adolescents and adults. In addition, it may also reflect that these items are more ambiguous because they each incorporated aspects of all three factors. In these circumstances, it is not unexpected that these loading may not occur consistently on the same factor during a different study.

We found good internal consistency coefficients for all three extracted factors and as a whole the reliabilities were also good; this implies that each of the measures is a relatively homogeneous construct. The criteria validity of the IES-R was demonstrated by the high validity index against the diagnosis of PTSD using a standardized interview schedule. The cutoff point suggested here is 19 of 20, although greater specificity, PPV, and OMR can be had by a cutoff of 20 of 21. However, the difference between the two cutoff points is small. There is a need for greater sensitivity when the IES-R is used as a screening tool and therefore it is suggested that adolescents with scores of 20 and more from the IES-R should be selected as possibly suffering from PTSD.

At this cutoff point, the PPV was 65.2%, indicating that 34.8% of the adolescents scoring 20 or more would not meet the PTSD diagnosis. This relatively higher false-positive rate may reflect that anxiety syndromes other than PTSD are present in the adolescent. In addition to PTSD, three other distinct types of posttraumatic response, namely depression, anxiety disorders, and behavioral disorders, have been proposed [21]. It was possible that those with anxiety disorder but not PTSD also show up with higher IES-R scores. Previous studies have found that anxiety disorder may include social phobia, agoraphobia, and separation anxiety disorder [22], [23]. Our findings seem to suggest that, in addition to the IES-R’s use as a screening tool, other anxiety symptomatology assessment tools are needed for adolescents in these circumstances. Nonetheless, overall, this study demonstrates that the IES-R performs well as a screening instrument for PTSD among adolescents and is very adequate in this role.

Study caveats need to be addressed. This study was conducted about 3 months after Typhoon Morakot had occurred. Thus, this result cannot be generalized to the very acute postdisaster stage. At the acute stress stage, it is possible that the posttraumatic reactions are more salient and show more variability. Resilience can help some people to return to a normal mental state from an acute stress reaction quickly, whereas others suffer from persistent mental dysfunction. They are the candidates for additional mental help during the reconstruction or rehabilitative stage after a disaster. In this context, we suggest that the IES-R needs to be testified for identifying this group to provide a higher range of application. In addition, convergence and divergence validity testing needs to be testified to strengthen the validation in the further studies.

In conclusion, our study suggests that the IES-R appears to be a promising and readily applicable instrument for the quick and early detection of psychological distress among adolescents who have experienced a natural disaster, such as a severe threat of flooding and mudslides.