A pilot randomized controlled trial of the e-couch anxiety and worry program in schools.

The aim of this pilot study was to evaluate the acceptability and feasibility of delivering an online anxiety prevention program in schools, and to assess the effectiveness of the intervention in reducing symptoms of anxiety. Three schools located in South Australia and the Australian Capital Territory were recruited to participate in the trial, with classes randomly allocated to the intervention or wait-list control condition. All participants (N = 225) were invited to complete a pre-intervention, post-intervention and 3-month follow-up questionnaire. Participants in the intervention condition completed the online e-couch Anxiety and Worry program during one class period a week for six weeks. No significant differences were found between the intervention and control conditions at post-intervention or 3-month follow-up for generalised anxiety (Cohen's d = - 0.09-0.08), social anxiety (d = 0.09 & - 0.26), anxiety sensitivity (d = 0.19 & - 0.15), depressive symptoms (d = 0.01 & 0.08) or mental wellbeing (d = 0.17 & 0.30). Online anxiety prevention programs are acceptable and can be feasibly delivered in schools. Although not significant, the sizes of some of the effects obtained in this pilot trial are consistent with earlier studies, and warrant further investigation in a larger trial.

RCT Entities:   Population Interventions Outcomes

Introduction

Anxiety disorders are one of the most common psychological problems in children and adolescents and often precede the development of depression (Essau, 2008, Kendall et al., 2004). During adolescence, between 8% and 27% of young people experience an anxiety disorder, with 1-year prevalence rates between 5% and 21% (Angold et al., 1999, Barrett et al., 2006, Boyd et al., 2000, Costello et al., 2005, Zimmermann et al., 2003). Anxiety disorders and symptoms can be associated with a range of problems for young people, including academic, social and other emotional difficulties (Donovan and Spence, 2000). Given the internal nature of anxiety, these disorders often go unrecognized by the young person, their parents and teachers (Barrett and Pahl, 2006, Donovan and Spence, 2000, Wells et al., 2001). Also, help-seeking by young people for anxiety is typically low (Essau, 2005, Farmer et al., 2003).

Due to the high prevalence of anxiety disorders and their associated ill effects, the prevention of anxiety in young people has become a focus (Donovan and Spence, 2000). This has led to the development of a number of effective school-based interventions designed to prevent and reduce symptoms of anxiety (Neil and Christensen, 2009). A review of anxiety prevention programs in schools reported that 69% of universal programs (which are delivered to all students regardless of their symptom level) significantly reduced symptoms of anxiety at post-intervention and/or follow-up (Cohen's d = 0.31–1.37; Median = 0.41; Neil and Christensen, 2009). The delivery of these interventions in schools is appealing given their reach to young people. Despite the apparent effectiveness of anxiety prevention programs, they are not routinely delivered in the school environment. Reasons for this are varied but the reluctance of teachers to deliver these programs due to a lack of expertise or training in mental health appears to be a key factor (Spence et al., 2005).

The delivery of web-based anxiety prevention programs in schools may circumvent this difficulty by providing self-directed programs that require little teacher training or knowledge and enable independent student learning. To date, very few online anxiety prevention programs have been evaluated in schools (Calear and Christensen, 2010, Clarke et al., 2015). One intervention that has been evaluated in this setting is the MoodGYM program, which was found to significantly reduce anxiety symptoms at post-intervention and 6-month follow-up (Cohen's d = 0.15–0.25; Calear et al., 2009) relative to a wait-list control condition. Another effective online prevention program that has been evaluated in schools is the Thiswayup Schools Depression and Anxiety intervention. Both the Depression (Cohen's d = 0.29) and Anxiety programs (Cohen's d = 0.18) were found to significantly reduce anxiety (Wong et al., 2014) in a universal randomized controlled trial. These programs provide preliminary support for the use of online anxiety prevention programs in schools.

The aim of the current pilot trial was to (a) test the acceptability and feasibility of delivering the e-couch Anxiety and Worry program in schools, an online anxiety program developed as part of the larger e-couch suite of mental health programs, (b) trial the study design, protocols, and materials ahead of a larger cluster randomized controlled trial of the e-couch Anxiety and Worry program, and (c) assess the potential effectiveness of the e-couch Anxiety and Worry program in reducing symptoms of anxiety. The e-couch Anxiety and Worry program is freely available to the public. As such, it is important to establish the acceptability, feasibility and effectiveness of this program in a range of settings, including schools, in order to tailor guidelines for its use.

Material and methods

Design

A cluster randomized controlled trial design was employed in the current study, with classes within participating schools randomly allocated to the intervention or wait-list control condition. An independent statistician randomly allocated classes to conditions using a computerized random number generator. The identity of the classes was concealed from the statistician during this process. All students were invited to complete a pre-intervention, post-intervention, and 3-month follow-up questionnaire.

Participants

Three schools from the Australian Capital Territory and South Australia were recruited to participate in the Y-Worri pilot trial. Of these schools, two were coeducational public high schools and one was a private single-sex girl's secondary school. Two of the three schools were located in a metropolitan area. A total of 225 students (38 males, 159 female) consented to participate in the trial. Participant age ranged from 13 to 17 years, with a mean age of 15.0 years (SD = 1.08). Approximately half (50.3%) of the participating students were in Year 10 at school; 10.2% were in Year 8, 17.3% in Year 9, 12.2% in Year 11 and 10.2% in Year 12. Just over 20% of students reported living on a farm or rural property, and 97% of participants reported English as their first language. The majority of participants (74.6%) reported living with both parents. Over a quarter (28.6%) of participants reported a prior history of anxiety.

Of the 225 consenting students, 127 students (82.1% female, 17.9% male) from six classes (17–25 students per class, M = 21.3 students) were allocated to the intervention condition. The mean age of students in the intervention condition was 14.89 years (SD = 1.20). In total, 98 students (78.8% female, 21.2% male) from six classes (10–21 students per class, M = 16.2 students) were allocated to the wait-list control condition. The mean age of students in the wait-list control condition was 15.16 years (SD = 0.85).

Measures

Demographics and process questions

Participants were invited to complete a range of demographic questions at pre-intervention, including their age, sex, school grade, location, first language, and living arrangements. At post-intervention and 3-month follow-up participants were also invited to provide feedback on the usability and acceptability of the e-couch Anxiety and Worry program. Usability items included how easy the website was to understand, the usefulness of the website and its provision of helpful information.

Generalised anxiety

Generalised anxiety was measured using the Spence Children's Anxiety Scale (SCAS; Spence, 1998) and the GAD-7 (Spitzer et al., 2006). The SCAS is a 44-item self-report measure composed of 38 items that assess specific anxiety symptoms relating to six subscales (generalised anxiety, social phobia, separation anxiety, panic attack/agoraphobia, obsessive compulsive disorder and physical injury fears) and six positive ‘filler’ items designed to reduce response bias. Each item is responded to on a four-point scale ranging from 0 (never) to 3 (always). The generalised anxiety sub-scale was utilised in the current study, with total scale scores on this 6-item sub-scale ranging from 0 to 18, with higher scores reflecting greater levels of anxiety. Scale internal consistency was high in the current study (Cronbach α = 0.81 [generalised anxiety sub-scale]). The GAD-7 is a brief seven-item self-report scale that measures generalised anxiety symptoms. Each item is rated on a four-point scale ranging from 0 (not at all) to 3 (nearly every day). Total scale scores can range from 0 to 21, with higher scores reflecting greater levels of generalised anxiety symptoms. The GAD-7 exhibited high internal consistency (Cronbach α = 0.84) in the current study.

Social anxiety

Social anxiety symptoms were assessed using the Social Anxiety Scale for Adolescents (SAS-A; La Greca, 1998). The SAS-A is a 22-item self-report measure composed of 18 items that assess a respondent's subjective experience of social anxiety, and four filler items. Each item is responded to on a five-point scale ranging from 1 (not at all) to 5 (all the time). Total scale scores can range from 18 to 90, with higher scores reflecting greater social anxiety symptoms. The internal consistency of the SAS-A was very high (Cronbach α = 0.95) in the current study.

Anxiety sensitivity

Anxiety sensitivity, which is associated with panic attacks and panic disorder, was measured using the Childhood Anxiety Sensitivity Index (CASI; Silverman et al., 1991). This 18-item self-report scale assesses the extent to which the respondent believes that the experience of anxiety will result in negative consequences. Items are rated on a three-point scale ranging from 1 (none) to 3 (a lot), with total scale scores ranging from 18 to 54. Higher CASI scores are associated with greater anxiety sensitivity. The CASI exhibited high internal consistency (Cronbach α = 0.89) in the current study.

Depressive symptoms

The Centre for Epidemiological Studies – Depression Scale (CES-D; Radloff, 1977) was used in the current study to measure depressive symptoms. The CES-D is a 20-item self-report scale assessing the frequency of depressive symptoms over the past 7 days. Items are rated on a four-point scale ranging from 1 (rarely or none of the time, less than one day) to 4 (most or all of the time, 5 to 7 days). Total scale scores can range from 0 to 60, with higher scores reflecting greater levels of depressive symptoms. A high level of internal consistency (Cronbach α = 0.91) was associated with the CES-D in the current study.

Wellbeing

Mental wellbeing was assessed by the Warwick-Edinburgh Mental Well-being Scale (WEMWBS; Tennant et al., 2007). The WEMWBS is a 14-item self-report measure that assesses different aspects of positive mental health. Each item is rated on a five-point scale ranging from 1 (none of the time) to 5 (all of the time), with total scale scores ranging from 14 to 70. Higher scores are indicative of a greater level of mental wellbeing.

Procedure

The trial was registered as ACTRN12610000408088. Ethical approval was obtained from the Australian National University Human Research Ethics Committee, as well as from the state and Catholic education departments responsible for the schools involved in the trial. Information and consent forms were distributed to all students and their parent/guardian in participating classes, with written informed consent required from both. All consenting students were invited to complete a pre-intervention, post-intervention and 3-month follow-up questionnaire. Questionnaires were administered by the classroom teacher (either online via a secure web-based survey or by paper and pencil) and took students approximately 20 to 30 min to complete. All consenting students were issued with a unique ID that allowed their questionnaires to be linked.

Following the pre-intervention questionnaire, students in intervention condition classes completed the e-couch Anxiety and Worry program during one class period (30 to 40 min) a week for six weeks. Classroom teachers supervised students' completion of the e-couch Anxiety and Worry program, which was delivered in a range of subject areas (e.g., pastoral care, religious education). The role of classroom teachers was to assist with program login and to respond to student questions and enquiries. No formal teaching or classroom discussion about the program was undertaken. During the intervention phase of the trial, students in wait-list control condition classes continued usual classes and were offered the intervention at the conclusion of the trial (after the 3-month follow-up questionnaire).

Intervention

The e-couch Anxiety and Worry program (www.ecouch.anu.edu.au) consists of two main sections: psychoeducation and evidence-based toolkits for anxiety consisting of cognitive behaviour therapy (CBT), relaxation and physical activity. The psychoeducation section includes a definition of worry, differentiation of worry, fear and anxiety, description of anxious thinking, risk factors for generalised anxiety, consequences of anxiety, and medical, psychological and lifestyle treatments for anxiety. The CBT toolkit focuses on the cognitive aspects of worry and how to change them. It essentially teaches participants about worry, what causes and compounds worry, how to detect and reduce worry, and how to problem solve and change thoughts to prevent and reduce worry. The relaxation toolkit contains a mindfulness meditation exercise and progressive muscular relaxation exercise, while the physical activity toolkit teaches participants about some of the benefits of being physically active and allows them to evaluate their own level of physical activity and learn some strategies for increasing or maintaining their current physical activity level.

Statistical analysis

Missingness and pre-intervention comparisons

Independent samples t-tests and chi-square analyses were used to identify any differences between participants who did and did not complete post-intervention and 3-month follow-up questionnaires. All outcome measures were analysed, as well as trial condition, age and sex. Independent samples t-tests and chi-square analyses were also used to identify any differences between the intervention and wait-list control conditions at pre-intervention. Comparisons were made between pre-intervention levels of generalised anxiety (SCAS GAD subscale and GAD-7), social anxiety, anxiety sensitivity, depressive symptoms, wellbeing, age and sex.

Universal outcome analyses

Outcome analyses were conducted using an intention-to-treat approach using mixed-model repeated measures (MMRM) analysis of variance (ANOVA), with measurement occasion as a within-groups factor and condition as a between-groups factor. School class was included as a random factor to reflect the clustered sampling of students within classes to each condition. The assessment of class effects was important, as class characteristics, such as classroom dynamics and teaching style, may have influenced the results of the study. A random effect of class was fitted to all models. The ICC coefficient was calculated by dividing the between classes variance by the sum of the between classes variance and the within-classes variance under this model (Donner and Klar, 2000). The test of the significance of the class effect is equivalent to testing whether the ICC is different to zero. ICCs of between 0.03 and 0.05 were obtained for the models in the current study. This indicates that approximately 3% to 5% of the variance in individuals' scores could be explained by between-classes effects.

The statistical techniques used yield an unbiased estimate of the outcomes and assume missing data to be missing at random or at random. Relationships between observations at different occasions were modelled as an unstructured covariance matrix. Error degrees of freedom were calculated with the Satterthwaite method. Planned contrasts were undertaken to compare differences between the intervention and wait-list control conditions in change from pre-intervention to post-intervention and 3-month follow-up. Visual inspection of the data was carried out to identify potential outliers at each time-point, as well as outlying change over time. No substantial or influential outliers were identified. The raw residuals of each model were examined with normal probability plots and were found to be within acceptable limits. Cohen's d between group effect sizes were calculated using observed mean gain scores from pre-intervention to post-intervention and from pre-intervention to 3-month follow-up. Positive effects sizes are indicative of changes in favour of the intervention condition. All analyses were conducted with the MIXED procedure in SPSS Version 22.0 for Windows.

Caseness analyses

Chi-square analyses were conducted to identify if there was a significant difference in the proportion of participants in the intervention and wait-list control conditions who met criteria for clinical caseness (GAD-7 ≥ 10) at post-intervention and 3-month follow-up.

Results

Usability and acceptability

Approximately 98% of participants completed the first two weeks of the e-couch Anxiety and Worry program, while 68% completed at least four weeks of the program and 45% completed all six weeks of the intervention. Approximately 95% of the participants reported the website as being easy to understand, while just over 81% of participants felt that the website included the information that they wanted to know about anxiety (including its causes, treatment options and prevention strategies). Over 60% of participants found the website to be useful or very useful, and at 3-month follow-up over 50% of participants reported that they would use the website again and a further 10% had already recommended the website to a friend in need. Many of the participants reported utilising the skills and strategies taught in the e-couch Anxiety and Worry program since its completion. The skills and strategies most frequently endorsed by participants included redirecting their attention to alleviate worry (46.8%), identifying specific worries (42.9%), and dealing with their feelings around worry (54.5%).

Missing data at post-intervention and follow-up

A higher percentage of participants from the wait-list control condition were missing assessments at post-intervention (38.8% vs. 27.6%) and 3-month follow-up (46.9% vs. 37.8%). However, these differences were not statistically significant (post-intervention: χ2(1) = 3.17, p = 0.08; 3-month follow-up: χ2(1) = 1.90, p = 0.17). Missing data were most often the result of a participant being absent from school on the day of questionnaire administration or having left the participating school since the last measurement occasion. No participant formally withdrew from the study. At post-intervention and 3-month follow-up, missingness (failure to complete the questionnaire) was not significantly related to age, sex, or pre-intervention levels of generalised anxiety (SCAS GAD subscale and GAD-7), social anxiety, anxiety sensitivity, depressive symptoms or mental wellbeing.

Pre-intervention comparisons

At pre-intervention there were no significant differences between the intervention and wait-list control conditions on age, sex, or the primary and secondary outcomes measures. Table 1 presents the observed means for generalised anxiety (SCAS GAD subscale and GAD-7), social anxiety, anxiety sensitivity, depressive symptoms and mental wellbeing at each measurement occasion for the intervention and wait-list control conditions. At pre-intervention, only 24 participants reported elevated levels of anxiety (GAD-7 ≥ 10).

Table 1

Observed means and standard deviations for each outcome measure at pre-intervention, post-intervention and 3-month follow up for the intervention and waitlist control conditions.

Outcome measure	Condition	Pre-intervention mean (SD)	Post-intervention mean (SD)	3-Month follow-up mean (SD)
SCAS-GAD	Intervention	6.25 (3.69)	5.42 (3.72)	5.76 (3.41)
	Control	5.99 (3.41)	5.45 (3.43)	5.18 (4.05)
GAD-7	Intervention	4.85 (4.34)	4.20 (4.25)	4.65 (4.12)
	Control	5.01 (4.59)	4.15 (3.96)	4.56 (4.96)
SAS-A	Intervention	43.86 (14.46)	40.44 (14.31)	41.51 (14.15)
	Control	44.03 (13.89)	41.95 (14.98)	37.69 (17.88)
CASI	Intervention	10.06 (6.90)	8.32 (7.11)	10.09 (8.68)
	Control	10.16 (6.63)	9.73 (6.81)	9.02 (7.21)
CES-D	Intervention	16.53 (11.25)	13.74 (10.69)	13.71 (10.36)
	Control	16.42 (10.31)	13.78 (9.89)	14.44 (11.37)
WEMWBS	Intervention	45.41 (10.90)	48.62 (11.27)	48.67 (12.85)
	Control	48.47 (10.24)	49.83 (10.70)	47.38 (16.76)

Note. SCAS-GAD = GAD subscale of the Spence Children's Anxiety Scale; SAS-A = Social Anxiety Scale for Adolescents; CASI = Childhood Anxiety Sensitivity Index; CES-D = Center for Epidemiological Studies Depression scale; WEMWBS = Warwick-Edinburgh Mental Well-being Scale.

Primary and secondary outcomes

Table 2 presents the results from the MMRM analyses for each of the outcome measures, as well as observed mean between group effect sizes. No statistically significant differences were observed between the intervention and wait-list control conditions for generalised anxiety, social anxiety, anxiety sensitivity, depressive symptoms or mental wellbeing.

Table 2

Estimates of condition × time interactions based on linear mixed models for each outcome and observed mean between group effect sizes.

Outcome measure	Condition × time interaction	Post-test effect size (d) [95% CI]	Follow-up effect size (d) [95% CI]
SCAS-GAD	F (2,145.1) = 1.26, p = 0.29	0.08 [− 0.24–0.41]	− 0.09 [− 0.44–0.27]
GAD-7	F (2,156.1) = 0.99, p = 0.37	− 0.05 [− 0.38–0.27]	− 0.06 [− 0.41–0.29]
SAS-A	F (2,130.0) = 1.23, p = 0.29	0.09 [− 0.25–0.43]	− 0.26 [− 0.62–0.11]
CASI	F (2,125.0) = 1.52, p = 0.22	0.19 [− 0.16–0.53]	− 0.15 [− 0.50–0.21]
CES-D	F (2,142.9) = 0.24, p = 0.78	0.01 [− 0.31–0.34]	0.08 [− 0.28–0.43]
WEMWBS	F (2,136.5) = 1.51, p = 0.22	0.17 [− 0.18–0.51]	0.30 [− 0.06–0.66]

Note. SCAS-GAD = GAD subscale of the Spence Children's Anxiety Scale; SAS-A = Social Anxiety Scale for Adolescents; CASI = Childhood Anxiety Sensitivity Index; CES-D = Center for Epidemiological Studies Depression scale; WEMWBS = Warwick-Edinburgh Mental Well-being Scale.

Clinical caseness

At pre-intervention, 14 (12.3%) participants in the intervention condition and 10 (12.8%) participants in the wait-list control condition had elevated levels of generalised anxiety (GAD-7 ≥ 10; χ2 (1) = 0.01, p = 0.91). At post-intervention, 11 (12%) participants in the intervention condition and 5 (8.3%) participants in the wait-list control condition had elevated levels of generalised anxiety. This difference was not significant (χ2 (1) = 0.51, p = 0.48). Similar findings were evident at the 3-month follow-up, with 12 (15.2%) participants in the intervention condition and 7 (13.5%) participants in the wait-list control condition reporting elevated levels of anxiety (χ2 (1) = 0.08, p = 0.78).

Discussion

Overall, the e-couch Anxiety and Worry program was found to be acceptable to staff and students and was feasibly delivered in the school environment. Just over two-thirds of participants completed at least four weeks of the intervention and just under half completed all six weeks of the program. Some participants were unable to complete all of the e-couch Anxiety and Worry program due to absence or other school activities (e.g., assemblies, excursions, sporting events). The level of program adherence obtained in the current study is comparable to other school-based trials of online interventions for anxiety and depression (Calear et al., 2009). The majority of participants reported that the website was easy to use, useful and included the information that they sought to know about anxiety. Over half of the participants also reported their intention to use the program again in the future. This finding is surprising, particularly given that this was a universal intervention and the majority of the students completing the program did not have elevated symptoms of anxiety and thus it might have been expected that the benefits of the intervention would not have been immediately apparent to them. Significantly, at 3-month follow-up 10% of participants reported having already recommended the program to a friend, despite the short timeframe since the completion of the program. Many of the participants also reported using the skills and strategies taught in the program after its completion. This finding again highlights the acceptability and appeal of the program to many of the students.

No significant differences were observed in generalised anxiety, social anxiety, anxiety sensitivity, depressive symptoms or mental wellbeing between the intervention and wait-list control conditions at post-intervention or 3-month follow-up. However, the effect sizes associated with a number of outcome measures, such as the effects obtained for anxiety sensitivity at post-intervention (d = 0.19) and for mental wellbeing at post-intervention (d = 0.17) and 3-month follow-up (d = 0.30), were consistent with the effects reported in other universal prevention trials (Calear et al., 2009, Neil and Christensen, 2009, Wong et al., 2014). Based on this pattern of findings, it is possible that an appropriately powered trial of the e-couch Anxiety and Worry program will demonstrate an improvement in mental wellbeing in the medium term and anxiety sensitivity in the short term. However, there was no evidence that the intervention will decrease the symptoms of mental ill health. Such a finding is not entirely surprising given that the current trial was delivered universally to all students of whom only 11% had elevated levels of anxiety at pre-intervention.

The positive effects emerging at post-intervention had diminished considerably by the time of the 3-month follow-up. This finding suggests that the therapeutic benefit of the intervention may be short-lived. One potential way to bolster the effect of the intervention long-term may be to encourage participants to re-access the intervention intermittently to review the program material. Such booster sessions would be consistent with the students' reported interest in further accessing the program. Such revision may be particularly important during stressful times, when the content of the intervention could be more relevant and potentially more impactful. Further exploring the benefits of scheduled revision of program material is warranted.

The lack of significant findings obtained in the current study reflects at least in part the low statistical power resulting from the small sample size and subsequent participant attrition from the trial. The level of participant attrition in the current trial was quite high, with over a third of participants missing one or more measurement occasions due to absence. Participant attrition from school-based trials due to assessment absenteeism is a significant problem, as schools often do not have the time or resources to follow-up participants at a later time. It is therefore important to highlight to schools the importance of research compliance and the need to obtain as many participant assessments as possible.

Some limitations of the current study include the small number of participating schools, the risk of control contamination due to the presence of intervention and control conditions within the same school, the use of self-report measures, the inability to collect complete data from all participants due to absence or school relocation, and the use of a wait-list control condition that does not allow the effects of adult attention or support to be controlled.

Conclusions

Overall, the results of the current study provide support for the feasibility and acceptability of an online anxiety prevention program in schools. The study also highlights the small, but likely robust, effects present in universal trials. The findings from the current study suggest a larger trial is warranted, with the need to recruit a more diverse sample of schools and participants. Given the adherence issues present in the current trial, a larger study might consider if intervention effects can be increased by having a youth worker deliver the intervention alongside the classroom teacher.