Evaluation of a school-based cognitive-behavioral depression prevention program.

AIM: The aim of this study was to investigate the feasibility and cost-utility of a school-based cognitive-behavioral (CB) depression prevention program.
METHODS: A quasi-experimental trial with an intervention group and a control group, with follow-up measurements obtained at three and 12 months after baseline, was conducted. The setting was six Swedish municipalities. The participants were students in grade 8 (median age: 14). A total of 462 students (79% girls) were allocated to the school-based CB prevention program, and 486 students (46% girls) were allocated to the control group. The school-based CB prevention program, Depression in Swedish Adolescents (DISA), was presented by school health service staff and teachers once per week for 10 weeks.
RESULTS: The main outcome measures were self-reported depressive symptoms and self-rated health; the secondary outcome measures were adherence and cost-utility. The intervention group decreased their self-reported depressive symptoms (as measured by the Center for Epidemiological Studies Depression Scale) and improved their self-rated health (as measured by the visual analog scale) at the 12-month follow-up more than the control group ( p < .05).
CONCLUSIONS: Given the challenges of conducting a study in a complex, everyday school setting with baseline differences between the intervention and control group, it is difficult to make accurate interpretations of the effectiveness of the intervention. However, with these limitations in mind, the results indicate that the DISA program is a feasible school-based prevention program.

RCT Entities:   Population Interventions Outcomes

Introduction

Supporting emotional resilience in adolescents is a crucial public health issue. Approximately 11% of adolescents report high levels of depressive symptoms [1]. Depressive disorders negatively affect academic achievement, peer and family relationships, and everyday functioning, and they often recur in adulthood [2]. Schools provide an ideal setting for mental health promotion, but in a recent systematic review of 39 universal depression preventing programs the effect sizes (g) obtained were small (g = 0.19, confidence interval (CI) 95%, 0.14–0.24) [3].

Background

Universal prevention programs targeting depressive symptoms have been predominantly based on cognitive–behavioral (CB) therapy [4]. The current study examined the universal school-based CB prevention program, Depression in Swedish Adolescents (DISA) [5], which is an abridged and modified version of the Adolescent Coping with Stress course [6]. The program is based on a multifactorial model [7] in which depression is assumed to result from multiple etiological elements, such as negative cognitions, stressful events, predisposing vulnerabilities, risk factors (i.e. being female, having a previous history of depression, or having parents with depression), and immunities to depression (i.e. high self-esteem, coping skills, or a high frequency of pleasant events and activities). The DISA program is frequently used in schools in Sweden and Finland for adolescents of both sexes, although it was primarily designated for girls [5]. The Coping with Stress course is an indicated program specifically designed for adolescents showing signs of depressive symptoms, whereas DISA is a universal program delivered in the classroom. The Coping with Stress course has demonstrated efficacy for adolescents with sub-threshold values for depression [8,9], and Lynch et al. [10] found it to be cost-effective as an indicated program. Positive effects among girls participating in the DISA program have been observed in two implementation studies [5,11]; however, with small effect sizes (g = 0.20 and 0.19, respectively). To the best of our knowledge, no large-scale feasibility studies have previously investigated DISA.

Aims

The primary aim was to investigate whether the school-based CB prevention program, DISA, could influence depressive symptoms and self-rated health among adolescents at the one-year follow-up. A secondary aim was to describe adherence to the program and its cost-utility.

Methods

The date range for participant recruitment and follow-up in this multisite evaluation was September 2012–November 2014. The study had a quasi-experimental design with intervention and control groups, and with testing at baseline and at three- and 12-month follow-ups [12].

Context

The Swedish educational system is highly decentralized. Therefore, despite legal regulations of the school curriculum and the availability of school health staff (i.e. school nurses, school physicians, school social workers, and special education teachers), each school principal has the opportunity to implement, for example, universal prevention programs at the school. Despite the large number of different school-based prevention programs in use, few of them have been sufficiently evaluated [13].

Study population

A power calculation was conducted a priori using the mean change in the Center for Epidemiological Studies Depression Scale (CES-D) as the primary outcome. For 80% power, a mean difference of 2.0 was regarded as clinically relevant [14]. The sample size needed for each sample separately was 99 participants (standard deviation (SD) = 5.0, α = .05).

The study population consisted of students in grade 8 (students aged 13–15 years, median 14 years) in six municipalities in southern Sweden, representing rural and urban areas. Of the 23 schools with grade 8 students in the participating municipalities, 14 schools offered the CB prevention program, DISA, within the regular school context. Schools without this mental health program in their curricula were recruited as control schools (see Figure 1). Gender inequity in the intervention and control groups arose because DISA was offered to more girls than boys. The questionnaires were completed by 367 girls and 95 boys in the intervention group and by 224 girls and 262 boys in the control group at baseline. The dropout rate for completing the questionnaire from baseline to the three-month follow-up was 24%, and the dropout rate from baseline to the 12-month follow-up was 20%. The reasons for dropout included weak motivation to complete the questionnaires and student absences on the day of assessment. The drop-out analysis (chi-square test) showed no significant differences regarding sex (p = .713) and age (p = .965).

Figure 1.

Consort flow diagram.

The intervention

The CB prevention program, DISA, was offered as a course in lieu of the ordinary school curriculum at the intervention schools. The program was conducted once per week for 1½ hours over a period of 10 weeks. The program was based on a manual, with a fixed curriculum for every session based on CB techniques designed to change negative thoughts, communication training and training in problem-solving strategies, and exercises to strengthen social skills and networks and to increase participation in health promotion activities [5]. The tutors were asked to complete a form pertaining to program fidelity and the time needed to conduct and prepare for each session. A total of 31 tutors, two of whom were men, completed the tutor questionnaire. All tutors had completed a three-day training course to be DISA tutors, and they consisted of school social workers (n = 12), school nurses (n = 9), teachers (n = 9), study counselors (n = 2), and school assistants (n = 3). The tutors reported that they required approximately 19.3 hours to prepare for and conduct the intervention. The mean group size was 12.5 students. The tutors reported that they followed the manual for 92% of the course in terms of the number of completed exercises.

Ethical considerations

The Regional Ethical Review Board in Lund approved the study (2012/462) before participant recruitment began. Students and their parents or guardians were provided with written information about the study, including that it was voluntary.

Outcome measurements

The CES-D is a self-reported measure of the frequency of 20 depressive symptoms occurring during the previous week [14]. The scale’s psychometric properties are good when used with adolescents [15], and the Swedish version has been used in research before [5]. The internal consistency of the CES-D in this sample was α = .91. Each item was scored for its frequency of occurrence in the previous week, with scores ranging from 0 (rarely or never) to 3 (most or all of the time), producing a total possible score range of 0–60. Higher scores indicated more depressive symptoms, with a threshold value to be at risk for depression of 20 points [14].

The Euro Qol (EQ) visual analog scale (VAS; included in EQ-5D) is commonly used in cost-effectiveness studies [16] and records the student’s self-rated health on a vertical visual analog scale with endpoints labeled “Best imaginable health state” and “Worst imaginable health state.” The scale ranges from 0 to 100, and it measures the self-rated health on a particular day. Higher scores indicate better self-rated health [17].

The questionnaire also included questions regarding the students’ year of birth, gender, country of birth for themselves and their parents, family situation (parents living together or separated), and perceived economic situation (“How financially well-off is your family?” with six possible answers: very well off, rather well off, average, not very well off, not well off at all, or do not know). The students in the intervention group also answered questions regarding attendance, whether they appreciated the course (three-month follow-up), and whether they had used the knowledge obtained from the course (12-month follow-up).

The questionnaires were completed in the classroom with paper and pencil, and the students were instructed not to look at one another’s answers. The tutors and/or teachers were available to offer support if problems arose in understanding the questionnaires. The students in the intervention group were asked to complete the questionnaires during the first and last sessions, and again after 12 months. The students in the control group were asked to complete the questionnaires at the same time points.

Statistical analysis

The analysis was conducted according to the intention to treat (ITT) principle [18]. Frequency analysis and chi-square analysis were performed to describe the demographic and background characteristics of the students. Differences in the mean changes on the CES-D and EQ VAS between the intervention and control groups from baseline to the three- and 12-month follow-ups were analyzed using an independent-samples t-test. A repeated measures ANOVA was also conducted [18].

To investigate the possibility of clustering, we performed multilevel linear and logistic regression analysis [19]. The linear regression used quantitative changes in the CES-D and EQ VAS. The logistic regression used improvement or unchanged status compared to deterioration. On the first level, we had students, and on the second level, we had schools. Analyses were performed with MLwiN (MLwiN, version 2.17, Centre for Multilevel Modelling, University of Bristol, UK).

A large number of students chose to answer the questionnaire anonymously. Therefore, responses could be paired from baseline to the 12-month follow-up in only 48% of cases. Multiple imputation was used to manage the missing data in the independent-samples t-test [20]. A complete case analysis (i.e. analysis without imputed data) was also performed. The significance level for all tests was set to .05. The statistical analyses were performed using IBM SPSS, version 21.0.

A cost-utility analysis was performed comparing the cost of program implementation to health gains measured by quality-adjusted life years (QALYs). Program costs included the intervention costs – that is, the cost for the DISA program – including the time used to train the tutors and to prepare for and conduct the 10-week program. We applied a difference-in-difference approach for the QALY gained, which accounts for possible baseline differences between the two groups and measures the differences in trends between the two groups over time [21,22]. QALYs gained were calculated as the difference in the mean change in EQ VAS from baseline to the three- and 12-month follow-up between the intervention and control group as the area under the curve. The incremental cost-effectiveness ratio (ICER) was the ratio of the difference in costs for the intervention group compared with those for the control group (incremental costs) to the QALYs gained. The time period was 12 months and, therefore, no discounts were used [16]. In the sensitivity analysis, cost-effectiveness was also calculated with 50% higher costs and with 50% reduced effect on QALYs.

Results

The baseline characteristics of the participants from the intervention and control groups did not differ in their birth year, family situation, perceived economic situation, school performance, or country of birth (see Table I). However, because the intervention was primarily offered to girls, the intervention group contained more girls. The students in the control group had fewer depressive symptoms (i.e. they scored lower on the CES-D) and had better self-rated health (i.e. they scored higher on the EQ VAS compared with the intervention group) at baseline.

Table I.

Baseline characteristics.

	Intervention (n = 462)	Control (n = 486)	df	p-value[a]
Sex			1	<.001
Girls	79%	46%
Age			2	.182
15 years	4%	3%
14 years	72%	72%
13 years	24%	25%
CES-D
Girls (SD)	14.8 (10.0)	13.1 (9.4)	48	.007
Boys (SD)	11.0 (9.5)	8.0 (7.3)	36	.001
VAS
Girls (SD)	67.6 (20.5)	73.2 (18.6)	42	.037
Boys (SD)	73.5 (21.5)	81.4 (18.3)	39	<.001
Country of birth			3	.144
Sweden	94%	91%
Nordic countries	1%	1%
Europe	1%	3%
Non-European	4%	5%
Country of birth, parents			3	.183
Sweden	83%	78%
Nordic countries	3%	3%
Europe	5%	10%
Non-European	9%	9%
Family situation			1	.172
Parents living together	72%	69%
Perceived economic situation			2	.807
Very good or quite good	76%	80%
Average	20%	18%
Not very good or not good at all	4%	2%

df: degree of freedom, SD: standard deviation; CES-D: Center for Epidemiological Studies Depression Scale; VAS: visual analog scale.

Chi-square test.

Adherence to the intervention

A majority (92%) of the students in the intervention group reported that they had attended at least eight of the 10 program sessions. The girls were more satisfied with the intervention than the boys were, and 74% of the girls found it beneficial compared with 55% of the boys. Approximately half of the girls (53%) reported that they had used the knowledge gained from the course in the subsequent year, compared with 40% of the boys.

Depressive symptoms and self-rated health

The complete case analysis (i.e. analysis without imputed data) revealed a significant improvement in the intervention group compared with the control group at the 12-month follow-up point in terms of self-reported depressive symptoms as measured by the CES-D and self-rated health as measured by the EQ VAS. There was a significant difference in the CES-D scores for the intervention group (mean (M) = −0.73, SD = 9.92) and the control group (M = 1.74, SD = 10.0); t(443) = −2.75, p = .006 at the 12-month follow-up. There was also a significant difference in the EQ VAS scores for the intervention group (M = 1.92, SD = 22.9) and the control group (M = −2.70, SD = 17.9); t(428) = 2.33, p = .020 at the 12-month follow-up. However, when the girls and boys were analyzed separately, the improvement remained significant only for the girls (p = .028 for the CES-D and .013 for the EQ VAS). The boys also showed improvement, but the improvement was not significant (p = .146 for the CES-D and p = .298 for the EQ VAS). However, when the imputed data (treated according to ITT) were used, the improvement for boys was also significant (see Table II). The difference in the mean change from baseline to the 12-month follow-up between the intervention and control groups (boys and girls) was 2.80 (CI 1.55–4.04) for the CES-D and 4.45 (CI 1.84–7.05) for the EQ VAS. No clustering was found at the school level in the multilevel analysis. The multilevel analysis confirmed the absence of a difference between tutors belonging to the school health team; that is, school nurses, school social workers, and teachers.

Table II.

Depressive symptoms and self-rated health over time in the intervention and control groups.

	Group	BL mean (SD)	3-month follow-upMean (SD)	p-value[a] BL – 3-month follow-up	p-value[b] (effect size)BL – 3-month follow-up	12-month follow-upMean (SD)	p-value[a] BL – 12-month follow-up	p-value[b] (effect size)BL – 12-month follow up
CES-D	Girls				.09(.11)			<.001 (.24)
	Intervention	14.8 (10.01)	12.8 (9.45)	<.001		14.2 (10.76)	.095
	Control	13.1 (9.40)	12.3 (9.66)	1.0		14.8 (10.91)	.019
	Boys				<.001 (.44)			<.001 (.29)
	Intervention	11.0 (9.47)	8.1 (7.97)	<.001		10.0 (9.61)	.226
	Control	8.0 (7.28)	9.3 (8.95)	.185		9.7 (9.54)	.005
	Total				<.001 (.27)			<.001 (.22)
	Intervention	14.0 (10.00)	11.8 (9.36)	<.001		13.2 (10.63)	.095
	Control	10.3 (8.67)	10.8 (9.41)	1.00		11.7 (10.39)	<.001
EQ VAS	Girls				<.001 (.25)			<.001 (.11)
	Intervention	67.6 (20.48)	71.2 (18.54)	.006		67.3 (21.51)	1.00
	Control	72.7 (18.57)	71.2 (18.19)	1.00		70.2 (19.55)	.065
	Boys				<.001 (.32)			<.001 (.31)
	Intervention	73.5 (21.44)	78.9 (14.32)	.004		77.1 (19.92)	.052
	Control	81.4 (18.92)	80.0 (18.96)	.210		78.4 (19.36)	.029
	Total				<.001 (.29)			<.001 (.15)
	Intervention	68.5 (20.70)	72.3 (18.17)	<.001		69.2 (21.54)	.345
	Control	77.7 (19.46)	75.5 (19.05)	.153		75.2 (19.82)	.002

Repeated measures ANOVA of differences in mean changes at different time points within groups based on data with multiple imputation.

Independent-samples t-test of differences in mean changes between intervention and control groups based on data with multiple imputation.

BL: baseline; CES-D: Center for Epidemiologic Studies Depression Scale (higher scores on the CES-D scale indicate more depressive symptoms); EQ VAS: visual analog scale (higher scores indicate better self-rated health).

Mean change = difference in the mean change from baseline to 3- and 12-month follow-ups.

A repeated measures ANOVA was conducted to investigate if there were any changes in the students’ depressive scores (as measured with the CES-D) when measured at baseline, and after three and 12 months. The results of the ANOVA showed significant time effect: Wilks’ Lambda = 0.95, F(2, 928) = 22.97, p < .001. Follow-up comparisons showed significant improvements (decrease of depressive symptoms) in the DISA group from baseline to the three-month follow-up (p < .001), and significant increase of depressive symptoms in the control group from baseline to the 12-month follow-up (p < .05).

The EQ VAS scores (divided by 100 for transformation to a 0–1 scale) were used as QALY weights. Based on these data, the QALYs gained at 12 months was 0.04 (i.e. area under the curve from baseline to the follow-ups at three and 12 months). The incremental cost per QALY gained was approximately US$6300 (see Table III).

Table III.

Program costs and cost-utility analysis.

• The total cost of the course in terms of tutor time per student was 6 h (3 h tutor training time and 3 h instruction time).

• The salary cost for the tutors was about US$34/h (including payroll taxes).

• The tutor training fee was approximately US$33 per student.

• The course material can be obtained for free, but the costs of making paper copies and for fruit were approximately US$13.

• In total, the cost was US$250 per student.

• Other costs were assumed to be equal between the groups.

• Year 2014 costs were converted from Swedish krona (SEK) to US$, using the exchange rate of 1 SEK = 0.15 US$ (www.riksbank.se).

• EQ VAS scores (divided by 100 for transformation to a 0–1 scale) were used as QALY weights.

• Cost per QALY gained was, therefore, US$6250 in the base case analysis (i.e. the cost divided with QALYs gained, 250/0.04).

• In the sensitivity analysis with 50% higher costs, the cost per QALY gained was US$9375 (375/0.04), and with 50% lower effect, the cost per QALY gained was US$12,500 (250/0.02).

EQ VAS: visual analog scale; QALY: quality-adjusted life year.

Discussion and conclusions

The students belonging to the intervention group slightly decreased their self-reported depressive symptoms and improved their self-rated health, whereas the outcomes for the students in the control group on average were worse at the one-year follow-up, with small effect sizes. The small effect sizes are consistent with previous studies [3,5,11]. Considering the baseline differences between the intervention and the control group, conclusions have to be drawn with caution. Analyses were performed both for differences between the intervention and control group with t-test and over time with repeated measures ANOVA. The incremental cost per QALY gained of approximately US$6300 was well below the “willingness to pay” values that are often referred to as “thresholds” in the US; that is, US$50,000–100,000 [23]. In Sweden, a low cost per QALY is considered to be below US$15,000, a medium cost is approximately US$75,000, and a high cost is greater than US$150,000 [24]. The fact that the intervention cost per person is low implies that also relatively modest utility gains will be considered cost-effective. Even with the sensitivity analysis aiming to investigate the cost-effectiveness ratio with 50% higher costs and 50% lower effect, the ICER remained well below these thresholds. Cost-effectiveness studies for universal prevention programs have rarely been conducted [3]. In our study, no significant differences were found if the tutors were teachers or school health staff (i.e. school social workers and/or school nurses).

The results showed high levels of adherence and student satisfaction. The intervention had been conducted at the schools for two years on average at the beginning of this study, and it was still being conducted at the 12-month follow-up. This finding is consistent with other feasibility studies in school settings, indicating that programs conducted in schools tend to have good reach [25] and are perceived as beneficial and meaningful [26]. However, although the DISA program is a universal prevention program, its theoretical underpinning in the Adolescent Coping with Stress course is treatment [6,27]. Tutors and students tend to change focus from the pathogenic focus in the manual to a salutogenic emphasis [26,28,29].

Limitations

Evidence from quasi-experimental studies is generally believed to be weaker than evidence from randomized trials. A significant limitation of this trial is that the intervention condition is made up of schools who were already using the program and, thus, already engaging with social and emotional learning. However, a naturalistic design may have ethical advantages, provide information about how a program is actually used in a busy, real-life setting, and be possibly less prone to overestimating benefits and, thus, more representative of expected benefits of a broader implementation [30]. Although the CES-D queries the previous week, and the EQ VAS queries a particular day, these instruments were chosen because the CES-D was the main outcome in the previous evaluations of the DISA and the Adolescent Coping with Stress program [5,8-10], and the EQ VAS is a well-established instrument in cost-effectiveness studies [16]. The dropout analysis showed that the two schools that declined to participate did not differ from the included schools in student characteristics. A study limitation is that a large number of students chose to answer the questionnaire anonymously. Therefore, multiple imputation was used as a complement to the complete case analysis to pair responses from baseline to the 12-month follow-up. The intervention group contained more girls than boys. This gender difference arose because the intervention is largely offered to girls in Sweden. It is the principal at each school who decides what extra-curricular subjects to offer and to whom. This gender imbalance has to be considered, and, therefore, the analyses were also performed with girls and boys separately. The results have to be interpreted with caution because of this baseline difference. Optimally, the baseline scores in the intervention and control group should be as similar as possible. The baseline scores for self-reported depressive symptoms and self-rated health were worse in the intervention group than in the control group, and poorer scores generally tend to improve more than better scores do. The intervention effect could, therefore, be overestimated. However, the difference-in-difference approach for measuring QALY gains considers the possible baseline differences because every individual is his/her own control. Our measurements focus on the average change for individuals in the groups.

The tutors completed a form pertaining program fidelity; however, it is a study limitation that no session was recorded. Other factors that would be beneficial to investigate are the influence of larger/smaller group sizes, seasonality, the focus on females only, greater/fewer number of times that tutors provide the intervention without new training, or the use of electronics instead of paper, were unfortunately not possible to investigate with the collected data.

Implications for research and practice

The school-based CB prevention program, DISA, appears to be feasible; however, the results have to be considered cautiously because of the baseline differences of the included students. Further investigations with a target group and implementation in schools, as well as exploration of the active elements in the prevention program, are recommended.