Differential intervention effectiveness of a universal school-based resilience intervention in reducing adolescent substance use within student subgroups: exploratory assessment within a cluster-randomised controlled trial.

OBJECTIVES: Interventions addressing the individual and environmental protective factors of adolescents are suggested to have potential for reducing adolescent substance use. While universally delivered school-based substance use prevention interventions are common, previous studies have suggested variable effectiveness by subgroups of students. An exploratory study was undertaken to examine the differential effectiveness of a universal school-based resilience intervention on adolescent substance use and protective factors according to their sociodemographic and previous substance use.
DESIGN: Secondary analysis of data from a cluster-randomised controlled trial.
SETTING: 32 Australian secondary schools. PARTICIPANTS: Cohort of grade 7 students (n=3155) followed up in grade 10 (aged 15-16 years; 2014; n=2105). INTERVENTION: Three-year universal school-based intervention implemented by school staff that targeted a range of student resilience protective factors (2012-2014). MEASUREMENTS: Primary outcomes included: tobacco (recent, number of cigarettes) and alcohol (recent, 'risk' and number of drinks) use, and secondary outcomes included: marijuana (recent) and other illicit substance (recent) use, and aggregate individual and environmental protective factor scores. Generalised and linear mixed models examined interactions between treatment and student subgroups (gender; socioeconomic disadvantage (low/high); geographic location (major city/inner regional/outer regional-remote); and previous substance use (non-user/user)) at follow-up (36 models).
RESULTS: Analysis of student follow-up data showed no differential intervention effect for any substance use or protective factor outcome for any subgroup, with the exception of one differential effect found by socioeconomic status for the outcome of mean number of cigarettes smoked by recent smokers (p=0.003). There was no evidence of an intervention effect within the low (mean difference (MD) -12.89, 95% CI -26.00 to 0.23) or high (MD 16.36, 95% CI -1.03 to 33.76) socioeconomic subgroups.
CONCLUSIONS: No evidence of an intervention effect on substance use and protective factors was found according to student subgroups defined by sociodemographic characteristics or previous substance use. TRIAL REGISTRATION NUMBER: ACTRN12611000606987. © Author(s) (or their employer(s)) 2018. Re-use permitted under CC BY-NC. No commercial re-use. See rights and permissions. Published by BMJ.

RCT Entities:   Population Interventions Outcomes

The cluster-randomised controlled study design and the real-world pragmatic intervention delivered by existing school staff was a strength of the study.

A further strength of the study was the use of best practice statistical methods for subgroup analyses.

While typical for school-based research, less than 50% of students completed both the baseline and follow-up surveys, which is a limitation of the study.

The study was limited by its exploratory nature as it was not designed to be powered to detect differences in subgroup analyses.

Background

Initiation of tobacco, alcohol and illicit substance use in high-income countries generally occurs during adolescence,1–3 with earlier use associated with greater dependence in adulthood and a range of negative health outcomes.4 Despite declining trends in adolescent use of tobacco, alcohol and illicit substances internationally,3 5 a considerable proportion of adolescents from high-income countries continue to report such use: 23%6 and 45%5 having smoked a cigarette, 43%6 and 71%5 having consumed an alcoholic drink and 15%7 and 40%6 having used an illicit substance.3 5 6 Similar levels of adolescent substance use are reported in Australia, with 19%,3 68%7 and 16%7 of adolescents respectively reporting having ever smoked a cigarette, had an alcoholic drink or used an illicit substance. Additionally within Australia, the prevalence of tobacco, alcohol and illicit substance use is reported to differ by gender,7 socioeconomic disadvantage8 and geographic location.8

Evidence from cross-sectional and longitudinal studies suggests that a range of individual and environmental factors are associated with a decreased likelihood, and considered protective factors, of adolescent tobacco, alcohol and illicit substance use.9–22 Individual factors found to be associated with a decreased likelihood of adolescent substance use include self-efficacy, problem solving and self-awareness, whereas environmental factors include caring relationships with adults and peers, and meaningful participation in home, school and community settings.9–22 These protective factors are also sometimes described as contributing to ‘resilience’.23–25 Most broadly, resilience has been defined as a ‘process, capacity or outcome of successfully adapting to challenging or threatening circumstances’.26 In disadvantaged populations in particular, ‘resilience’ or the presence of these protective factors has been found to characterise students with good health and life outcomes despite greater risk status.26 This is consistent with meta-analysis results from a recent systematic review that reported universal school-based interventions that address individual and environmental resilience protective factors to be effective in reducing illicit substance use by adolescents.27 Only one controlled trial has examined the effectiveness of an intervention approach focused solely on resilience protective factors in reducing adolescent substance use.28 While the study reported significant results for tobacco midintervention, no effect on tobacco, alcohol or marijuana use outcomes were evident at follow-up.29 In contrast, a non-controlled pre-post study of a 3-year universal intervention focused solely on individual and environmental protective factors reported significant reductions in the use of tobacco (50% to 27%), alcohol (34% to 17%) and marijuana (16% to 7%) use among cross-sections of students.30 Such evidence suggests the potential of school-based resilience interventions that address these individual and environmental protective factors as a means to reducing adolescent substance use.9 11–22

School-based substance use prevention interventions that are delivered to all students in a school or classroom regardless of risk; that is, universal interventions,31 32 are recommended and commonly implemented by governments worldwide.33–36 It has however been suggested that not all students within a population may benefit equally from universally implemented substance use prevention interventions, with certain subgroups of students either benefiting more or less than others.37 As a result, investigation of the generalisability of universal substance use intervention effects across subgroups of students has been recommended.38 This recommendation is consistent with the standards of evidence for effective programmes and policies developed by the Society for Prevention Research.39 For interventions found to be effective overall, investigation of the generalisability of effect across subgroups provides guidance for how to enhance intervention effectiveness for all student subgroups. For interventions that have a null effect overall, such investigation can provide guidance to hypothesise whether an intervention may be effective for particular subgroups and identify opportunities for future studies to test such hypotheses.

The only universally implemented school-based intervention focused solely on targeting resilience protective factors has not investigated any variable patterns of effect by participant subgroups. However, of those studies that have implemented school-based interventions that address resilience protective factors among other factors as part of a broader intervention approach, a variable pattern of effect by participant subgroup has been reported. Such variability has been reported to occur between students defined by sociodemographic and previous substance use characteristics.40–43 For example, studies have reported differential intervention effects on tobacco use by gender, such as reductions in tobacco use for either females40 or males,42 43 whereas other studies have reported differential effects by socioeconomic level, such as reductions in alcohol use for students of low socioeconomic schools but no effect in schools of medium or high socioeconomic level.41 No universally delivered school-based studies addressing protective factors could be found that examined differential intervention effects by subgroups of students defined by geographic location. Previous studies have also examined differential effectiveness of school-based substance use interventions in terms of students classified by risk of substance use, most often defined as substance use initiation prior to intervention. Such studies report mixed results,41 suggesting such interventions are more effective for existing substance users than non-users, more effective for existing non-users than users or no differential effect according to previous substance use.41 44–46

A cluster-randomised control study was undertaken to investigate the overall effectiveness of a universally delivered school-based resilience intervention in reducing substance use by adolescents (20 intervention; 12 control schools).47 As previously reported, the study found no effect on the primary (tobacco and alcohol use; mental health problems) or secondary (illicit substance use, individual and environmental resilience protective factors) student outcomes. The study also investigated the effect of the intervention on the implementation of resilience intervention strategies in both intervention and control schools, and of the 16 outcomes related to the implementation of resilience strategies, intervention schools were more likely than control schools to implement 9 hours of resilience curriculum.48 49 In order to investigate whether any student subgroups benefited from the intervention, a study was conducted to examine the differential effectiveness of the universally delivered school-based ‘resilience’ protective factor intervention on substance use by adolescents according to their sociodemographic and previous substance use characteristics. A secondary aim was to examine the differential effectiveness of the intervention on the hypothesised mechanism of effect, student resilience protective factors.

Methods

Study design and setting

The cluster-randomised controlled trial was conducted in secondary schools in one health district of New South Wales, Australia. Outcome assessments were conducted with a cohort of students at baseline in 2011 (when students were in grade 7: aged 12–13 years) and at follow-up in 2014 (when students were in grade 10).

Approximately 114 000 people aged 10–19 years reside in metropolitan, regional, rural and remote areas within the district.50 51 Further study details have been reported elsewhere.47

Participants and recruitment

Schools

A national schools database52 identified 172 schools with secondary enrolments within the study area. Schools were eligible if they: were a government or Catholic secondary school located within a socioeconomically disadvantaged local government area (defined by the Socio-Economic Indexes or Areas (SEIFA) Index of Relative Socio-economic Advantage/Disadvantage),53 had enrolments in grades 7–10 (aged 12–16 years) and had more than 400 total student enrolments. Schools were ineligible if they were: single gender, independent (private), special needs, selective, central (for students aged 5–18 years) or boarding schools.

Randomisation of schools

Eligible schools were randomly ordered using a random number function in Excel and approached in that order until a quota of 32 schools consented. The 32 consenting schools were then stratified according to participation in a government disadvantaged schools initiative (yes/no)54 and school size (medium 400–800; large >800). Schools were then randomly allocated to intervention or control in a 20:12 block design ratio (based on stakeholder request to increase the reach of anticipated intervention benefit) by an independent statistician using a random number function in Microsoft Excel prior to baseline data collection.

Students

All students enrolled in grade 7 (first year at secondary school) were eligible to participate (n=4589), and active parental consent for student participation in data collection was sought via a mailed study information pack. After 2 weeks, non-responding parents were prompted via telephone by school-affiliated staff who were blind to group allocation. A toll-free number was provided for parents who wished to decline the telephone prompt.

Intervention

A 3-year universal (‘whole of school’) intervention was delivered by school staff to all students in the cohort during grades 8–10. The intervention, based on a pilot study,30 involved 16 broad strategy areas (see box 1) seeking to build the protective factors of students implemented across all three domains of the Health Promoting Schools framework55 (box 1). Each broad intervention strategy addressed at least one individual (self-efficacy, problem solving, cooperation/communication, self-awareness, empathy and goals/aspirations) or environmental protective factor (school support, school meaningful participation, community support, community meaningful participation, home support, home meaningful participation, peer-caring relationships and prosocial peers). Such protective factors align with a ‘resilience’ approach.23–25 56 Schools were provided with details of existing available resources and programmes targeting the protective factors identified by researchers. While schools were required to implement all strategies, they were given the flexibility to select the order in which they were implemented and which resources or programmes they used when doing so.

Intervention strategies by Health Promoting Schools domain

Curriculum, teaching and learning

Age-appropriate lessons (9 hours) on individual protective factors across school subjects (eg, MindMatters78 or school-developed curriculum resources).I

Non-curriculum programmes (9 hours) targeting protective factors (eg, the Resourceful Adolescent Program).79 I, E

Additional programme targeting protective factors for Aboriginal students.*I, E

Ethos and environment

Rewards and recognition programme.I, E

Peer support/peer mentoring programmes.I, E

Antibullying programmes.I, E

Empowerment/leadership programmes.I

Additional empowerment/leadership/mentoring programmes for Aboriginal students.*I, E

Aboriginal cultural awareness strategies.I, E

Partnerships and services

Promotion/engagement of local community organisations/groups/clubs in school (eg, charity organisations).#

Additional/enhanced consultation activities with Aboriginal community groups.*I, E

Promotion/engagement of health, community and youth services in the school.I, E

Additional/enhanced Aboriginal community organisations promoted or engaged.*I, E

Referral pathways to health, community and youth services developed and promoted.I, E

Strategies to increase parental involvement in school (eg, school events).E

Information regarding student protective factors provided to parents via school newsletter.E

Engagement with school community including presentations at school staff meetings regarding planned intervention.†

Embedded staff support:

School intervention officer 1 day a week to support programme implementation.

Project coordinator to liaise with school sectors and support school intervention officers.‡

School intervention team formed (new team or realignment of existing team, inclusive of school intervention officer and school executive member) to implement intervention.

Structured planning process to prioritise and select appropriate resources/programmes:

Needs assessment of student protective factors (when study sample in grade 7).

Two school community planning workshops and one strategy review workshop.‡

School plan to address intervention strategies endorsed by the school executive.

Intervention implementation guide that described the intervention, planning process, available resources and programmes, tools and templates for intervention implementation.

Staff mental health training (minimum of 1 hour per school during staff meetings)

$A2000 per year each for:

Teacher release time for intervention implementation or professional development.

Strategies specifically for Aboriginal students.*

Feedback reports regarding student substance use and protective factors, and intervention implementation (termly).‡

An Aboriginal Cultural Steering group with representation from local Aboriginal community organisations was formed to provide Aboriginal cultural advice and direction regarding the study design, implementation, evaluation and dissemination.

Implementation support strategies

Following publication of the study protocol47 and based on advice received from an Aboriginal Cultural Steering Group intervention strategies 3, 8, 11 and13 were added.

*Implemented in years 2 and 3 only; †year 1 only; ‡years 1 and 2 only.

ITo target individual protective factors; Eto target environmental protective factors.

To ensure implementation of intervention strategies, schools were provided with a comprehensive range of support strategies, including an embedded implementation support officer, and other strategies that have been previously reported to facilitate implementation of interventions (box 1).57–64

Control schools implemented usual school curricula that may have included protective factor strategies and resources similar to or the same as those systematically provided to the intervention schools and were not provided with programme resources or implementation support. A report describing school-level student substance use and protective factor characteristics at baseline was provided to control schools.

Data collection procedures

Student demographic and protective factor characteristics and substance use

Students completed a confidential web-based survey in class time prior to intervention commencement (baseline: August–November 2011) and immediately following intervention completion (follow-up: July–November 2014). Neither the school staff nor researchers were blind to group allocation.

Measures

Student demographic characteristics

The student survey addressed: age, gender, residential postcode, Aboriginal and/or Torres Strait Islander status, ethnicity and non-English speaking background.

Student substance use characteristics

Substance use data were collected using items from an ongoing Australian triennial survey of school students’ health behaviours.3 Outcomes included recent tobacco, recent alcohol, ‘risk’ alcohol, recent marijuana and other illicit substance use as well as the number of cigarettes and alcoholic drinks consumed in the last week (table 1).

Table 1

Substance use and protective factor outcome measures at follow-up

	Survey item	Response options
Primary outcomes
Tobacco use: recent	Have you smoked a cigarette in the last week?	Yes/no
Number cigarettes: last week*	If yes, starting from yesterday, please record the number of cigarettes that you smoked on each day of last week.3	0–99
Alcohol use: recent	Have you had any alcoholic drinks, such as beer, wine or alcopops/premix drinks in the last week? (do not count sips or tastes)	Yes/no
Number alcoholic drinks: last week*	If yes, starting from yesterday, please record the number of alcoholic drinks that you had on each day of last week.3	0–99
Alcohol use: ‘risk’	In the last 4 weeks, how many times have you had five or more alcoholic drinks in a row?3	None/once/twice/3–6 times/7 or more times
Secondary outcomes
Marijuana use	How many times in the last 4 weeks have you smoked or used marijuana/cannabis (grass, hash, dope, weed, mull, yarndi, ganga, pot, a bong and a joint)?3	None/once or twice/3–5 times/6–9 times/10–19 times/20–39 times/40 or more times
Other illicit substance use	How many times in the last 4 weeks have you used any other illegal drug or pill to get ‘high’, such as inhalants, hallucinogens (eg, LSD, acid, trips), amphetamines (eg, speed, ice), ecstasy, cocaine or heroin?	None/once or twice/3–5 times/6–9 times/10–19 times/20–39 times/40 or more times
Individual protective factors65	Cooperation and communication subscale: two items; for example, ‘I enjoy working together with other students my age’.	1: Never true, 2: true some of the time; 3: true most of the time; 4: true all of the time
	Self-efficacy subscale: four items; for example, ‘I can do most things if I try’.	As above
	Empathy subscale: three items; for example, ‘I try to understand what other people feel and think’.	As above
	Problem solving subscale: three items; for example, ‘When I need help I find someone to talk with’.	As above
	Self-awareness subscale: three items; for example, ‘I understand why I do what I do’.	As above
	Goals and aspirations subscale: three items; for example, ‘I have goals and plans for the future’.	As above
Environmental protective factors65	School support subscale: six items; for example, ‘At my school there is an adult who really cares about me’.	As above
	School meaningful participation subscale: three items; for example, ‘At my school, I help decide things like class activities or rules’.	As above
	Peer caring relationships subscale: three items; for example, ‘I have a friend who helps me when I’m having a hard time’.	As above

*At baseline, students were asked whether they had ever smoked a cigarette/consumed an alcoholic drink.

Student individual and environmental protective factors

The Resilience and Youth Development module of the California Healthy Kids Survey was used to measure individual and environmental protective factors.65 Items for six individual and three environmental factor subscales (table 1) were selected. Aggregate individual and environmental protective factor scores were used as outcome measures. Consistent with a previous study of the survey tool,65 analysis of baseline responses showed the subscales were reasonably internally consistent and valid (Cronbach alpha coefficients: individual 0.55–0.81; environmental 0.77–0.88). Confirmatory factor analysis65 demonstrated the subscale factor structure to be a good model fit (X2 (1133)=6573, p<0.0001; comparative fit index 0.92, root mean square error of approximation 0.04).

Statistical analysis

Student sociodemographic subgroups

Student-reported residential postcode was used to calculate student socioeconomic status53 and remoteness of residential location.66 Students were classified into the following subgroups based on their baseline survey characteristics: gender (males, females), socioeconomic status (as defined by SEIFA Index of Relative Socio-economic advantage/disadvantage; low: scores of <990 (most disadvantaged), high: scores ≥99053 and geographic location (as defined by the Accessibility/Remoteness Index of Australia; major city: 0–0.2, inner regional: >0.2–2.4, outer regional/remote: >2.4–15).

Previous substance use subgroups

Ever use of tobacco was used to define baseline tobacco use, and ever use of alcohol was used to define baseline alcohol use (user and non-user). Use of marijuana in the last 4 weeks was used to define baseline marijuana use, and use of other illicit substances in the last 4 weeks was used to define baseline other illicit substance use (user and non-user). Baseline use of any substance was defined as use of at least one substance derived from baseline use of tobacco, alcohol, marijuana or illicit substances (user and non-user).

Primary outcomes: student alcohol and tobacco use

Recent tobacco use was defined as having smoked at least one cigarette in the last week, and recent alcohol use was defined as at least one alcoholic drink in the last week (yes/no). An average number of cigarettes and alcoholic drinks consumed in the last week was calculated from the responses for daily consumption. The response options for ‘risk alcohol use’ were dichotomised as ‘none’ or ‘use’ (‘once’/‘twice’/‘3–6 times’/‘7 or more times’).

Secondary outcomes: student illicit substance use

The response options for both marijuana and other illicit substance use were dichotomised as ‘none’ or ‘use’ (‘once or twice’/‘3–5 times’/‘6–9 times’/‘10–19 times’/‘20–39 times’/‘40 or more times’).

Secondary outcomes: student individual and environmental protective factor scores

Student protective factor subscale scores were calculated by averaging the responses to all items in each subscale. Aggregate individual and environmental protective factor scores were calculated by averaging all relevant subscale scores for each student.65 Mean scores ranged from 1 to 4, with higher scores more favourable.

Subgroup analyses

Best practice principles for subgroup analysis specify that such analyses should be: exploratory; limited to primary outcomes with a small number of predefined subgroups; analysis by formal statistical tests of interaction; and analysis within subgroups conducted only if an interaction is statistically significant.67 68 As such, comparisons between treatment groups for each dichotomous (five outcomes) and continuous (four outcomes) outcome at follow-up for the cohort grade 10 students in intervention and control schools by each of the four subgroups was undertaken to determine the effectiveness of the intervention using generalised linear mixed models (binomial distribution with a logit link) (20 models) and linear mixed models (16 models), respectively. All models included a fixed effect for treatment group (intervention vs control), a random effect for each school to account for clustering of responses within schools and an interaction term (treatment × subgroup) to determine differential intervention effect. ORs with 95% Wald CIs were calculated for each subgroup category. Where an interaction term was significant, comparisons between treatment groups within each of the subgroups was undertaken using the same modelling approach to determine the effectiveness of the intervention within each individual subgroup.

A criterion for statistical significance of p≤0.01 was used due to multiple testing.69 All analyses were undertaken using SAS Software V.9.4.70

Patient and public involvement

School staff from intervention schools were involved in implementing and selecting which resources and strategies to implement as part of the intervention. The deidentified and aggregate overall results of the study were disseminated to the principals of participating schools at the conclusion of the study.

Results

Sample

Forty-four of the 47 eligible schools were approached prior to obtaining the quota of 32 participating schools (73% consent rate). Of those 28 were government and four Catholic schools, and 21 were medium and 11 were large-sized schools (see figure 1). No schools withdrew following allocation.

Figure 1

Study flow diagram.

At baseline, parental consent was provided for 3530 grade seven students (76.9% of enrolled students), of which 3115 students participated in the baseline survey (67.9% of enrolled students; 88.2% of students with parental consent). Follow-up data were collected from 2149 of the students who completed the baseline survey (retention rate 69.0%; intervention 67.3%, control 71.6%; 46.8% of students enrolled at baseline). There was no differential loss to follow-up overall between intervention and control groups (p=0.1). Students who moved between schools (n=30) and those who participated but did not answer substance use items at baseline (n=14) were excluded resulting in a cohort of 2105 students for the primary analysis. The demographic characteristics of students who completed the baseline and follow-up survey are shown in table 2.

Table 2

Demographic and substance use characteristics by group at follow-up (n=2105)

Student characteristics	Intervention n (%)	Control n (%)	All students
Total students	1261	844	2105
Age, M (SD)	15.5 (0.5)	15.5 (0.5)	15.5 (0.5)
Aboriginal/Torres Strait Islander	128 (10.2)	95 (11.3)	223 (10.6)
Gender
Male	640 (50.8)	431 (51.1)	1071 (50.9)
Female	621 (49.3)	413 (48.9)	1034 (49.1)
Remoteness*
Major cities	525 (41.7)	508 (60.2)	1033 (49.1)
Inner regional	612 (48.6)	262 (31.0)	874 (41.5)
Outer regional/remote	123 (9.8)	74 (8.8)	197 (9.4)
Socioeconomic status (SES)*
Low (<990) (most disadvantaged)	704 (55.9)	534 (63.3)	1238 (58.8)
High (≥990)	556 (44.1)	310 (36.7)	866 (41.2)
Previous substance user
Tobacco user†	106 (8.4)	64 (7.6)	170 (8.1)
Alcohol user†	364 (28.9)	200 (23.8)	564 (26.8)
Marijuana user‡	14 (1.1)	7 (0.8)	21 (1.0)
Other illicit user‡	8 (0.6)	4 (0.5)	12 (0.6)
Any substance user§	387 (30.7)	215 (25.5)	602 (28.6)

*SES and remoteness could not be calculated for five students as postcode was missing (four intervention, one control).

†Ever use of tobacco and alcohol were used to define baseline tobacco and alcohol use, respectively.

‡Use of marijuana and other illicit substances in the last 4 weeks was used to define baseline marijuana and other illicit substance use, respectively.

§Baseline use of any substance was defined as use of at least one substance derived from baseline use of tobacco, alcohol, marijuana or illicit substances. This was the variable used for baseline use for protective factor outcomes.

Substance use

Table 2 shows the characteristics of the subgroups at follow-up. Online supplementary appendix A shows the proportion of, or mean score for, students in each subgroup reporting each outcome at baseline and follow-up, respectively. There was no difference between intervention and control students for any measure of substance use at follow-up for students overall (table 3). Similarly, there was no differential effect for any of the seven substance use outcomes between subgroups defined by gender, geographic location or previous substance use (table 3). A differential effect was found for one of the seven outcomes for subgroups defined by socioeconomic status; there was a significant interaction (p=0.003) between treatment and socioeconomic status on the mean number of cigarettes smoked (by students who were recent smokers). The CIs for the estimates within the both low (mean difference (MD) −12.89, 95% CI −26.00 to 0.23) and high (MD 16.36, 95% CI −1.03 to 33.76) socioeconomic subgroups included the null value (table 3).

Table 3

Unadjusted intervention versus control group substance use comparisons at follow-up by all students and subgroups*

Subgroups	Tobacco		Alcohol			Illicit substances		Protective factors
	Recent OR (95% CI)	Amount Mean difference (95% CI)†	Recent OR (95% CI)	‘Risk’ OR (95% CI)	Amount Mean difference (95% CI)‡	Marijuana OR (95% CI)	Other substance OR (95% CI)	Individual Mean difference (95% CI)	Environmental Mean difference (95% CI)
All students
Unadjusted	1.37 (0.85 to 2.22)	−2.30 (−11.94 to 7.69)	1.10 (0.77 to 1.57)	0.97 (0.71 to 1.33)	−0.77 (−2.80 to 1.25)	1.12 (0.74 to 1.68)	1.19 (0.67 to 2.10)	0 (−0.08 to 0.08)	0 (−0.10 to 0.10)
Adjusted§	1.48 (0.93 to 2.37)	−0.98 (−11.15 to 9.20)	1.10 (0.77 to 1.56)	1.03 (0.74 to 1.43)	−0.51 (−2.61 to 1.59)	1.18 (0.80 to 1.72)	1.42 (0.85 to 2.38)	−0.01 (−0.07 to 0.06)	−0.02 (−0.09 to 0.06)
Gender
Males	1.37 (0.77 to 2.42)	−4.56 (−19.35 to 10.23)	1.09 (0.72 to 1.66)	0.80 (0.55 to 1.16)	−1.40 (−4.29 to 1.49)	1.04 (0.65 to 1.67)	0.96 (0.51 to 1.82)	0.01 (−0.07 to 0.10)	0.02 (−0.09 to 0.13)
Females	1.38 (0.78 to 2.43)	−0.33 (−13.69 to 13.03)	1.12 (0.72 to 1.73)	1.21 (0.82 to 1.79)	0 (−3.01 to 3.00)	1.23 (0.74 to 2.04)	1.77 (0.80 to 3.92)	−0.02 (−0.11 to 0.07)	−0.02 (−0.13 to 0.10)
Remoteness
Major city	1.93 (0.91 to 4.07)	7.04 (−15.34 to 29.42)	1.30 (0.76 to 2.22)	1.03 (0.64 to 1.67)	−0.87 (−4.57 to 2.83)	1.49 (0.79 to 2.79)	2.09 (0.96 to 4.57)	−0.07 (−0.18 to 0.04)	−0.13 (−0.26 to 0.00)
Inner regional	1.36 (0.58 to 3.21)	−7.17 (−32.99 to 18.66)	1.08 (0.59 to 1.97)	0.92 (0.53 to 1.62)	−0.78 (−4.96 to 3.39)	0.97 (0.48 to 1.99)	0.75 (0.30 to 1.87)	0 (−0.12 to 0.12)	0.10 (−0.05 to 0.25)
Outer regional/remote	0.60 (0.18 to 2.07)	−17.91 (−56.18 to 20.36)	0.48 (0.17 to 1.33)	0.98 (0.40 to 2.39)	0.77 (−7.28 to 8.82)	0.52 (0.16 to 1.71)	0.36 (0.04 to 3.18)	0.23 (0.02 to 0.43)	0.15 (−0.11 to 0.40)
Socioeconomic status
Low	1.40 (0.79 to 2.48)	−12.89** (−26.00 to 0.23)	1.06 (0.70 to 1.61)	1.06 (0.72 to 1.55)	−0.91 (−3.65 to 1.84)	1.30 (0.79 to 2.15)	0.47 (0.72 to 3.01)	0.00 (−0.09 to 0.09)	0.01 (−0.10 to 0.12)
High	1.35 (0.65 to 2.81)	16.36 (−1.03 to 33.76)	1.21 (0.71 to 2.07)	0.84 (0.52 to 1.35)	−0.59 (−4.20 to 3.02)	0.85 (0.46 to 1.56)	0.82 (0.34 to 1.94)	−0.01 (−0.12 to 0.10)	−0.02 (−0.15 to 0.12)
Baseline use
Non-user	1.39 (0.83 to 2.31)	−5.16 (−16.55 to 6.23)	1.22 (0.76 to 1.95)	0.99 (0.69 to 1.41)	−1.05 (−3.76 to 1.67)	1.37 (0.13 to 14.23)	–¶	0.01 (−0.07 to 0.09)	−0.01 (−0.11 to 0.09)
User	1.34 (0.60 to 2.97)	6.83 (−11.76 to 25.43)	0.94 (0.63 to 1.42)	0.79 (0.52 to 1.21)	−0.95 (−4.1 to 2.20)	1.09 (0.69 to 1.72)	1.15 (0.58 to 2.27)	0.00 (−0.10 to 0.10)	0.05 (−0.08 to 0.18)

*Estimates presented originate from models inclusive of interaction term (treatment × subgroup). Sample sizes for each comparison ranged from 12 to 2105 and are detailed in online supplementary appendix B.

†Of students who reported tobacco use in the last week.

‡Of students who reported alcohol use in the last week.

§Models were adjusted for a priori selected prognostic variables (age, gender, school type, school size, Aboriginal/Torres Strait Islander status, ethnicity, non-English-speaking background and socioeconomic status).

¶OR estimated as infinity due to small sample size and zero cell counts.

**interaction term (treatment × subgroup) p<0.01.

Student individual and environmental protective factors

At follow-up, there was no difference in mean individual or environmental protective factor scores between intervention and control students (table 3). Similarly, there was no differential effect for any of the protective factor outcomes by any subgroup (table 3).

Discussion

This study sought to examine the differential effectiveness of a universally delivered school-based ‘resilience’ protective factor intervention on substance use by adolescents according to their baseline sociodemographic characteristics and previous substance use. The study found negligible evidence (1 of 36 tests) of differential intervention effectiveness between student subgroups. A differential intervention effect was found for number of cigarettes smoked by recent smokers, by socioeconomic status which, on further examination within subgroups, showed no significant difference in the consumption of cigarettes in intervention compared with control students residing within either subgroup. These findings, coupled with the primary study finding of no intervention effect on any measure of adolescent substance use or protective factors for the overall trial sample,49 suggests that a pragmatic approach to the universal delivery of a resilience-focused substance use intervention was not effective at any level.

The findings of the study with respect to subgroups defined by sociodemographic characteristics are generally consistent with previous universal school-based studies that targeted protective factors as part of a broader intervention approach. With respect to subgroups defined by gender, previous studies have reported equivocal support for such interventions being effective for alcohol or illicit substance use for males only,42 for females only71 or effective for both males and females40 in comparison with the current study that found no differential effect. Only one previous study was identified that investigated differential effectiveness of such an intervention approach by socioeconomic status, which reported an intervention effect for drunkenness for students of low but not high socioeconomic status.41 No previous studies were identified that investigated differential intervention effectiveness by geographic location for universal school-based protective factor interventions.

The findings of previous studies of intervention effectiveness by subgroups defined by previous substance use are equivocal. For example, previous studies have found no evidence of differential intervention effect between subgroups defined by previous substance use for both tobacco45and alcohol.45 46 In contrast, other studies have found results favouring students who have never used a substance44 and those who are previous substance users.46

The absence of an intervention effect for the hypothesised mechanism of intervention effect, resilience protective factors, for both the overall trial sample49 and across all subgroups assessed in this study precludes the ability to determine whether the enhancement of protective factors can lead to a reduction in adolescent substance use. However, the singular significant result in this study regarding the consumption of tobacco for recent smokers by socioeconomic status suggests that despite not impacting on protective factor levels, the intervention may have a differential effect on socioeconomically disadvantaged compared with advantaged students. Such results may suggest that an alternative mechanism may be responsible for the decrease in tobacco consumption. A more likely explanation for the singular result for 1 of 36 tests undertaken is that the result was false positive due to the number of tests that were conducted.

Various aspects of the intervention design may have contributed to the null finding for resilience protective factors both overall and within the student subgroups and have been reported previously.49 First, the universal delivery of the intervention without any differentiation or targeted intervention for students with lower protective factor scores or students at greater risk within particular student subgroups may have limited its ability to impact across all students. Second, the use of a pragmatic intervention approach involving school staff selection and implementation of existing readily available curriculum resources and programmes and flexibility regarding the order in which selected resources were implemented may have contributed to the null study findings as pragmatic intervention approaches have been reported to be less likely to be effective than non-pragmatic approaches.72 73 As previously reported, data regarding similar levels of implementation for 15 of 16 targeted strategies in both intervention and control groups suggests the use of programmes and resources that were also accessible to control schools may also have led to a lack of differential intervention exposure between group49 or may be an indication of cross-site contamination. Third, the duration of the intervention primarily over 2 years may have resulted in insufficient exposure to impact on student protective factors. Finally, the intervention’s focus on protective factors only, with no content addressing known risk factors of substance use (such as peer substance use74), and a limited focus on family and community-based protective factors of substance use75 may have impacted on the findings.

Strengths of the overall study included the cluster-randomised controlled study design, the use of implementation support strategies and it being a real-world pragmatic effectiveness trial. This study adhered to criteria for a best practice analytical approach to subgroup analyses67 with the exception that secondary outcomes (ie, resilience protective factor outcomes) were examined in subgroups, and the proposed subgroup analyses were not published a priori. In terms of further limitations, the number of comparisons undertaken within this study may have increased the risk of type 1 error and led to the sole significant result; however, the use of a formal statistical interaction test, rather than examination of intervention effect within individual subgroups, reduces this risk.76 In the overall study, the proportion of enrolled students completing both the baseline and follow-up surveys was below 50%, and while typical for school-based research,77 it may limit the generalisability of the study results. Additionally, despite the scale of the overall study in 32 secondary schools, it was not designed to be powered to detect differences in outcomes within participant subgroups. As a result, any actual differences in intervention effectiveness by subgroup may not have been detectable. The small sample size of the exposure groups in some of the subgroup analyses, such as examination of differences in the number of cigarettes smoked, further limited the statistical power to detect differences between groups. As a result, the subgroup analyses are considered exploratory in order to generate hypotheses for future research and should be interpreted with caution. While the effect of the intervention on illicit substance use among students with low socioeconomic status was double that of students with high socioeconomic status appears promising, it is unclear whether such a result would be substantiated in a study sufficiently powered to detect difference in outcomes within participants subgroups. Finally, the study was limited by its focus on adolescent substance use and resilience outcomes, and the impact of the intervention on other student outcomes, such as alcohol-related harm, or implementation outcomes, such as assessment of teacher impact on intervention implementation, was not assessed.

Conclusion

This exploratory study found negligible evidence of an intervention effect on adolescent substance use and protective factors outcomes according to student sociodemographic or substance use characteristics following implementation of a universal school-based resilience intervention. While there was some evidence of a differential intervention effect on tobacco use by smokers in subgroup analysis by socioeconomic status, which suggests there may be some potential for a resilience intervention in reducing tobacco use within this group of students, the result should be interpreted with caution and further sufficiently powered research conducted to confirm this.