Effectiveness of a neuroscience-based, harm reduction program for older adolescents: A cluster randomised controlled trial of the Illicit Project.

The prevention of risky adolescent substance use is critical. Limited age-appropriate, school-based programs target adolescents aged 16-19 years, despite this representing the age of initiation and escalation of substance use. The Illicit Project is a neuroscience-based, harm reduction program targeting late adolescents, designed to address this gap. The current study aims to evaluate the program's effectiveness in reducing risky substance use and related harms among late adolescents. A cluster randomised controlled trial was conducted involving 950 students (Mage = 15.9 years SD = 0.68; 60% Female) from eight secondary schools in Australia. Five schools received The Illicit Project program, and three schools were randomised into the active control group (health education as usual). All students completed a self-report survey at baseline and 6-months post-baseline and intervention students completed a program evaluation survey. Outcomes include alcohol and substance use, alcohol related harms and drug literacy levels (knowledge and skills). At 6-months post baseline, individuals in the intervention group were less likely to engage in weekly binge drinking (OR = 0.56), high monthly alcohol consumption (OR = 0.56), early onset cannabis use (OR = 0.35), risky single occasion cannabis use (OR = 0.48), MDMA use (OR = 0.16) or nicotine product use (OR = 0.59) compared to the control group. Students in the intervention group were less likely to have experience alcohol related harms (OR = 0.57) and more likely to have higher drug literacy scores (β = 2.44) at follow-up. These preliminary results support the effectiveness of The Illicit Project. Further follow-up is required to determine the durability of the results over time.

Introduction

Alcohol, cannabis and illicit substance use are major contributors of global morbidity and mortality, and harm falls disproportionately on young people (Griswold et al., 2018, Peacock et al., 2018). Adolescence 10–19 years) is a critical developmental period marked the onset of risk taking, such as substance use (Patton et al., 2016, Debenham et al., 2021a). Although early adolescents (12–14 years) are driving global downward trends in alcohol consumption (Kraus et al., 2018, Kerr et al., 2013, Meng et al., 2014), middle and late adolescents (15–19 years) continue to consume substances in risky quantities (Callinan et al., 2020, Statistics ABo, 2017). Risky/binge drinking, defined as five or more standard drinks per day (NHMRC, 2020), is the most common form of alcohol consumption in this age group. Over one quarter (26%) of Australians aged 16–17 years have engaged in risky drinking in the past fortnight, over one in ten (11%) have engaged in risky drinking in the past week, one in eight (16%) report monthly cannabis use and 4% report past month MDMA use (Guerin NaW, 2017). The harm from substance is cumulative (Hamidullah et al., 2020, Degenhardt et al., 2018)and adolescent onset use is linked to pervasive harm spanning cognitive, social, and physiological domains. As the foundation for future health, adolescent prevention requires urgent public health support.

The school environment is an effective and efficient place to reach large numbers of young people and the cost of implementation is low (Tancred et al., 2018). In Australia, mandatory health curricula ensure students in Years 8–10 (ages 13–16) receive substance use education, however senior students in Years 11–12 (ages 16–18) are not guaranteed the same education – despite this being the average age of initiation and escalation of substance useGuerin NaW, 2017; Xu et al., 2020;15 (Guerin NaW, 2017, Xu et al., 2020). Consequently, there are few programs targeting late adolescents (Tremblay et al., 2020)and late adolescent substance use prevention is lacking (Debenham et al., 2019).

The evidence for substance use prevention is mixed (Foxcroft and Tsertsvadze, 2012, Faggiano et al., 2014, Onrust et al., 2016). A systematic review reported that older adolescents benefited from the social influence approach, peer-resistance skills, personal goal development, self-control training and social problem solving (Onrust et al., 2016). High-risk adolescents responded positively to strategies around self-control and the biopsychosocial model of behaviour change (Onrust et al., 2016). Meta-analyses suggest prevention programs are associated with small reductions in the frequency and quantity of substance use rather than reducing the overall prevalence of use (Strøm et al., 2014, Hennessy and Tanner-Smith, 2015). The finding, that prevention programs are more effective at reducing risky use, over single time use, suggests harm reduction targets should be a key consideration in the development of school-based interventions. Exploring new and innovative methods to engage young people in substance use harm reduction within the school environment is urgently required (Debenham et al., 2019).

The Illicit Project is an age-appropriate, harm reduction program that leverages neuroscience to reduce the harms of substance use. Delivered to young people (aged 15–19 years), the program aims to improve drug literacy, which can be understood as the knowledge, skills and strategies required to identify and minimise substance use harms and seek support when necessary (Debenham et al., 2020). The Illicit Project has been piloted as a facilitator-delivered, face-to-face program and demonstrated high acceptance and credibility among students and teachers (Debenham et al., 2020). Following the pilot, the program underwent web-based adaptation to improve implementation and potential scalability (Debenham et al., 2021b). The current study aims to evaluate the effectiveness of The Illicit Project in reducing alcohol and other substance use, alcohol related harms and drug literacy levels in late adolescents.

Methods

Study design

A cluster randomised controlled trial (cRCT) was conducted in secondary schools across New South Wales, Australia in 2020–2021. Schools were recruited and randomly allocated to the intervention group or the active control group using block randomisation in R to avoid contamination between groups by an external researcher (Uschner et al., 2018). The five schools (seven cohorts; Mage = 15.8) in the intervention group completed The Illicit Project program during class over a 6-week period at the end of 2020. The three schools (six cohorts; Mage = 16.1) in the active control group completed health education as usual. All participants completed self-report surveys at baseline and 6 months post-baseline. Ethics approval for the study was received from the University of Sydney Human Research Ethics Committee (2020/053) and the State Education Research Applications Process (2020237). The trial follows the Consolidated Standards of Reporting Trials (CONSORT) statement is registered with the Australian New Zealand Clinical Trials Registry (ACTRN12620000805976). Further information on the study protocol have been reported elsewhere (Debenham et al., 2021b).

Participants and procedure

Eighty-two independent and state schools were invited to participate in the study and nine schools agreed to partake. Both passive parental consent and active student consent were required for participation, and one school required additional active parental consent. A trusted online data collection platform (REDCap) was used to generate unique identifiers for each participant to enable data linkage across time whilst ensuring confidentiality (Harris et al., 2019). Five schools were randomly allocated to the intervention group and four schools were randomly allocated to the active control group. Before the trial began, one school withdrew from the active control group due to scheduling issues. The final baseline sample included 950 participants from eight schools.

The Illicit Project intervention

Students in the intervention group received The Illicit Project, a the three-class, web-based program during class time. The Illicit Project is an interactive, neuroscience-based harm reduction program that aims to upskill young people in substance use harm minimisation. The evidence-based program encourages rapid skill development in areas including but not limited to peer resistance training, normative education, harm reduction with alcohol, MDMA and cannabis (Stockings et al., 2016, Stockings et al., 2016), help seeking, neuroscience and supporting friends (Foxcroft and Tsertsvadze, 2012, Faggiano et al., 2014). The program adopts a strengths-based approach to inspire young people to protect and nurture a healthy brain during protracted neurodevelopment. The three modules: 1) Alcohol and the Developing Brain; 2) MDMA, Cannabis Use, and Harm Reduction; and 3) Mental Health and Wellbeing, are comprised of interactive activities, case studies, quizzes, goal setting, interviews with neuroscientists, drug experts and a representative group of young people from around Australia (see Appendix A for key program outcomes). The program promotes inclusion and diversity through the representation of individuals from key ethnic groups and gender and sexual identities in the interviews, animations and activities. A pilot study of the face-to-face version confirmed The Illicit Project is both feasible and credible in the school environment (Debenham et al., 2020). For access to the program see www.theillicitproject.com and for illustrations see Fig. 1.

Fig. 1

Graphics from The Illicit Project program.

Control group

Schools in the active control group implemented health education as usual, which falls under the New South Wales Personal Development, Health and Physical Education curriculum and the Life Ready curriculum.

Outcome measures

Self-report surveys were administered to all students to collect demographic data (e.g., sex, age and grades) and primary outcomes (substance use, alcohol harm and drug literacy levels). All included measures are well-validated scales that reflect developmentally relevant patterns of substance use consumption as described below.

Alcohol and illicit substance use were measured through quantity and frequency, adapted from the Alcohol, Smoking and Substance Use Involvement Screening Test (ASSIST; (Humeniuk et al., 2008). Risky alcohol consumption (which at the time of the survey was defined as having over 5 standard drinks per day) was measured through the self-report item ‘how often did you have 5 or more standard alcoholic drinks on one occasion in the past 6 months?’ and dichotomised into weekly and monthly use (yes/no). Average monthly consumption was calculated by multiplying a frequency item ‘how often did you consume a standard alcoholic beverage in the past 6 months?’ by a quantity item ‘on average how many standard alcoholic drinks would you consume on a typical occasion when you are drinking alcohol?’ and high monthly alcohol consumption was quantified as consuming a total of 20 or more standard drinks per month (yes/no). The frequency of cannabis use was quantified as monthly (yes/no) and average quantity of cannabis use was assessed as risky if individual consumption surpasses one joint per occasion (yes/no). Exposure to MDMA, nicotine-products, methamphetamine, non-medical prescription medication and polysubstance use, will be assessed through any use in the past 6 months (yes/no).

Alcohol related-harms were measured by the 18-item, Rutgers Alcohol Problem Index scale, which demonstrates acceptable internal consistency (α = 0.85) (Neal et al., 2006, Earleywine et al., 2008). Greater harms are reflected by higher scores and will be examined by measuring those experiencing over one harm (yes/no).

Total Drug Literacy scores were measured through summing responses from a 20-item true or false knowledge scale and a 6-item attitude scale where students completed items such as ‘how confident are you in minimising the harms of substances?’ on a five-point Likert scale from not at all to extremely. Both scales were adapted from the School Health and Alcohol Harm Reduction Project (McBride et al., 2004)and demonstrate acceptable internal consistency (α = 0.67) (Debenham et al., 2020). Higher scores reflect greater drug literacy levels and the drug literacy scale was also administered in The Illicit Project pilot study (Debenham et al., 2020).

Sample size calculations

Based on power calculations for cRCTs using multi-level mixed effect regression models, this trial required three schools per arm, with at least 60 students per school, to achieve 80% power and a standardized, between-group mean difference of 0.3 (p = 0.05) at the end of the trial (Heo and Leon, 2009). Based on similar school-based trials, we expected 10% school dropout and the average year group size to be 100 students, therefore we aimed to recruit eight schools and 800 students to the study.

Statistical analysis

Multilevel mixed effect regression models, which account for clustering at the school level were used to determine the effectiveness of the intervention. Multilevel models are the preferred technique to analyse hierarchical data as they account for both fixed and random effects do not assume students from the same school have independent outcomes (Molenberghs and Verbeke, 2006). For the present analysis, two level models which account for clustering at the school level and baseline scores were applied, with multilevel logistic regressions modelling categorical outcomes (alcohol use, cannabis use, MDMA use, nicotine use, methamphetamine use, polysubstance use and alcohol harms) and multilevel linear regressions modelling continuous outcomes (drug literacy scores). Effect sizes for categorical outcomes are reported as odds ratios (OR) with 95% confidence intervals (CI) and as unstandardised beta coefficients with 95% CI for continuous outcomes. Missing data resulted from attrition (those who participated in the study at baseline but either changed schools or declined follow-up) and unavailability (those who remained in the study at follow-up but did not partake in outcome measurement). To determine whether baseline outcomes predict missingness at follow-up, chi-square tests for categorical outcomes, one-way analysis of variance for continuous outcomes and Mann-Whitney U for nonnormally distributed continuous outcomes were conducted. Multiple imputations were conducted to assess the robustness of the results against missing data, under the assumption that data were missing at random (see Appendix B for details). All data analysis is conducted on the intention-to-treat sample in Stata version 17 (StataCorp, 2021).

Results

Participant characteristics and baseline equivalence

The final sample included 950 students from eight schools in New South Wales (Mage = 15.9 years, SD = 0.68, 60% female, 94% born in Australia). See Table 1 for baseline characteristics of the sample and Table 2 for the frequency of outcome variables over time by trial group.

Table 1

Baseline characteristics of the sample.

	The Illicit Project n = 681	Control n = 269	Total Sample N = 950
Gender (% Female)	455/681 (67%)	140/269 (52%)	60%
Mean Age (SD)	15.8 (0.6)	16.1 (0.7)	15.9 (0.7)
Average grades
90–100%	12%	13%	12%
80–89%	29%	29%	29%
70–79%	24%	28%	25%
60–69%	14%	15%	15%
59% or below	21%	15%	19%
Year Group
Year 10	71%	54%	67%
Year 11	11%	26%	15%
Year 12	18%	20%	18%
School Type
Independent	2	0	2
State	3	3	6

Table 2

Outcome prevalence over time.

Primary Outcomes	Timepoint	Intervention	Control
Monthly binge drinking % (n)	Baseline	19%	33%
	6 months	20%	35%
Weekly binge drinking % (n)	Baseline	3%	4%
	6 months	3%	7%
High monthly alcohol consumption % (n)	Baseline	8%	14%
	6 months	10%	19%
Monthly cannabis use (%) n	Baseline	9%	9%
	6 months	7%	9%
High quantity cannabis use (multi puff)	Baseline	18%	23%
	6 months	17%	28%
Early onset cannabis use (<16 years)	Baseline	17%	14%
	6 months	12%	19%
Nicotine product use % (n)	Baseline	22%	26%
	6 months	23%	36%
MDMA use % (n)	Baseline	6%	6%
	6 months	2%	9%
Methamphetamine use % (n)	Baseline	2%	3%
	6 months	1%	3%
Polysubstance use % (n)	Baseline	6%	4%
	6 months	5%	4%
Alcohol harms % (n)	Baseline	30%	41%
	6 months	27%	42%
Mean Drug Literacy (SD)	Baseline	22.6	24.3
	6 months	25.9	24.0

Baseline N = 950; 6-months N = 532.

Retention

Of the 950 participants who completed the baseline survey (October-December 2019), 60% completed the 6-month follow-up survey (April-May 2020), with retention being higher in the control (68%) compared to the intervention group (51%) (see Fig. 2). Missingness was primarily due schools being unable to schedule in-class follow-up survey occasions and relying on students to complete the survey outside of school time, due to the impact of COVID-19. Missing data analyses revealed that students missing at follow-up, were more likely to be male (OR = 2.5, 95% CI 1.91–3.25), be in the intervention group (OR = 1.6, 95% CI: 1.2–2.2) and have lower grades at baseline F(5953) = 19.61, p = 0.00). Missing students tended to be “higher risk”, scoring lower on drug literacy (F(1,927) = 15.38, p = 0.0001), having higher odds of monthly binge drinking (OR = 1.5, 95% CI 1.12–2.05), monthly alcohol consumption (OR = 2.0, 95% CI 1.30–3.11), single occasion risky cannabis use (OR = 1.67, 95% CI 1.21–2.32), early onset-cannabis use (OR = 2.1, 95% CI 1.49–3.03), MDMA use (OR = 2.0, 95% CI 1.14–3. 45) and experiencing alcohol related harms (OR = 1.4, 95% CI 1.10–1.90). Missingness was not predicted by weekly binge drinking (p = 0.07) or nicotine use (p = 0.124).

Fig. 2

CONSORT flow chart of participant recruitment and retention.

Intervention effects over time

Risky alcohol and illicit substance use

At 6-months post baseline, primary analysis indicated the intervention group had reduced odds of weekly binge drinking (OR = 0.38, 95% CI 0.15–1.01), high monthly alcohol consumption (OR = 0.56, 95% CI 0.32–0.98), early-onset cannabis use (OR = 0.35, 95% CI 0.17–0.72) and high quantity cannabis use (OR = 0.48, 95% CI 0. 0.25–0.92), compared to the control group. By comparison, multiple imputations indicate the intervention group had reduced odds of early-onset cannabis use compared to control (OR = 0.10, 95% CI 0.01–0.79), however no differences between weekly binge drinking, high monthly alcohol consumption and single occasion risky cannabis use were evident between groups (see Appendix B). Students in the intervention group had reduced odds of past 6-month MDMA use compared to control (OR = 0.16, 95% CI 0. 0.05–0.49), which was consistent in multiple imputation analysis (OR = 0.23, 95% CI 0.05–0.97; see Appendix B). In the primary analysis the intervention group had reduced odds of nicotine product use compared to control (OR = 0.59, 95% CI 0.35–0.98), however this was not consistent in multiple imputation analysis (OR = 0.66, (5% CI0.30–0.70). There were no differences between groups in reports of monthly binge drinking, monthly cannabis use, methamphetamine use or polysubstance use at the 6-month follow-up.

Alcohol-related harms

Primary analysis indicated that students in the intervention group had lower odds of experiencing alcohol related harms at follow-up (OR = 0.57, 95% CI 0.35–0.92; see Table 3), however this was not supported by multiple imputation analysis (OR = 0.61, 95% CI 0.57–1.38; see Appendix B).

Table 3

Results from multilevel regression models at 6-month follow-up.

Multilevel logistic regression models	OR	95% CI	p-value
Monthly binge drinking	0.61	0.31–1.18	0.14
Weekly binge drinking	0.38	0.15–1.01	0.05*
High monthly alcohol consumption	0.56	0.32–0.98	0.04*
Monthly cannabis use	1.26	0.55–2.89	0.58
High quantity cannabis use	0.48	0.25–0.92	0.02*
Early onset cannabis use	0.35	0.17–0.72	0.00*
Nicotine product use	0.59	0.35–0.98	0.04*
MDMA use	0.16	0.05–0.49	0.00*
Methamphetamine use	0.44	0.09–2.13	0.31
Polysubstance use	1.86	0.58–5.99	0.30
Alcohol harms	0.57	0.35–0.92	0.02*
Multilevel linear regression models	β	95% CI	p-value
Drug Literacy	3.05	1.73–4.36	0.00*

OR, odds ratio; *significant outcome at p < 0.05.

Drug literacy levels

The intervention group were more likely to score higher in drug literacy than students in the control group at follow-up (β = 3.05, 95% CI 1.73–4.36p = 0.000; see Table 3), which was consistent in multiple imputation analysis (β = 2.16, 95% CI 1.19–3.12; see Appendix B).

Discusion

This is the first RCT to evaluate the effectiveness of a neuroscience-based harm reduction program, The Illicit Project, in secondary schools. The primary analysis and multiple imputation sensitivity analysis, indicate that at the 6-month assessment occasion, the intervention group was associated with reduced odds of early-onset cannabis use, past 6-month ecstasy use and increased likelihood of scoring high in drug literacy compared to the active control group. These important findings represent a large step away from the “just say no” abstinence-based approach to substance use prevention and provide an age-appropriate method to minimise harm in late adolescents, being a harm reduction approach. Harm reduction recognises that some people will engage in substance use and provides strategies to reduce the physiological and psychological harms associated with use (Stockings et al., 2016, Stockings et al., 2016). This approach addresses both proximal and distal harms of substances and can incorporate mental health and wellbeing strategies. These promising results contribute to a growing evidence-base that prevention investment should span over the entire breadth of development for young people (Das et al., 2016).

Cannabis is the most commonly used illicit substance and prevention is a national priority. It is one of the substances that the most Australians seek help for and use peaks during adolescence and early adulthood (Australian Institute of Health and Welfare, 2015). The Illicit Project’s large positive impact on reducing the odds of early-onset cannabis use, compares to an umbrella review and a meta-analyses of school-based prevention programs which report programs to have an average 17% risk reduction for cannabis sue outcomes (RR = 0.83, 95% CI: 0.69–0.99) (Das et al., 2016, Porath-Waller et al., 2010). Early cannabis use initiation is associated with a 30% increase in the likelihood of developing substance use dependence later in life (Marmet et al., 2021, Perkonigg et al., 2008)and the promising prevention results demonstrated in the current study may help reduce the burden of cannabis use disorder. The intervention had no impact on monthly cannabis use over the 6-month follow-up period, however longer-term follow-up such as 12- and 24-month survey occasions, may be required to detect changes.

The Illicit Project demonstrated beneficial effects in reducing the odds of MDMA use, which compares to a well-validated program targeting middle-adolescents, known as the Climate Schools: MDMA and Emerging Drugs module, which demonstrated a 10 fold reduction in the likelihood of MDMA use intention among young people (Champion et al., 2016). Considering the recent inquest into MDMA-related festival deaths and the Coroner’s recommendation for school-based MDMA education; these results could be of high public health interest (Hughes et al., 2019). The program had no impact on polysubstance use or methamphetamine use, potentially owing to the extremely low prevalence rates in this age group and further follow-up will be required to explore this relationship overtime.

Despite promising preliminary results across the majority of primary outcomes, the findings must be considered within the context of several limiting factors. First, retention at the 6-month follow-up was lower than anticipated due to difficulties with school scheduling and analyses revealed that high-risk students and students in the intervention group were more likely to be missing at follow-up. The three schools with scheduling issues all belonged to the intervention group and had 73% unavailability at follow-up, compared to the five schools with scheduled in-class survey completion, which had 25% unavailability at follow-up, which is within international standards (Lloyd et al., 2017). Although missingness of high-risk adolescents is common (Cook et al., 2002), the differential attrition may introduce bias to the results and limit the validity and generalisability of the results. The research team implemented rigorous retention strategies and worked closely alongside schools, however without scheduled in-class survey completion time missingness for this age group is expected to be high (Clary et al., 2021). It is likely that the global COVID-19 pandemic placed extreme pressure on school scheduling, which may explain some of the missing data. Nonetheless, reduced odds of early onset cannabis use and recent ecstasy use, and increased drug literacy scores were robust against missing data, which supports the validity and reliability of these three findings (Bondarenko and Raghunathan, 2016). Second, the trial relies on self-report measures of substance use, which despite demonstrated validated may lead to underestimations of volume consumption (Williams and Nowatzki, 2005). Finally, the current study presents the short-term results to the trial and continued follow-up will be important to determine whether results are sustained over time, particularly as substance use is expected to increase as students leave school. The current trial includes several strengths, including diverse sampling across New South Wales, including independent and state schools and the novel web-based delivery which upholds high program fidelity.

Conclusion

Delaying the onset and reducing the risky use of substances is an effective way to reduce the significant health costs of substance-related harms. High quality evaluation trials, with consistent replication are required to warrant widespread implementation and although the present study provides compelling results to support the effectiveness of The Illicit Project, high levels of missing data limit the generalisability of the results. Further follow-up is currently underway and results from the 12- and 24-month follow-up occasions will be important in refining and adapting the program to different target groups. In addition, pending further follow-up results, implementation trials will be required to inform the upscale of the program. It is important to continue to develop and test multidimensional prevention strategies for adolescents throughout the course of development and into adulthood.

Declaration of Competing Interest

The authors declare the following financial interests/personal relationships which may be considered as potential competing interests: JD is the developer of The Illicit Project, a program that will be made freely available pending the results of this trial.