Helping adolescents to better support their peers with a mental health problem: A cluster-randomised crossover trial of teen Mental Health First Aid.

BACKGROUND: teen Mental Health First Aid (tMHFA) is a classroom-based training programme for students aged 15-18 years to improve supportive behaviours towards peers, increase mental health literacy and reduce stigma. This research evaluated tMHFA by comparing it to a matched emergency Physical First Aid (PFA) training programme.
METHODS: A cluster-randomised crossover trial matched four public schools in two pairs and then randomised each to first receive tMHFA or PFA for all Year 10 students. In the subsequent calendar year, the new Year 10 cohort received the opposite intervention, giving eight cohorts. Online surveys were administered at baseline and 1 week post-training, measuring quality of first aid intentions, mental health literacy, problem recognition and stigmatising beliefs, towards fictional adolescents with depression and suicidality (John) and social anxiety (Jeanie).
RESULTS: A total of 1942 students were randomised (979 received tMHFA, 948 received PFA), 1605 (84%) analysed for the John vignette at baseline and 1116 (69% of baseline) provided post-training data. The primary outcomes, 'helpful first aid intentions' towards John/Jeanie, showed significant group-by-time interactions with medium effect sizes favouring tMHFA ( ds = 0.50-0.58). Compared to PFA, tMHFA students also reported significantly greater improvements in confidence supporting a peer ( ds = 0.22-0.37) and number of adults rated as helpful ( ds = 0.45-0.46) and greater reductions in stigmatising beliefs ( ds = 0.12-0.40) and 'harmful first aid intentions' towards John/Jeanie ( ds = 0.15-0.41).
CONCLUSIONS: tMHFA is an effective and feasible programme for increasing supportive first aid intentions and mental health literacy in adolescents in the short term. tMHFA could be widely disseminated to positively impact on help seeking for adolescent mental illness.

RCT Entities:   Population Interventions Outcomes

Background

Adolescents have the highest incidence and prevalence of mental illness across the lifespan (Kessler et al., 2005), yet the lowest engagement with treatment services, in comparison to other age groups (Merikangas et al., 2011). Level of social support received, and encouragement to seek out adequate adult help, are important determinants of adolescent help-seeking (Gulliver et al., 2010).

Mental health first aid (MHFA) is defined as the help provided to a person who is developing a mental health problem or experiencing a mental health crisis, until appropriate professional help is received or the crisis resolves (Kitchener et al., 2013). National surveys of Australian youth, aged 12–25 years, found that knowledge of supportive and appropriate first aid strategies was low (Jorm et al., 2007; Yap et al., 2011, 2012a). For example, encouraging a peer with a mental health problem to seek out adult help was not always endorsed by adolescents (Yap et al., 2012a). These findings have been replicated elsewhere (Olsson and Kennedy, 2010) and reveal that while adolescents have a preference for seeking help from peers, they are often ill-equipped to provide effective support when a friend has a mental health problem.

Intentions to provide supportive MHFA actions, and positive beliefs about adult sources of help, have been found to prospectively predict behaviours actually undertaken to support a peer. Participants in a national survey of Australian youth reported on their first aid intentions and beliefs based on one of four vignettes (Yap and Jorm, 2012). In a 2-year follow-up interview, the same participants reported on actions they had taken to help any family member or close friend with a problem similar to the vignette character since the initial interview. The quality score of the first aid intentions at baseline was found to prospectively predict the behaviours they reported using in response to an actual friend or family member with a mental health problem at follow-up. Hence, if adolescents could be trained in the provision of supportive first aid strategies and educated on the important role that adult helpers can play in assisting young people with mental health problems, adolescent peers could become an effective pathway for social support and professional help-seeking for mental illness in adolescence. Indeed, training adolescents in effective MHFA strategies could ultimately lead to early intervention and improved mental health in adolescence (Kelly et al., 2007).

MHFA training of adults began in Australia in 2001 and has spread to many other countries (Jorm and Kitchener, 2011). A recent meta-analysis of 15 separate evaluations of MHFA training (Hadlaczky et al., 2014) found that it is effective in improving knowledge, attitudes and helping behaviour, with a medium effect size on knowledge and small effect sizes on attitudes and behaviour (Hadlaczky et al., 2014). More recently, a version of MHFA training has been developed for adolescents in the senior secondary school years (Hart et al., 2012; Ross et al., 2012). teen Mental Health First Aid (tMHFA) is a three-session classroom-based training programme for students in Years 10–12 (aged 15–18 years), which aims to improve supportive first aid behaviours among peers, increase mental health literacy and decrease stigmatising attitudes.

An uncontrolled pilot trial of tMHFA was previously conducted (Hart et al., 2016) across four schools and 988 students, of whom 61% (n = 603) had parental consent and eligible evaluation data. A total of 520 students (Mage = 15.98 years) completed the baseline survey, 345 the post-test and 241 the 3-month follow-up. Statistically significant improvements in mental health literacy and confidence in providing MHFA to a peer were found over time, as were decreased stigmatising attitudes and reduced psychological distress.

This study reports the first randomised controlled trial of tMHFA, which examined student outcomes compared to an active, matched Physical First Aid (PFA) training programme. It was hypothesised that compared to PFA, students receiving tMHFA would report (1) significantly increased helpful first aid intentions, confidence in supporting a peer with a mental health problem and helpful beliefs about potential sources of help and (2) significantly decreased harmful first aid intentions and stigmatising attitudes.

Methods

The descriptions below follow the CONSORT 2010 statement and extension for cluster-randomised trials (Campbell et al., 2012).

Trial design

A cluster-randomised crossover (CRXO) trial was undertaken, with schools defining clusters. CRXO is a refinement of the matched-pair cluster design; in CRXO trials, all clusters receive all interventions with the sequence of administration being randomly assigned. It is particularly suitable when only a small number of clusters are available as it allows the effects of the intervention to be estimated within clusters, controlling for within-cluster variance (i.e. factors in specific school environments) that may influence the outcome of the study (Parienti and Kuss, 2007). With two interventions, two distinct waves of participants (periods; i.e. student year level cohort) undertake the separate trial conditions (Figure 1; Giraudeau et al., 2008; Turner et al., 2007).

Figure 1.

Design of the CRXO study.

In CRXO, all clusters receive all interventions, with the sequence of administration being randomly assigned. In this study, four schools enrolled in the study, and each of these were matched in a pair, based on Index of Socio-educational Advantage (ICSEA) and size of Year 10 cohort. Pairs of schools were randomised to receive either tMHFA or PFA in the first wave for their entire Year 10 cohort. In the subsequent year, the new Year 10 cohort received the opposite intervention. This design allows counterbalancing across schools, while also controlling for within-school variance, by using intervention crossover in subsequent waves. Individual school characteristics are shown in Table 3.

Table 3.

Participant characteristics by school and intervention.

	tMHFA[a]	PFA[b]	Combined
School 1 (ICSEA = 1031)
Eligible at assignment	159	170	329
Sample at baseline (n)[c]	116	115	231
Age (M, SD)	15.92, 0.52	16.03, 0.52	15.97, 0.52
Gender (% female)	36.21	38.26	37.23
English as first language (% yes)	74.14	74.78	74.46
School 2 (ICSEA = 1050)
Eligible at assignment	230	230	460
Sample at baseline (n)	200	209	409
Age (M, SD)	15.65, 0.39	16.06, 0.40	15.86, 0.45
Gender (% female)	48.00	45.45	46.70
English as first language (% yes)	84.5	87.08	85.82
School 3 (ICSEA = 1091)
Eligible at assignment	300	280	580
Sample at baseline (n)	231	233	464
Age (M, SD)	15.84, 0.42	15.76, 0.44	15.80, 0.43
Gender (% female)	38.53	45.92	42.24
English as first language (% yes)	85.28	83.26	84.27
School 4 (ICSEA 1097)
Eligible at assignment	300	273	573
Sample at baseline (n)	261	240	501
Age (M, SD)	15.89, 0.62	15.90, 0.64	15.89, 0.63
Gender (% female)	49.04	48.75	48.90
English as first language (% yes)	52.11	47.08	49.7
Total sample
Eligible at assignment	989	953	1942
Sample at baseline (n)	808	797	1605
Age (M, SD)	15.82, 0.51	15.92, 0.52	15.87, 0.52**
Gender (% female)	43.94%	45.55%	44.74%
English as first language (% yes)	72.77%	72.15%	72.46%

ICSEA: Index of Community Socio-educational Advantage; SD: standard deviation.

All students in Year 10 received teen Mental Health First Aid training (tMHFA).

All students in Year 10 received Physical First Aid (PFA) training.

All students who completed student assent, age and gender, and at least one item on the John vignette of the baseline survey, were included in primary outcome analyses.

Significant group difference found at the p ⩽ 0.001 level.

This research was registered with the Australian New Zealand Clinical Trials Registry (ACTRN12614000061639) and had ethics approval from a university ethics committee (HREC1341238) and the State Government Education Department for Victoria (2014_002268).

Participants

To be eligible, schools needed to be Government funded (rather than independent/private), agree to two consecutive cohorts of Year 10 students undertaking three survey sessions and three training sessions in regular class time, and agree to withhold any overlapping mental health classroom programmes until the completion of the research. The only exclusion criterion was having provided a mental health intervention designed to increase mental health literacy or help seeking to the current Year 9 or 10 cohorts over the previous 2 years. Students were eligible to participate in the evaluation surveys if they had parental consent and provided assent at the beginning of each survey. Students with a known mental health problem, previous experience of mental illness or suicide bereavement were encouraged to speak to their mental health professional, school counsellor and/or parents before deciding whether to participate. Passive parental consent was used. The research team provided parent, teacher and student information sessions and electronic and hardcopy information statements, 3 weeks prior to baseline measures. Parents could opt their child out of the training or the evaluation by returning a signed form to the participating school. To protect the identity of these families, no data were gathered on non-consenters.

Interventions

Both interventions consisted of three 75-minute classroom sessions, presented by trained instructors external to the host school, according to a manualised curriculum. In each intervention, students were provided with a specific programme booklet and completion certificate. Session content is provided in Table 1. Training was normally completed within 3 weeks (one session per week).

Table 1.

Structure and content of training interventions.

	tMHFA	PFA
Session 1: 75 minutes	Mental health problems What is mental health?What are mental health problems?Types of mental health problemsImpact on young peopleStigmaAppropriate help	DRSABCD – action plan Where to startWhen and how to call 000How to manage a conscious personHow to manage an unconscious personWhen and how to do cardio pulmonary resuscitation (CPR)When and how to use an automated external defibrillator (AED)Demonstration on a mannequin for CPR and AED
Session 2: 75 minutes	Helping a friend in a mental health crisis What is mental health first aid?What is a mental health crisis?Using the teen MHFA action plan to help a friend in crisisRecovery position	Basic first aid for SprainsStrainsWound careFractures and dislocationsConcussionAsthma
Session 3: 75 minutes	Helping a friend who is developing a mental health problem Importance of acting earlyUsing the teen MHFA action plan to help a friend developing a mental health problemHelpful links and resources	Basic first aid for AnaphylaxisPoisonsExposure to heatExposure to coldDiabetesSeizure
Action plan	Look – Look for warning signsAsk – Ask how they areListen – Listen upHelp – Help them connect with an adultYour Friend – Your friendship is important	D – DangerR – ResponseS – Send for help/Call 000A – Open AirwayB – Check for BreathingC – Start CPRD – Attach Defibrillator (AED)

MHFA: Mental Health First Aid training; PFA: Physical First Aid.

The central teaching of both MHFA and PFA is an action plan. Modelled on MHFA for adults (Kitchener et al., 2010; Kelly et al., 2010) and based on the key messages for adolescents from a Delphi expert consensus study (Ross et al., 2012), the tMHFA action plan provides five strategies taught in a mnemonic. The PFA action plan is based on the CPR recommendations (Eisenburger and Safar, 1999; Hazinski et al., 2005).

tMHFA

A detailed explanation of the tMHFA programme has been provided elsewhere (Hart et al., 2016), but relevant details are noted here. Training involved a PowerPoint presentation, videos, role-plays, group discussion, small group and workbook activities (Hart et al., 2012). tMHFA instructors completed at least 5.5 days of training.

Because a core message of the tMHFA training is to seek assistance from a trusted and reliable adult when a peer is experiencing a mental health problem, the Youth MHFA course (Kelly et al., 2017) was also offered to staff and parents at participating schools. One course was provided for teachers and another for parents. The course is a 14-hour programme for adults living or working with young people, analogous to basic PFA training for adults. It ensured that adults who were likely to be called upon to assist adolescents as a result of tMHFA were confident in providing support and could facilitate appropriate referral pathways to effective treatment interventions. A previous randomised controlled trial (RCT) has evaluated the effect of teacher training on students (Jorm et al., 2010). A parallel PFA course for parents and teachers was not provided for control condition cohorts, as the level of attendance at prior training was already high.

PFA

PFA training involved instructor introductions of topic content, role-plays using mannequins, bandages and splints and group discussions. PFA instructors underwent a minimum of 3 days training in first aid, plus an additional certificate-level course in Workplace Training and Assessment (8-week full-time equivalent; NRVET, 2017).

Outcome measures

The survey, which was modified from previous national mental health literacy surveys with youth (Yap et al., 2012b) and evaluations of tMHFA (Hart et al., 2016), measured quality of MHFA intentions (helpful and harmful intentions, confidence in providing help), mental health literacy (problem recognition, beliefs about helpfulness of adult sources of help) and stigmatising beliefs (social distance, weak-not-sick, dangerous/unpredictable and would not tell anyone). It presented two vignettes: one (John) depicting an adolescent with suicidal ideation and symptoms matching Diagnostic and Statistical Manual of Mental Disorders, Fifth Edition (DSM-5) and International Statistical Classification of Diseases and Related Health Problems–10th Revision (ICD-10) criteria for a depressive disorder (American Psychiatric Association, 2013; World Health Organization, 1994) and another (Jeanie) with symptoms matching criteria for social anxiety/phobia. The vignettes are provided in Additional File 1. A description of all measures, including reliability coefficient omega (ω), is provided in Table 2. Omega is considered a better indication of reliability than Cronbach’s alpha (Dunn et al., 2014), especially where response options to individual items involve binary responses or scales are brief (McNeish, 2017).

Table 2.

Measures used at baseline and post-training to assess student outcomes.

Outcome	Survey measure	Response	Example	Scores range and reliability statistics
Quality of MHFA intentions	If John/Jeanie was a friend I would …	Choose any of 13 possibilities:6 consistent with Action Plan6 discordant with Action PlanOther (please specify)	Helpful – Tell John I have noticed something seems wrong, and I want to make sure he is okay. Harmful – Ignore Jeanie because she is being attention-seeking	Helpful subscale, total score on6 items: 0 to 6John ω = 0.84Jeanie ω = 0.85Harmful subscale, total score on 6 items 0 to 6John ω = 0.76Jeanie ω = 0.78
Confidence	If John/Jeanie was a friend, how confident would you feel in helping him or her	5-point Likert-type scale1 = Not at all5 = Extremely		Test–retest for PFA students (approximately 4 weeks apart)John r = 0.52Jeanie r = 0.48
Problem recognition	What, if anything, do you think is wrong with John/Jeanie?	Open ended (blind scored according to a-priori criteria)	John: Depression/depressed/suicidalJeanie: Social anxiety/social phobia/anxiety/anxious/anxiety disorder	Scored for the presence of any of the acceptable labels: 0–1Test–retest for PFA students (approximately 4 weeks apart)John r = 0.47Jeanie r = 0.69
Beliefs about helpfulness of adult help	Which of the following people do you think would be helpful, harmful or neither for John/Jeanie’s problem?	Helpful, neither, harmful	Counsellor General practitioner (GP) or family doctor psychologist Teacher School counsellor/welfare Minister or priest	Total number of adults endorsed as helpful: 0 to 6Test–retest for PFA students (approximately 4 weeks apart)John r = 0.55Jeanie r = 0.58
Social distance	Would you be happy to	4-point Likert-type scale1 = yes definitely4 = definitely not	Develop a close friendship with John/Jeanie Go out with John/Jeanie on the weekend	Total score on 5 items: 5 to 20John ω = 0.91Jeanie ω = 0.94
Weak-not-sick	Please indicate how strongly you personally agree or disagree with each statement	5-point Likert-type scale1 = Strongly disagree5 = Strongly agree	A problem like John/Jeanie’s is a sign of personal weakness. John/Jeanie’s problem is a real medical illness.People with a problem like John/Jeanie’s could snap out of it if they wanted.It is best to avoid people with a problem like John/Jeanie’s so you don’t develop this problem.	Mean score of 4 itemsJohn ω = 0.77Jeanie ω = 0.78
Dangerous/unpredictable	Please indicate how strongly you personally agree or disagree with each statement	5-point Likert-type scale1 = Strongly disagree5 = Strongly agree	People with a problem like John’s/Jeanie’s are dangerous. People with a problem like John/Jeanie’s are unpredictable.It is best to avoid people with a problem like John/Jeanie’s so you don’t develop this problem.	Mean score of 3 itemsJohn ω = 0.59Jeanie ω = 0.77
Would not tell anyone	Please indicate how strongly you personally agree or disagree with each statement	5-point Likert-type scale1 = Strongly disagree5 = Strongly agree	If I had a problem like John’s/Jeanie’s I would not tell anyone.	Test–retest for PFA studentsJohn r = 0.54Jeanie r = 0.50

tMHFA: teen Mental Health First Aid; PFA: Physical First Aid.

ω: Revelle’s omega total for total scores and subscales. Omega is considered acceptable when above 0.70.

r: calculated based on the control condition measures taken at baseline and post-training which occurred approximately 4 weeks apart. Given that an intervention was provided in between the measurement occasions, this may have led to lower reliability estimates than would be reported by a training-naive sample. Other test–retest reliability data on these instruments have not been previously developed, as this was the first implementation of the John and Jeanie vignettes in a tMHFA-naive sample.

The primary outcome was quality of MHFA intentions. Participants were asked to endorse any of 12 possible actions towards John/Jeanie. The possible actions were designed to be consistent with the tMHFA action plan (helpful intentions) or contrary to the plan as distractors (harmful intentions; see Table 2 for examples). This measure was chosen because research based on the Theory of Planned Behaviour shows that intention ratings correlate strongly with future behaviours (Armitage and Conner, 2001) and because the quality of young people’s MHFA intentions has been shown to predict subsequent helping actions 2 years later (Yap and Jorm, 2012).

Problem recognition was assessed by asking students to identify what, if anything, was wrong with the person in the vignette. Responses were open-ended, and resulting data were coded by a researcher blind to period, cluster and cluster–period and in accordance with a structured protocol (Mason et al., 2015). Previous evaluations of MHFA training have found statistically significant increases in correct problem recognition (Hadlaczky et al., 2014). However, because of its brevity and specificity to adolescents, the tMHFA training does not focus on teaching correct labelling across different diagnoses of mental disorder. Only about 10 minutes of the entire 225-minute content is spent on explaining differences between broad psychiatric groups, such as mood versus psychotic illnesses (see Table 1; Hart et al., 2012). Instead, students are taught to assess for general signs that a person might be developing a mental health problem which is interfering with the usual activities. As such, no hypotheses about changes in students’ rates of correct problem recognition were made.

Beliefs about the helpfulness of adult help were assessed by asking participants to rate a range of individuals who have a clear role in protecting and promoting student well-being. Although examining beliefs about the helpfulness of a broad range of self-help and social support sources, has been a feature of other evaluations of mental health literacy (Reavley et al., 2011; Yap et al., 2010, 2012b, 2015), this study only analysed data on adult sources of help, because the necessity of engaging adult help is a key message of the tMHFA training.

Measures of stigmatising beliefs included the Social Distance Scale (SDS; Link et al., 1999) and the Depression Stigma Scale (DSS; Griffiths et al., 2004), modified for use with the John/Jeanie vignettes (Hart et al., 2016) and evaluated as four distinct stigma scales: ‘social distance’, ‘weak-not-sick’, ‘dangerous/unpredictable’ and ‘would not tell anyone’ (Yap et al., 2014).

Additional measures of self-reported mental health status, help-seeking, psychological distress (K6) and experiences with provision or receipt of first aid were administered at 12-month follow-up, but are not reported here as they were not administered at post-training. These measures assessed a time period of 1–12 months and thus may have overlapped with baseline measures if given earlier.

Statistical analysis

Mixed-effects models were used for analysing continuous and binary outcome variables, including group-by-measurement-occasion interactions, taking into account the hierarchical structure (i.e. the correlation of measurement occasions within clusters, periods and period–clusters; MAR; Rabe-Hesketh and Skrondal, 2012). All models included a random effect for student cohort (period–clusters) and fixed effects for school (cluster), year (period) and student age and gender to accommodate possible non-independence of student responses in cohorts. Gender and age were found to be associated with missingness, with older boys being less likely to provide post-training data, and were thus included as fixed effects in order to help meet the missing at random assumption. For outcome measures with no substantial baseline imbalance, effect sizes (Cohen’s d) were calculated by dividing the difference between the two group means at post-intervention by their pooled standard deviation (SD). With baseline imbalances, Cohen’s d was calculated by dividing the mean change in each condition by the pooled SD at post-intervention. Analyses were performed in Stata 13 (StataCorp, 2013).

Statistical power

It was conservatively estimated that there would be 100 Year 10 students per school, with a 50% consent + assent rate. Across four schools and eight clusters, this would give 400 adolescents (200 per intervention). The estimated intra-class correlation (ICC) for students (ICC = 0.003) at the school cluster level was based on findings from previous research (Hart et al., 2016) and not included in the power calculations due to the likely small design effect and the counterbalancing of schools. With an assumed 0.70 correlation between pre-post measurements, the study would have a 0.80 power to detect small (d = 0.17) group-by-measurement occasion differences at α = 0.05.

Implementation

The research team emailed a request to all accredited Youth MHFA instructors in the Melbourne area asking for an introduction to any secondary schools with an expressed interest in receiving training for staff and students. Two schools agreed to participate through this mechanism, with a further two schools selected for matched demographic variables: Index of Socio-Educational Advantage (ICSEA; M = 1000, SD = 100, Barnes, 2011) and Year 10 cohort size in the first year. Schools were required to be within 1 SD on ICSEA and within 100 students in size. The four schools were enrolled in the trial upon completion of a research agreement signed by the principal. A random sequence generator (SPSS) was used by the trial manager to generate the treatment sequence for the two pairs of schools, with the research assistant assigning the first school enrolled to sequence one, the second school enrolled to sequence two and the remaining schools assigned based on their matching demographic variables.Although randomisation was at the cluster level, the primary outcome was analysed at the individual level, as the aim of this research was to understand the impact of first aid training on students’ supportive intentions towards their peers, rather than impact on the school community. Research assistants, instructors providing training and students could not be blinded to intervention. However, research staff coding and analysing data were blinded to measurement occasion, school identity and condition.

Research staff completed a recruitment meeting with Principals and Assistant Principals from schools indicating interest in participating in the current research. Once schools had signed the Research Agreement, they were randomised to a sequence of treatment allocation. A planning meeting was then conducted, where the number of students enrolled in Year 10 and the number of classes requiring training were disclosed to the research team. Research staff relied on these verbal estimates from school staff to schedule training and survey sessions. Parent, teacher and student information sessions were provided 3 weeks before the baseline survey and parental opt-out forms could be returned at any time during the trial. The baseline survey was administered 1 week before the first training session. All surveys were completed by students during regular class times via the online hosting software SurveyMonkey. Each school was emailed a generic link to the survey, and school staff distributed this link to students via an intranet webpage or student email address. Upon accessing the survey, students were required to enter their unique identification number, which could only be matched to personally identifying information by the school and not research staff. The first session of tMHFA or PFA training usually occurred in the week following baseline completion and all three sessions were usually completed within 3 weeks. The post-training measure was scheduled in regular class time up to 1 week after the conclusion of the final training session. Students who were absent during scheduled survey sessions were tracked by school staff and provided with alternative opportunities to complete the survey during school time. Research staff were usually present to support the process of implementation for the baseline survey and initial training session across a whole school, though were not typically present in individual classrooms.

All survey and intervention sessions were conducted between April 2014 and October 2016. Although the paid provider of the PFA sessions changed after year 1, no changes to the content presented or research methods occurred after trial commencement.

Results

Participant flow

The flow of students through research stages is shown in Figure 2 (Boutron et al., 2008). At treatment allocation, 1942 students across four schools and 8-year-level cohorts were eligible to participate. Across the entire sample, only four parents opted their child out of the research (99.8% consent rate), 11 students did not provide assent and 8 provided assent but no data, giving a total sample consent rate of 98.8%. At baseline, 1642 students lodged surveys (84% participation rate); 26 of these provided demographic data only, 81 provided incomplete data on outcome measures and 1535 provided a complete response. The remaining 295 students who were eligible to participate but did not return baseline data were assumed to have been absent from the scheduled survey session during class time.

Figure 2.

Participant flow diagram.

All participant surveys were lodged anonymously online, with a student ID used to match responses over time. Attendance data were not collected from schools. Instead students indicated on their post-training surveys how many training sessions they believed they attended. Data presented here (*) are therefore known underestimates, as some students who did not complete surveys did attend training sessions.

For the intervention training sessions, attendance data were not collected from schools, as the research team tried to remain independent of programme implementation and gathered no identifying data from students. However, in the post-training survey, students were asked how many sessions they had attended, which gave an underestimation of attendance, as students who opted-out of the surveys may still have attended. From these data, it was estimated that the proportion reporting attendance at all three sessions was 46.87% (n = 765), and at least one session was 64.95% (n = 1060).

The post-training survey was lodged by 1116 students (67% of baseline): 50 provided incomplete data and 1066 complete responses. The main reason for attrition at post-training was assumed to be student non-attendance at the class scheduled for survey completion, as only a minority did not provide assent. Predictors of attrition were investigated with multiple logistic regression, adjusting for school-year clustering. Significant predictors were intervention group (tMHFA odds ratio [OR]: 1.36 [p = 0.008], 95% confidence interval [CI]: [1.08, 1.71]), gender (female OR: 0.75 [p = 0.015], 95% CI: [0.59, 0.94]) and age (OR: 1.54 [p < 0.001], 95% CI: [1.23, 1.93]). Attrition rates also varied substantially between schools. School 4 had an attrition rate at 24% of eligible students and school 2 had 27% attrition, but school 1 was significantly higher than school 4 (51% attrition; OR 2.59 [p < 0.001], 95% CI: [1.67, 4.01]) and school 3 had higher again (52% attrition, OR: 3.35 [p < 0.001], 95% CI: [2.46, 4.58]). Across intervention years, however, attrition rates were not significantly different, suggesting a school rather than cohort effect. English language (p = 0.524) was not associated with attrition.

Three of the four schools provided at least one Youth MHFA course for parents or teachers. The fourth school had previously engaged an independent instructor to provide training for staff and hence did not feel the need to undertake more.

Sample characteristics

Student characteristics by school and intervention group were well-balanced at baseline (see Table 3). Students were aged between 14 and 18.92 years, 55% were male and 28% reported that English was not their first language. Mean scores at baseline, on measures of mental health literacy and stigma, were similar to those reported in the previous National Survey of Youth Mental Health Literacy and Stigma (Reavley and Jorm, 2011).

Table 4 outlines the results for the primary outcome measures. Quality of first aid intentions showed significant group-by-time interactions, with students receiving tMHFA reporting greater increases in ‘helpful intentions’ towards John and Jeanie at post-training, than students receiving PFA. Students receiving tMHFA were also less likely to report ‘harmful intentions’ towards John and Jeanie than students receiving PFA. Effect sizes were medium and more pronounced for helping John than Jeanie. Students receiving tMHFA reported significantly higher levels of confidence in helping John and Jeanie after receiving their training, compared to levels reported at baseline. A significant group-by-time interaction was found, implying that improvements for tMHFA students were significantly greater than those reported for students receiving PFA, whose confidence was reduced at post-training.

Table 4.

Estimated marginal means and mean difference or proportions and odds ratio over time between Physical First Aid (PFA) and teen Mental Health First Aid (tMHFA).

Variable	Baseline				Post-training				M diff	95% CI	p	d [a]	ICC[b]
	PFA		tMHFA		PFA		tMHFA
John	M	SE	M	SE	M	SE	M	SE
Helpful intentions	3.78	0.05	3.78	0.05	3.65	0.06	4.61	0.06	0.95	[0.78, 1.13]	<0.001	0.58	0.000
Harmful intentions	1.08	0.08	1.04	0.08	1.14	0.08	0.77	0.08	−0.33	[−0.44 to −0.21]	<0.001	0.41	0.020
Confidence helping	3.63	0.04	3.54	0.04	3.53	0.04	3.78	0.04	0.34	[0.23 to 0.45]	<0.001	0.37	0.001
Adults rated as helpful[c]	3.46	0.09	3.46	0.10	3.23	0.09	4.01	0.10	0.78	[0.59 to 0.96]	<0.001	0.46	0.006
Stigma social distance[d]	1.97	0.04	1.96	0.04	1.95	0.04	1.80	0.04	−0.14	[−0.21 to −0.08]	<0.001	0.20	0.010
Stigma weak-not-sick	2.07	0.07	1.99	0.07	2.11	0.08	1.81	0.08	−0.20	[−0.28 to −0.13]	<0.001	0.40	0.028
Stigma dangerous/unpredictable	2.49	0.05	2.40	0.05	2.46	0.04	2.17	0.06	−0.19	[−0.28 to −0.11]	<0.001	0.40	0.015
Stigma would not tell anyone	2.52	0.05	2.43	0.05	2.52	0.05	2.17	0.05	−0.26	[−0.38 to −0.14]	<0.001	0.37	0.003
	%		%		%		%		OR
Correct recognition[e]	79.0		74.0		79.8		76.8		1.18	[0.71 to 1.96]	0.528		0.000
Jeanie
Helpful intentions	3.22	0.06	3.32	0.06	3.32	0.07	4.18	0.07	0.75	[0.57 to 0.93]	<0.001	0.50	0.001
Harmful intentions[f]	0.89	0.12	0.92	0.12	0.94	0.12	0.86	0.12	−0.11	[−0.23 to 0.01]	<0.001	0.15	0.046
Confidence helping	3.69	0.08	3.70	0.08	3.64	0.09	3.90	0.09	0.26	[0.14 to 0.37]	<0.001	0.22	0.023
Adults rated as helpful[g]	2.78	0.09	2.71	0.09	2.81	0.10	3.67	0.10	0.93	[0.71 to 1.14]	<0.001	0.45	0.004
Stigma social distance4	1.77	0.02	1.70	0.02	1.77	0.03	1.64	0.03	−0.07	[−0.14 to 0.00]	0.056	0.11	0.000
Stigma weak-not-sick	2.23	0.10	2.17	0.10	2.17	0.10	1.90	0.10	−0.20	[−0.28 to −0.12]	<0.001	0.38	0.047
Stigma dangerous/unpredictable	1.99	0.05	1.87	0.05	2.04	0.05	1.82	0.05	−0.09	[−0.18 to −0.01]	0.034	0.13	0.008
Stigma would not tell anyone	2.45	0.04	2.38	0.04	2.46	0.05	2.09	0.05	−0.30	[−0.42 to −0.17]	<0.001	0.36	0.001
	%		%		%		%		OR
Correct recognition6	47.2		45.0		49.7		58.2		3.34	[1.88 to 5.94]	<0.001		0.000

OR: odds ratio; GP: general practitioner; SE: standard error; PFA: Physical First Aid; tMHFA: teen Mental Health First Aid; CI: confidence interval.

Large effect size Cohen’s d = 0.8, medium effect size d = 0.5 and small effect size d = 0.2 (Fritz et al., 2012).

Period–cluster intra-class correlation coefficients (ICC) indicate the proportion of variability in the outcome attributable across clusters.

Scored when the following adults were rated as ‘Helpful’ (as opposed to ‘neither’ or ‘harmful’): counsellor, GP or family doctor, minister or priest, psychologist, school welfare coordinator/school counsellor and teacher (0–6).

For all stigma items, a reduction in scores was a positive outcome, indicating lower stigmatising beliefs.

To be considered correct, responses needed to mention either ‘depression/depressed’ or ‘suicide/suicidal thoughts’.

This variable was transformed due to non-normal distribution of model residuals. Estimated Marginal Means from untransformed data provided for ease of interpretation but p value based on variable transformation.

To be considered correct, responses needed to mention any one of ‘social anxiety’, ‘social phobia’, ‘anxiety/anxious’ or ‘anxiety disorder’.

Correct recognition of John’s problem as depression or suicidality was high at baseline, with 74% and 79% of the students receiving tMHFA and PFA using the correct labels before their respective training. This proportion did not increase after the training and there was no statistically significant group-by-time interaction. However, recognition of Jeanie’s problem as social phobia/anxiety disorder did improve for students receiving tMHFA, who were three times more likely to report the correct label after their training than students receiving PFA.

tMHFA training significantly improved beliefs about help-seeking compared with PFA, as more adults were rated as ‘helpful’ for John and Jeanie after training. The size of these effects was medium and similar for John and Jeanie, although the size of the effect for number of sources of help for John was slightly higher than for Jeanie at baseline.

Stigmatising attitudes towards John significantly decreased in students receiving tMHFA, but not in students receiving PFA. However, these reductions were not apparent on all scales measuring attitudes towards Jeanie. In particular, the ‘social distance’ and ‘dangerous/unpredictable’ scales for Jeanie showed very little change over time for both tMHFA and PFA students, although they were reasonably low at baseline. For the two stigma scales measuring attitudes towards Jeanie that did show improvements over time (‘weak-not-sick’ and ‘would not tell anyone’), the average reported levels at baseline were higher for Jeanie than for John, suggesting that students had more stigmatising attitudes towards individuals with an anxiety disorder, or females with mental illness, prior to receiving training and therefore room for improvement.

Discussion

A cluster-randomised crossover trial was carried out to establish the efficacy of tMHFA training as a school-based universal mental health intervention to increase student supportive behaviours towards peers, mental health literacy and beliefs about help. Given that adolescents’ first aid intentions have been found to prospectively predict first aid behaviours towards peers with a mental health problem, the findings of this study look particularly promising and are the first to demonstrate the impact of mental health (vs physical) first aid training on the intentions, confidence and stigmatising attitudes of adolescents.

The primary teaching of both interventions was a first aid action plan designed to help students remember key steps when assisting a peer. In the tMHFA programme, students were taught to Look, Ask, Listen, Help, Your Friend – a mnemonic designed to encourage assessment of risk for suicide, other at-risk states such as extreme intoxication, to be aware of developing mental health problems, and also to encourage listening non-judgmentally, seeking help from appropriate adults, and positive behaviours such as engaging in self-help strategies (like exercise or doing social activities) with the peer. Students who received tMHFA training were significantly more likely to report helpful first aid intentions when faced with someone like John – a character experiencing depression with suicidal thoughts – and significantly less likely to report potentially harmful strategies such as ignoring the person, doing nothing or suggesting the friendship might end. The statistically significant changes in behavioural intentions (which showed medium-sized effects), across the tMHFA and PFA groups, suggest that it is not just the teaching of a first aid action plan that improves supportive first aid intentions towards individuals with mental health problems, but the specificity of that action plan to the mental health context. These are very encouraging findings and point to the effectiveness of training adolescents to better support their peers with a mental health problem.

Level of confidence in helping John or Jeanie also bore out the importance of matching first aid teaching to the helping context. Students receiving PFA reported feeling slightly less confident in helping John or Jeanie after receiving their training, whereas students receiving tMHFA reported feeling much more confident. Bystander studies suggest that concerns about not knowing what to do are a significant barrier to providing support (Shotland and Heinold, 1985). The benefit of teaching a simple yet specific mental health action plan may be that students feel assured that they know how to intervene without doing harm, and this in turn increases confidence in at least giving some help rather than none.

At baseline, recognition of John’s problem was quite high: 79% and 74% of the students receiving PFA and tMHFA, respectively, correctly labelled the vignette as depression, suicidal thinking or depression with suicidal thoughts. Correct recognition of Jeanie’s problem as social anxiety at baseline was much lower (47% PFA and 45% tMHFA). Correct recognition was the only outcome for John that did not show a significant group-by-time interaction, though this is perhaps not surprising, given that three-quarters of students were already able to correctly label the vignette before training. What is surprising is that students receiving tMHFA were significantly more likely to report Jeanie’s problem correctly after training, compared to students receiving PFA. The tMHFA curriculum does not focus on diagnostic labels. Instead, a mental health problem is defined as when there is a change to a person’s thoughts, feelings or behaviours, which interrupts their daily life and does not go away quickly – and this definition is articulated repeatedly throughout the course. It may be that using this definition while analysing the Jeanie vignette allowed students receiving the tMHFA programme to more accurately identify a mental health problem and use an appropriate label.

Stigmatising attitudes reported at baseline were quite low. As has been reported previously (Reavley and Jorm, 2011), attitudes towards social anxiety were slightly less negative than towards depression. No group-by-time interaction was found on beliefs about Jeanie as ‘dangerous or unpredictable’ or on desired social distance, which is likely to be because these were particularly low at baseline. Previous research has found that young people reporting greater desire for social distance have decreased odds of assessing for suicide risk and making a doctor’s appointment on behalf of a person they are assisting with a mental health problem (Yap and Jorm, 2011). It is therefore encouraging that the tMHFA course was associated with significant reductions in students’ desired social distance from John and significant increases in the number of adults rated as helpful, as these likely increase the odds of students providing effective support to depressed or suicidal peers in the future.

Strengths, limitations and implications

Attrition from baseline to post-training was high, with approximately 31% of students counted at baseline not returning data at the second measurement occasion. Other similar school-based research has reported 16–23% attrition (Perry et al., 2014; Vogl et al., 2009; Wasserman et al., 2015), suggesting it was higher than expected in this study. Two factors likely contributed. First, it is possible that students at this particular age and stage (Year 10, Mage = 15.8 years) found the survey participation uninteresting. The online measures required reading of text and at times open-ended responses, both features known to be disliked by adolescents (Chow et al., 2012; de Bruin and Fischoff, 2000). This study did not include incentives to participate, as this is against the guidelines for research approved by the Victorian Government State Education Department. It may be that younger students engaged in the similar trials cited above are more compliant or committed to the research process or that incentives become more important for students in the 10th grade.

Second, the particular schools sampled appear to have contributed to a higher than expected attrition rate due to management and communication difficulties unique to their institution. It is clear that two of the participating schools met and exceeded participation rates, while the other two had much higher odds of non-completion. The particular demographic characteristics of those two schools (ICSEA, size, location) do not appear to explain higher attrition; however, anecdotally, these were the schools that the research team found communicating with more difficult throughout the trial; staff were less responsive to requests for information or offers of support, and engagement in student follow-up was minimal, perhaps due to staff workloads already being exceptionally high. Although attrition in school-based research is consistently noted as a barrier to effective trials (e.g. Boys et al., 2003; Flay and Collins, 2005), there are methods for overcoming it. In particular, enhanced participant tracking, such as individual phone calls, email and text/SMS messages, have been found effective in retaining adolescents (Meyers et al., 2003). However, given that our ethics review approval required collection of anonymous data, traceable only by student ID and re-identifiable by the host school alone, enhanced tracking methods were not possible in this study. Thus, it was dependent upon staff at individual schools to follow-up with students and this capacity varied among the schools sampled. Although attrition was higher than expected, the conservative methods used for estimating missing data (Rabe-Hesketh and Skrondal, 2012), and the previous power analyses showing small effects were still detectable, suggest that we can remain confident in the findings presented here.

Another limitation of this research includes the lack of observational or self-reported data on actual first aid experiences subsequent to receiving training. Although first aid intentions have been found to prospectively predict supportive first aid behaviours (Yap and Jorm, 2012), the effectiveness of the tMHFA programme must be further tested to ascertain whether the improvements in knowledge, confidence and intentions reported here translate into more supportive peer interactions, increase in help-seeking or effective interventions during mental health crises. Longer follow-up of students in this sample is planned.

Finally, the generalisability of this research is likely restricted to Australian adolescents residing in metropolitan areas. Further evaluation is required to understand whether schools with lower ICSEA scores (and thereby lower educational outcomes) and those located in regional areas show the same results. A larger RCT with longer follow-up (18 m), sampling of schools with low ICSEA and in regional areas, along with process evaluation (i.e. measures of fidelity), is currently underway (ACTRN12614000061639).

Strengths of this study include a large sample, a robust crossover trial design and advanced, conservative data analytic techniques. Furthermore, this is the first evaluation of a MHFA programme using a matched PFA training course, which allows parsing out of training effects in general versus those of the specific MHFA content. This study suggests that adolescents can be effectively trained in the provision of first aid strategies for supporting a peer with a mental health problem or crisis, and their attitudes towards adult and professional help improved. Importantly, the MHFA business model allows fast and efficient research translation into effective population-based training interventions (Gillinson et al., 2010). Since its inception in 2001, the MHFA programme has spread to more than 22 countries outside Australia (Jorm and Kitchener, 2011) and more than 2 million adults have received training worldwide (MHFA, 2017). Within this sustainable dissemination model, the tMHFA programme can be scaled and distributed, not only across Australia where it was created but also internationally to countries already hosting MHFA programmes.

Conclusion

Compared to PFA, tMHFA resulted in significantly improved supportive first aid intentions and mental health literacy and significantly decreased stigmatising attitudes among adolescents. Future research should assess the impact of tMHFA on first aid behaviours subsequent to receiving training, to test for any effects of these improvements on actual supportive behaviours. Given the success of the MHFA programme in disseminating training programmes globally, tMHFA could be widely disseminated to positively impact on adolescent mental health.