A cluster randomised effectiveness-implementation trial of an intervention to increase the adoption of PAX Good Behaviour Game, a mental health prevention program, in Australian primary schools: Study protocol.

Introduction: Implementation of evidence-based programs in school settings can be challenging, undermining the benefits these programs deliver for children. The primary aim of this study is to assess whether an enhanced implementation intervention increases adoption of the PAX Good Behaviour Game in New South Wales (NSW) primary schools in Australia. A secondary outcome is to investigate the impact of the PAX Good Behaviour Game on children's mental health in the Australian context. Methods and analysis: The study uses a cluster randomised hybrid type 3 effectiveness-implementation design and will involve 40 NSW primary schools. Randomisation will occur at the school level. All NSW primary schools trained in the PAX Good Behaviour Game are eligible for participation. The intervention is a multicomponent implementation strategy that has been iteratively co-designed by our research team and local stakeholders. Intervention schools will have access to eight implementation support strategies in addition to the training received as usual delivery to build knowledge and skills. Research staff will assess implementation and effectiveness outcomes using self-report online surveys with teachers and support staff at baseline, 6-weeks, 6-months and 12-months follow up. Semi-structured interviews with teachers and support staff will be used to examine which implementation strategies worked for whom and under what conditions. Discussion: If successful, this study will highlight effective strategies schools or education departments can use internationally to improve their translation of evidence-based programs into routine practices. This will lead to better outcomes for children and young people.

Introduction

Approximately 13.6% of Australian primary school children aged 4–11 years experience behavioural and emotional problems [1]. Without intervention, up to 40% of these children will experience life-long impacts and harm [2,3]. The health and societal costs of childhood mental illness are highly preventable through early intervention programs delivered in schools. Schools provide a useful setting for delivering programs to a wide reach of children as this is where they spend the majority of their time [4].

Evidence suggests that universal school-based mental health prevention programs are effective, potentially because all students are exposed to protective strategies regardless of risk [[5], [6], [7]]. The PAX Good Behaviour Game (PAX GBG) is one such program with demonstrable effectiveness, showing reductions in children's disruptive behaviour, improvements in educational outcomes and preventing substance use, crime and the development of psychiatric disorders later in life in randomised controlled trials (RCTs) across the United States, Europe and Canada [8,9]. Teachers deliver PAX's evidence-based strategies to support students' self-regulation and positive behaviours in the classroom [9]. Whilst PAX GBG is widely available, there are gaps in our understanding of what implementation supports facilitate successful adoption. Successful adoption is defined as the program being integrated into routine school practices.

Some efforts have been made to understand challenges with program implementation in schools. Barriers to program implementation at the teacher-level include time constraints, lack of training, lack of buy-in, and burnout, while at the school-level include lack of administrative support, logistics, staff turnover and financial resources [[10], [11], [12], [13]]. Implementation science is being increasingly used to develop strategies to overcome such barriers [[14], [15], [16], [17]]. Training and coaching has been found to enhance the fidelity of the PAX GBG in a RCT, which in turn was associated with better student outcomes [18]. In broader school-based research other promising implementation strategies include audit and provision of feedback [[19], [20], [21]], executive (administrative) support [15,[22], [23], [24]] and the use of school ‘program’ champions, when used in combination with other strategies [19,[25], [26], [27]].

To date, few studies have tested implementation strategies in schools using randomised (experimental) designs, with much of the evidence so far of what works based on pre- and post-test (non-experimental) designs.

The current study

This type 3 hybrid effectiveness-implementation trial of the PAX GBG in Australian primary schools has been designed to address gaps in our understanding of what strategies lead to program adoption (and sustainability in schools).

Aims

Primary aim

Evaluate whether an enhanced (multicomponent) implementation strategy [called PAX Plus] leads to higher rates of PAX GBG program adoption (more than 50% of teachers delivering the program) among primary schools at 6- and 12-months post-baseline compared to primary schools in a usual delivery (non-enhanced) condition.

Secondary aims

Effectiveness outcomes

Establish whether students in the PAX Plus condition report greater improvements in emotional and behavioural problems (EBPs) compared to students at 6-months post-baseline in the usual delivery condition.

Implementation outcomes

Evaluate whether schools in the PAX Plus condition report better overall implementation of the PAX GBG compared to the control condition measured as greater acceptability, fidelity, feasibility, reach, appropriateness, sustainability and normalisation and integration. These will be described in depth on p. 13–15.

Methods

Study design

This SPIRIT-compliant protocol describes the methodology of a hybrid-type 3 (effectiveness-implementation) cluster randomised controlled trial with two parallel arms, to test whether a multicomponent implementation strategy improves program adoption, whilst also confirming that the intervention is having its intended effect [28]. See supplementary file 1 for SPIRIT checklist. All primary schools will receive the PAX GBG and after registering for the trial will be randomised to receive the supported implementation strategy (‘PAX Plus’; intervention arm) or no implementation strategy (‘implementation as usual’; control). Schools will receive their allocated intervention for 12 months after registration, with the registration date varying between schools. A critical realist approach will be taken to exploring which implementation strategies worked for which school staff and under what conditions [29]. Critical realism is an epistemology that assumes knowledge about implementation strategies is gained through understanding how participants make sense of the resources provided to them in a way that leads to change [29].

Randomisation and blinding

Cluster randomisation will occur at the school level to avoid contamination that could occur with teachers implementing different strategies within the same school. School-level randomisation also allows for the exploration of school-level implementation factors. Schools will be randomised with a 1:1 allocation in blocks of four, stratified based on school student size (large: ≥300 enrolments, small: < 300 enrolments) and geographical region (rural/regional, metropolitan). Randomisation will be done using a computer-generated randomisation schedule. The research team will not be blinded to the treatment condition as this is not feasible or appropriate given the need to engage with schools to deliver the implementation intervention. The randomisation outcome will be communicated to intervention schools by the receipt of the implementation toolkit. No explication communication of the randomisation outcome will be made to control schools. The analyst for the quantitative outcomes of the trial will be blinded to condition allocation.

Setting

This study aims to recruit 40 government primary schools from New South Wales (NSW), Australia. Only government (public) schools will be invited to participate in the trial, as the NSW Department of Education is funding the teacher training costs. Trial management will take place at the Black Dog Institute, a translational research institute located in Sydney, Australia that is affiliated with the University of New South Wales.

Participants

Any NSW government (public) primary school (Kindergarten to Year 6; 5–12 years) that has teachers registered to be trained in the PAX GBG program is eligible to participate in this implementation trial. We aim to recruit two participant groups per school to address our primary and secondary aims. These are:

Classroom teachers: all teachers of participating Kindergarten to Year 6 classrooms delivering PAX GBG in their school.

Schoolsupport staff: Any staff who do not meet criteria for the ‘classroom teachers’ category but who provide support for the implementation of the PAX GBG in their school. This includes individuals in the school executive leadership (e.g. school principal, deputy), wellbeing staff (e.g., school psychologist, counsellor), or administrative staff.

Research team roles and responsibilities

The trial steering committee comprises senior investigators located at the Black Dog Institute, University of New South Wales, Deakin University, and Australian National University (MT, RI, PC, JT, PB, AC, AWS, LM). This group meets quarterly, or as needed, and is responsible for guiding the overall trial design, intellectual contribution to the scientific quality and strategy, oversight of trial progress, compliance with good clinical research practice, and dissemination. The day-to-day trial leadership team meets fortnightly (MT, RB, RI, PC, PK, RS) and leads the operational aspects of the trial, including management of teams responsible for recruitment, consent, data management, data privacy and security.

Recruitment and procedure

A flowchart of the study procedures is outlined in Fig. 1. Approximately 40 schools will be recruited to participate in the trial over 15 months, starting Term 2, 2021 (start April) until the end of Term 2, 2022 (June). Recruitment was interrupted by the COVID-19 school closures in Term 3, 2021 (July), however will resume in Term 1, 2022 (February).

Fig. 1

Study flow diagram.

The NSW Department of Education will email a brief expression of interest letter containing trial details to eligible schools. Those who wish to take part will be instructed to sign and return a letter of support via a digital consent form hosted on Qualtrics. Upon receipt of the letter, schools will be accepted into the trial.

After schools have consented, the principal will be sent a pair of URLs specific to teachers and support staff and be responsible for distributing these to the relevant groups approximately one week from baseline. Baseline is defined as the date on which the participant registers for the study.

Interested individuals will click on the survey link and be directed to an online registration portal via the Black Dog Institute's bespoke trial platform (the Research Engine). Here they will read the participant information sheet, provide digital consent, and complete a brief registration that requires them to provide an email address so that they can receive links to relevant surveys. Teachers and support staff will be invited to complete surveys at three timepoints (baseline, then 6- and 12-months post-baseline). If each survey is not completed within four days, they will receive one reminder email. At 6 and 12 month follow up teachers and support staff will be invited to complete a qualitative interview. Purposive sampling will be used to select teachers and support staff from a mix of regional/urban schools of varying size/geographical location. We aim to select four staff from each school, the principal, another member of the leadership team and two teachers. We want participants from both intervention and control schools to understand any organic strategies schools might have developed to support program implementation.

Intervention

The intervention is a multicomponent implementation strategy, PAX Plus, which will be tested against a usual delivery (control) condition. All schools will receive the PAX GBG.

PAX standard (usual delivery)

The standard implementation model will involve schools being offered the strategies described in Table 1.

Table 1

Description of PAX Standard (usual delivery) model.

Strategy	Description
Provide training	• PAXIS experts (US program developers) deliver one 8-h professional development workshop to teachers and support staff. • Experts provide information about PAX GBG, demonstrate delivery using role modelling and provide opportunities for staff to practice and ask questions.
Distribute educational materials	• Teachers will be provided a hard-copy training manual and access to online resources supplied by the program developers to support delivery. Teachers deliver the program to their students. • A short handbook outlining these strategies/resources will be provided to schools so that they are aware of what is being offered and how to access it if they wish.

The standard implementation arm is designed to mimic naturalistic contexts in which schools do or do not develop their own implementation strategies. There will be no formal requirements to utilise implementation strategies. Schools in the standard condition are encouraged to implement the PAX GBG for at least one academic calendar year, with changes in outcomes measured at 6-months and 12-months. Staff are encouraged to contact the research team if they have any questions about the trial, and to contact the NSW Department of Education with queries about PAX GBG including training and funding.

PAX plus (enhanced implementation strategy)

The PAX Plus ‘enhanced implementation strategy’ was iteratively co-designed by the research team, the NSW Department of Education and an advisory group of teachers and principals. This process will be described in detail in a separate paper. Briefly, the research team identified potential strategies through rapid review of the literature on school-based prevention program implementation and consultation with NSW Department of Education officials. These potential strategies were presented to school staff (teachers and principals) in focus group discussions. School staff provided feedback on the content and format of the implementation supports they wanted to use. This feedback informed the co-design of PAX Plus (enhanced implementation strategy). Schools in the PAX Plus condition will have access to all the same resources as in PAX Standard, but will additionally receive access to the eight implementation supports outlined in Table 2. PAX Plus schools will be encouraged by the research team to utilise implementation supports for 6 months, starting when they join the study. Implementation progress will be monitored through monthly calls with the school leadership team.

Table 2

Description of the PAX Plus intervention.

Strategy	Description
Provide training	• PAXIS experts (US program developers) deliver one 8-h professional development workshop to teachers and support staff. • Experts provide information about PAX GBG, demonstrate delivery using role modelling and provide opportunities for staff to practice and ask questions.
Distribute educational materials	• Teachers will be provided a hard-copy training manual and access to online resources supplied by the program developers to support delivery. • A short handbook outlining these strategies/resources will be provided to schools so that they are aware of what is being offered and how to access it if they wish.
Fortnightly e-newsletter	• Providing information about specific PAX GBG components (called kernels) and tips on how to implement them, delivered fortnightly to teachers and support staff for a period of 4-months.
Peer Learning network	• School staff will have access to an online peer learning network, moderated weekly by the Black Dog Institute. • It is a Questions and Answers forum for teachers from different schools to share their experiences with implementation challenges.
Promoting PAX Chats	• PAX Chats are live discussions between teachers and PAXIS in the USA to resolve common challenges with kernel implementation. • The leadership team will be encouraged to send reminders for teachers to participate in these PAX Chats available once or twice a term (depending on demand).
Continuous progress monitoring	• Teachers will complete weekly self-reported surveys to measure the frequency and fidelity in which they implement the PAX GBG in their classrooms. • Findings will guide iterative adaptations to the implementation supports. • Teachers will be provided with a fidelity self-checklist as a reference for monitoring their own progress.
Executive Support	• The researcher will schedule a 5 min phone call with the principal of each school every 4 weeks to check in and gain an understanding of their progress. • Principals will also receive an email every 4 weeks with strategies and tips for maintaining engagement.
School champions	• One teacher or staff member trained in the PAX GBG program will be nominated as a ‘program champion’ • The school champion does not receive formal training from the research team or NSW Department of Education. They might organically receive internal training within their school, which will be examined as part of this study. • Their role will be to promote the innovation and to identify strategies to enhance engagement within their school. • As part of this, the champion may chair weekly meetings with the relevant teachers and staff to share learning, problem solve, and maintain motivation.
Broader recognition system	• The leadership team or school champion will be supported to develop a system that rewards teachers and support staff for their contributions to the program. • They can utilise blank certificates of achievement with NSW Department of Education branding and/or staff tootle boards where they can post positive affirmations about their peers.
Audit and provide feedback	• Schools will be encouraged to develop a system to monitor the outcomes of the PAX GBG and feedback progress to staff as a motivation tool. • The research team provide school staff with recommendations for how they can monitor and provide feedback in light of what has worked for other schools.

Outcomes

Our evaluation design and outcome measures are guided by Proctor et al.'s (2009) conceptual model of implementation research as the evaluation framework for this trial (Fig. 2). This model posits that implementation strategies can enhance the effect that an evidence-based program has on implementation outcomes, which in turn determines its effect on service outcomes and student outcomes [30]. Proctor's model was developed for the healthcare setting, and has been adapted in this trial to suit an educational setting and also to include ‘appropriateness’ as an important implementation outcome as added in Proctor et al., 's 2011 model [31].

Fig. 2

Proctor et al.'s (2009) conceptual model of implementation research adapted for education.

The administration schedule for each of the assessment measures described in the following is presented in Fig. 3.

Fig. 3

SPIRIT figure outlining participant timeline.

Primary outcome measure

Program adoption – bespoke

Adoption will be defined as a school making a formal decision to commit to using the PAX GBG as part of their routine practices. We developed a bespoke self-report measure for adoption which is a binary item asking teachers and support staff whether the PAX GBG program is still being delivered in the classrooms of that school at 6 (primary end point) and 12 months post-baseline. ‘Being delivered’ means the classroom teacher is using at least one PAX behavioural strategy in the classroom on a daily basis. This is similar to the binary measurement used to distinguish schools who did and did not adopt the PAX GBG in a Dutch study [32]. Other trials that have measured adoption as a self-reported binary outcome, rate a program as ‘adopted’ if more than 50% of participants at that level are implementing the program. For example, in a school-wide trial of strategies to enhance a physical activity intervention, the program was ‘adopted’ by the school if four out of seven intended practices were in place [33].

Secondary measures

The secondary measures are assessed via online surveys.

Effectiveness

The following effectiveness measure will be administered to assess the effect of the PAX GBG on mental health outcomes of children in the classrooms.

Strengths and Difficulties Questionnaire [SDQ]

The primary measure for effectiveness will be the global difficulties score on the Teacher-Rated 25-item Strengths and Difficulties Questionnaire (SDQ) [34] for children aged 4–10 years. The SDQ will be used by teachers at baseline and 6-months post-baseline to rate the behavioural and emotional functioning of each student in their class. Teachers enter the SDQ scale data for each student online. The total difficulties score is generated by summing the four sub-scales together (emotional symptoms, conduct problems, hyperactivity/inattention, peer relationships problems) and scores range from 0 to 40. Higher scores indicate more severe emotional and behaviour problems. The SDQ total difficulties score had good internal consistency (Cronbach's α = 0.87) [34].

Process outcomes

The following process measures will be administered to assess how the PAX GBG is being delivered and integrated into regular school practices.

NoMAD

Normalisation and integration of the intervention into routine teaching practice will be measured using the 19-item Normalisation Measure Development questionnaire (NoMAD) [35] completed by school staff at 6-weeks, 6-months and 12-months post-baseline. The NoMAD includes items measuring constructs such as coherence, cognitive participation and collective action. The total normalisation and integration score will be generated by summing the 19 scales, with scores ranging from 5 to 95. This measure has good face validity, construct validity and reliability [36].

Custom fidelity scale

Fidelity will be measured using an 18-item custom-designed scale to capture core components of the intervention, such as use of ‘beat the timer’, ‘PAX Vision’ and other kernels. Total fidelity scores will be generated by summing the 18 items together with scores ranging from 5 to 90. Lower scores indicate greater fidelity. Teachers will complete the custom fidelity scale at 6-weeks, 6-months and 12-months post-baseline.

Implementation outcomes

The following implementation measures will be used to compare the effect of PAX Plus versus usual delivery on the implementation of PAX GBG in schools within this trial.

Acceptability of implementation measure

Acceptability will be measured using the 4-item Acceptability of Implementation measure (AIM) [37], which assesses teachers’ satisfaction with PAX GBG. Total acceptability score is generated by summing the subscales together with scores ranging from 4 to 20. Higher scores indicate better acceptability. AIM has been previously used to assess acceptability of school-based interventions in Australia [38,39]. Teachers will complete the 4-item AIM at 6-weeks, 6-months and 12-months post-baseline.

Intervention appropriateness measure

Appropriateness will be measured using the 4-item Intervention Appropriateness Measure, which assesses the perceived fit of PAX GBG within the school context. Total appropriateness scores will be generated by summing sub-scales with scores ranging from 5 to 20. Higher scores indicate better perceived appropriateness. This scale will be completed by the teachers at 6-weeks, 6-months and 12-months post-baseline [37]. The measure has good construct validity and test-retest reliability [37] and has been used to assess other school-based programs [39].

Feasibility of intervention measure

Feasibility will be measured using the 4-item Feasibility of Intervention measure (FIM) [37] that assesses the perceived workability of PAX GBG in schools. Teachers will complete the FIM measure at 6-weeks, 6-months and 12-months post-baseline. Total feasibility scores will be generated by summing the sub-scales with scores ranging from 5 to 20. Higher scores indicate greater feasibility. FIM has strong construct validity and test-retest reliability [37].

Reach

Reach will be measured using a 5-item bespoke reach survey completed by teachers at 6-weeks, 6-months and 12-months post-baseline. The survey will include questions about the proportion of students within the school exposed to PAX GBG.

Evidence-based practice attitudes scale-15

Teachers’ attitudes towards PAX GBG will be measured using the 15-item Evidence-Based Practice Attitudes Scale (EBPAS) [40] which measures four attitude domains: 1) intuitive appeal of the program, 2) likelihood of adopting the program, 3) openness to new practices, and 4) perceived divergence from usual practice. Teachers will complete the EBPAS at 6-weeks post-baseline. The measure has good construct validity and internal consistency [40].

Implementation leadership scale

The Implementation Leadership Scale [41] will be used to collect data on the leadership team's support for PAX GBG and factors that facilitated or hindered their ability to provide such support, such as knowledgeable leadership. The 12-item scale will be completed by support staff at 6-weeks, 6-months and 12-months post-baseline. Total leadership scores will be generated by summing subscales with scores ranging from 5 to 60. The measure has excellent convergent and discriminant validity and internal consistency reliability [41].

Measurement instrument for determinants of innovation

The barriers and enablers of implementing the PAX GBG at the teacher-level will be measured using an abridged version of the Measurement Instrument for Determinants of Innovation (MIDI) [42]. The 21-item MIDI assesses constructs such as procedural clarity and time available and will be completed by teachers at 6-weeks, 6-months and 12-months post-baseline. Total scores will be generated by summing subscales with scores ranging from 5 to 105. Higher scores indicate more enablers/fewer barriers. This measure has been used in other school-based implementation trials [39].

Barriers and facilitators to implementing survey

An abridged version of the Barriers and Facilitators to Implementing Survey will be used to assess school-level implementation barriers and facilitators such as executive support and scheduling conflicts [43]. The 14-item survey will be completed at 6-weeks, 6-months and 12-months post-baseline. Scales will be divided into barriers and facilitators, and summed separately with scores ranging from 5 to 45. Higher scores indicate more barriers/facilitators. This measure has been used in similar school-based trials [39].

Measures for school staff characteristics

Online self-report surveys will be used to determine the characteristics of the teachers and support staff delivering PAX GBG in schools. Multiple choice questions will be used to determine teacher/support staff's gender, the number of years they have been working at the current school in which they are employed and whether their current role is full-time, part-time or casual. Support staff will be asked to identify their current role out of a dropdown menu of; teacher's aide/support, year advisor/head teacher, guidance/wellbeing officer, school counsellor/psychologist, school principal/deputy principal or ‘Other’. The global positioning system will be used to determine if schools are rural/regional or metropolitan. Publicly accessible online information will be used to determine the school's socioeconomic context.

Qualitative measures

The semi-structured interviews will be directed by a guide informed by Pawson and Tilley's (1997) realist evaluation framework [29]. Semi-structured interviews will be used to assess the causal pathways that explain how implementation strategies bring about outcomes for some school staff in specific contexts. In line with Greenhalgh et al.'s (2017) [44] recommendation, the interviewer will inform the interviewee that the purpose of each implementation strategy is to improve schools' implementation of the PAX GBG. Participants will then be asked to identify what strategies worked to achieve this aim. For each identified strategy, the participant will be asked to reflect on the four causal propositions that explain how the strategy brought about its intended change: Context (C), Intervention (I), Mechanisms (M), and Outcomes (O), abbreviated to CIMO [45,46] (see Fig. 4).

Fig. 4

Realist evaluation framework (adapted from Pawson and Tilley 1997 and Eastwood et al., 2020).

The interviews will be conducted face-to-face or online (interviewee preference), take approximately 45 min to complete and will be audio recorded. All interviews will be transcribed for analysis.

Sample size

The sample size is powered based on the primary trial outcome of program adoption. Assuming that at 6-months of implementation 78% of the enhanced implementation schools and 40% of the control schools are still delivering the program [47], with 80% power and an alpha of 0.05, we would need 36 schools (18 in each condition). To detect moderate secondary implementation outcomes of d = 0.4, assuming an ICC of 0.05 and an average of 4 teachers/staff per cluster, 46 schools will be needed (23 in each condition). Assuming that four staff per school register for this trial, this will yield data for 184 teachers and/or staff on the primary implementation outcome of adoption (92 in each condition). Allowing for up to 30% attrition between baseline and 6-months, we need to recruit 240 teachers/staff (n = 120 in each condition) A sample of this size also provides appropriate statistical power to conduct robust secondary analyses related to establishing the effectiveness of the program on students’ emotional and behavioural problems. Collecting data for 32 students in each of the 46 schools (equivalent of 2 average classes) will provide 80% power to detect effects of d = 0.25 (effect size based on our pilot [48]) in student-level effectiveness outcomes (SDQ scores) between enhanced and comparison schools with an alpha of 0.05. Again, allowing for up to 30% attrition, we will aim to collect data on 1913 students.

We will also undertake qualitative interviews with 20 teachers and 10 support staff (combined total of the 6- and 12-month time points) which is sufficient to identify key themes, in light of similar research where sample sizes for key informant interviews have varied from 14 to 40 school staff [[49], [50], [51], [52], [53], [54], [55]].

Data collection and management

All research data collected in this trial will be stored using a unique participant ID code. A list of identifiable participant information associated with each ID code will be stored separately from the research data.

Coded survey outcome data will be stored securely on the Black Dog Institute online research platform. Data sets will then be directly exported from the research platform into Microsoft Excel following assessments so that they can be checked for data quality and accuracy. After checking, data will be exported into appropriate statistical software for analysis. The data manager will be responsible for extracting and securely transferring data to the research team. Only researchers whose analyses require access to the specific dataset collected from each survey will be able to access those data.

Analysis

Quantitative implementation data

The primary adoption outcome (binary) will be compared between conditions using Fisher's exact test. Changes in secondary implementation outcomes of PAX Plus will be analysed using linear mixed model repeated measures ANOVA, with fixed effects of condition, time and their interaction, and a random effect of school to account for clustering [56]. The primary endpoint for the trial is 6 months. Descriptive analyses will be used to provide an overview of the implementation factor scales across the collective study sample; linear regression models will be used to examine associations between the range of implementation scales and SDQ scores. Implementation factor scales include bespoke scales for demographics, reach and fidelity and standardised scales including NoMAD, the intervention appropriateness measure, acceptability of intervention measure, feasibility of intervention measure, evidence-based practice attitudes scale, measurement instrument for determinants of innovation and barriers and facilitators to implementing survey. T-tests will be used to examine associations between implementation factor scales and adoption. Alpha will be set at 0.05 for all analyses.

Qualitative data

Thematic analysis will be undertaken to identify, interpret, and report on patterns of meaning in the data, using Braun and Clarke's six phase model [57,58]. Transcriptions will be separated based on participant group. First, two researchers will familiarise themselves with the data by reading and re-reading transcripts. Second, they will review the same two transcripts and generate initial codes. Third, they will deductively identify the critical realist [29] themes included in the semi-structured interview and use inductive thematic analysis to capture non-predetermined themes. Fourth, initial themes will be reviewed and iteratively refined by the wider research team. Fifth, themes will be defined and consolidated into a thematic coding tree. Sixth, findings will be written up such that data is integrated into an analytic narrative, situated within existing literature.

Effectiveness outcomes

Child impact: We will use mixed models repeated measures analyses of variance, with maximum likelihood estimation and an appropriate covariance structure, to evaluate the change in emotional and behavioural problems - assessed by the SDQ - over time (baseline to 6-months and 12-months) in the intervention relative to control condition. Analyses will accord with the intention-to-treat principle and will include a random effect of school to account for clustering.

Ethics and dissemination

This trial has ethical approval from the University of New South Wales Human Research Ethics Committee (HC200759) and NSW Government State Education Research Applications Process (SERAP 2020364). It has been retrospectively registered with the Australian New Zealand Clinical Trials Registry (ACTRN12621001125819).

All trial findings will be presented at the aggregate level. Regular trial updates and findings will be communicated to participating schools using lay language summaries for distribution to staff and parents via email, school newsletters and/or school websites. Findings will be provided to the government in policy documents. The results of the trial will be disseminated via peer-reviewed publications in scientific journals and conferences.

Trial status

Participant recruitment and baseline data collection for this trial is underway, having commenced in April 2021 and will continue until April 2022.

Discussion

This protocol describes a novel effectiveness-implementation trial which aims to test whether a multicomponent implementation intervention enhances the adoption of the PAX GBG in Australian primary schools. Few implementation/hybrid trials have been conducted in schools [14,15,17,33] and so there is little existing knowledge about how to best implement psychosocial programs. If successful, this study will highlight effective strategies schools or education departments can use internationally to improve their translation of evidence-based programs into routine practices. In light of the literature, strategies that facilitate this knowledge translation will lead to better outcomes for children and young people [31,59]. More broadly, this protocol provides learning about how to conduct hybrid effectiveness-implementation trials, which are an emerging area of research as evidenced by the numerous hybrid trial protocols published between 2019 and 2021 [[60], [61], [62], [63], [64], [65], [66], [67], [68], [69]].

Strengths and limitations

The strengths of this study include the strong collaboration between researchers and the Department of Education, hybrid type 3 design, co-design of the implementation strategy, inclusion of a usual delivery control condition and use of mixed methods to test for effectiveness together with qualitative insights about what components of the strategies worked for whom under what conditions. Limitations of this study include the lack of quantitative measures testing the individual value of strategy components, lack of direct assessment of mental health outcomes in children and potential extraneous variables introduced by the highly open and complex nature of school systems.

Trial registration

Australian New Zealand Clinical Trials Registry, ACTRN12621001125819. Registered 23 August 2021 (version 1) – Retrospectively registered, https://anzctr.org.au/Trial/Registration/TrialReview.aspx?id=381346&isReview=true.

Funding

This work was supported by the and the Andrew & Geraldine Buxton Foundation. RB is a PhD student supported by funding from the Wellbeing Health & Youth (WH&Y) NHMRC Centre of Research Excellence in Adolescent Health [grant number 1134894]. RI is supported by Senior Research Fellowship [grant number 1136430]. PC is supported by NHMRC Early Career Fellowship [grant number 1158223]. ALC is supported by NHMRC fellowships [grant number 1122544] and [grant number 1173146]. PJB is supported by NHMRC Fellowship [grant number 1158707]. MT is supported by NHMRC Early Career Fellowship [grant number 1138710].

Data statement

All study materials are available from the research team upon request to the lead investigators.

Authors’ contributions

MT conceived the study, secured the funding and led the design of the trial with input from all authors and expert guidance from PJB, JT and ALC. RB drafted the manuscript with assistance from MT, RI and PC. PJB, JT, ALC, AWS and LM reviewed and edited the manuscript. RB, MT, RI and PC have an ongoing role in monitoring the conduct and outcomes of the effectiveness-implementation trial. All named authors contributed substantially to the approved final manuscript.

Ethics approval and consent to participate

This trial has ethical approval from the University of New South Wales Human Research Ethics Committee (HC200759) and NSW Government State Education Research Applications Process Approval (SERAP 2020364). To gain consent, principals will be emailed a letter providing a link to a digital consent form to register their school in the study. Upon registration, principals will be emailed a pair of URLs and encouraged to share this with support staff and teachers. This will provide information about the study as well as a link to provide individual digital consent. School staff will be provided with an email address and phone number to contact the lead investigators at the Black Dog Institute if they have any questions in regards to participation or once registered, if they wish to withdraw from the study.

Consent for publication

Not applicable.

Declaration of competing interest

The authors declare that they have no known competing financial interests or personal relationships that could have appeared to influence the work reported in this paper.