A health-education intervention to improve outcomes for children with emotional and behavioural difficulties: protocol for a pilot cluster randomised controlled trial.

INTRODUCTION: One in seven (14%) children aged 4-17 years old meet criteria for a mental illness over a 12-month period. The majority of these children have difficulty accessing clinical assessment and treatment despite evidence demonstrating the importance of early intervention. Schools are increasingly recognised as universal platforms where children with mental health concerns could be identified and supported. However, educators have limited training or access to clinical support in this area. METHODS AND ANALYSIS: This study is a pilot cluster randomised controlled trial of a co-designed health and education model aiming to improve educator identification and support of children with emotional and behavioural difficulties. Twelve Victorian government primary schools representing a range of socio-educational communities will be recruited from metropolitan and rural regions, with half of the schools being randomly allocated to the intervention. Caregivers and educators of children in grades 1-3 will be invited to participate. The intervention is likely to involved regular case-based discussions and paediatric support. ETHICS AND DISSEMINATION: Informed consent will be obtained from each participating school, educator and caregiver. Participants are informed of their voluntary participation and ability to withdrawal at any time. Participant confidentiality will be maintained and data will be secured on a password protected, restricted access database on the Murdoch Children's Research Institute server. Results will be disseminated via peer-reviewed journals and conference presentations. Schools and caregivers will be provided with a report of the study outcomes and implications at the completion of the study. TRIAL REGISTRATION NUMBER: ACTRN12621000652875. © Author(s) (or their employer(s)) 2022. Re-use permitted under CC BY-NC. No commercial re-use. See rights and permissions. Published by BMJ.

This study evaluates direct collaborations between primary school educators and clinicians that aim to empower educators to identify and support children with emotional and behavioural difficulties.

Given the limited evidence of such models, this study is a pilot and limited to 12 primary schools over a 6-month period to assess educator acceptability and feasibility.

The 12 primary schools represent a broad range of participants as recruitment involves metropolitan and rural settings as well as varying levels of socio-educational advantage.

The intervention is co-designed with schools to ensure the educators’ needs are accounted for in the design.

Introduction

Background and rationale

Childhood is a critical period of development and has lasting impacts on health and well-being throughout the life course.[1]

Emotional and behavioural difficulties are common in children and can be an early sign of mental illness. Over half of adult mental illness begins prior to 14 years of age,2 and one in seven Australian children meet diagnostic criteria for a mental illness over a 12-month period.3 While this national survey included children aged 4–17 years of age, evidence demonstrates that many childhood mental health problems become evident at a much younger age.4 5 Emotional difficulties include problems with anxiety, worrying and being withdrawn, while behavioural difficulties include hyperactivity and aggression.6 Prevention, early identification and intervention with evidence-based practices has been shown to reduce these problems.7 8 Unfortunately many children, especially those living in lower socioeconomic status settings, do not receive adequate support.9 10 This is in part because the individuals supporting children on a daily basis, such as caregivers (those with parental or legal responsibility)11 and educators, may not be able to identify that a child needs help. Many children therefore miss out on support due to a lack of health literacy regarding childhood development and well-being. Health professionals have a great deal of expertise in assessing and managing child mental health problems, but timely access to healthcare may be difficult. Barriers such as siloed services, complex referral pathways, long wait times and high costs mean that many families have difficulty accessing the help they require.12 13 Public health systems are experiencing increased numbers of presentations and referrals of children with emotional and behavioural difficulties, with families experiencing wait times in excess of 12 months to access support.14 This can result in worsening mental health, poor educational outcomes and family dysfunction.3 15 In addition, many children with emotional and behavioural difficulties, such as those with internalising symptoms, are either missed entirely or incorrectly diagnosed.10

Schools are an integral aspect of daily life for children, but educators often lack support from the health system regarding how best to identify and manage children with emotional and behavioural difficulties. Improved collaboration between the health and education sectors could combine the strengths of both to better support these children. While paediatricians frequently assess and treat children with emotional and behavioural difficulties, fewer than 10% of children over 4 years of age will visit a paediatrician in a 12-month period.16 However, almost every child spends at least 25 hours per week in primary school. The education system is therefore well placed to enable these children to be identified early and receive the support they need. Educators have an important and unique role in this area, but need to be better supported to detect and respond to problems early as there are concerns that many schools lack evidence-based approaches to supporting children.17

The Victorian state government has implemented a strategy to strengthen mental health capacity in secondary schools, but this does not help the large numbers (approximately 84 000 per annum) of children whose difficulties emerge in primary school.3 Primary schools need to be supported to ensure that children are identified early, have access to evidence-informed interventions within the school environment and can be referred to appropriate health and other services when school support is deemed insufficient. Families of children who require further assessment and treatment often need help navigating a complex and fragmented service system so as not to risk increased wait times or face mismatched referrals. Ideally all this needs to be designed collaboratively with each school, informed by an understanding of the student cohort and knowledge of local community services and resources.

Two relevant models of collaboration exist within the education and health sectors. Communities of Practice is an approach which has been adopted by the Victorian Department of Education and Training whereby school networks share knowledge, experiences and resources.18 Project ECHO is a similar model of interprofessional education and case-based learning used in clinical medicine to improve access to specialist expertise.19 Both models address the challenges of knowledge deficits and organisational silos. Aspects of each could be used to improve communication and provide a shared learning experience for the health and education professionals who support children with emotional and behavioural difficulties.

Objectives

The primary objective is to evaluate the feasibility and acceptability to educators and caregivers, of a structured health and education collaborative model designed to improve outcomes for children with emotional and behavioural difficulties.

The secondary objective is to assess any difference between intervention and control schools in educator identification and classifications of children as ‘struggling/always overwhelmed’ (single-item question) in those who score in the ‘borderline/clinical’ range on the Strengths and Difficulties Questionnaire (SDQ; validated against clinical diagnosis).

Trial design

Pilot cluster randomised controlled trial of a co-designed structured health and education collaborative model aiming to improve educator identification and management of children with emotional and behavioural difficulties and family uptake of services to assess and manage these difficulties.

Hypotheses

We hypothesise that the structured health and education collaborative model will be feasible and acceptable to educators and caregivers (primary outcome). We also hypothesise that educators in such a model will be able to identify and support children with emotional and behavioural difficulties using standardised measures, implementation of evidence-based strategies and health service use (secondary outcome).

Methods and analysis

Study setting

The study is conducted in Victorian primary schools. Caregivers and educators of students in grades 1 (the second year of formal schooling in Australia), 2 and 3 are recruited to assess children early in their school education. Educators of preparatory grade (ie, first year of schooling) students are not recruited because of their varying experiences of preschool education and the need to allow time for them to settle into the school environment and routine. Metropolitan and rural primary schools are recruited to test the feasibility and acceptability across diverse settings.

Eligibility criteria

The Australian school sector is made up of government (65.6%), Catholic (19.4%) and independent (15%) schools.20 To ensure consistency in supports within the education system, only Government schools are invited to participate. Schools are recruited from the Local Government Area of the City of Yarra (metropolitan) and the Wimmera Southern Mallee (rural). Schools must have a minimum of 40 students across grades 1, 2 and 3. Special schools (schools that provide specialist and intensive support in a dedicated setting for students with moderate-to-high learning and support needs) are excluded. Caregivers and educators of students completing grade 1, 2 or 3 in 2021 are invited to participate. Caregivers who have insufficient English to complete the surveys and who do not have access to interpreter services to support completion of the survey are excluded.

Patient and public involvement

The public was first involved in the project when community leaders within both communities were informed of the concept in early 2019. The lead paediatrician has worked in both communities for a number of years and through this work frequently engages with families, educators and clinicians. The community leaders assist with school recruitment and in shaping the project through conversation early in the design process. Once schools are recruited multiple caregiver information sessions are held at each of the 12 schools, both online and in person. Caregivers are asked to comment on the project in regards to their experience in supporting their children’s development and well-being.

Intervention

The intervention period is 6 months with baseline surveys pre-intervention and a follow-up survey at the completion of the intervention period. During the 6-month intervention period schools receive two key strategies.

Paediatrician (author WG)-led, fortnightly 1 hour videoconference seminar programme with case-based discussions covering topics of interest as selected by participating educators. The programme incorporates evidence-based approaches to identification and management of children with emotional and behavioural difficulties.

Paediatrician support to help identify, support and navigate health services for children whom educators perceive have emotional and behavioural difficulties.

Half of the participating schools are randomly allocated to the intervention (intervention group). At the beginning of the project WG meets with each school allocated to the intervention to co-design the programme for terms 3 and 4. Based on other research about improving collaboration, the programme involves discussions about certain topics (eg, anxiety, learning difficulties) selected by each school.18 19 This leads to a discussion of how to best use evidence to support children with these difficulties in the school environment. In addition, WG provides assistance to help identify, support and navigate health services for children who require assessment and therapy outside of the school environment. The individual school uses its normal communication pathways to discuss this with caregivers.

The other half of the schools (control group) operate as they usually do with no regular case discussion and knowledge of how best to navigate the healthcare system left to educators.

Outcomes

Outcomes are assessed by caregiver and educator-completed surveys at baseline and again at the completion of the trial period (6 months). Educators in both arms complete baseline and follow-up surveys for participating children only (caregivers have submitted consent and baseline surveys). Educators in the intervention group are also asked to take part in a focus group at the completion of the intervention (see table 1).

Table 1

Primary and secondary outcomes

Outcome	Variable	Measure	Participant group		Collection point
			Caregiver	Educator	Baseline	6 months
Primary	Feasibility	Study recruitment and retention	✔️	✔️	✔️	✔️
Primary	Acceptability	Intervention participation (intervention group only)		✔️	✔️	✔️
Primary	Acceptability	Study designed measures (intervention group only)		✔️	✔️	✔️
Primary	Confidence supporting students with emotional and behavioural difficulties	School Mental Health Self-Efficiency Teacher Survey		✔️	✔️	✔️
Secondary	Accuracy and timing of emotional and behavioural difficulties	Strengths and Difficulties Questionnaire	✔️	✔️	✔️	✔️
Secondary	Health service use	Study designed-measures	✔️			✔️

The primary outcome of the study is the feasibility and acceptability of the structured health and education collaborative model from the perspectives of educators and caregivers. This is measured using both quantitative and qualitative methods:

Quantitative

Number and proportion of eligible (1) educators and (2) caregivers who consent take part in the pilot.

Number and proportion of eligible educators who participate in the case-based discussions.

Number and proportion of eligible educators who use paediatric support to discuss how best to support individual students outside of the case-based discussions

Study-designed survey items asking educators about acceptability of the programme.

Educator confidence in supporting children with emotional and behavioural difficulties measured using the School Mental Health Self-Efficiency Teacher Survey (SMH-SETS).21

Qualitative

Open-ended questions about educators’ experience of the programme, relevance of the content, usefulness of experience and suggestions for improvement.

Educator perspectives of whether the model has impacted their ability to identify and support children with emotional and behavioural difficulties.

Secondary outcomes will measure in the intervention versus control groups:

Accuracy of educator identification of child emotional and behavioural difficulties (study-designed, single-item measure) compared with the standardised measure The SDQ, a reliable and validated measure, already used in Victoria’s School Entry Health Questionnaire of each child’s emotional and behavioural symptoms.22

Improvement in emotional and behavioural symptoms at the completion of the intervention, measured by the SDQ, compared with baseline.

Caregiver health service use for child emotional and behavioural problems.

To determine whether the intervention leads to identification of child emotional and behavioural difficulties, caregivers and educators are asked at each time-point (baseline and 6 months later) a single-item question as a measure of overall child emotional and behavioural difficulties, namely: “Thinking about your child’s mental health and well-being over the last 6 months, has [child’s name] been thriving, coping, struggling or always overwhelmed?”. This response is dichotomised (thriving/coping vs struggling/always overwhelmed) and compared against the SDQ, a validated measure of child emotional and behavioural difficulties to determine the sensitivity and specificity of the single-item question. It is a screening questionnaire for 3–16 year-olds, in which caregivers or educators rate 25 items. It provides a total score and subscale scores including emotional symptoms, conduct problems, hyperactivity/inattention, peer relationship problems and prosocial behaviour. ‘Normal’, ‘borderline’ and ‘abnormal’ ranges exist for total difficulties and each of the subscales.23 We will determine the sensitivity and specificity of the single-item measure against the SDQ total borderline/clinical cut point, as reported by educators and caregivers at baseline and at follow-up.

Health service use is measured by caregiver report asking the following questions: ‘Did the teacher suggest a referral to a health service (eg, general practitioner, paediatrician) for your child because of emotional or behavioural concerns?’. If yes ‘did you engage with that health service?’ If no, ‘why not?’ The baseline survey also collects child (eg, age, sex, existing emotional and behavioural support) and caregiver (eg, relationship to child, highest level of education, household income) demographic data.

Measurement of participant compliance

All participants are asked to complete the surveys and measurement of compliance is carried out at each of the data collection points (baseline and 6 months). Any surveys not submitted result in a reminder email or SMS being sent to the relevant educator/caregiver. Surveys that have not been submitted will result in the relevant educator/caregivers being contacted by WG to discuss any difficulties completing the survey and assess ongoing participation in the trial.

The time commitment for individual educators includes the co-design focus groups (90 min), seminar case discussions (1 hour per fortnight/month depending on co-design; intervention group only) and survey completion (10–15 min per student, that is, up to 4–6 hours at each data collection point). The primary objective of this research project is to assess the acceptability and feasibility of such an intervention in the context of the educators’ workload. Student well-being however is a core component of school responsibility, as detailed in the Royal Commission into Victoria’s mental health system,24 and participating schools and educators have expressed their willingness to take part in this pilot and complete relevant surveys.

Participant timeline

Schools were first enrolled in April 2021 and randomly allocated to the control or intervention group in May (see figure 1). Baseline survey collection took place in June with follow-up surveys planned for January 2022. Co-design workshops occurred with intervention schools in June 2021 and reflection focus groups with the same schools will take place in February 2022.

Figure 1

Flow chart of trial timeline.

Sample size

As this is a feasibility and acceptability pilot, a formal sample size calculation is not required. However, with a sample of 432 from 720 eligible (60% enrolment of eligible families presuming 12 schools with an average of 20 students in each of grades 1, 2 and 3), will provide meaningful data to ascertain the feasibility and acceptability of the intervention and study measures.25 Further, results will inform a sample size calculation of a planned future, adequately powered randomised controlled trial to test the effectiveness and cost-effectiveness of this intervention.

Recruitment

School recruitment

Schools are recruited through established relationships with existing community networks and invited to participate in the study. The paediatrician leading the study has been a member of both communities in a professional capacity for over 2 years; this assists with school and health service participation. The leadership team at each school, including the principal, assist in selecting the individual educators for the co-design process. This include the educators who will be receiving the intervention, along with any members of the school team involved in supporting children with emotional and behavioural difficulties (eg, student well-being team). Each participating school is asked to complete a School Participant Information Statement and Consent Form (see online supplemental appendix A).

Student recruitment

Educators distribute the Caregiver Study Interest Form (see online supplemental appendix B) where caregivers can state whether they do or do not wish to learn more about the study. In addition to this form, school communication networks are used to inform caregivers about the study (eg, school newsletter, caregiver information evening, flyers on the school campus). Interested families have 2 weeks to return the form after which time WG collects the forms of those families who have opted in. At the end of the 2 weeks, the classroom educator provides information about how many of the potential participants did not return the form. This demonstrates how many potential caregivers could be enrolled if families who did not opt out were contacted in the future. Any family who returns forms where opt out has been selected will not be contacted.

The study team then phones caregivers who have opted in to hear more about the study. They explain the study further, answer any questions they might have and ensure they meet inclusion criteria. Eligible and interested caregivers are emailed the baseline survey and Caregiver Participant Information Statement and Consent Form (see online supplemental appendix C) to return when completed. Should this ‘opt-in’ approach to recruitment yield insufficient caregivers, an ‘opt out’ approach is taken as per the Royal Children’s Hospital ethics committee and Department of Education approval. We start with an ‘opt-in’ approach first as this is the preference of the study’s ethics committee. Educators who have students enrolled in the study via caregiver survey completion will be provided with the Educator Participant Information Statement and Consent Form (see online supplemental appendix D).

Allocation

A statistician not directly involved in the analysis of the trial results prepares the randomisation schedule. Randomisation is stratified by region and tertile (see figure 2 below). For each stratum, we have two schools, which have been randomly allocated to treatment and control. This has been done using function sample in R V.4.1.0. The Index of Community Socio-Educational Advantage (ICSEA) is used to allocate each participating school into a relative tertile where the first tertile is the least advantaged third of schools and the third tertile is the most advantaged third of schools.26 The ICSEA score calculation is made up of the socio-educational advantage (SEA) plus remoteness and percentage of Indigenous student enrolment. The SEA is calculated using parental occupation and education. Schools in each region are paired based on their ICSEA score and within each pair, randomly allocated to either the control or intervention group. Allocation occurs after schools have consented to participate to ensure that the study includes a broad range of primary schools. Blinding is not possible given that educators (and caregivers) will be aware of their allocation based on participation in the intervention.

Figure 2

School pairing by region and Index of Community Socio-Educational Advantage ranking.

Data collection methods

As demonstrated in the participant timeline, measurements are collected at baseline and 6 months post baseline survey completion. Baseline surveys are collected prior to the 6-month intervention period, in which caregiver and educator surveys are conducted for each student in the control and intervention schools. Follow-up surveys are completed 6 months after the baseline measurements. At this time caregivers and educators are asked to again complete the single-item measurement, the SDQ and the SMH-SETS (educators only). In addition, the follow-up survey will ask caregivers and educators about health service referral and uptake. Participants complete surveys either via a link to a Research Electronic Data Capture (REDCap) survey database or using paper surveys which are then transferred to the REDCap database once submitted27 28 (see online supplemental appendix E and F for caregiver and educator surveys).

Data management and monitoring

All data collected is de-identified and stored in the restricted access folder on the Murdoch Children’s Research Institute (MCRI) server. This database is password protected and only the study investigators have access to this data. Confidentiality of the participants is maintained at all times. As per the Australian Code for the Responsible Conduct of Research, data are stored for a minimum of 5 years. To protect participant privacy, all data collected is de-identifiable with only the research team able to match participant names with ID numbers and stored on a secure network drive within the MCRI that is only accessible by the study investigators. Any paper forms used are scanned and stored on the secure network drive and subsequently destroyed. Given that this is a pilot study with limited resources, a formal data monitoring committee has not been created. The study team meet frequently however to monitor progress and independent oversight occurs as required by the overseeing University.

Statistical methods

Sample characteristics, participation rates and educator and caregiver reports of feasibly and acceptability are described using summary statistics (eg, number and proportion of caregivers who take part in the pilot, and educator confidence supporting students with emotional and behavioural difficulties).

To determine the usefulness of the single-item question about child mental health, we determine the sensitivity, specificity and positive and negative predictive values of the dichotomised response versus the educator-completed SDQ borderline/abnormal total score cut point for students in each group. We do this at baseline and at follow-up to determine if educators in the intervention group identify more children with symptoms of emotional and behavioural problems compared with educators in the control group. The single-item question completed by caregivers is also evaluated using the same method of comparison to the caregiver SDQ total score.

For our secondary outcomes, an intention-to-treat analysis at the level of the child is conducted. Scores from the SDQ at follow-up are dichotomised and compared between the intervention and control groups, with adjustments for baseline SDQ and the randomisation stratification factor (ICSEA). This will be conducted separately for educator and caregiver reported SDQ, using logistic regression. OR will be reported with 95% CIs and p values. These analyses will also be repeated using linear regression of the continuous SDQ scores on a standardised scale to provide effect sizes. Effect sizes are considered as small, ~0.20 SD; moderate, ~0.50 SD; and large, ~0.80 SD. Analysis is completed using Stata V.17.0.29

Harms

Participants are provided with contact details for the lead investigator (WG) and the Director of Research Ethics and Governance at The Royal Children’s Hospital Melbourne. They are advised to report any adverse events and such events will be reported to the ethics committee once the study team is aware that they have occurred. Oversight of the pilot includes monthly review meetings with HH and FO—two experienced paediatric researchers who have conducted over 15 randomised controlled trials in child health

Ethics and dissemination

Research and ethics approval

This study was approved by the Royal Children’s Hospital Human Research and Ethics Committee (#67653) and the Victorian Department of Education Research in Schools Ethics Committee (#2021_004349) on 16 March 2021. Informed caregiver consent will be obtained via a written or online participant information and consent form.

Protocol amendments

This study will be conducted in compliance with the current version of the protocol. Any change to the protocol document or informed consent form that affects the scientific intent, study design, participant safety or may affect a participant’s willingness to continue participation in the study is considered an amendment, and therefore will be written and filed as an amendment to this protocol and/or informed consent form. All such amendments will be submitted to the Human Research and Ethics Committee, for approval prior to becoming effective.

Consent

Selected schools are contacted via email and/or phone with an invitation to participate in the study. This correspondence includes the Participant Information and Consent Form (PICF, see online supplemental appendix). A signed consent form is obtained for each participating school principal, individual educator and caregiver.

Following family recruitment (described in 3.7) all participants are informed of their voluntary participation and ability to withdrawal their involvement at any time. In addition, each participant is informed of their anonymity in regard to the study including any publications resulting from the research. The lead investigator provides the PICF to the caregiver. This document describes the purpose of the trial, the procedures to be followed and the risks and benefits of participation.

The research team conducts informed consent discussion and checks the participant comprehend the information provided. The research team member answers any questions about the trial.

Participants are invited to provide consent following a phone call discussion about the study. Participants are given the choice of an online link or paper copy of the consent form to complete along with the baseline survey in the same format.

It is documented in the participant’s record that consent has been provided. When all the inclusion/exclusion criteria have been addressed and the eligibility of the participant confirmed, the participant will be assigned to a trial group, based on the group that their child’s school is randomised to.

Confidentiality

Participant confidentiality is strictly held in trust by the participating investigators, research staff and the sponsoring institution and their agents. The study protocol, documentation, data and all other information generated is held in strict confidence. No information concerning the study or the data will be released to any unauthorised third party, without prior approval by the participant and written approval of the sponsoring institution.