Study protocol for a self-controlled cluster randomised trial of the Alert Program to improve self-regulation and executive function in Australian Aboriginal children with fetal alcohol spectrum disorder.

INTRODUCTION: While research highlights the benefits of early diagnosis and intervention for children with fetal alcohol spectrum disorders (FASD), there are limited data documenting effective interventions for Australian children living in remote communities. METHODS AND ANALYSIS: This self-controlled cluster randomised trial is evaluating the effectiveness of an 8-week Alert Program school curriculum for improving self-regulation and executive function in children living in remote Australian Aboriginal communities. Children in grades 1-6 attending any of the eight participating schools across the Fitzroy Valley in remote North-West Australia (N ≈ 363) were invited to participate. Each school was assigned to one of four clusters with clusters randomly assigned to receive the intervention at one of four time points. Clusters two, three and four had extended control conditions where students received regular schooling before later receiving the intervention. Trained classroom teachers delivered the Alert Program to students in discrete, weekly, 1-hour lessons. Student outcomes were assessed at three time points. For the intervention condition, data collection occurred 2 weeks immediately before and after the intervention, with a follow-up 8 weeks later. For control conditions in clusters two to four, the control data collection matched that of the data collection for the intervention condition in the preceding cluster. The primary outcome is change in self-regulation. FASD diagnoses will be determined via medical record review after the completion of data collection. The results will be analysed using generalised linear mixed modelling and reported in accordance with Consolidated Standards of Reporting Trials (CONSORT) guidelines. ETHICS AND DISSEMINATION: Ethical approval was obtained from the University of Western Australia (WA) (RA/4/1/7234), WA Aboriginal Health Ethics Committee (601) and WA Country Health Service (2015:04). The Kimberley Aboriginal Health Planning Forum Research Sub-Committee and WA Department of Education also provided approval. The results will be disseminated through peer-reviewed journals, conference presentations, the media and at forums. TRIAL REGISTRATION NUMBER: ACTRN12615000733572; Pre-results. © Article author(s) (or their employer(s) unless otherwise stated in the text of the article) 2018. All rights reserved. No commercial use is permitted unless otherwise expressly granted.

RCT Entities:   Population Interventions Outcomes

This population-based self-controlled cluster randomised trial is part of a community-led fetal alcohol spectrum disorder strategy and follows extensive community consultation, piloting of outcome measures and adaptation of the Alert Program curriculum.

This trial will provide new information regarding the executive functioning abilities of predominantly Australian Aboriginal children living in remote communities.

Due to cost and geographical restraints, the design is non-traditional and will require careful statistical analysis.

Blinded measurement of outcomes was not possible and therefore the results must be interpreted with caution.

A highly mobile student population means a greater potential for contamination between clusters. It may also result in higher rates of study dropout and missing data.

Introduction

Children exposed prenatally to alcohol are at risk of acquiring a range of physical and central nervous system (CNS) abnormalities, collectively known as fetal alcohol spectrum disorder (FASD).1 2 Diagnostic criteria vary internationally, and FASD diagnoses have included fetal alcohol syndrome (FAS), partial fetal alcohol syndrome (pFAS) and neurodevelopmental disorder-alcohol exposed (ND-AE).3–5 Australian diagnostic guidelines have recently been developed in line with revised Canadian diagnostic guidelines, and specific diagnoses on the FASD spectrum include the following: (a) FASD with three sentinel facial features and (b) FASD without three sentinel facial features.6 7 Regardless of terminology, it is clear these disorders are characterised by pervasive neurodevelopmental impairment that impacts affected individuals across their lifespan.1 4 8–10

FASD prevalence internationally and in the Fitzroy Valley region of Western Australia

A recent systematic review and meta-analysis estimated global FASD prevalence in the general population at 7.7 per 1000 children and young people (0–16.4 years).11 Higher FASD rates have been identified in countries, including South Africa (111.1 per 1000 population) and Ireland (47.5 per 1000 population). Lower prevalence rates have been estimated in the Eastern Mediterranean Region (0.1 per 1000 population).11 From 2010 to 2012, the Lililwan Project was conducted. This population-based prevalence study used active case ascertainment of FASD in children born in 2002 and 2003 and living and attending school in the Fitzroy Valley region of Western Australia (WA). The project estimated a prevalence of 120.0 cases of FAS/pFAS per 1000.12 Furthermore, the prevalence of ND-AE was estimated at 74.1 per 1000 children, resulting in an overall FASD prevalence rate of 194.4 per 1000.13 These rates, determined using the 2005 Canadian FASD diagnostic guidelines and a careful diagnostic approach, are among the highest in the world.4 12 Aboriginal community leaders and school staff from the Fitzroy Valley have described FASD as a significant barrier to student learning, with the challenging behaviours of affected school-aged children being of major concern to school communities.14

FASD and executive functioning impairments

A confluence of research has identified executive functioning deficits as a common characteristic of individuals with FASD.9 15–27 Damage to neurological structures, including the prefrontal regions of the brain, is a significant hypothesised cause for these deficits.28 29 Executive functions are defined as a set of cognitive processes responsible for orchestrating purposeful, goal-directed behaviour.15 30 31 These processes are responsible for the ability to plan, organise, attend, problem solve and inhibit responses.31 It is also suggested that the ability to self-regulate emotional responses and behavioural actions is interrelated with the construct of executive functioning.17 28 32 Deficits in executive functioning and self-regulation can lead to learning and behavioural problems that impact a child’s educational outcomes as they struggle to cope with the complex demands of school life.16 20

The Lililwan Project identified the executive functions as one of the most commonly impaired domains of the CNS of children in the Fitzroy Valley, affecting 30% of the cohort of children assessed for FASD.13 High rates of executive functioning impairment were identified in children with both a diagnosis of FAS or pFAS (50%) and ND-AE (83%)13 Assessment of the executive functions in the Lililwan Project was accomplished through relatively brief testing.1 Of those children who did not meet diagnostic criteria for FAS, pFAS or ND-AE (n=87), 22% still showed executive functioning impairment13 suggesting there are multiple causes, including high rates of early life trauma and socioeconomic disadvantage commonly experienced in remote Aboriginal communities12 13 33–35

Interventions to support children with FASD

Early diagnosis and intervention for children with FASD are thought to be key to preventing behavioural, mental health and learning difficulties.36–38 However, Fitzroy Valley community members have reported that a current lack of diagnostic and intervention support for children with FASD impacts their children’s ability to reach their full potential.14 Children with FASD need access to interventions which support their development of emotional and behavioural regulation skills.38 It is recognised that educators, alongside the family, play a crucial role in supporting children with FASD to improve life outcomes through contextually appropriate and evidence-based interventions.36 While there is limited evidence for strategies that can assist children affected by FASD,2 particularly to improve self-regulation and executive functioning skills,8 17 32 39 the Alert Program for Self-Regulation has evidence to suggest it is a promising intervention.17 39

The Alert Program

The Alert Program, designed by occupational therapists, teaches children about self-regulation, which is the ability to change the arousal state of the nervous system or how ‘alert’ one feels appropriately to the task or situation.40 Self-regulation involves the sensory systems of the brain, including the vestibular and somatosensory systems.40 The program uses the analogy of a car engine to explain and provide vocabulary for arousal states by describing that the body can go into ‘high gear,’ ‘low gear’ and ‘just right gear.’ Participants then learn to use sensorimotor strategies and tools from five categories (look, touch, move, listen and by putting something in the mouth) to shift their arousal level into an optimal or ‘just right place’ to meet the demands of a particular task or situation as a means of self-regulation.40 The program is designed to be used within a range of settings and with clients of varying ages, making it suitable to adapt to remote school environments with a high proportion of Aboriginal students.40 Local occupational therapists and teachers have reported anecdotal success using the Alert Program with a limited number of children in the Fitzroy Valley.

Previous Alert Program studies

Four small North American studies until now have investigated the effectiveness of the Alert Program in improving neurobehavioural outcomes in children aged 6–12 years.17 39 41 42 Three of these four studies17 39 41 enrolled only children with FASD diagnoses. A randomised controlled trial (RCT) (n=40 treatment group, n=38 control group) of a 12-week neurocognitive habilitation programme,39 adapted from the Alert Program, for children with FASD reported statistically significant improvements in the Organization of Materials and Monitor subtests on the Behavior Rating Inventory of Executive Function (BRIEF) and the Roberts Apperception Test for Children Resolution one and two subtests.39 A between-subjects trial (n=12 treatment group, n=13 delayed treatment group) for children with FASD found statistically significant improvements on the Developmental Neuropsychological Assessment (NEPSY-II) Inhibition Naming and Affect Recognition subtests after 12 1-hour individual Alert Program sessions.17 A significant treatment effect was also detected using the BRIEF Behavioral Regulation Index and the BRIEF Emotional Control Sub-Scale.17 Importantly, structural MRI scanning also showed an increase in cortical grey matter in children with FASD following this 12-week Alert Program intervention.41 A small quasi-experimental pre-test/post-test school-based study (n=7 treatment group, n=5 control group) reported a significant positive group effect on teacher ratings of child efficacy in self-regulation after an 8-week Alert Program delivered to children with emotional disturbance.42

Evidence gaps and the significance of this work

While each of these four studies delivered an intervention using concepts from the Alert Program, the duration and frequency of each intervention varied, as did the setting in which the program was delivered. These studies had small sample sizes, so results must be interpreted cautiously. Three of the four studies trialled the Alert Program among children with confirmed FASD.17 39 41 The majority of participants resided in out-of-home care39 and received the Alert Program intervention in either a one-on-one17 or group39 therapist-delivered treatment setting, limiting the generalisability of study findings.

Nonetheless, the results from three Alert Program studies show promise for the effectiveness of the Alert Program in children with FASD. These studies provide a foundation for research in this area, including self-regulation and executive functioning. Our current study is examining, for the first time, the efficacy of the Alert Program within Australian schools, for children with or without a FASD diagnoses, and in a population with a high proportion of Indigenous children. Such research is extremely important as Aboriginal communities in North-Western Australia are actively seeking solutions for the large number of children and families living with a FASD.14

Aim

This paper describes the protocol of a study conducted to determine the effectiveness of the Alert Program in changing self-regulation and executive functioning in children aged between 5.5 and 12.5 years, with and without FASD, attending primary school in the remote communities of the Fitzroy Valley.

Methods and analysis

Setting and sample

The Fitzroy Valley, population 3500, is located in the remote Kimberley region 2500 km north of Perth, WA, and comprises 45 distinct communities.43 The Valley’s service centre is Fitzroy Crossing, which has a population of approximately 1200 people. The majority of residents (~80%) are Aboriginal people from the Bunuba, Walmajarri, Gooniyandi, Wangkatjunga and Nyikina language groups.43 Standard Australian English is a second or third language for many people in the Fitzroy Valley, with Kimberley Kriol or traditional languages spoken at home and within the community.1

Approximately 809 students are enrolled in kindergarten to grade 12 at six government and five independent community schools located within the Fitzroy Valley.44 Individual schools cater for between 14 and 278 students.44 Smaller schools are staffed by a teaching principal, a classroom teacher and an Aboriginal and Islander Education Officer (AIEO). High staff turnover can interrupt the continuity of staff knowledge and understanding specific to the cultural context.14 Classes often comprise multiaged groups of students due to low student numbers. Isolation is a key characteristic of schools in this region with some students travelling up to 220 km from their home community to and from school each day. Road access to some of the more remote communities is limited during the wet season (October–March) due to flooding which impacts school attendance and access to health services. Clinical services such as occupational therapy, paediatrics and school psychology are provided from major town centres with travel times to and from each school ranging between 6 and 10 hours. Due to their remoteness, visits by clinicians are limited to approximately 1 day, once or twice per 10-week school term. Consequently, effective and sustainable support for individuals with FASD must overcome the multiple complexities of service provision in a remote region such as the Fitzroy Valley. Interventions, such as the Alert Program which can be delivered by classroom teachers, provide an approach for enabling optimal access for children and could enhance program sustainability.45

Community consultation and pilot study

In 2008, community members of the Fitzroy Valley initiated a strategy to address FASD that included prevention, diagnosis and support/therapy.46 Implementing interventions for school children is a priority of this communityinitiative titled the Marulu FASD Strategy. Marulu means precious, worth nurturing and in this case refers to the children of the Fitzroy Valley. In 2015, seven visits of 2 to 3 weeks duration were made to the Fitzroy Valley. During these visits researchers provided information to the community about the proposed study and sought input from stakeholders. This included consultation with Aboriginal Elders, community leaders and representatives from health and education.47

A pilot study was conducted at a small Fitzroy Valley remote community school during this formative stage of the project in term three (July–September) 2015. The pilot study evaluated study procedures, Alert Program curriculum, resources and assessment measures to be used in this larger study. Following the pilot study, the Alert Program curriculum was modified in consultation with the program’s originators in response to feedback from teachers and the local community regarding contextual appropriateness. An initial battery of outcome measures was trialled and revised for the broader study, based on feedback from teachers and parent/caregivers, and following analysis of pilot study data.47 Principals and community members from eight of the other nine primary schools agreed for their school to be a part of the larger trial.

Study design, randomisation and blinding

The Alert Program study is a self-controlled cluster randomised trial based on modified stepped wedge cluster randomised trial principles. Following program pilot testing and obtaining community and stakeholder feedback on the intended design, it was determined the burden of data collection on participants would be too great in a stepped wedge trial. The nine required data collection points within each community would also have been logistically unrealistic and not possible under budgetary and resource constraints, particularly given the remote geographical location. Considering these factors, the design was modified to remove additional data collection time points; thus, the study is referred to as a self-controlled cluster randomised trial in that each cluster serves as its own control, with the control measures being made at the same time as the measures made in the preceding cluster before and after the intervention condition. The Alert Program was provided to all clusters in a phased roll out. The program was expected to have a positive effect compared with normal schooling, and simultaneous intervention conditions in multiple clusters would have been impractical. This process of study design and implementation remains faithful to Australian National Health and Medical Research Council’s Aboriginal and Torres Strait Islander research values whereby Aboriginal people and communities had input and agreed to the research processes and procedures prior to implementation.48

Eight schools were assigned to one of four clusters based on their geographical proximity and student enrolment numbers. This enabled relatively equal distribution of participants between clusters and allowed for efficient allocation of resources in this remote region. Using a computer-generated randomisation schedule, each cluster was randomly assigned to receive the intervention during school term two (April-July) or three (July-September) in either 2016 or 2017. Communities are frequently inaccessible and schooling is impacted due to weather conditions during terms one and four, so an interrupted implementation was necessary. Table 1 shows the sequence of data collection and intervention scheduling for each cluster. Cluster one participants received one pre-test measure prior to the intervention, while clusters two to four had an extended control phase prior to intervention. The nature of the study prevented blinding of schools and participants. Researchers measuring outcomes were unable to be blinded as (a) they were present in schools when the intervention was being delivered, (b) were required to explain to parents/caregivers at each data collection time point why those data are being collected in relation to the timing of the intervention and (c) different data collection time points required researchers to remind teachers about the need to collect other process-based data. It is recognised that results of unblinded trials tend to be biased towards finding beneficial effects and this will be carefully considered when interpreting findings and clearly stated in any manuscript reporting the results.

Table 1

Timeline for interventions in each cluster

	2016						2017
	April– May	May– June	End June	July– September	September	November– December	April– May	May– June	May– June	July– August	September	November– December
Cluster 1	0	X	0	0
Cluster 2	0		0	X	0	0
Cluster 3			0		0		0	X	0	0
Cluster 4							0		0	X	0	0

0, data collection; X, delivery of intervention.

Participants

Inclusion criteria

All children enrolled in grades 1 to 6 (between 5.5 and 12.5 years of age) at one of eight primary schools in the area at the time of recruitment were invited to participate.

Sample size

Based on school enrolment figures from 2015, there were estimated to be 363 children eligible for enrolment into the study.48 Sample size calculations cannot be performed for this self-controlled cluster randomised trial as the design is not standard. Therefore, sample size was estimated for a RCT and applying a design effect for a cross-over RCT.49 Sample size was calculated with power 80%, significance 0.05 and using pilot study outcomes on the Sutter-Eyberg Student Behavior Inventory—Revised (SESBI-R) Intensity Scale score. To detect a mean difference of 15 points (SD 40) in the primary outcome, a sample size of 112 children per treatment arm would be needed for an RCT, uncorrected for clustering or time. For a cross-over RCT, with 60 subjects per cluster across time periods, an intracluster correlation (ICC) of 0.05 and an interperiod correlation of 0.01, the number of subjects required is 240 per treatment arm. The ICC of 0.05 and the interperiod correlation of 0.01 are both considered conservative.50 Increasing the ICC towards 1 increases sample size requirements, though typical estimates for human subjects are below 0.05.51 Increasing the interperiod correlation towards 1 reduces the required sample size. This cross-over RCT sample size is likely to be an overestimate of actual requirements for a self-controlled cluster randomised trial as it does not account for the improved efficiency gained from repeated measures. Cluster sample size calculations allowing for repeated measures are complex and rely on specified parameter estimates and correlation structures,49 which are presently unknown.

While sample size estimates exist for stepped wedge designs,52 they were deemed unreliable as they are based on the structure of the design, which for this study would assume nine data collection points per cluster and calculate a total of 75 subjects needed. The removal of data collection points would increase the sample required, though the extent to which this increases cannot be determined. Due to the uncertainty of the sample size calculations, there is a possibility of being overpowered or underpowered. However, to address a request from community leaders and stakeholders to invite all families in the region to be involved in this population-based study, the maximum number of participants were invited to participate.

Recruitment and consent

Community leaders and school principals provided in-principle approval for the study via a series of engagement visits in 2015. School principals provided formal consent for their school’s participation between December 2015 and March 2016. Participants were recruited in the year their cluster commenced data collection via a series of home visits conducted in partnership with locally employed Aboriginal community researchers. Where necessary, parents/caregivers received language translation through community researchers to ensure comprehension of study documents before providing signed informed consent for their child’s participation in the study. Separate consent was requested for data collection, access to school attendance data, access to healthcare records, photos and videos. Child assent was obtained prior to each assessment with students. Parents/caregivers could withdraw consent at any point during the trial. In addition, school staff provided consent to deliver the intervention, to complete teacher-rated questionnaires for students in their class who were enrolled in the study and to provide feedback on the intervention, training and resources.

School-based Alert Program intervention

The Alert Program was delivered to students by their regular classroom teacher to ensure program concepts could be reinforced outside the discrete Alert Program lessons. This models real-world conditions, as it is impractical for local occupational therapists to provide weekly interventionist support at the classroom level. Eight 1-hour lessons were taught over 8 consecutive weeks using a curriculum guide containing lesson plans. To improve feasibility of delivery within the school curriculum, the lesson plans were based on Stages One and Two of the original three-stage Alert Program.

Stage One of the program focused on identifying engine speeds by learning the engine vocabulary and labelling engine levels. Stage Two allowed students to experiment with choosing strategies in each of the five sensorimotor areas (look, touch, listen, move and mouth) to change their engine speeds.40 Each lesson plan detailed a set procedure, vocabulary, activities and resources to be used by teachers.49 Ideas to embed Alert Program concepts into the classroom outside the discrete Alert Program lessons were also included. Stage Three of the program related to the development of student independence in using Alert Program strategies and receiving ongoing support.40 Suggestions on how to transition to Stage Three of the program were made at the conclusion of the 8-week curriculum guide.49

All materials and resources referred to in the curriculum guide were provided to teachers. This guide was developed by the research team and Fitzroy Valley-based occupational therapists by adapting commercial Alert Program resources for use in the remote school context with the authors’ permission. The Alert Program language, activities and resources were pilot tested at a remote Fitzroy Valley Aboriginal community school to evaluate the feasibility and cultural appropriateness of program delivery with minor adaptations made based on teacher, occupational therapist and original program developer feedback. Information about program, training and resource adaptations has been reported elsewhere.47

Training

The training was developed in consultation with the Alert Program developers, local occupational therapists and feedback from the implementation of teacher training during the pilot study.47 Teachers, AIEOs, education assistants (EAs) and school leaders participated in a 1-day training session prior to Alert Program implementation, This workshop introduced participants to Alert Program concepts, curriculum guide and resources to be delivered to their students. Participants viewed modules one and two from the Alert Program on-line leaders’ training which covered the first four of eight Alert Program key concepts. A second 1-day workshop was presented to school staff between the third and fourth week of Alert Program lesson delivery. In the second session, participants viewed on-line training modules three to five which covered the remaining four Alert Program key concepts. Both training sessions were facilitated by a researcher who had completed the on-line Alert Program leader’s course.

Intervention fidelity measures and support

The original program developers were contracted to review and provide feedback on aspects of the revised pilot study curriculum prior to the larger study. To assess fidelity and track intervention dosage, teachers were asked to record student attendance for each lesson, self-report any modifications made to lessons and to log any additional self-regulation activities they implemented outside the Alert Program lessons. Occupational therapist support was available through the Kimberley Population Health Unit during their planned school visits (once to twice per term). A researcher, who had undertaken on-line Alert Program training, was available via phone/email and at the training sessions to answer any queries relating to intervention delivery.

Outcome measures

The efficacy of the Alert Program to improve self-regulation and executive functioning was assessed using a series of standardised direct and indirect outcome measures, summarised in table 2. The primary outcome measure, of most importance to the school and community, will be the frequency of disruptive behaviours as measured on the Sutter-Eyberg Student Behavior Inventory—Revised (SESBI-R). It is hypothesised that improvements in children’s self-regulation following the Alert Program will lead to a reduction in disruptive behaviours.50–53 All other outcomes will be secondary.

Table 2

Outcome measures

Respondent	Measure	Outcomes
Teacher	Sutter-Eyberg Student Behavior Inventory—Revised version54 Behavior Rating Inventory of Executive Function (BRIEF) second edition—Teacher Screening Form28	Disruptive behaviours Behaviour, emotion and cognitive regulation
Parent/caregiver	Eyberg Child Behavior Inventory54 BRIEF second edition—Parent Screening Form28	Disruptive behaviours Behaviour, emotion and cognitive regulation
Child	Computerised test battery55 56 Go/No-Go Match to Sample Tower of London Berg’s (Wisconsin) Card Sorting Task Child Occupational Self-Assessment66	Areas of executive function Response inhibition Short-term memory Planning and strategy use Set shifting Occupational competence and values

Teacher-reported and parent-reported measures

Disruptive behaviours: Sutter-Eyberg Student Behavior Inventory—Revised (SESBI-R) and Eyberg Child Behavior Inventory (ECBI)

The 38-item teacher-rated SESBI-R and 36-item Eyberg Child Behavior Inventory (ECBI) are measures of disruptive behaviour in children aged 2–16 years. Both assessments incorporate a measure of frequency of disruptive behaviours (Intensity scale) rated on seven-point scales and a measure of the number of disruptive behaviours that are perceived as a problem (Problem scale). The SESBI-R and ECBI have been shown to have high internal consistency for both Intensity (α=0.98, α=0.95) and Problem scales (α=0.96, α=0.93).54 Reliability coefficients at 12-week intervals for SESBI-R and ECBI Intensity (r=0.94 and r=0.80) and Problem scales (r=0.98 and r=0.85) are also high.54 A reduction in score indicates fewer and/or less problematic disruptive behaviours. The SESBI-R and ECBI have been found to correlate significantly with a number of measures of related constructs, including the Externalizing Scale of the Child Behavior Checklist, indicating their convergent validity.54

Executive functioning behaviours: Behavior Rating Inventory of Executive Function 2 Teacher and Parent Screening Forms (BRIEF2 Teacher/Parent Screening Form)

The Behavior Rating Inventory of Executive Function 2 (BRIEF2) Teacher and Parent Screening Forms are 12-item questionnaires completed by teachers and primary caregivers. These forms assess executive functioning behaviours related to behavioural, emotional and cognitive regulation in the school and home environments, respectively.28 Each item is rated on a three-point scale with ‘1’ corresponding to ‘Never’, ‘2’ corresponding to ‘Sometimes’ and ‘3’ corresponding to ‘Often’. A reduction in total score indicates improvement. The BRIEF2 was designed for use with children aged 5–18 years, including children with developmental, neurological, psychiatric and medical conditions.28 The screening forms yield an Executive Functioning Screening raw score that has good internal reliability (α=0.87–0.91 for the standardisation sample) and test–retest reliability (r=0.79–0.87) and which correlates highly with the full BRIEF2 Global Executive Composite Score (r=0.93–0.96).28 For the present study, pilot testing with teachers and parents/caregivers in the Fitzroy Valley indicated that the full BRIEF was time-intensive and impractical for both teachers and parents. Furthermore, pilot testing revealed two items on the BRIEF2 Parent Screening Form were not relevant to their experience; ‘Does not plan ahead for school assignments’ and ‘Written work is poorly organized’.28 Consequently, these items have been omitted. Correspondence with the authors of the BRIEF informed the decision to prorate these cognitive regulation items based on other responses in this subscale to calculate the Executive Function Screening raw score. All items on the BRIEF2 Teacher Screening Form were administered.

Cognitive executive functioning measures

Executive function was directly measured using four subtests from the Psychology Experiment Building Language (PEBL) Test Battery55 56 each administered via a laptop computer. There is emerging evidence that tasks administered through the PEBL platform are reliable and valid compared with other methods of delivery.57 58 Short practice trials are completed before all tasks apart from Berg’s Card Sorting Test where it is not appropriate due to the effects of practice (children must infer the rules of the task from feedback given while performing the trials) on the outcomes of the task.

Go/No-Go

This is a measure of inattention and impulsivity where the participant must execute or inhibit a motor response (pressing a button) when presented with target stimuli (the letter P or R).59 Initially, button presses are required when viewing a ‘P’ for 75 trials, with a ratio of four ‘P’ presentations for each ‘R’. The target switches to ‘R’ followed by another 10 trial practices, and then a 75 trial experimental block with the same ‘P’ to ‘R’ ratio.55 56 Performance is assessed by calculating four values: (a) proportion of correct responses to the target letter (Go condition); (b) proportion of misses for the target letter (misses); (c) proportion of responses to the No-Go letter (false alarms) and (d) proportion of correct rejections of the No-Go letter. Furthermore, reaction times to the target (Go) letter is assessed. Missing the target letter (Go errors (b)) is thought to indicate inattention to the task.59–61

Match to Sample

This is a measure of working memory using a 4×4 matrix pattern filled with yellow and red squares.55 56 Participants are allotted 1000 ms to study the target pattern that appears on the screen. After the target pattern disappears, it is replaced (3500 ms after) by a pair of matrices. One of these is the same as the previously viewed stimulus and one differs randomly by one or more cells of the matrix. The participant indicates which of the two was the presented stimulus. Behavioural performance on the task is assessed by calculating the number of correct responses and mean reaction time. After two practice trials, participants perform 30 trials and are provided with feedback as to whether they made the correct selection.55

Tower of London

This is a measure of planning which involves two sets of three different coloured discs that are organised into piles with height restrictions of one, two and three discs.62 63 Participants are presented with a ‘goal’ state displayed at the top of the screen with coloured discs in a particular position. Participants are instructed to move the coloured discs in the active area to match the goal state in as few moves as possible, and as quickly as possible, using the computer mouse. Performance is assessed by number of moves used, time taken and time to first move. Participants completed three training trials followed by 15 trials of increasing complexity.55

Berg’s ‘Wisconsin’ Card Sorting Test

This is a measure of set-shifting that is a shortened standardised version of the Wisconsin Card Sorting Test.55 56 Participants are presented with four stimulus cards (single red triangle, two green stars, three yellow crosses and four blue circles) and asked to sequentially match 64 new cards which have various combinations of the attributes of colour, shape and number, but they are not provided with specific instruction on what aspects of the cards they should be matching.55 56 64 After making a decision, the child is told whether that match is right or wrong, based on the current matching rule in operation (match on colour, shape or number). After the participant sorts five successive cards correctly using the initial sorting category, the ‘correct’ sorting category shifts to another without informing the participant. Therefore, they must infer that the rule has changed and shift set to a new method of correct sorting. After the participant gets five cards correct in the new category, the ‘correct’ sorting category is again shifted (and so on). Behavioural performance on the task is assessed by calculating the number of correct responses, total errors, perseverative responses, perseverative errors (where they persist with the previous rule incorrectly), non-perseverative errors and unique errors.65

Child self-report measures

Sense of occupational competence: Child Occupational Self-Assessment (COSA)

The Child Occupational Self-Assessment (COSA) is a 25-item self-report measure of children’s perceptions of how well they do (Competence) on a range of activities they are likely to encounter at home, at school and in their community. This measure also assesses how important these activities are to them (Values). Items are rated on two four-point scales (Competence: 1 = ‘I have a big problem doing this’ to 4 = ‘I am really good at doing this;’ Values: 1 = ‘Not really important to me’ to 4 = ’Most important of all to me').66 The COSA has been shown to be reliable and to have good content, structural and substantive validity.67 68 For the present study, 10 items with particular face validity for children attending school (eg, ‘finish my work in class on time’) were selected and administered using a card-sort version of the assessment. This measure enables children to express their opinion regarding their performance on and value of a range of self-regulation activities typically demanded by the school environment. This will provide information to local teachers and occupational therapists about students’ self-rated competency at school and could lead to greater understanding of student strengths and weaknesses faced in the classroom.

Assessment battery in relation to FASD

Multiple outcome measures chosen for this study have previously been incorporated into FASD intervention studies or reported in papers describing the neurobehavioural profiles of children with FASD. These include use of the BRIEF,17 25 39 ECBI,18 Tower of London and Wisconsin Card Sorting Tests,18 25 29 69 Go/No-Go Task70 and variations of the Match to Sample Task.29

Identification of FASD

Parents/caregivers of children participating in the study were asked to provide consent for researchers to access their child’s medical records after the intervention but prior to data analysis. These records will be used to identify if the child has previously been diagnosed with FASD. To improve identification of children with FASD who may not have been previously diagnosed, children who receive a score at or above the clinically elevated cut-off level for their age range and gender on both the BRIEF2 Screening Form (parent and/or teacher) and the ECBI and/or SESBI-R are being referred through the local child health services for developmental screening and/or assessment for FASD by a multidisciplinary team with parent/caregiver consent. Due to clinical service limitations in these remote communities, not all children will necessarily receive a FASD diagnostic assessment.

Data collection

Student and teacher data were collected between April 2016 and December 2017, according to the schedule outlined in table 1. Student medical records will be accessed and reviewed between May and July 2018. For the intervention condition, data collection took place for 2 weeks immediately before and after the intervention. A follow-up assessment was conducted 7 to 8 weeks after completion of the intervention. For control conditions in clusters two to four, the control data collection matched that of the data collection for the intervention condition in the preceding cluster. Students completed assessments during class time and were facilitated by a research team member. Parents/caregivers were visited by a research assistant and an Aboriginal community researcher at their home to complete the BRIEF2 and ECBI. Parent questionnaires were translated into Kimberley Kriol and read to respondents when required to aid in comprehension of the questionnaires. Students or parents/caregivers who were unable to complete assessments at the initial point of contact were followed up within the 2-week period. Teachers were provided the BRIEF2 and SESBI-R to complete during the 2-week data collection period.

Process evaluation

School staff knowledge, attitudes and practices in relation to student behaviour, teaching and learning, and Alert Program key concepts, training and curriculum were measured via questionnaires, focus groups and lesson reflections. Questionnaires designed by the study team were conducted pre-training and post-training and at 7 weeks follow-up once the intervention was completed in each school. Focus groups were conducted with school staff at 8 weeks post-intervention to provide researchers with additional process-based data. Teachers were invited to complete a series of written reflections after each Alert Program lesson explaining whether the lesson aim was met and to provide researchers with information about the fidelity of lesson implementation.

Statistical analysis

Analysis will be undertaken following intention-to-treat principles, with children’s allocation to cluster determined by the school at which they were enrolled at the time of recruitment. Sample and baseline characteristics will be reported. All continuous variables will be summarised using standard measures of central tendency, with mean and standard deviation or median and interquartile range. All categorical variables will be summarised with frequencies and percentages.

Post and follow-up outcomes data will be compared with baseline (control) to determine the average intervention effect on the outcome. The outcome variables will be analysed using generalised linear mixed effects models (GLMM) with gamma errors and will include the following: (a) fixed effects for age, sex, and the interaction between intervention and time; (b) random effects for individuals nested within cluster and (c) an exchangeable correlation structure. This approach will analyse at the individual level while adjusting for clustering, consider both within-cluster and between-cluster comparisons, and adjust for systematically different observation periods.71–73 Missing data will not be imputed as GLMM is robust even when data are missing; subjects will be included in the models if they have at least two data points, with one being prior to the intervention and one being after the intervention. No adjustment to the significance level of 5% will be made for the primary outcome. All secondary outcomes will be adjusted for multiple testing using an appropriate false discovery rate.

Secondary analyses may include the following: a fixed effect for the interaction of FASD with intervention; a fixed effect for lesson attendance and, for PEBL outcomes, a fixed effect for assistance required. All analyses will be carried out using IBM SPSS Statistics 23 or above and the statistical software package Stata 15.1or above. All results will be reported according to Consolidated Standards of Reporting Trials (CONSORT) guidelines.74

Ethics and dissemination

Ethics approval has been granted by the University of WA (approval number RA/4/1/7234), the WA Aboriginal Health Ethics Committee (approval number 601) and the WA Country Health Service (approval number 2015:04). Approval was also received from the Kimberley Aboriginal Health Planning Forum Research Sub-Committee and the WA Department of Education. The results will be disseminated by the research team in partnership with Aboriginal community researchers to local governance structures, including the Marulu FASD Leadership Team, Fitzroy Valley Futures Forum, Kimberley Education Regional Office and Kimberley Population Health Unit. A presentation of results will also be offered to schools participating in the study and through a series of community forums. A written report will be provided to the WA Department of Education. The results will be submitted for publication in peer-reviewed scientific journals and presented at international scientific conferences.

Discussion

This study will contribute substantively to a growing, though limited, body of evidence evaluating the effectiveness of the Alert Program for improving neurobehavioural outcomes of children with FASD. This innovative population-based trial is nested within the broader Fitzroy Valley community-led Marulu FASD strategy.1 The Alert Program therapy intervention was conceived as a direct response to high rates of FASD identified in school children in the Fitzroy Valley. As such, it represents a responsible approach to research whereby a long-term approach is taken to implement a response to the high burden of impairment documented through epidemiological studies.1 12 The study has strong support from participating communities and schools following an extensive period of community consultation with feedback extensively incorporated into the research design to collaboratively ensure that the study is culturally and regionally appropriate.47 Data from the Lililwan FASD prevalence study also highlight the extent of poor neurobehavioural outcomes for children in this population and the need to provide interventionist support.

A pilot study and stakeholder input have provided valuable information about culture and context that have subsequently informed the Alert Program intervention as described in this protocol. Having the intervention developers involved in the manualising and adaptation of their Alert Program is a strength of the current study. This intervention targets (and provides more information about) known developmental challenges in the target population, addressing the importance of the domains of executive function and self-regulation to children’s outcome and occupational performance. In addition, should the current study demonstrate beneficial findings, both real-world sustainability and a dissemination plan have been proactively considered in the study design and implementation. This is likely to ensure strong adherence to the Alert Program curriculum by teachers and maintenance of the intervention after the research period ends. Involvement of Aboriginal community researchers in this project is ensuring culturally safe and contextually relevant research is being undertaken. Furthermore, training of Aboriginal community members as researchers is building local expertise and capacity.

Community and stakeholder feedback during the formative stage of this project led to a modification of the research study design. This included reducing the number of data collection time points and choosing outcome measures that could be completed by participants in a reasonable time frame (≤30 min). These changes should increase participant retention rates.47 A study limitation is the highly mobile student population. This could lead to contamination between clusters and to variable student attendance that could affect treatment dose of the intervention. Because the intervention has been rolled out over a 2-year period, there is a potential for contamination between schools/clusters if teachers and other school staff have talked to each other about the Alert Program. The inability to blind researchers involved in the measurement of outcomes will mean results must be interpreted cautiously. Normative data for Australian Aboriginal populations are also lacking for study outcome measures, as is true for many psychometric instruments. Limited FASD diagnostic capacity in the region has led to an inability to assess all children within the study for FASD, instead relying on a medical record review of participating children to determine if they have been diagnosed with FASD. Where diagnoses have been made, parents and teachers may not be blinded to whether a child has FASD. An under-representation of children with FASD may occur given not all children in the study may have undergone a FASD assessment.

By building the capacity of teacher and school support staff to deliver the Alert Program to students, an important public health goal is that children in the Fitzroy Valley will improve their executive functioning and self-regulation skills. Evaluation of the effectiveness of this school-based Alert Program could support future school and health sector policy decisions in the Fitzroy Valley. If shown to be effective, there is also potential to translate the program to additional schools where similar problems exist. Ultimately, this study aims to evaluate a culturally and contextually appropriate program that will support children, families and teachers who are impacted by FASD, and other psychosocial factors, in a way that can contribute to overcoming some of the challenges of remote service delivery.