The Irie Classroom Toolbox, a universal violence-prevention teacher-training programme, in Jamaican preschools: a single-blind, cluster-randomised controlled trial.

BACKGROUND: Violence is a leading global public health problem, and interventions in early childhood are important in the primary prevention of violence. We tested whether the Irie Classroom Toolbox, a violence-prevention teacher-training programme reduced violence against children by teachers and reduced class-wide child aggression in Jamaican preschools (catering to children aged 3-6 years).
METHODS: We did a single-blind, cluster-randomised controlled trial in 76 preschools in Kingston and St Andrew, randomly selected, using simple randomisation, from 120 eligible preschools. Inclusion criteria were two to four classes of children; at least ten children per class; and located in an urban area. We randomly assigned preschools (1:1) to either the Irie Classroom Toolbox intervention or waiting-list control that received no intervention, using a computer-generated randomisation sequence by an independent statistician masked to school identity. The Toolbox involved training teachers in classroom behaviour management and promoting child social-emotional competence. All assessors were masked to group assignment. All teachers and classrooms in the selected schools participated in the study. Within each school, we used simple randomisation to randomly select up to 12 children aged 4 years for evaluation of child outcomes. The Toolbox intervention was implemented from August to April the following year. Teacher and classroom measures were done at baseline (the summer school term; ie, May to June), post-intervention (after 8 months of intervention; ie, May to June of the following year), and 1-year follow-up (ie, May to June 2 years later). The primary outcomes were observations of violence against children (including physical violence and psychological aggression) by teachers occurring across one full school day, and class-wide child aggression occurring over five 20-min intervals on another school day, all measured at post-intervention and 1-year follow-up and analysed by intention to treat. This trial is registered with ISRCTN, number ISRCTN11968472.
FINDINGS: Between June 22, 2015, and April 29, 2016, (after baseline measurements were completed), we assigned 38 preschools (with 119 teachers) to the Toolbox intervention and 38 preschools (with 110 teachers) to control. 441 children in the intervention schools and 424 in the control schools were included in the evaluation. All schools were included in the post-intervention and follow-up analyses. There were fewer counts of violence against children by teachers in the intervention schools compared with control schools at post-intervention (median counts 3 [IQR 0-11] vs 15 [3-35]; effect size -67·12%, 95% CI -80·71 to -53·52, p<0·0001) and 1-year follow-up (median counts 3 [IQR 0-9] vs 6 [1-16]; effect size -53·86, 95% CI -71·08 to -36·65, p<0·0001). No differences between groups were found for class-wide child aggression at post-intervention (effect size 0·07, 95% CI -0·16 to 0·29, p=0·72) or 1-year follow-up (-0·14, -0·42 to 0·16, p=0·72).
INTERPRETATION: In Jamaican preschools, the Irie Classroom Toolbox effectively reduced violence against children by teachers. The Toolbox was designed for use with undertrained teachers working in low-resource settings and should be effective with early childhood practitioners in other LMICs. Additional research is needed to further develop the Toolbox to reduce class-wide child aggression. FUNDING: Medical Research Council, Wellcome Trust, UK Aid, and the National Institute of Health Research.

RCT Entities:   Population Interventions Outcomes

Introduction

More than 50% of all children aged 2 to 17 years experience physical, emotional, or sexual violence each year globally. Schools are an important context for violence-prevention initiatives because of their population reach and the high proportion of time that children spend there. School-based programmes can target multiple types of violence including bullying, peer aggression, and violence against children by teachers, but there is little evidence of their effect in low-income and middle-income countries (LMICs).2, 3

Violence against children by teachers is prevalent in schools across the world and is linked with mental health and behavioural problems, physical injury, and poor academic achievement. The few trials of school-based, violence-prevention programmes with quantitative measures of teachers' use of violence against children have been done in primary and secondary schools.5, 6, 7 Early-onset conduct problems are the most common mental health concerns in children and predict juvenile delinquency, and crime and violence in adulthood; these disruptive behaviour problems are reduced through early preventive interventions. However, few trials have been done in LMICs, especially in the pre-primary years.3, 9, 10

Evidence before this study

There are no systematic reviews of interventions to prevent violence against children by school staff. We searched the Cochrane and Psychinfo databases from the first record to May 1, 2020 using the terms “corporal punishment', “physical punishment”, “violence against children”, “physical discipline”, “violence”, and “school”. We found three randomised trials that assessed teachers' use of violence against children, one done in secondary schools and two in primary schools. The most rigorous trial was an evaluation of the Good School Toolkit in Ugandan primary schools that found a 42% reduction in past-week violence against children by school staff in the group allocated to the toolkit.

A systematic review of interventions for disruptive behaviour problems in low and middle-income countries (LMIC) identified three school-based trials published up to November 2016, targeting preschool age children. We searched the Cochrane and Psychinfo databases to identify school-based, randomised trials with preschool-age children in LMIC from Nov 12, 2016, to May 1, 2020, with the terms “aggression”, “disruptive behaviour”, “conduct problems”, “externalising disorders”, and “school”. One further trial from Uganda was identified. Three of the four trials were small-scale and done in ten schools or fewer. The largest study was an efficacy trial in 24 Jamaican preschools that reported reductions in behavioural problems of preschoolers at high risk for antisocial behaviour by independent observation and reports from teachers and parents.

Added value of this study

This is the first study of a preschool violence-prevention programme with direct measurements of teachers' use of violence against children and the first trial in any school setting to include follow-up measurements of this outcome. The teacher-training programme led to large and sustained reductions in teachers' use of violence against children. We found no benefits with this programme on child aggression at the classroom level. However, the intervention led to benefits for a wide range of secondary outcomes at the level of (1) the classroom (ie, the quality of the classroom environment and class-wide child pro-social behaviour); (2) the teacher (ie, teachers' wellbeing and retention); and (3) the children (ie, child inhibitory control and mental health of high-risk children). The results inform the provision of in-service training initiatives for preschool teachers in LMIC to prevent child maltreatment and improve child functioning.

Implications of all the available evidence

School-based violence-prevention programmes implemented in LMIC are an effective strategy to reduce teachers' use of violence against children. Further research is required to identify a broader set of strategies to eliminate child maltreatment in schools. Research is also needed on how to fully integrate school-based interventions into ongoing pre-service and in-service teacher training and other school improvement initiatives with sufficient quality to maintain effectiveness, and to identify how the changes to teacher behaviour can be sustained over time.

The Irie Classroom Toolbox is a universal, early childhood, violence-prevention teacher-training programme designed for use in LMICs, that aims to reduce violence against children by teachers and prevent the early development of antisocial behaviour in children aged 3–8 years. The Toolbox also aims to improve the quality of the classroom environment and to promote child mental health, self-regulation, and prosocial skills. In this study we evaluated the effectiveness of the Irie Classroom Toolbox in a cluster-randomised trial in inner-city Jamaican preschools.

Methods

Study design and participants

We did a two-arm, single-blind, cluster-randomised trial with parallel assignment in community preschools located in disadvantaged areas of Kingston and St Andrew, Jamaica. The preschools are run through community organisations, with government oversight, for children aged 3 to 6 years living in the locality. Parents pay a small fee and provide school supplies. Preschool was the unit of randomisation to prevent contamination among teachers. Inclusion criteria for preschools were two to four classes of children, at least ten children per class, situated in urban areas of Kingston and St Andrew, and all teachers consenting to participate in the trial. We assessed 173 preschools for eligibility and 120 preschools met the inclusion criteria; 76 of these eligible preschools were randomly selected to participate in the study using simple randomisation with a computer-generated random number sequence (figure). Within each school, we used simple randomisation (with a random number table) to select up to 12 children aged 4 years for evaluation of child outcomes. In preschools with 12 or fewer children aged 4 years and in schools with 13 eligible children, all children were recruited. We selected children in the 4-year-old classes only because children in the 5-year-old classes were transitioning to primary school before the start of the intervention and some children in the 3-year-old class transition to government preschools at age 4 years. Inclusion criteria for children were no absences totalling more than 1 month in the past term, attends school at least 3 days per week, no obvious disability, and parental consent. Ethical consent for the study was given by the Ethics Committees of the School of Psychology, Bangor University (2014-14167) and the University of the West Indies (ECP 50, 14/15). The trial protocol has been published. Written, informed consent was obtained from preschool principals, all teachers, and the parents of the selected children. Child assent was not obtained because of their young age.

Figure

Trial profile

Preschools with children aged 3 to 6 years. ITT=intention to treat. *3 schools declined to participate because of current or imminent leadership changes. These schools were replaced by the next school on the randomisation list. †Up to 12 children aged 4 years randomly selected. ‡47 children were excluded and were replaced by the next randomly selected child: 23 for poor attendance, 21 left school between recruitment and testing, and three had a disability.

Randomisation and masking

Preschools were randomly assigned to the Irie Classroom Toolbox intervention or waiting-list control, that did not receive the intervention, using a computer-generated simple randomisation sequence by an independent statistician masked to school identity. All preschools, teachers, and children were recruited, and all baseline measures were done before randomisation.

Data collectors were masked to the group assignment at post-intervention and follow-up with the following steps: (1) they were not informed that this was an intervention trial and were unaware that some teachers had received additional training; (2) all study schools received the same educational materials; (3) where data collectors were employed for more than one round of data collection, they were rotated across schools and classrooms so they observed different classrooms each time; and (4) teachers were asked not to reveal intervention status. Furthermore, through the intervention, teachers were encouraged to choose strategies according to their individual preferences, classroom context, and the needs of the children; thus, we expected substantial variation among teachers making group allocation less obvious. The statistical analyses were done by the trial statistician who was masked to group assignment. All decisions related to the analyses were taken without seeing the estimated effects of the intervention.

Procedures

The Irie Classroom Toolbox intervention involved training teachers over two school terms (ie, 8 months) through five full-day, teacher-training workshops; eight 1-h sessions of in-class support; and fortnightly text messages. Teachers were trained in classroom behaviour management and promoting child social-emotional competence (panel). The Toolbox does not explicitly challenge teachers' attitudes towards violence against children. Rather, the theory of change suggests that the Toolbox reduces violence against children by helping teachers to gain skills, motivation, and opportunity to use positive discipline techniques. Teachers in control schools participated in the full Irie Classroom Toolbox training when the 1-year follow-up measurements were completed. All study schools were given a gift of construction toys, toy cars and animals, and a set of storybooks after each round of measurements.

Content and materials

The Irie Classroom Toolbox consists of four modules:

creating an emotionally supportive classroom environment;

preventing and managing child behaviour problems;

teaching social and emotional skills; and (4) individual and class-wide behaviour planning.

Intervention materials for teachers include (1) a tools book that provides simple guidelines on how to use each strategy and the underpinning rationale; (2) an activity book of songs, games, activities, lesson plans, and behaviour planning forms; (3) three sets of picture cards to help teachers teach classroom rules, friendship skills, and understanding emotions; and (4) a problem-solving stories book consisting of 14 pictorial stories depicting common classroom problems that children encounter in school, and strategies that children can use to overcome them (eg, how to work together as a team, and how to share classroom materials). All materials are available online.

Procedures

The teacher-training workshops include the use of brainstorms, live and videotape modelling, role-play and rehearsal, discussions, card-sorting activities, and small group activities. In our study, five full-day workshops were undertaken, and full details of the content and process for each workshop were provided in a training manual. Eight in-class support sessions were designed to support the content covered in workshops. Guidelines for the in-class support sessions were given in a coaching manual. All sessions included three steps: (1) a brief 5-min planning discussion with the teacher; (2) supporting the teacher in the classroom by modelling the strategies, prompting the teacher to use them, providing supportive feedback, and helping the teacher to problem-solve (around 45 min); and (3) debriefing and goal-setting (around 10 min). Teachers also received practical classroom assignments to be completed after each in-class support session. Fortnightly phone text messages were sent to all intervention teachers using an SMS service.

Who provided the Toolbox intervention?

Eight female staff assisted in delivery of the intervention; when recruited, all staff had a Masters' degree but no or little experience in child-behaviour management and early childhood education. Each teacher-training workshop was done by one facilitator and one co-facilitator who received 2–3 days of training for each day of workshop delivered (a total of 12 days). Teachers were divided into four groups, based on geographical location, and workshops were done with groups of 24–35 participants. The four co-facilitators of the workshops provided in-class support, and each was responsible for 9–10 preschools (around 30 teachers). The co-facilitators were supervised by two of the workshop facilitators who supported them in field visits and met with them individually approximately once a month to discuss the progress of each teacher. Weekly group supervision meetings lasting around 1 h were held to problem-solve common issues faced and to discuss and practice each session before implementation.

Further details for the Toolbox intervention

Teacher training workshops were done in community centres (eg, preschools and church halls) in urban areas of Kingston, Jamaica. Teachers were given a small stipend to cover transportation costs (US $4 per workshop) and lunch was also provided.

Teachers attended five full-day (6 h) workshops over one school year. 3 days of workshop were done in the summer holiday before the start of term, one workshop day was held in the half-term break in October, and the remaining workshop day was held in the half-term break in February. In-class support was provided to each intervention teacher once per month for 8 months from September to April, for approximately 1 h each session. Teachers in the intervention schools were offered one additional training workshop in the summer holiday as a refresher before the start of the next school year; 50% of teachers opted to attend this additional workshop.

The intervention was manualised and largely implemented in a standard way with all teachers. Workshops were fully scripted and done as planned with minimal modifications. All teachers were offered the same number of in-class support sessions. During these sessions, the content was the same for all teachers, although teachers would receive different types of assistance in the classroom depending on their needs, competence, and engagement with the intervention.

The workshop facilitators completed a training protocol and self-evaluation questionnaire after each workshop; all of the prescribed workshop content was covered. Teachers rated their opinion of the content, videos, facilitator skills, group discussion, role plays or demonstrations, and small-group work on a six-point scale from not at all helpful (0) to extremely helpful (5). The mean score across all workshops was 27·1 (out of a maximum score of 30), showing a high level of teacher satisfaction with the workshops. 52 teachers (48%) in the intervention group attended all five workshops; 98 (91%) attended three or more. Facilitators similarly covered all prescribed content in the in-class support sessions; 100 teachers (93%) participated in all eight in-class support sessions; 105 (97%) participated in seven or more. Teachers were given classroom assignments after the first seven in-class support sessions. 53 teachers (49%) did four or more of these assignments; 89 (82%) did two or more.

The study involved observations of teachers' use of violence. Corporal punishment is against the law in Jamaican early childhood institutions; however, the law is not enforced and corporal punishment is widespread.11, 13 During the study, we followed the reporting requirements of the Jamaican Child Protection Act that mandates reporting of severe corporal punishment. Severe corporal punishment is rare in preschools and no instances were observed.

Outcomes

The primary outcome measures were observations of violence against children by teachers and class-wide child aggression. Violence against children by teachers was a count of teachers' use of physical violence and psychological aggression across one full school day. Class-wide aggression was measured through ratings of the frequency, intensity, and proportion of children engaged in aggressive behaviour over five 20-min intervals on another school day (table 1). Secondary outcomes were measures collected through observation, teacher report, and direct child testing (table 1). Observational measures were (1) class-wide child prosocial behaviour, (2) the quality of the classroom environment, and (3) a binary measure of violence against children by teachers over 2 school days. Teacher-report measures were (1) teacher wellbeing; and (2) child behaviour difficulties and prosocial skills for the selected children. Direct tests were used to measure child inhibitory control. Child attendance from school records was also included as a secondary outcome in the protocol. However, classroom registers were incomplete, and the data are not presented.

Table 1

Description of outcome variables measured by observation, teacher questionnaire, and direct child testing

		Measures used
Primary observational outcomes (measured at baseline, post-intervention, and 1-year follow-up)
Teachers' use of violence against children (count)*		Continuous observations of teachers' behaviour over one school day; event sampling was used to code each discrete act of violence against children; total score is the sum of teachers' use of violence in all three categories
	Physical punishment	Hitting with hand or object, forcefully pushing or pulling, shaking, pinching, poking, or making the child stand or kneel in uncomfortable positions
	Verbal abuse	Calling the child by a derogatory name (eg, idiot or dummy), threatening physical punishment, threatening a child in a way that would frighten them (eg, threats to call the police), encouraging children to harm, insult, or exclude a child (eg, encouraging a child to hit another child)
	Other abuse	Intimidation (eg, banging a stick with force on the desk in front of a child), non-verbal threats (eg, using a stick or ruler to threaten child)
Class-wide child aggression†		Classroom observations over five 20-min periods during one school day; mean score over five observations used in the analysis; observers coded child aggression on a seven-point rating scale (1–7) after each 20-min observation period; the score reflects the frequency, intensity, and number of children involved in aggressive acts. Higher scores indicate more aggression
Secondary observational outcomes (measured at baseline, post-intervention, and 1-year follow-up)
Quality of classroom environment14		Classroom observations over five 20-min periods during one school day: mean score over five observations used in the analysis; all scales are scored on a seven-point rating scale (1–7) with higher scores indicating higher quality
	Emotional support‡	Four scales: Positive Climate, Negative Climate, Teacher Sensitivity, and Regard for Student Perspectives (internal reliability 0·74, range 0·71–0·76)††
	Classroom organisation§	Three rating scales: Behaviour Management, Productivity, and Instructional Learning Formats (internal reliability 0·80, range 0·79–0·81)††
	Instructional support¶	Three rating scales: Concept Development, Quality of Feedback, and Language Modelling. (internal reliability 0·61, range 0·59–0·62)††
Class-wide child prosocial behaviour‖		Classroom observations over five 20-min periods during one school day: mean score over 5 observations used in the analysis. Observers coded child prosocial behaviour on a seven-point rating scale (1–7) after each 20-min observation period; the score reflects the frequency, intensity, and number of children involved in prosocial acts (ie, sharing, helping, and cooperating); higher scores indicate more prosocial behaviour
Violence against children by teachers (binary)**		Observations of teacher behaviour over 2 school days (binary variable); we recorded whether teachers used violence during each of the five 20-min intervals used for the class-wide ratings above on one school day, and combined these data with data for whether teachers used violence against children over another school day
Teacher outcomes (measured at baseline, post-intervention, and 1-year follow-up)
Teacher wellbeing		Teacher questionnaire administered in face-to-face interview; composite of three subscales used in the analysis
	Depressive symptoms	Centre for Epidemiological Studies Depression Scale:15 20 questions, potential range of scores 0–60
	Teacher burn-out	Teacher Burn-Out Scale:16 20 questions, potential range of scores 20–100
	Teaching self-efficacy	3 subscales from Bandura's Teacher Self-Efficacy Scale:17 disciplinary self-efficacy, efficacy to enlist parent involvement and efficacy to create a positive school climate: 16 questions, potential range of scores 16–112
Child outcomes (measured at baseline and post-intervention)
Inhibitory control		Direct child testing; composite score of the three tests were used in the analysis
	Big or little stroop	Children are presented with pictures of big cats and little cats and are told to say ‘little’ when the tester points to a big cat and ‘big’ when the tester points to a little cat; score is the number correct out of a total of 18 items
	Silly sounds stroop	Children are presented with pictures of a cat and a dog and told to say ‘woof’ when the tester points to a cat and ‘meow’ when the tester points to a dog; score is the number correct out of a total of 18 items
	Frog or bear	Children are told to do the actions that the bear tells them to do and not to do the actions that the frog tells them to do; score is the number of times children perform the action suggested by frog out of a total of ten items (higher scores indicate worse inhibitory control)
Child behaviour		Teacher report using the SDQ18 administered in a face-to-face interview
	Behaviour difficulties	SDQ total difficulties: 20 questions, potential range of scores 0–40
	Prosocial skills	SDQ Prosocial Subscale. Five questions, potential range of scores 0–10
	Clinical range for behaviour difficulties	A score ≥16 on the SDQ Total Difficulties Scale=abnormal (1), <16=normal (0)18
	Impact of difficulties on daily life	Generated from the Impact Scale of the SDQ. A score of ≥2=abnormal (1), <2=normal (0)18

SDQ=Strengths and Difficulties Questionnaire. Stability of observational measures over 1 year from baseline to post-intervention of control group only (n=92): intraclass correlation coefficients

0·63,

0·59,

0·48,

0·42,

0·06,

0·13, and

0·53.

Mean (range) for the internal reliability of Classroom Assessment Scoring System Pre-Kindergarten (CLASS Pre-K) domain scores across three rounds of data collection using Cronbach's α.

The Toolbox intervention was implemented from August to April. All teacher and classroom measurements were done at baseline (summer term, May to June), after 8 months of intervention (post-intervention; May to June of the following year), and 1-year follow-up (May to June 2 years later). Individual child measurements were done at baseline and post-intervention only, because children transitioned to primary school after the post-intervention measurements. For teacher and individual child measures, the same participants were measured at each data collection point. For class-wide behaviour, a different class of children was measured at each timepoint because data were collected in the final term of each school year.

The data collection team included four groups of data collectors assigned to do (1) observations of violence against children by teachers (teacher observers), (2) observations of class-wide child behaviour and the classroom environment (classroom observers), (3) child tests (child testers), and (4) face-to-face teacher interviews (teacher interviewers). The data collection team comprised ten teacher observers, ten classroom observers, five child testers, and one teacher interviewer. In addition, there was one senior data collector for each group of data collectors with responsibility for ongoing quality control. The senior data collectors were researchers with previous experience undertaking observations, teacher interviews, or child tests in Jamaican preschools. Each teacher and classroom was observed over 2 school days. On the first day, a classroom observer coded class-wide child aggression and prosocial behaviour, the quality of the classroom environment, and violence against children by teachers over five 20-min periods. On the second day, a teacher observer undertook continuous observations of teachers' use of violence against children across the school day. Child testers and the teacher interviewer were not present in a school at the same time as the teacher and classroom observers. All data collectors did equal numbers of measurements from each group and were rotated across schools.

Training for data collectors was done over a 4-week period at each timepoint including 1 week in office training, 2 weeks field training, and 1 week field reliabilities for each measurement. Training continued until inter-rater reliabilities, using intraclass correlation coefficients (ICCs), were greater than 0·95 for child tests and teacher interviews, greater than 0·90 for teachers' use of violence against children, and greater than 0·80 for all other observational measures. The senior data collectors did ongoing monitoring of reliabilities once per week with each data collector (on 8% of observational measurements, and 10% of child tests and teacher interviews). Reliabilities of ICCs greater than 0·95 on child tests and teacher interviews, greater than 0·90 on violence against children by teachers, and greater than 0·80 on all other observations were maintained. Data were collected over an 8-week period at each timepoint. The data collection period lasted approximately 5 days in each school.

All questionnaire measures had good internal consistency (Cronbach's α median 0·87, IQR 0·83–0·88) and test-retest reliability over 2 weeks (ICC median 0·81, IQR 0·78–0·83) (appendix p 1). Test-retest for the composite score of child inhibitory control was ICC 0·82. For observational measures, we examined the stability (test-retest) over 1 year using data from the control group only. Stability was good for the primary outcomes of violence against children by teachers (ICC 0·63) and class-wide child aggression (ICC 0·59), and acceptable (ICC >0·40) for the secondary observational outcomes, except for class-wide prosocial behavior and instructional support which was poor (ICCs 0·13 and 0·06, respectively) (table 1).

Statistical analysis

In a previous efficacy trial in Jamaican preschools, we noted mean effect sizes for classroom ratings and individual child outcomes of 0·75 and 0·50 respectively. In this larger effectiveness trial, we hypothesised effect sizes of 0·50 for classroom-level outcomes and 0·25 for child-level outcomes. These hypothesised effect sizes are also of similar magnitude to those reported in meta-analyses of early childhood development programmes; ie, 0·39, 0·44, and 0·26 for classroom-level, teacher-level, and child-level outcomes, respectively. This study was powered to detect differences in the primary outcomes of counts of violence against children by teachers through observation, and ratings of the level of class-wide child aggression, at post-intervention and 1-year follow-up. Using the Bonferroni method, we required a significance level of 0·0125 instead of 0·05 for each primary outcome. Minimum detectable effects were computed assuming an ICC of 0·10 for violence against children by teachers and 0·05 for class-wide child aggression, guided by efficacy trial data.9, 20 With 80% power and allowing for a loss of two schools per group, giving 36 schools in each group with three teachers per school (108 teachers), the minimum detectable effects were a reduction of 0·50 SD in class-wide child aggression and a reduction of 10·31% in the counts of violence against children by teachers. For child-level measures, with 36 schools in each group, and assuming an average of 11 children per school, 80% power, 0·05 significance, and an ICC of 0·05, the minimum detectable effect for continuous outcomes was 0·25 SD, and for child behaviour difficulties in the clinical range, a reduction from 15% of children to 7% of children.

Analyses followed a prespecified protocol that was approved by a trial steering committee. A negative binomial multilevel mixed-effects model was used in the analyses of the count data of violence against children by teachers over 1 day (details for data distribution are in the appendix p 1). The choice between Poisson and negative binomial was based on the estimate of the overdispersion parameter of the negative binomial and the fit assessed using a χ2 goodness-of-fit test. The binary outcomes were analysed using multilevel mixed-effect logit models. A multilevel mixed-effect linear model was used to estimate the effect of the intervention on all other outcomes. The residuals of linear models were tested for normality and the dependent variable (and its baseline value) transformed if normality was rejected. Class-wide aggression, prosocial behaviour, and instructional support were log-transformed; teacher-reported child behaviour difficulties and prosocial behaviour and child inhibitory control were transformed using a Box-Cox transformation.

For teacher wellbeing (depression, self-efficacy, and burnout), depression and burnout were reverse coded and exploratory factor analyses gave one factor (appendix p 1); factor scores were used in the analyses. For the child inhibitory control tests (big or little stroop, silly sounds stroop, and the frog or bear test), the frog score was reverse coded and exploratory factor analysis produced one factor (appendix p 1). The raw scores of the three scales were summed to create a formative indicator of inhibitory control. Before summing, the frog score was multiplied by 18/10 to put it on the same scale as the other two tests. Random effects were included at the school level in all analyses. Fixed effects in the model for teacher or classroom outcomes included a constant term, the value of the outcome variable at baseline, group assignment, and covariates for which there were theoretically important differences (rather than statistically significant differences) between treatment groups or that were significantly different between those lost to analysis and those who continued. The set of variables considered were (1) number of children in the class at baseline, (2) number of years teaching, (3) number of years teaching in the school, (4) high school complete, (5) qualified teacher, and (6) primary or secondary baseline outcome variable. The following covariates were included in all regressions of teacher and classroom outcomes: number of years teaching in current school, high school completed, qualified teacher, and number of children in the class. Child-level regressions included age and sex as fixed effects.

All models were estimated using multiple imputation to adjust for loss to post-intervention and 1-year follow-up as well as missing in the baseline value of the dependent variable (one teacher and one child). Imputation models included group assignment, baseline level of the outcomes, and the same covariates as the main models and were estimated using 20 imputations (appendix p 2). In sensitivity analyses, the imputation models were expanded to include interactions between the covariates and group assignment to allow for differential loss.

The p values of the four primary outcome measures were adjusted using Holm step-down procedure to control for multiple hypothesis testing. Effect sizes for continuous outcomes were calculated by dividing the regression coefficient by the SD of the control group for each dependent variable. The analyses of the secondary outcomes were considered exploratory and thus no adjustments were made for multiple outcomes. Statistical analyses were done with STATA, version 15.1. This trial is registered with ISRCTN, number ISRCTN11968472.

Role of the funding source

The funders of the study had no role in study design, data collection, data analysis, data interpretation, writing of the report, or the decision to submit for publication.

Results

Between June 22, 2015, and April 29, 2016, after baseline measurements were completed, we assigned 38 preschools (with 119 teachers) to the Toolbox intervention and 38 (with 110 teachers) to control. All preschools were followed up at post-intervention and 1-year follow-up (figure). Because of teacher turnover, 29 teachers (13%) were lost at post-intervention (11 intervention, 18 control) and 39 teachers (17%) were lost at 1-year follow-up (14 intervention, 25 control). 865 children were recruited; 441 in the Toolbox intervention schools and 424 in the control schools (figure). 82 children selected for evaluation (9%) were lost at post-intervention (43 intervention, 39 control); reasons for loss are in the figure. There were no significant differences between teachers and children lost and those retained on any of the baseline outcome measures (appendix p 3). However, teachers lost had been teaching at their current school for fewer years than those retained. Compared with intervention-assigned teachers, more of the control teachers lost at post-intervention had completed high school (17 [94%] control vs seven [64%] intervention) and more control teachers lost at post-intervention and at follow-up were trained teachers (11 [61%] control vs two [18%] intervention; and ten [40%] control vs one [7%] intervention, respectively). There were no significant differences between the study groups on child and teacher characteristics. Classrooms in intervention schools had significantly more children than classrooms in control schools (table 2).

Table 2

Baseline characteristics of schools, classrooms, teachers, and children

			Irie Classroom Toolbox	Control
School characteristics			n=38	n=38
	Number of children enrolled		56·8 (24·6)	48·24 (17·8)
Classroom characteristics			n=119	n=110
	Number of children in class		17·8 (7·0)	15·4 (5·8)
Teacher characteristics			n=119	n=110
	Total number of years teaching		17·0 (8·0–23·0)	14·0 (8·0–23·3)
	Number of years teaching at current school		12·0 (5·0–20·0)	9 (4·0–20·0)
	Completed secondary school (grade 11)		95 (80%)	97 (88%)
	Trained teacher		41 (35%)	40 (36%)
	Currently attending teacher-training college		16 (13%)	12 (11%)
	Sex
		Female	116 (97%)	108 (98%)
		Male	3 (3%)	2 (2%)
Child characteristics			n=441	n=424
	Age (years)		4·9 (0·3)	4·9 (0·4)
	Sex
		Female	202 (46%)	211 (50%)
		Male	239 (54%)	213 (50%)

Data are mean (SD), median (IQR), or n (%).

Intervention teachers attended a mean 4·11 full-day workshops (SD 1·19) and participated in a mean 7·89 in-class support sessions (0·56). All prescribed content was covered, and teachers reported high levels of satisfaction with the intervention (panel).

There were no significant baseline differences between the groups for the raw data for all outcomes (Table 3, Table 4). At baseline, there were a median seven instances of violence against children over one school day in both groups (IQR 2–15 Toolbox, 1–18 control, respectively); 27 teachers (12% of all teachers) used no violence over 2 days of observation. The most commonly used form of violence was physical violence, which usually involved hitting a child with the hand. Threatening children with physical punishment was the most common form of verbal abuse. Scores for observed class-wide child aggression and class-wide prosocial behaviour were in the low range. Scores for quality of the classroom environment were in the mid-range for emotional support and classroom organisation, and in the low range for instructional support. 152 children (18%) were in the abnormal range for behaviour difficulties by teacher report.

Table 3

Raw data for primary and secondary teacher and classroom outcomes at baseline, post-intervention, and 1-year follow-up

	Irie Classroom Toolbox			Control
	Baseline (n=119)	Post-intervention (n=108)	1-year follow-up (n=105)	Baseline (n=110)	Post-intervention (n=92)	1-year follow-up (n=85)
Counts of observed violence against children by teachers across 1 school day
Physical violence	4 (0–11)	2 (0–9)	0 (0–3)	3 (0–12)	11 (0–26)	3 (0–7)
Verbal abuse	1 (0–3)	0 (0–1)	0 (0–2)	1 (0–4)	2 (0–4)	2 (0–5)
Other violence	0 (0–2)	0 (0–1)	0 (0–0)	0 (0–1)	1 (0–6)	0 (0–1)
All violence	7 (2–15)	3 (0–11)	3 (0–9)	7 (1–18)	15 (3–35)	6 (1–16)
Binary measure of observed violence against children by teachers across 2 school days
No physical violence	21 (18%)	30 (28%)	44 (42%)	27 (25%)	16 (17%)	27 (32%)
No verbal abuse	40 (34%)	56 (52%)	44 (42%)	37 (34%)	21 (23%)	22 (26%)
No other violence	51 (43%)	67 (62%)	73 (70%)	52 (47%)	37 (40%)	51 (60%)
No total violence	12 (10%)	19 (18%)	23 (22%)	15 (14%)	8 (9%)	14 (16%)
Rating scales of class-wide child behaviour
Child aggression	2·4 (1·6–4·0)	3·2 (2·2–4·4)	2·6 (1·6–3·4)	2·4 (1·6–3·8)	3·0 (1·8–4·8)	2·8 (1·6–4·1)
Child prosocial behaviour	1·8 (1·4–2·2)	2·4 (1·8–3·0)	2·2 (1·6–2·6)	1·8 (1·2–2·3)	2·0 (1·5–2·6)	1·8 (1·6–2·6)
Rating scales of the quality of the classroom environment using the CLASS Pre-K scales
Emotional support	4·2 (0·8)	4·4 (0·6)	4·1 (0·8)	4·3 (0·7)	3·9 (0·9)	3·7 (0·8)
Classroom organisation	4·8 (0·8)	4·7 (0·8)	4·5 (0·8)	4·9 (0·8)	4·4 (0·9)	4·3 (0·8)
Instructional support	1·4 (1·3–1·5)	1·5 (1·4–1·7)	1·4 (1·3–1·6)	1·4 (1·3–1·6)	1·4 (1·3–1·6)	1·3 (1·2–1·5)
Teacher wellbeing
Depression	14 (8–21)	10 (6–18)	10 (5–18)	12 (6–19)	12 (7–19)	12 (5–19)
Burn-out	28 (23–35)	28 (23–34)	27 (22–36)	28 (23–36)	29 (23–38)	31 (23–36)
Teaching self-efficacy	92 (82–98)	92 (84–100)	97 (88–102)	90 (83–96)	90 (81–98)	92 (85–100)

Data are median (IQR), n (%), or mean (SD).

Table 4

Raw data for primary and secondary individual child outcomes at baseline and post-intervention

				Irie Classroom Toolbox		Control
				Baseline (n=441)	Post-intervention (n=398)	Baseline (n=424)	Post-intervention (n=385)
Individual child outcomes
	Child inhibitory control
		Big or little stroop score		12 (8–16)	15 (11–17)	12 (8–16)	14 (10–17)
		Silly sounds score		9 (9–11)	11 (9–14)	9 (9–11)	10 (9–13)
		Bear or frog score		3 (0–9)	1 (0–2)	2 (0–8)	1 (0–2)
		SDQ scores
			Behaviour difficulties	9 (5–13)	7 (4–11)	10 (6–14)	7 (4–12)
			Prosocial behaviour	7 (5–9)	8 (6–10)	7 (5–9)	8 (6–10)
	Child in clinical range for behaviour difficulties on the SDQ scale
		Behavioural difficulties in abnormal range*		75 (17·0%)	40 (10·1%)	77 (18·2%)	65 (16·9%)
		Impact on daily living in abnormal range†		53 (12·0%)	41 (10·3%)	59 (13·9%)	63 (16·4%)

Data are median (IQR) or n (%). SDQ=Strengths and Difficulties Questionnaire.

Teacher reported SDQ difficulties scale ≥16.

Teacher reported SDQ impact score ≥2.

In the Toolbox intervention group there were significantly fewer counts of violence against children by teachers over one school day compared with the control group at both post-intervention and 1-year follow-up (post-intervention: Toolbox group median counts 3 [IQR 0–11] vs control group median counts 15 [3-35]; effect size −67·12%, 95% CI −80·71 to −53·52, p<0·0001; 1-year follow-up: Toolbox median counts 3 [IQR 0–9] vs control median counts 6 [1-16]; effect size −53·86%, 95% CI −71·08 to −36·65, p<0·0001; table 5). There were no differences between the groups in class-wide child aggression at post-intervention (effect size 0·07, 95% CI −0·16 to 0·29), or 1-year follow-up (effect size −0·14, −0·42 to 0·16; table 5).

Table 5

Intention to treat analyses of the Irie Classroom Toolbox on primary and secondary outcomes at post-intervention and 1-year follow-up*†

		Post-intervention				1 year follow-up
		Regression coefficient B (95% CI)	ICC	Effect size‡ (95% CI)	p value	Regression coefficient B (95% CI)	ICC	Effect size‡ (95% CI)	p value
Primary outcomes
Violence against children by teachers (number of times)		−1·11 (−1·53 to −0·70)	0·05	−67·12%§ (−80·71 to −53·52)	<0·0001¶	−0·78 (−1·15 to −0·40)	0·00	−53·86%§ (−71·08 to −36·65)	<0·0001¶
Class-wide child aggression‖		0·04 (−0·09 to 0·16)	0·06	0·07 (−0·16 to 0·29)	0·72¶	−0·07 (−0·21 to 0·08)	0·01	−0·14 (−0·42 to 0·16)	0·72¶
Secondary outcomes
Classroom and teacher outcomes
Class-wide child prosocial behaviour‖		0·15 (0·06 to 0·25)	0·00	0·42 (0·17 to 0·71)	0·001	0·08 (−0·03 to 0·19)	0·12	0·22 (−0·08 to 0·53)	0·15
	Emotional support CLASS scores	0·55 (0·36 to 0·74)	0·08	0·65 (0·43 to 0·88)	<0·0001	0·39 (0·16 to 0·62)	0·13	0·50 (0·20 to 0·79)	0·001
	Classroom organisation CLASS scores	0·41 (0·20 to 0·62)	0·19	0·49 (0·24 to 0·74)	<0·0001	0·33 (0·11 to 0·54)	0·04	0·42 (0·14 to 0·69)	0·003
	Instructional support CLASS scores‖	0·12 (0·06 to 0·19)	0·00	0·61 (0·31 to 0·97)	<0·0001	0·05 (−0.01 to 0·10)	0·09	0·29 (−0·06 to 0·57)	0·08
	Teacher wellbeing**	0·18 (−0·03 to 0·39)	0·04	0·18 (−0·03 to 0·39)	0·09	0·25 (0·02 to 0·47)	0·15	0·26 (0·03 to 0·48)	0·03
	Violence against children by teachers over 2 school days (binary)†† (OR [95% CI])	0·23 (0·07 to 0·74)	0·00	..	0·01	0·68 (0·31 to 1·50)	0·00	..	0·34
Individual child outcomes
	Child inhibitory control‡‡	55·19 (13·84 to 96·55)	0·09	0·18 (0·05 to 0·32)	0·01	..	..	..	..
	Child behaviour difficulties§§	−0·10 (−0·60 to 0·40)	0·30	−0·05 (−0·27 to 0·18)	0·76	..	..	..	..
	Child prosocial behaviour‡‡	−0·23 (−4·46 to 4·00)	0·30	−0·01 (−0·24 to 0·21)	0·92	..	..	..	..
	Clinical range for behaviour difficulties by teacher report¶¶ (OR [95% CI])	OR 0·46 (0·22 to 0·94)	0·11	..	0·03	..	..	..	..
	Impact on daily living‖‖ (OR [95% CI])	OR 0·56 (0·29 to 1·08)	0·11	..	0·08	..	..	..	..

ICC= intracluster correlation coefficient. OR=odds ratio.

Analyses of primary outcomes and secondary outcomes relating to teacher and classroom were adjusted for baseline score, number of children in class, number of years teaching at current school, high school completed, and qualified teacher as fixed effects and school as a random effect. Analyses of individual child outcomes were adjusted for baseline score, and child age and sex as fixed effects, and school as a random effect. Estimates were obtained using multiple imputation (20) to adjust for losses to post-intervention or follow-up.

Intervention group=1, control group=0.

The effect size is the regression coefficient divided by the SD of the control group at post-intervention and follow-up.

The percentage change in the number of times that teachers used violence against children.

p values for primary outcomes were adjusted for multiple outcomes using Holms step-down procedure.

Transformed using natural logarithm.

Factor scores comprising teacher-reported depressive symptoms, burn-out, and self-efficacy.

0=no violence, 1=violence.

Transformed using Box-Cox of order 2.

Transformed using Box-Cox of order 0·5.

Above cutoff (≥16) on Strengths and Difficulties Questionnaire behaviour difficulties scale, 0=normal, 1=abnormal.

Above cutoff (≥2) on impact supplement of SDQ, 0=normal, 1=abnormal.

At post-intervention, there were significantly better scores in the Toolbox intervention group for class-wide prosocial behaviour and the quality of the classroom environment across all domains of emotional support, classroom organisation, and instructional support (table 5). Teachers in the Toolbox intervention group were also less likely to use violence against children across two school days (table 5). However, only 19 (18%) of 108 teachers in the intervention group used no violence at all at post-intervention (table 3). Differences in teacher wellbeing at post-intervention were not significant (table 5).

At 1-year follow-up, significantly better scores were noted in the Toolbox intervention group for emotional support and classroom organisation compared with the control group (table 5). Scores for teacher wellbeing were also significantly greater in the intervention group at this point compared with control. At 1-year follow-up there were no between-group differences in scores for class-wide prosocial behaviour, instructional support, or teachers' use of violence against children over 2 days.

Children in intervention schools had higher scores in tests of inhibitory control (table 5) and had reduced odds of clinical-level behaviour problems at post-intervention. There were no significant between-group differences in child behaviour difficulties or prosocial skills by teacher report or to the effect of behaviour difficulties on daily life.

Post-hoc analyses showed that there was a greater number of teachers retained in schools in the Toolbox intervention group at 1-year follow-up compared with control (105 teachers retained [88%] of 119 in intervention schools vs 85 retained [77%] of 110 in control schools; p=0·03). The between-group difference in teacher retention at the post-intervention timepoint was not significant (108 teachers retained [91%] in the intervention group vs 92 [84%] in the control group; p=0·11).

In sensitivity analyses, the magnitude and significance of the effects of the Toolbox intervention on all outcomes at the post-intervention and 1-year follow up timepoints were similar to the main analyses (appendix p 4).

Discussion

In Jamaican preschools, the Irie Classroom Toolbox effectively reduced violence against children by teachers compared with no intervention but had no effect on class-wide child aggression. To our knowledge, this is the first trial of a school-based, violence-prevention programme implemented in a pre-primary educational setting with rigorous, quantitative measures of teachers' use of violence. The intervention led to reductions in violence against children by teachers, including physical violence and psychological aggression at post-intervention, with benefits sustained 1 year after the end of the intervention. No benefits were found for observations of class-wide child aggression at post-intervention or 1-year follow-up; however, baseline values were low. Improved scores were noted in the intervention group for a wide range of secondary classroom-level outcomes at post-intervention, including observations of children's prosocial behaviour and the quality of the classroom environment in emotional support, classroom organisation, and instructional support. The greater scores for emotional support and classroom organisation in the intervention group were sustained at 1-year follow-up, and we noted increased teacher wellbeing and teacher retention in the intervention group at this timepoint. Benefits were also found for the individual child outcomes of inhibitory control and risk of clinical-level behaviour difficulties by teacher report at post-intervention.

At baseline, there were a median seven counts of violence against children over one day in both groups. This finding extrapolates to 35 counts in a week and 1400 counts of violence each school year. At post-intervention and 1-year follow-up, there were significantly fewer counts of violence against children by teachers in the intervention group (median three per day at both timepoints, extrapolating to 15 per week and 600 per year). These large, sustained reductions in violence against children by teachers from in-service teacher-training are important for informing global initiatives to prevent violence against children at school. The Irie Classroom Toolbox is theory-informed, uses evidence-based core content and implementation strategies, and was developed in close participation with early-childhood teachers. This integration of scientific evidence and theory with teachers' preferences and needs might be important in promoting meaningful benefits to teacher practices. A meta-analysis of teacher-training programmes reports that those including an individual coaching component are the most effective, and the combination of workshops and in-class support is likely to be instrumental to the effectiveness of the Irie Classroom Toolbox. These implementation strategies are also widely used in government in-service training globally, thus increasing the feasibility of widescale dissemination of the Toolbox through existing services.

The reductions in teachers' use of violence against children were accompanied by increased scores for the quality of the classroom environment, indicating that teachers were using more developmentally appropriate teaching practices including positive discipline techniques. This finding is consistent with the theory of change underpinning the intervention and has been reported in previous studies.5, 24 Increased teacher wellbeing is also part of the hypothesised mechanism of change, because poor teacher wellbeing is associated with negative teacher behaviour, and predicts negative child outcomes including poor inhibitory control and behaviour problems. The benefits of the Toolbox intervention for teacher wellbeing, (including lower burn-out and increased self-efficacy), might also be linked to the increased teacher retention noted in our study, which is an important outcome as high turnover of early childhood educators is a recognised problem in LMICs.

Although we reported large, sustained reductions to violence against children by teachers in the intervention group, most teachers continued to use violence against children in the classroom at post-intervention and 1-year follow-up. Previous studies5, 6 have also reported that violence against children by teachers is reduced following an intervention, but still remains common. Additional research is required to identify how to further reduce teachers' use of violence against children. Self-management strategies and attitudes to violence against children have been shown to predict caregivers' use of corporal punishment; thus, incorporating content to promote teachers' self-regulatory capacities, and addressing beliefs related to violence against children may be necessary.27, 28 Full integration of the Toolbox into government training and support structures may also reduce violence against children by teachers. Given the widespread use of physical violence and the absence of consequences for teachers who use violence against children, monitoring and enforcing compliance with the legal ban on corporal punishment is also necessary.

There were no benefits from the Toolbox intervention noted for class-wide child aggression; there are several potential reasons for this absence of an effect. First, it is possible that teachers need additional training on how to manage more severe child behaviour difficulties, including child aggression. The Toolbox has a strong focus on promoting positive behaviour and preventing negative behaviour, and although strategies for managing misbehaviour are included, there are few appropriate consequences for misbehaviour that are suitable in the context—eg, teachers are not trained to use time-out for misbehaviour because of the difficulties in implementing the strategy in overcrowded classrooms. There are also few privileges available in these low-resource contexts that can be removed as punishment for misbehaviours. Second, the facilitators providing in-class support to teachers were inexperienced and might not have had the skills needed to support teachers in managing more severe misbehaviour. Finally, the absence of effects on child aggression might be measurement related. In previous studies9, 19 we noted reductions in ratings of class-wide child inappropriate behaviour (including aggressive and disruptive acts) and in observed conduct problems of high-risk children. In this study, to maximise inter-rater reliability, only salient acts of aggression were coded. Class-wide child aggression scores were in the low range at baseline and this might have reduced sensitivity to change with intervention.

It is encouraging that the training changed teacher practices sufficiently to lead to benefits in child outcomes. The benefits to child inhibitory control were small, although meaningful at the population level. Child inhibitory control is an important component of self-regulation that predicts criminal offending, educational achievement, and wealth in adulthood. No benefits were noted with the intervention for teacher-reported child behaviour difficulties or prosocial skills. School-based preventive interventions have been shown to benefit high-risk children the most; the absence of effect in our study might be because we selected a random sample of children. Fewer intervention children had behaviour difficulties in the clinical range at post-intervention, indicating that benefits were found for children at the highest risk of continued behaviour problems.

The strengths of our study included the robust cluster-randomised study design, masked assessors, prespecified analyses, high levels of intervention fidelity, teacher and classroom assessments at post-intervention and 1-year follow-up, and primary outcomes measured through independent observation. Observational measures are less open to bias than teacher reports; child report is not a feasible option for preschool-age children. Teachers were aware of their group allocation and it is possible that intervention teachers behaved differently during the observational assessments. However, when observations are done over a full day, teacher reactivity to being observed is minimised. In this study, teachers were observed over two full school days at each data collection period, with the continuous measurements of violence against children by teachers done on the second day. The structural conditions of community preschools, with classrooms usually sharing a common space and high noise levels, reduce any intrusiveness of the observer. Furthermore, our data show that intervention effects on violence against children by teachers did not significantly differ across the school day (appendix p 5). The psychometric properties of the outcome measures were good, except for the low stability of observations of class-wide prosocial behaviour and instructional support over 1 year. Study limitations included the differential teacher attrition across study groups, indicating that the intervention reduced teacher turnover. Contamination was minimised through the use of a cluster design; however, schools were situated in a common geographical region and it is possible that some sharing occurred. Schools with fewer than two and more than four classrooms, and schools with fewer than ten children per class were excluded for logistical reasons related to the evaluation; it is likely that the intervention would have similar effects in these community preschools in Jamaica. Children with poor attendance were ineligible for the evaluation and it is probable that poor attendance would reduce the benefits of intervention. However, only around 3% of children were excluded for poor attendance.

The Irie Classroom Toolbox is suitable for teachers of children aged 3–8 years and was designed for use with undertrained teachers working in low-resource contexts. The Toolbox was developed by linking evidence-based content and behaviour change techniques to the enablers and barriers of intervention implementation by early-childhood teachers. The process of development has been described in detail previously and provides a framework for adapting the Toolbox to early childhood educational settings in other LMIC. Adaptation would involve identifying to what extent the enablers and barriers to intervention implementation differ in the new context and making changes to the content, structure, materials, or process of delivery as needed. Adaptations would also need to be made to the visual aids used with the children to ensure that they reflect the cultural context. As the Toolbox is theory informed, incorporates empirically derived content and behaviour change techniques, and was developed to be adaptable and feasible in low-resources settings, we anticipate that it has potential for effectiveness in early childhood educational contexts in other LMIC. This needs to be tested in future trials. The Toolbox is suitable for use in pre-service and in-service teacher-training and all teacher manuals and materials are available online.

Data sharing

The principal investigator and those authorised by the principal investigator shall have exclusive rights to analyse and publish such data until 5 years after the study completion date. Thereafter, the data will be available, on reasonable request, from the corresponding author (HB-H). The Irie Classroom Toolbox is licensed through a Creative Commons Attribution-NonCommercial-ShareAlike license.