Building elementary students' social and emotional skills: A randomized control trial to evaluate a teacher-led intervention.

The Faith and Wellness: A Daily Mental Health Resource is a school-based, teacher-led social-emotional learning (SEL) intervention resource for elementary students. It is designed to address the challenges faced by existing SEL interventions, including lack of time, training, and resources. Using a randomized control trial design, this study evaluates short-term outcomes associated with the use of this resource. Participants were elementary teachers (NT1 = 201, NT2 = 129) and students (NT1 = 242, NT2 = 183; ages 4-14; 47.5% girls) from 19 Catholic school boards in [PROVINCE]. Using multi-level models, significant small to medium effect sizes indicated that intervention group teachers: taught SEL more frequently; had higher confidence in teaching SEL; and had more positive perceptions of the classroom climate, students' SEL, and students' school engagement at Time 2 than comparison group teachers. Results for students were less robust, though there was indication of dosage effects. Results highlight the role of teachers and frequent delivery in effective SEL implementation. Supplementary Information: The online version contains supplementary material available at 10.1007/s12310-022-09538-x.

School-Based Social-Emotional Learning

Educational research and policy have shown an increasing focus on promoting the overall well-being of children and youth through the use of social-emotional learning (SEL) programs in classroom settings (Atwell & Bridgeland, 2019). The burgeoning interest in youth well-being in school contexts is underscored by a growing awareness of the internalizing and externalizing challenges that young people face, including anxiety, depression, bullying, and aggression (Fonagy et al., 2009; Jones et al., 2010). In Canada, for instance, not only does the Mental Health Commission of Canada recognize child and youth mental health as a priority, but it also identifies schools as an important context for fostering the social-emotional skills necessary to promote positive social, emotional, mental, and academic development (Hymel et al., 2017; Mental Health Commission of Canada, 2012). SEL has been linked with positive short- and long-term outcomes for students, which has prompted researchers to focus on the effective development, implementation, and evaluation of school-based social-emotional learning programs (Taylor et al., 2017; Wanless & Domitrovich, 2015). As such, the goal of this study was to evaluate a school-based, teacher-led SEL intervention for elementary school students (kindergarten to grade 8) in Ontario, Canada, that was designed to be brief, accessible, and cost-effective.

What is SEL?

SEL refers to the process of acquiring, developing, and applying the knowledge, skills, and attitudes that are considered foundational to the promotion of positive mental health and well-being (Weissberg et al., 2015). SEL often includes the skills necessary to understand and manage emotions, develop both self-awareness and self-control, set and achieve positive goals, establish and maintain positive relationships, and engage in responsible decision-making (Collaborative for Academic, Social, and Emotional Learning, 2016). Evidence-based SEL programs provide students with the opportunity to develop and expand their SEL knowledge and skills (Taylor et al., 2017). SEL programs are associated with a slew of positive outcomes for students, including: (a) engagement in greater prosocial behavior; (b) improved academic engagement and achievement; (c) reduced aggression; (d) reduced emotional distress; (e) the formation and maintenance of positive relationships; and (f) the promotion of general well-being (Berry et al., 2016; Castillo et al., 2013; Denham et al., 2012; Espelage et al., 2015; Low et al., 2015; Oberle et al., 2014; Rhoades et al., 2011; Sklad et al., 2012). Existing research supports the importance of SEL for healthy development and further suggests that school-based SEL programs may be uniquely situated to promote the skills that are essential to healthy cognitive, social, and emotional development (Durlak et al., 2011; Hough et al., 2017).

SEL in a Classroom Setting

With children spending at least six hours a day in school, the classroom is an important context for reaching children and youth (Bird & Sultmann, 2009; Kress & Elias, 2007; Schonert-Reichl et al., 2012). School age (approximately 4–14 years) represents a period of invaluable social, emotional, and cognitive development. With higher brain plasticity than adults, children and youth are particularly amenable to developing SEL competencies (Bradshaw et al., 2012). In addition, children’s social-emotional well-being has been shown to decline over the course of elementary school years and many internalizing and externalizing behaviors are known to increase in frequency and severity (Eccles, 2004; Murray-Close et al., 2007).

Aside from representing a key developmental period in children’s lives, school settings also present the potential to engage with a wide range of young people. In schools, it is possible to provide equitable delivery of SEL programs to diverse children. School-based SEL programs are identified as more cost-effective than programs that utilize extra-curricular or community-based delivery methods (Cipriano et al., 2020; Greenberg, 2010). Further, school-based interventions can make SEL available to students regardless of ethnic, racial, socioeconomic, or geographical differences, as supported by meta-analytic research (Durlak et al., 2011; Taylor et al., 2017). Recent school-based SEL research is focused on the potential for SEL interventions to support equitable treatment of students (Jagers et al., 2019). Importantly, SEL skills are malleable and teachable, placing teachers in an optimal position to deliver SEL programming to students (Durlak et al., 2011; Mahoney et al., 2018; Sklad et al., 2012).

School-Based SEL Intervention Programs

Many successful, evidence-based SEL programs have been implemented worldwide. (Mahoney et al., 2018; Wigelsworth et al., 2016). For instance, Promoting Alternative Thinking Strategies (PATHS), a program that has been facilitated in multiple countries, has shown positive effects on children’s rates of aggression, social competence, and academic engagement (Greenberg et al., 1995; Shonfeld et al., 2015). Likewise, the Positive Attitude intervention in Portugal purports to increase students’ social awareness, self-control, and self-esteem, as well as decrease social anxiety and isolation (Coelho et al., 2015). In a Canadian context, interventions such as The Roots of Empathy and MindUP were found to support students’ engagement in prosocial behavior, as well as increased cognitive control, empathy, and peer acceptance (Schonert-Reichl et al., 2012, 2015).

The feasibility of an intervention is an important consideration for implementation. While it is crucial that interventions be evidence-based, they must also stand up to real-world implementation challenges. In schools, teachers and staff often lack the time, resources, and training necessary to carry out programming (Durlak et al., 2011). Utilizing class time for SEL programming also places pressure on administrators and teachers, who must combat the common public misconception that investing time in SEL unnecessarily detracts from academic learning (Malecki & Eliot, 2002). The delivery of SEL interventions in schools is further challenged by the resources often required for program implementation (e.g., available for purchase or requiring additional materials such as books and videos). Additionally, many programs are delivered by external program facilitators or require additional teacher training (Greenberg et al., 1995; Schonert-Reichl et al., 2012). Often, the onus is placed on individual teachers to pursue SEL training, which can be costly and time consuming (Miyamoto et al., 2015). Indeed, while research suggests that teachers play a crucial role in their students’ SEL, a lack of accessible teacher training, knowledge, and confidence are often cited as barriers to the delivery of SEL programming in classroom settings (Bridgeland et al., 2013; Durlak et al., 2011).

The Intervention

The Faith and Wellness: A Daily Mental Health Resource was developed as an implementation-sensitive resource for the delivery of SEL by teachers in the classroom. The Faith and Wellness Resource was designed to emphasize brief (e.g., 5–15 min), accessible (i.e., no fee for access, no external resources or materials needed), and easily implemented practices (i.e., no specialized training required, can be implemented into regular scheduling) for teachers to use with students. In order to meet these goals, the Faith and Wellness Resource was co-created by multiple organizations and stakeholders to address the need for universal, school-based SEL offerings which promote positive mental health in Ontario’s English Catholic School System. The Faith and Wellness Resource was based on a prior intervention, originally developed in partnership by School Mental Health Ontario (SMHO) and the Elementary Teacher’s Federation of Ontario (EFTO) for use in the public school system. The original resource was developed with input from teachers, child and youth workers, and children’s mental health specialists (Short & McVey, 2018). The resource was developed by and for teachers, with a focus on affordability, brevity, and ease of delivery.

The original intervention was the result of a scoping literature review and content analysis that identified and refined the common elements of existing evidence-based SEL programs, resulting in six core components of SEL (stress management and coping, identification and management of emotions, positive motivation and perseverance, healthy relationship skills, self-awareness and sense of identity, and executive functioning [the category of “Executive Functioning” has since been modified to “Critical and Creative Thinking (Executive Functioning)]; see Provincial System Support Program, 2017). Within these six categories, SEL practices were created (See Supplementary Materials, Section A for example practices).

To suit the needs of the Ontario Catholic school system, SMHO and the Ontario English Catholic Teacher’s Association (OECTA) adapted the prior intervention. Modifications were made for additional information regarding how SEL practices could apply to the Catholic curriculum, but the core SEL component of each practice was based on the original intervention that was designed to be applicable across a wide range of students (see Short & McVey, 2018 for additional details on how the resource was adapted). The resulting, adapted intervention included 80 daily SEL practices. The Faith and Wellness Resource provides a centralized database of practices from which teachers can select those best suited to students’ skill sets and areas of need. The resource itself provides information about each practice to support teachers’ selections, including: length of practice (ranging between 5 and 15 min); grade level (and, where applicable, modifications that could be made to the practice to cater to other grade levels); the SEL skill being targeted by the particular practice; and the evidence supporting the practice (i.e., citation for relevant peer reviewed research).

In creating the Faith and Wellness Resource, efforts were made to address potential barriers to delivery and implementation. The resource is intended to be flexible, recognizing teachers’ professional judgment regarding students’ needs. The resource’s practices are accessible online at no cost (https://smho-smso.ca/fw/) and are designed for delivery by teachers across all subject areas and grade levels (Kindergarten to Grade 8), with each practice amenable to modification. The practices aim to promote active and passive engagement of students and teachers. The resource requires no training and is designed to be accessible to teachers with little to no prior knowledge of SEL interventions (though, there is additional information to support teachers provided on the website). Perhaps most importantly, the practices are short, easy to learn, and intended for frequent, daily use.

Current Study

While the Faith and Wellness Resource purports many strengths (e.g., evidence-based, short, no cost, easy to implement) and was developed to address some of the challenges typically faced by SEL interventions, formal evaluation is required. As such, the overarching objective of this study is to offer a preliminary evaluation of the Faith and Wellness Resource for Ontario Catholic school teachers and students. In particular, the goals of this study are to assess teacher and student use of the Faith and Wellness Resource and its effects on SEL in the classroom over the period of implementation. Using a randomized control design, we examined changes in teachers’ SEL knowledge, SEL confidence, and perceptions of students in their classrooms, as well as changes in students’ SEL skills, general well-being, and perceptions of school. Overall, it was anticipated that there would be positive changes for both teachers and students associated with the use of the resource. For teachers in the implementation group, we expected increased levels of SEL knowledge and confidence, compared to the control group, for whom we expected no changes. Likewise, we anticipated that students in the implementation group would demonstrate increased SEL competence compared to students in the control group. The analyses included in this study are confirmatory in nature, while those investigated further in the Supplementary Materials (e.g., examining implementation patterns) are exploratory.

Method

Recruitment and Procedures

Teachers

The research team invited English Catholic school boards across Ontario (N = 29) to partake in the study, 19 of which agreed to participate. Teachers (Kindergarten-Grade 8) within these boards were eligible to participate in the research project. Teachers were informed about the opportunity to participate in the study by either school board staff (i.e., principal, colleague, etc.) or through their federation (i.e., local or provincial units). Participating school boards (N = 19) were match-paired based on unit size and urban/rural makeup, then randomly assigned to either Group 1 (intervention group) or Group 2 (waitlist comparison group). Each school board has a senior clinician involved in supporting system-wide mental health strategy development and implementation. In this case, their role included supporting teachers actively implementing the Faith and Wellness Resource. Given the role of these board-level clinicians, randomization was conducted at the school board level to avoid contagion. All procedures, for both teacher and student participation, were approved by Brock University Research Ethics Board, as well as the research ethics boards from each participating school board.

After providing informed consent, teachers in Groups 1 (n = 124) and 2 (n = 77) completed an online questionnaire package (October/November Time 1; T1). The questionnaire took approximately half an hour to complete. Following completion of T1 surveys, teachers were informed of their group membership. Group 1 teachers were given access to the Faith and Wellness Resource website. Group 1 teachers were provided with an overview of the intervention website, including where to find an introductory video that explained use of the resource, background information, and supporting evidence for practices within the resource.

Based on the findings from the prior intervention (Short & McVey, 2018). and a pilot study of the current intervention, Group 1 teachers were asked to complete at least one practice from each of the six SEL categories (each repeated three times), for a total of at least 18 practices over the course of the 3 month implementation period. Teachers selected which practices they used from each category and with what frequency (beyond the minimum three uses of each of the six practice categories). The resource provides teachers with information on each practice, including target age range and target skills, allowing teachers to select practices based on their knowledge of the students in their classrooms. Teachers completed electronic feedback forms approximately every two weeks, which they completed after implementing a practice at least three times.

Following the implementation period (February/March Time 2; T2), teachers were invited to complete another questionnaire package, like the T1 questionnaire and utilizing the same procedures. After Group 2 teachers completed the T2 questionnaire, they received access to the Faith and Wellness Resource website, with the intent to begin the process of implementing a minimum of 18 practices (one from each of the six categories, repeated three times) over the following 3 months. All implementation and data collection ceased, however, in mid-March due to COVID-19 school closures.

Students

Student participation procedures were different than for teachers. Of the teachers who had already agreed to participate (N = 201, see above), 19 teachers (42% Group 1, n = 8) from 11 schools (45% Group 1, n = 5) were selected to have the students in their classrooms participate in the student portion due to practical constraints (i.e., hundreds of participating teachers translates into thousands of students; therefore, due to limits on resources for data collection, only a subset of students, numbering in the hundreds, could be invited to participate). Parental consent for student participation was attained before groupings were assigned. Selection was random, with stratification to ensure that selected classrooms represented a variety of grade levels and both Group 1 and 2. Teachers in each of the 19 participating classrooms sent informed consent forms home with students; students whose parents provided consent and who themselves provided assent were able to participate.

Participating students were asked to complete a T1 questionnaire (approximately 20 min). For older students, each item was read aloud by trained research assistants, and students could follow along and complete the pencil-and-paper questionnaire. This was done as a whole class (Grades 6–8) or in small groups of 2–4 students (Grades 3–5), with research assistants available to provide assistance as needed. For younger students (Grades K-2), the questionnaire was completed one-on-one in an interview style; research assistants read each question aloud and students could respond verbally or by indicating their response on a visual scale (Martin et al., 2017). Measures for younger students used a simplified and visual 3-point Likert scale to reflect younger students’ levels of comprehension (e.g., 1 = Not at All, 3 = A Little Bit, 5 = A Lot), whereas older students used a 5-point Likert scale (e.g., 1 = Not at All, 2 = Not Really, 3 = A Little Bit, 4 = Pretty Much, 5 = A Lot) (for both groups, scales ranged from 1 to 5; see Martin et al., 2017). Following the 3 month implementation period, students completed a T2 questionnaire package following the same procedures as T1.

Participants

A total of 201 teachers consented to participate and completed a T1 survey (62% Group 1, n = 124). At T2, 129 teachers completed a survey (46% Group 1, n = 60) (64% total retention; 47% Group 1 retention). Retained teachers were compared to non-retained teachers on all T1 study variables; no differences were found, with one exception. Retained teachers reported lower T1 SEL knowledge (M = 3.82, SD = 0.74) compared to non-retained teachers (M = 4.32, SD = 0.63; t[199] = 0.85, p < 0.001).Teachers varied in their years of teaching experience, from less than one year of experience (1%), 1–5 years’ experience (8%), 6–10 years’ experience (15%), 11–20 years’ experience (52%), 21–30 years’ experience (22%), to over 30 years of experience (2%). Participants included teachers at all grade levels from Kindergarten to Grade 8 (See Supplementary Materials, Section C).

From the 19 selected teachers’ classrooms, there were a total of 413 students. At T1, the response rate of returned consent forms was 74%, with 83% of these parents consenting for their child to participate, resulting in a total of 257 participating students (39% Group 1). Of these 257, 242 completed surveys (2 students chose not to participate; the remaining students were absent during data collection). At T2, 183 students completed T2 surveys (1 student chose not to participate, 4 students had changed schools; an additional 32 students were absent during data collection) (76% total retention; 63% Group 1 retention). Data at T2 were unable to be collected from 53 students with permission to participate (5 classrooms) due to COVID-19 school closures.

There were a total of 271 student participants, 154 of which participated at both T1 and T2 (T1 only N = 88, T2 only N = 29). Retained students were compared to non-retained students on all T1 study variables. The following differences were found: retained students reported higher classroom climate (M = 4.08, SD = 0.63) and school liking (M = 3.80, SD = 0.79) compared to non-retained students (M = 3.85, SD = 0.73, t [239] = 2.54, p = 0.01; M = 3.46, SD = 1.00; t [147.06] = 2.77, p = 0.01, respectively). Students ranged across grades: Kindergarten (9%), Grade 1 (4%), Grade 2 (22%), Grade 3 (4%), Grade 4 (3%), Grade 5 (17%), Grade 6 (19%), and Grade 7 (22%). Approximately half of students reported they were female (49%; 49% male, 1% other, 1% prefer not to say). According to Statistics Canada (2016), the representation of people of color within the neighborhoods in which the 11 schools were situated varied, ranging from 2 to 55% (M = 19%) across neighborhoods, with 0.4–7% (M = 3%) identifying as Indigenous Peoples. Median income levels ranged from $24,000–47,000 (M = $36,000).

Measures

SEL Measures

At both T1 and T2, many of the measures asked specifically about the six skill categories represented in the Faith and Wellness Resource: stress management and coping, identification and management of emotions, positive motivation and perseverance, healthy relationship skills, self-awareness and sense of identity, and executive functioning. These six categories and associated measures were adapted from the Centre for Addictions and Mental Health (2017) and Assessment Work Group (2019). Teachers were given a description of these skill categories at the beginning of the questionnaire package. Using 5-point Likert scales from 1 = Not at All to 5 = Extremely (the ends of the response scales changed slightly based on the question), each of the following was assessed and averaged across the 6 SEL items: SEL knowledge (i.e., the degree to which they have the knowledge required to introduce each competency to students, model it in the classroom, and answer related student questions for the 6 SEL skill categories)(αT1 = 0.93, αT2 = 0.90); SEL confidence (i.e., confidence in teaching each SEL skill category) (αT1 = 0.92, αT2 = 0.93); and frequency of structured SEL teaching (αT1 = 0.91, αT2 = 0.93).

General SEL Teaching Confidence

General SEL confidence was assessed using a 3-item scale rated on a 5-point Likert scale ranging from 1 (Not at all confident) to 5 (Extremely confident) (Assessment Work Group, 2019). Teachers indicated their level of confidence related to the following items: 1) “discussing social-emotional learning with students”; 2) “delivering social-emotional learning materials to students”; and 3) “handling questions from students related to social-emotional learning.” Responses on these three items were averaged to create a total score for teachers’ general SEL confidence.

Perceptions of students’ SEL

To assess teachers’ perception of their students’ SEL skills in the classroom, teachers rated the number of students in their classroom who currently demonstrate competencies in each of the six SEL skill categories with each of the six categories representing one item on the questionnaire (Assessment Work Group, 2019; see above). Teachers responded using a four-point Likert scale ranging from 1 = None/almost none of the students in my class demonstrate competencies in this category to 4 = All/almost all of the students in my class demonstrate competencies in this category (e.g., “How many of your students have competencies in…positive motivation skills?”). Responses were averaged to create a total score for teachers’ perceptions of students’ SEL skills (αT1 = 0.88, αT2 = 0.85).

Perceptions of Classroom Climate

Teachers reported on their perceptions of classroom climate with a 12-item scale (adapted from Strizek et al., 2016; see also Short & McVey, 2018). An example item from this scale reads: “Students in this class take care to create a pleasant learning atmosphere.” Reponses were rated on a four-point Likert scale ranging from 1 = Strongly Disagree to 4 = Strongly Agree and were averaged to create a total score for teachers’ perceptions of classroom climate (3 items were reverse coded prior to averaging) (αT1 = 0.86, αT2 = 0.90).

Perceptions of Students’ School Motivation and Engagement

Teachers reported on their perceptions of students’ overall school motivation and engagement using a 5-item scale, adapted from Field et al. (2017). Example items include: “Students in my class are doing the best they can” and “Students in my class want to try hard in school.” Responses were rated on a four-point Likert scale ranging from 1 = None/almost none to 4 = All/almost all. Responses were averaged (1 item was reverse coded prior to averaging) to create a total score for teachers’ perceptions of students’ school motivation and engagement (αT1 = 0.82, αT2 = 0.85).

Perceptions of Students’ Academic Performance

Teachers rated their students’ overall academic performance using a 4-item scale with items such as: “Students in my class are doing well academically.” Reponses were rated on the same four-point Likert scale as school motivation. Responses to all items were averaged (1 item was reverse coded prior to averaging) to create a total score for teachers’ perceptions of students’ academic performance (αT1 = 0.80, αT2 = 0.80).

Satisfaction

At T2, Group 1 teachers were asked to rate overall, how helpful they found the resource to be, on a 5-point Likert scale ranging from 1 = Not at all helpful to 5 = Extremely helpful (Assessment Work Group, 2019). Teachers were also asked to indicate whether they would recommend the resource to others (No/Yes).

Students

SEL Confidence

Students’ confidence with each of the six specific SEL skill categories was measured with a six-item scale, using a five-point Likert scale ranging from 1 = Not at All Confident to 5 = Extremely Confident (Assessment Work Group, 2019). Students were provided with some examples to illustrate each of the six SEL skill categories (see above). For example: “How confident are you with your…coping skills? These are skills like relaxation calming, mindfulness, helpful thinking, or asking for help.” Responses were averaged to create a total score for students’ SEL confidence (αT1 = 0.81, αT2 = 0.78).

General Well-Being

Students’ well-being was measured using the Stirling Children’s Well Being Scale (Liddle & Carter, 2015). This scale comprises 12 items, assessing both positive emotional state and positive outlook. Items were rated on a five-point Likert scale ranging from 1 = Never to 5 = All of the Time and were averaged to create a total general well-being score (αT1 = 0.89, αT2 = 0.88).

Classroom Climate

Students reported on their perceptions of the classroom climate using a 5-item scale (adapted from Battistich et al., 1995; Andrews et al., 2016) (e.g., “Students in my class work well together”). Responses for this measure, as well as school liking and school motivation and engagement which follow, were rated on a five-point Likert scale ranging from 1 = Not at All to 5 = A Lot. Responses were averaged to create a total score for students’ perceptions of classroom climate (αT1 = 0.72, αT2 = 0.68).

School Liking

Students reported on how much they liked school using a 10-item, 5-point Likert scale (adapted from Ladd et al., 2000). An example item reads: “I like being at school.” Responses to all items ranged from 1 = not at all to 5 = a lot and were averaged (5 items were reverse coded prior to averaging) to create a total score for students’ school liking (αT1 = 0.89, αT2 = 0.91).

School Motivation and Engagement

Students reported on their school motivation and engagement using a 5-item, 5-point Likert scale (similar to the teacher version; e.g., “I try hard in school”) (Field et al., 2017). Responses to all items were averaged (1 item was reverse coded prior to averaging) to create a total score for students’ school motivation and engagement (αT1 = 0.71, αT2 = 0.83).

Satisfaction

At T2, Group 1 students were asked to rate overall, how helpful they found the Faith and Wellness Resource practices to be, both for themselves and for other students in their class. Both items were rated on a 5-point Likert scale ranging from 1 = Not at all helpful to 5 = Extremely helpful. Students were also asked to indicate whether they would like to continue doing the practices in their class (no/not sure/yes).

Data Analytic Plan

Teacher data were nested within school boards. Intra-class correlations (ICCs) indicated that, with the exception of SEL knowledge, the extent of nesting was quite small (between 0 and 3%; SEL knowledge had ICC of 0.26). Despite small ICCs, multi-level modeling procedures were used. Multiple imputation procedures (20 imputations) were conducted in Mplus 8 to account for missing data. To assess the overall effects of the intervention and examine potential change over time in outcome variables, a series of multi-level models, using maximum likelihood estimation, were specified in Mplus to account for nesting at Level 2 (using imputed data sets; Mplus provides average results across all datasets). A separate model was specified for each outcome variable. Each model included the T1 variable predicting the T2 variable at Level 1, and group (Group 1 versus Group 2) predicting the T2 variable at Level 2.

Group 1 teachers (N = 124) were asked to indicate via weekly feedback forms how many practices they implemented in their classrooms. To examine the possibility that the more times teachers implemented practices across the total 3-month implementation period, the more change in outcome variables, multi-level models were specified (separately for each outcome variable) with the total number of times teachers reported implemented practices predicting differences scores in each of the outcome variables of interest (difference scores were calculated as T2 – T1, such that a positive value indicated an increase over time in the variable and a negative value indicated a decrease over time in the variable) (see Castro-Schilo & Grimm, 2018; McArdle, 2009). All variables were specified as Level 1, but models accounted for nesting at Level 2 (school board).

A similar set of analyses were conducted for student data. Student data were nested both within teacher (Level 2) and school board (Level 3). ICCs indicated nesting between 7 and 13% at Level 2, and between 0.2 and 0.7% at Level 3. Given negligible Level 3 ICCs, 2-level models were run. Multiple imputation (20 imputations) was also used to account for missing data. As with teachers, a separate model was specified for each outcome variable, and included the T1 variable predicting the T2 variable at Level 1, and group (Group 1 versus Group 2) predicting the T2 variable at Level 2 (nesting within teacher).

Again, as with teachers, we examined the possibility that the more times Group 1 teachers implemented practices across the implementation period, the more change in student outcome variables. Thus, multi-level models were specified (separately for each outcome variable) with the total number of times teachers reported implemented practices (Level 2) predicting differences scores in each of the outcome variables of interest.

Results

Descriptive Statistics and Baseline Comparisons

All variables were normally distributed, as indicated by skewness < 2 and kurtosis < 7 (Field, 2018). See Tables 1 and 2 for means and standard deviations for all variables for teachers and students, respectively. To confirm that, through randomization to groups, there were no baseline differences on key measures, we conducted a series of ANOVAs comparing teachers in Group 1 and Group 2 on all T1 measures and another series of ANOVAs comparing Group 1 and 2 students on all T1 measures. For teachers, the ANOVAs indicated no significant differences between groups at T1, with one exception (T1 SEL knowledge was higher for teachers in Group 1 than 2; F[1,199] = 154.07, p < 0.001, η2 = 0.44). Likewise, the ANOVAs indicated no significant differences between students at T1, with one exception (students reported less positive classroom climate in Group 1 than 2; F[1, 239] = 4.39, p = 0.037, η2 = 0.018).

Table 1

Means and Standard Deviations for Teachers Group 1 and 2 at Time 1 and Time 2

	Time 1				Time 2
	Group 1		Group 2		Group 1		Group 2
	M	SD	M	SD	M	SD	M	SD
SEL knowledgea	4.38	.51	3.39	.63	3.81	.53	3.37	.61
General SEL confidencea	3.61	.72	3.45	.85	3.93	.58	3.47	.77
SEL confidencea	3.52	.71	3.51	.73	3.81	.56	3.41	.73
Frequency of SELa	2.63	.93	2.66	.90	3.12	.85	2.67	1
Student SELb	2.24	.50	2.36	.44	2.66	.47	2.37	.39
Classroom climateb	2.54	.38	2.56	.46	2.79	.37	2.62	.47
Student School Motivation and Engagementb	2.85	.46	2.82	.50	3.12	.45	2.80	.46
Student Academic Performanceb	2.47	.50	2.58	.44	2.59	.58	2.61	.39

Ngroup 1 = 124, Ngroup 2 = 77. SEL = Socio-emotional learning

aMeasure rated on a 1–5 scale

bMeasure rated on a 1–4 scale

Table 2

Means and Standard Deviations for Students Group 1 and 2 at Time 1 and Time 2

	Time 1				Time 2
	Group 1		Group 2		Group 1		Group 2
	M	SD	M	SD	M	SD	M	SD
SEL confidence	3.72	.90	3.81	.81	3.70	.85	3.83	.64
General well-being	3.86	.73	3.94	.72	3.70	.76	3.89	.64
Classroom climate	3.88	.73	4.07	.63	3.72	.70	3.96	.59
School liking	3.59	.88	3.73	.88	3.62	.90	3.80	.81
School motivation and engagement	4.31	.74	4.40	.69	3.70	.85	3.83	.64

Ngroup 1 = 102, Ngroup 2 = 169. SEL = Social emotional learning. All measures rated on 1–5 scale

Effects of the Intervention: Group Differences in Change over Time

For all models, T1 variables significantly predicted T2 variables (with the exception of SEL knowledge; see Table 3, Panel A). The grouping variable (dummy-coded with Group 2 as the reference group) significantly and positively predicted the T2 variable for general SEL confidence (marginal), SEL knowledge, frequency of SEL, student SEL, classroom climate, and student school motivation and engagement. This indicates that, for all variables except SEL knowledge and academic performance, the intervention group had more favorable outcomes at T2 compared to the comparison group, controlling for T1 variables.

Table 3

Parameter Estimates from Multilevel Models

	T1 Variable (Level 1)	Group (Level 2)	Effect Sizes (Cohen’s d)
Panel A: Teacher T2 Variables
SEL Knowledge	.20	.10	.15
General SEL confidence	.58***	.22+	.31
SEL confidence	.33**	.29**	.41
Frequency of SEL	.49***	.30*	.31
Student SEL	.37***	.29***	.63
Classroom climate	.70***	.23***	.52
Student school motivation and engagement	.48***	.33***	.65
Student academic performance	.59***	.04	.08
Panel B: student T2 variables
SEL confidence	.51***	.05	.06
General well-being	.49***	− 09	− 12
Classroom climate	.65***	− 09	− 13
School liking	.56***	− 01	− 01
School motivation and engagement	.54***	− 15	− 19

Group coded as Group 1 = 1, Group 2 = 0. SEL = Socio-emotional learning

+p < .06.;*p < .05.; **p < .01.;***p < .001

For all models, T1 variables significantly predicted T2 variables (see Table 3, Panel B). Contrary to expectations, group membership did not significantly predict any outcome variables. That is, there was no evidence that being in Group 1, compared to being in Group 2, significantly predicted T2 variables, accounting for T1.

Implementation and Changes Based on Dosage

Approximately one quarter (27%) of Group 1 teachers reported implementing practices between 3 and 10 times, 20% reported implementing practices 11–20 times, 9% implemented practices 21–30 times, and 3% implemented practices 31 or more times (overall M = 12, SD = 10.38). Of the total 124 teachers in Group 1, 41% did not complete any feedback forms (see Supplementary Materials, Section B for further information regarding completion versus non-completion of feedback forms). The numbers presented here are based on teachers’ submitted feedback forms; it is possible that teachers may have implemented practices but did not complete an accompanying feedback form. Subsequent analyses should be considered with that in mind.

For models examining dosage effects within Group 1, results indicate the same pattern for SEL knowledge (b = 0.01, p = 0.02), general SEL confidence (b = 0.01, p = 0.03), SEL confidence (b = 0.01, p = 0.05), frequency of SEL (b = 0.03, p = 0.003), and student SEL (b = 0.01, p = 0.03): the more times Group 1 teachers reported implementing practices, the more positive change in outcome variables. For the remaining variables (classroom climate, student school motivation, and student academic performance), there was no significant prediction from number of times teachers implemented the practices (bs < 0.01, ps > 0.05). Teaching experience and grade level taught were also tested and did not significantly predict the implementation of practices (see Supplementary Materials, Section C).

In the Group 1 classrooms involved in student data collection, teachers reported implementing the practices between 7 and 26 times over the 3-month implementation period (M = 14.67, SD = 6.22). Models examining dosage effects indicated that the number of times teachers implemented practice significantly predicted a positive difference score for SEL confidence (b = 0.03, p 0.03), such that the more times teachers reported implementing practices, the more positive change in students’ confidence in their SEL skills. The same pattern was found for general well-being (b = 0.03, p = 0.01), and classroom climate (b = 0.03, p = 0.05). For the remaining variables (school liking, school motivation and engagement), there was no significant prediction from number of times teachers implemented the practices (bs < 0.01, ps > 0.05).

Satisfaction with the Intervention

Descriptive statistics were examined to understand Group 1 teacher and student satisfaction with the intervention. Group 1 teachers reported high levels of satisfaction with the Faith and Wellness Resource, with most teachers (62%) rating the resource as either “very helpful” or “extremely helpful.” Almost all teachers in Group 1 (97–98%) reported that they would recommend the resource to other teachers. See Supplementary Materials, Section D for more detailed teacher perceptions and satisfaction with resource. Students also reported high levels of satisfaction with the Faith and Wellness Resource practices. Specifically, students rated the practices as being generally helpful both for themselves (M = 3.82, SD = 1.11) and for other students in their class (M = 3.82, SD = 1.00), on a 1–5 scale. The majority of students (81%) indicated that they would like to continue doing the practices in their class, while an additional 16% were not sure (only 3% said that they would not like to continue the practices).

Discussion

The Faith and Wellness Resource aims to provide time efficient, accessible, and cost-effective SEL opportunities for elementary school students and teachers. Results of this RCT offer support for the Faith and Wellness Resource as an effective SEL resource. Though the results suggest no difference between students in the intervention group compared to the control group, use of Faith and Wellness Resource was associated with positive outcomes for teachers, suggesting that the Faith and Wellness Resource encouraged their engagement with and understanding of SEL for themselves and their students. Further, we found that teachers were able to implement the practices regularly and frequently, which emerged as a key factor in the intervention’s efficacy. Finally, both teacher and students reported high levels of satisfaction with the resource.

Outcomes for Teachers

Over time, teachers who implemented the Faith and Wellness Resource reported increases in their own SEL confidence and frequency of teaching SEL concepts in the classroom. They also reported positive increases in their perceptions of students’ SEL competencies, classroom climate, and school motivation and engagement compared to teachers in the control group who reported no changes on the aforementioned measures. Teachers in the intervention group also demonstrated dosage effects, suggesting that the more frequently teachers reported implementing practices, the more positive changes were reported in their SEL knowledge, general SEL confidence, specific SEL skills’ confidence, and perceptions of students’ SEL skills. It is important to note that the directionality of these associations is still unknown: it is unclear whether teachers’ SEL knowledge and confidence improved because they taught more lessons or whether they taught more lessons because they felt more knowledgeable or confident. Despite this lack of directionality, these results offer preliminary support for the effectiveness of the Faith and Wellness Resource for teachers. The teachers’ results regarding their perceptions of their students suggest that the Faith and Wellness Resource may have benefits for their students as well. The more frequently the practices were implemented, the better self-reported teacher outcomes and teacher-reported student outcomes were found, suggesting that regular practice may play an important role in fostering SEL skills for teachers and students alike. Changes in teacher-reported outcomes were generally consistent regardless of grade level or teaching experience.

Outcomes for Students

For students, group membership (intervention versus waitlist group) did not significantly predict differences in any of the outcome variables. Lack of between-group changes in students’ outcomes may underscore the importance of the teacher in supporting SEL. That is, teachers play an important role not only in modeling SEL in the classroom, but also in fostering the behaviors that contribute to positive classroom climate and group norms surrounding SEL (Durlak et al., 2011; Hawkins et al., 2004). The current study indicates that teachers—particularly those who used the resource frequently—improved in their own SEL confidence and skills. It is possible, therefore, that changes in teachers’ SEL knowledge and confidence must occur first, with teachers’ attitudinal and behavioral changes subsequently supporting students after a longer period of time than what was assessed during this particular RCT.

The lack of students’ change in SEL competencies over time may also be attributed to the delivery of the resource. Implemented using a group-based, whole class-approach, it would be expected that there would be differences between different classrooms, more so than among students within the same classroom. That is, students within the same classroom would share similar experiences in terms of the classroom climate and classroom behavioral norms, compared to students in another classroom with a different teacher. Given the importance of the classroom context, it may be that this sample did not include enough discrete classrooms to discern changes between the control and intervention groups (particularly since data collection from some classrooms was not possible due to COVID school closures).

Students in the intervention group, however, did demonstrate positive dosage effects (similar to those reported by teachers). For students who were exposed to the intervention, the number of times their teachers implemented the practices significantly predicted increases in students’ SEL confidence and general well-being, as well as students’ perceptions of their classroom climates. The presence of dosage effects for students suggests that frequent practice of SEL skills in a group setting, like the classroom, may help to establish group norms that encourage and reinforce the sustained use of SEL skills, bolstered using common language, as well as peer and student support (Durlak et al., 2011; Jennings & Greenberg, 2009; Peddigrew et al., 2022).

Benefits of the Intervention

The results obtained from this RCT are encouraging, particularly considering the relatively short 3-month implementation period. The Faith and Wellness Resource seems to address some of the challenges typically faced by school-based SEL interventions, such as costliness, time constraints, lack of teacher confidence, and/or needing to coordinate program facilitation by organizations external to the education system (Durlak et al., 2011; Miyamoto et al., 2015). Teachers who engaged with the program did not report any perceived barriers to its use (see Supplemental Materials); however, 41% of teachers reported not using the practices, potentially indicating that there may be barriers to this intervention that need further consideration and investigation in future evaluation. It may be that some teachers had pre-existing SEL practices within the classroom and did not feel the need to add in more, or teachers felt that certain practices were not of value. Despite this, the results of this study support the rationale for designing the Faith and Wellness Resource to be as accessible to classroom teachers as possible. For example, the consistency of change for teachers supports the importance of certain components of the Faith and Wellness Resource: namely, that it is easy to implement without training or experience and is flexible to apply to several grade levels.

The results of this RCT also support the continued importance of SEL intervention evaluation. First and foremost, an evaluation is necessary to determine the efficacy of the intervention. Beyond determining effectiveness, the results of evaluation allow for the continued improvement of SEL instruction and implementation (CASEL, 2019). As a result, it is important to evaluate, not only for efficacy, but for feasibility. In short, are teachers going to use the resource and will students be receptive to its delivery? Our results suggest that not only does the Faith and Wellness Resource support positive outcomes for teachers and students, but that teachers and students found the practices to be both helpful and enjoyable. The high levels of satisfaction suggest, even beyond its efficacy as an intervention, the Faith and Wellness Resource is a tool that teachers and students alike want to use. Our results suggest that teachers are receptive to short, daily practices that can be adjusted to fit individual class needs, schedules, and developmental stages, such as those included in the Faith and Wellness Resource. The study’s participants are members of the Catholic school system; as such, it is unknown whether results can generalize to teachers or students in other geographic contexts or school systems. However, participants in this study do represent diversity across Ontario (e.g., both rural and urban areas), and the intervention itself is designed to be applicable regardless of context. Thus, it may be that results can generalize to other elementary school populations, though further research is needed to determine this potential generalizability.

Limitations and Future Directions

The evaluation of the Faith and Wellness Resource was limited by school closures due to the COVID-19 global pandemic. As a result, the planned follow-up evaluation was not possible. Though qualitative teacher data supports our expectations that students were able to retain some of the SEL skills even through the stressful experience of the pandemic (Peddigrew et al., 2022), this was not investigated using quantitative methods. In addition to the impacts of the COVID-19 pandemic, the retention of teacher participants may have been limited by the ongoing labor unrest that occurred in the province, from T1 through the implementation period. The pandemic and the labor strikes did contribute to teacher attrition and subsequent missing data, which represents a limitation for the current study. However, these events—while certainly challenging to the research process—offer a strength as well: the results of this study under the conditions of real-world interruptions (e.g., COVID-19 pandemic, labor strike) might reflect the feasibility of this Faith and Wellness Resource in imperfect, practical circumstances.

Another limitation to the current study reflects teacher reporting, including both fidelity and dosage. It is important to note that the teacher effects are the results of teacher report only and may reflect a bias in reporting since the teachers in Group 1, who participated in the SEL lessons, knew that they were part of the intervention group. Similarly, the number of times each Group 1 teacher implemented practices was based on teacher’s own initiative. Though teachers were asked to complete 18 practices, not all did (or did not report doing so). Indeed, 41% of Group 1 teachers did not complete any feedback forms; thus, we assumed that they did not complete any practices. By including all Group 1 teachers in analyses, including some who may not have completed practices (i.e., intent to treat), it is possible that treatment effects would be even stronger had all teachers completed the recommended dosage of practices.

In contrast, some teachers implemented more practices, and such were able to choose how many practices to utilize and with what frequency. It may be possible that more enthusiastic teachers chose to implement more than the minimum number of practices. In other words, the current findings may reflect teachers who are willing to frequently implement SEL practices. Future research could consider assigning teachers to dosage groups to better understand SEL dosage effects. Further, teachers’ level of fidelity was not monitored. It is possible teachers were not always implementing practices as planned; however, by design, the practices included in the Faith and Wellness Resource were constructed to be simple and straightforward so that they could be delivered accurately without training. In addition, teachers did report on modifications made to their delivery of the practices. All modifications were structural (e.g., increased length of time spent on practice, modifications for physical abilities) rather than based on content, supporting the notion that teachers were implementing the core components of the practices with fidelity.

Implications and Conclusions

The results of this offers preliminary evidence to support Faith and Wellness Resource as an SEL intervention. The Faith and Wellness Resource seems to support teachers’ use of SEL practices in the classroom as well as to be associated with teachers’ confidence in the delivery of such practices. The results of the Faith and Wellness Resource RCT underscore that SEL is both important and effective, especially when delivered by teachers in classroom settings. Considering the common barriers to SEL implementation such as cost, time commitment, and lack of teacher confidence, the results of this evaluation suggest that not only is a fast, no-cost, and simple evidence-based and implementation-sensitive intervention possible, but that it can also potentially result in positive changes for teachers and their students. Additionally, results support the notion that teachers can be effective changemakers for students’ SEL, even through the use of short, simple practices that do not require formal training. While a more rigorous study is needed to determine the overall efficacy of the Faith and Wellness Resource, the findings from this study offer promising results.

More broadly, the results of this RCT support the ways in which evaluations can offer empirical support for the incorporation of evidence-based SEL interventions in educational practice, with the potential to inform educational policy (Liu & Huang, 2017; Stillman et al., 2018). For instance, through evaluation, researchers can highlight effective strategies for school- and board-wide improvements in SEL programming (Stillman et al., 2018). This evaluation further supports the importance of providing teachers with SEL practices and strategies that are centralized, short, adaptable, and require little to no formal training. In addition, the results of the Faith and Wellness Resource RCT offer support for the positive teacher and student outcomes associated with SEL delivery in schools. In particular, these results highlight the important role that teachers play in the implementation and delivery of effective SEL. As such, evaluations also offer support for the inclusion of SEL training in teacher education programs in order to support teachers’ implementation of SEL curriculum (Archambault et al., 2012; Main, 2018). It is clear that SEL contributes to healthy psychosocial development (e.g., Durlak et al., 2011; Taylor et al., 2017). The results of the current evaluation support that classroom teachers who frequently use short, simple, and cost-effective SEL practices, such as the Faith and Wellness Resource, may play an important role in the promotion of children’s cognitive, social, and emotional development (Hough et al., 2017).

Below is the link to the electronic supplementary material.

Supplementary file1 (DOCX 15 KB)