Evaluation of a health promoting schools program in a school board in Nova Scotia, Canada.

A Health promoting schools (HPS) approach aims to make schools a healthy place through a holistic approach that promotes a supportive 'school ethos' and emphasizes improvements in physical, social, and emotional well-being and educational outcomes. A HPS initiative in rural Nova Scotia (Canada) provided an opportunity for a population-level natural experiment. This study investigated student well-being and health behaviours between schools with and without HPS implementation and schools with high and low school ethos scores. Student well-being, nutrition, and physical activity were examined in a cross-sectional survey of elementary students in Nova Scotia, Canada in 2014. Multiple regression was used to assess the relationship with student well-being using the Quality of Life in School (QoLS) instrument and health behaviours. The main exposure was attending one of the 10 HPS schools; secondary exposure was the school ethos score. The overall QoLS score and its subdomain scores in the adjusted models were higher in students attending HPS schools compared to those in non-HPS schools, but the differences were not statistically significant and the effect sizes were small. Students in schools that scored high on school ethos score had higher scores for the QoLS and its subdomains, but the difference was only significant for the teacher-student relationship domain. Although this study did not find significant differences between HPS and non-HPS schools, our results highlight the complexity of evaluating HPS effects in the real world. The findings suggest a potential role of a supportive school ethos for student well-being in school.

Introduction

School is an important part of a child's life, and the school years are considered a crucial period of childhood development (Eccles, 1999). Healthy child development in turn is associated with better health outcomes later in life (Mikkonen and Raphael, 2010). Schools offer an ideal setting for health promotion interventions as most children spend a large part of their day there (Baranowski et al., 2000, Veugelers and Fitzgerald, 2005a, Sacchetti et al., 2013). Many school-based health promotion interventions have traditionally focused on changing individual behaviour (Alvaro et al., 2011), rather than targeting broader social or environmental determinants that influence behaviour (Coburn et al., 2003). Multicomponent interventions in schools that combine educational, curricular, policy, and environmental elements are thought to be more effective than interventions targeting single components or behaviours (van Sluijs et al., 2007, Kriemler et al., 2011). Health Promoting Schools (HPS, also known as Coordinated School Health or Comprehensive School Health) is such a multicomponent intervention that emphasizes improvements in educational outcomes as well as physical, social, and emotional well-being (International Union of Health Promotion and Education, 2009). Internationally, HPS has been found to have small, but positive effects on health behaviours and some aspects of social well-being (Langford et al., 2014). Within Canada, there has been less formal research on HPS approaches; however, some studies have demonstrated effectiveness of HPS in improving children's health behaviours (Fung et al., 2012, Veugelers and Fitzgerald, 2005b, Reed et al., 2008, Naylor et al., 2006).

The framework is adapted from recommendations by the World Health Organization and focuses on fostering health and learning, engaging all school partners (staff, students, parents, and community), providing an environment that supports health, and implementing healthy policies and practices (World Health Organization, 2016). The Pan-Canadian Joint Consortium has developed a framework for comprehensive school health (CSH) in Canada that includes four distinct but interrelated pillars: Teaching and Learning, Healthy School Policy, Physical and Social Environments, and Partnerships and Services (Pan-Canadian Joint Consortium for School Health, 2016). The adaptability of HPS is an important feature as it ensures flexibility to diverse school contexts across the country (Veugelers and Schwartz, 2010, Keshavarz et al., 2010). However, this variability has also led to considerable uncertainty as to how HPS should be implemented and evaluated across schools (Deschesnes et al., 2003, Mũkoma and Flisher, 2004, McIsaac et al., 2015a).

School ethos reflects the various physical and psychosocial structures that may shape school environments (Parcel et al., 2003) and, in turn, influence the health and well-being of students. Although school ethos is understood as being an essential component of HPS (Samdal and Rowling, 2011, Rowling and Samdal, 2011), there is little published research on how it might influence HPS, and there are no existing measures of this construct. Implementation of CSH and HPS in Canada has varied according to jurisdictional support (Veugelers and Schwartz, 2010). In Nova Scotia, HPS has evolved to a provincial initiative that aims to create healthier school communities for all children in the province (McIsaac et al., 2015b, McIsaac et al., 2012). The HPS initiative is based on the needs and assets of individual school communities. With provincial funding and support, the Tri-County Regional School Board (TCRSB) began implementing HPS in some of its schools starting in 2006 with voluntary enrollment in the program ongoing. As of 2014, ten of the 18 elementary schools in the school board had adopted the HPS approach, setting the stage for a population health natural experiment (Hawe and Potvin, 2009). The objective of the current study was to compare student well-being and health behaviours between schools in the TCRSB with and without voluntary HPS implementation, thereby representing a natural experiment (Hawe and Potvin, 2009). The secondary objective was to compare the same student outcomes between schools with high and low school ethos as an additional measure for HPS implementation.

Methods

The TCRSB HPS approach

HPS in the TCRSB relies on school interest and the readiness of schools to become involved. Schools implementing a HPS approach receive funding to support planning, development, and implementation of school-based action plans. These plans are developed by “school action teams” that meet regularly and include school staff, community partners and students. A “school supporter,” employed by public health, recreation, or the school board, works with each school to assist with planning, priority-setting and evaluation. These supporters represent a member of the board-level steering committee and ensure that school actions are consistent with the HPS approach. Each school determines its own priorities based on information collected from parents, students, and school staff regarding needs and community assets. Notably, schools not formally a part of the HPS initiative may still implement health-promoting activities, but do not receive specific funding or other HPS support described above. For example, all schools in the TCRSB receive funds to implement a mandatory provincial food and nutrition policy and after-school programs. The difference between HPS schools and non-HPS schools is that for HPS schools, these strategies would be part of a planned comprehensive program with additional funding and support. Those schools not enrolled in HPS may implement some programs independently, possibly by individual staff members, but not as part of a broader school plan (Tri-County Regional School Board, 2016).

Study design

Study design and procedures of the TCRSB HPS evaluation have been reported previously (Ghotra et al., 2016). The project was a cross-sectional evaluation of a natural experiment comparing TCRSB HPS and non-HPS schools. Data were collected in spring 2014 through a population-based survey of students in grades 4–6 (about 9–12 years old) and their parents in the TCRSB in Nova Scotia, Canada. The TCRSB is a rural school board in southwestern Nova Scotia covering an area of over 7000 km2. Approximately 6400 students attend 27 schools in the TCRSB. Data collection included information on student health behaviour and well-being, and school environment through surveys with students, parents, school leaders, and teachers, along with an audit of the school environment. All 18 elementary schools with grade 4–6 students in the school board were invited to participate. Packages containing consent forms and a survey were sent home with all students to obtain parental consent. Trained research assistants visited schools to administer a survey to participating students that assessed physical activity and sedentary behaviour, self-efficacy, and quality of life in school, along with a version of the Harvard Youth Adolescent Food Frequency Questionnaire (YAQ) that was modified to reflect the Canadian context (Rockett et al., 1995). All participating students completed the questionnaires by themselves in their classroom as a group. The parent survey contained questions on sociodemographic factors, the home environment, their child's health and their dietary and physical activity behaviours. All eligible schools agreed to participate, and parental consent was obtained for 670 students resulting in a response rate of 46% (46% in HPS schools and 48% in non-HPS schools).

Ethics approval for this study was obtained from the Health Sciences Research Ethics Board at Dalhousie University (file #2013–3094). Informed written consent was obtained from the parents of participating children; children provided written assent. Permission for data collection was also granted from the TCRSB.

Outcomes

The primary outcome was the students' quality of life in school as assessed by the Quality of Life in School (QoLS) instrument, a measure of students' general well-being and satisfaction that is based on positive and negative experiences of school activities (Weintraub and Bar-Haim, 2009). The original version of the QoLS was developed and validated in the Hebrew language and consists of 37 items in four domains: teacher–student relationship and school activities, physical environment, negative feelings towards school, and positive feelings towards school (Weintraub and Bar-Haim, 2009). The current study used the English translation (by its creators) with some minor changes to the wording of some of the items (by the authors). Factor analysis in the current sample confirmed the 4-factor structure of the instrument and was used to remove three items, leaving 34 items in four domains (psychosocial, attitude towards school, school environment and teacher-student relationship) (Ghotra et al., 2016). Items were scored on a 4-point Likert scale (from “always true” to “never true”) with some items being reverse scored; overall and domain scores were calculated by averaging the items in the respective scale.

Secondary outcomes were diet quality, physical activity, screen time, and self-efficacy. Diet Quality was assessed using the Diet Quality Index (DQI). The DQI is a composite score ranging from 0 to 100 that includes aspects of diet adequacy, variety, balance, and moderation, with higher scores indicating better diet quality (Kim et al., 2003). This score was calculated based on student responses on the YAQ that were linked with information from the Canadian Nutrient File database (Health Canada, 2015). Physical activity was assessed with the Physical Activity Questionnaire for Children (PAQ-C), which was filled out by the students. The PAQ-C is a self-administered, validated, 7-day recall instrument that was developed to assess general levels of physical activity throughout the school year for elementary students, including time spent during and after school and on weekends (Kowalski et al., 2004). The PAQ-C score ranges from 1 (low) to 5 (high) and was calculated as the mean score of nine questions related to frequency and intensity of physical activity. Daily screen time was estimated by combining the parent-reported hours per day (< 1 h per day, 1–2 h per day, 3–4 h per day, 5 or more hours per day) children spent using a computer, playing video games, or watching television and then dichotomized (> 2 vs. ≤ 2 h per day). Self-efficacy for healthy eating and physical activity was assessed using single questions (“I can eat fruits and vegetables every day” and “I can be physically active most days of the week”, respectively) and responses were dichotomized as Agree vs. Not sure/Disagree. Continuous primary and secondary outcomes were converted to within-sample z-scores.

Exposures

The main exposure of interest was attending one of the 10 schools in the TCRSB that implemented the board's HPS program before the 2012–13 school year (1.5 years before the data collection for the current study). Reflective of the real-world nature of this study, health promoting activities were present in all 18 schools as a result of a mandatory nutrition policy, school health curricula, and an emphasis on after-school physical activities and mental well-being throughout the province. Therefore, the secondary exposure was a measure for school ethos that was conceptualized to represent school structures and processes (administrative support and leadership, staff support, school connectedness, resources) (Parcel et al., 2003). This Health Promoting School Ethos (HPSE) score was intended to assess the health-support environment of each school regardless of its state of HPS. The HPSE included a 120-item instrument representing eight theoretical domains (consciousness of health, safe surroundings, reinforcement of health, sense of belonging, availability, accessibility, aesthetics, and resources) developed by the authors in consultation with schools (Penney et al., 2016). Data for HPSE were collected through surveys from school leaders and teachers and an audit of the school environment by a trained observer allowing for a range of data sources relevant for each concept. Relative sub-scores were created for each domain of the conceptual framework and tested for internal consistency. Sub-scores were then summed to create an overall HPSE score. Schools with a high HPSE score ≥ 5 (out of 8) reported high on several indicators for supporting student health and well-being, irrespective of their HPS school status.

Other variables

Other variables included in the analysis were student sex; age; household income ($0 to $40,000; $40,001 to $60,000; $60,001 to $100,000; >$100,000 CDN); highest parental education attainment (secondary school or less; college; university); and area of residence (urban vs. rural, based on the Canadian postal code).

Statistical analysis

Sample characteristics were summarized by HPS status of the school and compared using t-test or chi-squared test as appropriate. A series of linear, logistic, and Poisson regression models with a clustered (school) sandwich estimator was used to examine the associations between a school's HPS status or HPSE score, respectively, and the outcomes. Continuous outcomes were converted to within-sample z-scores prior to the analysis. Beta coefficients from the linear regression models therefore represent the z-score difference between HPS and non-HPS schools. Poisson regression with robust standard errors was used instead of logistic regression for binary outcomes with a prevalence > 10% to avoid overestimation of risk ratios by odds ratios (Knol et al., 2012). Models were adjusted for household income, household education, and area of residence based on a priori assumptions about confounding. The model with the outcome DQI was further adjusted for energy intake (Willett et al., 1997). Children with energy intakes < 500 kcal or > 5000 kcal were excluded from analyses of DQI. The statistical analysis was performed using Stata/SE 13 (Stata Corp., College Station, TX, United States).

Results

A total of 636 students (401 in HPS and 235 in non-HPS schools) had complete information for the primary outcome and were included in the analysis. Characteristics of the study sample are summarized by HPS status of the school in Table 1. Children in HPS schools tended to come from households with higher socioeconomic status and were significantly less likely to live in rural areas. There was no difference in HPSE score between HPS (median: 3, range: 1–6) and non-HPS schools (median: 3, range: 1–8) (P = 0.82).

Table 1

Sample characteristics by HPS status of the school in the Tri-Country Regional School Board, NS, Canada in 2014. Numbers are presented as mean (standard deviation) unless indicated otherwise.

	HPS (n = 401)	Non-HPS (n = 235)	P
Male sex [%]	47.7	48.3	n.s.
Age (years)	10.9 (1.0)	10.9 (0.9)	n.s.
Household education [%]			n.s.
High school or less	24.1	29.3
College	51.1	47.1
University	23.1	22.3
Household Income [%]			n.s.
$40,000 or less	25.9	27.6
$40,001–$60,000	12.9	17.6
$60,001–$100,000	24.3	20.4
>$100,000	14.5	10.4
Missing	22.4	24.0
Rural residence [%]	53.8	85.6	< 0.01
Quality of Life in School z-scores
Overall	0.06 (1.01)	− 0.11 (0.97)	< 0.05
Psychosocial domain	0.06 (0.98)	− 0.10 (1.03)	n.s.
Attitude towards school domain	0.08 (0.96)	− 0.14 (1.04)	< 0.05
School environment domain	0.06 (1.03)	− 0.10 (0.93)	n.s.
Teacher domain	0.01 (1.02)	− 0.03 (0.96)	n.s.
PAQ-C z-score	− 0.01 (0.96)	0.01 (1.06)	n.s.
Diet Quality Index z-score	0.04 (0.99)	− 0.07 (1.02)	n.s.
Screen time > 2 h/day [%]	51.4	49.2	n.s.
Self-efficacy (diet) [%]	76.1	73.7	n.s.
Self-efficacy (physical activity) [%]	80.2	81.7	n.s.

Abbreviations: HPS Health promoting schools, PAQ-C Physical Activity Questionnaire for Children.

Results from the regression analysis for the comparison between the HPS and non-HPS schools are shown in Table 2. The overall QoLS score and its subdomain scores in the adjusted models were higher in students attending HPS schools compared to those in non-HPS schools, but the difference was not statistically significant and the effect size was small. The secondary outcomes also did not show any statistically significant differences between HPS and non-HPS schools.

Table 2

Comparison of students' quality of life in school and health behaviours between HPS and non-HPS schools in the Tri-Country Regional School Board, NS, Canada in 2014. Associations are presented as beta coefficients and prevalence ratios with 95% confidence intervals; beta coefficients represent the z-score difference between HPS and non-HPS schools.

	Unadjusted	Adjusted
Beta coefficient (95% CI)
Overall QoLS	0.24 (− 0.06, 0.54)	0.23 (− 0.06, 0.52)
QoLS Psychosocial domain	0.21 (− 0.07, 0.50)	0.24 (− 0.02, 0.49)
QoLS Attitude towards school domain	0.25 (− 0.02, 0.51)	0.21 (− 0.05, 0.48)
QoLS School environment domain	0.23 (− 0.14, 0.60)	0.22 (− 0.14, 0.59)
QoLS Teacher-student relationship domain	0.09 (− 0.18, 0.35)	0.07 (− 0.20, 0.35)
PAQ-C	− 0.02 (− 0.26, 0.22)	0.00 (− 0.31, 0.31)
Diet Quality Index	0.15 (0.01, 0.28)	0.11 (− 0.04, 0.25)
Prevalence ratio (95% CI)
Screen time > 2 h	1.05 (0.84, 1.30)	1.04 (0.82, 1.32)
Self-efficacy (diet)	1.03 (0.86, 1.25)	1.05 (0.86, 1.28)
Self-efficacy (physical activity)	0.98 (0.82, 1.17)	0.99 (0.81, 1.20)

Models were adjusted for household income, household education, and area of residence. The Diet Quality Index model was further adjusted for energy intake.

Abbreviations: CI confidence interval, HPS Health Promoting Schools, PAQ-C Physical Activity Questionnaire for Children, QoLS Quality of Life in School.

Statistically significant estimates are highlighted in bold.

Table 3 shows the results of the comparison between schools with a higher or lower HPSE score, regardless of HPS status. Students in schools that scored high (≥ 5) on the HPSE had higher scores for the QoLS and its subdomains, but the difference was only significant for the teacher-student relationship domain in the adjusted model, and the effect size was small. There were no statistically significant differences between the two groups for any of the other outcomes.

Table 3

Comparison of students' quality of life in school and health behaviours between schools with high (≥ 5) and low Health Supporting School Ethos scores in the Tri-Country Regional School Board, NS, Canada in 2014. Associations are presented as beta coefficients and prevalence ratios with 95% confidence intervals; beta coefficients represent the z-score difference between schools with high and low Health Supporting School Ethos scores.

	Unadjusted	Adjusted
Beta coefficient (95% CI)
Overall QoLS	0.26 (− 0.07, 0.60)	0.30 (− 0.01, 0.61)
QoLS Psychosocial domain	0.27 (− 0.04, 0.58)	0.23 (− 0.03, 0.50)
QoLS Attitude towards school domain	0.07 (− 0.26, 0.40)	0.11 (− 0.19, 0.40)
QoLS School environment domain	0.31 (− 0.10, 0.71)	0.34 (− 0.05, 0.74)
QoLS Teacher-student relationship domain	0.22 (− 0.06, 0.50)	0.31 (0.04, 0.58)
PAQ-C	0.04 (− 0.22, 0.30)	0.00 (− 0.31, 0.31)
Diet Quality Index	− 0.14 (− 0.28, 0.01)	− 0.09 (− 0.24, 0.07)
Prevalence ratio (95% CI)
Screen time > 2 h	0.86 (0.68, 1.10)	0.85 (0.65, 1.11)
Self-efficacy (diet)	1.10 (0.90, 1.33)	1.07 (0.86, 1.33)
Self-efficacy (physical activity)	0.98 (0.81, 1.19)	0.95 (0.77, 1.18)

Models were adjusted for household income, household education, and area of residence. The Diet Quality Index model was further adjusted for energy intake.

Abbreviations: CI confidence interval, PAQ-C Physical Activity Questionnaire for Children, QoLS Quality of Life in School.

Discussion

The present study compared student well-being and health behaviours in schools based on HPS status and school ethos score. Although the effects were mostly positive, we did not find statistically significant differences between students attending HPS schools compared to non-HPS. Students from schools scoring high on the HPSE score had significantly higher scores in the teacher-student relationship domain of the QoLS instrument, but there were no statistically significant differences between the two groups for any of the other outcomes.

The current study is one of the first in Canada to evaluate a school-board HPS approach in terms of student outcomes, and specifically within the context of a natural experiment. The study is therefore both unique and offers important insight into how a HPS approach might contribute to student health and wellbeing. We learned that school ethos is an important, but as yet, poorly understood construct that might contribute to student health and wellbeing and additional study is warranted on this aspect of HPS. Three school-based programs in Canada have published findings on multi-component school interventions focused on obesity prevention (Fung et al., 2012, Reed et al., 2008, Campbell et al., 2012). The APPLE Schools (Alberta Project Promoting active Living and healthy Eating) program uses a CSH approach via a dedicated school-health facilitator to tailor health promotion strategies to each school's unique needs. In comparison to changes observed in students elsewhere in the province, early results have demonstrated positive student outcomes in terms of diet, physical activity and weight status (Fung et al., 2012). Healthy Buddies™ is a multi-component school intervention that is facilitated by teachers and emphasizes peer support. Relative to comparison schools, students attending intervention schools have demonstrated increases in healthy-living knowledge and behaviours (Campbell et al., 2012). Finally, Action Schools BC provides teachers and schools with training and resources to implement physical activity and healthy eating using a comprehensive approach. Overall, findings from this intervention have found modest enhancements in physical activity, improved cardiovascular fitness, and increases in the number of servings and variety of vegetables and fruit compared with other schools (Reed et al., 2008, McKay et al., 2015). Evaluations of natural policy experiments for HPS on student outcomes are less common. Although a previous study in Nova Scotia did not find consistent or significant favorable physical health benefits resulting from enhanced implementation of HPS, there were fewer negative trends among schools at these enhanced levels (McIsaac et al., 2015b).

The overall intent of the HPS approach is to improve social and emotional well-being as well as physical health. Previous studies have often used diet quality or physical activity as primary HPS outcomes (Fung et al., 2012, Reed et al., 2008, Campbell et al., 2012, McKay et al., 2015), but capturing the effects of such a holistic intervention is difficult if only individual health behaviour outcomes are measured (Keshavarz et al., 2010, Mũkoma and Flisher, 2004). A previous study from Canada that compared health-related quality of life between schools with and without a HPS program did not find any differences despite higher levels of physical activity and better diets in the HPS schools (Wu, 2012). Our study included a measure of social and emotional well-being to assess students' health, educational and social outcomes across different aspects of the school environment (Ghotra et al., 2016), and provided a better measure of the effects of the holistic HPS intervention. Given the similarities between the concept of quality of life in school and HPS, similar measures (Konu and Lintonen, 2006, Mok and Flynn, 2002) may be important for future evaluations of HPS programs.

Because a school's HPS status may not accurately reflect implementation of the HPS approach, we included the HPSE as an additional exposure variable. Few tools exist to specifically measure HPS implementation given its extensive variability (Veugelers and Schwartz, 2010), and the tool developed for the current study focused on school ethos as one conceptual dimension that may shape school environments. School ethos is often understood as being essential for HPS, but no published research exists to describe the relationship of school ethos to the overall HPS approach (Samdal and Rowling, 2011, Rowling and Samdal, 2011). Although our HPSE instrument requires further testing and refinement, we did observe higher (but not statistically significant) QoLS scores in schools with high HPSE scores compared to schools with low scores, although the difference was only significant for the teacher-student relationship domain. Considering this result, future research should investigate teachers' competence to support relationships and promote positive social and emotional learning for students (Jennings and Greenberg, 2009).

Our study has several other notable limitations. The challenges of demonstrating positive outcomes in natural policy interventions are well known (Hawe and Potvin, 2009, Petticrew et al., 2005) and may have contributed to the lack of differences between the HPS and non-HPS schools. The researchers were not involved with the implementation of the initiative, which began over six years preceding this study. As a result, outcomes were not tracked from the beginning and schools joined the initiative at different times resulting in varied progress in implementation. The cross-sectional design of the study made it impossible to examine longitudinal trends in student well-being and health behaviours, or examine long-term outcomes of HPS. Further, our modest response rate was mainly due to the lack of support for data collection among schools as result of other initiatives and may have contributed to the apparent lack of difference between HPS and non-HPS schools. Another limitation was the concurrent implementation of HPS-related activities among non-HPS schools, which may have diminished any difference that may exist between the two groups.

One of the strengths of our study was the use of the validated QoLS instrument (Ghotra et al., 2016, Weintraub and Bar-Haim, 2009) as a holistic measure of student well-being in school. As previously mentioned, the HPS label may be limited in its distinction of implementation differences but our use of the HPSE provided another measure for understanding differences at the school-level using both self-reported and observational information to describe features of school ethos. A full description of the development of the HPSE tool is currently under review and further research will be needed for validation.

Conclusions

Although this study did not find significant differences between HPS and non-HPS schools, our results highlight the complexity of evaluating HPS in the real world. The findings suggest the potential role of a supportive school ethos for well-being in school. Evaluations of HPS should continue to explore measures of implementation, gather detailed information on the school environment to help interpret outcomes observed, and consider a measure of student well-being as an outcome. Further, longer term and longitudinal research is needed to demonstrate the potential effects of HPS on student health, well-being, and academic achievement into the future.

Funding

This work was supported by a Population Health Intervention Research grant from the Canadian Institutes of Health Research (CIHR) (FRN 127082). The funder had no role in study design; in the collection, analysis, and interpretation of data; in the writing of the manuscript; and in the decision to submit the manuscript for publication.

Sara Kirk holds a CIHR Canada Research Chair in Health Services Research. Jessie-Lee McIsaac was supported by a postdoctoral fellowship from KT Canada (CIHR) as well as a Bright Red Graduate Research Award from the Heart and Stroke Foundation of Canada and the Canadian Cancer Society (grant #703878). Tarra Penney was supported by the Cambridge International Scholarship from the Cambridge Commonwealth, European and International Trust.

Conflict of interest

Lori Munro-Sigfridson (Active Healthy Living Consultant with the Tri-County Regional School Board) and Jane Cunningham (Nutritionist with the Nova Scotia Health Authority) are involved in the implementation of the HPS intervention studied. The remaining authors have no conflict of interest to declare.