A longitudinal cohort study of participation in the Boys & Girls Clubs of Winnipeg.

Introduction: Out-of-school or after-school programs are designed to support healthy behaviours, boost academic achievement and strengthen social skills. The Boys and Girls Clubs of Winnipeg (BGCW) in Manitoba, Canada, have been offering out-of-school programs to children and adolescents in low-income neighbourhoods for more than 40 years. Many BGCW participants face considerable risk of poor health, social and educational outcomes due to challenges they experience at home, in school and in their communities. Objective: We examined whether BGCW participation among children and adolescents aged 5-18 is associated with improved school outcomes and lower risk of justice system encounters and adverse health outcomes.
Methods: We linked de-identified data on BGCW participation for children and adolescents born 1987-2010 to administrative data from the healthcare system, education system, and social services in the Population Research Data Repository, Manitoba Centre for Health Policy. The exposure group (n = 8,990) included children and adolescents who visited BGCW at least once from 2005-2016. The comparison group (n = 69,980) comprised children and adolescents matched on school year and neighbourhood who had never participated in BGCW.
Results: Participation in BGCW was significantly associated with better scores in grade 3 numeracy and grade 7 student engagement assessments. The risk of justice system encounters among adolescents (aged 12-17) dropped as the frequency of BGCW participation increased, as did justice system encounters among young adults (aged 18-24) who had participated in BGCW as adolescents. The likelihood of teen pregnancy among female adolescents (aged 13-19) and sexually transmitted infections among adolescents (aged 13-19) also declined as the frequency of participation in BGCW increased. Conclusions: The findings suggest that participation in BGCW is associated with positive outcomes for children and adolescents in multiple domains and despite socioeconomic and family challenges faced by many in this population.

Introduction

Poverty has been recognised as a critical social determinant of child health in high-income countries [1-3]. There is abundant evidence that children in low-income households are disadvantaged across a range of health, social and education outcomes when compared to their higher-income peers. On average, children who grow up in low-income households have poorer health and lower scores on tests of cognitive, social and behavioural development [4-7]. They perform more poorly in educational assessments, have lower self-esteem as adolescents, and are more likely to become involved in criminal or delinquent behaviour [8-10]. A growing body of literature is exploring whether there is a direct causal link between household income and children’s developmental outcomes and whether the relationship is mediated by one or several interacting mechanisms [11, 12]. For example, ‘investment theory’ posits that parents with higher incomes have more financial resources to help their children thrive [13], and the ‘family stress model’ proposes that economic hardship can cause parental stress and limit nurturing parenting behaviours [14]. Regardless of mechanism, however, it is clear that childhood poverty has long-term impacts on health, academic outcomes and employment opportunities later in life, all of which are societal measures of success and upward social mobility.

Addressing poverty as a social determinant of health should be a priority for national, provincial and municipal governments, but progress is often slow and the available funding is not always adequate to bring about significant change. Meanwhile, the immediate challenges for lower-income families persist and disparities in health, social and education outcomes continue to grow. Many children of middle- and high-income families have access to a variety of organised recreational and learning activities with fees typically paid by their families [15]. In contrast, children growing up in low-income families often have limited opportunities to participate in extracurricular activities [16, 17]. In this context, localised approaches and interventions can play an important role in supporting the healthy development of socioeconomically disadvantaged young people. One example of this type of intervention is after-school or ‘out-of-school’ programs, which provide organised activities for school-aged children and adolescents aimed at promoting their physical, mental, educational and social well-being.

The benefits of out-of-school programs have been widely studied in the USA. The evidence suggests that high quality programs designed to keep young people safe and in appropriately structured and engaging environments can produce many positive outcomes [18-23]. Key features of high quality programs include clear, pre-determined goals and objectives; activities that elicit a sustained engagement and effort, and provide opportunities to build or develop skills; and high quality instructional support, including supportive and positive relationships between staff and participants [16, 20, 23]. Regular attendees of such programs have better standardized test scores and school attendance and fewer behavioural problems [24], they exhibit more positive self-perceptions and social behaviours [25], and they have better concentration and regulation skills [26] than those who rarely or never attend.

A prominent organisation providing out-of-school programs is the Boys & Girls Clubs, with about 4,000 clubs across the USA and more than 700 in Canada. The Boys & Girls Clubs of America have produced annual outcomes reports since 2011. Through collection of survey data from regular participants, they have demonstrated positive associations between participation in the clubs and outcomes across multiple areas. For example, compared to children and youth who never or rarely attend club activities, regular participants in the clubs are more likely to believe that their school work is meaningful, achieve better grades, graduate from high school, and have the intention to pursue post-secondary education [27]. They are more likely to consume the daily recommended servings of fruits and vegetables and be physically active 5+ days per week [27]. As well, they are less likely to consume alcohol or smoke cannabis, and more likely to volunteer or be involved in community service [27]. In some jurisdictions, the individual clubs have also been independently evaluated, providing additional insights into their positive impacts on school attendance and achievement [28], identity development and social skills [29], and family engagement in youth development [30]. These are important findings for the American clubs, but may not reflect the reality of the Canadian ones. There are substantial differences between the two countries’ social policies, social safety nets and healthcare systems that impact population-level poverty and employment rates [31]. Because of these differences, interventions and supports for low-income families are worth exploring in both settings.

To date, there have been very few research studies on the impacts of out-of-school programs like the Boys & Girls Clubs in Canada. One of these few is a qualitative study from 2014 that examined the social and emotional development of teens aged 16–18 who participated in a club in Alberta [32]. The interviewees reflected on the ways in which their sense of self and their relationships with others changed through their involvement with the club. To the best of our knowledge, quantitative studies on health, social or education outcomes associated with participation in Boys & Girls Clubs in Canada have not yet been conducted. However, in Manitoba, we recognised a unique opportunity to evaluate the Boys & Girls Clubs’ programming using population-based administrative data. The information-rich Population Research Data Repository, located at the Manitoba Centre for Health Policy, is a collection of 50+ years of routinely-collected administrative data on Manitobans’ interactions with the healthcare system, education system, social services and justice system [33, 34]. Using the Repository for research allows evaluation of programs and services in a ‘real-world’ setting using non-intrusive data unbiased by survey sampling and recall errors and without incurring the expense of primary data collection. The breadth and depth of population data available in Manitoba are unmatched in Canada.

Thus, a partnership between the Boys & Girls Clubs of Winnipeg (BGCW) and researchers at the Manitoba Centre for Health Policy was formed to examine BGCW participants. The study aim was to gain a better understanding of BGCW participants’ family and socioeconomic characteristics and to examine their health, social and education outcomes.

Methods

Study setting and approach

Manitoba is a central Canadian province with a population of 1.4M. The major urban centre is the capital city of Winnipeg (pop ~800,000). The remainder of the Manitoba population lives in smaller towns and cities or in rural/remote communities. Manitoba’s demographic and educational profiles are similar to national averages, which means that research findings from Manitoba population-based studies are typically generalisable to Canada as a whole [35, 36]. The larger of the two Boys & Girls Clubs in Manitoba is located in Winnipeg with 12 sites operating in neighbourhoods across the city. The current study was initiated by BGCW leadership to learn more about the population served by the club and to inform their future planning and programming. Members of the BGCW staff, leadership and board were kept apprised of study progress throughout the research project and were invited to give input on the interpretation of the results when the analyses were completed.

The Boys & Girls Clubs of Winnipeg

The BGCW is one of Winnipeg’s core youth-serving agencies focused on the well-being of children and adolescents. Their programs are designed for young people whose social and family circumstances may affect their overall health, education, social and recreational opportunities. The BGCW works to identify financial, cultural and transportation barriers families may face in enrolling their children in city and community programs. BGCW programs are offered at no charge to give participants the opportunity to discover, develop and achieve their full and positive potential.

BGCW programs

BGCW activities are guided by professional staff and dedicated volunteers, providing participants with positive role models and mentors in a safe and facilitated environment. The staff and volunteers take a preventative and developmental approach to tackling a broad range of critical issues participants may face, and this approach is manifested in their five core program areas:

Education and career exploration: programs that support the development of skills and knowledge for school (e.g. help with homework, tutoring) and employment success

Sports and physical recreation: programs that promote physical fitness and participation (e.g. sports camps), and the development of social and interpersonal skills

Arts and cultural appreciation: programs that encourage creative expression (e.g. through music, dance and photography workshops), cultural awareness and critical thinking

Health awareness and life-skills development: programs that nurture physical, mental and functional well-being (e.g. nutrition classes)

Leadership and service to community: programs that empower participants to support and influence their community, sustain meaningful relationships with others and develop a positive self image (e.g. youth council)

The BGCW are based at schools and community recreation centres and share some facilities (e.g. gymnasiums, kitchens, playgrounds) with these institutions. A wide variety of activities are offered at each club and the specific programs are tailored to meet the diverse needs and interests of participants at each location (more here: https://www.bgcwinnipeg.ca/). For example, some clubs may offer cultural programming in neighbourhoods with large Indigenous, immigrant or refugee populations; some may have a focus on school performance or recreational programming. Some clubs offer programming seven days/week, while others may be open only afternoons on school days. Some programs are available on an ongoing drop-in basis, while others require registration for a certain number of weeks (still free of charge); in both cases, participation is recorded by staff and volunteers daily as children and adolescents arrive at the activity.

Data sources

The data for this study were derived from the Manitoba Population Research Data Repository, housed at the Manitoba Centre for Health Policy, University of Manitoba. The population-based Repository contains approximately 100 administrative, survey and registry databases [37]. This information is routinely collected during the administration of the province’s publicly funded healthcare system, social services, education system and justice system. All records in the Repository are de-identified (i.e. names and addresses are removed), but are linkable at the person level via a nine-digit numeric identifier (the scrambled version of each person’s Personal Health Identification Number) attached to each record. Using this identifier, all records belonging to a single individual or group of individuals can be linked together across different databases and over time, allowing for a broad yet detailed population-based perspective on the health and well-being of Manitobans. The Repository data have been used extensively in health and social research and their validity in population studies has been well documented [33, 37–40].

To examine BGCW participation, an additional database was required. Participation records from the BGCW containing the dates on which individual children and adolescents visited the BGCW from 2005-2016 were de-identified and then linked to existing Repository databases. For more details on the databases used in this study, see Appendix 1.

Study cohort

The exposure group (BGCW participants) included all Manitoba children and adolescents born 1987–2010 who participated in BGCW at least once during the study period of 2005–2016 (Figure 1). The comparison group was created by identifying adolescents who had never participated in BGCW but who were born in the same year (±60 days) and had the same 5-digit postal code as the BGCW participants (i.e. they lived across the street or within one block of each other and had similar neighbourhood characteristics).

Figure 1: Study cohort development

Variables

Exposures

We examined exposure to BGCW in two different ways: 1) as a binary exposure (participated either 1+ times or never), and 2) as a continuous time-varying exposure (by counting the number of times individuals participated).

Outcomes

We examined outcomes in three domains.  Appendix 2 provides detailed definitions for each outcome.

Education outcomes: assessments for competencies in Grade 3 (numeracy, reading), Grade 7 (student engagement, mathematics), and Grade 8 (reading & writing), and high school graduation within two years of the expected date (based on age). For the student assessments, teachers ranked students as “meeting expectations”, “approaching expectations” or “not meeting expectations”, and we counted those who were assessed as meeting expectations in each outcome.

Social outcomes: justice system encounters, defined as any criminal or administrative charges (not including traffic offences) against adolescents aged 12–17 and young adults 18–24, or any record of the individual being a witness to a crime or a victim of a crime. Adolescents in the 12-17 year old age range fall under the provisions of the Youth Criminal Justice Act in Canada.

Health outcomes: teen pregnancy, defined as any pregnancy (including those that did not result in a live birth) among female adolescents aged 13–19, where multiple pregnancies to a single individual were counted as separate events; and testing positive for a sexually transmitted infection (syphilis, chlamydia or gonorrhea), which was determined from the laboratory test results among adolescents aged 13–19.

Descriptive variables

Using data from the Repository, we calculated frequencies and means for the following study cohort characteristics: age of participants at first BGCW visit, biological sex, small-area level income, whether or not the family received income assistance, how often they had moved residences, whether they had ever had any involvement with Child Protection Services (and more specifically, whether they had ever had a child taken into care), the mother’s age at first birth, birth outcomes (prenatal care, low birth weight, preterm birth), and the mother’s mental health.

Statistical analyses

All analyses in this study were conducted using SAS Version 9.4. We first tested for differences between BGCW participant and comparison group characteristics using the binary exposure variable (had participated/had never participated). We used a generalised linear model with binomial distribution for categorical variables and normal distribution for continuous variables with an identity link function and reported the differences at a significance level of p<0.05.

For the outcomes, we developed a generalised propensity score, incorporating the characteristics we observed to be significantly different between groups to predict the number of visits participants made to BGCW. The generalised propensity score was calculated using a zero-inflated negative binomial distribution, with number of BGCW visits as the outcome and the following characteristics as explanatory variables: mother’s age at first birth, the socioeconomic factor index (SEFI-2) [41], number of residential moves, receipt of income assistance, receipt of services from Child Protection Services, child taken into care, and mother diagnosed with a mental disorder. To validate the model, we checked the balance of the model covariates between exposed and comparison groups using a generalised additive model. The outcome model used the continuous exposure variable (number of visits to BGCW) to determine whether more visits to BGCW were associated with better outcomes.

For the education outcomes, we used the generalised additive model to examine the relationship between number of visits to BGCW and “meeting expectations” in various school assessments and graduating high school. We applied the generalised propensity score as an inverse probability of treatment weight to adjust for measured confounders and included a time-varying factor to account for the number of visits to BGCW increasing as time went by. The model generated graph output demonstrating how the likelihood of “meeting expectations” changed with the number of BGCW visits.

For the health and social outcomes, we conducted time-to-event analyses to determine how participation in BGCW was related to the risk of justice system encounters, teen pregnancy, or testing positive for a sexually transmitted infection, and whether a larger number of visits was associated with a reduction in risk. We used the PHREG procedure in SAS, a regression analysis based on the Cox proportional hazards model, adjusting for potential confounders by including the generalised propensity score in the model. The model generated hazard ratios with 95% confidence intervals.

Data privacy, sharing and ethics

As the custodian of the Manitoba Population Research Data Repository, the Manitoba Centre for Health Policy adheres to the rules for privacy and protection of personal information outlined in the province of Manitoba’s Personal Health Information Act [42] and Freedom of Information and Protection of Privacy Act [43]. The records in the Repository are provided under data sharing agreements between the University of Manitoba and the data providers. Multiple layers of protections ensure that the individual privacy of Manitobans represented in the Repository is secure, and that the data linkage process does not compromise the security of their personal information [37]. Research studies proposing to use the Repository data are subject to prior approval from the University of Manitoba Health Research Ethics Board and the Government of Manitoba’s Health Information Privacy Committee.

Results

Study cohort characteristics

The characteristics of the study cohort are presented in Table 1. Approximately half of participants (51.9%) were male. The average age at which children first participated in BGCW was 9.4 years; additional information about the distribution of visits across age groups is available in Appendix 3. Most participants were from lower-income families (55.2% were in Q1 and 18.0% in Q2), and 64.7% were from families who had received income assistance at some point before the children participated in BGCW. Among participants, 49.0% lived in families who had previously received services from Child Protection Services, and 19.9% had been taken into care of Child Protection Services for at least one day. Nearly 44% of participants had a mother who was a teen when she had her first child, and 48.8% had a mother with a previous diagnosis of a mental disorder.

Table 1: Study cohort characteristics

	BGCW Participants Comparison group
	n	% or Mean	n	% or Mean	% or Mean Difference (95% CI)	p-value
Age at first participation in BGCW (mean)	8,990	9.35	69,980	9.97	–0.62 (–0.70, –0.55)	–0.001
Sex (%)
Male	4,661	51.85	35,741	51.07	0.77 (–0.32, 1.87)	0.167
Female	4,329	48.15	34,239	48.93
Income Quintile (%)
Q1 (lowest)	4,960	55.17	39,527	56.48	–1.31 (–2.40, –0.22)	0.019
Q2	1,622	18.04	11,178	15.97	2.07 (1.23, 2.91)	<0.001
Q3	834	9.28	6,709	9.59	–0.31 (–0.95, 0.33)	0.341
Q4	703	7.82	5,444	7.78	0.04 (–0.55, 0.63)	0.893
Q5 (highest)	551	6.13	4,371	6.25	–0.12 (–0.64, 0.41)	0.664
Not found	320	3.56	2,751	3.93	–0.37 (–0.78, 0.04)	0.075
Geography (home residence) (%)
Urban (Winnipeg)	8,271	92.0	59,494	85.02	6.99 (6.37, 7.61)	<0.001
Rural	719	8.0	10,486	14.98
SEFI-2 (mean)	8,586	0.87	67,532	0.91	–0.04 (–0.07, –0.02)	0.001
Family received income assistance (%)	5,818	64.72	31,634	45.20	19.51 (18.46, 20.57)	<0.001
Number of residential moves (mean)	8,934	1.49	57,007	1.36	0.12 (0.09, 0.16)	<0.001
Services provided by Child Protection Services (%)	4,394	48.88	25,280	36.13	12.75 (11.66, 13.84)	<0.001
Child was in care of Child Protection Services (%)	1,785	19.86	11,621	16.61	3.25 (2.38, 4.12)	<0.001
Mom’s age at first birth (mean)	8,835	21.66	64,088	22.28	–0.63 (–0.75, –0.51)	<0.001
Mom was a teen at first birth (%)	3,929	43.70	24,778	35.41	8.30 (7.21, 9.38)	<0.001
Birth outcomes (%)
No prenatal care before third trimester	2,355	26.20	18,436	26.34	–0.15 (–1.11, 0.82)	0.763
Low birth weight	338	3.76	3,109	4.44	–0.68 (–1.10, –0.26)	0.002
Preterm birth	486	5.41	4,089	5.84	–0.44 (–0.94, 0.06)	0.086
Mom had at least one physician visit for mental health reasons (%)
Mood/Anxiety disorder	4,368	48.59	28,614	40.89	7.70 (6.60, 8.79)	<0.001
Psychotic disorder	111	1.23	828	1.18	0.05 (–0.19, 0.29)	0.676
Personality disorder	123	1.37	942	1.35	0.02 (–0.23, 0.28)	0.865
Mom had a diagnosis for a mental disorder (%)	4,386	48.79	28,740	41.07	7.72 (6.62, 8.81)	<0.001

BGCW: Boys & Girls Clubs of Winnipeg; CI: confidence interval; SEFI-2: socioeconomic factor index 2 (on a scale of 0-1, smaller numbers represent higher socioeconomic status).

In the comparison group, a number of these characteristics were significantly less prevalent, including the percentage of families who received income assistance (45.2%), received services from (36.1%) or had a child taken into care of Child Protection Services (16.6%), and the percentage of mothers who were teens when they had their first child (35.4%) and had been diagnosed with a mental disorder (41.1%). Despite the matching of using age and 5-digit postal codes, there were clearly still differences in baseline characteristics between the groups, and we recognised that it would be important to account for these observed differences in our statistical models (see below).

Education outcomes

Crude education outcomes (numeracy, literacy and engagement assessments) for students in grades 3, 7 and 8 and high school graduation rates for grade 12 students are presented in Table 2. We calculated the percentage of students in each study group assessed as ‘meeting expectations’ or being ‘established’ in all of the competencies for each outcome (details on these competencies available in Appendix 2). In this crude analysis, the comparison group performed better than the BGCW participants in all education outcomes except for grade 3 numeracy (no difference between groups).

Table 2: Education Outcomes (crude %)

	BGCW Participants		Comparison Group		% Difference (95% CI)	p-value
	n (N)	%	n (N)	%
Grade 3 Numeracy
Meets expectations in 4 competencies	516 (2,256)	22.87	2,481 (10,948)	22.66	0.21 (–1.69, 2.11)	0.828
Grade 3 Reading
Meets expectations in 3 competencies	603 (2,256)	26.73	3,571 (10,944)	32.63	5.90 (3.88, 7.93)	<0.001
Grade 7 Student Engagement
Established in 5 competencies	861 (3,885)	22.16	4,359 (18,352)	23.75	1.59 (0.15, 3.03)	0.031
Grade 7 Mathematics
Meets expectations in 6 competencies	736 (3,887)	18.93	4,426 (18,352)	24.12	5.18 (3.80, 6.56)	<0.001
Grade 8 Reading & Writing
Meets expectations in 6 competencies	879 (3,676)	23.91	5,620 (18,740)	29.99	6.08 (4.55, 7.60)	<0.001
High School Graduation	995 (1,686)	59.02	10,012 (15,012)	66.69	7.68 (5.21, 10.14)	<0.001

Adjusted education outcomes are presented in Figure 2; here we accounted for the baseline differences in study cohort characteristics and for participation in BGCW by incorporating the generalised propensity score into the statistical model. Figure 2a shows that the likelihood of students meeting expectations in Grade 3 numeracy was significantly associated with the number of visits students made to BGCW. Among all students combined, the range of visits for which this relationship was significant was about 20-450 visits, although the association leveled off around 300 visits. This association was evident among both male and female students (Figure 2b). Figure 2c shows that the likelihood of students being assessed as ‘established’ in learning engagement was significantly associated with the number of visits students made to BGCW (1-600 visits, plateauing at around 320 visits). The association was moderately evident for both male and female students (Figure 2d). Notably, more girls were assessed as ‘established’ in student engagement than boys even before they had any contact with BGCW, although the gap between them diminished at higher levels of BGCW participation (400-500 visits). All other education outcomes (for which we found no significant association with BGCW participation) are presented in Appendix 4.

Figure 2: Education Outcomes associated with participation in the Boys & Girls Clubs of Winnipeg

Social outcomes

Justice system encounters among adolescents aged 12–17

Among 12 to 17-year-olds in the study cohort, 12.2% of BGCW participants (n = 808 of 6,638 individuals) and 10.6% of the comparison group (n = 6,269 of 59,146 individuals) had a justice system encounter during the study period (Figure 3a). Using a time-to-event analysis, the crude hazard ratio was 1.61, indicating that adolescents who had participated in BGCW had 1.61 times the risk of a justice system encounter vs adolescents in the comparison group. We then created an adjusted model that included the generalised propensity score to account for baseline differences between the groups (as described above). When we incorporated the binary exposure variable into the model, the hazard ratio dropped to 1.34 (i.e., the difference between the BGCW group and the comparison group became smaller, but adolescents in the BGCW group were still significantly more likely to have a justice system encounter). However, after including the continuous exposure variable that accounted for the specific number of times they had participated in BGCW, the hazard ratio fell to 0.99 (i.e. the higher the number of visits to BGCW, the less likely adolescents in the BGCW group were to have a justice system encounter).

Figure 3: Social outcomes associated with participation in the Boys & Girls Clubs of Winnipeg

Justice system encounters among young adults aged 18–24

Among young adults aged 18–24 who participated in BGCW before the age of 18, 22.5% of the BGCW group (n = 744 of 3,306 individuals) and 14.9% of the comparison group (n = 5,645 of 37,890 individuals) first had a justice system encounter between the ages of 18–24 (Figure 3b). The crude time-to-event analysis generated a hazard ratio of 1.77, indicating that BGCW participants had 1.77 times the risk of the comparison group of a justice system encounter. In the adjusted model, the hazard ratio dropped to 1.14 after including the binary exposure variable, and to 0.99 after including the continuous exposure variable. This means that in this group of young adults, the higher the number of visits to BGCW while they were age 12-17, the less likely they were to have a justice system encounter while they were age 18-24.

Justice system encounters among young adults (aged 18–24) who had a prior justice system encounter between the ages of 12–17

About 60% of young adults aged 18–24 previously had at least one justice system encounter (Figure 3c). The crude hazard ratio for justice system encounters for ages 18–24 for this subgroup was 1.60. In the adjusted model, the hazard ratio dropped to 1.06 after adjusting for any participation in BGCW, and then to 0.99 after accounting for the number of visits to BGCW. This means that, despite prior justice system involvement as adolescents, higher numbers of visits to BGCW were associated with lower risk of a justice system encounter in the ensuing years.

Health outcomes

Teen pregnancy

A total of 13.1% of female BGCW participants aged 15–19 (n = 373 of 2,845 individuals) and 11.5% of female adolescents aged 15–19 (n = 3,168 of 27,550 individuals) in the comparison group became pregnant during the study period (Figure 4a). In the time-to-event analysis, the crude model generated a hazard ratio of 1.77. After adjusting for different characteristics between the study groups as well as for binary participation in BGCW, the hazard ratio dropped to 1.48, and after adjusting for the number of times adolescents participated in the BGCW, the hazard ratio fell to 0.98.

Figure 4: Health outcomes associated with participation in the Boys & Girls Clubs of Winnipeg

Sexually transmitted infections

A total of 23% of BGCW participants aged 13–19 (n = 1,410 of 6,077 adolescents) and 19.0% of adolescents aged 13–19 (n = 10,705 of 56,344 adolescents) in the comparison group tested positive for an STI during the study period (Figure 4b). In the time-to-event analysis, the crude model generated a hazard ratio of 1.79. After adjusting for different characteristics between the study groups as well as for binary participation in BGCW, the hazard ratio dropped to 1.54, and after adjusting for the number of times adolescents participated in the BGCW, the hazard ratio fell to 0.996.

Discussion

In this study, we examined the socio-demographic characteristics and the health, social and education outcomes of a cohort of children and adolescents aged 5-18 who participated in programming offered by the Boys & Girls Clubs of Winnipeg (BGCW) from 2005-2016. We found that many BGCW participants and their families had experienced circumstances likely to contribute to parental stress and to affect their development and well-being, including poverty, poor maternal mental health and/or involvement with Child Protection Services. When we compared BGCW participants to an age- and income-matched cohort of individuals who had never participated in BGCW and adjusted for other family characteristics, we saw that participants were more or at least as likely to do well in Grade 3 numeracy and to be engaged in learning in Grade 7. Among teenage participants, the risk of having a justice system encounter, a teenage pregnancy or a sexually transmitted infection appeared to be mitigated so that it was not significantly different from non-participants. Although children and adolescents participating in BGCW face many challenges in their home lives, BGCW out-of-school programming offers them a supportive environment and many opportunities for growth and development, and these may contribute to positive health, education and social outcomes in this population. At the same time, many of the outcome effect sizes were small, and for some of the later education outcomes we measured, there were no significant associations with BGCW participation. Further study may be needed to determine why early associations were not sustained and did not translate into better academic performance in later years.

The Boys & Girls Clubs of Canada are a widespread organisation with many member clubs in cities across the country. Although many of the Canadian clubs (and their sister organisations in the USA) collect regular feedback from participants and their families, there is very little peer-reviewed published literature on the Boys & Girls Clubs’ impact on participants’ proximal and distal outcomes. Our Manitoba-based study was unique in being able to leverage a linked population-based administrative data resource to examine longitudinal health, social and education outcomes of participants in BGCW, resulting in one of the first empirical studies on the role the clubs play in influencing young people’s health and social well-being, and contributing to a growing body of literature on the outcomes of participation in out-of-school programming.

The study findings have important real-world implications for BGCW and other youth-serving agencies organisations, for researchers who evaluate them, and for government and private funders.

At BGCW, the findings of this study have informed data collection efforts and have promoted interest in doing further research, e.g. revisiting the analysis with a larger longitudinal cohort and/or doing a “deeper dive” into some of the findings reported here. As the first study of its kind within the Boys & Girls Clubs of Canada organisation, the study also serves as a springboard for future research and evaluation. Nationally and internationally, the study provides an example for other clubs or community organisations interested in establishing their own research and evaluation streams using administrative data.

This study exemplifies a successful research and evaluation partnership and has been a fruitful learning experience for researchers and community partners alike. Researchers at the Manitoba Centre for Health Policy worked closely with leadership at BGCW throughout the research process, including during the study design, data transfer and analysis, and interpretation of the study results. Input from staff who were familiar with the data collection process and what participation in BGCW programs looked like “on the ground” was essential in working with the data, particularly when we detected unexpected patterns. Their involvement also informed our decision to find a way to quantify participation in the analysis, so that they could apply the findings to their member recruitment and engagement strategies. Throughout the study, regular updates to the BGCW board helped to guide the study direction and keep the research questions focusedon what was useful to stakeholders, including participants and their families, staff and volunteers, and funders of BGCW.

Critically, the study provides information for policymakers and other funders to make evidence-informed decisions about how grants, tax dollars and other funds are best spent to improve the health and well-being of Manitobans. For example, programs that work to prevent youth justice system encounters represent a large financial benefit for the province, as detailed in previous reports about the substantial costs associated with youth criminal incidents and court cases [44, 45].

With the recognition that the life-long health of children and adolescents is influenced by many different factors, supporting their growth and development becomes essential for building a vibrant, successful society. Organisations like BGCW play a role in achieving this outcome by helping to set young people on a positive trajectory towards success in adulthood.

Study considerations and limitations

There were a number of data-related challenges that we needed to address in designing this study. Complexities of this nature are typical of real-world evaluations, which are often hampered by logistical challenges [46]. For example, the diversity in the BGCW programming made it difficult to determine which participants attended which kind of program and for how long. The programs offered by each club are tailored to the needs and characteristics of the communities in which they are located. We dealt with this by choosing to examine all BGCW programs together and by defining “participation in BGCW” as both a binary exposure variable and a continuous exposure variable that accounted for accumulating visits over time. However, this means that our study could not pinpoint specific programs or sites where participants were more engaged or experienced better outcomes than others, which would have been useful for program development. Future studies could potentially include a more in-depth data quality assessment and examine the program data at a more granular level. With respect to data linkage, not all BGCW participants had provided the clubs with their Personal Health Identification Number, a key identifier for linking participation to the Manitoba Population Data Repository. The linkage process therefore required additional time and expertise to complete.

We recognise the study’s potential for selection bias (i.e. individuals who participated in BGCW were different from those who did not) and other unmeasured confounding factors that we could not account for using matching or propensity scores. We noted that there were many individuals in the matched comparison group who had never participated in BGCW, and wondered why they had had no contact with BGCW. Despite the breadth of information in the Manitoba Data Repository, we were also somewhat limited by the specific administrative data available for this study. For example, the data from educational assessments in Manitoba (‘meeting expectations’, ‘approaching expectations’, ‘not meeting expectations’, etc.) are very broad measures of how children are doing in school, and do not give a detailed account of academic performance. There could be many factors other than those we measured that impact upon school performance, including additional information on participants’ physical and mental health, their home environments (e.g. single parenthood households, paternal mental health), influences from friends, and time allocated to activities outside of school (such as part-time jobs or programs offered by organisations other than the BGCW). These factors would have provided additional context and information on participant engagement and the contribution of BGCW programs to the associated outcomes we measured, but were not available in the administrative data. There may be future opportunities to combine administrative data and survey data to gain a more detailed understanding of participant engagement and related outcomes.

Conclusion

Through participation in BGCW, many children and adolescents living in difficult socioeconomic circumstances and facing challenges in their home lives gain access to a safe and supportive environment in which they can experience new opportunities, overcome barriers and build positive relationships. This study of the association between BGCW participation and the health, social and education outcomes of children and adolescents in Winnipeg, Manitoba, Canada, provides evidence for program stakeholders to advocate for program funding, and to policymakers to make evidence-informed decisions about how taxpayer dollars are best spent to improve the health and well-being of Manitobans.