A protocol for coordinating rural community stakeholders to implement whole-of-community youth physical activity surveillance through school systems.

Accurate and effective local data collection systems are needed to inform community change on youth health behaviors such as physical activity (PA). Systematic methods are particularly important for understanding PA behaviors that may be influenced by individual, interpersonal, organizational, and regional factors. The purpose of this study was to describe a protocol for coordinating community stakeholders to implement an online youth PA surveillance instrument. The research team collaborated with local health departments (LHDs) from two rural communities to coordinate schools in implementing school-wide youth PA surveillance. A data sharing agreement was established between all partners. School administrators and teachers attended in-person training sessions for an online PA survey and how to use the data. Following the training, students were provided individualized logins to complete the survey once a semester over a two-year academic period. Across both communities, 23 teachers and administrators attended the training sessions that were facilitated by the LHDs and research team. In Year 1 (Y1), a total of 465 3rd through 6th grade students were enrolled in the participating schools (community 1 = 227; community 2 = 238). Survey response rates ranged from 86.1% to 95.4% completion, depending on the community and semester. In Year 2 (Y2), a total of 501 3rd through 6th grade students were enrolled (community 1 = 260; community 2 = 241). Response rates ranged from 86.3% to 89.6% in the fall term. A protocol for coordinating LHD and community stakeholders was an effective strategy for implementing population-level youth PA surveillance with high levels of reach.

Introduction

Routine monitoring of youth physical activity (PA) behaviors through systematic surveillance efforts is key for identifying age- and gender-related trends, detecting underlying disparities, and informing decisions around resource allocation and policy-making (National Academies of Sciences, Engineering, and Medicine, 2019). Existing national surveillance systems such as the Youth Risk Behavior Surveillance System (YRBSS) (Kann et al., 2018) may aid communities in making local data-driven decisions regarding PA opportunities. However, they are limited in providing timely and community-specific data. To impact community-based PA programming and inform local action, it is important to capture primary sources of data from a localized target population that is bound geographically and temporally within the community of interest (King et al., 2016). Coordinating local surveillance efforts across multiple systems concerned with youth PA and health can be a useful approach for gathering relevant, local data.

Recently, Crooks and colleagues (2017) described a protocol for sustainable, local monitoring of youth obesity in Australia. A key element of the protocol that led to its success was engaging and training community stakeholders (e.g., dieticians, local government officials, school nurses, etc.) in implementing behavioral and anthropometric data collection activities within the school setting (Crooks et al., 2017). Incorporating a feedback, or quality improvement, cycle to improve processes and guide data-driven decisions can further support community health monitoring efforts (Crooks et al., 2017, Lacy et al., 2015, Coleman et al., 2012). School systems routinely conduct annual educational testing to guide programming, and similar efforts have been used to capture youth physical fitness data (The Cooper Institute, 2010). Similarly, a locally led approach for surveilling youth PA coupled with a quality improvement cycle may support efforts for estimating the prevalence of PA and informing local change. Further, engaging with and building capacity of key community stakeholders to carry out PA surveillance efforts can aid in sustainability and foster a sense of ownership (Crooks et al., 2017, Coleman et al., 2012, Dzewaltowski et al., 2002, Dzewaltowski et al., 2009). In the present study, we describe a protocol for engaging community stakeholders in implementation of an online youth PA surveillance instrument within the school system.

Methods

Project design, setting, and participants

The Wellscapes Initiative (www.wellscapes.org) is a two-year community development initiative (baseline infrastructure year, intervention year) targeting promotion of youth PA across rural Nebraska community wellness landscapes of school classrooms, after school program groups, youth club groups, and youth sport teams. This initiative includes a Type 3 – Hybrid Implementation-Effectiveness community randomized controlled trial (ClinicalTrials.gov Identifier: NCT03380143) where two rural Great Plains communities, representative of communities with a concentration of primarily white children (Wave 1) and two communities representative of communities with a concentration of Hispanic/Latino children (Wave 2), were planned for enrollment. Within each wave, communities were randomized to receive either the Wellscapes Model or the standard practice Collective Impact Model (Kania and Kramer, 2011). This paper reports data collected concurrently with Wave 1.

Participating rural communities were large enough to contain a public-school district catchment area with one public high school and were greater than 10 miles from an urbanized area. Community population and distance classifications were drawn from rurality criterion from the Department of Education Rural and Low-Income School Program (RLIS) based on the National Center for Education Statistics (U. S. Department of Education, 2020). Distance was classified as: Town, Distant; Town, Remote; Rural, Distant, and Rural Remote (Geverdt, 2015). Wave 1 rural communities were predominantly non-Hispanic white (based on 2010 American Community Survey 5-Year estimates; Community 1 = 93.3%; Community 2 = 98.2%).

All 3rd through 6th grade youth (ages 8–12 years) from participating schools were eligible to participate in the study. The PA surveillance activities were adopted by schools as a standard educational practice. That is, as part of education activities, children completed an educational learning experience during class time that was linked to a surveillance system platform. Following the United States Family Educational Rights Privacy Act (FERPA), schools shared data from the learning experience with the research team (the data hub) for processing on the PA surveillance system platform as part of the PA promotion initiative. Data collection took place during the Fall 2018, Spring 2019, and Fall 2019 terms. Data collection was planned to take place again during the Spring 2020 term. However, due to school closures related to the novel coronavirus pandemic, these data were not collected. All study activities were approved by the University of Nebraska Medical Center Institutional Review Board (IRB #439-18-EX). Students within a subset of school classrooms were recruited by active parental consent for data collection as part of the randomized controlled trial (IRB #446-18-EP).

The Wellness Landscape (Wellscapes) Whole-of-Community System Intervention Trial

The Wellscapes Initiative posits that communities are dynamic wellness landscapes comprising inter-related, organized group settings that provide opportunities and constraints for youth health behavior (e.g., school classrooms, after-school program groups, youth club groups, youth sport teams) (Dzewaltowski, 2017). Informed by the Healthy Youth Places Framework (Dzewaltowski et al., 2002) and its application in school (Coleman et al., 2012, Chen et al., 2020, McLoughlin et al., 2020, Chen et al., 2018) and out-of-school group interventions (Dev et al., 2020, Dzewaltowski et al., 2010, Hastmann et al., 2013) the Wellscapes whole-of-community randomized trial compared two systems interventions by manipulating the characteristics of four essential elements: 1) a whole-of-community, multi-organization stakeholder group we call a community hub, 2) a community improvement process, 3) an organized, adult-led group evidence-based PA practice intervention, and 4) multi-level data collection and feedback that lends to dynamic, multi-level, data-driven decision making.

During the baseline and intervention years, each community group, comprising key stakeholders (e.g., school personnel, youth sport coaches, club leaders, health department officials), participated in one of two separate, four-time yearly workshops with protocols targeting distinct capacity improvement processes. The standard public health practice protocol for the standard practice community group followed five key conditions of the Collective Impact approach: a common agenda, shared measurement, mutually reinforcing activities, continual communication, and support from a backbone institution (Kania and Kramer, 2011). The Wellscapes community group protocol followed an iterative improvement cycle of Investigate (e.g., “What is our community wellness landscape?”), Design (e.g., “What community opportunities do we want to design for children and families?”), Practice (e.g., “How do we try to practice implementing our design?”), and Reflect (e.g., “Did we develop or improve our community's wellness landscape?”) (IDPR cycle) (Dzewaltowski, 2017). The IDPR cycle is similar in theoretical rationale to the “Plan Do Study Act” (PDSA) cycle that is used in clinical settings for quality improvement, but is designed to address the properties of complex community social systems with dynamic and autonomous organizations (Coleman et al., 2012).

In the present study, both communities participated in the data collection protocol, which was an essential element of the Wellscapes community systems intervention (i.e., data collection and feedback element) (Dzewaltowski, 2017). The Wellscapes Model community completed the protocol as part of the “Investigate” phase of the IDPR cycle, whereas the standard practice community completed the protocol in the context of the Collective Impact Model. Both communities systematically collected data about the prevalence of youth PA behavior and related contextual information and the Wellscapes Model community received enhanced feedback compared to the standard practice community.

Physical activity surveillance system

The Youth Activity Profile (YAP) is a self-report PA instrument that was developed specifically for school-based applications and has been used as a surveillance instrument in the National Cancer Institute’s Family Life, Activity, Sun, Health, and Eating (FLASHE) Study (Saint-Maurice and Welk, 2014, Oh et al., 2017). A unique feature of the YAP is that youth self-report behaviors on a simple survey and items are subsequently calibrated to provide group-level estimates of PA behaviors (Welk et al., 2021). Specific algorithms were recently developed for the online version of the YAP (Welk et al., 2021). Group-level estimates from the YAP were found to be statistically equivalent to values from a validated PA monitor (i.e., a SenseWear Armband Pro3) (Welk et al., 2021, Saint-Maurice et al., 2015). Group-level estimates of in-school and out-of-school PA values from the YAP were within 23% and 21% of values derived from the PA monitor, respectively, based on mean absolute percent error (MAPE) calculations (Welk et al., 2021).

Participants in this study completed an online version of the YAP that allowed schools to easily administer the assessment to large groups of youth in a classroom setting supervised by a teacher. Students were guided to respond to five questions related to in-school participation in PA (e.g., active transportation, activity breaks, physical education, and recess), 5 questions related to out-of-school PA (e.g., activity before and after school, activity on weeknights, and activity on weekends), and 5 questions pertaining to sedentary behaviors (e.g., watching television, videogame use, computer use) based on the last seven days. The online platform employed updated calibration equations to provide estimates of both time spent in moderate-to-vigorous physical activity (MVPA) as well as time spent in sedentary behavior (Welk et al., 2021). Please visit www.youthactivityprofile.org to complete a full demonstration of the YAP.

To provide a more robust evaluation of PA behaviors, students also completed additional questions within the online system to capture demographic variables (i.e., gender, grade) and activities in which they participated during the last month and 12 months (i.e., after school program, sports lessons, clubs or organizations, organized activities or lessons). Activity participation questions were derived from the National Survey of Children’s Health (NSCH) (Census Bureau and Survey, 2017).

Youth physical activity surveillance implementation protocol

Principals at the elementary (Communities 1 and 2) and high school (Community 1) were invited by members of the community’s local health department and the research team to participate in the online youth PA surveillance system protocol and the community randomized trial. Community health departments were granted a service agreement of $34,000 in baseline and intervention years to provide small incentives and support a part-time public health position to coordinate the Wellscapes Initiative. Both communities offered financial incentives to schools for participating in this study; however, teachers and staff within the schools were not individually incentivized.

Under the guidance of the research team, the local health department worked directly with participating schools to define a community data system. The research team imitated the role of an educational system service unit and served as the data hub for data processing and report generation for school and community improvement. A Data Sharing Agreement (DSA) was established between the school district and data hub. DSAs are legal documents that are commonly used to share electronic education or health information between various entities (e.g., school systems, health systems, hospitals, long-term care facilities, etc.) to manage and conduct research on population health outcomes (Allen et al., 2014). The DSA, a corresponding Principals of Collaboration Agreement (PCA), and IRB protocols identified the type of student information that could be shared and detailed the terms and conditions of data sharing. The DSAs and PCAs were thoroughly reviewed by the university’s IRB and legal teams. Shared data included student name, identification number, date of birth, grade, race/ethnicity, and free/reduced lunch status. Under the conditions of the DSAs, the data hub housed within the research team received shared data from the schools on a secure platform, created de-identified data sets, and provided aggregate reports of YAP data to schools and health departments.

After the DSA was established and a school data coordinator was identified, the local health department and research team collaborated to deliver a Community Measures Training workshop. The purpose of this workshop was twofold: to provide training and resources to school personnel (i.e., data coordinator, school administrators, teachers) on how to use community health reports, and to train teachers on how to implement the YAP surveillance system. The in-person workshops were delivered once per academic year in each community and were approximately one hour long. The research team facilitated the workshop during Year 1 (Y1), and the local health departments facilitated the workshop during Year 2 (Y2).

Facilitators of the Community Measures Training (Y1 = research team; Y2 = local health department) delivered a brief presentation about how to use community reports, specifically PA data, to inform decision-making. Additionally, all 3rd through 6th grade teachers and school administrators received a printed workbook that provided step-by-step instructions for implementing the YAP. Prior to data collection, the research team created a profile for school data coordinators on the YAP online platform and granted them permissions to upload student profiles. The school data coordinators then created individual profiles and unique account information (i.e., usernames and passwords) for all 3rd through 6th grade students and distributed this information to classroom teachers. School administrators and teachers chose when to implement the YAP survey during the first week of school in the fall terms, and the first week of May in the spring term. Teachers administered the YAP in the classroom setting using media carts or a school media center, and students were instructed to use their individualized login information to access the YAP survey on the online platform. After entering login information, students were able to view and complete the YAP survey and received an automatically generated report of their individual PA and sedentary behavior. Schools received an automatically generated summary report of students’ PA and sedentary behavior. Teachers provided technical support to their students, and the research team was available to provide remote technical support to the teachers and school data coordinators as needed. YAP data were also included as part of more comprehensive community reports distributed throughout the project.

Analysis

Descriptive statistics, including frequencies and percentages, were calculated using Excel to summarize enrollment information and to describe YAP response rates in each community. The research team’s project coordinator recorded and summarized instances of technical support provided by the research team to the schools.

Results

Participating schools adopted the YAP surveillance system as a standard educational practice for all 3rd through 6th grade students. As such, all eligible students were enrolled in the YAP online platform but may not have completed the survey if they chose to opt out, or if they were absent on the day of data collection. Across both communities, 23 teachers and administrators attended the Community Measures Training workshops that were facilitated by the local health department and research team. The DSA allowed the research team to receive school enrollment information (see Table 1) and calculate YAP response rates at each data collection time point. Across both communities, a total of 465 and 501 3rd through 6th grade students were eligible to complete the YAP in Y1 and Y2, respectively. In Y1, response rates ranged from 86.1% to 95.4% (see Table 2). In Y2, response rates ranged from 86.3% to 89.6% during the fall term; data were not collected for the spring term.

Table 1

School enrollment information acquired through the data sharing agreement.

	Community 1		Community 2
	Year 1	Year 2	Year 1	Year 2
All Participants, n	227	260	238	241
Gender, n (%)
Male	114 (50.2)	126 (48.5)	123 (51.7)	113 (46.9)
Female	113 (49.8)	134 (51.5)	115 (48.3)	128 (53.1)
Grade, n (%)
3	50 (22.0)	70 (26.9)	70 (29.4)	53 (22.0)
4	64 (28.2)	56 (21.5)	50 (21.0)	74 (30.7)
5	61 (26.9)	67 (25.8)	66 (27.7)	48 (19.9)
6	52 (22.9)	67 (25.8)	52 (21.8)	66 (27.4)
Race/Ethnicity, n (%)
Non-Hispanic White	208 (92.0)	236 (91.1)	226 (95.0)	231 (95.9)
Other	18 (7.9)	23 (8.9)	12 (5.0)	10 (4.1)
Lunch Status, n (%)
Full Pay	177 (78.0)	186 (71.5)	146 (61.3)	154 (63.9)
Free/Reduced	50 (22.0)	74 (28.5)	92 (38.7)	87 (36.1)

Table 2

Youth Activity Profile (YAP) response rates by community and term.

	Community 1			Community 2
	Year 1		Year 2	Year 1		Year 2
	Fall 2018	Spring 2019	Fall 2019	Fall 2018	Spring 2019	Fall 2019
Enrolled participants (n)	227	227	260	238	238	241
Completed YAP (n)	214	213	233	227	205	208
Response rate (%)	94.3	93.8	89.6	95.4	86.1	86.3

Minimal technical assistance was provided to school coordinators by the research team. During Y1, there were no difficulties with implementing the YAP. In the fall of Y2, there was a miscommunication between school personnel regarding who would create student profiles in the online system. There were also minor issues when uploading student profiles due to use of duplicate email addresses. All technical assistance by the research team was provided through email or over the phone.

Discussion

The protocol described in the present study built the capacity of local stakeholders (i.e., county health department personnel, school administrators, and teachers) to implement school-wide PA surveillance over two years. All children in the participating school systems participated in PA surveillance as a standard educational practice. Coordinating efforts among stakeholders was an effective approach to implementing youth PA surveillance in the two participating rural communities and resulted in high response rates on the YAP with minimal technical assistance.

In a qualitative study of rural health system stakeholders in the Northwest United States, Bekemeier and colleagues (2019) identified key challenges to collecting and monitoring population health behaviors. Rural health department personnel expressed significant challenges with accessing data relevant to their communities and indicated that datasets were often outdated or collected without a systematic protocol. Further, participants expressed little motivation for establishing community monitoring systems due to lack of training and expertise in data collection and analysis procedures (Bekemeier et al., 2019). Though national PA surveillance methodologies are well-established and remain a significant public health priority (National Academies of Sciences, Engineering, and Medicine, 2019) it is clear that systematically monitoring local health behaviors to inform community-level change is also needed (Bekemeier et al., 2019, Institute of Medicine, 2011).

Building the capacity of community stakeholders to establish local data systems can equip communities with data to inform community change related to youth PA behaviors. In the present study, the use of a DSA engaged community stakeholders from two organizations, school districts and health departments, in establishing a data system for monitoring youth PA over time. This approach allowed for stakeholders to have access to timely data that was relevant and useful to their community, an important need in rural communities (Bekemeier et al., 2019). The addition of the in-person Community Measures Training workshops informed stakeholders about the importance of systematically collecting youth PA data and provided them with sufficient resources to be able to implement a school-wide, online PA surveillance system with little technical assistance from the research team.

The surveillance protocol used in the present study led to considerably higher survey response rates than what is typically observed. However, it should be noted that traditional surveillance methods and those of the present study have different goals. Traditional surveillance activities rely on representative sampling to estimate population-level patterns in health behaviors. The present study utilized a protocol to assess all youth in the pre-determined system in order to estimate population PA and provide the local data as feedback to the communities. Existing surveillance systems, such as the widely used YRBSS, provide important estimates of youth PA behaviors at a national level and typical response rates on the YRBSS are reported to range from 60 to 71% (Brener and Kann, 2013, Kann et al., 2018). The National Health and Nutrition Examination Survey (NHANES) is also commonly used in youth PA surveillance, and response rates from two recent data cycles (i.e., 2015–2016, 2017–2018) for youth ranged from 56.3 to 68.0% (CDC, n.d.). These response rates are sufficient for estimating population health behaviors using representative sampling. Variations in response rates between national surveillance methods and those of the present study may be due, in part, to the ways in which the surveillance tools were promoted within the school systems. Efforts to support national or state level surveillance may not lead to the same level of school engagement since schools do not directly benefit from the local data. Our response rates, however, were similar to those reported by Crooks and colleagues (2017) which suggests that the involvement of local leaders, and the value of the data for both students and the individual schools likely enhanced the engagement with the YAP in the present study.

Variation in response rates can be attributed to methodological factors, such as consenting procedures, that pose challenges for obtaining accurate estimates of population-level youth PA behaviors. For example, traditional surveillance methodologies that require a parent or guardian to consent prior to reporting youth health behaviors (i.e., opt-in) result in lower participation rates, ranging from 30 to 60% (Tigges, 2003, Strugnell et al., 2018). A passive (i.e., opt-out) approach in which participation hinges on a parent or caregiver refusing to participate yields considerably higher response rates (86–100%) (Crooks et al., 2017, Tigges, 2003, Strugnell et al., 2018). Strugnell and colleagues investigated the differences in participation bias between two consent protocols used in a study exploring population-level prevalence of youth overweight and obesity (Strugnell et al., 2018). Surveillance activities took place in a school setting and an active, opt-in consent protocol was implemented the first year of the study. In the second year, a passive, opt-out consent protocol was used, and all other study methods remained the same. Youth participation rates were significantly higher in the opt-out protocol compared with the opt-in protocol (84.4% vs. 36.3%; p < 0.001). Further, there was significant non-participation bias, particularly among girls, which resulted in underestimated prevalence of overweight and obesity among this population (Strugnell et al., 2018). The passive consent protocol used in the present study was a significant strength, as it resulted in high response rates among study participants, which provides an externally valid population indicator of youth PA behaviors in their respective communities.

Importantly, the communities in which the present study took place were primarily non-Hispanic white, rural communities, which does not provide information about broader application of the protocol to more diverse and urban communities. Next, to minimize burden, we did not request that teachers report the reasons why youth missed completing the YAP survey, so we are unable to determine whether a student opted out of completing the survey due to absence from school, or due to moving to a different school district. Lastly, we did not examine demographic differences among youth who completed the YAP and those who opted out due to the parameters established in the DSA. That is, demographic data provided by the school was not linked to YAP data without an additional level of parental consent. Despite the limitations, the protocol established in this study, coupled with the online YAP platform, yielded more precise estimates of population youth PA behavior and allowed for communities to have timely access to relevant, local data.

Conclusions

While national-level youth PA surveillance data are important drivers for policy change and resource allocation, they have limited utility or value for more localized applications. The YAP is a valid assessment tool that can be effectively used in school and community settings to capture representative data on youth PA behaviors. The protocol described in the present study was shown to be a useful approach for implementing school-wide youth PA surveillance in rural communities with high response rates. Further, the community data system and use of a DSA and data hub ensured that community stakeholders with a shared vision of improving youth PA and health could access timely, and relevant population-level reports. Equipping communities with the necessary infrastructure for collecting and monitoring youth health behaviors is a vital step toward data-driven decision making and coupled with collaborative efforts for community change, can have a significant impact on youth population health.

Funding

Research reported in this publication was supported by the National Cancer Institute of the National Institutes of Health under Award Number R01CA215420. Study sponsors were not involved with data collection, analysis, interpretation, or writing of the manuscript. The content is solely the responsibility of the authors and does not necessarily represent the official views of the National Institutes of Health.

CRediT authorship contribution statement

Michaela A. Schenkelberg: Conceptualization, Methodology, Formal analysis, Investigation, Resources, Writing - original draft, Writing - review & editing, Visualization, Supervision, Project administration. Ann M. Essay: Conceptualization, Methodology, Formal analysis, Investigation, Data curation, Resources, Visualization, Writing - original draft, Writing - review & editing, Project administration. Marisa S. Rosen: Conceptualization, Methodology, Investigation, Data curation, Resources, Writing - review & editing, Project administration. Arissa E. Bavari: Formal analysis, Investigation, Writing - original draft, Writing - review & editing. Sara J. Norgelas: Conceptualization, Methodology, Formal analysis, Investigation, Data curation, Resources, Writing - review & editing, Project administration. Richard R. Rosenkranz: Conceptualization, Methodology, Resources, Writing - review & editing, Supervision, Funding acquisition. Gregory J. Welk: Conceptualization, Methodology, Software, Data curation, Resources, Writing - review & editing, Supervision, Funding acquisition. David A. Dzewaltowski: Conceptualization, Methodology, Investigation, Resources, Writing - review & editing, Supervision, Funding acquisition.

Declaration of Competing Interest

The authors declare that they have no known competing financial interests or personal relationships that could have appeared to influence the work reported in this paper.