Developing a core outcome set for physical activity interventions in primary schools: a modified-Delphi study.

OBJECTIVES: To develop a core outcome set (COS) for physical activity interventions in primary schools.
DESIGN: Modified-Delphi study.
SETTING: The UK and international. PARTICIPANTS: 104 participants from four stakeholder groups (educators, public health professionals, health researchers, parents); 16 children (aged 8-9 years) from 1 London primary school.
INTERVENTIONS: Physical activity interventions.
METHODS: Four-stage process: (1) outcomes extracted from relevant studies identified from an umbrella review and a focus group; (2) list of outcomes produced and domains established; (3) stakeholders completed a two-round Delphi survey by rating (Round 1) and re-rating (Round 2) each outcome on a nine-point Likert Scale from 'not important' to 'critical': a>70% participant threshold identified the outcomes rated 'critical' to measure, and outcomes important to children were identified through a workshop; and (4) a stakeholder meeting to achieve consensus of the outcomes to include in the COS.
RESULTS: In total, 74 studies were extracted from 53 reviews. A list of 50 outcomes was produced and three domains were established: 'physical activity and health' (16 outcomes), 'social and emotional health' (22 outcomes) and 'educational performance' (12 outcomes). 104 participants completed survey Round 1; 65 participants completed both rounds. In total, 13 outcomes met the threshold; children identified 8 outcomes. Fourteen outcomes achieved consensus to produce the COS: five outcomes for physical activity and health (diet (varied and balanced), energy, fitness, intensity of physical activity, sleep (number of hours)); seven outcomes for social and emotional health (anxiety, depression, enjoyment, happiness, self-esteem, stress, well-being); and two outcomes for educational performance (concentration, focus).
CONCLUSIONS: We have developed the first COS for physical activity interventions in primary schools in consultation with those interested in the development and application of an agreed standardised set of outcomes. Future studies including these outcomes will reduce heterogeneity across studies. TRIAL REGISTRATION NUMBER: Core Outcome Measures in Effectiveness Trials Initiative registration number 1322; Results. © Author(s) (or their employer(s)) 2022. Re-use permitted under CC BY-NC. No commercial re-use. See rights and permissions. Published by BMJ.

This is the first study to develop a core outcome set (COS) for physical activity interventions in primary schools.

The COS has been developed in consultation with participants from key stakeholder groups.

This study uses robust methodology as recommended by the Core Outcome Measures in Effectiveness in Trials Initiative.

There were an unbalanced number of participants in each stakeholder group.

The low representation of international participants may limit the use of this COS to UK schools only.

Introduction

Increasing children’s physical activity is a global health goal given the vast evidence showing benefits on physical, social, mental and cognitive health outcomes.1 Health behaviours may become embedded in childhood; providing opportunities for children to engage in physical activities during the primary school years may lead to physically active lifestyles and improved health during adolescence and adulthood.2 Many governments support the need for increased physical activity promotion in schools.3 The WHO recommends that schools should organise and promote opportunities for children to regularly participate in physical activities.4

School settings are ideal as they have the potential to reach the majority of children across society5 6 including those living in poverty. Socioeconomic inequalities have been associated with moderate and vigorous physical activities and may contribute to widening health inequalities.7 Targeting schools therefore could help towards reducing the gap in physical activity among children.7 8 As a result of governments and the WHO recommendations of physical activity promotion and engagement in schools, there are many physical activity interventions that are implemented. However, the interventions vary in design. Some interventions integrate additional physical education classes alongside compulsory physical education lessons,9 while some may incorporate 10 min of physical activity into every school day.10 There are also others which implement classroom movement breaks11 or active mile interventions.12 13

There is considerable evidence showing the benefits of physical activity interventions in schools successfully increasing children’s fitness14–17 and reducing sedentary time18 19 There is also increasing evidence of improvements to children’s social, emotional and cognitive outcomes.20–23 However, due to the heterogeneity of the outcomes assessed across studies, definitive conclusions are challenging.20 22 For example, to assess children’s emotional health, one study may measure children’s ‘happiness’, while another may measure ‘depression’. Both these outcomes are conceptually different and difficult to compare. In 2013, a Cochrane review of 44 randomised control trials of physical activity interventions in schools for children aged 6–18 years found considerable variations in the outcomes measured, and the results could not be synthesised to establish intervention effects.24 The review was updated in 2021; the authors concluded that due to the variability of results, heterogeneity and risk of bias across studies, the impacts of physical activity interventions in schools have shown small effects. These interventions may show small improvements to children’s physical fitness but have little or no impact on other outcomes such as body mass index (BMI).25

Synthesising results from studies are likely to be of interest to a number of key groups including public health professionals, teachers, parents, healthcare researchers and policymakers. However, many of the outcomes measured in existing studies, although important to measure, may vary in relevance to specific groups. For example, BMI is a frequently measured outcome from which important conclusions have been identified.26 27 BMI may be considered highly important to healthcare practitioners but may not be considered as important to teachers who may instead place higher importance on cognitive outcomes. Lack of consultation with key groups when deciding which outcomes to measure in studies limits the relevance of findings to specific groups and may has possibly led to differences of outcomes measured across studies, thus preventing comparisons.

A core outcome set (COS) is an agreed set of standardised outcomes in a specific research area that is recommended to measure and report.28 These sets should be developed in consultation with those who are interested in the development and application of an agreed set of outcomes.29 The COS should be viewed as a minimum to measure and does not restrict additional outcomes of interest to be assessed. COSs were originally developed for clinical trials but are increasingly being used in other study designs, for example, in observational studies by practitioners and researchers to conduct their own assessments of interventions.28 To our knowledge, there is not a COS for physical activity interventions in primary schools. Therefore, the development of a COS (the aim of this study) would contribute to this field of research by identifying the key outcomes to be studied, allowing for evidence synthesis to better understand the impact of physical activity interventions in schools on children’s health.

Methods

Design

The protocol for this work has been published in online supplemental file 130; it was developed in accordance with the Core Outcome Measures in Effectiveness Trials (COMET) criteria29 and prospectively registered accordingly. 31 We used a modified-Delphi method consisting of four stages to develop the COS (figure 1). First, we extracted outcomes and how they had been defined/described by the authors of relevant studies identified through an umbrella review and through a focus group with our steering committee (our steering committee includes health professionals, health researchers, academics and sports representatives from organisations such as Sport England and The Daily Mile Foundation). Second, after deduplication and combining similar outcomes, we created a long list and established domains determined by the outcomes. Third, we recruited participants from four key stakeholder groups (educators, health researchers, public health professionals and parents of children aged from 5 to 11 years) to complete a two-round Delphi survey. We also obtained children’s views of what is important to them through a workshop. Fourth, we held a stakeholder meeting to achieve consensus on the outcomes to be included in the COS. We report the study following the COS–STAndards for Reporting checklist (online supplemental file 2).32

Figure 1

Process for developing a core outcome set for physical activity interventions in primary schools.

Stage 1: extraction of outcomes

For the umbrella review, we searched six databases (MEDLINE, EMBASE, PsycINFO, CINAHL, CENTRAL and the Cochrane Database of Systematic Reviews). Keywords used for the search were ‘school’, ‘physical activity’, ‘exercise’, ‘physical education’, ‘fitness’ and ‘energy expenditure’ and adapted to use database specific filters, that is, subject headings or medical subject headings. Reviews were limited to systematic reviews, meta-analyses or meta-syntheses and those published between 1990 and 2019. Relevant studies from these reviews were identified from which the outcomes extracted. We also held a focus group with our steering committee and used a nominal group technique to brainstorm outcomes and rate their importance to extract further outcomes that may not have been captured in our literature review. Descriptions of each outcome were guided by the published literature and discussions with our steering group.

Stage 2: list of outcomes and establishing domains

We removed duplicate outcomes and merged those that were closely related, for example, outcomes of ‘light physical activity’, ‘moderate physical activity’ and ‘vigorous physical activity’ were combined into ‘intensity of physical activity’, to create a long list of outcomes. Descriptions were generated for each outcome based on those provided by authors of the relevant studies and discussions with our steering committee. Guided by the outcomes and descriptions, we established relevant domains by grouping similar outcomes that captured a broader concept.

Stage 3: stakeholder recruitment, Delphi surveys and children’s workshop

The purpose of the Delphi surveys was to identify which outcomes, from the long list we produced, were considered the most important to measure across key stakeholder groups.

Stakeholder recruitment

Through emails to our public health research and practitioner networks and through snowballing and social media, we recruited participants from four key stakeholder groups (educators (teachers, head teachers, school governors), health researchers, public health professionals and parents of primary school-aged children). Through discussions with our steering group, we identifed the key stakeholder groups that would be the most interested in the development and implementation of an agreed set of outcomes to enhance this field of research. An information leaflet was made available to participants which included an electronic link to the Round 1 Delphi survey and study contact details. Through the Round 1 survey link, we obtained consent for participation, followed by participants registering their details (name and email address) and indicating which of the four stakeholder groups they identified with.

Delphi surveys

Using DelphiManager software,33 we listed the outcomes with their descriptions by each domain in a Delphi survey conducted over two rounds (Round 1 took place during June 2020 and Round 2 in August 2020). Using the predefined Delphi survey guidelines,33 we asked participants to rate the importance of each outcome using a nine-point Likert Scale ranging from ‘not important to measure’ to ‘critical to measure’ in Round 1. A rating of 10 could be indicated if participants felt they were unable to score an outcome. Ratings were grouped into three categories: ‘not important to measure’ (ratings of 1, 2 or 3); ‘important but not critical to measure’ (ratings of 4, 5 or 6); and ‘critical to measure’ (ratings of 7, 8 or 9). In addition, participants were asked to suggest any other outcomes that they felt were not captured. In line with our protocol, if more than two individual participants suggested the same additional outcome, this would be included in Round 2 for all participants to rate. For ratings in Round 2, participants were provided with feedback of Round 1 ratings categorised by stakeholder group and an option to rerate their initial ratings based on this feedback. Participants were sent three email reminders to complete Round 1; those who rated all outcomes in Round 1 were invited to complete Round 2. The criteria for outcomes considered most important to measure for each domain after Round 2 were defined a priori, >70% of all participants rating an outcome ‘critical’ and 15% or less rating it ‘not important’.30 None of the outcomes were removed between rounds.

Children’s workshop

We recruited primary school children to take part in a workshop in December 2020 with consent obtained from parents via the school. Due to COVID-19, our access to schools was restricted. We partnered with one primary school in Greater London, UK. Guided by the list of outcomes, we engaged the children in a series of activities and discussions on physical activity and elicited the children’s views on what they thought was important to measure.

Stage 4: stakeholder meeting

Participants who completed both survey rounds were invited to attend the stakeholder meeting in December 2020. Due to COVID-19 restrictions, the meeting was held virtually using the Zoom platform and we adapted the voting method (70%/15% threshold) as described in our protocol. Instead, to achieve consensus on the outcomes to be included in the COS, we led discussions around the ratings of outcomes in the Delphi surveys and children’s views. We used the Zoom chat function for participants to indicate the most important outcomes and further discussion to agree the outcomes to be included in the COS.

Patient and public involvement

We have consulted with professional and public representatives within our steering committee and as part of The Daily Mile Research Advisory Group. Both groups include public health professionals, health researchers, academic researchers and representatives from The Daily Mile Foundation, Sport England, London Marathon and London Sport. Our COS has been developed in consultation with educators, health researchers, public health professionals, parents and children through focus groups and workshops. We will widely advertise our COS through those involved in the development and also to child public health policymakers through our research networks.

Results

Our umbrella review identified 53 relevant papers from which 74 individual studies were extracted (online supplemental file 3); around 181 outcomes were identified from these studies. However, we identified variations across studies of how the outcomes were defined or described if at all. The steering committee focus group identified 34 outcomes. We created the description for each outcome guided by the literature and from discussions with our Steering Group.

The final list consisted of 50 outcomes (table 1) representing three domains: (1) physical activity and health (16 outcomes); (2) social and emotional health (22 outcomes); and (3) educational performance (12 outcomes). Two outcomes, ‘sleep’ and ‘diet’, were included in two domains as authors agreed that these outcomes in particular could be both a ‘physical activity and health’ and a ‘social and emotional health’ outcome. For example, sleep defined as number of hours slept as recommended for children was included in the physical activity and health domain, while sleep times/patterns/broken sleep was included in the social and emotional health domain. Similarly for the outcome of diet, eating well-balanced meals was included in the physical activity and health domain, while appetite was included in the social and emotional health domain (see table 1 for descriptions).

Table 1

List of 50 outcomes and their descriptions by domain and the number of studies from which the outcomes were extracted

Domain	Outcomes measured	Description*	Studies†
1: Physical activity and health	Active travel	To get to and from school, for example, walking, public transport, that is, train/tube/bus (do not include car, van, motorcycle), cycling and scooter	FG‡
	Anthropometry§	Weight, height, body mass index body fat, body mass and waist circumference	34
	Blood lipids	Fatty substances found in the blood (ie, cholesterol, triglycerides) which increase the risk of heart attack	2
	Blood pressure	The force at which your heart pumps blood around your body and the resistance to the blood flow in the blood vessels	2
	Diet	Varied and balanced diet including fruit and vegetables	FG‡
	Energy levels/expenditure	The amount of energy needed to carry out physical functions such as breathing, exercising or digesting food	5
	Fitness	Being fit and healthy for optimal health and overall well-being	16
	Heart rate	Number of beats per minute to establish normal resting heart rate, high or low heart rate	5
	Intensity of physical activity	Includes light activity (ie, taking a stroll); moderate activity (ie, cycling/swimming at regular pace, sweeping, washing windows); and vigorous activity (ie, aerobics, running, fast cycling, climbing stairs)	42
	Leisure time activity	Time spent in activity for leisure during the day (ie, walking in the park, playing sports with friends/family)	FG‡
	Motor skills	Skills that require using large muscles of the arms/legs/torso, that is, standing, walking, going up and down stairs, running, swimming, jumping, skipping, leaping and kicking	8
	Musculoskeletal	Bone strength, bone mineral density and muscle¶	8
	Peak oxygen intake	The maximal rate at which oxygen can be used by the body during maximal work	1
	Sedentary time	Time spent sitting at desk, reading, sitting or lying down to watch television	7
	Sleep	Between approximately 10–12 hours per night	FG‡
	Step counts	Number of steps taken in a day	13
2: Social and emotional health	Anxiety	Persistent feeling of worry, fear or nervousness	FG‡
	Appetite	Eating well and regularly	FG*
	Body awareness	The ability to recognise one’s body moves helping to understand how to relate to objects and people at home, at school and outdoors	1
	Body image	The perception one has of their physical self	1
	Depression	Feeling persistently sad for more than a few days	FG‡
	Empowerment	Feeling a sense of becoming stronger and more confident	FG‡
	Enjoyment	Taking pleasure in doing something	3
	Happiness	Feeling a sense of joy and contentment	FG‡
	Mood	A state of mind or a feeling such as happy, sad, cheerful or angry	FG‡
	Peer support	Using one’s own experiences to help others	1
	Resilience	The ability to recover quickly from difficulties	FG‡
	Satisfaction	A sense of fulfilling a need, desire or appetite	FG‡
	Self-confidence	A feeling of trust in one’s abilities, qualities and judgement	FG‡
	Self-efficacy	A person’s belief of their capacity to perform behaviours necessary to produce specific performance attainments	2
	Self-esteem	A factor that influences people’s choices and decisions which results in them either taking or not taking care of themselves and explore their full potential	1
	Self-expression	The communication of one’s personality, feelings or opinions	FG‡
	Self-perception	Attitudes towards own preferences and behaviour	1
	Sickness	Feeling unwell, nauseous and dizzy	FG‡
	Sleep patterns	Sleep patterns/achieving less than recommended (10–12 hours)/broken sleep	FG‡
	Social interaction	An exchange between two or more people	FG‡
	Stress	Feeling under pressure or threatened	FG‡
	Well-being	Feeling well, happy, healthy and ability to manage stress	FG‡
3: Educational performance	Academic performance	Measurement of a child’s achievement over a range of academic subjects	20
	Attention	Taking notice of someone or something	6
	Classroom behaviour	How children are acting in the classroom in response to what is going on or present around them	15
	Cognitive development/ function	How children think, explore and figure things out	6
	Concentration	Ability to focus on task	16
	Engagement	The degree of attention, curiosity, interest, optimism and passion that children show when they are learning or being taught	5
	Executive functioning	A set of mental skills including working memory, flexible thinking and self-control to apply to everyday learning, work, and daily life	4
	Focus	Ability to concentrate and not easily distracted	16
	Maths	The study of numbers, shapes and patterns	16
	Reading	A cognitive process that involves decoding symbols to arrive at meaning, the primary purpose of which is to understand the text	8
	Working memory/inhibition	A cognitive system with a limited capacity that can hold information temporarily and is important for reasoning, decision-making and behaviour	6
	Writing	A form of communication to express language using symbols; being able to understand grammar, punctuation, spelling and vocabulary	5

*Descriptions were guided by the published literature and our steering group.

†From the 74 studies identified from the 53 relevant reviews.

‡FG=outcome identified by our focus group (steering group).

§Anthropometry was presented as ‘bioimpedance’ to participants, changed to ‘anthropometry' based on reviewer suggestions.

¶Muscle was not included in the original description presented to participants. This was added based on reviewer suggestions.

A total of 104 participants consented and registered their details. Ninety (87%) completed Round 1 in full of whom 65 (72%) also completed Round 2 in full. The 65 participants included 16 (25%) educators, 24 (37%) researchers, 13 (20%) public health professionals and 12 (18%) parents and represented 9 countries: the UK (80%), Brazil (6%) and Korea (5%), Australia, France, the Netherlands, Romania, Spain and Taiwan (2%).

In total, 13 outcomes met the >70% participant critical threshold: sleep (number of hours) and diet (varied and balanced) in ‘physical activity and health’; happiness, well-being, anxiety, self-esteem, depression, self-confidence, enjoyment and stress in ‘social and emotional health’; and concentration, attention and focus in ‘educational performance’ (table 2). In Round 1, a further 29 outcomes were suggested, but after internal discussions, it was agreed that 16 of the suggestions overlapped with the outcomes that were listed in the survey, and the remaining 13 were proposed by only one participant and therefore not carried forward to Round 2. Mean Round 1 ratings of participants completing both Rounds were similar to those who completed Round 1 but did not complete Round 2 (6.33, SD 2.08 vs 6.48, SD 1.95, respectively) suggesting those who did not complete Round 2 would have scored similarly to those who did.

Table 2

Outcomes rated ‘not important’ and ‘critical’ to measure after Delphi survey Round 2 (n=60)

Domain	Outcome	Participants rating outcomes 'not important' %	Participants rating outcomes ‘critical’ %
1. Physical activity and health	Active travel	3	51
	Anthropometry*	15	26
	Blood lipids	32	14
	Blood pressure	28	14
	Diet (varied and balanced)†	3	71†
	Energy	8	26
	Fitness	0	60
	Heart rate	20	17
	Intensity of physical activity	3	63
	Leisure time activity	3	62
	Motor skills	8	46
	Musculoskeletal	12	20
	Oxygen peak intake	29	9
	Sedentary time	3	63
	Sleep (number of hours)†	3	85†
	Step counts	12	23
2. Social and emotional health	Anxiety†	0	78†
	Appetite	8	42
	Body awareness	2	46
	Body image	2	66
	Depression	3	74
	Empowerment	2	42
	Enjoyment†	0	74†
	Happiness†	0	85†
	Mood	0	51
	Peer support	0	46
	Resilience	3	55
	Satisfaction	2	46
	Self-confidence†	0	74†
	Self-efficacy	2	68
	Self-esteem†	0	75†
	Self-expression	8	34
	Self-perception	2	51
	Sickness	12	40
	Sleep patterns	3	69
	Social interaction	0	65
	Stress	0	72†
	Well-bein†	0	85†
3. Educational performance	Academic performance	2	57
	Attention†	0†	74†
	Classroom behaviour	2	68
	Cognition	2	54
	Concentration†	0	75†
	Engagement	0	69
	Executive functioning	2	46
	Focus†	3	72†
	Maths	8	55
	Memory	2	48
	Reading	8	51
	Writing	8	48

*Anthropometry was presented as 'bioimpedance' to the participants. This was changed based on reviewer comments.

†Ratings that met the threshold (<15% agreement of the outcome rated ‘not important’ and >70% agreement of the outcome rated ‘critical’ to measure.

In total, 16 children aged 8–9 years took part in the workshop, of which 50% were girls; 13% were Caucasian, 56% were Asian and 31% were black; 6% had special educational needs; and 75% had English as a second language. The children identified eight outcomes important to measure: five in ‘physical activity and health’ (energy, fitness, heart rate, muscle strength and weight) and three in ‘social and emotional health’ (happiness, mood and stress). Interestingly, children did not associate physical activity with any educational performance related outcomes.

In total, 13 participants attended (2 educators, 2 parents and 9 researchers). Participants expressed that they had expected more outcomes under the domain of physical activity and health to be rated critical, that is, intensity of physical activity which had been rated critical by 63% (table 2). Through discussion, agreement was reached that this outcome is important to measure to be able to assess sustainability of physical activity interventions in schools. After review of the outcomes identified critical in the survey and the outcomes considered important to children, six outcomes were dropped and the additional outcome of intensity of physical activity was included (online supplemental file 4). Therefore, a total of 14 outcomes reached consensus for the COS: diet (varied and balanced), fitness, intensity of physical activity and sleep (number of hours) in the physical activity and health domain; anxiety, depression, enjoyment, happiness, self-esteem, stress and well-being in social and emotional health domain; and concentration and focus in the domain of educational performance (table 3). We sent the agreed set of outcomes for review to the stakeholders unable to attend the meeting. The wider group approved the COS.

Table 3

Core outcome set for physical activity interventions in primary schools

Domain	Outcome
Physical activity and health	Diet (varied and balanced)
	Energy
	Fitness
	Intensity of physical activity
	Sleep (number of hours)
Social and emotional health	Anxiety
	Depression
	Enjoyment
	Happiness
	Self-esteem
	Stress
	Well-being
Educational performance	Concentration
	Focus

Discussion

We have developed the first COS for physical activity interventions in primary schools. By using robust consensus methods and multidisciplinary stakeholder groups, we have achieved consensus on the outcomes considered important to measure. Implementation of this COS in future studies will reduce heterogeneity between studies allowing for evidence synthesis and will also be relevant to wider audiences.

During the consensus meeting, it was noted that the survey identified only two outcomes (sleep and diet) in the domain of physical activity and health as critical to measure, while the outcomes ‘physical activity intensity’ and ‘fitness’ did not meet the threshold. Outcomes that may fit under this domain include moderate physical activity, vigorous physical activity, moderate-to-vigorous physical activity and heart rate, which are more commonly studied but these did not meet the critical threshold in our survey. This potentially reflects the heterogeneity across studies of the outcomes that should be measured under broader concepts. As discussed in our consensus meeting, the under-representation of outcomes rated critically important in the physical activity domain may have been due to the specificity of outcomes listed. For example, researchers agree that physical activity should be measured but do not agree on which specific outcome to measure it. This would explain the wide variation of physical activity outcomes that were identified from the published literature. Physical activity can have many benefits beyond measuring its impact on particular health or clinical outcomes. Therefore, our participants agreed that measuring physical activity is important and should be included.

In the published literature, we found only 10 studies which measured outcomes that related to mental health, yet all our stakeholders placed critical importance on many of the outcomes under the domain of social and emotional health. These findings may be explained by the growing awareness of poor mental health in children and the growing evidence base of associations between increased physical and better mental health. The importance placed on mental health perhaps indicates a shift in focus from measuring physiological outcomes and towards measuring mental health when assessing physical activity interventions in primary schools. This may allow health professionals/researchers/teachers/parents to be able tackle better mental health in childhood which may lead to better mental health in adolescence and adulthood. These findings further support the need for a COS in this field. Our study has provided a better understanding that to achieve better overall health and well-being in children, both physical and mental health are important to measure.

Functional precursors of performance-related outcomes (concentration, attention and focus) met the critical threshold than actual educational attainment outcomes of reading, writing and maths which are more commonly assessed in previous studies and by schools. A possible explanation for this is that to improve educational attainment, physical activity interventions need to help to improve cognition (ie, concentration, focus). These interventions may therefore have an indirect effect on improving reading, writing and maths by improving cognition. Schools provide children with learning a range of subjects. However, if increased physical activity in schools enhances children’s learning by improving their physical and mental health, this will likely increase the acceptability of physical activity interventions in schools. This may therefore generate a greater interest from schools to implement these interventions.

Although we are not aware of another COS that specifically evaluates interventions aimed at increasing children’s physical activity in primary schools or other settings such as in the community, there are several existing frameworks for assessing these interventions. A systematic review by Cassar et al34 identified 14 frameworks applied across 27 papers34 which included reach, effectiveness, adoption, implementation and maintenance framework,35 ecological framework for understanding effective implementation,36 multilevel implementation quality framework37 and a conceptual framework for implementation.38 The review found that the frameworks were primarily used for interpreting results and analyses rather than being used as a planning tool for outcomes to be measured or for understanding results.34 Another review by Damschroder et al39 also found little evidence that frameworks for school-based physical activity interventions were used to guide the data collection.39 Findings from these reviews imply that the frameworks to assess these interventions provide little emphasis on the planning of what should be measured and perhaps explain the heterogeneity of outcomes measured to date. A study by McKay et al40 prioritised a list of frameworks to improve the quality and consistency of implementing interventions to ensure that interventions are effectively delivered to achieve population level benefits.40 COSs should be used to inform the choice of outcomes41 and our COS contributes to an important gap in these frameworks and can add to them by providing a guide on the minimum set of outcomes to measure in future studies of physical activity interventions in primary schools. It is important to note however that the existing research from physical activity intervention studies has enabled important findings of outcomes that are more commonly measured such as BMI42 and physical activity43 and have allowed for a better understanding of the impacts of these interventions on these outcomes. But any COSs currently being developed are mainly centred around childhood obesity44–46 which is complex; tackling childhood obesity requires comprehensive, multicomponent strategies. Developing COSs require the need to consider the aims and scale of the intervention, the population groups being targeted and the needs of the stakeholders. Our COS, focused on physical activity interventions in primary schools and developed in consultation with those who would benefit the most to better understand intervention effects, should be considered as part of a set of tools for wider improvement of health in primary schools.

Our study’s strengths include: we have developed the first COS for physical activity interventions in primary schools, to our knowledge, and used robust methodology as recommended by the COMET to capture a wide range of outcomes to reach consensus. Our inclusion of participants from four key stakeholder groups representing nine countries, as well as incorporating views of children, ensures the relevance of outcomes to measure for the target population. We also ensured that the domains were not predetermined. We instead established the domains led by the list of outcomes and their descriptions, thus avoiding any researcher bias. However, there are limitations to our study. The descriptions of each outcome were guided by the published literature. We had found variations in how the outcomes were described across studies. This resulted in our descriptions for each outcome either being a definition, suggestion, implying a positively directed relationship or a combination of these. Further research is needed to identify neutral descriptions of outcomes. The low attendance of participants in our consensus meeting which did not include a representation for the educators stakeholder group, may have possibly limited further discussions of the outcomes that should be included in the COS. However, the final list of outcomes was circulated to all the participants who completed both rounds of the Delphi survey and an opportunity to comment further was provided before the final outcome set was agreed. As we recruited participants through several methods including advertising on our research network websites and through snowballing, we are not aware of how many potential participants were targeted for our research and did not participate. Although our participants represented nine countries, most were UK based. The educators and health researcher stakeholder groups included participants from five countries, while participants from two countries represented the public health professional and parent groups. All stakeholder groups had a UK participant representation between 71% and 95%. The outcomes identified from our umbrella review were not limited to UK-based studies, but the lower proportion of participants representing other countries and in each stakeholder group may have prevented the identification of other outcomes that may be more relevant. Other countries and cultures may differ in the importance placed on physical activity in schools and may focus on other aspects such as educational attainment. This may bias our COS towards outcomes relevant to UK audiences. COVID-19 restrictions limited our reach to primary schools and year groups to target for our workshops; children from different year groups may have considered additional or fewer outcomes important. In addition, our representation of children with English as a second language was much higher (75%) than the average number of children with English as a second language in London primary schools (48%).47 The development of our COS during the COVID-19 pandemic may have influenced our findings. It has been widely reported that school closures and restrictions have reduced opportunities for children to be physically active and has increased poorer mental health.48 49 This may perhaps explain the higher number of outcomes in the domain of social and emotional health that met the threshold in our surveys. Finally, it may be challenging for future studies to include all 14 outcomes identified in our COS. However, as our outcomes have been grouped into three main domains, researchers may choose to include the outcomes within the domain of interest.

The development of our COS is timely; several interventions that have been implemented in schools in recent year may have stopped due to COVID-19. These interventions are likely to resume and may be more important to assess now due to the negative impacts the pandemic has had on children’s physical activity and mental health. Our COS would be relevant to future studies assessing the impact of physical activity interventions in primary schools such as The Daily Mile, a popular active mile intervention reaching one in five state-funded primary schools in England,50 and recommended by England’s National Obesity Plan.51 Despite its reach, the evidence of its impact remains limited or inconsistent.52–55

Our COS would benefit from identifying the best assessment tools to measure the outcomes that are readily available to those implementing physical activity interventions in schools. COMET suggests that a COS use should first aim to establish which outcomes are important to measure, and then aim to identify which assessment tools would be the most accessible for end users.56 There is a low uptake of COSs in randomised control trials due to lack of recommendations of valid measures, lack of involvement of key stakeholders and those implementing or assessing interventions not being aware of a COS in their field of research.56 Our next step is to identify assessment tools that are readily available to measure the outcomes in our COS. Recommendations of assessment tools would further enhance the quality and consistency of results in studies using our COS.

Prevention and public health approaches in early life to reduce health inequalities and improve health of the whole population may be a better investment than treating disease in the population that generally arises later in life.57 58 The robust processes that we have applied in this study could be repeated to inform an adolescent (young people aged 12–17 years) focused COS. Physical activity is low among the secondary school population59 and poorer mental health is also increasing among this age group.60 We recommend that our COS is included as part of a wider set of tools and frameworks that should be developed to standardise the outcomes to measure other areas of children and young people’s health such as weight and nutrition.61 This would allow for improved health to continue during adolescence and adulthood.

Conclusion

Our COS identifies the outcomes that are most important to measure for studies of physical activity interventions in primary schools. Next, we aim to identify the assessment tools to measure these outcomes. Wide use of our COS in future studies will reduce heterogeneity allowing for evidence synthesis to better understand intervention effects on children’s health and cognition during the primary school years.