The evolution of multiagency partnerships for safety over the course of research engagement: experiences from the NoGAPS project.

OBJECTIVE: Implementation of effective population-level injury prevention interventions requires broad multiagency partnerships. Different stakeholders address this from varying perspectives, and potential conflicts in priorities need to be addressed for such partnerships to be effective. The researcher-led National Guidance for Australian football Partnerships and Safety (NoGAPS) project involved the engagement and participation of seven non-academic partners, including government health promotion and safety agencies; peak sports professional and advocacy bodies and health insurance organisations.
DESIGN: The partnership's ongoing development was assessed by each partner completing the Victorian Health Promotion Foundation Partnership Analysis Tool (VPAT) annually over 2011-2015. Changes in VPAT scores were compared through repeated measures analysis of variance.
RESULTS: Overall, mean total VPAT scores increased significantly over the 5-year period (125.1-141.2; F5,30=4.61, p=0.003), showing a significant improvement in how the partnership was functioning over time. This was largely driven by significant increases in several VPAT domains: 'determining the need for a partnership' (F5,30=4.15, p=0.006), 'making sure the partnership works' (F5,30=2.59, p=0.046), 'planning collaborative action' (F5,30=5.13, p=0.002) and 'minimising the barriers to the partnership' (F5,30=6.66, p<0.001).
CONCLUSION: This is the first study to assess the functioning of a multiagency partnership to address sport injury prevention implementation. For NoGAPS, the engagement of stakeholders from the outset facilitated the development of new and/or stronger links between non-academic partners. Partners shared the common goal of ensuring the real-world uptake of interventions and research evidence-informed recommendations. Effective multiagency partnerships have the potential to influence the implementation of policies and practices beyond the life of a research project. Published by the BMJ Publishing Group Limited. For permission to use (where not already granted under a licence) please go to http://www.bmj.com/company/products-services/rights-and-licensing/.

Introduction

Implementation of effective solutions to population-level health and safety issues requires a broad multiagency approach,1 2 especially if interventions are to be sustained as ongoing practice.3 The World Health Organisation (WHO) defines a partnership as ‘a collaborative relationship between two or more parties based on trust, equality, and mutual understanding for the achievement of a specified goal’ (p. 1).4 Partnerships between researchers and stakeholders, including government departments and non-government agencies, are necessary to positively impact on public health and injury issues.5–9 Multilevel approaches are particularly important when designing and implementing prevention strategies,10 including those for injury prevention.3 9 11 Partnerships need to be sustained to ensure long-term injury prevention gains.12 However, research into which features of partnerships are most likely to achieve sustainability is scarce. In sports injury prevention, for example, fewer than 4% of studies describing the efficacy/effectiveness of exercise training programmes have considered programme maintenance factors.13

The ability of any partnership to operate effectively and meet defined targets depends on the ‘extent to which the perspectives, resources, and skills of its participating individuals and organisations contribute to and strengthen the work of the group’ (p. 187).14 Partnerships addressing public health problems can be difficult to sustain,5 particularly when varying stakeholder perspectives, with potentially conflicting priorities, exist.15 Measuring the performance of partnerships is critical to determine whether a partnership approach is working and achieving desired outcomes.16 Although evaluation of a partnership can involve an assessment of whether key targets have been reached,17 for some partnerships those targets might only be realised in the long term. Most partnership evaluation research has examined the ability of the partnership to function as an entity.5 14 Important factors for partnership effectiveness include active participation of members, member diversity, collaboration among member agencies, trust and respect between partners and having clear procedures for managing conflict and power differentials.5 16 18 An effective partnership evaluation can provide accountability to stakeholders and support the continuous improvement of the collaborations.19

Several challenges to evaluating partnership progress exist, including attributing successful outcomes to partnership arrangements and determining whether observed benefits outweigh the costs of partnership. The Partnership Continuum20 is useful for conceptualising different levels of a partnership and provides a framework for evaluating health promotion partnerships.1 It can be used to guide the identification of needs of partnership members along an engagement continuum that ranges from informal networking through to formal collaboration, with the latter representing the deepest level of engagement and synergy.

The National Guidance for Australian football Partnerships and Safety (NoGAPS)21 project is a multiagency partnership approach to the development of practical guidelines to help community sporting organisations adopt a sustainable approach to safety and to implement specific injury prevention interventions. The larger NoGAPS project aimed to develop and evaluate the strategic implementation of an evidence-based sports safety programme in community-Australian Football (AF),21 and was the first to bring together a range of stakeholder groups to improve community sport safety in Australia. The aim of this paper is to describe the evolution of the stakeholder partnership over the 5-year NoGAPS project. Differences in outcome expectations across partners, depending upon their own major operational focus, were expected. Nonetheless, it was hypothesised that these expectations would change during the project, as the partners were exposed to, and encouraged to engage with each other and the staged research activity and outputs.

Methods

The main focus of the NoGAPS partnership was on preventing injuries in community-AF. Additionally, most partners were interested in developing a robust process for scale-up to other sports. Full details of the partnership rationale, the specific research and implementation questions addressed and the evaluation plan have been published.21 The NoGAPS project was jointly initiated by researchers and the partners in response to a funding call by the National Health and Medical Research Council (NHMRC). The research team developed the funding application and scientific research plan, in response to an identified sport injury prevention need presented by the Australian Football League (AFL). Six other partners were approached to participate because of their key national/state roles in sport injury prevention. All agreed to participate. Table 1 describes these partner agencies.

Table 1

The National Guidance for Australian football Partnerships and Safety (NoGAPS) project partner organisations, their role in sports injury prevention and specific interest in Australian football

Partner organisation	Role in general sport injury prevention and/or participation promotion	Specific role/interest in AF
AFL	–	The national non-government peak sports body responsible for the setting and delivery of safety rules, policies and practices at all levels of the sport.
VicHealth	A statutory health promotion body that encourages, facilitates and supports sustained participation in sport/physical activities, including through the removal of barriers to participation, such as injury.	Given the sport is one of the major participation sports in Victoria, it invests funding in increasing/maintaining participation in community-AF and also supports AF clubs to provide healthy and welcoming environments for physical activity.
SRV	A state government department responsible for providing strategic leadership, funding and support to the sport and recreation industry to provide safe sports participation opportunities across all sports, improve facilities and increase community participation in sport and active recreation.	Provides direct funding to community-AF clubs/regions for participation initiatives, safety programme delivery and administration. Also provides funding to bodies such as Sports Medicine Australia National and Victorian Branches for the development and dissemination of sports safety resources, including to all those involved in community-AF.
New South Wales Sporting Injuries Committee	A not-for-profit statutory state government organisation that provides insurance coverage for serious injuries and promotes injury prevention and safe sports practices at a state level, across a range of sports.	–
Jardine Lloyd Thompson Sport	A national sports insurance broker agency, providing insurance cover for many sports nationally. It has invested significantly in risk management approaches for sports safety to reduce the cost of insurance coverage and payouts for sports injury treatment claims.	Provides sports injury insurance coverage to all AF clubs and registered participants Australia-wide. Has developed, and promotes, risk management resources and safety checklists specifically for AF.
Sports Medicine Australia	A peak sports medicine body, which is a not-for-profit non-government organisation that provides sports first aid training, advocates for sports safety, provides professional development in sports safety and develops sports safety guidelines and programmes. Both the state branch and the Australian national body were represented as separate partners.	Works with the AFL to help deliver sports trainer courses to educate about injury prevention and management. Develops and disseminates resources specifically for the prevention of injuries in community-AF, when funded to do so by agencies such as SRV and VicHealth.

AF, Australian football; AFL, Australian Football League; SRV, Sport and Recreation Victoria Division; VicHealth, Victorian Health Promotion Foundation.

Participants who contributed to the results presented in this paper were the agency-nominated representatives of the seven partners involved throughout the NoGAPS project. Membership of the partnership group was restricted to organisations. No players, coaches or others were included, though these individuals were engaged throughout the project.22

Ethics approval was obtained from the Monash University Human Research Ethics Committee. Anonymity was guaranteed for all agencies.

The development of the partnership was evaluated over a 5-year period from February 2011 to March 2015 using the Victorian Health Promotion Foundation Partnership Analysis Tool (VPAT).20 The VPAT was based on the Partnership Continuum and designed to help groups reflect on established partnerships and monitor their effectiveness. It reflects theory about how community partnerships can positively influence health. It has previously been used in community-based projects in Victoria, Australia23 and to explore the development of sports safety policy by a multiagency partnership.24

Formal whole-of-partnership meetings, involving the research team and partner representatives, were held biannually to discuss interim research findings and inform ongoing programme and partnership development (figure 1). During these face-to-face meetings, partners were able to have direct input into the design of the injury prevention programme content and implementation plan in real time, as it evolved. Each meeting also included planning of whole-of-partnership actions for the forthcoming 6 months.

Figure 1

Timeline of the partnership assessments (VPAT) and partnership meetings conducted through the National Guidance for Australian football Partnerships and Safety project. VPAT, Victorian Health Promotion Foundation Partnership Analysis Tool.

The first VPAT administration was before the first whole-of-partnership meeting (February 2011) and it was readministered in April 2011 to identify any immediate changes following the first partner meeting. It was then administered annually in 2012–2015, before the first whole-of-partnership meeting each year (figure 1).

Total scores (possible range 35−175) were categorised into the three recommended VPAT levels indicating the degree of partnership functionality (table 1).20 The VPAT Partnership Checklist20 was then used to assess the partners’ perceptions across seven domains: (1) determining the need for a partnership; (2) choosing partners; (3) making sure partnerships work; (4) planning collaborative action; (5) implementing collaborative action; (6) minimising the barriers to partnership and (7) reflecting on and continuing the partnership. Items within each VPAT domain were rated on a 5-point Likert scale ranging from 1 (strongly disagree) to 5 (strongly agree), then summed to give an overall domain score representing partnership function (table 2). Higher scores indicated greater agreement with the VPAT concepts of a successful partnership. Descriptive analyses were used to summarise checklist domain scores and aggregated checklist scores both within and across partners. Repeated measures analysis of variance was used to compare total and domain scores over time. The least significant difference (LSD) was used to undertake post-test multiple comparisons of significant trends. One partner did not complete the last survey and their most recent response was carried forward to the final time point. All analyses were performed in the statistical package R (R Core Team 2013, http://www.r-project.org, V.3.1.2).

Table 2

VPAT checklist total score per organisation at six time points

	Partnership analysis tool total score†
Partner organisation*	Survey 1 (pre 1st meeting)	Survey 2	Survey 3	Survey 4	Survey 5	Survey 6
A	126	141	132	134	144	143
B	135	140	140	136	135	143
C	126	151	160	151	164	152
D	126	126	126	146	116	137
E	128	127	124	132	140	142
F	123	131	141	139	143	NA‡
G	112	127	129	124	135	130
Mean (SD)	125.1 (6.9)	134.7 (9.5)	136.0 (12.4)	137.4 (9.0)§	139.6 (14.2)§	141.2 (7.3)§
VPAT partnership strength	2	3	3	3	3	3

*To protect the anonymity of the partners, they have been randomly assigned letters in the above table. †The VPAT20 suggests three recommended categories of partnership level: 3: partnership based on genuine collaboration had been established and the challenge is to maintain its impetus and build on the current success (127–175); 2: the partnership is moving in the right direction, but it will need more attention if it is going to be really successful (85–126); 1: the whole idea of a partnership should be rigorously questioned (35–84).

‡No response for survey 6 from this organisation (the data have been analysed after replacing this score by survey 5 score of the same organisation, ie, 143).

§Comparison with first survey, p<0.05.

VPAT, Victorian Health Promotion Foundation Partnership Analysis Tool.

Results

Across partners, the median total VPAT score (table 2, solid line in figure 2) increased over time and the range of scores across agencies was less variable. The mean total VPAT scores increased significantly over the 5-year period (F5,30=4.61, p=0.003). Post-hoc comparisons demonstrated significant differences in the mean total VPAT scores between the first and fourth surveys (LSD 12.30, 95% CI 1.25 to 23.32; p=0.03), the first and fifth surveys (LSD 14.43, 95% CI 3.40 to 25.47; p=0.01) and the first and sixth surveys (LSD 16.30, 95% CI 5.25 to 27.32; p=0.005).

Figure 2

Box and whisker plots showing the changes in overall VPAT scores across seven partner organisations and 5 years of the partnership. Survey 1 corresponds to the first partnership assessment before the first whole-of-partnership meeting. The second survey was conducted 2 months later and all subsequent surveys were conducted annually thereafter. The line joins the mean VPAT scores across surveys. VPAT, Victorian Health Promotion Foundation Partnership Analysis Tool.

Table 3 shows that, across all partners, mean VPAT domain scores increased significantly over the 5-year period for the following four domains: domain 1: ‘determining the need for a partnership’ (F5,30=4.15, p=0.006); domain 3: ‘making sure the partnership works’ (F5,30=2.59, p=0.046); domain 4: ‘planning collaborative action’ (F5,30=5.13, p=0.002) and domain 6: ‘minimising the barriers to the partnership’ (F5,30=6.66, p<0.001).

Table 3

Overall rating of VPAT checklist domains across partner organisations (n=7) over time

		Checklist score* mean (SD)
Domain description		Survey 1 (pre 1st meeting)	Survey 2	Survey 3	Survey 4	Survey 5	Survey 6
1	Determining the need for the partnership†	19.9 (1.5)	20.3 (2.5)	20.6 (2.6)	21.4 (1.4)	22.6 (2.1)‡	22.8 (1.8)‡
2	Choosing partners	19.6 (1.8)	19.7 (2.0)	20.1 (2.4)	18.4 (2.6)	20.1 (2.5)	20.8 (2.5)
3	Making sure the partnership works†	16.3 (2.6)	18.9 (2.0)‡	18.6 (2.7)	19.1 (1.3)‡	18.7 (2.6)	19.5 (2.5)‡
4	Planning collaborative action†	16.9 (1.3)	19.9 (1.3)‡	19.7 (1.8)‡	20.0 (0.6)‡	19.3 (2.6)‡	20.8 (1.8)‡
5	Implementing collaborative action	18.3 (1.4)	18.9 (1.1)	18.4 (1.8)	18.9 (2.3)	19.3 (1.5)	18.7 (1.6)
6	Minimising the barriers to partnership†	16.4 (1.7)	18.1 (2.5)	19.1 (2.3)‡	20.4 (1.9)‡	19.7 (2.8)‡	20.0 (2.3)‡
7	Reflecting on and continuing the partnership	17.9 (0.7)	19.0 (2.6)	19.4 (2.4)	19.1 (1.5)	19.9 (3.5)	18.5 (1.4)

*Maximum domain score =25, maximum domain item score =5.

†Significant increasing trend over survey time points according to the repeated measures analysis of variance.

‡Comparison with first survey, p<0.05.

Although there were improvements in all domains across surveys, there was some variability in change in VPAT scores across domains. Nonetheless, these results suggest that the partnerships had strengthened over the 5-year NoGAPS project, notably in many aspects not related to the initial research that brought the partners together in the first place. Online supplementary file 1 shows trends in the individual items within each domain.

Discussion

To the best of our knowledge, this is the first study to bring together a range of stakeholder groups in a formal partnership to address sports injury prevention. Government investment in sports injury prevention has been limited and therefore the funding and in-kind contributions provided by the partners to this project represented major new investments in this area. While some partners had previously worked together, this project brought together industry-wide representation to develop a coordinated and strategic approach to sports safety. Discussion of the VPAT results during the whole-of-partnership meetings encouraged partners to examine the reason for the partnership and evaluate a number of its features. This evaluation of partnership functioning was useful for identifying concerns related to communication, roles and responsibilities, shared interests and trust.

The ability of any partnership to operate effectively and meet targets depends on the ability of the participating individuals and organisations to contribute to and strengthen the work of the partnership. Understanding the partners’ perspectives and any potentially conflicting priorities provides important information on how well a partnership is functioning and the ability of the partnership to influence policy and practice. This is one of very few studies to describe the views of different partners about a multiagency and cross-sectoral injury prevention partnership and how this changes over time.

A partnership between researchers and policy officers from governmental health departments has previously been shown to have strong benefits for falls prevention.8 There has also been a call for governmental health agencies to be a lead agency in cross-sectoral injury prevention efforts more widely.9 However, very few studies have sought to understand the processes involved in the formation of such partnerships to address injury prevention and an even smaller number have investigated partnerships specifically aimed at sports injury prevention.24 In the context of general sports injury prevention, previous sports safety efforts have largely been undertaken in isolation, by either the sports or the health sectors but rarely together, and this may well have limited their success.25 To our knowledge, this is one of the first studies worldwide to assess the functioning of a multiagency partnership, crossing both sport and health sectors, created specifically to address sports injury prevention.

The NoGAPS partnership was initiated as a research project through one-to-one contact of the research team with individuals from the partner agencies and this is reflected in the fact that the ‘choosing partners’ domain was stable and collaborative action was consistently high. Although the partners had all agreed to participate in this collaborative research initiative, they did not meet as a whole team until April 2011. Comparison of the VPAT scores before and after that whole-of-partnership meeting showed that bringing everyone together for the first time contributed to a very large improvement in the functioning of the partnership. This was not unexpected, but ongoing annual VPAT evaluations demonstrated that the partnership strength grew each year. Overall, the partners began with a shared common goal of ensuring the real-world uptake of evidence-informed sports injury prevention. They identified the need for trust, mutual respect and a desire to work and communicate with each other, as important factors in an effective partnership. Lower baseline VPAT domain scores relating to the partners’ roles and responsibilities, organisational priorities and communication within the partnership improved over the 5-year period.

The need for attention to communication, roles and responsibilities, shared interests and trust as important influences on partnership functioning have previously been reported in the literature.1 14 26 Open communication among partners can help the partnership to focus on a common objective and helps to increase trust and sharing of resources between members.27 The observed VPAT trends indicate that the partners perceived improvements in the collective decision-making processes over the 5 years, presumably through interactions at the regularly scheduled whole-of-partnership meetings.

Despite this, the partners were unconvinced that strategies were in place to enhance the skills of the partnership over time. Apart from the regularly scheduled whole-of-partnership biannual meetings and frequent dissemination of research findings from the research team, there were no formal processes instituted to facilitate collaboration or action to address partnership deficiencies. Specific strategies to enhance the partnership functioning, such as using partnership brokers or workforce training in collaborative skills,28 could be used to address this.

A strength of this partnership evaluation was that it was conducted annually over 5 years. Understanding the changes in relationships between individuals and organisations is essential for both adoption and sustainability of a partnership. This study used the VPAT to show change over time and demonstrates that this tool is sensitive enough for these purposes. The strength of the relationships that an organisation develops will also depend on the level of organisational support and opportunities for interaction and collaboration with other partners. Group interaction analysis may be useful for this in the future.29

Several limitations of this study need to be acknowledged. The partnership was instigated for a specific research aim and the NHMRC Partnership Project funding opportunity was used to leverage funding and in-kind support for the project from each partner. Each had a strong stake in the project to succeed, but the partnership could have been influenced by the NoGAPS funding and contractual arrangements. It was the needs of the research project that initiated the annual whole-of-partnership meetings, which then mediated the partnership growth and goals of the partnership. The surveys relied on self-reported perceptions of the partnership and the views expressed from the organisational representatives may not have been the same as those of their colleagues. Moreover, the validity of the VPAT is not known. For two partners, the representative who completed the VPAT changed and this may have influenced within-organisation trends. This study focused on the functioning of the partnership and, while satisfaction of the stakeholders with the partnership was considered, other outcomes of effective partnerships, such as the sustainability of the partnership and changes in their policies and practices, were not measured. Finally, the reasons postulated for the improvements in the partners’ perceptions need to be interpreted with caution. During the 5-year partnership period, no specific processes were put in place to facilitate collaboration or action to address partnership deficiencies. In the absence of any specific strategies to enhance the partnership functioning or evaluation of how partner relationships outside of NoGAPS changed over time, the reasons for changes in the partners’ perceptions during the project are unknown.

It is not yet known if the partnership will be sustained, or the extent to which the sports injury prevention programme it developed will be able to be maintained. The trends in VPAT item 7.4 (supplemental file) give some suggestion that the partners may have become concerned about the availability of resources to continue the partnership as the funded project neared its end. It was not clear if this decline in the VPAT item score related to the continuation of the partnership group itself, or was more about the maintenance of the injury prevention programme. There would be value in reassessing the partnership 12 months after the formal project end, to identify determinants of, and barriers towards, its sustainability.

In addition to the promising VPAT score changes, there have been other demonstrable successes arising from collaborative effort on the part of the research team and the partner organisations. An evidence-informed exercise training programme to prevent lower limb injuries has been developed30 and is now available through the AFL's website (http://www.aflcommunityclub.com.au/index.php?id=1905). In 2015, the AFL formally endorsed the sports safety programme, committing to including it in future national coach education programmes. New learnings about how to target sports injury prevention programmes specifically for their implementation context, including engaging the community in developing an optimal delivery plan also arose.22

In conclusion, effective multiagency partnerships have the potential to influence implementation of injury prevention policies and practices beyond the life of a research project. Sports injury is a multifactorial public health problem requiring a multiagency approach to implement effective preventive solutions across all levels of sports.3 The lack of a strategic, coordinated approach to sports safety has contributed to a recognised limited adoption of policy and poor levels of implementation of research evidence and interventions to date.3 15 21 25 31

Further investment in sports safety partnerships should focus on management and action to address the identified deficiencies for the benefit of all partners and could enhance the effectiveness of the partnership. How well any partnership functions will influence its potential to contribute to, and directly influence the policy and practices of organisations delivering sports safety messages and policy solutions. The development of this NoGAPS partnership has already informed the research process, improved the knowledge of stakeholders and developed connections to help ensure the real-world uptake of interventions and research evidence recommendations.

Development and implementation of effective population-level injury prevention interventions requires broad multiagency partnerships.

Different stakeholders address this from varying perspectives, and potential conflicts in priorities need to be addressed from the outset for such partnerships to be effective.

This is the first study to assess the functioning of a multiagency partnership to address sports injury prevention intervention implementation in the sport and recreation setting.

Engagement of stakeholders from the outset in a research-driven partnership facilitates the development of new and/or stronger links between non-academic partners. Researchers and stakeholder agencies then share a common goal of ensuring the real-world uptake of interventions and research evidence-informed recommendations.