The Evolving Collaborative Relationship between Practice-Based Research Networks (PBRNs) and Clinical and Translational Science Awardees (CTSAs).

PURPOSE: Clinical and Translational Science Awards (CTSAs) and Practice-based Research Networks (PBRNs) have complementary missions. We replicated a 2008 survey of CTSA-PBRN leaders to understand how organizational relationships have evolved.
METHODS: We surveyed 60 CTSA community engagement (CE) Directors and 135 PBRN Directors and analyzed data using between and within-group comparisons.
RESULTS: Forty-three percent of CTSA CE Directors (26/60) and forty-two percent of PBRN Directors (57/135) responded. Quantitative responses revealed growing alignment between CTSA/PBRN perceptions, with a few areas of discordance. CE Directors noted declining financial support for PBRNs. PBRN Directors identified greater CTSA effectiveness in PBRN engagement, consultation, and collaborative grant submissions. Qualitative data revealed divergent experiences across CTSA/PBRN programs.
CONCLUSIONS: Relationships between CTSAs and PBRNs are maturing; for some that means strengthening and for others a growing vulnerability. Findings suggest a mutual opportunity for PBRNs and CTSAs around applied research. Studies to characterize exemplar CTSA-PBRN collaborations are needed.

Introduction

Essential elements in the research pipeline are the translational steps: moving evidence from bench to bedside and then out into routine use in practices and community-based settings [1, 2]. Woolf et al. summarized the 2 distinct “translational blocks” identified by the Institute of Medicine’s (IOM) Clinical Research Roundtable which are still relevant today: first, moving evidence from laboratory discoveries into their first testing in humans and second, from clinical studies into “everyday clinical practice and health decision making [3].” A recent systematic review identified an emerging consensus 5-phase definition of translational research, which emphasizes research along a continuum rather than across “gaps” and proposed a circular rather than linear trajectory of the research phases from basic research to populations and back [2]. Clinician and Translational Science Awards (CTSAs) and Practice-Based Research Networks (PBRNs) play an important role in current and historical efforts to improve translation research. However, few studies explore the structure and changing relationships between CTSAs and PBRNs over time [4, 5].

The CTSA program was established by the National Institutes of Health (NIH) in 2006 to improve the conduct of biomedical research, reduce the time required for translation of discoveries into practice, engage communities in research, and train the next generation of translational scientists [4]. The CTSA program was designed in part to help address a key focus of the 2004 NIH Roadmap, to speed-up the translational research timeline [1, 6]. Yet over the past decade, the NIH Roadmap and CTSA program have experienced many changes [7]. For example, in 2012 the CTSA program shifted leadership to the National Center for Advancing Translational Sciences (NCATS) with an espoused mission to “transform the translational science process so that new treatments and cures for disease can be delivered to patients faster” by supporting a national network of medical research institutions—called hubs—that work together to improve the translational research process [8]. Concurrently, NCATS stopped requiring that a community engagement core was proposed within CTSA renewal applications despite later conclusions by the IOM that training and education as well as community engagement were important strengths of the CTSA program [7]. Despite growing awareness of the work done by CTSAs, in late 2015 the cornerstone journal for the CTSA program, Clinical and Translational Science, mandated a shifted ownership, editorial staff, and focus away from the full, rich spectrum of translational research back to spotlighting laboratory discoveries [9]. There are currently CTSAs in ~60 academic medical institutions in the USA [10].

PBRNs were established in the USA in the 1970s, as collaborative groups of primary care clinicians committed to conducting research focused on the experience and delivery of care to the patients they serve [11, 12]. Early PBRN studies explored problems faced by practitioners on the front lines of primary care; findings demonstrated the gap between research conducted in controlled specialty settings and real-world practices [13]. Over the past 40 years PBRNs have evolved and expanded their focus to engage additional front-line providers and to address increasingly complex questions that emerge from daily practice [13]. In addition to engaging primary care clinicians, PBRNs now engage and/or are led by clinicians from multiple specialties, public health professionals, and key population groups (e.g., patients and providers for people with disabilities) [13-18]. Moreover, PBRNs have become multifaceted health improvement networks [19] that conduct community-engaged research [13, 20–22], lead quality improvement and practice transformation initiatives [23], support participatory implementation research [24], enable continuing education and maintenance of board recertification [25, 26], and help train the future generation of translational researchers [27]. Concurrently, PBRNs have expanded beyond the walls of primary care to engage patient and community-based partners in addressing the social determinants of health [28, 29] and in supporting dissemination and implementation research [30]. Field leaders estimate that PBRNs reach ~15% of the US population [31]; many PBRNs serve regions and populations experiencing health disparities. In 2012, prior to disbanding of funding for the PBRN resource center, there were 143 active PBRNs in the USA and nearly half reported a CTSA affiliation [32].

CTSA and PBRNs have many complementary characteristics and goals. PBRNs can play an important role in helping questions across the translational research pipeline stay grounded in the needs and interests of practicing clinicians and community-based stakeholders. In 2008, we published findings from a web-based survey that examined early relationships between PBRNs and CTSAs as perceived by CTSA community engagement (CE) directors as well as PBRN directors [4]. Our findings revealed important opportunities and challenges for these partnerships in relation to program roles, relationships, and structures for collaboration [4]. Although our findings suggested that both CTSA CE Directors and PBRN Directors found the relationship important and that the PBRN environment created opportunities for bidirectional exchanges for participatory and translational research we concluded that with only 3 years of experience together, PBRN/CTSA relationship were in the early discovery phase, with the collaborators negotiating expectations [4]. Although a recent study found that PBRNs who reported an affiliation with a CTSA was more likely to conduct pragmatic clinical research trials, participate in research projects with more than one other PBRN, to perceive fewer barriers to securing funding to support PBRN infrastructure [5], we found no additional studies that have explored the continued evolution of CTSA and PBRN relationships over time.

Many years have passed since we first characterized PBRN and CTSA relationships. These changes in the CTSA program focus and the expansion and diversification of PBRNs may have important implications on the relationships between CTSAs and PBRNs. Therefore, we conducted this new study to ascertain how CTSA CE and PBRN Directors viewed current relationships and to assess the changes in these associations since the original survey [4].

Methods

Modeled after the 2008 survey content and methods, we conducted an anonymous cross-sectional web-based survey of CTSA CE and PBRN Directors [4]. The current study was approved by the Case Western Reserve University Institutional Review Board.

Selection of Participants and Survey Administration

The Collaboration/Engagement Domain Task Force (formerly Community Engagement Key Function Committee, KFC) and the Community Engagement Survey Facilitation Reviewers of the CTSA endorsed the survey in October 2013, and provided access to the email listserv of 60 voting members from the Collaboration/Engagement Domain Task Force, excluding those members that were NIH employees. We identified 135 eligible PBRN Directors or other PBRN points of contact published on the Agency for Healthcare Research and Quality (AHRQ) PBRN Directory listserv.

Between December of 2013 and July of 2014, we distributed 3 sequential email requests for participation to CTSA CE and PBRN Directors. These requests explained the study premise and provided the appropriate online survey link (i.e., CTSA CE Director or PBRN Director) in Qualtrics survey software (www.qualtrics.com). Survey software was configured to prevent participants from taking the survey multiple times.

Survey Content

Separate web-based, anonymous surveys consisting of fixed and open-ended questions were prepared for CTSA CE Directors and PBRN Directors. The CTSA CE Director Survey consisted of 24 questions that explored the PBRN/CTSA relationship, support characteristics, and future expectations. The PBRN Director Survey consisted of 32 questions that explored similar topics with the addition of feedback on whether affiliated CTSAs understood the importance of PBRNs meeting their clinician members’ research needs/interests. CTSA CE Directors that reported they did not have any PBRN affiliation as well as PBRN Directors that were not affiliated with a funded CTSA grant/planning grant were directed to the end of the survey. The current study reports on CTSA CE and PBRN Directors who completed the full surveys describing their affiliations/working relationships with one another, support characteristics, and future expectations.

Two open-ended questions were asked of both CTSA CE and PBRN Directors: How has the collaborative relationship with your (CTSA/PBRN) changed since 2008? What expectations do you have for your future relationship between your organization and your (CTSA/PBRN)? One additional question was asked of PBRN Directors only: Do your CTSA leaders understand the importance of addressing the research needs and interests of your PBRNs’ clinician-members—please explain? One additional question was asked of CTSA CE Directors only: Does your PBRN have existing limitations to the CTSA mission—if yes, please describe? As described, participants could request a summary of the survey results by providing their identifying information in a separate link.

Data Management and Analysis

Quantitative data were downloaded into SPSS Version 22 (IBM, 2013), cleaned, then analyzed using descriptive statistics. When individual cells contained an expected count of <5, we conducted transformations when possible or utilized the Fisher exact test. To compare quantitative questions that contained >1 answer choice, we dichotomously coded each response option [i.e., Role of PBRN in CTSA: research methods (0=No; 1=Yes)]. We analyzed data using χ2 tests of independence to explore between (CTSA vs. PBRN responses on the 2014 survey) as well as within (2014 vs. 2008 responses)-group comparisons.

Qualitative data from open-ended questions were transferred to ATLAS-ti 7.0 (Scientific Software Development GmbH, 2014) for data management and analysis. We conducted semantic thematic analysis guided by the study’s research questions but were open to new information and perspectives [33, 34]. Two investigators (M.R.-B., J.J.W.) jointly developed the codebook a priori and periodically met to clarify code definitions, discuss emerging themes, and check reliability [35]. In addition, narrative responses from PBRN Directors regarding whether their CTSA leaders understood the importance of PBRN(s) addressing the research needs and interests of their clinician members were grouped into 3 categories: “yes,” “yes and no,” and “no” for review. A small number of emergent codes were added during the analysis; a coding reliability threshold of 75% was met [36]. In addition to identifying themes from open-ended questions on the 2014 survey, we compared emergent themes to published results from our 2008 study of CTSA CE and PBRN Directors [4].

Results

In total, 43% of all invited CTSA CE Directors (26/60) and 42% of all invited PBRN Directors (57/135) responded to the 2014 survey. Respondents indicating PBRN/CTSA affiliations were eligible to complete the full survey and included 70% (n=18) of the CTSA CE Directors and 73% (n=42) of the PBRN Directors. Participating PBRN Directors denoted that half (50%, n=21) of their PBRNs had been established between the years 2000 and 2013; participating CTSA CE Directors relayed that 56% (n=10) of their CTSA organizations were formed between the years 2006 and 2008. The majority of PBRNs (83%, n=50) were affiliated with only 1 CTSA. CTSAs were most often associated with 1 (40%, n=24) or 2 (40%, n=24) PBRNs. Both CTSA CE Directors and PBRN Directors reported that their CTSA renewals were due between 2015 and 2018 (CTSAs=81%; PBRNs=92%).

Quantitative Findings: Greater Alignment and Reduced Discordance

As summarized in Table 1, CTSA CE Directors survey responses in 2014 and 2008 varied less than the responses of PBRN Directors (significant difference in 4 items vs. 10 items, respectively). Comparing 2014 with 2008 responses, significantly more CTSA CE Directors reported no changes in budgetary support for PBRNs (63% vs. 4%) yet they provided significantly greater “in-kind” services to PBRNs in some areas (i.e., accounting and training). Responses of PBRN Directors in 2014 compared with 2008 also noted significantly greater use of CTSA resources (i.e., informatics, training) and mostly highlighted growth and improvement in the perceived relationships. For example, PBRN Directors indicated that significantly more PBRNs were involved with CTSAs (74% vs. 57%), were seen as very important to their CTSAs (51% vs. 24%), and that CTSAs were more often “very effective” in engaging with the PBRN (42% vs. 14%). However, in 2014, significantly more PBRN Directors indicated that their CTSAs had become less effective at referring investigators who had projects ready for implementation in a PBRN than in 2008 (52% rated poorly vs. 31%). Although more PBRN Directors in 2014 reported that research grants had been submitted or funded through NIH (44% vs. 12%), significantly more reported that no training grants had been submitted (73% vs. 36%).

Table 1

Quantitative responses from Clinical and Translational Science Awards (CTSA) community engagement (CE) Directors and Practice-Based Research Networks (PBRN) Directors on the 2008 and 2014 web-based surveys regarding PBRN/CTSA relationships*

	Comparisons between 2008 and 2014 survey responses
	CTSA CE Directors		PBRN Directors
Variable	2008 (n=25) [n (%)]†	2014 (n=26) [n (%)]†	2008 (n=69) [n (%)]†	2014 (n=57) [n (%)]†
PBRN(s) involved with CTSA (Yes)	17 (65.4)	18 (69.2)	39 (56.5)	42 (73.7)1
How important are PBRNs to CTSA?
Minimally	13 (50.0)	4 (22.2)	19 (45.2)	11 (26.8)
Somewhat important	10 (38.5)	8 (44.4)	13 (31.0)	9 (22.0)
Very important	3 (11.5)	6 (33.3)	10 (23.8)	21 (51.2)2
CTSA effectiveness in engaging with PBRN
Not effectively	11 (42.3)	6 (33.3)	16 (37.2)	10 (24.4)
Somewhat effectively	12 (46.2)	7 (38.9)	21 (48.8)	14 (34.1)
Very effectively	3 (11.5)	5 (27.8)	6 (14.0)	17 (41.5)3
Role of PBRNs in CTSA (Yes)
Study recruitment	20 (76.9)	14 (77.8)	30 (43.5)	23 (56.1)
Research methods	20 (76.9)	12 (66.7)	27 (39.1)	27 (65.9)4
Educate researchers	15 (57.7)	9 (50.0)	24 (34.8)	17 (41.5)
Effectiveness of PBRNs in engaging community
Not effectively	8 (30.8)	1 (7.70)	5 (12.5)	3 (9.10)
Somewhat effectively	15 (57.7)	7 (53.8)	23 (57.5)	24 (72.7)
Not effectively	3 (11.5)	5 (38.5)	12 (30.0)	6 (18.2)
Effectiveness of PBRNs engaging academic investigators
Not effectively	6 (23.1)	3 (16.7)	3 (7.30)	8 (19.5)
Somewhat effectively	16 (61.5)	9 (50.0)	13 (31.7)	17 (41.5)
Very effectively	4 (15.4)	6 (33.3)	25 (61.0)	16 (39.0)
Financial support provided by CTSA to PBRNs
None	0 (0.00)	4 (26.7)	17 (41.5)	10 (26.5)
<$50,000	8 (32.0)	2 (13.3)	6 (14.6)	6 (15.8)
$50,000–100,000	3 (12.0)	6 (40.0)	8 (19.5)	11 (28.9)
>$100,001	14 (56.0)	3 (20.0)5	10 (24.1)	11 (28.9)
CTSA budget cut for PBRNs
Smaller than rest of CTSA budget	8 (32.0)	1 (12.5)	8 (20.5)	3 (8.30)
Proportionate to rest of CTSA budget	7 (28.0)	2 (25.0)	4 (10.3)	2 (5.60)
Larger than rest of CTSA budget	1 (4.00)	0 (0.00)	5 (12.8)	5 (13.9)
Not changed/not applicable	1 (4.00)	5 (62.5)6	1 (2.6)	26 (72.2)7
Increased	8 (32.0)	0 (0.00)	21 (53.8)	0 (0.00)
Types of support CTSA provides to PBRNs (Yes)
IRB	9 (34.6)	11 (64.7)	13 (18.8)	11 (27.5)
Regulatory	5 (19.2)	5 (29.4)	9 (13.0)	5 (12.50)
Human resources	6 (23.1)	6 (35.3)	8 (11.6)	7 (17.5)
Accounting	1 (3.80)	6 (35.3)8	2 (2.90)	1 (2.5)
Biostatistics	12 (46.2)	8 (47.1)	15 (21.7)	14 (35.0)
Informatics	9 (34.6)	9 (52.9)	12 (17.4)	18 (45.0)9
Payroll	1 (3.80)	3 (17.6)	2 (2.90)	3 (7.50)
Training	10 (38.5)	12 (70.6)10	14 (20.3)	19 (47.5)11
Consultation	15 (57.7)	14 (82.4)	21 (30.4)	19 (47.5)
Lab	4 (15.4)	3 (17.6)	7 (10.1)	3 (7.50)
NIH grants from CTSA-PBRN collaboration (>1 response)
Research grant applications
Submitted/funded	8 (30.8)	8 (50.0)	8 (11.6)	18 (43.9)12
None submitted	14 (53.8)	4 (25.0)	23 (33.3)	17 (41.5)
Training grant applications
Submitted/funded	5 (19.2)	2 (12.5)	6 (8.70)	5 (12.2)
None submitted	17 (65.4)	12 (75.0)	25 (36.2)	30 (73.2)13
How well CTSA-referred investigators have done in proposing study types that can readily be implemented in a PBRN?
Below average/poor	N/A	N/A	13 (31.0)	18 (51.4)14
Average	N/A	N/A	4 (9.50)	8 (22.9)
Above average	N/A	N/A	25 (59.5)	9 (25.7)
CTSA affiliation has required significant changes in PBRN(s) (Yes)	N/A	N/A	16 (39.0)	15 (37.5)

NIH, National Institutes of Health; N/A, not asked; FET, Fisher exact test.

Only respondents indicating PBRN/CTSA affiliations were eligible to complete the full survey (CTSA n=18; PBRN n=42).

Because of rounding or multiple response options, some percentage grouping totals do not add up to 100%.

1 χ2 (2)=6.75, p<0.05, Cramer’s V=0.29 (medium effect size).

2 χ2 (2)=8.00, p<0.05, Cramer’s V=0.31 (medium effect size).

3 χ2 (1)=7.35, p<0.01, Cramer’s V=0.26 (weak effect size).

4 FET, p<0.001, Cramer’s V=0.78 (strong effect size).

5 FET, p<0.01, Cramer’s V=0.59 (large effect size).

6 FET, p<0.01, Cramer’s V=0.68 (large effect size).

7 χ2 (1)=9.68, p<0.01, Cramer’s V=0.30 (medium effect size).

8 FET, p<0.05, Cramer’s V=0.42 (medium effect size).

9 χ2 (1)=8.88, p<0.01, Cramer’s V=0.29 (weak effect size).

10 χ2 (1)=4.01, p<0.05, Cramer’s V=0.18 (weak effect size)

11 χ2 (1)=14.87, p<0.01, Cramer’s V=0.37 (medium effect size).

12 χ2 (1)=14.04, p<0.01, Cramer’s V=0.36 (medium effect size).

13 FET, p<0.05, Cramer’s V=0.34 (medium effect size).

14 χ2 (1)=14.04, p<0.01, Cramer’s V=0.36 (medium effect size).

Perceptions of the CTSA-PBRN relationship appeared more similar than different in 2014 as reported by CTSA CE and PBRN Directors (see Table 1). Three significant areas of difference related to the types of support provided by CTSAs to their PBRNs focused on use of “IRBs,” “accounting,” and “consultation.” Nine additional items that significantly differed between CTSA CE and PBRN Directors in 2008 were no longer significantly different in the 2014 survey. For example, with respect to the roles of PBRNs within CTSAs, in 2008 CTSA CE Directors indicated that PBRNs were CTSA resources for study recruitment, research methods expertise, and research education significantly more often than did PBRN Directors; these differences were not observed in 2014. Similarly, in 2008 CTSA CE Directors compared with PBRN Directors reported that CTSAs provided significantly higher levels of financial support to PBRNs, significantly more biostatistical support, and submitted significantly greater numbers of CTSA-PBRN collaborative research and training grant applications; none of these differences were observed in 2014. Although in 2008 PBRN Directors as compared with CTSA CE Directors noted significantly higher effectiveness ratings related to how their PBRN engaged academic investigators, this item did not significantly differ for either respondent group in 2014.

Qualitative Findings: Divergent Experiences of CTSA/PBRN Programs Over Time

Qualitative responses on the 2014 survey indicated how perceptions of the CTSA/PBRN relationship were changing over time. As summarized in Table 2, both CTSA CE Directors and PBRN Directors indicated that there had been an improvement in collaboration and awareness of the role PBRNs could play in CTSA, increased value placed on PBRNs as infrastructure for community-engaged research, and an improved understanding of the benefits that PBRNs could provide. Some CTSA CE and PBRN Directors also noted, however, that their PBRN(s) lost the financial support of their CTSA(s) and that the CTSA(s) “lost interest” in working with the PBRN. In contrast, other respondents reported that their PBRNs had continued collaborating with their CTSA and were currently receiving financial, instrumental, or in-kind support.

Table 2

Clinical and Translational Science Award (CTSA) community engagement (CE) and Practice-Based Research Network (PBRN) Directors perceptions of PBRN/CTSA relationships over time and in the future

Reported by CTSA CE Directors (n=18)	Reported by PBRN Directors (n=42)
Question: How has the CTSA/PBRN collaborative relationships changed since 2008?
CTSA became more aware of benefits of PBRNs “Since initial funding, our CTSA has become more aware of the PBRN’s benefits as institutional infrastructure.” “More PBRN involvement since the importance of the primary care community is increasingly recognized.” No longer see PBRNs as solely a recruitment source “…we thought we would use the PBRN for recruitment of already planned studies. When there was appropriate resistance by our PBRN, the relationship soured. We’re now rebuilding to a positive interaction.” PBRNs have lost CTSA financial support “We helped start the PBRN with a pilot grant and in-kind support… we continue with in-kind support but no longer have the money for continued financial support.” “We’ve had less involvement with PBRN over time.” “The PBRN is losing strength due to underutilization and lack of funding”	CTSA/PBRN collaborative relationship has grown “Evolved from no interest in practice-based research to our PBRNs being essential to community engagement as described in grant applications and progress reports” “The relationship has improved due to recognition by the universities that they need external collaborators and we are more than just a data source” PBRN has grown through CTSA affiliation Our CTSA involvement has increased our visibility” “We are evolving and having more interactions between all of our PBRNs and the Community Engagement Core” “We have grown, joined a statewide coalition of PBRNs, and work with more community-based investigators” CTSA’s interest in PBRNs has declined “The CTSA’s enthusiasm for the PBRN has waned” “We had a closer relationship with the CTSA in the first round of funding, but that decreased in the 2nd round”
Question: What expectations do you have for the CTSA-PBRN relationship in the future?
Increase big data and data sharing infrastructure “Robust linkages via data connectivity and access to big data for NIH investigators at our CTSA.” “Increased data sharing.” Increasing community engagement activities “We hope to better integrate our primary care CE activities through the PBRN with residential, educational, and service projects.” “We will continue to work with the CTSA to fund projects where we can write in support for PBRN infrastructure, especially community outreach support.” Focus on healthcare transformation research “We hope the CTSA develops a stronger focus on healthcare transformation.” “Increased emphasis on practice change and transformations, per network interests, but it is unclear that the CTSA holds this as an important issue.” Limited funding for PBRNs in the future “If some infrastructure funding can be secured, the current situation could be improved… we are working on this”	CTSA/PBRN relationships will get stronger “Strong and getting stronger. PBRN success is a positive reflection on the CTSA.” “We both derive benefit. The marriage seems to be going well.” “We anticipate continued growth of collaborative relationships with CTSA cores and resources.” The CTSAs should appropriately compensate the PBRNs “PBRNs should be a required entity under each institution’s CTSA award…if they don’t fund us, then they should stop taking credit for the work we do independently.” “Sustained funding level via the CTSA for PRBN infrastructure.” Desire for PBRN involvement in decision making “…that there actually be a seat at the CTSA table for the PBRN.” “Hope to see PBRNs and community groups being engaged in CTSA governance.” An uncertain future “Not sure what to expect since it seems like the goals of the CTSA funders are always changing.” “They may stop supporting us”

CTSA CE and PBRN Directors also displayed variability in their future expectations for CTSA-PBRN collaborative relationships (see Table 2). CTSA CE Directors acknowledged the need for continued flexibility in the research and healthcare environments, specifically the desire for PBRNs to expand into broader community engagement and to develop robust linkages for increased data sharing. PBRN Directors tended to describe their expectations for future CTSA-PBRN relationships with less certainty. Although some PBRN directors voiced positive expectations for continued collaboration and shared success, many indicated that meaningful collaborations will require increased levels of CTSA support and more shared decision making between the leaders of CTSAs and PBRNs.

PBRN Directors reported seeing significant variability in their CTSA CE Director’s understanding and prioritization of the research needs of the clinicians in their networks, see illustrative quotes in Table 3. Some PBRN Directors characterized CTSA leaders as understanding that the support of PBRN clinicians for local research was necessary in order to engage practices in larger studies, other CTSA leaders were reported to be less receptive to this message and continued to view PBRNs as resources for subject recruitment into clinical trials. Still others indicated that although their CTSA leaders believed the PBRNs to have much potential, the CTSA had not committed substantial infrastructure resources due to competing demands for financial support. As summarized in Table 4, when asked to describe the limitations of PBRNs with respect to the CTSA’s mission, CTSA CE Directors indicated that the priorities of CTSAs and PBRNs do not perfectly align, that CTSAs need to see PBRNs as more than a recruitment source for clinical trials, that the foundational ideologies might differ, and that it is often challenging to agree upon a common agenda that meets the needs of CTSAs and PBRNs within the confines of declining CTSA budgets. Overall, qualitative data suggest that the receptivity of CTSAs to PBRNs and vice versa was related to the level of congruence between CTSAs’ needs and of the goals of the associated PBRN(s).

Table 3

Practice-Based Research Network (PBRN) Directors’ perceptions of Clinical and Translational Science Award (CTSA) Leaders’ understanding of the importance of addressing/prioritizing the research needs and interests of PBRN clinician-members (n=42)

Yes	Yes and No	No
“The leadership is committed to community practice engagement” “Yes, but it is still a minority view” “They understand that it is necessary for maintaining practice engagement” “Our CTSA leaders understand that the PBRNs should do work relevant to clinicians in the PBRN and help identify academic researchers with those interests. It’s a dance and balance between academic driven topics and practice”	“Yes and no. It is what we are trying to TEACH the CTSA leaders that the study BEGINS with the practitioner not with the investigator” “Some leaders yes, others no. Those who have community practice research experience are more likely to understand the needs and interests of the clinicians” “They realize that there is an unrealized potential, but it has not been a high priority for them to invest in it”	“They know we exist and would like to use us as a recruitment mill. They do not engage us as stakeholders or ask for our clinicians’ needs or interests” “I don’t think it’s a lack of understanding, but a lack of even thinking about it. The PBRN is an afterthought” “Always a big problem getting campus folks to understand that primary care clinicians are busy and have their own research interests and priorities”

Table 4

Clinical and Translational Science Award (CTSA) community engagement (CE) Directors’ perceptions of the existing limitations of Practice-Based Research Network (PBRNs) to the CTSA mission (n=18)

The PBRN’s agenda does not always match the CTSA’s agenda “The networks have their own agendas/priorities. Those priorities do not always overlap with the priorities of the CTSA; the challenge is to develop a common agenda while still making the networks responsive to provider interests”

CTSAs need to see PBRNs as more than a recruitment source for clinical trials “Our CTSA is still navigating the tension between PBRNs as a recruiting center for biomedical research and being an autonomous entity for projects meaningful to practice and their communities”

The CTSAs and PBRNs may have differing ideologies “The CTSA has only slowly recognized our PBRN as a resource and its capacity for conducting research. Our PBRN has remained independent of CTSAs office of clinical trials for both ideological and logistical reasons”

Lack of funding to support PBRNs “PBRNs continue to struggle with financial sustainability in the face of declining support from the institution, given the mandate to reduce CTSA expenditures by [X%] over the next funding cycle”

Discussion

Despite the overlapping goals of CTSA and PBRNs, there are a paucity of studies that explore these relationships and how they have changed over time [4, 5]. In fact, comparisons of the findings from our current study with the original 2008 surveys highlight the continually evolving relationship between PBRNs and CTSAs. Our quantitative findings indicate greater alignment between CTSA CE Director and PBRN Director perceptions in 2014 than in 2008 [4]. However, qualitative results suggest that there is growing discordance within respondent categories. For example, PBRN Directors described experiences with their CTSAs that were highly variable—some report strong collaboration and support while others described how their PBRNs were an afterthought for their CTSA and that they held little sway in leadership and decision making. Financial support of PBRNs by CTSAs was also wide-ranging—some CTSA Directors noted that PBRNs were spared from across-the-board budget cuts because the networks were so highly valued whereas other PBRNs lost CTSA financial support and were not provided with alternate funding to continue the collaboration.

These findings suggest that in some circumstances PBRNs are closely aligned with the CTSA structure and mission, while in other settings PBRNs are working to find new ways to fund their operations so they are not so reliant on the CTSA and changing organizational missions. These findings echo recent results from a survey of PBRN Directors in the USA and Canada which found that PBRNs affiliated with CTSA programs were less likely to report maintaining funding as a significant barrier [5]. Moreover, they suggest interesting changes as both CTSA and PBRNs evolve from programs that support “translational research” to those that are integral partners in a broader, community-focused learning healthcare system [7, 19, 37]. Although early visions of translational research specified divisions between research and practice, learning healthcare systems facilitate an “iterative innovation process designed to generate and apply the best evidence for the collaborative healthcare choices of each patient and provider; to drive the process of discovery as a natural outgrowth of patient care; and to ensure innovation, quality, safety, and value in healthcare [38].”

Despite evidence of growing congruence between CTSAs and PBRNs generally, both are responding to different environmental challenges as they seek to stay rigorous and relevant to their stakeholders and funders. Challenges for CTSAs include reductions in core funding and an increasing emphasis on the development of partnerships for research and collaboration across the CTSA consortium. Many changes in CTSAs have been driven by evolving priorities of the CTSA program and recommendations from a 2013 IOM report [7]. In parallel, PBRNs face increasing demands on clinicians’ time, the need to make successful value propositions to healthcare systems for practices’ engagement in research, and pressure to develop the capacity to effectively utilize electronic health information and large administrative data sets in PBRN studies [13, 39–41]. As PBRNs have responded to the needs of their stakeholders, many have recognized that community partners and thus community-engaged research methods are critical to PBRN success [20, 42, 43].

Community engagement provides an exemplar case for exploring how changes in funding and healthcare delivery may be shaping CTSA and PBRN relationships. As PBRNs shifted their focus “beyond the clinic walls” it provided an ideal opportunity to partner with CTSAs around community engagement, an area where many CTSAs expressed early challenges [7]. For example, PBRNs had a long-standing history of linking academic investigators and community-based practices to identify, develop, and to conduct research and were expanding this approach to bridge with community. For example, the Community Health Improvement and Research Partnership model is one approach used by PBRNs to conduct community-engaged research and to mobilize research partnerships between academicians and diverse community stakeholders (e.g., health system, schools, business, service providers, engaged citizens) [44-47].

Thus, it is possible that CTSAs could learn from and leverage existing PBRN expertise and infrastructure rather than to create parallel yet distinct programs covering the same regions [5]. However, in 2012 NCATS stopped requiring that a community engagement core was proposed within CTSA renewal applications, and many interpreted this to mean that CE had become a lower priority for NCATs. In reaction, some CTSAs de-emphasized their CE groups in renewal applications and prioritized core functions that were perceived as closer to the center of NCATS vision for clinical and translational research, which had deleterious effect on CTSA CE-supported PBRNs. Although the 2013 IOM report stressed the importance of community engagement within CTSAs, this did not result in NCATS’ reinstatement of CE as a required key function. Instead NCATS’ clarified that all CTSAs must have core resources across the full spectrum of translational research and emphasized that each CTSA program should emphasize it’s particular strengths [7]. These changes may have prompted a shift away from CTSAs supporting PBRNs that are important developing assets but are not viewed as major strengths. Such changes would align with perceptions within the PBRN community that networks that were viewed as strengths within their CTSAs have received increased CTSA support over time, while PBRNs on the margins of their CTSAs have experienced funding cuts.

There are a few important limitations of this study. First, we only administered the survey instrument to CTSA CE Directors and PBRN Directors. It is possible that some PBRNs may be affiliated with the CTSA more broadly, or that other stakeholders (e.g., patients, investigators, community stakeholders) may have varied perceptions on CTSA/PBRN relationships. This approach was congruent with the original 2008 survey; participants were selected based on our understanding that these positions would have the most comprehensive view of CTSA/PBRN relationships. Second, less than half of all respondents approached completed the survey. However, this appears higher than the average response rate on web-based surveys [48]. Third, we conducted cross-sectional surveys in 2008 and 2014. In both surveys there was considerable heterogeneity in participant responses. Our analysis identified statistically significant differences between CTSA CE and PBRN Directors as well as across the 2008 and 2014 surveys. Qualitative findings reveal heterogeneity in participant responses that cannot fully be explored. Finally, because we deidentified participant responses we cannot link CTSAs or PBRNs to look at change over time within an individual settings. Despite these limitations, replication of the 2008 survey in 2013–2014 allowed us to understand incremental changes in PBRN/CTSA relationships and provides key considerations to inform for future research.

Our current findings indicate that even after 6 additional years, PBRN and CTSA relationships are still evolving. Although perceptions appear to be aligning in the quantitative data, qualitative themes revealed considerable variation across the participating sites. These variations may be due to historical relationships, federal changes to the CTSA mission or funding priorities for PBRNs, or to local competing priorities. A few case studies have described how CTSAs, PBRNs, and other regional programs have effectively partnered to promote research and community engagement [44, 49]. Future research is needed to explore how CTSA and PBRN relationships evolve over time and to identify factors that contribute to successful program alignment. We suggest that longitudinal as well as qualitative case studies may provide a rich opportunity to identify promising practices in CTSA and PBRN relationships. For example, conducting comparative case studies of “high” and “low” performing CTSA/PBRN collaborations could evaluate contextual factors and resource sharing structures over time. Qualitative methods could allow data collection from multiple CTSA and PBRN stakeholders (e.g., CTSA Directors, PBRN Network Managers, Affiliated Investigators, community-based stakeholders) and be used to identify promising practices in fostering collaborative rather than competitive partnerships for these programs. Conducting research and providing technical assistance to CTSAs/PBRNs in parallel could present an opportunity to share transportable lessons from what is working across the great variability in different CTSA and PBRN cultures, priorities, and settings. As increasing numbers of CTSAs submit for renewal of their applications under the modified RFAs, we may continue to see increasing variability in CTSA-PBRN relationships in the years ahead.

CTSAs and PBRNs have great overlap in their core missions of supporting the generation of relevant new knowledge to improve person and population health, and of speeding translation of research into practice for community benefit. Robust engagement of PBRNs has the potential to ensure that the questions asked by CTSA affiliated faculty serve the needs of “real world” practitioners and stakeholders across all phases of translational research, from basic research to population-based studies and back again [2]. Our 2008 study reported that in many cases PBRNs and CTSAs were in the discovery phase [4]. In the interim, PBRNs and CTSAs have undergone many changes in their internal and external contexts, yet a paucity of research has explored further evolution in these relationships [5]. Our updated 2014 survey results indicate that despite greater alignment in perspectives and a recognized value in a collaborative relationship, the future of CTSA and PBRN relationships is uncertain. Notably, the ability to respond with support to build the PBRN/CTSA relationship is limited as budgets are cut, primary care research is threatened, and the priority for community engagement by academic health centers changes. Our study shows growing recognition of the challenges and possibilities for continued evolution of collaborative opportunities between CTSAs and PBRNs to advance scientific knowledge, foster learning healthcare systems and to improve practice and community health.