Facilitative Components of Collaborative Learning: A Review of Nine Health Research Networks.

OBJECTIVE: Collaborative research networks are increasingly used as an effective mechanism for accelerating knowledge transfer into policy and practice. This paper explored the characteristics and collaborative learning approaches of nine health research networks. DATA SOURCES/STUDY
SETTING: Semi-structured interviews with representatives from eight diverse US health services research networks conducted between November 2012 and January 2013 and program evaluation data from a ninth. STUDY
DESIGN: The qualitative analysis assessed each network's purpose, duration, funding sources, governance structure, methods used to foster collaboration, and barriers and facilitators to collaborative learning. DATA COLLECTION: The authors reviewed detailed notes from the interviews to distill salient themes. PRINCIPAL
FINDINGS: Face-to-face meetings, intentional facilitation and communication, shared vision, trust among members and willingness to work together were key facilitators of collaborative learning. Competing priorities for members, limited funding and lack of long-term support and geographic dispersion were the main barriers to coordination and collaboration across research network members.
CONCLUSION: The findings illustrate the importance of collaborative learning in research networks and the challenges to evaluating the success of research network functionality. Conducting readiness assessments and developing process and outcome evaluation metrics will advance the design and show the impact of collaborative research networks.

Introduction

Collaborative networks and learning communities are increasingly used as effective mechanisms for accelerating knowledge transfer into policy and practice. Given the information explosion facilitated by technological advancement, organizations across diverse sectors – from business to economics to psychology – rely on networks for internal and external knowledge sharing, communication and collaboration. Collaborative networks provide a structure for individuals and organizational entities that are autonomous, geographically dispersed and heterogeneous in their operating environment and culture, to work collectively to achieve a common or compatible goal (Camarinha-Matos and Afsarmanesh 2006; Shuman and Twombly 2009). The benefits of collaborative networks are clear: they stimulate creativity and the identification of innovative approaches to solve complex problems; they align organizational objectives and activities to achieve efficient and high-quality results; they enhance sharing of individual and collective assets (e.g., lessons learned, tools, funding); and they foster trust, teamwork, reciprocity and mutuality (Camarinha-Matos and Afsarmanesh 2006; Sorgenfrei and Smolnik 2014).

The implementation and funding of collaborative health networks has flourished throughout the past two decades. The 2006 Inventory and Analysis of Clinical Research Networks identified nearly 300 clinical research networks in the US and Canada. Approximately half carried out clinical trials as their primary activity, and others supported observational research, outcomes research or best-practice modelling (Kagan et al. 2009). Furthermore, the number of research networks is increasing. Beginning in the 1990s, commentators noted a move towards “big science”: large, collaborative research initiatives with annual budgets of $5 million or more (Kagan et al. 2009). In 2013, the Patient-Centered Outcomes Research Institute invested >$100 million to develop 29 health data networks and a coordinating centre (Fleurence et al. 2014; PCORI 2013).

For health services and clinical research, networks offer analytical advantages such as increased sample size and population diversity for enhanced statistical power, subgroup analyses and generalizability (Go et al. 2008). Networks allow researchers to answer a broader array of questions, for example, about variation in process and outcomes by region and setting (Ayanian et al. 2004). Networks facilitate collaboration on analyses that require the expertise of methodologists at other institutions.

In addition to these analytical functions, some research networks emphasize shared learning among participants through collaborative learning models and techniques from the business and organizational development fields. Scientific collaboration can be limited by the independent culture of scientists, disciplinary specialization and decentralization of research capabilities (Bos et al. 2007). However, through meetings, presentations and training of junior researchers, research networks promote collaboration, professional development and shared learning in both informal and formal ways. A growing trend capitalizes on the contributions of scientists with different perspectives by fostering interdisciplinary, multidisciplinary and transdisciplinary research (Adler and Stewart 2010; Chilingerian et al. 2012; Fiore 2008; Hall et al. 2012; Popp et al. 2014). The interdisciplinary aspect of research networks is the most obvious in community-based research including practice-based research networks (Israel et al. 1998; Schmittdiel et al. 2010) but is also apparent in clinical research networks (Go et al. 2008).

Despite the growth in research networks, the mechanisms and structures through which research networks promote collaborative learning have not been systematically explored. How do health research networks that seek to facilitate shared learning motivate researchers to participate? Once participation is established, how do networks promote key objectives such as exchanging information, sharing innovation and collectively focusing on a topic? The purposeful combination of study-specific support and collaborative learning functions in research networks may be one of the most effective ways to catalyze broader innovation in science because it brings together both analytic and collaborative learning functions.

The large data sets and systematic research methods available to networks support more complex analyses than a single study. The infrastructure of collaborative learning networks facilitates the exchange of ideas to promote development and dissemination of state-of-the art approaches and the training and retention of a skilled scientific workforce. Research networks with strong collaborative learning functions may be especially valuable for accelerating new and complex fields of research that rely on interdisciplinary methods, including health services research (Bowers et al. 2013).

This study originated from our efforts to design and implement a Technical Assistance Center for the Agency for Healthcare Research and Quality (AHRQ) Multiple Chronic Conditions Research Network (MCCRN). AHRQ established the MCCRN to foster collaboration among 45 research grant recipients funded between 2008 and 2010 to conduct studies on MCC. The purpose of the MCCRN was to expand and enhance the existing body of knowledge and evidence on care for people with MCC. The role of the Technical Assistance Center was to convene the 45 investigator teams and facilitate a series of in-person and virtual network activities (LeRoy et al. 2014). In addition to evaluating the MCCRN and Technical Assistance Center, we observed and documented the facilitative elements of collaboration among the MCCRN over time.

The objective of this paper is to explore the characteristics of nine health research networks; illustrate how they used collaborative approaches to develop a shared vision and structure to promote collaborative learning; and offer recommendations for enhancing collaboration in health research networks.

Methods

Study design

To learn from the experiences of health research networks and compare the facilitative components of the MCCRN with other networks, we conducted a qualitative study using telephone interviews with leaders of research networks. We wanted to understand the phenomenon of collaboration among network participants, including the best ways to facilitate shared learning when research studies are diverse and topics are in emerging fields of study (Moustakas 1994). Therefore, we gathered perspectives and experiences on collaborative learning research networks from investigators in research networks outside the MCCRN. This information was combined with findings about the MCCRN from project evaluations.

Sample selection

To identify research networks that were currently in operation and incorporated learning collaborative functions and were advancing an emerging field of health services research, we searched and reviewed public websites and peer-reviewed and grey literature. We searched the PubMed database of the US National Library of Medicine at the National Institutes of Health, Google Scholar and ScienceDirect using the terms: “research network,” “learning OR research collaborative,” “health research network,” “health collaborative,” “interdisciplinary research” and “transdisciplinary science.” Searches were limited to articles published in English on collaborative healthcare research networks administered in the US. We also asked AHRQ staff members who facilitate and coordinate research networks and network officials that we contacted to identify eligible networks. Through this process, we identified 18 potential networks. We did not conduct an exhaustive scan of research networks, rather we sought to identify a sample of networks with a collaborative learning emphasis but varied structures and focuses. We searched for mature networks whose leaders could reflect on collaborative learning and related processes. After a careful review, we limited our non-MCCRN sample to eight diverse and established health-related research networks.

Interview guide and interview procedures

We developed a short, semi-structured interview protocol with questions about network mission, funding, organizational structure and membership, and methods for collaboration and knowledge dissemination. We also asked about barriers and facilitators of coordination and collaboration among network participants, and elicited recommendations for funding, designing and sustaining future research networks. The Abt Associates Institutional Review Board determined that the study was exempt from review. Interviews were conducted between November 2012 and January 2013. Two trained researchers facilitated the interviews, along with one assigned note-taker. Respondents were network leaders, usually the steering committee chair or project officer for the sponsoring organization.

For the MCCRN, characteristics, barriers and facilitators were based on the project's final evaluation report, which summarized data on the experiences of MCCRN participants collected via online survey and one-on-one telephone interviews. We included our own observations on implementing the Technical Assistance Center, which was part of the evaluation.

Analyses

Multiple team members reviewed detailed notes from the interviews to distill salient themes. Coding was based on a priori codes from the literature, as well as themes that emerged from the data. Network websites were reviewed for additional information if information was missing. The coding team held three analytical retreats to discuss and compare codes across the nine networks and to interpret the data.

Characteristics of Collaborative Networks

In the following section, we describe the characteristics of the nine networks, including mission, funding and membership (Table 1), as well as governance structures and approaches for collaboration and dissemination (Table 2).

Table 1.

Comparison of research network organizational characteristics, 2013

Network	Mission and/or goals	Members	Funder(s)	Coordinating centre
Cancer Prevention and Control Research Network (CPCRN)	Accelerate the adoption of evidence-based cancer prevention and control to reduce the burden of cancer	10 organizations 180 individuals	Centers for Disease Control and Prevention & National Cancer Institute	University of North Carolina Chapel Hill
Community Health Applied Research Network (CHARN)	Conduct comparative effectiveness and patient-centred outcomes research to improve patient care at federally supported community health clinics	23 organizations (including health centres) 73 individuals	Health Resources and Services Administration	Kaiser Foundation Hospitals Center for Health Research
Collaborative Care Research Network (CCRN)	Conduct practice-based primary care research that examines the impact of behavioural health on primary care and health outcomes	78 organizations 111 individuals	None	American Academy of Family Physicians (AAFP) National Research Network
MacArthur Research Network on Socioeconomic Status (SES) & Health	Enhance learning on socioeconomic factors that affect the health of individuals and their communities	13 organizations	MacArthur Foundation	None
Medicaid Medical Director's Learning Network	Advance the health of US Medicaid patients by increasing the sharing of knowledge between state Medicaid Medical Directors	45 states* 59 individuals	Agency for Healthcare Research and Quality	AcademyHealth
HMO Research Network (HMORN)	Improve healthcare delivery through comparative effectiveness research that connects resources and capabilities of healthcare systems	18 organizations 400 individuals	Member dues support cross-project infrastructure, in close coordination with specific projects (e.g., Cancer Research Network)	N/A; organized under a Board of Governors and several executive committees, in close coordination with the leadership of individual projects; one member organization administers the budget
Pediatric Emergency Care Applied Research Network (PECARN)	Conduct multi-institutional research on prevention and management of acute illnesses and injuries in children and youth across the continuum of emergency medicine healthcare	18 organizations 19 individuals	Health Resources and Services Administration	University of Utah
Washington University (WU) and Barnes-Jewish Hospital (BJH) Epicenter for Prevention of Healthcare Associated Infections	Develop improved systems to detect and prevent healthcare-associated infections	13 organizations 20 individuals	Centers for Disease Control and Prevention	None
Multiple Chronic Conditions Research Network (MCCRN)	Advance the field of multiple chronic conditions through comparative effectiveness research, infrastructure development, and dissemination of collective work	45 organizations 75 individuals	Agency for Healthcare Research and Quality	Abt Associates

All states are invited to participate in the Medicaid Medical Director's Learning Network; however, the number of states represented fluctuated over time. At the time of the interview, 45 states were active in the network.

Table 2.

Research network governance structure and methods for collaboration and dissemination

Network	Governance structure				Methods of collaborative learning
	Steering committee	Sub-committees	Annual in-person meetings	Conference calls	Workgroups	Webinars
Cancer Prevention and Control Research Network (CPCRN)	✕		✕	✕	✕	✕
Community Health Applied Research Network (CHARN)	✕	✕	✕	✕	✕	✕
Collaborative Care Research Network (CCRN)	✕	✕		✕
MacArthur Research Network on Socioeconomic Status (SES) & Health			✕	✕	✕
Medicaid Medical Learning Network	✕		✕		✕	✕
HMO Research Network (HMORN)	✕	✕	✕	✕	✕	✕
Pediatric Emergency Care Applied Research Network (PECARN)	✕	✕	✕	✕	✕	✕
Washington University (WU) and Barnes-Jewish Hospital (BJH) Epicenter for Prevention of Healthcare Associated Infections	✕	✕	✕	✕	✕	✕
Multiple Chronic Conditions Research Network (MCCRN)			✕	✕	✕	✕

All networks also maintained typical communication vehicles, such as a website and LISTSERV.

Network purpose and evolution

Respondents shared similar motivations for forming their networks: to advance a field of research, collect data on understudied populations and accelerate the implementation of research findings into practice. In each network, collaborative learning was an explicit part of the mission. In most cases, networks were designed to bring individuals together from a range of disciplines to answer similar research questions, pool study subjects or data sets, and share and disseminate methods and knowledge among network members and with the larger community. Inception varied across networks; for example, the Collaborative Care Research Network was born out of a Collaborative Care Conference, during which the founders identified both the need for an evidence-base on mental health–primary care integration and an organizational mechanism to support it. Two networks (PECARN and MCCRN) were established through the American Recovery and Reinvestment Act to advance patient-centred outcomes research, one with a focus on infrastructure development and another on collaboration among community-based providers and researchers. One network aimed to “create a community of people who engage in both research and clinical practice, in an attempt to accelerate research findings into the care setting.” Similarly, the Medicaid Medical Directors Network originated to increase knowledge sharing among state officials, to decrease independent struggles with common issues and to implement multistate measurement and quality improvement projects. Finally, training young researchers was cited as a motivation, helping them develop their careers and encouraging them to focus on important research topics.

Funding

Six networks were funded by federal healthcare agencies, two with grants from private organizations and one from multiple-funding sources (Table 1). Of the networks that received federal funding, the Cancer Prevention and Control Research Network and HMO Research Network (HMORN) were jointly funded by multiple agencies. As previously mentioned, two networks were funded through the American Recovery and Reinvestment Act. Funding varied substantially, ranging from $350,000 to $1 million per year. The length of initial funding varied but as of 2013, half the networks had been functioning for more than 10 years. Respondents stated that network duration and sustainability were primarily dependent on available funding and on the level of effort and interest among members. Some networks suffered budget cuts when the financial climate worsened. These cuts reduced the ability to convene or support travel to in-person meetings, and maintain network websites and data registries. When asked if funding was adequate to achieve intended network goals, all respondents but one said that funding was insufficient and that obtaining funding was always a challenge. Respondents noted that some members sought additional funding for individual projects developed within the network. One respondent said that a few established investigators served as magnets for research network funding. Thus, while their network intended to help less-experienced investigators become project leaders, funders tended to award grants to senior investigators, hindering the professional development of younger researchers.

Membership

Network membership comprised researchers in geographically diverse organizations and in academic medicine and research institutions, universities, hospitals and health centres. Membership size ranged from 10 member organizations to ~78, and from 13 individual participants to more than 400. Researchers and their organizations typically applied to serve as a research network “node” (site) and/or a coordinating centre through a funding agency's request for applications. In one instance, a network director invited individuals to participate in her proposed network based on their disciplines and their level of interest and engagement in topics outside their research specialty. She especially sought early career researchers and individuals with expertise in interdisciplinary research. The MCCRN included researchers who received individual AHRQ grants on MCC who were later brought together by the agency to participate in the network.

Research network participants represented various disciplines and diverse content expertise in the areas of healthcare, research, economics and policy who shared common interests. In the words of one respondent:

“Part of what makes our centre work is that there are common themes … even though individuals have different research projects and strengths, they are all associated with healthcare-associated infections … having some thematic consistency and common interest makes a big difference.”

According to several respondents, diverse knowledge and skills were valuable to the network and essential for cross-disciplinary collaboration.

Several respondents reported that their networks experienced yearly member turnover. For example, one network experienced a 50% turnover for the network as a whole, and a 40% turnover among principal investigators. Reasons for turnover included job changes and retirement. Also, one respondent noted that network participation is difficult for sites without academic infrastructure because they lack ongoing research support.

Network governance

In all but two networks, a steering/advisory committee oversaw implementation and day-today management (Table 2). Several networks did not initially have governance structures, but created steering committees as missions and membership evolved. One of these networks had minimal structure for the first three years as network members primarily engaged in sharing knowledge. As the network's vision and research matured, members wanted more structure and created a formal steering committee and governing bylaws. One respondent explained:

“Initially, it was difficult to keep the network together … without having a group leader, which was why a steering committee was developed.”

Steering committee roles and responsibilities were fairly consistent across networks: typically, the steering committee developed agendas and facilitated monthly or quarterly meetings, monitored research and collaborative activities and managed key decisions (e.g., future research projects, authorship criteria for publication). Steering committees usually comprised a chair and vice chair, a funding agency representative and a few representatives from the research sites or coordinating centre. Generally, a new steering committee was elected every few years. In addition to steering committees, five networks maintained subcommittees or workgroups for executing work effectively and efficiently. More than half of the research networks were supported by a coordinating centre that provided administrative and technical support to network participants (e.g., data assistance, organization of member collaborative activities, guidance on dissemination of research, products and tools).

Methods of collaboration and dissemination

Respondents reported that the best method for promoting collaborative learning among research network members was in-person meetings. Research networks regularly brought members together at least once a year, and most networks held two to four annual face-to-face meetings. Three networks, for convenience to members, coordinated in-person meetings with national conferences. One network exclusively used this method. Two others held both dedicated network meetings and meetings coordinated with national conferences. Four networks held additional in-person meetings throughout the year for steering committees or special interest/working groups.

Half the networks used teleconferencing. Conference call frequency varied considerably: one network held weekly member calls, while another held quarterly, 6-hour conference calls. The other two networks held periodic calls with subgroups such as the steering committee and working groups. All networks used a website and eight also used a LISTSERV to exchange information and foster collaboration.

Facilitators and Barriers to Collaboration

Below we describe the facilitators and barriers to collaboration identified by respondents.

Facilitators of collaboration

As mentioned above, one of the main facilitators of effective collaboration was in-person time with members, funders and key stakeholders. Building trust between members arose as a key theme throughout our interviews, with respondents saying that trust served as a crucial facilitator for overcoming differences in research and disciplinary approaches. One respondent emphasized the importance of bringing members together to enhance and maintain trusting relationships:

“It's essential to create free time for individuals to get to know one another. At the beginning of the in-person meetings [we] would always have a dinner meeting. These dinner meetings helped facilitate trust and a common connection between members. Having this trust made it possible for everyone to work together more effectively as a group. This process would not have been as successful via web conference or through email.”

A few respondents mentioned the importance of maintaining a “shared vision” or a common set of agreed-upon goals and objectives. Traditionally, researchers are trained to pursue their own projects independently. Given the non-collaborative tradition of research, as well as the challenge in managing multiple investigators and competing ideas, it is essential for networks to reach a consensus on the mission and focus of the work, identify strategies to integrate diverse interests and find common ground among network members. With the MCCRN, we found that surfacing methodological and substantive issues of mutual interest to participants were essential in motivating investigators to collaborate. Common problems and research challenges brought network participants together to problem-solve and consult with each other on solutions. Most respondents spoke about the difficulty and time commitment of conducting collaborative research and network involvement:

“Many sites wanted to do the work but didn't have the patience it took to engage with the network. Because the workload is about twice as much as the investigators are actually paid to do, everyone participating really has to be passionate about the work that they are doing.”

In addition, respondents thought it important that network members believed in collaboration as the best method to answer their research questions.

“Being a part of a network requires tolerance for ambiguity, a certain humility about your own discipline, an appreciation and a passion for a particular problem, and the realization that you can only solve that problem if you work together.”

Furthermore, all respondents spoke of the importance of establishing strong leadership and a culture of transparency for developing common goals and equitable participation:

“Strong leadership was essential in this circumstance. Everyone went into the network with certain assumptions. The group needed guidelines – and a system to be accountable to. In the beginning, there was a constant pushing and pulling.”

Barriers and challenges to collaboration

Funding and financial sustainability were identified as the greatest barriers to research collaboration. Collaboration takes time and thought, and many researchers are responsible for attracting funding that pays for their own salaries. Collaborative research across multiple institutions is expensive to organize and implement, and funding for this type of work is limited. Established research networks reported that their funding declined over time. In addition, respondents discussed the limitations of short funding periods. Building trust, a collaborative spirit, infrastructure and systems took years. Thus, short funding periods were a serious barrier, especially for research networks with ambitious goals. As an example, investigators in the MCCRN took almost two years to coalesce as a group and identify areas for collaboration. The network was funded for three years without a mechanism for extending funding. This was not long enough for participants to build sufficient momentum around collaborative efforts. When asked how many years are needed to develop an effective research network, respondents recommended a minimum of 5 to 6 years, with 7 to 10 as the ideal:

“You need to have a long-term investment because it's inefficient in the short term. Researchers need to know if the network is going to be supported for long enough to get the payoff.”

To offset the challenges of sustainable funding, one network created a supplemental funding pool, which allowed the investigators to quickly apply for and obtain support for add-on collaborative or multi-member work. Although flexible funding facilitated collaborative work in this network, the mechanism was not used by the other networks. Several respondents cited changes in travel regulations for federally funded work as a specific barrier that hindered or even prevented in-person network meetings.

Furthermore, respondents pointed to busy schedules and competing demands for time as a barrier to collaborative research. Many network investigators were practising physicians or professors balancing research network activities with other institutional demands and requirements. For example, one respondent stated that while some sites wanted to “do the work,” they did not have the time or patience to engage in all network activities. Finally, as previously described, high membership turnover was common. This too added to the difficulty of establishing trust and maintaining collaboration.

Discussion

Our analysis of a set of US health research networks shows variation in their governance, focus, membership and funding. In all networks, however, the work of building collaborative structures – establishing a culture of trust, compromise and sharing – took time and thought. Each research network in the study came up with its own mechanisms and ways of creating infrastructure, but it expressed a common recognition of the need to carefully craft processes and techniques that fostered learning among the participants. All network representatives mentioned the importance of holding face-to-face meetings, finding time for regular communication and interaction, and maintaining ongoing network structures and processes in the midst of competing demands.

In addition to structural facilitators such as a meeting organizer, our results highlight the essential role of actively building trust and relationships for establishing collaborative learning processes. The importance of effective communication in developing trust and strengthening relationships is a common theme in studies of research networks (Williams et al. 2008). Effective networks do not simply throw people and ideas together, but intentionally promote and build on the dynamic and emergent relations between members. As described by Vangen and Huxham (2003), trust building is a cyclical process. Positive outcomes form the basis for trust development. With each consecutive positive outcome, trust builds incrementally, over time, in a virtuous cycle (Vangen and Huxam 2003). Scott and Hofmeyer (2007) stress the centrality of network theory and social capital in determining network outcomes. Members themselves shape the identity, function and products of their networks through their individual interests, and through shared properties including common goals; trust; compatibility of language, culture and methods; transparency; rewards (e.g., building professional relationships and reputations); and level of collaboration readiness and skill (Scott and Hofmeyer 2007; Stokols et al. 2008; Williams et al. 2008).

Despite the central role of learning network functionality, assessing the specific components that make research networks effective in promoting trust between members and achieving research goals is challenging. As our findings illustrate, one difficulty in evaluating the success of learning network functionality in networks is that objectives vary over time, especially as funding changes and individual research efforts move to completion. Techniques for surfacing the common substantive interests of members need to be documented and tested, as they may predict participation and engagement in collaborative networks. A starting point for these techniques could be readiness assessments from areas such as health innovation improvements (Weiner et al. 2008) and community–academic partnerships (Goytia et al. 2013).

Another limitation in determining the factors that contribute to research network effectiveness is the lack of established outcome measures. The most common metric for evaluating outputs from research networks is publications. However, research collaboration will not always lead to a publication and other valuable – but difficult to measure – results from learning collaborations include intellectual and social capital, personal satisfaction, fun and pleasure, quality of results, prestige, training, communication, implementation, sustainability (Bleeker et al. 2010; Bukvova 2010; Fenton et al. 2007; Kreger et al. 2007), and training and career development for junior research staff.

Our findings may be useful for others forming and evaluating research networks. While our analysis is based on a small sample of research networks in one country, our interview results are consistent with previous research (e.g., Pless et al. 2010; Williams et al. 2008) in emphasizing the importance of collaborative learning in research networks, and mechanisms for fostering it. Further, the findings resonated with our applied experience facilitating trust-building and information-sharing through the MCCRN Technical Assistance Center. While we used asynchronous collaboration methods such as a shared website and newsletter, we found in-person meetings especially useful. In these, we used specific learning community techniques to foster group conversations and learning. Meetings were structured according to interests expressed by grantees. We found that collaborative activities increased over time, but interest and engagement in collaborative research varied across participants.

Given the technological advances and the financial costs of in-person learning communities, more networks are turning to virtual collaboration to meet their organizational goals and address geographic dispersion among network members. Over the past 10 years, business and organizational development practitioners have assessed the management and performance of virtual networks and teams. While research reveals that virtual teams outperform co-located groups, such teams are successful only when managers implement task-related processes that capitalize on specialized knowledge and expertise of virtual groups and promote cultures that prioritize diversity (Siebdrat et al. 2009). Our own experience and the reports of representatives from other networks suggest that virtual collaboration is unlikely to be successful until familiarity and trust are established through in-person experiences.

Based on our analysis, we offer several suggestions for the development of research networks. First, the time and resources to facilitate collaboration cannot be underestimated and underfunded. According to our respondents, infrastructure needs do not decline over time, but rather change over the life of the network. In turn, readiness assessment may be a valuable tool for developing network structure and activities to meet members' needs. As noted above, few evaluations of health services research networks have been published and metrics for assessing network success are nascent. Ultimately, the benefits of collaborative networks may need 5 or 10 years to be realized, and evaluations must take this into account. Developing process and outcome evaluation metrics would greatly advance the design of research learning networks and show their impact. Last, continued sharing of research network experiences and success stories can help current and developing collaborative endeavours refine mechanisms to meet their objectives.