The Council of Academic Hospitals of Ontario (CAHO) Adopting Research to Improve Care (ARTIC) Program: Reach, Sustainability, Spread and Lessons Learned from an Implementation Funding Model.

Despite evidence on what works in healthcare, there is a significant gap in the time it takes to bring research into practice. The Council of Academic Hospitals of Ontario's Adopting Research to Improve Care program addresses this research-to-practice gap by incorporating the following components into its funding program: strategic selection of evidence for implementation, education and training for implementation, implementation supports, executive champions and governance, and evaluation. Funded projects have been sustained (76% reported full sustainability) and spread to over 200 new sites. Lessons learned include the following: assess readiness, develop tailored implementation materials, consider characteristics of implementation supports, protect champion time and consider evaluation feasibility.

Background

Despite a growing body of evidence on what works in healthcare, there is a significant delay in the time it takes to bring research into practice (Brownson et al. 2006); without infrastructure in place to support research implementation, it may take up to 17 years for research to be implemented in practice (Balas and Boren 2000). Because health systems are not maximizing research uptake, there are large inefficiencies in these systems that result in reduced quantity and quality of life (Davis et al. 2003; Kennedy et al. 2004; Madon et al. 2007; McGlynn et al. 2003; Pimlott et al. 2003; Shah et al. 2004). For example, an estimated $240 billion is invested annually in health and biomedical research, but approximately 85% does not result in evidence implemented into practice (Chalmers et al. 2014); even when evidence is implemented, it often results in little to no meaningful practice change (Davidoff et al. 2015). Knowledge translation (KT) science is the field of study that was developed to address this research-to-practice gap. KT is the “dynamic and iterative process (including the synthesis, dissemination, exchange and ethically sound application of knowledge) to improve the health of Canadians, provide more effective health services and products and strengthen the healthcare system. This process takes place within a complex system of interactions between researchers and knowledge users, which may vary in intensity, complexity and level of engagement depending on the nature of the research and the findings and the needs of the particular knowledge user” (Graham 2000).

The amount of funding set aside for KT is a fraction of the money dedicated to research; it is, therefore, sometimes considered “decimal dust” despite KT's integral role in changing population-level outcomes (Kerner 2006; Tetroe et al. 2008). In an environmental scan of KT funding opportunities worldwide (24% from Canada), approximately 20% of the KT funding supported implementation activities (as opposed to dissemination or synthesis activities; Timmings et al. 2015). In recent years, there has been a push to fund more KT activities, including efforts from the Canadian Institutes of Health Research (CIHR), the National Institute of Health Research in the UK, the Health Research Council of New Zealand and the Michael Smith Foundation for Health Research in British Columbia (Holmes et al. 2012, 2014; Timmings et al. 2015). For example, CIHR funds several KT initiatives, which have been found to perform well against traditional funding mechanisms in terms of producing KT and academic outputs, providing training opportunities, improving the health of Canadians, strengthening the healthcare system and creating more effective health services and products (Graham et al. 2014; McLean and Tucker 2013). Simply funding more KT projects, however, is not enough to move research into practice, and there is a move to incorporate more implementation supports from funders, as “greater involvement of funding agencies in all forms of KT … is essential for the maintenance of the health research enterprise in the face of many competing and compelling demands on the tax base” (Kitson and Bisby 2008). As the number of implementation funding opportunities increases, there is a need to understand the impact of these funding mechanisms, particularly those using less traditional approaches to funding that incorporate implementation supports (Tetroe et al. 2008). Based on the Council of Academic Hospitals of Ontario (CAHO) Adopting Research to Improve Care (ARTIC) program's experiences through three rounds of funding and an evaluation, ARTIC has identified several lessons learned that could inform future funding opportunities/ agencies interested in supporting the uptake of evidence-based practices using effective implementation strategies. The aim of this paper is to describe an implementation funding model; evaluate reach, sustainability and spread of this model; and share lessons learned.

Implementation Funding Initiative

The CAHO ARTIC program is a model for accelerating and supporting the implementation of research evidence into practice across the healthcare system to drive improvements in the quality of care. CAHO is the non-profit association of Ontario's 24 research hospitals and provides a focal point for strategic initiatives for these hospitals. CAHO developed ARTIC to accelerate the adoption of research evidence within hospital settings; this funding model was CAHO's first attempt to implement and evaluate an implementation funding mechanism. It aims to transform the healthcare system by using evidence to drive quality and therefore make the best use of resources, enabling a culture of continuous quality improvement and creating a jurisdiction where implementation strategies to support clinical interventions are sustained and spread across the province.

The ARTIC model is based on the Knowledge-to-Action (KTA) model for moving research into practice, a model that is based on a review of over 30 theories of planned action (Graham et al. 2006). The funding mechanism of change is drawn from work by the CIHR KT funding program and associated logic model, which hypothesizes that funding KT projects will produce meaningful researcher and knowledge user partnerships, facilitate the dissemination and application of knowledge and advance the science of KT, which, in the long-term, will improve services to and the health of Canadians (McLean et al. 2012). The partners (i.e., funder, implementers and supports) and their relationships to each other are presented in Figure 1, an adapted version of the Interactive Systems Framework (ISF; Wandersman et al. 2008). Driven by the KTA and CIHR's KT funding model, the implementation infrastructure includes five implementation enablers: (1) strategic selection of evidence; (2) implementation supports (e.g., coaching, technical expertise and communities of practice); (3) education and training for implementation; (4) executive champions and governance; and (5) evaluation.

Figure 1.

ARTIC program and project structure

As a funder, ARTIC provides $800,000 to $1.7 million for the implementation of interventions across multiple sites over a two-year period. However, an integral component of this structure is ARTIC's role not only as a funder but also as an implementation support, knowledge broker and partner, which is different from the roles of most funding agencies. As a partner, ARTIC is involved in engaging senior leadership and fostering the idea of working and thinking as a community of academic hospitals rather than as individual institutions. This partnership model aligns with the integrated KT approach, in which there is a partnership between researchers and knowledge users who actively contribute to the research agenda and implementation activities and make research findings directly relevant to the knowledge user (Graham et al. 2014). ARTIC enhances the typical integrated KT approach by also incorporating senior leadership who can serve as a facilitator of project implementation (Aarons and Sommerfeld 2012) and central ARTIC Program staff who are accountable to support the Communities of Practice and facilitate implementation on a wide range of projects.

Strategic selection of evidence for implementation strategy and clinical intervention

The program uses a systematic process to identify and fund KT implementation projects. Projects comprise clinical interventions (i.e., evidence-based practices) delivered using effective implementation strategies. Projects are selected based on key criteria and annually selected themes, which align with government (e.g., Ontario Ministry of Health and Long-Term Care) and organizational (e.g., CAHO member) priorities. Table 1 presents descriptive information for the six funded projects. The project selection process includes three stages: (1) letter of intent (LOI) stage, (2) full submission stage and (3) readiness assessment stage. The ARTIC Task Force reviews the LOIs and then requests full submissions for five to eight proposals that are based on robust research evidence, have potential for high system impact, demonstrate implementation feasibility and present an evaluation plan to assess outcomes. They then select the top two to four submissions to proceed to the readiness assessment stage. The readiness assessment stage is the most unique feature of the project selection process. Organizational readiness for change is “the extent to which organizational members are both psychologically and behaviorally prepared to implement change” (Chaudoir et al. 2013; Gagnon et al. 2011; Weiner et al. 2008). Typically, readiness is rarely assessed before implementation (Weiner et al. 2008), yet measuring readiness is associated with better outcomes (Amatayakul 2005; Jones et al. 2005; Kotter 1996). ARTIC requires each project to have a lead project team to coordinate implementation delivery across the hospital; in addition, each participating hospital delivers the implementation strategies through a local project team. ARTIC works with the lead team to develop a readiness assessment that includes the target population for intervention, resource requirements and key stakeholders to be engaged in the implementation process. CAHO collects data from all sites and creates a readiness report. This report is assessed by the ARTIC program Task Force to determine implementation feasibility and select the funded project. Each lead team is provided with the report to assist in developing their implementation roadmap.

Table 1.

Description of CAHO ARTIC funded projects

Projects	Funding year	Number of participating sites	Spread* and scale
Canadian C-Spine Rule (CCR) to assess the clinical impact of CCR on reducing emergency department wait times and increasing efficiencies by maximizing the use of inter-professional resources	2011–2012	9	Relevant to emergency department only; no internal spread
HandyAudit to increase the efficiency of hand hygiene compliance reporting in hospitals	2011–2013	16	Delivered hospital-wide; no internal spread; external spread to over 170 hospitals
Antimicrobial Stewardship Program (ASP) in Intensive Care Units (ICUs) to optimize antimicrobial use in ICUs where critically ill patients are the sickest and most vulnerable	2012–2014	12	In progress; internal hospital spread in 6 of 12 sites; implementing a hub and spoke model to spread to community hospitals
Mobilization of Vulnerable Elders in Ontario (MOVE ON) to promote early mobilization and prevent functional decline in older patients admitted to hospital	2012–2014	14	Internal hospital spread in 10 of 14 sites; external spread to over 28 hospitals
Implementing an Enhanced Recovery After Surgery Guideline to Optimize Outcomes following Colorectal Surgery (iERAS) to implement a range of interventions aimed at improving patient outcomes and reducing hospital stay after surgery	2013–2015	15	Data not yet available
Implementing the Transitional Discharge Model (TDM) supports the successful transition from the hospital to the community for people diagnosed with a mental illness	2013–2015	9	Data not yet available

Internal spread refers to implementation within the same hospital beyond the originally funded units; external spread refers to implementation in other hospitals not funded through ARTIC.

Implementation supports

The funder–fundee partnership is different from a typical grant, in that ARTIC staff are involved in facilitating the implementation strategy: along with the lead project team and executive champions, an ARTIC program manager is one of three key stakeholders who make up the infrastructure in the implementation support system (illustrated in Figure 1). These infrastructure stakeholders build both general capacity for the program and capacity within each participating organization by enhancing leadership skills, improving knowledge of implementation science and developing clinician champions to lead evidence-based change. Together, these three stakeholders provide implementation support in the form of coaching, technical expertise and communities of practice. Lead project teams create timelines, submit progress reports and prepare outcome presentations; the program manager then offers feedback on meeting objectives and timelines. The program manager identifies areas of alignment and integration within ARTIC projects and with other government initiatives to support a coordinated approach to improving quality implementation and maximize system, organizational and patient impact and outcomes. This approach allows the ARTIC program manager to understand how the lead project teams and local project teams are functioning.

Lead project teams support the local project teams through education and training on implementation. The lead project team provides tools, resources, supports and coaching to local project teams. The lead project team also develops the implementation roadmap to guide participating hospitals through the implementation and evaluation process. The lead project team helps ensure that as projects are tailored to the local context, hospitals maintain fidelity to the goals of implementation. Each project develops a community of practice (CoP), which facilitates communication and peer support (Wenger et al. 2002) across projects.

Education and training for implementation

Scaling up implementation requires education and training, including coaching and written materials, to facilitate the adoption of evidence within and across hospitals. Education materials are designed to produce consistency across the system and build general implementation capacity and clinical intervention capacity (Wandersman et al. 2008). General implementation training is intended to teach implementers (i.e., hospital end users) about best practices in implementation. Clinical intervention education focuses on the content of the project (e.g., changes in clinical practice). The lead project teams provide tools, training, resources and coaching and host a launch event in collaboration with ARTIC.

Executive champions and governance

The governance structure, illustrated in Table 2, ensures that the delivery of the implementation strategy is properly resourced and supported. The three-tiered governance structure includes the CAHO ARTIC program Task Force (ARTIC Task Force), the CAHO Practice and Education Committee (P&E) and the CAHO Council.

Table 2.

CAHO ARTIC governance structure

	Responsibilities	Members	Meeting frequency	Reports to
CAHO ARTIC Program Task Force	Provides operational oversight of program and responsible for reviewing and assessing proposals submitted for funding consideration	Representation from diverse groups (research, clinical practice, KT and system partners), including the P&E committee, CAHO's research committee, the MOHLTC and HQO	Every two months	P&E Committee
CAHO P&E Committee	Provides strategic guidance and oversees program implementation, including reflecting on the CAHO ARTIC Program Task Force recommendationsP&E committee members are executive champions for the projects in their hospitals, providing senior leadership support for implementation projects to obtain staff engagement, create a culture receptive to change, prioritize the initiative, spread the initiative, allocate resources and resolve challenges	Clinical practice leaders from all CAHO hospitals (e.g., chief nursing executives, vice presidents of quality, vice presidents of medical affairs)	Every two months	CAHO Council
CAHO Council	Provides strategic oversight to and ultimate accountability for the CAHO ARTIC Program by being accountable for and approving funding decisions and developing the program's strategic direction	CEOs from each of the CAHO member hospitals	Every two months	N/A

Evaluation

Embedded in the ARTIC's structure is an evaluation at two levels: program and project (Figure 1). An interim program evaluation including interviews with 43 senior leaders and relevant stakeholders representing 17 of the 25 CAHO organizations was conducted following the first round of funding. The goal of the interim evaluation was to understand the organizational decision-making practices regarding participation in ARTIC and to recommend ways to improve the selection of future projects. Results of this evaluation provide support for the existing structure and enablers of ARTIC. A second evaluation was conducted to examine reach, sustainability and spread of ARTIC.

At a project level, each team develops and executes a monitoring and evaluation component. Regular performance monitoring and evaluation is designed to ensure greater fidelity to the implementation strategy and clinical intervention and to provide opportunities for continuous quality and process improvement.

Program Reach, Sustainability and Spread

An evaluation was conducted to examine ARTIC's impact on reach, sustainability and spread. Sustainability is defined as continued implementation of the initiative (Shediac-Rizkallah and Bone 1998), whereas spread is the horizontal diffusion or active dissemination of best practices or programs and implementing these across settings (Institute for Healthcare Improvement 2008). Data for the evaluation were collected through interviews, with 18 program stakeholders (e.g., executive champions and members of governance), interviews with all four lead project teams, surveys with 27 of the 35 participating sites across all projects (79%), interviews with 12 local teams (hospital site leads) and a review of documents from the lead project teams and the ARTIC program office.

Program reach

For the six projects, ARTIC funded $7.6 million in direct costs, which was matched by $12.2 million in in-kind contributions for a total of $19.8 million invested into delivering evidence into practice and a cost to in-kind ratio of 1:1.6. Through ARTIC, over 25 researchers/program developers learned about implementation, over 220 site-level champions connected with researchers and gained implementation experience, over 1,500 staff or volunteers worked on the projects and quality of care was improved for over 18,000 patients (Cathexis Consulting 2013, ARTIC Phase I evaluation report).

Sustainability

Sustainability and spread were assessed in the four projects that had completed the funding phase. Using self-report surveys, sites were asked whether they were still using the implementation strategy to support the clinical intervention 1.5 or 2.5 years after initial implementation (depending on when they were first funded). Of the 50 sites, 46 provided data and 76% (35 of 46 sites) reported fully sustaining the implementation strategy, 13% (6 of 46 sites) reported partially sustaining the implementation strategy and 11% (5 of 46 sites) reported not sustaining the implementation strategy. Given the challenges in sustaining implementation efforts and the typically low rates of sustainability (Stirman et al. 2012), these findings are promising and indicate that the ARTIC model supports a high level of sustainment.

Spread

Program funding spread four projects to 50 sites (Table 1). Two of the four projects subsequently spread internally within the original hospitals. In addition to spreading internally, two of the projects were spread to new sites: one project developed into a commercial venture and spread to over 170 hospitals; the other spread to over 28 additional sites through new funding and hospital-driven initiatives. Patient outcome data from these additional sites are being published by individual projects and therefore cannot be reported here.

Lessons Learned

Based on the program's experiences through three rounds of funding and an evaluation, CAHO ARTIC has identified several lessons learned linked to each of the five implementation enablers/implementation infrastructure.

Strategic selection of evidence for the implementation strategy and clinical intervention

Prior to including the readiness assessment in project selection, hospitals reported that expectations and required resources were not clear before beginning the project, and project teams expressed challenges related to hospitals not committing necessary resources. The readiness assessment was included to make potential participating hospitals aware of the resources required to successfully implement the project and to inform CAHO of each hospital's readiness to deliver the implementation strategy. An external evaluation found the readiness assessment to be brief, focused and valuable for identifying implementation resources prior to committing to the project.

Education and training

Education and training ran more smoothly when hospital teams were provided with ready-to-use implementation materials that required minimal time and effort to adapt to their context. Developing materials de novo was a burden on local hospital teams, which delayed implementation, resulted in duplication and/or reduced the quality of materials. Challenges to education and training included the lack of funding to backfill participants' time in training, lack of buy-in from frontline staff and difficulty identifying the appropriate people to train.

There were several lessons learned with regards to implementation supports, including understanding the optimal characteristics of project teams, building key implementation enablers into the program and the project design, identifying ways to develop and maintain CoPs and supporting use of KT experts by project teams. Ideally, lead project teams should have the expertise in the development and delivery of evidence-based implementation strategies and be perceived as credible, approachable and flexible. The team should include an appropriate mix of professions/disciplines (e.g., clinical areas, KT, evaluation and project management).

The roles and responsibilities of each group need to be clearly defined at the outset and communicated to all relevant stakeholders. While CoPs were available across all projects, their use was not optimized. Future CoPs could be provided guidance on the purpose, structure and types of CoP activities (e.g., ways to engage participating site leads via teleconference or use of project websites to sustain CoPs).

The executive champion in each hospital should be in a position to prioritize the project administratively (e.g., protecting staff time) and champion the project by encouraging engagement and participation and boosting staff morale.

Project evaluation

The project evaluation was resource intensive in terms of data collection and feedback. Therefore, the monitoring and evaluation requirements of future projects should consider the feasibility of data collection and incorporate an efficient, timely feedback mechanism.

Limitations

Several limitations should be noted. The evaluation and lessons learned did not consistently link the implementation projects with improved patient outcomes, and there is no comparison group that did not receive ARTIC funding. Each funded project improved patient outcomes, but because of the program evaluation research design, we are not able to determine whether the funding model successfully improved outcomes. In addition, the measures of sustainability and spread are self-reported; an organization reporting that they continue to implement the initiative may not be representative of actual behaviour.

Conclusion

With an increased focus on implementing research evidence into practice in the healthcare sector, organizations that fund implementation work have an opportunity to play an integral role in funding and supporting research uptake. The CAHO ARTIC program attempts to fill the KT funding gap and transform the healthcare system by supporting the use of evidence to drive quality, enabling a culture of continuous quality improvement and sustaining and spreading the implementation of evidence across Ontario. To date, 76% of the sites have sustained implementation for at least 1.5-years' post-implementation and the project has spread to over 200 new sites.