Unlocking the "black box" of practice improvement strategies to implement surgical safety checklists: a process evaluation.

BACKGROUND: Compliance with surgical safety checklists (SSCs) has been associated with improvements in clinical processes such as antibiotic use, correct site marking, and overall safety processes. Yet, proper execution has been difficult to achieve.
OBJECTIVES: The objective of this study was to undertake a process evaluation of four knowledge translation (KT) strategies used to implement the Pass the Baton (PTB) intervention which was designed to improve utilization of the SSC.
METHODS: As part of the process evaluation, a logic model was generated to explain which KT strategies worked well (or less well) in the operating rooms of a tertiary referral hospital in Queensland, Australia. The KT strategies implemented included change champions/opinion leaders, education, audit and feedback, and reminders. In evaluating the implementation of these strategies, this study considered context, intervention and underpinning assumptions, implementation, and mechanism of impact. Observational and interview data were collected to assess implementation of the KT strategies relative to fidelity, feasibility, and acceptability.
RESULTS: Findings from 35 structured observations and 15 interviews with 96 intervention participants suggest that all of the KT strategies were consistently implemented. Of the 220 staff working in the department, that is, nurses, anesthetists, and surgeons, 160 (72.7%) knew about the PTB strategies. Qualitative analysis revealed that implementation was generally feasible and acceptable. A barrier to feasibility was physician engagement. An impediment to acceptability was participants' skepticism about the ability of the KT strategies to effect behavioral change.
CONCLUSION: Overall, results of this evaluation suggest that success of implementation was moderate. Given the probable impact of contextual factors, that is, team culture and the characteristics of participants, the KT strategies may need modification prior to widespread implementation.

Background

Each year, an estimated 234 million surgeries are performed globally.1 Of these, approximately seven million deaths occur,2 half of which are believed to be preventable.3,4 One strategy that has been advocated by patient safety experts worldwide to reduce patient morbidity and mortality is the implementation of surgical safety checklists (SSCs).1,5 The main intent of checklists is to ensure that those “must do” clinical tasks are not forgotten and can be embedded into routine workflow patterns. In 2008, the Safe Surgery Saves Lives Group at the World Health Organization (WHO) launched a perioperative SSC1 to minimize the risk of wrong site surgery. Following development, the WHO SSC was tested in eight diverse settings, including developing countries around the world.6 When the WHO SSC was used, there was a reduction of ~50% in surgical mortality, from 1.4% to 0.8% and a near 40% reduction in inpatient complications, from 11% to 7%.6 Since the publication of that landmark study, several meta-analyses suggest associations between checklist use and reductions in patient mortality, wound infection, pneumonia, blood loss, and postoperative complications.7–9 Notwithstanding, improvements have been noted more broadly than just morbidity and mortality: for example, some researchers have found improvements in cost savings10 whereas others showed using an SSCenhanced communication, teamwork, and safety climate.11–14 Taken together, these results suggest that safety improvements may be attributable to a change in work culture and improved team climate, although the mechanism of action to explain these changes remains less clear.

The challenges of implementation and sustained compliance have become apparent following widespread dissemination of the SSC across diverse countries and health care systems. Notably, implementation of checklists does not always lead to compliance. Full compliance to checklists is often difficult to achieve and largely depends on many contextual factors. Compliance differs between hospitals and staff members and between specific sections and individual items of the checklist.5–7,15–19 Variation in compliance has been noted: Borchard et al7 summarized 15 studies evaluating checklist compliance and found that overall compliance rates ranged from 12% to 100%. Other prospective studies based on administrative data have indicated that, while items were documented as being checked, clinical behaviors underlying those checks were not observed.18,20–22 While researchers have evaluated outcomes relative to checklist compliance,5–7,15–19 there are currently no studies in this field that have evaluated strategies that support its implementation. As the main intent of using SSCs is to improve interdisciplinary communication among members of the surgical team, the implementation process should be aligned with this objective. Clearly, designing implementation interventions that promote behavioral engagement is central for such a checklist to fulfill its potential (or conversely avoid negative unintended impacts on safety). Thus, continuous evaluation of an implementation intervention should be part of its implementation and not an afterthought.

Complex interventions

Complex interventions are commonly described as interventions that contain several interactive components.23 Components include techniques to facilitate behavior change that constitute the active ingredients of the interventions and the strategies used to deliver them.24 Dimensions of complexity are characterized by the range of possible outcomes or their variability in populations and across contexts. Saliently, complexity is not characterized by the number of components within the intervention itself, rather it encompasses the interactions between components, the difficulty of behaviors by those delivering and receiving the intervention, and the degree of flexibility in tailoring that is allowed.25 The lines between simple and complex interventions are blurred as complexity runs along a continuum; so in reality, few interventions are considered simple.25 Consequently, the development and evaluation of complex interventions must be founded on a sound theoretical understanding to isolate the mechanisms that guide behavior and test the extent to which the “active ingredients” of the intervention magnify or diminish behavioral engagement.25,26 A lack of impact may reflect implementation failure rather than effectiveness failure. In addition, the context in which the complex intervention is implemented may render strict fidelity inappropriate because the intervention may work better if adapted to the nuances of the context.25,27 An important consideration while evaluating practical effectiveness centers on understanding the full range of an intervention’s effects and how these vary between participants and sites, over time, and the causes of those variations.23,25

Materials and methods

Objective

The need to better understand how, when, and why particular strategies work prompted unlocking the “black box” in terms of checklist implementation. To this end, the objective was to undertake a process evaluation of the implementation of four knowledge translation (KT) strategies designed to support a behavior change intervention intended to promote better utilization of the WHO SSC. As part of this evaluation, a tentative logic model was generated to explain which KT strategies used to support the Pass the Baton (PTB) intervention worked well (or less well) in the operating rooms of a large tertiary referral facility in Queensland, Australia. This paper focuses on process outcomes, guided by the following questions:

To what extent were the KT strategies used to support the PTB intervention successfully and consistently implemented (ie, aspects of fidelity, dose, and coverage)?

To what extent were the KT strategies used to support the PTB intervention feasible and acceptable from participants’ perspectives (ie, what components were useful, why/why not)?

What are the processes underlying the change (or lack of change) (ie, evaluation of a logic model)?

Design

In this process evaluation, an observational design and triangulated data were used based on audit, fieldwork observations, and interviews. Process evaluations (as distinct from outcome evaluations) are important because they can clarify to what extent a complex intervention was delivered in a uniform way, whether the target population actually received the planned activities, what factors inhibited or promoted effectiveness, and what the participants’ actual experiences with the executed strategy were.25,28 Process evaluations also provide information essential to understanding and validating theory-informed practice improvement strategies.25 Analysis of process and outcome data allows evaluations to explore associations between strategy delivery and receipt, and outcomes on effectiveness.29 In this process evaluation, a logic model was developed to help explain causal mechanisms of the implementation strategies that were used to support PTB.

Logic model development

Arising from process evaluation, logic models are based on the need to explore and make explicit links between inputs, activities, actions, and outcomes and thus offer a conceptual model about how the intervention might work.28,29 Logic models may be used to help identify under what environmental conditions the implementation would work best and/or to evaluate an existing strategy as it is being implemented.25 The purpose of this process evaluation was to focus on the short-term outcomes and mechanisms of action that gave rise to those outcomes. In the current study, the logic model presented in Figure 1 framed the evaluation questions, focusing on the KT strategies and their mechanism of action.

Figure 1

Checklist implementation logic model.

Abbreviation: PTB, Pass the Baton.

Intervention context

The setting for this process evaluation was a 750 bed tertiary hospital with 22 commissioned operating rooms in Queensland, Australia. The aim of the PTB intervention was to improve checklist use and therefore the communication of clinical information as a team.

The theoretical underpinnings of the PTB intervention were based on two sources: 1) literature on checklist implementation,27 and 2) data derived from an analysis of the barriers and enablers of checklist implementation in surgery.15,30,31 The first source, a realist review,27 was undertaken to explain when and how implementation of SSC KT strategies worked, or did not work well, and why. Key findings revealed that using surgical checklists was discipline-specific and that greater uptake was achieved when clinician stakeholders were involved in tailoring and evaluating implementation.

The second source, a prospective study, included a barrier analysis based on 33 interviews with 70 stakeholder participants.15,31 The Theoretical Domains Framework30 informed stakeholder interview questions to identify modifiable barriers and enablers that influenced the participation of surgical teams in all safety checks. The top four barriers were workflow, a lack of knowledge about timing/content of checks, a lack of clinical leadership, and negative/skeptical attitudes. Stakeholders considered that workflow had the greatest impact on their ability to be present and participate in all phases of the SSC. Consequently, workflow was systematically mapped with stakeholders.31 Findings indicated that the organization of work activities and the need to multitask during time-critical periods prevented team members from participating in various safety checks. Pragmatically, this meant that current checking processes were not feasible and needed to be tailored to incorporate workflow needs.31

Ethics review

This study was reviewed by the ethics committees of hospital (HREC/13/QGC/154) and the university (NRS/06/14/HREC). Participants gave informed consent for observations and interviews and were assured that participation was voluntary and their decision to decline participation would not prejudice their professional or employment status. Participants were advised of their rights to discontinue participation anytime throughout the data collection period.

Data collection and analysis

Data were collected over 3 months, during 2015–2016. To evaluate consistency (ie, fidelity, dose, coverage) of implementation, 35 surgical teams and 30 education sessions were audited using a standardized data collection tool specifically developed for the evaluation. In assessing feasibility and acceptability of implementation, semi-structured interviews were used. Interviews and audits were conducted by two members of the research team. Observations of surgical teams and interview participants were sampled conveniently. An interview guide was developed, and the interviews were intended to enrich the observational findings and provide supporting data on feasibility and acceptability. Implementation fidelity, dose, and coverage were captured through observational audit. The number of posters displayed in the department, the number of staff who received lanyards and pens, the number of education sessions delivered (by whom, number of attendees, and duration), and the presence/number of change champions were documented.

Individual or group interviews with participating interdisciplinary team members lasted between 10 and 30 min. Interview questions included: Were the implementation strategies delivered? If so, how were they delivered? Were they delivered the way they were intended to be delivered? How did you find using the implementation strategies? In all, 15 semi-structured interviews were conducted, including five group interviews, with a total of 96 participants. Of these, 56 were nurses, and 40 were physicians from anesthetics and surgery.

Observational data were analyzed using the Statistical Package for Social Sciences (SPSS V 22; IBM Corp, Armonk, NY, USA). Data entry was checked for accuracy and analyzed descriptively using absolute (n) and relative (%) frequencies. For interviews, a deductive content analysis approach was used as described by Elo and Kyngash32 to prepare, organize, and report findings relative to feasibility and acceptability of the implementation of the KT strategies. As part of the analytic process, words or phrases were coded and categorized as either barriers or enablers. The decision to classify codes as either a barrier or an enabler was relative to its perceived impact on participants’ clinical practice.

Underpinning assumptions and intervention

The following underpinning assumptions were supported by behavior change theories33–35 and checklist literature:16,27,36

Staff value the SSC as a safety tool for enabling team-based communication.

Staff are accessible and willing to participate in the KT strategies and the PTB intervention.

Implementation of the PTB intervention will result in improvements in team communication and work processes.

The KT strategies were “cocreated” and implemented with stakeholder input, with the ultimate aim of supporting the PTB intervention. “Cocreation” involves developing a shared body of knowledge to close the gap between evidence and practice in partnership with the end users – clinicians.37,38 Table 1 details each of the four strategies and their processes and the proposed mechanisms of action.

Table 1

KT strategies, targeted barriers, and processes that support the implementation of PTB

KT strategy	Targeted barrier	Intervention process	Proposed mechanism of action
Opinion leaders and change champions	Lack of clinical leadership: senior staff reluctant to take ownership of the checking process	Type: role modeling, social comparison, and information of others’ approval, social supportMode: real-time demonstration in clinical practiceContent: building checks into current workload, normalizing the process as part of practice	• Provides an example to which others can aspire• Role modeling from credible sources• Provides social comparison of behavior• Provides social support and improves motivation to perform checks
Education	Lack of knowledge: staff lack the necessary knowledge, skills, and training in the timing and content of checks	Type: education, demonstrate behavior, provide instructionMode: inservices, clinical skills training, staff emailsContent: outline of PTB strategies, building in, and timing of checks taking into account workflow issues	• Increases self-efficacy through manipulated mastery experience (ie, practicing a behavior) and vicarious experience (ie, observing a model performing the behavior)• Feedback on past performance increases self-efficacy
Audit and feedback	Negative/skeptical attitudes: perceived uncertainty about the value of the safety checks and their intended effect	Type: education, persuasion, behavioral feedback, self-monitoringMode: inservices, printed materials, staff emailsContent: information about performance	• Improves attitudes toward checking processes• Promotes reflection, feedback, and self-monitoring on the checking process• Improves motivation and behavioral action
Prompts and reminders	Workflow: staff feeling pressured because of time constraints and productivity demands (hectic setting, multitasking, heavy workload)	Type: environmental restructuringMode: posters, lanyards, pensContent: PTB logo	• Prompts and reminders are cue to action to get people to act and maintain behavior – build a habit• Increases behavioral maintenance

Abbreviations: PTB, Pass the Baton; KT, knowledge translation; TDF, Theoretical Domains Framework.

As workflow was a major barrier, it was important that the PTB intervention itself (as distinct from the KT strategies that supported it) addressed this barrier. The most problematic phases of the SSC where staff participation/usage was at its lowest were during the “sign in” (52.1%) and “sign out” (0.0%) phases.15,31 When checks were performed during “sign in,” they were not verbally confirmed with others in the surgical team because they were not present or were performing other priority tasks during that time. “Sign out” was not done at all as a team despite that most members were still present during wound closure.31 Enabling staff to better utilize the SSC required tailoring the SSC in relation to: 1) the person delegated to performing/leading the checks, and 2) the timing and confirmation of these checks with others. Stakeholders agreed collectively that nurses were pivotal to facilitating the checks and acting as conduits of information to others. The PTB intervention is depicted in Figure 2 and described below:

Figure 2

Pass the Baton intervention, mapping flow of communications using the checklist.

Abbreviations: A/B, antibiotics; PTB, Pass the Baton; AN, anesthetic nurse; ANA, anesthetist; SS, scrub/scout nurse; SUR, surgeon; VTE, venous thromboembolism.

To increase team member participation in “sign in” anesthetic nurses initiated “sign in” and verbally confirmed relevant information to other members of the team (in the operating room) at an appropriate time.

To facilitate team member participation in “sign out” instrument nurses led this phase, and verbally confirmed each checklist item with other team members during wound closure.

Results

Selection of KT strategies, that is, change champions/leaders, education, audit and feedback, and prompts and reminders, was data driven.15,31 Results of the observational audit indicated moderate consistency in implementation. Prior to implementation, staff from surgery, anesthetics, and nursing were notified via email on two occasions. In terms of overall coverage, 160/220 (72.7%) staff including nurses, anesthetists, and surgeons knew generally about the implementation strategies. Introduction of each KT strategy was staggered over four 1-week periods.

Week 1 – Opinion leaders/change champions

The strategy was to identify opinion leaders from surgery, anesthetics, and nursing. The greatest buy in and engagement came from the nurse leaders in education. The Perioperative Educator identified four nurses working across surgery and anesthetics who assumed the role of change champions to lead implementation. Despite attempts to identify change champions in surgery and anesthetics, very little engagement occurred. Implementation of opinion leaders and change champions was perceived as useful by 81/96 (84.4%) interview participants.

Weeks 2–4 – Information sessions

Brief education sessions combining audit and feedback were implemented, and the results from process mapping workflow were presented to staff. Process map results, detailing workflow issues, resonated well and increased buy in, particularly from nursing staff. Of the 30 education sessions audited, eight (26.6%) involved PowerPoint presentations that were delivered by a designated change champion and lasted between 20 and 40 min. Of these, 7/8 sessions targeted nursing staff with 20–40 nurses in attendance at each session. The remaining education session was attended by 35 anesthetists (consultants and registrars) in the anesthetic department and was delivered by one of the study investigators. The other 22 (73.3%) informal sessions were delivered ad hoc by two nurses who were designated change champions. These sessions targeted any nursing or medical staff present, lasted between 5 and 10 min, and occurred during case setup periods, between cases, and during downtime periods. Assembling surgeons en masse was unfeasible because of competing work commitments that spanned across several different hospitals in the health services district. As such, surgeon consultants and registrars were targeted during the ad hoc education sessions that occurred serendipitously. Of the 96 participants interviewed, 91/96 (94.8%) believed that receiving some form of education was useful, particularly including audit and feedback as part of the education strategy.

Week 4 – Prompts and reminders

A total of 14 A3-sized posters were visible throughout the 22 theatre complexes. Ten posters were displayed in department corridors, whereas four others were located on the education board, reception area, recovery room, and staff tearoom. During audit, at least three members in 16/35 (45.7%) teams were wearing lanyards with the PTB logo. While 200 pens (with the PTB logo) were distributed to nursing and medical staff, they were not visible during the audit period. In relation to reminders, 68/96 (70.8%) participants believed that the posters and lanyards were useful prompts.

Table 2 displays the breakdown of findings relative to the feasibility and acceptability of implementation. A notable barrier to feasibility was physician engagement. An obstacle to acceptability was participants’ uncertainty about how implementation strategies could/would improve clinical practice.

Table 2

Verbatim based on 15 informal qualitative interviews* with 96 participants

Component of implementation	Supporting verbatim
	Barriers	Enablers
Feasibility	• Do not expect the doctors to participate but many of us will support the nurses (ANA).• Got to factor-in the continual change-over of registrars every 6 months which means that they have to be told about PTB. I think it (PTB) could get lost in change-overs (RN).• How will giving out pens and putting up posters help? What is the outcome you are trying to achieve (SUR)?• Hard to get the surgeons together to tell them about PTB. Monthly meetings with consultants and registrars are conducted in each craft group but attendance is a bit hit-and-miss, depending on the day (SUR).	• Nurses often champion everything. Eventually everyone else comes on board (RN).• The Friday morning inservice is a good time to get staff together because more nurses are able to attend (RN).• It is very helpful to watch the Level 2 nurses talking about how to do PTB (RN).• Any strategy that increases patient safety is worth doing (RN).• The idea of PTB is pretty simple and easy to do (ANA).• The message of PTB is really apt in relation to the checklist (RN).• An education meeting on Fridays would be the best time to catch anesthetists (ANA).• Have the support of the NUM and educators to use PTB (RN).
Acceptability	• PTB is not working well in its current form; some other strategy is needed or at least modify it somehow (SUR).• What is the point of PTB? I do not think giving people pens and putting up posters will change practice (SUR).• The strategies in PTB are good but they are not enough; a 10 min DVD showing teams how the checklist should be done would help (ANA).• It is difficult in the theaters because posters cannot be put up on the walls. The best that can be done is to put smaller (A4 sized) posters on the clip boards in each room (RN).• As time passes, the posters just become background noise. People are less inclined to notice them on the wall and walk straight pass them (RN).	• I like the idea of having lanyards and pens to remind me. Nurses always like pens (RN)!• Always handy to have an extra pen (ANA).• I think the posters work well and they are visible for everyone to see (RN).• All good strategies (PTB) which seemed to have made a change and improved “sign out” (SUR).• Having posters and lanyards are a good advertisement for PTB (RN).• The posters are easy to read, with not much writing, and nice colors (ANA).

Note:

Data based on individual and group interviews. Interview questions were based on the Theoretical Domains Framework.

Abbreviations: RN, registered nurse; ANA, anesthetist; SUR, surgeon; PTB, Pass the Baton.

Discussion

This is the first study of its kind in this area to evaluate the implementation of KT strategies designed to support use of the SSC. Clinical stakeholders were engaged a priori to undertake an evaluation of the implementation, and a logic model was used to inform the process component of the evaluation. The results of this evaluation suggest that implementation of KT strategies was somewhat successful relative to fidelity, feasibility, and acceptability. The four KT strategies employed do not encompass the full gamut of potential KT interventions, and others may need to be considered to ensure sustainability. However, the development of the strategies was both theoretically informed and data driven. Most practice improvement interventions that rely on behavior change are implemented using pragmatic approaches, in response to emergent local needs rather than first identifying barriers and enablers to implementation within a theoretical framework. The approach to implementing these KT strategies, while responsive to the ever-changing contextual demands of the surgical environment, was informed by theory27,38 and empirical data, triangulated from a barrier analysis.15,31

Mechanism of impact

In designing this process evaluation, Pawson26 context–mechanism–outcome was applied to articulate the probable causal mechanisms and the effects that context (ie, anything external to the intervention that acts as barrier or enabler) had on the success (or failure) of implementation. In unlocking the “black box” of implementation, “mechanism” yields an understanding of the workings of the intervention that bring about the effects or changes in behaviors.39 Seldom can any observed change be attributed to one mechanism (ie, cause). Thus, attribution of any one of the four KT strategies to desired behavioral outcomes might be contestable as there are usually several mechanisms of action operating simultaneously that could explain the success of implementation. Understanding how participants interact with a complex intervention is critical to understanding how they work.25 The impact of context on participation in implementation was evident in varying levels of interaction and engagement: contextual factors such as team culture (ie, history, interpersonal relationships) and the characteristics of participants (ie, professional role, level of knowledge/skills/experience, beliefs/attitudes) likely influenced the degree to which participants interacted with the implementation strategies. Indeed, the biggest barrier to the implementation was physician engagement. Physician interaction with implementation was generally limited but when senior nurses initiated and led implementation, their support and participation increased.

During the evaluation, nurse leaders reported variable uptake of the KT strategies among some surgical specialties, depending on the context and the individuals involved. Engagement of nursing leaders was pivotal to implementation success. A key strategy targeting participant engagement with PTB was the use of opinion leaders and change champions. During implementation, opinion leaders led both formally and informally through their attitudes and overt behavior and were at the center of interpersonal communication networks within the department. A systematic review of 12 studies on the effectiveness of opinion leaders’ strategies concluded that their implementation led to improvements in compliance, with desired practice ranging from an absolute improvement of 25% to a decline of 6% with an overall median improvement of 10%.40 In this evaluation, the education/information sessions were well attended, and stakeholder participation was interactive. Educational sessions offered to large numbers of learners are common, but some evidence suggests that this type of KT strategy produces little, if any, behavior change. Yet, other studies have indicated that sessions where there is increased interactivity, variation in teaching methods, and linked to assessment are likely to be more effective in improving clinical performance.41 The education sessions implemented in this study focused on strategies designed to optimize checklist use taking into account workflow issues. Audit and feedback on performance using the checklist were included, and overall the feedback from stakeholders was positive. The underlying mechanisms of action with audit and feedback are increasing motivation by showing the gap between actual and desired performance.42 A Cochrane Review of 140 randomized controlled trials concluded that audit and feedback lead to small but potentially important improvements in clinical practice.43 However, effectiveness depends on baseline performance and how feedback is provided.43 Delivery of feedback varies according to format (ie, verbal, paper, electronic), frequency and duration, content, and use of various resources.42 During implementation of PTB, reminders such as posters, lanyards, and pens were used as prompts. While reminders may become ineffective over time if they are not modified,42 they enable self-regulation and reduce the cognitive load for clinicians, thus promoting habit formation for the behavior. However, as habit formation depends on the repetition of a behavior, it may be that other behavior change techniques focused on self-regulation and sustaining motivation, like goal setting, self-monitoring, learning associations, behavior substitution, and repetition are needed.44 Therefore, the role of habit formation as a key mechanism involved in the development and maintenance of using the checklist is worthy of further investigation.

Limitations and strengths

One of the key strengths of this process evaluation was that it was cocreated with stakeholders across nursing, surgery, anesthetics, management, and education. Through its participatory design, stakeholders at all levels within the organization were actively involved in specifying the KT strategies designed to support the implementation of the PTB intervention. Hence, it is more likely that the intervention will be sustained over time. Logic model methodology was incorporated to inform the evaluation, but while logic models represent intention, they do not reflect reality.28 Nevertheless, in developing the model, stakeholders and evaluators are forced to focus on and be accountable for what matters – formative and summative outcomes. Furthermore, by adopting a theoretically grounded intervention which is operationally supported by logic model methodology, probable causal mechanisms that bring about any observed effects or changes in clinical behavior could be identified. Fidelity in the implementation of the KT strategies herein was reasonable – however, external validity may be limited because day-to-day implementation is needed to respond to the ever-changing clinical demands of the surgical environment. Consequently, participants adapted implementation to local conditions on some occasions during the 3 month roll out period. These minor variances in fidelity were documented as part of the process evaluation.

Conclusion

Results of this process evaluation suggest reasonable success in implementation of four KT strategies to embed the PTB intervention but refinement and/or addition of other strategies is necessary given the limited engagement of some stakeholder groups. Outcome (ie, summative) evaluation of the medium- and long-term impacts of the PTB intervention itself is underway. After further refinement and tailoring, PTB will be implemented on a larger scale, across multiple hospital sites. Stakeholders and evaluators should continually reflect on and modify the implementation of any behavior change intervention and the KT strategies that support it in response to theory, context, practice, and policy changes that ensue over time. While a body of work has informed this implementation effort, this evaluation is exploratory and further research is needed to move toward undertaking a definitive evaluation.