Conceptualising contexts, mechanisms and outcomes for implementing large-scale, multisite hospital improvement initiatives: a realist synthesis.

DESIGN: Realist synthesis. STUDY
BACKGROUND: Large-scale hospital improvement initiatives can standardise healthcare across multiple sites but results are contingent on the implementation strategies that complement them. The benefits of these implemented interventions are rarely able to be replicated in different contexts. Realist studies explore this phenomenon in depth by identifying underlying context-mechanism-outcome interactions.
OBJECTIVES: To review implementation strategies used in large-scale hospital initiatives and hypothesise initial programme theories for how they worked across different contexts.
METHODS: An iterative, four-step process was applied. Step 1 explored the concepts inherent in large-scale interventions using database searches and snowballing. Step 2 identified strategies used in their implementation. Step 3 identified potential initial programme theories that may explain strategies' mechanisms. Step 4 focused on one strategy-theory pairing to develop and test context-mechanism-outcome hypotheses. Data was drawn from searches (March-May 2020) of MEDLINE, Embase, PubMed and CINAHL, snowballed from key papers, implementation support websites and the expertise of the research team and experts. INCLUSION CRITERIA: reported implementation of a large-scale, multisite hospital intervention. RAMESES reporting standards were followed.
RESULTS: Concepts were identified from 51 of 381 articles. Large-scale hospital interventions were characterised by a top-down approach, external and internal support and use of evidence-based interventions. We found 302 reports of 28 different implementation strategies from 31 reviews (from a total of 585). Formal theories proposed for the implementation strategies included Diffusion of Innovation, and Organisational Readiness Theory. Twenty-three context-mechanism-outcome statements for implementation strategies associated with planning and assessment activities were proposed. Evidence from the published literature supported the hypothesised programme theories and were consistent with Organisational Readiness Theory's tenets.
CONCLUSION: This paper adds to the literature exploring why large-scale hospital interventions are not always successfully implemented and suggests 24 causative mechanisms and contextual factors that may drive outcomes in the planning and assessment stage. © Author(s) (or their employer(s)) 2022. Re-use permitted under CC BY-NC. No commercial re-use. See rights and permissions. Published by BMJ.

An iterative process was used to search, extract data, validate and analyse results using evidence and expertise from researchers and partners.

RAMESES Reporting Standards were used to ensure rigour of each staged step.

In spite of a systematic and thorough search for literature, only 51 papers were found; while large-scale hospital interventions abound, implementation activities and outcomes are not commonly reported.

The wealth of data constrained our study to consider only a single formal theory, and a subset of implementation strategies.

Introduction

The implementation of large-scale, multisite, hospital-based improvement initiatives, developed from high quality evidence have the potential to standardise practice, improve safety, continuity and quality of care for patients, reduce unnecessary, unwarranted treatments and provide better value for money.1 Large-scale hospital interventions, as discussed here, are projects that are typically intended to be implemented across multiple hospitals (eg, all public hospitals in a region). They are usually ‘top down’ in nature, in contrast to local, clinician-initiated ‘grass-roots’ projects. The mandate to implement these initiatives is typically from the hospitals’ funding or governing bodies (eg, State Health Departments, or local health networks), or high-level clinical agencies (eg, a national Quality and Safety Commission). Such interventions may be supported by additional staff and resources and align with other high-level health system priorities. The focus of these initiatives is improvement of care and did not include mandated, enforceable health orders.

The QUARISMA intervention in Quebec, Canada, for example, was implemented in 32 hospitals.2 The intervention was based on best practice guidelines derived from recommendations of the Society of Obstetricians and Gynaecologists. The hospitals that implemented it, successfully and safely reduced the rates of clinically unwarranted caesarean sections in low risk mothers.2 Another example of a large-scale hospital intervention is the WHO’s surgical safety check-list3 which was successfully adopted in six high performing hospitals in The Netherlands. This significantly reduced surgical complications and mortality.4

Large-scale interventions are expensive and time consuming to implement.5 Their success is contingent on the implementation programmes that accompany them6; that is, the suite of individual implementation strategies designed to prepare the hospitals for change, and equip the focal stakeholders to adopt new practices and adapt or discard old ones. Recent systematic reviews have identified a range of strategies linked to successful implementation programmes, such as conducting a needs assessment, recruitment of champions or opinion leaders, use of audit and feedback, engaging organisational leaders and developing implementation teams (eg,7 8). For large-scale interventions, these implementation programmes are often required to fit a range of hospitals of different size, geographical and socio-demographic contexts, and health consumer needs.

In recent years, implementation strategies have been compiled, described and categorised9 but research has failed to explain why strategies that work as intended in one context (eg,10) may be a failure in another (eg,11). Results suggest that those designing implementation strategies have failed to take into account local contextual features,12 and that contextual features are poorly conceptualised and defined in reporting.13 Moreover, the underlying mechanism of action, working within that context, is only rarely defined, implying that the way strategies work is poorly understood. A programme theory that lies beneath the implementation programme and that articulates how the strategies are thought to work is often not explicitly stated. Davies and colleagues showed in their review of 235 guideline dissemination and improvement projects in healthcare, only 23% used theory of any kind to inform the development of the implementation strategies.14 This, they argue, can result in a poor choice of implementation strategy for the context (eg, settling for a ‘default strategy’ such as an education session15) and corresponding poor results.

Realist approaches take a deep dive into why programmes work as intended some of the time but not all of the time.16 A realist approach asserts that all programmes have an underlying programme theory that explains how the strategies bring about intended or unintended results. This holds the promise of unpicking the link between the context and outcomes. A realist synthesis is the ideal approach to understand implementation programmes for large-scale hospital interventions, as it explores the links between strategies, mechanisms of action, contexts, the responses of clinicians and outcomes. Terms used in this synthesis referring to types of theories are defined in box 1.

Formal theories: here, this refers to general implementation science theories that have been used to explain how implementation strategies work broadly and for which there is some empirical support. Also called mid-range theories.42

Programme theory: a theory that explains how and why particular types of interventions work to generate the outcome/s of interest.16

Initial programme theory: a programme theory that is hypothesised, tested and refined as a result of the realist synthesis to explain how the focal type of intervention generates the outcome/s of interest.

Potential initial programme theories: a suite of programme theories being considered as an initial programme theory.

A realist synthesis is a generative process, first understanding the nature of the implementation programme and then proposing potential initial programme theories around the way a programme works. These initial programme theories, configured as contexts (circumstances under which the programme works), mechanisms (generative causes of how programmes elicit results) and outcomes (the results of the programme), are then tested using published literature.16 The context–mechanism–outcome configurations (CMOs) that are found through analysis of the literature can be explored and used to formulate and refine initial programme theories which explain how and under what circumstances programmes achieve different outcomes. Consequently, realist research does not apply value judgements on programme outcomes such as ‘successful’ or ‘unsuccessful’. Instead, it acknowledges that programmes produce intended and unintended outcomes.17

The aim of this realist synthesis was to synthesise evidence and generate initial programme theories that explain how implementation strategies work in large-scale hospital interventions; in other words, to gather evidence on what works as intended for whom, in what circumstances and why. This realist synthesis is divided into two parts. First, we scope the literature seeking to understand the concepts and features of implementation programmes for large-scale hospital interventions to understand the sorts of formal theories that may be relevant here. Second, we focus on a single group of implementation strategies and generate initial programme theories18 and CMO configurations to test against the literature.

Both parts of the synthesis are part of a larger project19 examining seven Leading Better Value Care projects implemented in metropolitan, remote and regional-based hospitals (n=100) across New South Wales (NSW), Australia, between 2016 and 2018.19 These projects are based on a value-based care paradigm and address unwarranted clinical variation, and preventable hospitalisations across seven high impact conditions.20 Early results from this project showed that implementation strategies accompanying the projects were variably successful across sites at eliciting buy-in and adoption of the interventions. This current study is informing a realist evaluation of the implementation strategies used to build a nuanced model to support future large-scale hospital implementations; specifically, by defining relevant concepts and proposing initial programme theories.

Methods

We followed the Reporting Standards for realist syntheses recommended by the RAMESES group.17 We used a combination of academic database and grey literature searches, data extraction and fortnightly research team discussions to collate evidence for the synthesis. Throughout the work, research team discussions around data extraction and interpretation were informed by ongoing discussions with partners at the NSW Ministry of Health, Agency for Clinical Innovation and Bureau of Health Information who were experienced in design and implementation of large-scale hospital initiatives, and colleagues from Macquarie University’s Centre for the Health Economy. All searches were conducted between March and August 2020. Table 1 shows the four iterative steps of our method.

Table 1

The four iterative steps used to search, find, extract and synthesise evidence to generate initial programme theories that explain how implementation strategies work in large-scale hospital interventions

Step	Purpose	Research questions	Activities
1 2	To conceptualise implementation of large-scale hospital interventions.	What are the key concepts and features of large-scale hospital initiatives and their implementation?What mechanisms might these suggest are key to driving the programme?	Build an initial list of concepts and associated features based on research team’s research and clinical experience.Add to the list through a search of published literature on implementation of large-scale hospital interventions.Consider antecedents and outcomes of each feature to identify putative relevant mechanisms.
	To scope suites of implementation strategies used with large-scale hospital interventions.	What implementation strategies are used for large-scale hospital initiatives?How do they fit with Step 1?What do they tell us about possible mechanisms, contexts and the underlying programme theories?	Collation of implementation strategies extracted from Step 1 literature.Search of additional published literature including extracted studies from systematic reviews.Grey literature search: targeted websites and search terms.Strategies aggregated and sorted then mapped to ERIC implementation strategies.
3 4	Identify potential initial programme theories.	What formal theories might explain the mechanisms of action for the strategies listed?	Identification of formal theories from the published literature. Consideration of theories in the context of implementation strategies we have listed.Refinement of the initial programme theory-implementation strategy pairing through research team workshops using all data generated from the project.
	Focus on a promising implementation strategy-theory pairing and development of CMOs.	What context–mechanism–outcome configurations can we develop and test with the literature around implementation strategies linked to Organisational Readiness Theory?	Research team workshop to develop initial CMO statements.Testing and refinement of CMO statements through review of literature from Steps 1–3.Final iterations of CMOs.

CMO, context–mechanism–outcome configurations; ERIC, Expert Recommendations for Implementing Change.

Step 1: conceptualising large-scale hospital interventions

The first step towards generating initial programme theories in a realist synthesis is to identify the key concepts of the topic of interest. Concepts are tightly linked to programme theories as they help to understand where key mechanisms leading to expected outcomes are likely to occur.18 Here, we identified and defined key concepts associated with the implementation of large-scale hospital initiatives by exploring the focal stakeholder cohort, arena of action, social processes, intended outcomes and the nature of support for the programme.

This step drew data from three sources: the research team’s knowledge, expert consultation and a published literature search across three iterative stages. The research team (JCL, MNS, EFA, CP, HMN) was made up of four experienced health services researchers, two with clinical backgrounds, one sociology and the other psychology, and a research assistant (HMN). The team was actively mentored, and work validated by an experienced realist researcher (RH).

First we built a list of concepts and associated features characterising implementation programmes for large-scale hospital interventions from key articles (eg,1) our own research and clinical experience. This was done by the research team in two 1-hour meetings. This list was verified and expanded through ongoing discussions with partners involved in large-scale, multisite initiatives at the NSW Ministry of Health (senior policymakers), Agency for Clinical Innovation (senior implementation support strategists) and the Bureau of Health Information (senior data management and analysis professionals). Discussions occurred as one-on-one interactions (via email) or part of project meetings/updates.

Next, we examined the published literature for evidence to support or refute our list and to look for other concepts and features we had not considered. We searched MEDLINE, PubMed, Embase and CINAHL, using the search string: health AND ((((implementation OR driver) OR change) AND large-scale) AND ((innovation OR intervention) OR program)) AND hospital. We set limits on English language but no date limits were set. We snowballed papers from the reference lists and added known key papers not captured by the search, and included individual studies reported in reviews. We assessed whether each of the concepts and features on our list were supported by the literature, noting each as being reported ‘always’, ‘nearly always’, ‘often’, ‘sometimes’, ‘rarely’ or ‘not at all’.

Using an iterative approach, the research team refined our definition of large-scale hospital interventions as we built up the list of associated concepts and features. Finally, antecedents and intended outcomes of the features as a whole and individually were developed and considered to further explore possible mechanisms that may be relevant. Articles that we included involved implementation across multiple hospital sites for interventions aimed at improving patient safety or quality of care. We did not include programmes situated outside the hospital setting (eg, implemented solely in community-based health services), interventions at only one site, locally driven interventions (eg, internally developed, ward-based improvements) or tightly controlled research trials that were not considered ‘real world interventions’ (eg, randomised controlled trials). We did consider pragmatic trials if they met other parts of our definition. A data extraction sheet was used to organise concepts described in the papers found. Papers not reporting implementation strategies or activities were not included.

Step 2: 'scoping suites of implementation strategies

Our next task was to identify and collate all implementation strategies that were reported as part of these types of large-scale interventions. Together with the concepts and features of the initiatives found in Step 1, this list of strategies and any information reported on how they were intended to work, were needed to understand possible contexts and mechanisms leading to outcomes.

We started our search for implementation strategies with the papers found in Step 1. Next we scanned papers found in an existing systematic review of implementation strategies used in hospital avoidance interventions for people with chronic conditions, choosing projects that met our large-scale, multisite criteria.21 We also searched more broadly for systematic reviews looking at implementation strategies targeting other cohorts of patients (Web of Science (all databases selected): ‘implementation’ AND ‘systematic review’). We included protocol papers hoping these might provide a fuller rationale for their choice of strategies. We also included selected grey literature from a targeted search of implementation materials from agencies known to actively support large-scale implementation programmes: UK’s National Health Service, Canada’s Advance Care Planning, NSW Agency for Clinical Innovation, Australian Medical Research Council, Enhanced Recovery After Surgery Society, and WHO. We set up a data extraction matrix, recording reported implementation strategies for each project. We also ran a Google search on ‘implementation guide’ and ‘implementation healthcare guides.’ Implementation strategies were collated and reviewed in each source.

Initially, we used our own descriptors for the strategies, but then aggregated similar strategies and mapped them to the Expert Recommendations for Implementing Change (ERIC)9 taxonomy of 73 implementation strategies. Any strategies that did not map to an ERIC strategy were still included but noted.

Step 3: identifying potential initial program theories

In this next step, the research team held two 2-hour meetings to workshop ideas towards identifying potential initial programme theories.18 Many theories were proposed in the workshop, mainly from our prior research experience and discussed one by one. We also read up on theories proposed by other realist researchers and added them for consideration. This work was being done in parallel with the realist evaluation of the actual state-wide initiative so this also guided our thinking. This resulted in a short list of promising theories.

The process included consideration of all the data generated so far in the project as well as searching published literature around known formal theories; in particular, we examined together the concepts identified in Step 1 and the implementation strategies identified in Step 2. That is, we considered what existing formal theories or types of theories might be relevant to explain particular implementation strategies given the concepts and putative mechanisms we had identified. For example, an implementation strategy of conducting a local needs assessment, fitted with the concept of facilitation through provision of resources and the feature ensuring a formal period of planning. Organisational Readiness Theory was identified as a formal theory that promised to explain how this implementation strategy of conducting a local needs assessment would work across different contexts. These formal theories became the basis for our initial programme theories and were matched with implementation strategies using this process. Theories were retained or excluded on their ability to broadly describe what was happening in one or more implementation strategies, how and why across a range of contexts, and a range of levels (micro, meso and macro).

Step 4: further scoping and focus on a key strategy

As realist syntheses aim to explain how and why a programme works and have the potential to generate vast amounts of data to do this well, it was necessary to carefully scope the results generated and narrow our focus. Following the example of other realist syntheses,18 22–24 we looked for a single set of implementation strategies and their accompanying initial programme theory that (a) was deemed highly important in informing our parallel tranche of work—the realist evaluation of the Leading Better Value Care projects in NSW, Australia—and (b) had not already been researched using realist methodology.

Results

Results of the activities used to synthesise evidence and generate initial programme theories that explain how implementation strategies work in large-scale hospital interventions are outlined below. The process was driven by the fortnightly research team meetings, separate 2-hour workshops, validation by other authors and stakeholders, and iterative refinements. Table 2 summarises the results from the four steps. Figures 1 and 2 show the Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA)-style flow diagrams for Steps 1 and 2, respectively.

Table 2

Summary of search strategy, activities and results at each of the four steps

Step	Purpose	Activity	Interim results	Final result
1	To conceptualise implementation of large-scale hospital interventions.	Build an initial list of concepts and associated features based on research team’s research and clinical experience, validated by key informants on the wider project.Search databases for implementation of large-scale hospital interventions, screen title and abstract for relevance, data extraction.	5 concepts and 12 features listed.381 articles found.51 relevant articles.Exclusions: not hospital-based, not large-scale, implementation not described.4 additional features identified from the literature.	5 concepts and 16 features identified and described.
2	To scope suites of implementation strategies used with large-scale hospital interventions.	Extracted data from Step 1 literature that report implementation strategies.Search for published literature on implementation and screen for large-scale hospital criteria.Include known literature.Individual studies extracted from reviews.Search of targeted websites and other grey literature.Strategies aggregated and sorted then mapped to ERIC implementation strategies.	45 articles.585 reviews found:31 found to include relevant studies some reporting on multiple implementation strategies.Data extracted from 5 sets of grey literature documents.	302 reports of 28 different implementation strategies identified and collated.28 implementation strategies mapped to ERIC taxonomy, 1 did not map.
3	Identify potential initial programme theory areas.	Identification of potential initial programme theories using all data generated from the project so far plus other realist studies, compilations of programme theories and literature describing individual formal theories.	3 broad domains of action identified.	5 initial programme theories mapped to implementation strategies.
4	Focus on a promising implementation strategy-theory pairing and development of CMOs.	Research team workshop to develop initial CMO statements informed by Organisational Readiness Theory.Testing and refinement of CMO statements through review of literature from Steps 1–3.Final iterations of CMOs.	All data collected so far.51 articles+4 articles that focused on organisational readiness assessment.	24 CMOs were hypothesised and literature used to support or refute them.

CMO, context–mechanism–outcome configurations; ERIC, Expert Recommendations for Implementing Change.

Preferred Reporting Items for Systematic Reviews and Meta-Analyses-style flowchart for data sources in Step 1.

Concepts associated with large-scale hospital intervention implementation programmes (step 1)

The research team initially listed 5 concepts associated with 12 features of large-scale hospital interventions, which grew to a final set of 16 features after further scoping of the literature. Over 400 titles and abstracts were accessed via database searching and data were extracted from a subset of 51 full-text articles that met our inclusion criteria. Table 2 summarises results of Step 1 and online supplemental file 1 shows the full data extraction sheets.

The five concepts of large-scale hospital improvement initiative implementation were: (i) External, top–down source, (ii) Evidence-based interventions, (iii) Safety and quality focus, (iv) Facilitation through provision of resources and (v) Harnessing of local resources and encouraging adaptation. Between two and four features of each were identified.

External, top–down source

Features found associated with this concept were that the interventions being implemented were externally developed: either by peak agencies or research institutes (eg, WHO,25 American College of Surgeons26), quality collaboratives (eg, Michigan Surgical Quality Collaborative,27 German Quality Network28) or in one case, mandated, evidence-informed policy (eg, US Veterans’ Affairs (VA) National Disclosure Policy29). Support for implementation for the intervention itself was frequently built into this package by the external source: interventions were often presented as a ‘bundle’ of interventions all aimed at addressing a single issue (eg, surgical site infections,30 treatment of blunt chest injury31). Checklists and implementation guides may also be provided by the external agency that developed the intervention. Contrary to our expectations, the offer of incentives or disincentives for implementation was rarely reported.

Evidence-based interventions

All interventions were identified by the authors as being evidence-based, although the evidence (eg, the randomised control trial on which the intervention was built) itself was rarely cited. Contrary to the expectations of our research team, de-implementation of processes and practices that presumably were no longer ‘best practice’ was rarely reported. This applied even to upgraded information technology systems where legacy systems were allowed to remain alongside the new programmes.32

Safety and quality focus

A clear aim of improving patient outcomes was consistently found, often by making a case for change from baseline data. Implicit in most programmes was the assumption that a positive safety culture, that saw improvement of patient outcomes as core business was present at the site. Also implicit was that there was consensus at each site that the intervention was needed, and that the implementation support provided would be acceptable.

Facilitation through provision of resources

As well as implementation guides and intervention resources, external support was seen in many projects in the form of new equipment, customised forms for documentation, and care pathways. Project officers skilled in the intervention and tasked with data collection or training were funded in some projects, often budgeted as part of an associated research component (eg,33). Partnership agreements with external agencies facilitated implementation by providing access to specialist advice. Funding for the projects was often a mix of external (eg, VA (USA)29), internal (eg, Hornsby Ku-ring-gai District Hospital (Australia)33) and research-based (eg, National Institutes of Health grants (USA)).34 Facilitation was not always a feature. All studies relied on the goodwill of clinicians, and some did not factor in any quarantined time for implementation activities such as performing audits. Interventions developed by clinical collaboratives were often framed as partnerships, including access to practical support and expert advice (in-kind) for the implementation and monitoring of outcomes, some allowing dissemination of learnings from other sites, and benchmarking. Data was often provided to make a case for change, and support for ongoing audit and feedback were common features.

Harnessing local resources and encouraging adaptation

The provision of a lead-in time for each site to assess for readiness and local needs was sometimes reported, and internal support for implementation from senior management was reported in most papers. Design amenable to adaptation to fit different local practices, patient cohorts or workflows, developed by clinically based implementation teams, was also frequently reported. Clinical leadership, mentoring, supervision and in-house education were also key features.

Following this, features were refined by determining their antecedents and intended outcomes, to help with the next step of defining associated implementation strategies, mechanisms and potential initial programme theories. Online supplemental file 2 shows the results of this step.

Collated suites of implementation strategies (step 2)

We found 302 reports of 28 different implementation strategies associated with large-scale hospital interventions from 45 peer-reviewed papers and five sets of grey literature documents (each linked to a single website). Figure 2 shows the PRISMA flow diagram for this step. All of the strategies except one mapped to one or a combination of strategies in the ERIC taxonomy.9 The strategy that did not map was Aligned with organisational/District and Departmental priorities. Some strategies that were similar were combined as descriptions in the articles were not sufficient to determine exact details (eg, Involve executive boards was combined with obtain formal commitments as it was often the executive group which was negotiating on behalf of the site.) The 28 strategies are summarised in table 3 and shown in full in online supplemental file 3. Most frequently reported or recommended strategies were: Promote adaptability/purposely re-examine the implementation (n=34); Involve executive boards/obtain formal commitments (n=24); and Assess for readiness and identify barriers and facilitators (n=24).

Figure 2

Preferred Reporting Items for Systematic Reviews and Meta-Analyses-style flowchart for data sources in Step 2.

Table 3

List of implementation strategies and their frequency, found in the set of 51 grey and black literature documents

ERIC taxonomyERIC,9	Implementation strategy	Frequency (n=51 sources)
Access new funding	Extra staffing as needed; salary support	6
Assess for readiness and identify barriers and facilitators	Readiness	24
Audit and provide feedback	Audit and feedback	11
Build a coalition; create new clinical teams; create a learning collaborative	Multidisciplinary involvement; clinical leadership	16
Capture and share local knowledge	Community of practice/knowledge network of clinicians	11
Change physical structure and equipment (a)	Funding for equipment	6
Change physical structure and equipment (b)	Tools to improve communication	4
Conduct cyclical small tests of change	PDSA cycles	5
Conduct local consensus discussions; facilitator	Local facilitator/project officer	10
Conduct local needs assessment	Identify resources required	12
Create a learning collaborative	Engaging stakeholders	7
Develop a formal implementation blueprint (a)	Implementation guides	14
Develop a formal implementation blueprint (b)	Intervention toolkit	10
Develop academic partnerships; use an implementation advisor; use advisory boards and workgroups	Support from external experts/external support	14
Develop and implement tools for quality monitoring	Monitoring	6
Develop educational materials; distribute educational materials	Education	18
Develop resource sharing agreements	Resources shared	1
Distribute educational materials	Clinical practice guidelines	8
Facilitation	Problem solving	2
Identify and prepare champions	Champion	4
Inform local opinion leaders	Opinion leaders	7
Involve executive boards; obtain formal commitments	Executive sponsorship/engagement with the state-wide collective	24
Organise clinician implementation team meetings	Quarantined time for skill acquisition	4
Promote adaptability; purposely re-examine the implementation	Local adaptation	34
Provide clinical supervision	Mentoring/supervision/coaching	16
Recruit, designate, and train for leadership	Clinical leadership	10
Use data experts	Information technology and communication support for new processes	6
(No ERIC equivalent)	Align with organisational/district or departmental priorities	12
Total		302

ERIC, Expert Recommendations for Implementing Change; PDSA, Plan, Do, Study, Act.

Identify potential initial programme theories (step 3)

The research team workshop started by considering both the concepts and features from Step 1 and the strategies from Step 2 to identify high level domains in which our potential initial programme theories and their underlying mechanisms would be expected to work. Four of these domains were identified: social processes and influences; assessment and planning; accessing resources; and partnering outside the organisation. Domains were not seen as mutually exclusive but connected and interdependent. A list of formal theories that addressed these domains was compiled through researcher knowledge and discussion, searching other published realist studies, literature on programme theories and online searches. Five formal theories that explained in a very broad sense, how various strategies might be expected to work were selected through discussion. The theories that were selected were: Organisational Readiness Theory, Social Cognitive Theory, Partnership Synergy Theory, Diffusion of Innovation and the Theory of Planned Behaviour. Table 4 summarises the selected formal theories. Table 5 shows the strategies, concepts, domains and their matched theories.

Table 4

Summaries of formal theories selected as potential initial programme theories to explain mechanisms across different contexts of the implementation strategies identified

Theory	Overview (sources)
Organisational Readiness Theory	Readiness for change refers to organisational members’ shared resolve to implement a change (change commitment) and shared belief in their collective capability to do so (change efficacy).35
Social Cognitive Theory	Behaviour is influenced by three mechanisms operating in concert: direct personal agency; proxy agency that relies on others to act on one’s behalf to attain the desired goals; and collective agency where the larger group acts.43
Partnership Synergy Theory	Partners who effectively collaborate and share knowledge, skills and perspectives are able to achieve more value than the sum of the individual parts contributed.44
Diffusion of Innovation	Explains how an innovation, new idea or product spreads, mediated by social processes within a population over time. A slow start by innovators and early adopters demonstrates the innovation in practice, increasing confidence. A tipping point is reached after a time when the majority take up the new practice. A small group of conservative and risk aversive ‘laggards’ will be the last to adopt.45
Theory of Planned Behaviour	Three independent constructs determine a person’s intention to perform a specific behaviour: ‘attitude’ refers to how positively or negatively a person perceives the behaviour; ‘social norm’ refers to the perceived pressure from others to perform the behaviour; ‘perceived behaviour control’ relates to how easy or difficult the person thinks it will be to perform the behaviour.46

Table 5

Theory areas associated with implementation strategies

ERIC strategy	Domain	Associated concepts (bold) and intended outcomes	Associated initial programme theories
Develop a formal implementation blueprint	Baseline assessment and planning	Clear implementation plan or blueprint for change Clear aim of improving patient outcomes: Clear communication of expectations across sites; tool for planning changes. Provide support for comparison across sites implementing the intervention	Social Cognitive Theory
Conduct cyclical small tests of change	Ongoing assessment	Designed with adaptation to local settings in mind: Incremental changes easier than multifaceted ones.	Social Cognitive Theory
Promote adaptability; purposely re-examine the implementation	Ongoing assessment	Designed with adaptation to local settings in mind: Negotiation, needs assessment, ownership of change.	Social Cognitive Theory
Build a coalition; create new clinical teams	Partnering	Facilitate access to reputable advice and problem-solving assistance: Inclusion, trust, common goal, breadth of expertise.	Partnership Synergy Theory
Develop academic partnerships; use an implementation advisor; use advisory boards and workgroups	Partnering	Facilitate access to reputable advice and problem-solving assistance: Breadth of expertise, social support.	Partnership Synergy Theory
Align with other priorities	Social processes	Formal period of planning and needs assessment: Assess the fit with current workflow, personal and organisational goals aligned.	Organisational Readiness Theory Social Cognitive Theory
Conduct local needs assessment	Baseline assessment and planning	Formal period of planning and needs assessment: Assessing readiness; understanding implications of change on workflow and practice. Designed with adaptation to local settings in mind	Organisational Readiness Theory
Assess for readiness and identify barriers and facilitators	Baseline assessment and planning	Formal period of planning and needs assessment: Setting up conditions that support change.	Organisational Readiness Theory
Change physical structure and equipment	Accessing resources	Provide or facilitate practical support in the form of resources and equipment: Aligning structure with process.	Partnership Synergy Theory
Use data experts	Partnering	Provide data support for new or changed IT systems, baseline audits and ongoing monitoring: Partnership with experts to support change.	Partnership Synergy Theory
Develop resource sharing agreements	Partnering	Provide or facilitate practical support in the form of resources and equipment: Working with others to effect change.	Partnership Synergy Theory
Develop educational materials	Baseline assessment and planning	Formal period of planning and needs assessment: Setting up educational support/conditions that support change.	Organisational Readiness Theory
Distribute educational materials	Accessing resources	Provide practical support in the form of education and skill acquisition: Knowledge and skill acquisition, increase in personal and collective competence and confidence.	Social Cognitive Theory
Provide clinical supervision	Social processes and influences	Provide practical support in the form of education and skill acquisition: Social support, role modelling, and practice of new behaviours. Provide social support from executive sponsorship, supervised practice, project officers, opinion leaders or champions	Social Cognitive Theory
Access new funding	Accessing resources	Provide practical support in the form of resources and equipment: Setting up conditions that support change.	Partnership Synergy Theory
Create a learning collaborative	Social processes	Provide social support from executive sponsorship, supervised practice, project officers, opinion leaders or champions: Social influences supporting change and learning.	Diffusion of innovation Organisational Readiness Theory
Facilitation	Social processes	Provide social support from executive sponsorship, supervised practice, project officers, opinion leaders or champions: Breadth of expertise, social support.	Diffusion of innovation
Identify and prepare champions; inform local opinion leaders	Social processes	Provide social support from executive sponsorship, supervised practice, project officers, opinion leaders or champions: Social influence supporting change.	Diffusion of innovation Organisational Readiness Theory
Involve executive boards; obtain formal commitments	Social processes	Provide social support from executive sponsorship, supervised practice, project officers, opinion leaders or champions: Trust, social support, legitimacy, accountability.	Social Cognitive Theory Diffusion of Innovation Organisational Readiness Theory
Recruit, designate, and train for leadership	Social processes	Provide social support from executive sponsorship, supervised practice, project officers, opinion leaders or champions: Social influence supporting change. Provide or facilitate practical support in the form of education and skill acquisition:	Social Cognitive Theory Diffusion of innovation
Organise clinician implementation team meetings	Social processes	Provide social support from executive sponsorship, supervised practice, project officers, opinion leaders or champions: Social influence supporting change, setting common goals and expectations.	Organisational Readiness Theory Theory of Planned Behaviour
Conduct local consensus discussions	Social processes	Provide social support from executive sponsorship, supervised practice, project officers, opinion leaders or champions: Social influence supporting the setting of clear objectives, building local trust, planning. Designed with adaptation to local settings in mind	Organisational Readiness Theory Partnership Synergy Theory Theory of Planned Behaviour
Audit and provide feedback	Baseline assessment and planningOngoing assessment	Formal period of planning and needs assessment: Setting up tension for change. Provide support for comparison across sites implementing the intervention: Standardised collection of data sets up a tension for change, diagnoses areas for individual sites to work on, and tracks progress locally and across sites.	Organisational Readiness Theory
Capture and share local knowledge	Social processes	Support for comparison across sites implementing the intervention: Increase the breadth of expertise, social support.	Social Cognitive Theory
Develop and implement tools for quality monitoring	Baseline assessment and planningOngoing assessment	Support for implementation built into intervention: Setting up conditions that foster change and decrease participant effort. Provide support for comparison across sites implementing the intervention	Organisational Readiness Theory

ERIC, Expert Recommendations for Implementing Change; IT, information technology.

CMO statements from the organisational readiness theory (step 4)

Weiner defines organisational readiness as a multilevel and multifaceted construct referring to organisational members’ shared commitment to change—encompassing both willingness and capacity.35 This readiness for change is crucial in producing collective engagement; that is achieving buy in and commitment from those at the front lines enacting the change. This engagement results in valuable implementation outcomes: a collective commitment to initiate change, greater effort to make the change successful, greater perseverance when barriers are encountered and an increase in pro-social collaborative behaviours that promote the change.35 Holt, Amenakis and colleagues,36 state the most potent mechanisms were shared perceptions and beliefs among stakeholders in the organisation that (a) they are capable of implementing the proposed change (ie, change-specific efficacy), (b) the proposed change is appropriate for the organisation (ie, appropriateness), (c) leaders are committed to the proposed change (ie, management support) and (d) the proposed change is beneficial to organisational members (ie, personal valence). Perceptions about resources are considered the active means to achieve readiness rather than the resources themselves.35

In an iterative process undertaken by the research team, CMOs were configured, to understand what circumstances (context) needed to be present in an implementation strategy to trigger an identified mechanism leading to an outcome. Since many of the strategies overlapped in their mechanisms and outcomes, we considered them both together and separately. We limited our enquiry to how the mechanisms worked on the implementers within an organisation; that is, the people delivering the intervention directly to patients, rather than the designers or facilitators of the intervention. The outcomes associated with the Theory of Organisational Readiness were all around engagement, buy-in and commitment to the change.

At the same time as the CMO statements were being configured, articles that reported enough detail on these strategies were reviewed for evidence looking for specific contextual factors (external, organisational or individual13) and mechanisms. A further search specifically for implementation projects across multiple sites that reported using organisational readiness theory was also performed, yielding another four papers. The final column of table 6 indicates the articles that give evidence to support or not support the CMO configurations.

Table 6

Context–mechanism–outcome configurations for implementation strategies aligning with Organisational Readiness Theory. The broad context is for individual and collective implementers of large-scale hospital interventions

Implementation strategy (ERIC wording)	Context	Mechanism	Outcome	Component of Organisational Readiness Theory	Evidence from the literature on large-scale hospital projects
Baseline audit results shared with implementers(Audit and provide feedback)	When implementers see their baseline audit results and perceive that current practice is not optimal	…a tension for change is developed leading to	… members being more likely to engage in the project	AppropriatenessPersonal valence	Support10 47
Clear evidence provided on effectiveness of intervention(Audit and provide feedback)	When implementers see clear evidence that the intervention is effective and will improve patient care	… implementers value the change	… members are more likely to engage in the project	AppropriatenessPersonal valence	Support48
	When implementers do not see clear evidence of the effectiveness of the intervention / do not see the link with improved outcomes for patients	… implementers do not value the change	… members are less likely to engage in the project	AppropriatenessPersonal valence	Limited support39 40
Sharing the positive experience of early adopters of the intervention(Create a learning collaborative)	When implementers are told of the success of early adopters at other sites	…a tension for change is developed and perceptions of feasibility at their own site will improve leading to	… members being more likely to engage in the project	AppropriatenessPersonal valence	Support49
A lead-in period is provided when local needs are assessed(Conduct local needs assessment)	When local needs of implementers are assessed before any proposed change	… confidence in capability rises, resulting in greater levels of commitment and collaboration	… resulting in more effective implementation	AppropriatenessChange-specific efficacy	Support 10 31 38 50–52
	When local needs are not accurately assessed (eg, time needed for new practice underestimated)	… confidence in capability falls, resulting in poorer levels of commitment and collaboration	… resulting in poor adoption and outcomes		Not supported37 Support38
Executive and management are engaged and support the intervention(Involve executive boards; obtain formal commitments)	Executive /management support that is visible to the implementers	… increases perceptions of feasibility and organisational capacity	… resulting in increased engagement	Management support	Support10 52–55
	Commitment to support the change from executive level is communicated to implementers	… increases perceptions of feasibility and organisational capacity	… resulting in increased engagement		Support28 48 50–52
	Executive /management support is inadequate or not visible to the implementers	… decreases perceptions of feasibility and value of the change	… resulting in lack of engagement		Support32 38
	Executive /management support is inadequate or distant, but local or within team leadership is seen as strong and autonomous	… does not decrease perceptions of feasibility and value of the change	… and does not impact intention to commit		Supported56
	Executive /management support is inadequate, but local or within team leadership is seen as strong	… increases perceptions of siloed change, decreasing perceptions of feasibility	… resulting in lower staff buy-in and commitment		Supported55
	Executive /management support is inadequate, and local or within team leadership is also inadequate/ non participatory	… decreases perceptions of feasibility and value of the change	… resulting in lack of engagement		Supported55
Support from external agencies/peak bodies for the intervention(Develop academic partnerships; use an implementation advisor; use advisory boards and workgroups)	When external support and/or endorsement of the proposed change is present	implementers may value the change more favourably or feel a greater tension for change	…resulting in increased engagement and commitment	Appropriateness	Support10 27 48 50 53 57
Clear and consistent communication with identified/designated leaders of the intervention(Identify and prepare champions; Recruit, designate, and train for leadership)	Consistent messages and actions from leaders, opinion leaders and champions	… increase perceptions of organisational capacity	… resulting in more effective engagement	Management supportAppropriateness	Support50 57
	Mixed or missed information from leaders, opinion leaders and champions	… decrease perceptions of organisational capacity and disempowerment	… resulting in poorer engagement		Support38 58
Align intervention with other organisational priorities	When the proposed change aligns with other organisational or national priorities	… implementers may value the change more favourably and see their efforts as contributing to a larger, more significant programme	… resulting in more effective engagement	Personal valence Appropriateness	Support48 57
	When the proposed change is part of a collaborative effort across multiple sites	… stakeholders’ perceptions of the value of the change may increase	… resulting in greater commitment		Support27 28 48 59
Align with known concerns/priorities of implementers	When the proposed change aligns with the personal priorities of implementers	…the change is valued more highly by implementers	… resulting in more effective engagement	Personal valence	Support32 51
	When the proposed change does not align with personal or group priorities/ do not make sense	… the value of the change is discounted	…resulting in poor engagement	Individual and group valence	Support58
Provide opportunities for formal and informal planning and knowledge exchange around the intervention(Create a learning collaborative; Capture and share local knowledge)	When there is appropriate and timely information sharing through social interaction, and shared experience	… may increase collective vision and purpose	… resulting in greater engagement and persistence		Support10 38 47 51 52 57
Providing appropriate education(Develop educational materials)	Development of educational packages appropriately pitched at key implementers	…… increase perceptions of feasibility and organisational capacity	… members are more likely to engage in the project	Change-specific efficacy	Support31 47 51 57 59
	Development of educational packages not tailored to specific group’s knowledge base perceived as inappropriate	… decreases perceptions of capability	…members are less likely to engage or commit to the project		Support38
Providing appropriate implementation support(Facilitation: Develop a formal implementation blueprint)	Provision or preparation of implementation blueprints or plans	…… increase perceptions of feasibility and organisational capacity	… members are more likely to engage in the project	Change-specific efficacy	Support31
Appealing to past successes	In spite of previous successes and capabilities, if local needs and capabilities are not considered adequate by those enacting this specific change proposed ….	… collective capability will be seen as deficient	… levels of commitment will be poor	Change-specific efficacy	No evidence found

ERIC, Expert Recommendations for Implementing Change.

Online supplemental file 4 shows the RAMESES checklist for this synthesis. Online supplemental file 5 shows the full search string used in the early steps.

Discussion

In this realist review of implementation strategies for large-scale hospital interventions we have used a four-step process to build a clearer picture of the nature and purpose of implementation and identify likely mechanisms driving intended and unintended outcomes. In the final step we focused on early implementation strategies around baseline assessment and planning to define and test CMO statements explaining outcomes.

In Step 1, we articulated the key concepts associated with implementation programmes of large-scale hospital interventions. Providing practical and social support figured prominently, as did establishing credibility, level of evidence and intended outcomes of the intervention through clear blueprints and collaborative learning and planning activities. While many of the interventions themselves were prescriptive (eg, surgical checklists10–12), the need for implementation to include local needs assessments and tailored activities was also clear. In Step 2, we identified suites of implementation strategies for large-scale hospital interventions and found them to be multifaceted, directed at both individual and organisational levels, and often interdependent. For example, while nearly all the large-scale projects reported education and local leadership, these would only be successful as strategies if they were combined with executive support for the project, and a collective sense of the need for change. It can be argued that the precursor to all implementation strategies is the engagement of the implementers, as without their commitment to change, no substantive change can be achieved. The choice to use Organisational Readiness Theory to further develop the initial programme theories was prompted by this observation.

Organisational Readiness Theory postulates that engagement and commitment to any proposed change will be strongly influenced by individual and collective perceptions around the need for the intervention, its quality and effectiveness, the level of support from management and executive that is apparent and the feasibility of using it. Support for the hypothesised CMOs was found across multiple projects providing strong evidence of the theory’s applicability in large-scale hospital interventions.

Evidence found in our set of literature almost all supported CMOs that led to positive, desirable implementation outcomes of engagement and commitment. There was some refuting evidence that pointed to the interdependence of some factors, and that at times one contextual factor could interact and outweigh another. For example, Wyld and colleagues found that although all stakeholders involved with a new biobank highly valued the initiative, doctors tasked with collecting the samples felt early consultation, management support and consideration of the feasibility for them had been lacking.37 In spite of this, the implementation of the programme had been successful with almost universal adherence to the new processes by the doctors. Possibly, the patients’ altruistic enthusiasm for the initiative, that was often voiced to the doctors during the informed consent process may have put greater value on the initiative, outweighing the doctors’ difficulty.

Evidence for contextual factors that triggered mechanisms leading to poorer outcomes were also found. Bayley and colleagues note the mismatch in perceptions of feasibility found between managers and implementers, and between different healthcare professionals contributing to the multidisciplinary team effort of implementing stroke rehabilitation guidelines.38 This same project found that perceptions of feasibility were also negatively affected by overly complicated statements of the intervention and called for a ‘plain English’ version that would be more accessible for busy clinicians. Both contextual factors were considered barriers as this collective perception of lack of support and lack of feasibility triggered disengagement and lack of commitment to the change.

Some authors, lacking high quality evidence, suggested the cause of poor implementation outcomes might be linked to contextual factors. For example, Reames and colleagues suggest that the failure of their large-scale hospital intervention to effect change might be because it required staff to follow processes that were not strongly associated with clear patient improvements.39 This perception of lack of effectiveness would not trigger the mechanism of building a tension for change, but complacency leading to poor adoption of the intervention. Wand and colleagues found that disagreement voiced by a senior clinician at one site in the early planning stages of an intervention was likely to adversely affect the project’s success there unless it could be resolved.40 Here the perception of implementers would be that the intervention was not feasible or appropriate, and trigger disengagement. While these examples do not claim to be high level evidence, but rather the informed opinion of the authors, they are intuitively correct and consistent with the other evidence found in this study.

Strengths and limitations

Our search for literature was systematic and thorough yet only resulted in 51 papers. This was because while large-scale hospital interventions abound, implementation activities and outcomes are not commonly reported.41 This meant many articles reporting interventions were not relevant to the present study. Even for papers reporting implementation, reporting of these strategies and the contexts in which they were used was often not detailed enough to develop theories. Notable was the lack of accounts of patient involvement in implementation plans. Articles that were found were mostly reporting successful implementations and this is an acknowledged bias of published literature. While CMOs are useful in explaining single factors, multiple contextual factors may arise that modify how mechanisms work. Lack of detail in reporting meant the O (outcomes) in our CMO configurations were high level and dichotomous: implementers were engaged or implementers were not engaged. Another limitation was the need to constrain our search and inquiry to a subset of strategies and a single formal theory. Strengths included the expertise of the research team (including clinical and implementation science expertise) and the systematic four step iterative investigation.

Conclusions

Large-scale hospital interventions hold the promise of standardising high quality, evidence-based care for large numbers of patients but must be supported with appropriate implementation strategies to support and effect change. The study has used realist methodology to tease out how initial planning activities can drive engagement and commitment and delineate the contextual factors required to trigger mechanisms. These findings, using Organisational Readiness Theory, will add to understandings around why large-scale projects work some of the time but not all of the time. Evidence has been presented around a set of CMO hypotheses, showing the importance of implementers’ perceptions around feasibility, support and value in triggering engagement and commitment to the proposed change.