Addressing the evidence to practice gap for complex interventions in primary care: a systematic review of reviews protocol.

INTRODUCTION: Getting the results of research implemented into routine healthcare is often a challenge. The disconnect between the development and implementation of evidence into practice is called the 'second translational gap' and is particularly apparent in primary care. To address this gap, we plan to identify, summarise and synthesise currently available evidence by undertaking a systematic review of reviews to: (1) explore barriers and facilitators of implementation of research evidence or complex interventions, and (2) assess the effectiveness of strategies in facilitating implementation of complex interventions in primary care. METHODS AND ANALYSIS: This is a protocol for a systematic review of reviews. We will search MEDLINE, EMBASE, the Cochrane Library, CINAHL and PsycINFO up until December 2013. We will check reference lists of included studies for further studies. Two authors will independently screen the titles and abstracts identified from the search; any discrepancies will be resolved by discussion and consensus. Full-text papers will be obtained and relevant reviews will be selected against inclusion criteria. Eligible reviews have to be based on predominantly primary care in developed countries and examine either factors to implementation or, the effectiveness of strategies to optimise implementation. Data from eligible reviews will be extracted using standardised data abstraction forms. For barriers and facilitators, data will be synthesised using an interpretative meta-synthesis approach. For implementation strategies, findings will be summarised and described narratively and synthesised using a framework approach. All findings will be reported in accordance with the Preferred Reporting Items for Systematic Reviews and Meta-Analyses guidelines. ETHICS AND DISSEMINATION: Ethical approval is not required. The review findings will inform the work of the design and implementation of future studies and will be of interest to a wide audience including health professionals, researchers, health service or commissioning managers and policymakers. TRIAL REGISTRATION NUMBER: Protocol registration number (PROSPERO CRD42014009410). Published by the BMJ Publishing Group Limited. For permission to use (where not already granted under a licence) please go to http://group.bmj.com/group/rights-licensing/permissions.

This systematic review of reviews aims to produce a comprehensive overview related to the field of implementation of complex interventions in primary care; and it will not be restricted to any topic or health condition.

Comprehensive search strategy; all citations identified from the search will be double screened.

Like all systematic reviews, the search may not identify all relevant literature; this will be minimised by screening reference lists of all included papers for additional literature.

Introduction

Evidence-based medicine is the conscientious, explicit and judicious use of current best research evidence in making decisions about the care of individual patients.1 Traditionally, the practice of evidence-based medicine means integrating individual clinical expertise with the best clinical evidence available from systematic research.1 The application of evidence-based practices is becoming increasingly normative in health systems. Examples of evidence-based products include clinical guidelines and quality indicators for measuring performance.

The term ‘complex intervention’ refers to interventions with several interacting components2 such as an (innovative) product (eg, self-management tool) or set of new processes (eg, referral process, integrated care pathway for people with diabetes) or behavioural and working practices (eg, changing prescribing behaviour). Often it is not clear which of these components is the active ingredient for achieving any intervention effect. Research has consistently shown that many effective interventions are not commonly used in practice. For example, interventions recommended as core treatment (particularly exercise, weight loss and the provision of written information) for knee pain among older adults were underused.3 Despite clinical trial findings showing a clear impact on primary care management of Bell's Palsy, a significant proportion of patients did not receive effective treatment (ie, corticosteroids).4 Conversely, many interventions that have been proven to be ineffective continue to be used, for example, antibiotics for acute respiratory tract infection.5 Estimates suggest that at least 30–40% of patients do not receive care according to current scientific evidence, while 20% or more of the care provided is not needed or potentially harmful to patients.6 This delayed or lack of the translation of evidence-based medicine and complex interventions into every day clinical practice and policy is known as the ‘second translational gap’.7

The process of implementation is complex and it involves activities that occur between making an adoption commitment and the time that an innovation either becomes part of the organisational routine or is discontinued.8 How best to implement complex interventions in clinical practice is a common question raised by academic researchers, healthcare professionals and health service managers. This gap is particularly apparent in primary care9 10 which has its own distinctive research and implementation culture.2 Primary care organisations vary in characteristics such as team composition (eg, number of general practitioners (GPs), practice nurses, practice managers, administrative staff), organisational structures or cultures and working practices; and this diverse context can make it challenging to implement complex interventions.

To bridge the evidence to practice gap in primary care, we need to consider: the causes, usually described as barriers to, and facilitators of implementation; and the effectiveness of strategies in optimising implementation of complex interventions. Implementation strategies are aimed at optimising the uptake and/or implementation of research evidence or complex intervention, by overcoming barriers identified by those charged with implementation (eg, practice nurses, GPs), to ensure fidelity (deliver the intervention as intended). In order to classify the range of possible implementation strategies, we will use the effective practice and organisation of care (EPOC) taxonomy of intervention aimed at achieving practice change,11 which includes the following categories: professional, organisation, structural, financial and regulatory interventions.11 Further details about the EPOC are available in data online supplementary file 1.

The purpose of this study is to summarise and synthesise published review literature on the causes of, and the methods for, closing the second translational gap in primary care. The rationale for undertaking a systematic review of reviews is that although there are already a large number of published reviews in the field of implementation, they are often based on one particular topic or health condition12 13 and it can be hard for health service managers, healthcare professionals or novice researchers to access a comprehensive and comprehensible summary of evidence which would help them plan appropriate, evidence-based implementation strategies. Undertaking a review of reviews allows new syntheses and interpretations of a large amount of data across topics. We will also be able to identify areas where the available evidence is insufficient and further research is indicated.

In order to make the findings more usable and applicable for the UK primary care setting, we are focusing attention on primary care (defined as general practices and primary care teams, including GPs, practice nurses, non-clinical staff such as practice managers or administrative staff) in developed countries.

Methods and analysis

This will be a systematic review of reviews including data from qualitative and quantitative reviews.

Criteria for considering studies for review

Review question (RQ) 1: What barriers and facilitators impede or enable the implementation of research evidence and complex interventions in primary care?

Review question (RQ) 2: What is the effectiveness of different strategies in facilitating implementation of complex interventions in primary care?

Overall effectiveness

In what circumstances? For whom, how and why?

Search methods for identification of studies

A comprehensive electronic search will be performed in MEDLINE, EMBASE, the Cochrane Library, Cumulative Index of Nursing and Allied Health and PsycINFO. The search will be conducted by the primary reviewer (RL), supported by a specialist librarian (RP). Search strategy will be developed using medical subject headings, for example, ‘translational medical research’, ‘evidence-based practice’, ‘general practice’, ‘review’, ‘review literature as topic’ and free-text words, for example, evidence to practice, evidence practice gap, family doctor, implementation, adoption, barriers. We provide the search strategy for MEDLINE in data online supplementary file 2. Small modifications of the strategy will be used for the other databases. Articles published up to December 2013 will be eligible for inclusion in this review. Citation searches will be carried out in ISI Web of Science and reference lists of all included articles will be screened for additional literature.

Selection of studies

Titles and abstracts of all the records obtained from the search will be independently double screened to ensure consistency. The primary review author, RL will screen all identified citations (titles and abstracts) for potential inclusion; coinvestigators will act as the second reviewer(s). RL will obtain the full text articles that are potentially eligible; they will be assessed for eligibility against a prespecified inclusion and exclusion criteria by two reviewers (RL, EM). Any discordance or uncertain inclusions will be resolved through discussion between the two reviewers initially and a definitive judgement will be made by retrieval of the full paper, and the involvement of a third reviewer if necessary.

Data extraction and management

Standardised structured data abstraction forms will be developed for both review questions. We will pilot the forms on five articles to assess their usability. Data from the included studies will be extracted by a single reviewer (RL). Contextual information of the reviews (including titles, aims and objectives, setting, review methodology, number of included primary studies, theoretical framework and definitions) will be extracted for both review questions.

RQ1: Each article will be carefully read by the primary reviewer (RL) and a list of barriers and facilitators will be extracted from results and discussion sections of the included papers. We include the discussion because it often contains further interpretation(s) from the reviewer(s), which offers important insights and enhances the richness of the findings in our review of reviews. A purposive sample (based on topic) of 20% of included reviews will be selected, read and checked by the coinvestigators. They will examine critically the extraction performed by RL using a quality assurance form. The coinvestigators will check the accuracy of data extraction, including review characteristics (eg, author, reference, aims and objectives, setting, number and type of primary studies included in the review) and list of barriers and facilitators.

RQ2: Owing to the substantial literature relevant to this review question, a systematic process is being developed that will enable us to manage the review more effectively, similar to the methodology described in the review by Campbell et al.14

Step 1: sorting papers

First, the identified eligible reviews will be put into different implementation strategy categories according to the EPOC taxonomy. The categorisation has been developed by the EPOC review group within the Cochrane Collaboration.11 All the papers in each category will then be sorted chronologically.

Step 2: selection of a ‘benchmark’ review paper

Following that, a ‘benchmark’ (or reference) review paper will be selected for each implementation strategy (eg, reminders). A benchmark paper will be selected based on its completeness (eg, breadth of topic), rigour (eg, reviewing methods) and publication year; most recent/latest review usually reflects the inclusion of a greater number of relevant studies. Other relevant reviews that report on similar topic and inclusion/exclusion criteria but less recent (ie, smaller number of included studies) will be compared against the ‘benchmark’ review paper during synthesis.

Step 3: selection of important outcomes

Published reviews tend to evaluate the effectiveness of the strategies on a large number of outcomes; some may be more relevant or important than the others, in terms of outcome measures of implementation. In order to produce a useful review, we will focus on relevant outcomes. After selecting a benchmark paper, RL will extract a list of outcomes reported in the results from that chosen paper. The coinvestigators will then select and agree on at least one and no more than three important outcomes for each implementation strategy. Any disagreement will be discussed and consensus will be reached.

Step 4: data extraction

An assumption has been made that reviews of similar topic focus and selection criteria (eg, setting, intervention) should identify or include the same primary studies. Based on that assumption, we have decided to fully extract the results of each chosen benchmark review paper. The results of the remaining relevant reviews will be summarised and entered into the synthesis individually at a later stage (see data synthesis RQ2). All included papers will be read carefully and if the subsequent papers have a somewhat different selection criteria (eg, different population or setting, one review may apply a quality filter; and exclude studies that are of high risk of bias) or use different methods of analysis (eg, calculation of mean risk difference, instead of median adjusted risk difference) that deviate from the benchmark review paper, this information will be extracted and used for synthesis.

The coinvestigators will check the accuracy of data extraction (benchmark review paper plus two randomly selected subsequent papers for each strategy), including review characteristics, information about the derivation of the overall effect size or analysis, results (eg, overall dichotomous or continuous outcome measures, active components that contribute to the effectiveness of the intervention) and interpretation (eg, consistency of findings between benchmark and subsequent reviews) using a quality assurance form.

Data synthesis

For both review questions, we will present the characteristics of included reviews and a flow chart that describes the study selection process. We will report our findings in accordance with the Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA) guidelines.15

RQ1

Data will be synthesised using meta-synthesis techniques16 and an interpretive and inductive approach. Meta-synthesis rests on the reviewer’s interpretation of the findings, which may include themes, categories and relationships, arising from the data of the original findings, to produce new interpretations that incorporate the meanings of the included studies.17

The first stage involves identification of themes and concepts. A pilot synthesis will be conducted by taking a sample of 10 reviews. An initial coding framework will be developed through mapping and coding of data. This process will allow the identification of common and recurring elements18 in the literature. Any uncertainty about coding will be discussed between RL, EM, FS and BNO. Each theme and subtheme will be carefully defined (also known as descriptors) to facilitate coding. The second stage involves refinement of themes and concepts, and construction of lines of argument.18 It is an iterative process. The group including all the coinvestigators will examine and discuss the data, the list of descriptors and the initial coding framework, to look for conflicting (refutational) relationships18 through meetings and email correspondences. In addition, a purposive sample of 20% of included reviews will be selected and they will be read and checked thoroughly (double-coding) by coinvestigators who will critically examine the data coding of the data performed by RL. The coding framework will be refined; if required, data will be recategorised from one theme to another, and some themes will be reconfigured. Final discussion and agreement will be reached by all coinvestigators.

RQ2

The heterogeneous nature of reviews (eg, topic or focus, intervention type, number of components, outcome measures) makes it inappropriate to pool the data by undertaking a meta-analysis; therefore, a qualitative approach will be employed. Results will be described narratively and synthesised. A synthesis table will be developed in accordance with our research question:

1. Overall effect size (if provided), presented as risk difference (the difference in outcome after the intervention minus the difference before the intervention) for dichotomous measures, and/or percentage change relative to the control mean postintervention for continuous measures.

Comparison between different strategies (impact)

Single strategy/multiple component strategies versus no strategy, or

Single strategy/multiple component strategies versus another strategy

Comparison within strategy (consistency)

Benchmark review versus other subsequent review(s)

Findings will be summarised, compared and contrasted across reviews. If a subsequent review reports findings that are considerably different to those reported in the benchmark review, we would attempt to explore why and what causes this discrepancy, by looking closely at the population or setting, intervention, comparison, outcomes, study types, number of included studies, type of analysis, as well as targeted behaviour.

Descriptions of the size of effect are variable across reviews; hence, we have used the definitions proposed by Grimshaw et al19 for dichotomous outcomes:

‘Small’ to describe effect sizes ≤5%

‘Modest’ to describe effect sizes >5% and ≤10%

‘Moderate’ to describe effect sizes >10% and ≤20%

‘Large’ to describe effect sizes >20%

2. Differential effects of implementation strategies on process or professional outcomes, by targeted behaviour (eg, prescribing versus diabetes or cardiovascular management).

3. Active components or features that make implementation strategies successful – why and how it works.

Assessment of methodological quality

Owing to the fact that we are aiming to describe and synthesise a body of qualitative literature, and not determine an effect size, we will not conduct quality appraisal of the included studies in this part of the review as it will not affect this interpretive synthesis.

The PRISMA checklist will be used to critically appraise the quality of reporting of the included benchmark review papers. PRISMA is a 27-item checklist consisting of preferred reporting items for systematic reviews and meta-analyses and it is focused on randomised trials and quantitative data.15

Discussion

To the best of our knowledge, this is the first systematic review of reviews that will attempt to produce a comprehensive overview related to the field of implementation in primary care; and this review will not be restricted to any clinical topic or focus. We will examine extensively the gap between evidence and practice by reviewing, summarising and synthesising the available literature. We will explore factors that inhibit or enable implementation of research evidence together with a wide range of complex interventions across different disciplines and assess the impact of different implementation strategies. This will enable the findings to be maximised and enhance utility to a number of audiences, including researchers who wish to design or conduct a complex intervention study or trial in primary care, healthcare professionals, commissioning and health service managers who wish to implement a new product or process, as well as policymakers. Gaps in knowledge and future directions for research will also be identified. Potential limitations include the possibility that not all relevant primary research studies will be captured by included reviews, so some findings may be missed by concentrating on reviews.

Given the rapid pace of change in the National Health Service and the current drive of the Clinical Commissioning Groups to implement more effective and cost-effective interventions, this review will provide a deeper understanding of how to bring about change and optimise the uptake or implementation of complex interventions in primary care, in order to improve service delivery and patient care.

Participants	General practice/primary care teams in developed countriesReviews that include ≥50% original studies based on primary care (general practices, primary care teams)
Intervention	Implementation of research findings/complex interventions in practice
Comparator	Not applicable
Outcome	Barriers and facilitators (mainly qualitative data)
Study type(s)	Reviews that provided descriptions of methods, including systematic reviews, narrative reviews, meta-analyses, meta-syntheses, meta-ethnographies

Participants	As above
Intervention	Implementation of complex interventions that focus on changing health professional behaviours or practices, by using single or multifaceted implementation strategies (either as a component of the intervention or an add-on)Exclusion: complex interventions targeted at patients
Comparator	Standard implementation processes, usual care, control or no strategy, another implementation strategy (single or multifaceted)
Outcome	Degree of implementation, integration, embedding and normalisation. For example, measures of process of care (eg, referral rates or appropriateness of referral) and professionals’ behaviour or performance (eg, prescribing, adherence to guidelines)
Study type(s)	As above