Effectiveness of implementation strategies for the improvement of guideline and protocol adherence in emergency care: a systematic review.

OBJECTIVE: Guideline and protocol adherence in prehospital and in-hospital emergency departments (EDs) is suboptimal. Therefore, the objective of this systematic review was to identify effective strategies for improving guideline and protocol adherence in prehospital and ED settings.
DESIGN: Systematic review. DATA SOURCES: PubMed (including MEDLINE), CINAHL, EMBASE and Cochrane.
METHODS: We selected (quasi) experimental studies published between 2004 and 2018 that used strategies to increase guideline and protocol adherence in prehospital and in-hospital emergency care. Pairs of two independent reviewers performed the selection process, quality assessment and data extraction.
RESULTS: Eleven studies were included, nine of which were performed in the ED setting and two studies were performed in a combined prehospital and ED setting. For the ED setting, the studies indicated that educational strategies as sole intervention, and educational strategies in combination with audit and feedback, are probably effective in improving guideline adherence. Sole use of reminders in the ED setting also showed positive effects. The two studies in the combined prehospital and ED setting showed similar results for the sole use of educational interventions.
CONCLUSIONS: Our review does not allow firm conclusion on how to promote guideline and protocol adherence in prehospital emergency care, or the combination of prehospital and ED care. For ED settings, the sole use of reminders or educational interventions and the use of multifaceted strategies of education combined with audit and feedback are all likely to be effective in improving guideline adherence. © Author(s) (or their employer(s)) 2018. Re-use permitted under CC BY-NC. No commercial re-use. See rights and permissions. Published by BMJ.

A strength of this systematic review is the extensive and systematic search of the literature and the inclusion of controlled study designs.

A second strength is the overview of strategies used, and their effectiveness in improving adherence to guideline and protocols in prehospital and emergency department settings.

A third strength is the use of a systematic and careful selection process, quality assessment and data extraction, all performed by pairs of independent reviewers.

A limitation of this study is the absence of a meta-analysis, which was impossible due to heterogeneity of designs, guidelines, methods and outcomes used in the studies.

A second limitation is that most of the identified studies came with a high risk of performance and detection bias.

Background

Clinical practice guidelines and protocols are developed to improve quality of care, to reduce variation of practice and to ensure that evidence is actually used when appropriate.1 A guideline consists of systematically developed recommendations to assist practitioners and patients in decisions about appropriate healthcare for specific clinical circumstances.2 A guideline recommendation is defined as any statement that promotes or advocates a particular course of action in clinical care.3 To assist implementation of guidelines, a protocol can be developed which yields a specification of a guideline and exactly formulates how to act and what steps to follow.4

Similar to other settings, guidelines and protocols have become an important aspect of prehospital ambulance care provided by physicians, paramedics and ambulance nurses (from now on referred to as prehospital), and in hospital-delivered emergency care (from now on referred to as ED) provided by emergency physicians and emergency nurses.5 6 Although patient-specific circumstances might lead to guideline deviations, in most cases, improved guideline adherence improves patient outcomes.7–9 Yet a relatively recent (2013) systematic review on guideline adherence in the prehospital and ED settings showed adherence rates to be highly variable and ranging from 7.8% to 95% for guideline adherence rates in the prehospital setting and 0%–98% for guideline adherence rates in the ED setting.10 Studies published after this systematic review show similar ranges.11–14 These results indicate that patients might be exposed to undertreatment or overtreatment and related risks.

To improve guideline adherence, it is important to identify implementation strategies which have been shown effective in improving adherence to guidelines and protocols. A commonly used classification for strategies is the EPOC Taxonomy (see online supplementary appendix 1 for selected examples of strategies and their definitions from the EPOC Taxonomy).15

General literature on healthcare delivery improvement shows that opinion leaders, educational meetings, printed educational materials, educational outreach visits, and audit and feedback show positive effects in improving professional practice, although effects are small and do not always occur.16–20 To successfully select or develop implementation strategies for improved healthcare delivery and to optimise chances of effectiveness, it is generally recommended to tailor strategies to identified barriers and facilitators.21 Although combining and tailoring of interventions seems logical, literature is not totally decisive on the effectiveness of combining strategies as compared with the sole use of strategies.22 23 Tailored interventions can change professional practice with small to moderate effects,24 which might be due to a mismatch between the identified barriers and the designed or selected intervention.25 In addition, evidence is not compelling that a multifaceted intervention is more effective than a single-component intervention.26

Though the implementation literature at large offers guidance on the effectiveness of implementation strategies, for prehospital and ED settings it is unknown which strategies are effective to improve guideline and protocol adherence.27 As guideline adherence and implementation in prehospital and ED settings are influenced by multiple factors of which many are context specific,28–30 and as there are time constraints that might influence guideline adherence and the effectiveness of implementation strategies, a review for prehospital and ED settings is necessary. Therefore, the objective of this systematic review is to provide an overview of implementation strategies used, and to assess the effectiveness of these strategies in the improvement of guideline and protocol adherence in emergency care settings (including prehospital emergency care and in-hospital ED settings). More specifically, this review looks at the content and the effectiveness of single and/or multifaceted implementation strategies in relation to healthcare professionals’ adherence to guidelines/protocols in prehospital and emergency care settings.

Methods

A systematic review of the literature was performed according to the steps of the Cochrane Handbook for Systematic Reviews of Interventions.31 This review is reported in concordance with the Preferred Reporting Items for Systematic reviews and Meta-Analyses (PRISMA) statement.32

Literature search

First, the Cochrane database for systematic reviews and the DARE database were checked for a similar review. Second, systematic searches were performed in PubMed (including MEDLINE), CINAHL and EMBASE in April 2018. Search strategies were structured to represent ‘terms for professionals’ OR ‘terms for settings’ AND ‘terms for strategies’ AND ‘terms for adherence’ AND ‘terms for guidelines/protocols’. Full search strategies per database are given in online supplementary appendix 2. Searches were restricted by year of publication (≥2004) to avoid outdated studies from the perspective of progress in prehospital and emergency care treatment and care. In addition to the electronic searches, we used included articles as a source to identify relevant studies by searching the reference lists.

Study inclusion criteria

We included study reports published in any peer-reviewed scientific journal between 1 January 2004 and 10 April 2018. As for designs, (cluster-) randomised controlled trials and quasi-experimental designs (no random allocation) that described the use of one or more strategies to improve adherence to guidelines or protocols in the prehospital ambulance care and ED settings were considered for inclusion. The sole inclusion of (quasi) experimental designs is recommended for implementation effectiveness studies.33 Conference abstracts, editorials, personal communications or unpublished studies were excluded. Studies using simulation, training manikins or self-report methods were excluded since they insufficiently represent actual clinical practice and incorporate a risk of overestimation.34 The guideline or protocol described in the studies had to be a national or international evidence-based emergency care guideline or protocol. We included studies concerning all types of medical conditions/procedures. Local guidelines/protocols were excluded as their evidence base is often less explicit and/or not transparently reported.

Study selection

Pairs of two reviewers (RHAE, FS, URM, LCMV, TvA) independently screened the search results on title and abstract. Differences were then discussed and resolved, and in case of doubt, a third reviewer was asked. Articles were included if the title or abstract described attempts at improving adherence to a guideline or protocol in a prehospital or emergency care setting. The remaining articles were screened full text by pairs of two reviewers (RHAE, FS, URM, LCMV, TvA) independently. In addition, reference lists of included articles were screened (RHAE, FS) and potentially relevant publications were screened in a similar way.

Outcome measures and data extraction

For data extraction, we were guided by the Cochrane Effective Practice and Organization of Care Review Group (EPOC) Data Collection Checklist. From the EPOC perspective, this review focuses on assessment of change at the process level of care provided to patients. Therefore, one of the outcome measures of the study had to include professionals’ adherence to guidelines or protocols measured by observations or registration, and for which self-reporting was excluded. For transparent and uniform reporting on implementation strategies, we mapped the implementation strategies described in the studies to the EPOC taxonomy (2015 version). Though this enabled our overview and interpretation of results, some of the EPOC’s use of terminology and definitions can be debated. EPOC for instance defines audit and feedback as “A summary of health workers’ performance over a specified period of time, given to them in a written, electronic or verbal format. The summary may include recommendations for clinical action”. Given this definition, we did not classify instant feedback as generated through resuscitation devices (eg, instructing the professional to adjust the frequency or depth of heart massage) under this label, as the definition excludes immediate feedback. Instead, we decided this was closest to the EPOC definition of reminders, being “Manual or computerized interventions that prompt health workers to perform an action during a consultation with a patient, for example computer decision support systems”. However, we recognise that instant feedback, as generated and delivered through resuscitation devices, would fit the definition of feedback as found in other, more theoretically informed classifications such as the Taxonomy of Behavior Change Methods,35 which defines feedback as “Giving information to individuals and environmental agents regarding the extent to which they are accomplishing learning or performance, or the extent to which performance is having an impact”.

We extracted the following study characteristics: setting, design, methods, involved organisations, professionals targeted, units of measurement, the guideline and the type of intervention/improvement strategy used. Data were extracted by two independent researchers (RHAE, FS, LCMV). If these data extractors disagreed, the difference was discussed and they tried to reach consensus. If consensus was not reached, a third reviewer (LCMV and TvA) decided. The EPOC taxonomy15 was used to classify the strategies in well-defined categories, which was again done by a pair of two independent reviewers with the same procedure to reach a final decision (RHAE, FS, URM, TvA).

Intervention reporting

Studies describing usage of all types of implementation interventions to improve adherence to guidelines and protocols were included; no interventions were excluded. Interventions were analysed and reported to conform to the Workgroup for Intervention Development and Evaluation Research (WIDER) recommendations,36 meaning a detailed description of the intervention, clarification of assumed change process, access to intervention manuals/protocols and a detailed description of active control conditions. All interventions were assessed according to the WIDER recommendations by a pair of two independent reviewers (RHAE, FS, URM, LCMV). In the first round, there was an agreement on an average of 15.7 out of 20 items. After discussion, we reached total agreement. Furthermore, for each intervention, we described if the intervention was tailored at identified influencing factors (barriers and incentives identified before the intervention was developed and applied) and if the intervention was multifaceted or single-component.

Quality assessment of included studies

To assess the risk of bias, we used the ‘risk of bias assessment tool’.31 This tool is a domain-based evaluation to assess selection bias, performance bias, attrition bias, detection bias and reporting bias. For non-randomised studies, the Cochrane collaboration recommends to add additional domains. Therefore, we added the domain randomisation (yes/no). The quality assessment was performed by two researchers (RHAE, FS, URM, TvA) independently. After the first round of quality assessment, there was agreement on an average of 6.7 out of 8 items. After discussion, in the second round of quality assessment, we reached agreement on an average of 7.9 out of 8 items.

Data synthesis and presentation

Due to heterogeneity of the studies with regard to clinical topics, patient populations and outcome assessment, a meta-analysis was not possible. Instead, we extensively analysed and synthesised the studies by describing strategies used and their effects in detail. As most studies used multiple primary outcomes, we only concluded positive effects were found if significant and positive results were reported for at least two-thirds of these outcomes.

Patient and public involvement

There were no patients involved in this study.

Results

Review statistics

The initial electronic searches resulted in 1829 unique hits and successive selection rounds identified eight articles meeting the selection criteria. In addition, three articles were identified after searching the reference lists (figure 1), bringing the total number of articles to 11. During the full-text selection process, 126 articles were screened and 115 were excluded. Reasons for exclusion were no prospective intervention design (n=77), locally developed guidelines or protocols (n=13), adherence is not an outcome (n=11), another setting than prehospital or ED (n=8), having a guideline is the only intervention (n=5) and the full text article could not be retrieved (n=1).

Figure 1

Literature search flow diagram.

Study characteristics

The characteristics of the included studies (n=11) are displayed per setting in table 1. Nine studies were conducted in the ED setting and two were conducted in the combined setting of prehospital and ED. The studies were conducted in North America (n=4), Europe (n=3), Australia (n=3) and Asia (n=1). Professionals included in these studies were ED physicians, residents and medical officers; ED nurses; paramedics; and physician assistants. Four of the studies were monocentric, meaning they included one unit, either one ED or one prehospital Emergency Medical Service, while the rest were multicentric. Of the 11 included studies, nine were (cluster) randomised controlled trials and two were quasi-experimental studies (QESs).

Table 1

Characteristics of included studies (n=11)

First author (year) Country (ref ID)	Design	Methods/data sources	Number of organisations involved	Professionals	Units of measurement	Guideline (publication year)	Strategies	EPOC taxonomy	Tailored (Y/N)	Multifaceted (M)/single component (S)
Prehospital and emergency department
De Luca  (2009)  Italy  37	CRCT	EMS run sheets Computerised ER medical records The Stroke Surveillance System The Hospital Information System	EMS (n=29) ED (n=18)	EMS and ER employees (physicians, nurses and ambulance drivers)	Patients (n=4895) with suspected stroke	Acute treatment of ischaemic stroke: European Stroke Initiative (2004)	1. Training of the employees in the intervention group about applying the emergency clinical pathway procedures 2. ‘On-site training’ in small groups	Educational meeting Educational outreach visits, or academic detailing	N	M
Hutchison  (2009)  Australia  38	QES	Data collection form	ED (n=3)	Paramedics	Patients (n=122) with (suspected) ST-elevation myocardial infarction	American College of Cardiology/American Heart Association guideline for the management of patients with ST-elevation myocardial infarction (2007)	Education Training Structural intervention: field triage system	Educational meetings	N	S
Emergency department
Bin Nisar  (2011)  Pakistan  39	CRCT	Observation form	ED (n=3)	ED physicians	Patients (n=248) with life-threatening situations	ABCD-approach guideline (no date)	Lectures Real-life scenarios using simulations on manikins, equipment, animal specimens and anatomical dummies	Educational meetings	N	S
Dexheimer  (2013)  USA  42	RCT	Medical records Electronic triage application ED patients status board	ED (n=1)	ED physicians ED nurses Respiratory therapists	Children (n=704) with asthma	National Heart, Lung, and Blood Institute guidelines for the diagnosis and management of asthma (1997)	Computerised asthma informatics system for automatic detection and management Paper-based protocol	Educational materials Reminders	N	M
Doherty  (2006)  Australia  40	CRCT	Data collection form/database review	ED (n=8)	ED staff	Patients (n=187) with asthma	New South Wales (NSW) Department of Health guideline for the optimal treatment of chronic respiratory diseases (2003)	Reformatting the guideline Reminders Audit and feedback Educational sessions Implementation team	Educational materials Educational meetings Audit and feedback	N	M
Doherty  (2007)  Australia  47	QES	Data collection form/database review	ED (n=2)	ED physicians Medical officers ED nurses	Patients (n=283) with asthma	NSW Department of Health guideline for the optimal treatment of chronic respiratory diseases (2003)	Reformatting the guideline Reminders Audit and feedback Educational sessions Implementation team	Educational materials Educational meetings Audit and feedback	Y	M
Geurts  (2017)  The Netherlands  43	RCT	Electronic hospital patient record	ED (n=1)	ED nurses	Children (n=222) with acute gastroenteritis	European Society for Paediatric Gastroenterology, Hepatology, and Nutrition/European Society for Paediatric Infectious Diseases evidence-based guidelines for the management of acute gastroenteritis in children in Europe: executive summary (2008)	Electronic clinical decision-support system Lectures Posters Email+newsletter	Reminders Educational meetings	N	M
Raja  (2015)  USA  44	RCT	Computerised Patient Order Entry Chart review	ED (n=1)	ED physicians	Patients (n=2167) with pulmonary embolism	Guidelines on the diagnosis and management of acute pulmonary embolism: the Task Force for the Diagnosis and Management of Acute Pulmonary Embolism of the European Society of Cardiology (ESC) (2008)	Quarterly performance feedback reports	Audit and feedback	N	S
Rankin  (2013)  Canada  45	RCT	Standard instruments Chart audits Interviews	Multiple hospitals across Canada	ED nurses	Patients (n=367) who were triaged	Revision to the Canadian emergency department Triage and Acuity Scale (CTAS) implementation guidelines (2004)	6-week web-based CTAS workshop	Educational materials Educational meetings Audit and feedback	Y	M
Roy  (2009)  France  41	CRCT	Handheld computer sheets Medical records	ED (n=20)	ED physicians	Patients (n=2637) with clinically suspected pulmonary embolism	ESC guidelines on the diagnosis and management of acute pulmonary embolism (2008)	Handheld clinical decision-support system	Reminders	N	S
Tajmir  (2017)  USA  46	RCT	Medical records	ED (n=1)	ED physicians Physician assistants	Patients (n=613) with foot or ankle complaints	Ottawa Ankle Rules	Clinical decision support	Reminders	N	S

CRCT, cluster randomised controlled trial; ED, emergency department; EMD, Emergency Medical Dispatch; EMS, Emergency Medical Service; EPOC, Cochrane Effective Practice and Organization of Care Review Group; ER, emergency room; QES, quasi-experimental study; RCT, randomised controlled trial.

Prehospital setting

No studies in the prehospital setting were identified.

Combined prehospital emergency care and ED setting (n=2)

Two studies described adherence improvement initiatives in the prehospital and ED setting, one being a cluster-randomised controlled trial37 of relatively good quality, while the other had a quasi-experimental design38 with a higher risk of bias (online supplementary appendix 3). These studies focused on the implementation of guidelines on acute treatment of ischaemic stroke and the management of myocardial infarctions, respectively.

As for the WIDER assessment (online supplementary appendix 4), the first study37 met all criteria for detailed description of the intervention, clarification of assumed change process and design principles, and detailed description of active control conditions. The other study38 only met six out of eight criteria for detailed description of the intervention; criteria for the other WIDER recommendations were not met. Both studies used sole educational implementation strategies and showed improvements in adherence rates (table 2).

Table 2

Results (n=11)

First author (year) Country (ref ID)	Primary adherence outcomes	Statistical test performed	Results (pre-test post-test differences and compared for intervention and control group when possible within design) (+) improvement (−) deterioration (NS) non-significant	Effect
Prehospital and emergency department
De Luca  (2009)  Italy  37	Appropriate referral to stroke unit (ITT analysis) (%) Appropriate referral to stroke unit (PP analysis) (%) Appropriate thrombolysis (ITT analysis) (%) Appropriate thrombolysis (PP analysis) (%)	Relative risk (95% CI) Fisher’s exact test	Intervention-control group at post-test 24.4%–13.1%, RR 2.01, 95% CI 79 to 4.00 (+) 43.2%–13.8%, RR 3.21, 95% CI 1.62 to 4.98 (+) 8.6%–1.7%, p=0.02 (+) 15.1%–1.9%, p=0.001 (+)	+
Hutchison  (2009)  Australia  38	Time to presentation between 8:00 and 17:00 Monday to Friday (in minutes with IQRs) Time to presentation other hours (in minutes with IQRs) Pain-to-balloon time (in minutes with IQRs) Door-to-balloon time (in minutes with IQRs) Door-to-cardiac catheter laboratory time (in minutes with IQRs) Cardiac catheter laboratory-to-balloon time (in minutes with IQRs)	3-way ANOVA with Bonferroni correction Kruskal-Wallis test Mann-Whitney U test χ2 test	Pre-test/post-test 86 (65.5–114)−36.5 (30–66), p<0.001 (+) 107.5 (77–130.5)−62 (54–73), p<0.001 (+) Not reported 98 (73–127)−56 (36.5–70), p<0.001 (+) 69 (40–94.5)−28 (13–39), p<0.001 (+) 28 (23–39)−26 (22–33), p=0.23 (NS)	+
Emergency department
Bin Nisar  (2011)  Pakistan  39	Life-threatening emergency episodes managed according to ABC approach (%)	χ2 test ANOVA (OR) T-test	Intervention-control group at post-test 63.7%–37.1%, OR 2.98. 95% CI 1.78 to 4.99, p=0.0001 (+)	+
Dexheimer  (2013)  USA  42	Asthma education ordered (%) Protocol found on chart (%) Asthma scoring (%) Hospital admission rate (%)	ANOVA Wilcoxon rank-sum test	Intervention-control group at post-test 92%–93% (NS) 18%–1%, p<0.001 (+) Not reported 37%–35% (NS)	−
Doherty  (2006)  Australia  40	Overall adherence (%)	χ2 test Fisher’s exact test	Intervention-control group at post-test 36%–62%, p<0.001 (+)	+
Doherty  (2007)  Australia  47	Overall adherence (%)	χ2 test Fisher’s exact test	Intervention hospital 38%–79%, p<0.01 (+) Control hospital 44%–43% (NS)	+
Geurts  (2017)  The Netherlands  43	Oral/nasogastric ORS rehydration therapy (%)	χ2 test T-test	Intervention-control group at post-test 86%–61% (95% CI 0.78 to 0.92) (+)	+
Raja  (2015)  USA  44	Adherence to total guideline (%)	Not reported	Control group: 78.8%–77.2% (NS) Intervention group: 78.3%–85.2%, p=0.0043 (+)	+
Rankin  (2013)  Canada  45	Accuracy of triage based on Canadian Triage and Acuity Scale (%) Number of undertriaged patients Number of overtriaged patients	T-test Kruskal-Wallis test Fisher’s exact test	Intervention-control group at post-test 72.1%–67.3%, p=0.36 (NS) 9–43, p<0.01 (+) 42–17, p<0.01 (+)	+
Roy  (2009)  France  41	Appropriate diagnostic testing (%)	Fisher’s exact probability test	Difference between intervention group/control group 1. +17.1%, p=0.030 (+)	+
Tajmir  (2017)  USA  46	Patients with ankle injury with appropriate workup with Ottawa Ankle Rules (%) Patients with foot injury with appropriate workup with Ottawa Ankle Rules (%)	χ2 test	Difference between intervention/control group 92.6%–61.8%, p=0.015 (+) 81.0%–63.6%, p<0.001 (+)	+

ITT, intention to treat analysis; ORS, oral rehydration solution; PP, per protocol analysis.

Emergency department setting (n=9)

Nine studies described adherence improvement initiatives in the ED setting. Three were cluster-randomised trials,39–41 five were randomised controlled trials42–46 and one was a quasi-experimental study.47 The quality assessment of the studies in the ED setting indicated overall higher risk for performance and detection bias, and lower risk for selection, attrition and reporting bias (online appendix 3). The guidelines covered in the ED studies related to the ABCD approach,39 asthma,40 42 47 pulmonary embolism,41 44 triage,45 acute gastroenteritis43 and foot/ankle complaints.46

None of the studies conducted in the ED setting fulfilled all four WIDER recommendations (online appendix 4). Only one study met all criteria for recommendation 1 (detailed description of the intervention), and the majority of the studies reported the characteristics of the recipients, the setting, the mode of delivery, the intensity and a detailed description of the intervention content provided for the intervention group. Two studies met all criteria for clarification of assumed change process and design principles, with the rest of the studies reporting either the development of the intervention or the change techniques used while failing to provide details on the causal processes targeted by these techniques. Three articles offered access to intervention manuals or protocols, either within the article itself or through a reference to an earlier published article. Out of all WIDER recommendations, criteria for the detailed description of active control conditions were met the least. Only one study reported on all criteria for the detailed description of active control conditions.

In the ED setting, eight out of nine studies showed positive effects on guideline adherence.39–41 43–47 Six studies used educational components in their implementation strategy.39 40 42 43 45 47 Of these, three combined education with audit and feedback, and all showed improved guideline adherence.40 45 47 Two other studies combined education with reminders,42 43 one of which showed positive effects on guideline adherence.43 One study solely used educational implementation strategies and showed improvement in adherence.39 Two studies used reminders as a sole intervention and showed improvements in adherence.41 46 Finally, one study used audit and feedback as a sole intervention and showed positive effects on guideline adherence.44

Tailoring in relation to outcome

In our review, only two studies used strategies tailored to preidentified determinants for adherence.45 47 Both studies showed significant improvements in adherence.

Discussion

This systematic review attempted to give an overview of strategies used in improving adherence to guideline and protocols in emergency care and their effectiveness. In total, 11 studies were identified.

The studies performed in the ED setting indicated that educational strategies such as educational meetings or distribution of educational materials as sole intervention or in combination with audit and feedback are probably effective in improving guideline adherence. We found similar results in the two studies of good to moderate quality, performed in the combined settings of prehospital and ED care; here, sole use of education also improved guideline adherence. Whether studies used educational meetings and/or distributed educational materials did not seem to make a difference to these results. Our findings on the sole use of education seem in contrast with systematic reviews indicating that educational meetings alone are not likely to be effective for changing complex behaviours17 or that when used alone, printed educational materials may have only small beneficial effects on professional practice outcomes.18 Perhaps this is due to limited awareness or knowledge of existing guidelines and protocols among emergency care professionals,28 awareness that can be enhanced by the sole use of education.

Another finding from our review is that sole use of reminders to improve guideline and protocol adherence in the ED was effective. This finding can be understood from general findings on using reminders to improve healthcare delivery indicating that the effect sizes of reminders are larger than those of most other strategies.48 Also, reminders included in the projects on ED care were usually automated and very instant and specific reminders, which could have positively contributed to their effectiveness.

The identified studies performed in the ED setting also indicated that combining implementation strategies is often effective in increasing adherence to guidelines and protocols. This finding is in line with conclusions of several other reviews which already concluded that multifaceted implementation strategies are likely to improve practice49 and that addressing combinations of different determinants shows good results.50 The latter of these two reviews addressed adherence to hand hygiene prescriptions in hospital nurses and indicated that effect sizes went up with each additional determinant of adherence being addressed by the implementation approach. Whereas Huis et al found that using strategies addressing up to five determinants of practice was effective, we found that fewer strategies per implementation approach could also be effective. In our review, studies reported on one or two different strategies in their approach. Also, the variation in strategies was relatively limited (n=5 unique strategies: educational meetings, educational material, educational outreach visits/academic detailing, reminders, and audit and feedback) in relation to the range of options as indicated in taxonomies of implementation strategies (eg, over 100 types of strategies in EPOC subcategories).

Our review also showed that most of the studies identified through our systematic search were conducted in the ED setting, and fewer in the combined settings of prehospital and ED care. Although guidelines and protocols are an important aspect of prehospital care and guideline adherence can be suboptimal,10 we did not identify rigorous research on effective implementation strategies. Yet, the need for well-designed experimental studies on the effectiveness of such strategies is clear from previous studies describing that there is a time gap between publication of a guideline and actual usage and adherence.51–53

A final finding from our review is that only two studies tailored implementation strategies to previously identified factors influencing adherence. This is despite the general recommendations and evidence for the need to tailor strategies,21 24 and also supports the relevance of conducting rigorous research on the identification of influencing factors for guideline adherence in emergency care settings.29

Limitations of included studies

A first limitation is that most of the included studies came with high risk of performance and detection bias, although risk for selection, attrition and reporting bias was lower. Also, no studies addressed strategies to improve guideline adherence in the prehospital setting, and only two studies addressed the mixed prehospital and ED settings. The need to assess effectiveness in these (combined) settings is urged by growing attention for implementation in (combined) emergency care settings.54–56 In addition, we considered reporting on guideline and protocol adherence after discontinuation of implementation strategies to assess sustainability of effects. However, this was not possible given the limitations of the included studies. Several studies had no follow-up other than directly after strategy delivery, several studies mentioned a follow-up but not its results, and studies that reported on follow-up used highly variable follow-up periods and formats (reporting on a single point in time vs reporting on adherence over the period of a whole year).

With regard to the WIDER recommendations, overall reporting on the nature of the implementation strategies used in interventions group was suboptimal, and descriptions of what was done in control groups was poor. Regarding the detailed description of the intervention, most of the studies described the characteristics of recipients, the setting, the mode of delivery, the intensity, the duration and a detailed description of the content (criteria 2–6, 8). The characteristics of those delivering the intervention and adherence to delivery protocols were less well described. Only two studies gave information relevant to all criteria for the clarification of assumed change processes and design principles, with item B (the change techniques used) being described the most. Only three studies provided access to intervention manuals/protocols. As for description of the control conditions, three studies described eight or nine criteria; the other studies described less or no criteria. This poor reporting of change strategies is reported in literature,57 58 and limits the replicability and generalisability of used strategies.

Study strengths and limitations

We conducted our review in concordance with the EPOC taxonomy, used the PRISMA statement for transparent reporting and applied the WIDER recommendations to report the interventions. Our procedures were up to date and in complete concordance with these tools. Also, we had a high inter-rater reliability during the assessment of quality of the studies included in this systematic review, with an average agreement of 6.9 for the eight items. Usage of the EPOC taxonomy (2015 version) implies that our results and conclusions should be seen in the light of the EPOC taxonomy as a reference for use of terminology. Although the EPOC taxonomy is widely applied, other taxonomies of implementation strategies and behaviour change techniques are developed and described.59–62 A limitation of the EPOC relates to the absence of clear links to relevant theory or determinants of practice for many of its strategies. For this, some of the alternative taxonomies could provide more guidance.

As for limitations, we were unable to do a meta-analysis due to a high heterogeneity of guidelines and methods used in the studies. In relation to this, we looked for significant effects in study outcomes and did not so much focus on effect sizes, which was also hindered by incomparability of outcomes and the way they were reported. This implies that we were able to arrive at general conclusions, but perhaps not to cover all relevant details on effects. Furthermore, the cut-off point of at least two-thirds of the outcomes showing significantly positive improvements was used to conclude on positive results, but the exact cut-off point is necessarily arbitrary. Another limitation concerns the fact that we were not able to retrieve the relationship between length of time following guideline implementation and outcomes.

Finally, excluding studies on the implementation of local guidelines and protocols from our review might have influenced our results. We excluded these studies as (1) from perspective of evidence-based practice only guidelines and protocols with a solid evidence base should be implemented, (2) the generalisability of local guidelines and protocols might be less clear or restricted to the specific context, and (3) as findings from these studies might not be as informative to professionals outside this specific context. We can only speculate as to how this could have influenced our results.

Conclusions and further recommendations

There is a lack of rigorous experimental studies to inform the promotion of guideline adherence in the prehospital care setting, while there is limited evidence for the ED setting. For ED settings, sole use of educational interventions, sole use of reminders, and using education combined with audit and feedback are all likely to improve adherence to guidelines. As reporting on change strategies is poor, replicability of implementation strategies is limited. Finally, factors influencing guideline adherence in emergency care settings are little considered in care quality improvement studies, thus hindering tailoring the implementation strategies.

Based on these conclusions, we offer several suggestions for research and practice. First, we have found that implementation strategies targeted at guidelines adherence in the prehospital settings and in the combined chain of care are limited. Thus, in the future, more studies targeted at prehospital care and across emergency care settings are needed. As for study quality, performing more rigorous RCTs or QES studies should be encouraged for all settings in order to reduce the risk of bias. In addition, change strategies should be reported using the WIDER recommendations. Second, our results indicate that further research on effectiveness of tailored implementation strategies in prehospital and emergency settings should be encouraged. Third, reminders seem to be an important change strategy when it comes to increasing adherence to the guidelines and protocol at the ED, given their usually larger effect size. Integrating this strategy more frequently may in fact improve results. Finally, a rather limited range of strategies for improving healthcare practice seems to be used in emergency care. Some creativity in the selection of implementation strategies in the sense of using more and different combinations of implementation strategies from the EPOC taxonomy can possibly increase the level of adherence and improve the quality of care further, although tailoring strategies to context should always be taken into account.