Evidence-based interventions to reduce adverse events in hospitals: a systematic review of systematic reviews.

OBJECTIVE: To provide an overview of effective interventions aimed at reducing rates of adverse events in hospitals.
DESIGN: Systematic review of systematic reviews. DATA SOURCES: PubMed, CINAHL, PsycINFO, the Cochrane Library and EMBASE were searched for systematic reviews published until October 2015. STUDY SELECTION: English-language systematic reviews of interventions aimed at reducing adverse events in hospitals, including studies with an experimental design and reporting adverse event rates, were included. Two reviewers independently assessed each study's quality and extracted data on the study population, study design, intervention characteristics and adverse patient outcomes.
RESULTS: Sixty systematic reviews with moderate to high quality were included. Statistically significant pooled effect sizes were found for 14 types of interventions, including: (1) multicomponent interventions to prevent delirium; (2) rapid response teams to reduce cardiopulmonary arrest and mortality rates; (3) pharmacist interventions to reduce adverse drug events; (4) exercises and multicomponent interventions to prevent falls; and (5) care bundle interventions, checklists and reminders to reduce infections. Most (82%) of the significant effect sizes were based on 5 or fewer primary studies with an experimental study design.
CONCLUSIONS: The evidence for patient-safety interventions implemented in hospitals worldwide is weak. The findings address the need to invest in high-quality research standards in order to identify interventions that have a real impact on patient safety. Interventions to prevent delirium, cardiopulmonary arrest and mortality, adverse drug events, infections and falls are most effective and should therefore be prioritised by clinicians. Published by the BMJ Publishing Group Limited. For permission to use (where not already granted under a licence) please go to http://www.bmj.com/company/products-services/rights-and-licensing/

This review offers a unique overview of effective patient-safety interventions based on data from systematic reviews, thereby producing a stronger evidence-based oversight of effective interventions compared to the outcomes of a systematic review of primary studies.

For several patient-safety interventions that are implemented worldwide, there is a lack of high-quality studies in which these interventions are evaluated.

The found estimates of effectiveness of patient-safety interventions might vary across contexts, such as small versus large hospitals, academically affiliated hospitals versus those that are not and the availability of factors that stimulate successful implementation of interventions.

Introduction

Improving patient safety is an ongoing concern for healthcare providers, managers and policymakers. Worldwide, the prevalence of patient harm and death as a result of adverse events is about 10% among hospitalised patients. Half of these adverse events are considered avoidable.1 Despite the widespread implementation of interventions to reduce patient harm, patient safety is not improving.2–4

Substantial effort has been invested into developing and implementing safety improvements.5–7 Patient-safety improvement interventions have been defined as: practices, strategies, structures, procedures, behaviour or actions to prevent or mitigate unintended patient harm, resulting from the healthcare process across a range of diseases and procedures.8–11 Several reviews have studied the nature and effectiveness of a broad range of these patient-safety interventions.5 12–15 However, the findings of these reviews need to be seen in the light of several limitations. The reviews included studies with weak designs, lacking a systematic approach, or were conducted more than a decade ago. Most importantly, none of the reviews reviewed or prioritised patient-safety interventions based on their effects on adverse event and mortality rates. So far, patient-safety interventions have not been reviewed or prioritised based on effect measures.

Better insight into the effectiveness of interventions aimed to reduce adverse events and preventable deaths within hospitals is needed to assist managers and healthcare providers with deliberately selecting patient-safety interventions based on available evidence16 and to disseminate effective patient-safety improvement interventions into routine practice.3 Therefore, the aim of this study was to systematically review systematic reviews of interventions aimed at improving patient safety in hospitals by evaluating interventions, the studies they were tested in and the effect sizes found.

Methods

We conducted this systematic review with a prespecified protocol (see online supplementary appendix 1), in accordance with the preferred reporting items for systematic reviews and meta-analyses (PRISMA) and the AMSTAR (A MeaSurement Tool to Assess systematic Reviews) checklist for systematic reviews (see online supplementary appendices 2 and 3).17 18

Data sources and searches

We searched for systematic reviews from inception to 22 July 2013, using the following scientific databases: PubMed (including MEDLINE), CINAHL, PsycINFO, the Cochrane Library and EMBASE. We used the filters for searching papers on patient safety developed by Tanon et al19 to maximise the sensitivity of our literature search. The search terms used are described in detail in online supplementary appendix 4. We updated the search until 6 October 2015 (see flow chart in figure 1).

Figure 1

Summary of evidence search and selection. *See online supplementary appendix 5 for the exclusion reason per systematic review after full-text selection.

Additional hand searches were conducted in high-impact journals and online databases in the field of patient safety, from April 2010 to May 2015, including: Systematic Reviews Journal, Annals of Internal Medicine, BMJ, BMJ Quality and Safety in Healthcare and the International Journal of Quality in Healthcare. Finally, references from the included systematic reviews and bibliographies of published and unpublished reviews related to our study objective were scanned to identify eligible systematic reviews.

Systematic review selection

Two researchers (MZ and GH) independently assessed the inclusion eligibility of the retrieved systematic reviews according to a standardised format (see online supplementary appendix 1). The initial selection for inclusion was based on the title and abstract of the systematic reviews. A full-text copy of the article was retrieved and reviewed, in case the title and abstract provided insufficient information to determine its relevance. For the final selection, a full-text copy of the systematic reviews was examined to determine whether it fulfilled the inclusion criteria. Disagreement about inclusion was solved by discussion. When no consensus could be achieved, a third reviewer (HW) made the final decision.

Each systematic review had to meet the following criteria (see online supplementary appendix 1):

English-language, full-text published and unpublished systematic reviews;

including any study matching the Cochrane Effective Practice and Organisation of Care (EPOC) criteria for study designs, including: randomised controlled trials, non-randomised controlled trials, controlled before–after studies and interrupted time series;20

focusing on population of hospitalised patients across a range of diseases and procedures;

regarding patient-safety interventions (aimed at changing healthcare processes, structures, strategies, behaviour or actions) targeted at reducing adverse events;

reporting quantitative effect measures.

Systematic reviews that met any of the flowing criteria were excluded from the review:

only obtaining observational studies;

only obtaining pharmacological studies;

only obtaining psychiatric, obstetric patients or neonates as the study population/sample;

only including process errors or consequences of adverse events (eg, readmission and length of stay).

Systematic reviews were included if they included observational studies and studies that met the EPOC criteria. Of these systematic reviews, only the studies that met the EPOC criteria for study designs were studied and were called ‘eligible studies’.

Data extraction and quality assessment

One researcher (WG) extracted the data from the included systematic reviews, using a standardised form (see online supplementary appendix 1). The extracted data were checked by a second researcher (GH). Disagreement was resolved through discussion, and a third person (MZ) was consulted if needed. We limited the data extraction to the prespecified elements, including the intervention components, design and number of included studies, study sample (nature and size) and effect measures. Of all of the studies in a systematic review, only data from studies that met our selection criteria (called ‘eligible studies’) were extracted and analysed.

Three reviewers (MZ, GH and WG) independently assessed the extent to which the systematic review was conducted to the highest possible standards, using a quality assessment form (see online supplementary appendix 1) that included the 11 AMSTAR quality criteria.18 Systematic reviews scored 1 point for each fulfilled criterion, and a total score for each systematic review was calculated. A score of 0–3 was classified as ‘low’, 4–7 as ‘moderate’ and 8–11 as ‘high’.21

Data synthesis and analysis

The study characteristics and patient outcomes for all of the systematic reviews that met our inclusion criteria were organised in a tabular form. The systematic reviews included were classified into patient-safety areas. The classification was adapted from previous reviews on patient-safety interventions.11 12 14

The overlap in primary studies between systematic reviews was studied. Systematic reviews of which all included studies were included in a more recent systematic review (100% overlap) were excluded. We reported the proportion (%) overlap between included systematic reviews per patient-safety area.

We compiled the pooled effect sizes of meta-analyses reported in the systematic reviews and analysed the intervention components. Subsequently, we ranked the effective interventions based on their effect size.

Results

Search results

Our initial search identified 11 032 records (figure 1). The title and abstract scan resulted in 172 articles that underwent full-text review. Thirty-six articles met our selection criteria after the full-text review. The exclusion reasons for the 136 articles are given in online supplementary appendix 5. Four additional articles were identified through hand searching and snowballing, and 20 additional articles were identified through an update of our search action. The final set consisted of 60 articles22–81 that underwent data abstraction and analysis.

Methodological quality

Four (6.7%) systematic reviews scored low, 30 (50.0%) scored moderate and 26 (43.3%) scored high on methodological quality. Their AMSTAR scores ranged from 2 to 10 (see online supplementary appendix 6), with a mean score of 6.9 (SD ±2.2). None of the included systematic reviews fulfilled all of the AMSTAR criteria. Online supplementary appendix 7 shows the proportion of studies satisfying each of the 11 AMSTAR quality criteria. Most (>80%) of the included systematic reviews carried out a comprehensive literature search, reported the characteristics of the included studies, assessed the scientific quality of the included studies and used the scientific quality of the included studies appropriately in formulating conclusions. One-third of the systematic reviews referred to a study protocol in which the research questions and inclusion criteria were established before the study was conducted, and provided a list of included and excluded studies. None of the systematic reviews reported the conflicts of interest of the included studies (see online supplementary appendix 7). Six systematic reviews (10.0%) did not include a statement on the presence or absence of potential conflicting sources of support for carrying out the systematic review.42 45 46 52 68 78

Characteristics of the included systematic reviews

The characteristics of the included systematic reviews are summarised in online supplementary appendix 8. More than half (56.7%) of the systematic reviews were published between 2013 and 2015. The total number of included studies ranged from 267 81 to 138;65 the number of eligible studies (ie, met the inclusion criteria) ranged from 167 80 81 to 33.29 The number of participants in the eligible studies ranged from 93875 to 225 68671 and was not reported or unknown in 26 (43.3%) reviews.

The included reviews covered 14 patient-safety areas (table 1). Most of the reviews were about preventing adverse drug events (n=15), followed by infection prevention (n=8), delirium prevention (n=7) and adverse events after hospital discharge or clinical handover (n=7).

Table 1

Identified systematic reviews (n=60) classified by patient-safety area (n=14)

Patient-safety area		Number of systematic reviews (references)	Intervention components relevant to patient safety (effective components are in bold)
Adverse drug event	Subarea
	CPOE system	222 23	CPOE system
	Medication review	424–27	Medication reconciliation
	Computer-assisted decision support/alerts	328–30	Computerised advice or decision support; computerised drug-laboratory alerts for clinicians on prescribing or monitoring decisions
	Multicomponent interventions	631–36	Multicomponent interventions, including pharmacist involvement and support of care teams or physicians; guideline implementation, including academic detailing, reminders and feedback of data; multicomponent intervention, including CPOE system, changes in work schedules, education, support systems for clinical decision-making
Infection*	Device-related infections (CAUTI; CLABSI; VAP)	437–40	Care bundles and checklists; empowerment to stop procedure; surveillance; infrastructure and organisational changes; training on appropriate catheter placement; catheter restriction and removal protocols; reminder or stop order to decrease catheter placement; use of specific technologies
	Sepsis	141	Multicomponent programme aimed at improving compliance to sepsis care bundles, including education and decision support tools
	Hand-hygiene compliance	242 43	Education; audit and feedback; health promotion; variations in the availability and type of products used for hand hygiene
	Overall hospital-acquired infection	144	Education; protocols to remove catheters
Delirium		745–51	Psychiatric assessment; special care; daily visits by a liaison nurse; interdisciplinary team; supportive psychotherapy; multicomponent intervention, including cognitive screening, proactive geriatric consultation and psychotherapy; multicomponent intervention, including early mobility, cognition and orientation, sleep–wake cycle preservation; multicomponent intervention, including physiotherapy, family involvement and staff/family-member education
Adverse event after hospital discharge or clinical handover		752–58	Postacute intermediate care units; geriatric assessment; liaison nurse; predischarge assessment of risks; patient engagement; individualised patient record; multidisciplinary discharge planning team; clinical follow-up; nurse-led early-discharge planning programmes
Fall		459–62	Addressing risk factors by a multidisciplinary team; care planning; environmental changes; movement alarms; physiotherapy; management of urinary incontinence; multicomponent interventions, including risk alert card, exercise, education, hip protectors and geriatric assessment
Adverse event in surgery		563–67	Screening and decolonisation of surgical-site infections; subspecialisation; benchmarking; technology or training; surgical safety checklist
Cardiopulmonary arrest		468–71	Critical-care outreach service; rapid response teams
Venous thromboembolism		272 73	Alerts and education; real-time audit and feedback; multicomponent interventions to improve appropriate administration of thromboprophylaxis
Staffing		374–76	Increasing proportion of support staff; addition of specialist nursing post to staffing; reducing shift length; protected sleep time; night float; education among residents; interdisciplinary team interventions
Pressure ulcer		177	Standardisation of interventions; multidisciplinary teams and leadership; designated skin champions; education; audit and feedback
Mechanical complication and underfeeding		178	Total parenteral nutrition team: nutrition support for patients who are unable to obtain adequate nutrition either via the oral or enteral route
Clinical pathway		179	Clinical pathways: multidisciplinary care plans with essential steps in care, supporting the translation of clinical guidelines into local protocols and application in practice
Safety culture		180	Error-prevention training; restructured patient-safety governance; lessons-learnt programme; cause analysis programme; executive rounds
External inspection		181	External inspections of compliance with standards (eg, accreditation)

*Surgical-site infections were classified as ‘prevention of adverse events in surgery’.

CAUTI, catheter-associated urinary tract infection; CLABSI, central-line-associated bloodstream infection; CPOE, computerised physician order entry; VAP, ventilator-associated pneumonia.

There was overlap in the included studies between systematic reviews within specific patient-safety areas (see online supplementary appendix 9). The overlap ranges from 25%45 to 86%47 for ‘delirium prevention’ and from 66%62 to 75%59 60 for ‘fall prevention’.

Effects of patient-safety interventions

The results of all included systematic reviews are summarised in online supplementary appendix 10. A meta-analysis was carried out in 30 of the 60 (50.0%) systematic reviews (table 2). The authors addressed the following reasons for not performing a meta-analysis: too few studies identified (n=5); the heterogeneity of the respective study designs (n=9), interventions (n=8), subject groups (n=5) and reported outcomes (n=5); and methodological limitations (eg, lack of available valid data) of the included studies (n=5).

Table 2

Effect sizes of patient-safety interventions: results from meta-analyses (n=30) reported in the 60 included systematic reviews

Patient-safety area	Reference meta-analysis	Intervention	Patient outcome	Effect size (95% CI) significant effect sizes are bold	p Value	Studies in meta-analysis (n) (eligible studies* (n))
Adverse drug eventMedication review	Holland et al24	Pharmacist-led medication review	Mortality	RR 0.96 (0.82 to 1.13)	0.62	22
	Christensen and Lundh26	Medication review	Mortality	RR 0.98 (0.78 to 1.23)	0.86	4
	Hohl et al27	Medication review	Mortality	OR 1.09 (0.69 to 1.72)	0.71	3
Adverse drug eventComputerised advice on drug dosage	Durieux et al28	Computerised advice on drug dosage	Mortality	RR 0.81 (0.37 to 1.81)	0.61	6
	Gillaizeau et al29	Computerised advice on drug dosage	Mortality	RR 1.08 (0.80 to 1.45)	0.61	10
	Bayoumi et al30	Computerised drug-laboratory alerts	Adverse events (bleeding and thrombosis)	OR 0.88 (0.78 to 1.00)	0.05	4
Adverse drug eventMulticomponent interventions	Davey et al33	Intervention for antimicrobial therapy	Mortality	RR 0.92 (0.69 to 1.22)	0.56	3
		Antibiotic guideline for pneumonia	Mortality	RR 0.89 (0.82 to 0.97)	0.01	4
		Decrease excessive prescribing	Mortality	RR 0.92 (0.81 to 1.06)	0.25	11
	Wang et al36	Pharmacist interventions	Preventable adverse drug events	OR 0.23 (0.11 to 0.48)	<0.01	3 (2)
Infections	Blot et al39	Care bundle/checklist interventions	CLABSI	OR 0.39 (0.33 to 0.46)	<0.01	41 (5)
			CLABSI rate at 3 months	OR 0.30 (0.10 to 0.88)	0.03	6 (4)
	Meddings et al40	Catheter reminder and stop order	CAUTI episodes per 1000 catheter days	RR 0.47 (0.30 to 0.64)	<0.01	11 (1)
			CAUTI	RR 0.72 (0.52 to 0.99)	0.05	8 (2)
	Damiani et al41	Sepsis bundle	Mortality	OR 0.66 (0.61 to 0.72)	<0.01	48 (3)
Delirium	Hempenius et al47	Multicomponent interventions, including cognitive screening, proactive geriatric consultation and psychotherapy	Incidence of delirium	OR 0.58 (0.38 to 0.92)	NR	5
		One-component interventions	Incidence of delirium	OR 1.05 (0.09 to 11.57)	NR	2
	Hshieh et al50	Multicomponent intervention, including early mobility, cognition and orientation	Incidence of delirium	OR 0.47 (0.38 to 0.58)	<0.01	11 (7)
	Martinez et al51	Multicomponent intervention, including physiotherapy, daily reorientation, family involvement and staff/family-member education	Incidence of delirium	RR 0.73 (0.63 to 0.85)	<0.01	7
Adverse event after hospital discharge or clinical handover	Griffiths et al52	Nursing-led inpatients units	Mortality	OR 1.10 (0.56 to 2.16)	0.64	7
			Mortality 3 or 6 months post admission	OR 0.96 (0.63 to 1.47)	0.62	6
	Conroy et al53	Comprehensive geriatric assessment	Mortality	RR 0.92 (0.55 to 1.52)	0.77	5
	Niven et al54	Critical-care transition programmes	Mortality	RR 0.84 (0.66 to 1.05)	0.1	3 (2)
	Shepperd et al56	Discharge planning from hospital to home	Mortality at 6–9 months	RR 1.00 (0.79 to 1.26)	0.69	6
			Falls	RR 0.87 (0.50 to 1.49)	0.61	1
	Lowthian et al57	Optimised ED discharge	Mortality up to 18 months postdischarge	OR 1.01 (0.70 to 1.47)	0.94	2
	Zhu et al58	Nurse-led early-discharge planning	Mortality	RR 0.70 (0.52 to 0.95)	0.02	5
Fall	Oliver et al59	Multicomponent intervention	Falls	RaR 0.82 (0.68 to 1.00)	NR	12
			Fallers	RR 0.95 (0.71 to 1.27)	NR	12
			Fractures	RaR 0.59 (0.22 to 1.58)	NR	12
	Coussement et al60	Multicomponent intervention	Falls	RR 0.82 (0.65 to 1.03)	NR	4
			Number of fallers	RR 0.87 (0.70 to 1.08)	NR	4
	Cameron et al61	Multicomponent interventions	Rate of falls	RaR 0.69 (0.49 to 0.96)	0.03	4
			Risk of falling	RR 0.71 (0.46 to 1.09)	0.12	3
		Exercises	Risk of falling	RR 0.36 (0.14 to 0.93)	0.04	2
Adverse event in surgery	Bergs et al66	WHO surgical safety checklist	Any complication	RR 0.59 (0.47 to 0.74)	<0.01	5
			Mortality	RR 0.77 (0.60 to 0.98)	0.04	4 (3)
			Surgical-site infections	RR 0.57 (0.41 to 0.79)	<0.01	5
Cardiopulmonary arrest	Chan et al69	Rapid response team	Mortality	RR 0.92 (0.82 to 1.04)	NR	16
			Cardiopulmonary arrest	RR 0.65 (0.55 to 0.77)	NR	16
	Maharaj et al71	Rapid response team	Mortality	RR 0.91 (0.85 to 0.97)	<0.01	4
			Cardiopulmonary arrest	RR 0.74 (0.56 to 0.98)	0.04	2
Venous thromboembolism	Kahn et al72	Alerts	All venous thromboembolism	RR 0.85 (0.49 to 1.46)	NR	3
		Multicomponent interventions	All venous thromboembolism	RR 1.01 (0.51 to 1.98)	NR	5
			Symptomatic deep vein thromboembolism	RR 0.59 (0.18 to 1.98)	NR	3
Staffing	Butler et al75	Addition of specialist nursing post to staffing	In-hospital mortality	RR 0.96 (0.59 to 1.56)	0.86	1
			Postdischarge adverse events	RR 1.03 (0.70 to 1.53)	0.87	1
		Increasing the proportion of support staff	Mortality in trauma unit	RR 0.41 (0.16 to 1.01))	0.05	1
			Mortality in hospital	RR 0.56 (0.29 to 1.09)	0.09	1
			Mortality at 4 months	RR 0.57 (0.34 to 0.95)	0.03	1
	Pannick et al76	Interdisciplinary teams	Mortality	wRR 0.92 (0.82 to 1.05)	NR	7
		Team practice interventions	Mortality	wRR 0.67 (0.45 to 0.99)	NR	2
Clinical pathway	Rotter et al79	Clinical pathway	Mortality	OR 0.84 (0.64 to 1.11)	0.23	3
			Complications up to 3 months	OR 0.31 (0.13 to 0.72)	0.07	1
			In-hospital complications	OR 0.58 (0.36 to 0.94)	0.03	5

*Study design in accordance with methodological criteria of the Cochrane EPOC review group and quantitative data on adverse event rates were reported.

CAUTI, catheter-associated urinary tract infection; CLABSI, central-line-associated bloodstream infection; EPOC, Effective Practice and Organisation of Care; NR, not reported; RaR, rate ratio; RR, risk/relative ratio; wRR, weighted risk ratio.

Seventeen meta-analyses showed a statistically significant effect on adverse drug events,36 catheter-associated urinary tract infection (CAUTI) rates,40 central-line-associated bloodstream infection (CLABSI) rates,39 delirium incidence,47 50 51 fall rates,61 surgical-site infections,66 incidence of cardiopulmonary arrest,69 71 complications66 79 and mortality rates.33 41 58 66 71 75 76 Patient-safety interventions with statistically significant effect sizes are discussed below.

Adverse drug event

Of the 15 included systematic reviews about adverse drug events, 2 reported statistically significant results. Davey et al33 found that interventions aimed at increasing antibiotic guideline compliance for pneumonia were associated with a significant reduction in mortality: risk ratio (RR) 0.89 (95% CI 0.82 to 0.97; p=0.01). This found effect was based on four studies. Effective intervention components were formal presentations, academic detailing, letters, frequent reminders by pharmaceutical representatives, preprinted outpatient and admission order sheets and reporting of outcome data to providers.

Wang et al36 found that participation of a pharmacist in physician rounds and timely information exchange and advice of physicians by the pharmacist (ie, on drug interactions, appropriate dosages, dose intervals and routes of administration) was associated with a statistically significant reduced adverse-drug-event rate: OR 0.23 (CI 0.11 to 0.48; p<0.01). The found effect was based on three studies, of which two complied with the Cochrane EPOC inclusion criteria for study designs.

Infection

Three systematic reviews reported statistically significant effects on the reduction of infection and mortality rates as a result of implementing interventions and care bundles.39–41 The meta-analysis performed by Blot et al39 showed a reduction in the CLABSI rate (OR 0.39 (CI 0.33 to 0.46; p<0.01)) and reduction in the CLABSI rate at 3 months post intervention (OR 0.30 (CI 0.10 to 0.88; p=0.028)) as a result of care bundles and checklists.39 These found effects were based on 41 and 6 studies, respectively, of which 5 and 4 studies met our inclusion criteria, respectively.

Meddings et al40 reported that the use of a reminder and/or stop order to prompt removal of unnecessary urinary catheters led to a 53% reduction of CAUTI episodes per 1000 catheter days: rate ratio (RaR) 0.47 (CI 0.30 to 0.64; p<0.01). This meta-analysis was based on 11 studies, of which only 1 study complied with the inclusion criteria for study designs.

The implementation of a programme to improve compliance to sepsis care bundles led to a statistically significant decreased mortality rate: OR 0.66 (CI 0.61 to 0.72; p<0.01). This rate is based on 48 studies, of which 3 fulfilled the criteria for study designs.41

Delirium

Three systematic reviews reported a statistically significant reduction in delirium incidence.47 50 51 There was a 16% overlap (3 of the 19 studies) between these systematic reviews (see online supplementary appendix 9).

Hempenius et al47 pooled the effects of five studies and found a statistically significant effect of multicomponent interventions to prevent delirium: OR 0.58 (CI 0.38 to 0.92). Components were education, systematic cognitive screening, geriatric consultative services, supportive psychotherapy and a scheduled pain protocol.

Hshieh et al50 reviewed studies evaluating non-pharmacological interventions, including the following components: early mobility, cognition and orientation, sleep–wake cycle preservation, hydration, hearing and vision. They found a statistically significant reduction in delirium incidence: OR 0.47 (CI 0.38 to 0.58); p<0.01. This rate was based on 11 studies, of which 7 complied with the inclusion criteria for study designs.

Martinez et al51 found a statistically significant reduction in delirium incidence: RR 0.73 (CI 0.63 to 0.85); p<0.01. This rate was based on seven studies, using different multicomponent interventions, but a number of specific components were shared: physiotherapy, daily reorientation, family involvement in care, stimulation programmes with avoidance of sensorial deprivation and staff/family-member education.

Adverse event after hospital discharge or clinical handover

Six systematic reviews pooled the effect of interventions to improve clinical handover or hospital discharge. One systematic review reported a statistically significant effect size: nurse-led early-discharge planning programmes were associated with a lower mortality rate: RR 0.70 (CI 0.52 to 0.95; p=0.02).58 This found effect was based on five studies. Effective intervention components were an individual discharge plan to address identified transitional care needs, comprehensive discharge plan and home-based follow-up visits or telephone calls by providers to patients after their hospital discharge.

Fall

One systematic review61 reported the effectiveness of fall-prevention interventions. Additional physiotherapy reduced the risk of falling: RR 0.36 (CI 0.14 to 0.93). Multicomponent interventions reduced the fall rate: RaR 0.69 (CI 0.49 to 0.96). These rates were based on two and four studies, respectively. Effective components of the multifactorial interventions were fall-risk alert card and information brochure, exercise programme, education programme, hip protectors, comprehensive geriatric assessment and treatment of fall-risk factors by a multidisciplinary team.

Surgical adverse event

The implementation of a surgical checklist was associated with a reduction of complications, deaths and surgical-site infections: RR 0.59 (CI 0.47 to 0.74), 0.77 (CI 0.60 to 0.98) and 0.57 (CI 0.41 to 0.79), respectively. These pooled rates were based on five studies.66 The authors reported that the results were statistically significant but cannot be regarded as definitive in the absence of high-quality studies.66

Cardiopulmonary arrest

Two systematic reviews found an association between the implementation of a rapid response team and improved patient outcomes. There is an 11% overlap (2 of the 19 studies) between these systematic reviews (see online supplementary appendix 9). Chan et al69 performed a meta-analysis on 16 studies and found a statistically significant reduction of cardiopulmonary arrests outside the intensive care unit, following the implementation of the rapid response team: RR 0.65 (CI 0.55 to 0.77). The authors of the systematic review raised questions about the effectiveness of rapid response team implementation given the lack of an effect of rapid response teams on mortality.

The systematic review of Maharaj et al71 found a statistically significant reduction in cardiopulmonary arrests based on two studies: RR 0.74 (CI 0.56 to 0.98; p=0.04) and a statistically significant reduction of deaths based on four studies: RR 0.91 (CI 0.85 to 0.97; p<0.01).

Staffing

Butler et al75 found 6202 studies that were potentially relevant to studying the effect of hospital-nurse staffing models on mortality and adverse events. However, one study reported a statistically significant effect: increasing the proportion of support staff (ie, dietetic assistants) reduced mortality at 4 months: RR 0.57 (CI 0.34 to 0.95; p=0.03). The authors stated that they were unable to draw conclusions because of the small number of eligible studies.

Pannick et al76 found that interdisciplinary team interventions reduced mortality rates: RR 0.67 (CI 0.45 to 0.99). The finding was based on two studies. Effective intervention components were interdisciplinary rounds, including physician, nurse, pharmacist, nutritionist and social worker; expanded senior clinical nurse roles; incorporating structured detailed assessments of premorbid functional and social patient data and investment in allied health professionals as consistent staff members.

Clinical pathway

Rotter et al79 found an association between the use of clinical pathways and a reduction of in-hospital complications, based on five studies: OR 0.58 (CI 0.36 to 0.94). Examples of reported complications were postoperative confusion, infection, uncontrolled bleeding and deep vein thrombosis, ventilator-associated pneumonia, joint dislocation and decreased postdischarge mobility up to 3 months postsurgery. The OR for complications up to 3 months, based on one study, was 0.31 (CI 0.13 to 0.72).

Summary of effective patient-safety interventions

Patient-safety interventions that result in a significant reduction in adverse event or mortality rates are presented in table 3.

Table 3

Effective patient-safety interventions (n=14*)

Intervention effect estimates based on meta-analysis with only eligible studies†	Patient outcome	Effect size (95% CI)	Sample size (n patients)	Study size (n studies)	Designs of studies (n)
Exercises61	Risk of falling	RR 0.36 (0.14 to 0.93)	83	2	RCT (2)
Surgical safety checklist66	Surgical-site infections	RR 0.57 (0.41 to 0.79)	15 198	5	ITS (5)
Increasing the proportion of support staff75	Mortality at 4 months	RR 0.57 (0.34 to 0.95)	302	1	RCT (1)
Rapid response team69	Cardiopulmonary arrest	RR 0.65 (0.55 to 0.77)	1 143 495	16	Non-RCT (2); CBA (12); ITS (2)
Nurse-led early-discharge planning programmes58	Mortality	RR 0.70 (0.52 to 0.95)	2503	5	RCT (5)
Multicomponent interventions, including physiotherapy, daily reorientation, family involvement and staff/family-member education51	Delirium	RR 0.73 (0.63 to 0.85)	1691	7	RCT (7)
Antibiotic guideline for pneumonia33	Mortality	RR 0.89 (0.82 to 0.97)	22 526	4	RCT (1); CBA (3)
Rapid response team71	Mortality	RR 0.91 (0.85 to 0.97)	209 639	4	RCT (2); CBA (1); ITS (1)
Interdisciplinary team interventions76	Mortality	wRR 0.67 (0.45 to 0.99)	2640	2	Non-RCT (2)
Multicomponent interventions61	Falls	RaR 0.69 (0.49 to 0.96)	6478	4	RCT (4)
Multicomponent interventions, including cognitive screening, proactive geriatric consultation and psychotherapy47	Delirium	OR 0.58 (0.38 to 0.92)	1343	5	Non-RCT (3); CBA (2)
Clinical pathway79	In-hospital complications	OR 0.58 (0.36 to 0.94)	664	5	RCT (4); CCT (1)
Intervention effect estimates based on meta-analysis with eligible and non-eligible studies	Patient outcome	Effect size (95%CI)	Sample size (n eligible patients) and proportion of eligible patients of all patients (%)	Study size (n) and proportion of eligible studies (n; %)	Designs of eligible studies (n)
Catheter reminder and stop order40	Infections (CAUTI)	RR 0.72 (0.52 to 0.99)	U	8 (2; 25)	RCT (1); non-RCT (1)
Pharmacist interventions36	Adverse drug events	OR 0.23 (0.11 to 0.48)	2794 (30.4)	3 (2; 66.7)	CBA (2)
Care bundle and checklist39	Infections (CLABSI)	OR 0.39 (0.33 to 0.46)	70 358 (2.8)	41 (5; 12.2)	BA (36); ITS (5)
Multicomponent interventions, including early mobility, cognition and orientation50	Delirium	OR 0.47 (0.38 to 0.58)	2914 (68.3)	11 (7; 63.6)	RCT (3); non-RCT (4)
Sepsis bundle41	Mortality	OR 0.66 (0.61 to 0.72)	11 720 (2.7)	48 (3; 6.3)	ITS (3)

*17 systematic reviews reported about 14 types of interventions.

†Studies with a design in accordance with methodological criteria of the Cochrane EPOC review group.

CCT, controlled (clinical) trial; CAUTI, catheter-associated urinary tract infection; CBA, controlled before after; CLABSI, central-line-associated bloodstream infection; EPOC, Effective Practice and Organisation of Care; ITS, interrupted time series; NR, not reported; RaR, rate ratio; RCT, randomised controlled trial; RR, risk/relative ratio; U, unclear; wRR, weighted risk ratio.

Exercises to reduce the risk of falling, surgical safety checklist to reduce the rate of surgical-site infection, rapid response team to prevent cardiopulmonary arrest and multicomponent interventions to prevent delirium have significantly better results compared to changes in staffing and interventions to improve hospital discharge to prevent mortality. Pharmacist interventions and care bundle interventions and checklists were significantly associated with, respectively, reduced rates of adverse drug events and infection rates. These effect measures are, however, partly based on experimental studies (table 3).

Fourteen of the 17 significant effect sizes (82.4%) were based on five or fewer studies that comply with the inclusion criteria for study design. The effect measures were based on sample sizes varying from 83 to 1 143 495 patients, for exercises to reduce the risk of falling and rapid response team to reduce the rate of cardiopulmonary arrest, respectively (table 3). The AMSTAR scores of the systematic reviews of the 17 effective patient-safety interventions ranged from 4 to 10, with a mean score of 7.5 (SD ±1.9).

Three systematic reviews evaluated multicomponent interventions to prevent delirium (all with different compositions of the multicomponent intervention and different effect measures); two systematic reviews evaluated the effects of rapid response teams, resulting in 14 unique patient interventions (box 1).

Antibiotic guideline for pneumonia to reduce mortality rates.

Catheter reminder and stop order to reduce the risk for developing catheter-associated urinary tract infection.

Care bundles and checklists to reduce rates of central-line-associated blood stream infections.

Clinical pathways to avoid complications.

Exercises to reduce the risk of falling.

Increasing the proportion of support staff to reduce mortality rates.

Interdisciplinary team interventions to reduce mortality rates.

Multicomponent interventions to reduce the risk of falling.

Multicomponent interventions to prevent delirium.

Nurse-led early-discharge planning programmes to reduce mortality rates.

Pharmacist interventions to prevent adverse drug events.

Rapid response team to reduce the risk for cardiopulmonary arrest and reduce mortality rates.

Sepsis bundle to reduce mortality rates.

Surgical safety checklist to reduce the risk for surgical-site infections and reduce mortality rates.

Discussion

We systematically reviewed the literature for effective interventions aimed at reducing adverse event rates and preventable deaths in hospitals. The results showed that there were 14 effective patient-safety interventions (box 1), including: multicomponent interventions to prevent delirium; rapid response teams to reduce cardiopulmonary arrest and mortality rates; exercises and multicomponent interventions to reduce the risk of falling and surgical safety checklist to reduce the rate of surgical-site infection. Other effective interventions were pharmacist interventions to reduce adverse drug events, care bundles and checklists to reduce infection and mortality rates, changes in staffing and interventions to improve hospital discharge to reduce mortality rates. The evidence base that supports the interventions is moderate because 82% of the found effect measures were based on five or fewer primary studies that fulfilled the Cochrane EPOC criteria for study designs.20

This review offers a unique overview of effective patient-safety interventions based on data that are synthesised from systematic reviews, thereby producing a stronger evidence-based oversight of effective interventions compared to the outcomes of a systematic review of primary studies.16 The overlap of primary studies in existing reviews is analysed to minimise the potential effects of ‘double-counting’ primary studies in multiple reviews.82 Moreover, most of the systematic reviews included in our review were of high methodological quality (mean AMSTAR score of 6.9 for all included reviews and 7.5 for the reviews with positively pooled outcome effects), thereby increasing the credibility and validity of our findings.18

Despite the growing number of experimental studies evaluating the effectiveness of patient-safety interventions, our findings show that the evidence base for patient-safety improvement is still not strong. Furthermore, our findings are in contrast to the findings of previous research on this topic. Shekelle et al83 strongly supported the adoption of 10 patient-safety practices, including hand-hygiene strategies, the do-not-use list for hazardous abbreviations and multicomponent interventions to reduce pressure ulcers. We found limited support for the effectiveness of these interventions while finding strong support for delirium prevention interventions and rapid response teams. Our review placed more emphasis on assessing interventions on the basis of patient outcomes (ie, reduced adverse event and mortality rates) and testing within high-quality designs; this emphasis on the quality of studies produces a very different assessment of which safety interventions are most beneficial for patients and which should be implemented.

Evidence is still lacking for medication reconciliation and several interventions to improve the safety of clinical handover or discharge of hospitalised patients, which are incorporated in national and international patient-safety campaigns and are recommended by the WHO.84 However, the results of our review showed that by looking strictly at patient outcomes and only including high-quality studies, the evidence that these interventions reduce adverse event or mortality rates remains incomplete.

The lack of evidence for patient-safety interventions does not mean that these interventions do not work; it primarily addresses the lack of valid effect. Policymakers and clinicians show good intentions by implementing ambitious patient-safety programmes and investments of resources. However, implementing unproven interventions can lead to the opposite of what is intended with patient-safety improvements: waste of resources, energy and enthusiasm.85 86 In times of limited resources, we concur with Shekelle et al and underscore previous, urgent calls for more research on the effectiveness of patient-safety interventions.7 12 83 85 87 88 Patient-safety interventions should be tested on their effectiveness based on the same high-quality standards used for drug studies.3 89

This systematic review has several limitations. First, we did not retrieve data from the primary studies; instead, we used the information reported by the authors on aspects, such as the description of the interventions and reported outcomes. As a result, the information for some patient-safety interventions and outcomes reported in our systematic review is limited. However, by focusing on the results of the systematic reviews rather than each individual primary study, we were able to obtain a broad overview of the field of patient safety.90 Second, the found estimates of effectiveness of patient-safety interventions might vary across contexts, such as small versus large hospitals, academically affiliated hospitals versus those that are not and the availability of factors that stimulate successful implementation of interventions, for example, strong leadership and an electronic patient record.91 Third, in two-thirds of the included systematic reviews, publication bias was not assessed (see online supplementary appendix 7), meaning that the pooled rates in these reviews may present an overestimation of the effect size.92 Fourth, in this study, valuable narrative syntheses from systematic reviews may have been under-reported, because we focused on the quantitative evidence of safety interventions. The large amount of eligible systematic reviews and subsequent data from primary studies restricted us to focus on the results from meta-analyses, which are widely considered as the highest level of evidence for the effectiveness of interventions (Oxford Centre for Evidence-Based Medicine—Levels of Evidence). Fifth, the focus of our systematic review was to summarise quantitative evidence for existing patient-safety interventions. A limitation of this approach is that the found statistically significant effect measures may not be clinically significant and, vice versa, effects that are clinically relevant may not be statistically significant and were not captured in our systematic review.

In conclusion, patient-safety interventions are implemented worldwide, even though evidence for these interventions remains incomplete. A major cause for this problem is the lack of high-quality studies in which interventions are evaluated on their effects. To contribute to evidence-based patient safety, interventions need to be evaluated based on high-quality research standards, including experimental research designs, measured outcomes at the patient level and description of the intervention, implementation process and context in detail. Description of these aspects is necessary to know which factors lead to optimal effects and how to replicate the patient-safety intervention in practice.93 94 Policymakers and clinicians should stop taking shortcuts but need to spend more time and money conducting high-quality research on the effectiveness of patient-safety interventions to establish progress in patient safety.