Improving the governance of patient safety in emergency care: a systematic review of interventions.

OBJECTIVES: To systematically review interventions that aim to improve the governance of patient safety within emergency care on effectiveness, reliability, validity and feasibility.
DESIGN: A systematic review of the literature.
METHODS: PubMed, EMBASE, Cumulative Index to Nursing and Allied Health Literature, the Cochrane Database of Systematic Reviews and PsychInfo were searched for studies published between January 1990 and July 2014. We included studies evaluating interventions relevant for higher management to oversee and manage patient safety, in prehospital emergency medical service (EMS) organisations and hospital-based emergency departments (EDs). Two reviewers independently selected candidate studies, extracted data and assessed study quality. Studies were categorised according to study quality, setting, sample, intervention characteristics and findings.
RESULTS: Of the 18 included studies, 13 (72%) were non-experimental. Nine studies (50%) reported data on the reliability and/or validity of the intervention. Eight studies (44%) reported on the feasibility of the intervention. Only 4 studies (22%) reported statistically significant effects. The use of a simulation-based training programme and well-designed incident reporting systems led to a statistically significant improvement of safety knowledge and attitudes by ED staff and an increase of incident reports within EDs, respectively.
CONCLUSIONS: Characteristics of the interventions included in this review (eg, anonymous incident reporting and validation of incident reports by an independent party) could provide useful input for the design of an effective tool to govern patient safety in EMS organisations and EDs. However, executives cannot rely on a robust set of evidence-based and feasible tools to govern patient safety within their emergency care organisation and in the chain of emergency care. Established strategies from other high-risk sectors need to be evaluated in emergency care settings, using an experimental design with valid outcome measures to strengthen the evidence base. Published by the BMJ Publishing Group Limited. For permission to use (where not already granted under a licence) please go to http://www.bmj.com/company/products-services/rights-and-licensing/

This is the first systematic review of the literature that has evaluated the effects, reliability, validity and feasibility of interventions aimed to improve the governance of patient safety (ie, the ability for higher management to monitor and manage patient safety) in emergency care settings.

The review provides an overview of a variety of promising tools and their characteristics to monitor and manage patient safety in various types of emergency medical service organisations and in emergency departments. However, robust evidence to support these tools is absent.

The small number of included studies and the heterogeneity in the selected studies in terms of design, aims, intervention activities, population samples and presented outcomes make generalisations difficult.

To date, no studies have examined the effectiveness of interventions aimed to improve the governance of patient safety in the chain of emergency care, nor have they evaluated their psychometric properties and feasibility.

Introduction

Executives of healthcare services are increasingly held accountable for patient safety.1 2 Therefore, they have a fundamental governance role in overseeing and managing safety risks within their service.3 Governance of patient safety is especially important in the field of emergency care, because emergency care involves high patient safety risks. Care is often delivered to high-acuity patients with unstable vital signs in a fast-paced setting under unpredictable conditions.4 Also, emergency care often involves collaboration between different emergency medical service (EMS) organisations, including: general practitioner out-of-hours services (GP OHS), ambulance EMS, helicopter EMS (HEMS) and psychiatric EMS, and between EMS organisations and the emergency department (ED) in the hospital. Frequent patient handovers between the different services involve inherent opportunities for miscommunication and adverse events (AEs) to occur.5–7

Executives of emergency care organisations, however, seem to fall short in the governance of patient safety. Evidence shows that suboptimal emergency care is an important cause of patient harm and mortality. Between 6% and 8.5% of the patients who receive care in the ED experience an AE.8 9 Furthermore, 36–71% of the AEs in the ED are believed to be preventable.10 Preventable AEs also occur in ambulance EMS, HEMS and GP OHS.11–13 Causes of AEs relate to system failures, stressed and fatigued care providers, medication errors, communication problems, lack of professional skills and problems with medical equipment.14 15

Several studies investigated board engagement with quality and safety issues in their health service,16 17 and systematically reviewed the effectiveness and usefulness of governance systems and tools.18 19 However, evidence on effective safety governance activities in emergency care is unknown. More insight into available valid, reliable and feasible means to monitor and manage safety risks could provide boards better oversight of patient safety and accountability of their emergency care organisation, and the chain of emergency care. We defined the chain of emergency care as: the interprofessional structure in which emergency care is delivered by multiple providers with the aim to provide seamless care to patients with acute care needs.6

The purpose of this study is to systematically review interventions aimed at improving the governance of patient safety in (the chain of) emergency care, and to evaluate their effects, reliability, validity and feasibility.

Methods

We planned and reported this systematic review in accordance with the guideline for performing and reporting systematic reviews and meta-analyses (PRISMA, Preferred Reporting Items for Systematic Reviews and Meta-Analyses).20

Data sources and searches

We searched for English and Dutch language studies published between January 1990 and July 2014 in the following databases: PubMed (including MEDLINE), Cumulative Index to Nursing and Allied Health Literature (CINAHL), Embase, PsychInfo and the Cochrane Library. Online supplementary appendix 1 provides a detailed listing of the search terms. We also searched for additional relevant studies (ie, ‘snowballing’): (1) via Google with the use of major key terms (ie, ‘governance’ AND ‘emergency care’ AND ‘patient safety’); (2) by reviewing references from the included studies and (3) by reviewing online archives/bibliographies of three high-impact journals in the field of emergency care (Annals of Emergency Medicine, Injury, Journal of Trauma and Acute Care Surgery).

Study selection

Two reviewers (GH and SB) independently screened the titles and abstracts of all studies identified by the search strategy for their eligibility. For inclusion, each study had to meet four criteria: (1) experimental or non-experimental study published as a full-text article or dissertation, (2) evaluating an intervention aimed at improving the governance of patient safety (ie, the ability for higher management to monitor and manage patient safety); (3) within the emergency care setting and (4) reporting data on the effect, validity, reliability or feasibility in terms of time and cost investment, and user friendliness of the intervention. Studies with a focus on acute dental care, intensive care (IC) and disaster medicine were excluded. When the title and abstract did not clearly indicate whether the inclusion criteria were met, a full-text copy was retained and reviewed.

Full-text copies of the potentially relevant studies were retrieved and evaluated for inclusion as described previously by two reviewers (GH and TB). A final set of studies was identified for data extraction. Inclusion discrepancies were reconciled by discussion.

Data extraction

GH and TB independently extracted data from each study meeting the inclusion criteria. A standardised form was used to ensure consistency of data extracted from each article. The extracted data described the study objectives, underlying theory-based concepts, setting, sample, intervention characteristics and findings. Disagreement between the reviewers was resolved by discussion. If no consensus was reached, a third reviewer (SB) was consulted.

Quality assessment

GH and TB independently assessed the study quality using a quality appraisal tool developed by Kmet et al.21 Studies were scored on up to 24 items: 14 items for studies with a quantitative research design and 10 items for studies with a qualitative research design. Items were scored depending on the degree to which the specific criteria were met (‘yes’=2, ‘partial’=1, ‘no’=0). Items not applicable to a particular study design were marked ‘NA’ and were excluded from the calculation of the summary score. Discrepancies were resolved through discussion. If no consensus was reached, a third reviewer (SB) was consulted. A study quality score (percentage) was calculated for each paper by summing the total score obtained across relevant items and dividing the obtained score by the total possible score.

Data synthesis

Study outcomes were organised in tabular form and a classification was made based on the study design, setting, sample size, intervention characteristics and outcomes, namely: effects and reported statistical significance, psychometric properties (ie, reliability and validity) and feasibility of the intervention.

Results

Search results

Our initial search identified 4287 records. After exclusion of duplicates, 3713 records were screened by title and abstract. Seventy full-text studies were retrieved and reviewed, of which 57 were excluded. Five articles were identified through snowballing. The final set consisted of 18 published studies that underwent full-text extraction (figure 1). Owing to the heterogeneity of the study designs, participants and outcome measures, a meta-analysis of the results was not possible.

Figure 1

Flow chart of the study selection and review process.

Study quality

Thirteen articles had a quantitative study design.22–34 Two articles had a qualitative study design.35 36 Three articles combined both quantitative and qualitative methods.37–39 The study quality scores ranged between 41% and 100% (tables 1 and 2). Two articles scored low (ie, <55%),32 34 10 articles scored high (ie, >75%),23–28 30 31 33 36 1 article scored high on the qualitative study and low on the quantitative study part,38 1 article scored high on the quantitative study and moderate on the qualitative study part,39 and 1 other article scored high on both (qualitative and quantitative) study parts.37 The three remaining studies scored a moderate in-between rating.22 29 35 Of the five articles with qualitative research, four had no or an unclear qualitative data analysis description (eg, omitting the types of analysis). Three qualitative studies failed to fully describe their qualitative data collection methods (eg, not mentioning an interview guide or the number of consensus rounds conducted in a Delphi study).35 38 39 Three qualitative studies showed no or poor use of verification procedures to establish credibility.35 36 39 Compared with the qualitative studies, the quantitative studies lacked in points related to sampling. Of the 16 articles with quantitative research, 8 had no or poor description of their sampling strategy (eg, inclusion and exclusion criteria),22–24 29 32–34 38 lacked an appropriate sample size23–26 32 34 37 38 and described sample characteristics insufficiently.25 28–30 33 35–37 Only two articles with quantitative research reported to appropriately control for confounding variables.28 31

Table 1

Quality assessment of studies with quantitative design

	Wolff and Bourke22	Hendrie et al23	Patterson et al37*	Clunas et al24	van Noord et al25	Patterson et al26	Patterson et al27	Flowerdew et al38*	Jaynes et al39*	Evans et al28	Zwart et al29	Reznek and Barton30	Pham et al31	Jones et al32	Patterson et al33	Shaw et al34
Question/objective sufficiently described?	2	2	2	1	2	2	1	1	2	2	2	2	2	2	2	2
Study design evident and appropriate?	2	1	2	2	2	2	2	1	2	2	2	2	2	2	2	1
Method of subject/comparison group selection or source of information/input variables described and appropriate?	1	1	2	0	2	2	2	1	2	2	1	2	2	1	1	1
Subject characteristics sufficiently described?	2	2	2	1	2	2	1	1	1	2	2	1	2	1	1	1
If interventional and random allocation was possible, was it described?	NA	NA	NA	NA	NA	NA	NA	NA	NA	2	0	NA	NA	1	NA	NA
If interventional and blinding of investigators was possible, was it reported?	0	NA	NA	NA	NA	NA	NA	NA	NA	2	NA	NA	NA	NA	2	NA
If interventional and blinding of subjects was possible, was it reported?	0	NA	NA	NA	NA	NA	NA	NA	NA	2	NA	NA	NA	NA	NA	NA
Outcome and exposure measure(s) well defined and robust to measurement/misclassification bias? Means of assessment reported?	2	2	1	2	1	2	2	1	1	2	2	2	2	1	1	1
Sample size appropriate?	2	1	1	1	1	1	2	1	2	2	2	2	2	0	2	1
Analytic methods described/justified and appropriate?	1	2	2	2	2	2	1	1	1	2	2	2	2	2	2	0
Is some estimate of variance reported for the main results?	2	2	0	2	1	2	2	0	0	2	0	2	2	0	2	0
Controlled for confounding?	0	1	NA	NA	NA	1	1	0	NA	2	1	NA	2	0	NA	0
Results reported in sufficient detail?	2	2	2	2	2	2	2	1	2	2	1	2	2	2	2	1
Conclusions supported by the results?	2	2	2	2	2	2	1	1	2	2	2	2	2	1	2	2
Total points	18	18	16	15	17	20	17	9	15	28	17	19	22	13	19	10
Max points possible	26	22	20	20	20	22	22	22	20	28	24	20	22	24	22	22
Summary score, in percentage (%)	69	82	80	75	85	91	77	41	75	100	71	95	100	54	86	46

NA, not available.

*Study using quantitative and qualitative research methods.

Table 2

Quality assessment of studies with qualitative design

	Patterson et al35	Patterson et al37*	Flowerdew et al38*	Jaynes et al39*	Schull et al36
Question/objective sufficiently described?	2	2	1	2	2
Study design evident and appropriate?	2	2	2	2	2
Context for the study clear?	2	2	2	2	2
Connection to a theoretical framework/wider body of knowledge?	2	2	2	1	1
Sampling strategy described, relevant and justified?	2	2	2	1	2
Data collection methods clearly described and systematic?	1	2	1	1	2
Data analysis clearly described and systematic?	1	1	1	0	2
Use of verification procedure(s) to establish credibility?	0	2	2	1	1
Conclusions supported by the results?	2	2	1	2	2
Reflexivity of the account?	0	1	1	1	1
Total points	14	18	15	13	17
Maximum points possible	20	20	20	20	20
Summary score, in percentage	70	90	75	65	85

*Study using quantitative and qualitative research methods.

Study characteristics

Table 3 shows a summary of the study characteristics. A more detailed overview of the study characteristics is provided in online supplementary appendix 2. Of the 18 included studies, 10 (56%) were performed in the USA,26 27 30–35 37 39 4 (22%) in Australia,22–24 28 2 (11%) in the Netherlands,25 29 1 (6%) in the UK38 and 1 (6%) in Canada.36 Thirteen studies (72%) were non-experimental. Five studies (28%) were quasi-experimental using an interrupted time series design,30 33 a non-equivalent group design28 29 and a before–after design.22

Table 3

Study characteristics

First author (year) (country)	Design	Setting	Sample	Intervention	Findings
					Effects	Psychometrics	Feasibility
Wolff (2002) (Australia)22	Quasi-experimental (BA)	ED (n=1)	Reviewed patient medical records (n=20 050)	Incident reporting in addition to standardised screening of medical records on AEs	Reduced AEs*	NR	NR
Hendrie (2007) (Australia)23	Non-experimental	ED (n=1)	Patient case histories (n=3332)	AE screening	NR	Inter-rater reliability	Time†
Patterson (2012) (USA)35	Non-experimental	EMS (n=NR)	Patient case reports (n=250)	AE identification and severity rating method	NR	Internal reliability; construct validity	NR
Patterson (2014) (USA)37	Non-experimental	HEMS (n=NR)	Expert clinicians in emergency medicine and HEMS (n=10)	AE identification and severity rating method	NR	Content and face validity	NR
Clunas (2009) (Australia)24	Non-experimental	ED (n=1)	Reviewed patient deaths (n=303)	Audit of all deaths that occurred within 48 h of ED presentation in addition to auditing all deaths that occurred in the ED itself	NR	NR	Usability‡
van Noord (2010) (The Netherlands)25	Non-experimental	ED (n=31)	Closed and settled claim files (n=47)	Root Cause Analysis using PRISMA method	NR	Inter-rater reliability; face validity	Time†
Patterson (2010) (USA)26	Non-experimental	EMS agencies (n=3)	EMTs and paramedics (n=71)	EMS-SAQ	NR	Internal reliability; construct validity	Response rate‡; user friendliness‡
Patterson (2010) (USA)27	Non-experimental	EMS agencies (n=61)	Care providers (n=1595)	EMS-SAQ	NR	Inter-rater reliability; face validity	NR
Flowerdew (2012) (UK)38	Non-experimental	ED (n=2)	NR	Observational physician (non-technical) skills assessment	NR	Face and content validity	NR
Jaynes (2013) (USA)39	Non-experimental	EMS (n=NR)	EMS care providers (n=380)	EMS and HEMS working relationship satisfaction questionnaire	NR	Internal reliability; face, content and construct validity	NR
Evans (2007) (Australia)28	Quasi experimental (NEG)	ED (n=4)	ED (n=2) attendances (n=66 669) with intervention vs ED (n=2) attendances (n=78 264) with usual procedure	Incident reporting programme comprising intense staff education, 24/7 reporting options, changes in report management and enhanced feedback	Increased IRs*	NR	NR
Zwart (2011) (The Netherlands)29	Quasi experimental (NEG)	GP OHS (n=3)	GP OHS with intervention (n=1); GP OHS with usual procedure (n=2)	Local incident-reporting vs centralised incident reporting (usual procedure)	Increased IRs; increased IR types	NR	Time‡; costs†
Reznek (2014) (USA)30	Quasi experimental (ITS)	ED (n=1)	IRs (n=314)	Standardised non-punitive peer review of IRs	Increased monthly frequencies of IRs*	NR	NR
Schull (2011) (Canada)36	Non-experimental	ED (n=NR)	Candidate indicators (n=170)	Patient safety indicators	NR	Face validity	Usability‡
Pham (2011) (USA)31	Non-experimental	ED (n=1)	Patients seen in the ED within 72 h of prior visit (n=6858) and patients not seen in the ED within 72 h (n=211 321)	Patient safety indicator	NR	NR	Usability†
Jones (2013) (USA)32	Non-experimental	ED (n=2)	Care providers (n=60)	Teamwork training on patient safety (TeamSTEPPS)	Positive change in safety culture perception	NR	NR
Patterson (2013) (USA)33	Quasi experimental (ITS)	Paediatric ED (n=1)	Care providers (n=151)	Multidisciplinary simulation-based training	Increased staff safety knowledge*; increased staff safety attitude*	NR	Time‡
Shaw (2006) (USA)34	Non-experimental	Paediatric ED (n=1)	Staff (n=99)	Unit-based Patient Safety Walk-rounds	Increased IRs; increased hand hygiene compliance	NR	NR

*Statistical significant effect (p<0.05).

†Negative finding with regard to the feasibility of the intervention.

‡Positive finding with regard to the feasibility of the intervention.

AE, adverse event; BA, before–after; ED, emergency department; EMS, emergency medical services; EMS-SQA, EMS-Safety Attitudes Questionnaire; EMT, emergency medical technician; GP OHS, general practitioner out-of-hours services; HEMS, helicopter EMS; IR, incident report; ITS, interrupted time series; NEG, non-equivalent group; NR, not reported; PRISMA, Prevention and Recovery Information System for Monitoring and Analysis.

Of the 18 included studies, 12 (67%) evaluated a safety governance intervention within EDs,22–25 28 30–34 36 38 4 (22%) within EMS organisations,26 27 35 39 1 within an HEMS37 and 1 (6%) within GP OHS.29 One study focused on monitoring the quality and safety of ambulance and HEMS collaboration.39 The sample size ranged from 60 to 1595 studied care providers, 6858–211 321 studied patients and 47–20 050 studied files (eg, incident reports, medical records, claim files). One study (6%) described a study panel of 10 expert clinicians as the study sample.33

Four studies (22%) reported statistically significant effects.22 28 30 33 Nine studies (50%) reported data on the reliability and/or validity of the intervention.23 25–27 35–39 Eight studies (44%) reported on the feasibility of the intervention.23–26 29 31 33 36

Intervention characteristics and findings

Six studies (33%) examined methods for screening and assessing AEs, incidents and patient deaths.22–25 35 37 Four studies (22%) evaluated safety culture and care provider behaviour measures.26 27 38 39 Three studies (17%) evaluated incident reporting systems.28–30 Two studies (11%) evaluated patient safety indicators.31 36 Two studies (11%) evaluated training methods for improving care provider safety skills and attitudes.32 33 One study (6%) evaluated the effectiveness of Patient Safety Walk-rounds (PSWs).34

Screening and assessment methods

Four studies described methods to screen and assess AEs. Wolff and Bourke22 described retrospective screening of medical records in the ED with the use of an AE severity scale to assess AEs. A clinical risk manager performed the screening and assessment of AEs, and created weekly reports for the ED management, describing the type and severity of identified AEs and improvement actions. Aggregated quarterly reports detailing actions taken and AE rates were presented to the hospital's main quality improvement committee. In addition, uniform reporting of incidents by ED staff was stimulated with the use of one definition of a clinical incident and a standardised incident report form. Over 2 years, the number of AEs reduced—a relative risk reduction of 85.3% (95% CI 62.7% to 100%). Hendrie et al23 evaluated an AE screening and assessment method of case records. AEs were identified using a validated data collection instrument and classified on management causation, outcome and preventability. Inter-rater agreement on the classification of AEs (ƙ=0.15), on judgements about management causation (ƙ=0.50) and on preventability (ƙ=0.58) was poor. Furthermore, the researchers considered the time to detect an AE to be substantial. The study did not report any measure of effect (eg, regarding the number of detected AEs). Patterson et al35 evaluated a method for AE identification and severity rating in medical charts in ambulance EMS. A definition of an AE in EMS and an AE severity-rating index were developed in a consensus study for uniform identification of AEs in medical charts. Multirater agreement on classification of AEs was poor (ƙ=0.24). Patterson et al37 used a modified Delphi study to develop a consensus-based AE definition and a framework for AE detection in HEMS. Subsequently, the framework evaluated on content validity, using the item and scale content validity index. The framework was composed of three main components: (1) a trigger tool to operationalise AE detection, using key words or phrases contained within patient care reports that have a high probability of being linked to patient harm, (2) a method for rating AE severity, (3) a method for rating proximal cause of AEs. All three components of the framework showed content validity. The study did not report any measure of effect.

Clunas et al24 evaluated an audit of patient deaths that occurred within 48 h of ED presentation in addition to auditing all deaths that occurred in the ED itself. The authors tested the audit by reviewing 303 deaths, including 75 deaths in the ED and 228 deaths within 48 h of ED presentation. Results showed that 36% of the death cases within 48 h of ED presentation that required a major external hospital review were not identified by the standard hospital incident monitoring system.

The psychometric properties and the feasibility of the Prevention and Recovery Information System for Monitoring and Analysis (PRISMA) was evaluated by van Noord et al,25 to retrospectively analyse root causes of incidents that have led to malpractice claim files in the ED. The authors found a high inter-rater agreement on classification of root causes (ƙ=0.78). Validity of the root cause profile of claims was considered moderate. The delay between incident occurrences and their detection and reporting made it difficult to draw firm conclusions from the analyses. Finally, the PRISMA analyses were time consuming. The study did not report any measure of effect.

Safety culture and care provider behaviour measures

Patterson et al26 27 evaluated the Safety Attitudes Questionnaire (SQA). The EMS-SQA is a modified version of the validated Intensive Care Unit SAQ (ICU-SAQ). The anonymised questionnaire is administered in paper form and/or via the internet. Respondents are asked to rate 60 items on a five-point Likert scale (strongly agree to strongly disagree). The responses are used to characterise six safety domains (eg, safety climate and teamwork climate). Evaluation of the six safety domains, using Confirmatory Factor Analysis (CFA), revealed acceptable internal consistency and model fit validity of the EMS-SQA. Patterson et al26 confirmed feasibility of the EMS-SQA based on the high response rate and positive feedback on instrument utility from EMS chief administrators. In contrast, the authors stated that some chief administrators raised concerns about the respondent burden and the face validity of several questionnaire items. The study did not report any measure of effect.

Flowerdew et al38 evaluated a method to assess care provider non-technical skills in the ED. A behavioural marker system was developed for the observational assessment of 12 specific non-technical skills required by physicians, for example, maintaining standards, managing workload and resolving conflict. Skills were assessed on a nine-point rating scale and divided into ‘unacceptable’, ‘acceptable’ and ‘exemplary’. The tool was considered to be valid based on the input of evidence-based literature, and the input of interviews with staff and observations, to determine whether, in practice, the skill list contained any significant omissions and whether skills were observable. A survey among experts proved content validity of the developed list of skills and behavioural markers. The study did not report any measure of effect.

Jaynes et al39 evaluated an instrument to assess the working relationship between ambulance and HEMS care providers. The questionnaire consisted of 22 items that were rated on a five-point Likert scale (never/very poor to always/very good). The questionnaire was developed based on the input of providers, medical directors and administrators (n=12), who defined the activities involved in the EMS–HEMS working relationship and generated items (eg, We have the information we need for making transport decisions). HEMS and EMS personnel reviewed the questionnaire and determined content validity based on consensus. The measure had good internal reliability, with a Cronbach's α for each domain varying between 0.85 and 0.88. Explanatory factor analysis showed that a single underlying factor could best account for all questionnaire items. The study did not report any measure of effect.

Incident reporting systems

Evans et al28 evaluated an incident reporting programme in the ED. The programme included the display of posters and manuals for staff describing the importance of reporting, the possibility of anonymous reporting, the use of a one-page report form, a 24 h/7 days open telephone reporting service, and feedback on statistics and root-cause analysis findings to all ED staff. A patient safety manager initially assessed incident reports. Also, anonymous reports were validated and managed without the involvement of unit heads. The intervention resulted in a statistically significant improvement in reporting by the ED staff; an overall increase of 39.5 incident reports per 10 000 ED attendances (95% CI 17.0 to 62.0; p<0.001). Zwart et al29 compared a local incident reporting procedure (LIRP) with a centralised incident reporting procedure (CIRP) in Dutch GP OHS. In the LIRP, a local multidisciplinary committee is trained to screen and analyse incident reports, whereas in the CIRP, incident analysis is performed by an advisory committee of the board of directors of the GP OHS collaboration. The local committee was responsible for feedback to reporters and for follow-up measures when appropriate. Furthermore, reported incidents were analysed within 2 weeks instead of the usual every 2 months. The number of incidents in the GP OHS, using the LIRP, increased 16-fold compared with the GP OHSs using the CIRP. The implementation of a LIRP was associated with extra costs for administration and analysis. Reznek and Barton30 evaluated the effectiveness of a standardised, non-punitive peer review process of incident reports in one ED compared to analysis of incident reports by a single reviewer. Relevant reports were peer reviewed each month by a committee of board certified physicians, and involved structured analysis and discussion of incidents with staff that participated in open peer review proceedings. The authors stated that the monthly frequency of reporting increased over time compared with that of a control group of practitioners from outside the hospital (p=0.0019; p<0.0001).

Patient safety indicators

Pham et al31 evaluated the usability of one indicator: patient ED returns within 72 h of prior visit. Findings did not support the use of 72 h returns as a safety indicator: patients who return to the ED within 72 h do not use more resources, are not more severely ill and do not have a higher hospital admission rate than those who had not been previously seen. Schull et al36 sought to develop a set of evidence-based quality of care indicators for EDs. An expert panel reached consensus on a set of 48 indicators of which six focused on the measurement of patient safety. Of these six patient safety indicators, four were classified as feasible based on the use of current national administrative databases (eg, Percentage of patients with headache discharged home from the ED who were admitted to hospital with subarachnoid haemorrhage in the subsequent 14 days). The two other indicators (ie, ‘Percentage of central lines inserted in the ED that developed catheter-related bloodstream infections’ and ‘Percentage of intubated patients for whom end-tidal carbon dioxide was monitored’), could be feasibly measured with enhanced quality and completeness of data (eg, coding of injuries, medical interventions and time registrations) in existing database fields.

Training of safety attitudes and skills

Jones et al32 evaluated the effect of a teamwork training method (TeamSTEPPS) on improved staff perception of safety culture within the ED. The training was given in a period of 4 weeks, educating staff on how to communicate safety concerns, and report errors and system failures. Video vignettes were used illustrating good communication—as well as barriers to communication—to facilitate group discussion. Participants used hand-outs with communication techniques for practice, both in class and after the training sessions. Findings showed no statistical difference of perceived safety culture before and after the training. Patterson et al33 evaluated the effectiveness of multidisciplinary simulation-based training. Care providers learned techniques to prevent medical errors, develop resilience, and to improve situation awareness and closed loop communication. Via debriefing of video-based simulations and a videotaped clinical scenario, ED personnel were trained to recognise high risk situations and to use the acquired skills to prevent or decrease the impact of unexpected events and errors. The training resulted in a statistically significant increase of patient safety knowledge and attitudes of personnel. The time required to conduct the training reduced over time from 12 to 4 h.

Safety walk-rounds

Shaw et al34 evaluated the effectiveness of PSWs in one ED. PSWs were performed by a physician and two staff nurses, and lasted approximately 30 min. Each PSW was conducted in the clinical area of the ED and included data collection on two of the following clinical quality improvement topics: (1) accuracy of weight and allergy documentation; (2) compliance with hand washing; (3) accuracy of medication orders, administration and documentation; (4) appropriateness of patient monitoring and alarm parameters/central monitoring; (5) reasons for prolonged length of stay (>3 h) and (6) patient/family communication. Rounds were followed by a general discussion with ED staff on, for example, staff near-miss experiences and suggestions for improvement. Subsequently, the ED Patient Safety Committee (ie, directors, managers) reviewed results and incident reports. An email was sent to all staff regularly, to inform on positive outcomes and needs for improvement. Study findings showed 44% increase of medication near-miss incident reports and 23% overall increase in hand hygiene compliance within the ED.

Discussion

To the best of our knowledge, this is the first systematic review of studies evaluating the effects, reliability, validity and feasibility of interventions to improve the governance of patient safety in emergency care. Our review highlights the lack of evidence on effective safety governance strategies in emergency care settings, particularly in the field of prehospital emergency care. Only four studies examining an intervention in EDs and GP OHS reported statistically significant effects on reduced AEs, an increase of reported incidents, and an increase of patient safety attitudes and knowledge among care providers. The validity, reliability and feasibility of interventions varied greatly. Moreover, the information provided in terms of time investment, costs and usability, was limited.

We identified two types of interventions that showed to be effective in improving the governance of patient safety within organisations. First of all, simulation-based patient safety training proved to be an effective intervention for improving the patient safety culture and safe medical practice in the ED. These findings correspond with the literature on medical education and training. Simulation-based training is increasingly valued as an effective method to enhance safety knowledge and behaviour of providers and healthcare teams, in addition to didactic education methods.40 41 In a controlled setting, care providers can experience infrequent and unexpected events, and learn to practice resilient behaviour.42 This is especially important in a high-risk sector such as emergency care. Second, the use of well-designed incident reporting systems leads to an increase of incidents reported by GP OHS and ED staff, which is an important source of data for executives to use for monitoring safety risks. Effective incident reporting systems shared the following components: (1) education of staff on the importance and the learning purpose of reporting; (2) multiple and constantly available reporting options for staff; (3) a short reporting form to minimise the burden of reporting and (4) structural feedback by presenting descriptive statistics, findings of incident root-cause analyses and improvement actions. These findings are supported by other publications on successful incident reporting systems.43–46 In a setting such as emergency care, where providers constantly have to deal with time pressure, it is important that sufficient resources for effective and efficient reporting are available. Additionally, a non-punitive reporting system is imperative for a culture of self-reporting to thrive.47 Interestingly, the effective incident reporting systems had different approaches towards anonymous reporting and the management of reports. One system had the ability for care providers to report anonymously, and anonymous reports were validated and followed-up only by the patient safety manager. This is consistent with previous studies suggesting that anonymous reporting and validation of reports by an independent party can increase the quality of reporting by care providers.48 49 In contrast, the other system invited care providers to participate in a non-anonymous peer review process that involved analysis and structured discussion of incident reports submitted to ED physician leadership. This suggests that anonymity of reporting and management of incident reports by an independent party may not be necessary if an incident reporting and review process is perceived to be safe.

No effective interventions were found that aim to monitor or improve patient safety in the chain of emergency care. This is a disturbing finding considering the high number of patient transitions and the unique challenges to safe handoffs between EMS organisations.5–7

Our hope is that this systematic review will act as a stimulus to gather more evidence on safety governance improvements in the field of emergency care. Characteristics of the interventions included in this review (eg, anonymous reporting and validation of reports by an independent party) could provide useful input for the design of an effective tool to govern patient safety in EMS organisations and hospital-based EDs. However, at the moment, executives cannot rely on several evidence-based strategies to govern patient safety within their organisation and in the chain of emergency care. A variety of established and effective tools are used in other healthcare domains and high-reliability sectors, such as the aviation and chemical industry. For example, safety indicators,50 patient safety dashboards and checklists,51 52 prospective risk analysis techniques (eg, Bow-tie, Failure Mode Effect Analysis)53 and safety audits.54 55 These strategies need to be evaluated on effectiveness and feasibility in studies with multiple (types of) EMS organisations as study sample, a control group, and uniform and valid outcome measures. Executives, quality officers and researchers should therefore keep in mind that these interventions need to correspond with the organisation's current patient safety stage.56 For example, the use of risk surveillance and educational interventions are doomed to fail without a culture of openness about errors among staff, and a proactive attitude towards safety improvement.

Review limitations

Our review has several limitations. First, the heterogeneity in the selected studies in terms of design, aims, intervention activities, sample, outcome measurements and presented outcomes prevented us from performing quantitative meta-analyses. Second, we experienced difficulties with including relevant studies, because most studies did not explicitly address if interventions were meant to improve safety governance at the executive level (ie, board of directors), or at the middle or lower management level (ie, heads of department, unit leaders) or both. Third, the outcome measures used by the studies may not reflect the impact of safety governance activities. For example, a reduced AE rate may be caused by factors other than an improved reporting system. Moreover, an increase of incident reports may also be an indicator of over-reporting by care providers. There are no uniform and clear criteria for measuring effective governance of patient safety in healthcare organisations. Therefore, the effects found need to be interpreted with caution. Fourth, evaluations with an observational design dominated the studies we identified. The design of these studies limits the ability to draw firm conclusions on the effectiveness of individual interventions. Fifth, the effectiveness and feasibility of reviewed interventions may relate to a specific medical or demographical setting. Two-thirds of the studies included in this review were performed in one or more EDs. More than a third of the included studies were conducted in a single organisation. Sixth, restricting the literature search to studies published in the English and Dutch languages may have introduced a study selection bias based on language. However, we did not find non-English publications that met our inclusion criteria.

Conclusion

Simulation-based training and incident reporting systems with a focus on reducing the fear of reporting, reporting burden, and structural and systematic feedback, are promising interventions to improve the governance of patient safety in emergency care. However, the weak study designs, the lack of valid outcome measures and information on feasibility hinder the demonstration of robust evidence to support these interventions. Promising interventions for the governance of patient safety in the chain of emergency care are absent. Further research evaluating established governance tools on effectiveness and feasibility from other sectors within emergency care organisations is warranted.