Review of alternatives to root cause analysis: developing a robust system for incident report analysis.

Medical errors and other system failures lead to patient injury and significant costs—estimates of potential societal costs range from $393 to $958 billion.1 2 To improve safety, the healthcare system has looked to industries with impressive safety records in high-risk contexts, often referred to as high reliability organisations (HRO), for ideas.3 In the 1990’s the Veterans Health Administration (VHA) adopted root cause analysis (RCA) from HROs to learn from the most serious incident reports (IRs).4 Over the ensuing decades, other healthcare organisations followed developing distinct, yet similar approaches.5–12

Systematic learning from IRs is a central component of reliable healthcare delivery systems. Methods to learn from IRs work best when they highlight areas of risk, which can lead to improved performance.12–19 Investigating IRs with RCA is consistent with the core characteristics of HROs. The structured feedback process on a defect in care provides the environment for organisational mindfulness. RCA ‘enables simultaneous adaptive learning and reliable performance’20 and promotes mindfulness by acting as ‘a window on the system’.21 However, it is neither feasible nor desirable to complete an RCA following every IR. For example, one 600-bed private hospital generates 15 000 IRs annually—an impossible number to investigate with RCA given that the process takes more than 20 person-hours and over $8000 to complete.22–24 Therefore, the RCA process is typically reserved for medical errors that lead to the greatest harm. Unfortunately, in the remainder of the cases, incident reporting alone does not improve safety.13 21 Thus, healthcare organisations need additional strategies to learn from no-harm and low-harm IRs that are rarely the subject of RCA.25 26

One example of a harmful IR that would lead to an RCA is a medication error that contributed to a patient death. On the other hand, if a nurse administering the medication noticed the error and alerted the prescribing provider a resulting IR would not indicate patient harm. No RCA or systemic organisational learning would result.

The current RCA process seeks to improve patient care through organisational learning and identifying specific actions to improve performance. Several similar RCA tools are in wide use in healthcare.6–8 10 12 In an RCA a multidisciplinary team asks three questions: ‘What happened? Why did it happen?’ and ‘How to prevent it from happening again?’27 They seek to analyse safety events through ‘a human factors engineering approach—entailing a search for system vulnerabilities rather than individual human errors and other less actionable root causes’.28 RCA has been shown to improve safety and compliance with clinical processes.29–45 Studies have also identified problems with the methods by which RCAs are conducted and actions enacted.12 23 25 46–56 While imperfections in RCA methods occur, they do not characterise all RCAs.

The feasibility of RCA remains a problem. A single RCA can take 20–90 person-hours or more to complete.23 The workload means that few, if any IRs of low-harm and no-harm events are addressed. If some of these events are particularly worrisome to frontline staff, this can lead to staff questioning the relevance of reporting safety events and contribute to a poor safety culture.57 Additional tools are needed to investigate IRs of no-harm and low-harm IRs. Ideally, the level of harm and potential frequency of the risk should match the depth of the investigation following an IR.6 8 The ideal analytic tool for otherwise unexamined, less-harmful events should consume fewer resources than RCA to allow more investigations and more opportunities for organisational learning.58 The aim of this paper was to identify and describe the range of tools used to investigate and analyse no-harm and low-harm IRs with a specific focus on alternatives to the conventional RCA process.

Methods

This narrative review searched PubMed and Embase. There were no search limits (language, study types) used. The search strategy initiated with several concise analytical tools known to the authors and was further developed with a Medical Librarian. Each database’s search is described in online supplementary appendix 1.

The reviewer looked at each citation, and selected items for inclusion if the source described a tool that could be used for investigating and analysing IRs that consumed fewer person-hours than RCA. Studies were excluded if they did not describe or analyse such a tool or if they were not in English. Reference lists for relevant articles were also scanned for additional studies.

Results

Seven tools were identified in the literature to investigate IRs of medical errors that appear to be less resource-intense than RCA (table 1). We began with the least structured and progress to those that are more resource-intensive.

Table 1

Learning tools: measures in the literature

Learning tool	Structure measures	Process measures	Outcome measures	Safety culture (Outcome)
AAR	5–10 min62	Improved team performance, team efficacy, team communication and cohesion after training scenarios;70 Improved safety norms72	Improved psychological safety72
Debrief or huddle	Approximately 30 min for team86	Effective mechanism to reflect on staff performance after an adverse event74	Improved team culture;78 Decrease in medical complications;83 Decrease in adverse drug events;86 May mitigate the ‘second victim’;95 ‘create a culture of collaboration and collegiality that increases the staff's quality of collective awareness and enhanced capacity for eliminating patient harm’;97 Reduce compassion fatigue99
LFD tool	Associated with decreased nurse turnover when used as part of CUSP104 105	CUSP improved teams’ ability to identify risk and solutions.104	Implementation of CUSP teams was associated with a decrease in length of stay and medication errors105	Improved safety culture and climate when used as part of CUSP102 104 105
SWARM	Suggested 1 hour for multidisciplinary team106	75% of SWARMS occur within 16 days of event106	Decrease in pressure ulcers during treatment; decrease in the observed-to-expected mortality ratio; improved staff culture106	Improved safety culture106
CIA	Measured average of 11 person-hours for multidisciplinary team107	89% of test sites rated tool ‘Easy’ or ‘Very Easy’ to use; 89% rated tool as ‘Effective’ or ‘Very Effective’; 67% of action items were implemented107
‘Concise tool’ from the NHS and Canadian Incident Analysis Framework	The Canadian Incident Analysis Framework uses the CIA tool cited above.	The Canadian Incident Analysis Framework uses the CIA tool cited above
Aggregate RCA/Multi-Incident Analysis	Measured average of 87.5 person-hours; median and mode are 60 person-hours (N=697).122	61.4% of the recommended actions were implemented114	Decrease in falls with injury;113 114 114

AAR, After-Action Review; CIA, Concise Incident Analysis; CUSP, comprehensive unit-based safety programme; LFD, Learn From Defect; NHS, National Health Service; RCA, root cause analysis.

After-Action Review (AAR)

After-Action Review (AAR) originated in the US military to extract lessons from training and missions but has been adapted to healthcare. Its original purpose was ‘to help Army leaders adapt quickly in the dynamic, unpredictable situations they were sure to face’.59 60

Several investigators have modified the military's 10-step AAR for healthcare.61–63 Each investigator suggests at least four questions: ‘What was expected to happen? What actually happened? What is the difference between these? What has been learnt?’63 A trained facilitator improves the quality of reflection and resultant strong, measurable actions. The AAR takes place in the team's work environment. This method of AAR can be a means to reflect on training, a work shift or IR.63 64 There are also examples of variations of AAR used after public health crises and mass casualty events.65–69 Sawyer et al suggest that AAR provides a structure to enhance team reflection, learning and enlightened action.61 Available research demonstrates improved team performance after AAR in training scenarios,70–72 but does not specifically address evaluating IRs in healthcare organisations.

Adverse event Debriefing and Huddles

Debriefing, including after event huddles, assume varying structures but share the core characteristic of process-oriented, ‘rigorous reflection’ to find actionable solutions.73 The literature uses the terms huddle and debriefing interchangeably.74 75 They frequently occur as part of a clinical workflow. A team may debrief after a procedure such as surgery. Checklist driven postoperative debriefings are recommended by the VHA and WHO.76–78 Debriefings and huddles may also take place after a safety event, such as a fall.74 75 79–89

When a debrief or huddle retrospectively assesses a safety event, several recommend three stages. These are the (1) reaction phase, (2) understanding or analysis phase and (3) summary phase.73 90 One study provided a checklist.74 All formats initiate group reflection soon after the incident.74 77 91 Often a trained facilitator guides a team. Several actionable items are generated to mitigate similar future incidents.74 92 93 One paper suggests that a postadverse event debriefing could act as a precursor to a more in-depth RCA.74

Literature suggests numerous benefits for debriefing in healthcare. They have been linked to improved safety, team culture and clinical performance.74 75 77 85 94–96 Team members report feeling supported by colleagues after an adverse event debriefing.74 97 They can also reduce provider stress and burnout associated with second-victim syndrome, where a caregiver is deeply stressed by his/her association with an adverse event.95 98 99

Learn From Defect (LFD) tool

Johns Hopkins Medicine developed the Learn From Defect (LFD) tool to systematically guide clinical teams to improve clinical performance in the comprehensive unit-based safety programme (CUSP).100 101 CUSP teams use the LFD tool to analyse one defect per month. All staff who were directly or indirectly involved in the defect participate in the investigation. After describing the event, the format guides the team to reflect on various potential contributing factors and whether the factors positively or negatively contributed to the defect. Last, the team creates measurable interventions to prevent future, similar harm.101

The LFD tool has been evaluated as part of CUSP in various healthcare settings.102–105 The CUSP programme, which includes other quality improvement tools, was associated with an improved safety culture and improved clinical performance.102 105 Furthermore, CUSP equips staff with improved ability to identify risks and create meaningful interventions.104

SWARM

The University of Kentucky HealthCare Lexington introduced the ‘SWARMing’ process to improve on their RCA system building on the idea of ‘swarming intelligence’.106 An IR is reviewed by a department administrator who initiates a preliminary investigation. If it merits a brief review, a multidisciplinary team is chartered including individuals involved in the event.106 The team progresses through five phases within an hour. The first two phases train team members on the purpose of the SWARM and review ground rules. The known facts of the incident are reviewed in the third phase. A root cause is determined in the fourth. In the fifth phase, the team develops actions to prevent a similar incident from recurring. An assigned action owner has 60 days to ensure that the task leaders fulfil their duties. Finally, a report is distributed among organisational leaders highlighting the incident, root causes and remedial actions.106

After over 1200 SWARMs, UK HealthCare Lexington studied the effectiveness of its tool. The number of IRs increased by 52%. Across the healthcare system, there was a ‘37% decrease in the observed-to-expected mortality ratio’.106 While preliminary evidence suggests an improved patient safety and safety culture, UK HealthCare Lexington wants to improve the implementation and sustainment of the action plans.106

Concise Incident Analysis (CIA)

The Concise Incident Analysis (CIA) was developed in 2013 under the guidance of an advisory group from the WHO Reporting and Learning Systems Community with several objectives. One was to develop an efficient tool to analyse more IRs than the RCA process. It was also aimed to analyse no-harm and low-harm incidents at the unit-level to empower caregivers and strengthen safety culture at the front line.107 CIA is administered by a trained facilitator in a small group. A checklist guides the investigation. Like others, it questions what happened and why it occurred. Further, it leads the team to develop measurable actions.107

The CIA development team surveyed the pilot sites. Among the study sites, most groups (59%) found it helpful when implementing actions and 95% learnt from the analysis. While 94% shared their lessons within the organisation, only 6% shared their lessons outside the organisation.107 Yet, limitations exist. One is that it may be prone to bias since those involved in the incident are also among the group analysing the IR.107 Second, the quality of analysis likely rests in the capability of the facilitator. Last, while the tool will allow more IR investigations, the CIA tool was not designed to study sentinel events, which should be analysed by a more robust tool such as RCA.107

Comprehensive frameworks for incident report investigation and analysis

Both the National Health Service (NHS) in the UK and the Canadian healthcare system developed multiple learning tools to investigate IRs. NHS analyses IRs in one of three levels. The Concise Investigation is used for Level 1, which are no-harm, low-harm and moderate-harm events.108 Serious incidents and ‘never events’ fall under Level 2 or 3 and are investigated using the Comprehensive or Independent Investigation Reports, which are similar in scope to RCAs required by VHA and The Joint Commission.6 8 109 110 The Canadian Incident Analysis Framework adopted the Concise Incident Analysis (CIA) described in the previous section for no-harm and low-harm events. More serious events are investigated with a Comprehensive Analysis, which also mirrors RCAs required by the VHA, and The Joint Commission.6

There are few differences between NHS’s and the Canadian framework’s concise tools. Each is led by a trained facilitator and seeks to determine the root cause of the incident. The Canadian tool forms a team only of those proximal to the adverse event.6 Similarly, NHS tool has a local investigation team, but also encourages patients and or relatives to be involved in the investigation if they were directly affected. Both systems develop a report and encourage lessons to be shared in the work unit, with organisational leadership, or nationally, if appropriate.6 8 107 Each tool encourages evaluation of the resulting actions from the analysis. While there are no studies evaluating the NHS’s Concise Investigation, a 2015 ombudsman report was critical of the entire framework’s function. Investigations, analyses and actions were not consistently executed as designed.111 112 The feasibility and effectiveness of the Canadian concise tool matches that of the CIA tool (table 1).107

Aggregate RCAs and the multi-incident analysis

Aggregate RCAs are different than the other forms of investigation discussed. Here, multiple IRs can be analysed in one RCA. This is practiced both within the VHA and the Canadian Incident Analysis Framework.6 113 114 The Canadian framework offers guidance for the use of their Multi-Incident Analysis. Scenarios where it can be used include: (1) a series of low-harm and no-harm events, (2) a group of events similar in origin or composition, (3) a group of patients inhibited by similar factors such as patients receiving care in an emergency department who do not receive the septic bundle within an acceptable time frame and (4) an analysis of a group of completed safety investigations.6

There has been little evaluation of aggregate RCAs. In one study, investigators interviewed 97 VA medical centres teams, who had completed 176 aggregate RCAs on falls. These teams reported a reduced rate of falls in 34.4% of the locations. 43.8% reported no change and 20.8% stated that it was too soon to tell.114 The rates of falls with injuries similarly improved at some facilities, but did not at others.114 Aggregate RCA has also been recommended as a method to improve the ‘bird's eye view of incidents’ across an organisation. Trends can be identified to determine systemic root causes.51 The primary limitation of aggregate RCA is the significant person-hours required.

Discussion

In this structured narrative literature review, we identified and described seven tools to analyse IRs, which have a lower time investment and less depth of analysis than traditional RCA. Several of these tools have already been incorporated into tiered frameworks that guide investigative teams to the appropriate tool in large health systems. In these frameworks, IRs resulting in greater harm or perceived potential for future harm receive more resource-intense investigations.

The brief tools differ from RCA in several characteristics (table 1). One, staff involved in the IR are members of the multidisciplinary investigation team. This should speed up the fact-finding process. Yet, it may bias the investigation and analysis. RCA seeks greater objectivity by requiring a third-party investigation. The tools also diverge from RCA on their method to share lessons from the analysis. AAR does not mention sharing lessons beyond the work group. Campbell et al suggest that their debrief tool may act as an initial screen prior to a RCA.74 The SWARM reports the analysis to leadership like RCA. Leadership shares the lessons as needed.106 A similarity among all concise tools and RCA is the recommendation for a trained facilitator. In evaluating the CIA tool, Pham et al remarked that the quality of investigation likely rests on the ability of the facilitator.107 This observation is likely true of all the analytical tools discussed.

The literature describing and evaluating the tools is not complete. The organisational context for each tool should be clear.115 An investigative tool may function better in a specific work environment. For example, the LFD tool was evaluated as part of the CUSP team approach in multiple settings.102–105 This tool is effective in a small clinical team environment. It may not function well if practiced in a multidisciplinary team recruited from diverse areas of a medical centre to address a system-wide problem. Second, each tool should be validated to promote safety and a safety culture, which several have done.36 76 77 105 106 114 Some studies on huddles and debriefs have also analysed their effects on team functioning,101 102 105 106 which influences both safety culture and potentially clinical outcomes. Third, tools should demonstrate improved clinical processes or outcomes. RCAs have been critiqued on the strength and implementation percentage of their actions.28 32–36 56 116 Better outcomes can include a decrease in the observed-to-expected mortality ratio as in the SWARM study.27 28 106

Our study had limitations. Some suggest that morbidity and mortality (M&M) rounds are a method to learn from adverse events.117–121 Yet, this review did not feature M&M rounds. They retrospectively present on the resolution of a defect in care.118 It does not actively guide a group to systematically discern root causes and remedial actions for an IR. Second, there was limited literature evaluating the brief analysis tools. The reports were often descriptive and lacked performance metrics. Divergent methods were used to evaluate each tool. The evaluation of the CIA tool described the safety event characteristics and then surveyed the investigation teams on their perceptions of the ease of use and effectiveness of the tool.107 Other studies described their tool’s effects on safety and safety culture.36 76 77 105 106 114 Still others studied its influence on standardised outcome measures like observed-to-expected mortality.106 Emotional and social aspects of the tools were also measured differently. Some assessed the tool’s effects on staff stress and compassion fatigue.98 99 The LFD tool noted a decrease in nurse turnover.104 105 Each tool is measured against different criteria making comparisons challenging. The diverse measures highlight the difficulty of defining effectiveness in safety investigations. Third, we may not have found all relevant articles due to the non-standardised titles of the tools and search terms. Additional tools exist that have not been evaluated in the literature. Our review falls into the wider emerging literature that has pointed out limitations of the commonly occurring RCA system.23 25 48 50–52 57 Most of that literature has focused on the short comings of RCA as an investigative tool. We have examined alternative approaches.

It is clear that healthcare organisations need systematic frameworks to learn from error.48 50 51 Each medical error potentially provides new knowledge to improve the reliability of a healthcare system. Yet, the best method to investigate is unclear. Each tool has trade-offs. A huddle benefits from its proximity to a medical error but lacks the objectivity of a resource-intense third-party investigation. Aggregate analysis tools can potentially optimise feasibility and effectiveness by studying multiple no-harm and low-harm medical error at once.21 48 Thus, several tools can be uniquely combined into a coherent system like the British and Canadian frameworks.6 8 The appropriate investigative framework may vary on the context of the healthcare organisation. An organisation’s chosen tools should enable it to improve the reliability, safety and quality of care provided to their local community.