Teaching Root Cause Analysis Using Simulation: Curriculum and Outcomes.

BACKGROUND: Clinicians are key drivers for improving health care quality and safety. However, some may lack experience in quality improvement and patient safety (QI/PS) methodologies, including root cause analysis (RCA).
OBJECTIVE: The Department of Veterans Affairs (VA) sought to develop a simulation approach to teach clinicians from the VA's Chief Resident in Quality and Safety program about RCA. We report the use of experiential learning to teach RCA, and clinicians' preparedness to conduct and teach RCA post-training. We provide curriculum details and materials to be adapted for widespread use.
METHODS: The course was designed to meet the learning objectives through simulation. We developed course materials, including presentations, a role-playing case, and an elaborate RCA case. Learning objectives included (1) basic structure of RCA, (2) process flow diagramming, (3) collecting information for RCA, (4) cause and effect diagramming, and (5) identifying actions and outcomes. We administered a voluntary, web-based survey in November 2016 to participants (N = 114) post-training to assess their competency with RCA.
RESULTS: A total of 93 individuals completed the survey of the 114 invited to participate, culminating an 82% response rate. Nearly all respondents (99%, N = 92) reported feeling at least moderately to extremely prepared to conduct and teach RCA post-training. Most respondents reported feeling very to extremely prepared to conduct and teach RCA (77%, N = 72).
CONCLUSIONS: Experiential learning involving simulations may be effective to improve clinicians' competency in QI/PS practices, including RCA. Further research is warranted to understand how the training affects clinicians' capacity to participate in real RCA teams post-training, as well as applicability to other disciplines and interdisciplinary teams.

Introduction

The Institute of Medicine maintains that patient safety and unintentional patient harm is an urgent public health issue.[1] Patient harm is often a consequence of medical errors, which can involve system-based mistakes, including missed or delayed diagnoses, miscommunication between care teams, and misadministration of medication.[2-5] The resulting impact on the patient ranges from no harm to death.[2-5] While current initiatives and collaborative practices strive to make patient care safer, adverse events still occur. In the United States, medical errors have emerged as the third leading cause of death, accounting for more than 250 000 deaths each year.[6] This further emphasizes the need to improve patient safety. Common efforts include standardization of care processes, or recognizing and improving specific aspects of care.[2-5] Moreover, hospitals dedicated to patient safety may use a more foundational approach, such as establishing and promoting a patient safety culture.[7,8] This may result in system-wide changes that improve a broader range of care aspects. One framework for this approach is high-reliability organizations (HROs), which are often used in industries such as commercial aviation.[7,8]

High-reliability organizations adhere to 5 core principles: (1) sensitivity to operations and how they affect the organization; (2) reluctance to accept simple explanations and striving to identify the root source of problems; (3) preoccupation with failure including anticipating and correcting potential problems before they occur; (4) deference to individuals with expertise of the particular task at hand, regardless of hierarchy; and (5) commitment to resilience, and adaptive when developing new solutions to unexpected problems.[7,8] As a result, major harm events in HROs are rare.[7,8] When applied to the health care system, a culture of safety means identifying systematic causes contributing to patient harm, encouraging error reporting and removing individual blame, and promoting multi-level collaboration to develop solutions that may prevent similar errors from happening again.[7,8] There are existing tools intended to support this process, such as root cause analysis (RCA).[9,10] Clinicians in particular are key drivers for improving health care, but some may lack experience or knowledge in the use of such systems-based practices or quality improvement and patient safety (QI/PS) methodologies, including how to conduct an RCA.[11-13]

Although RCA has been promoted as both tangible evidence of high reliability and an important tool for improving patient safety, literature suggests that execution of RCA within the health care system is often suboptimal.[13,14] One leading cause of this is a lack of knowledge and skill of personnel carrying out the RCA.[15,16] There is substantial literature outlining the ideal RCA process; however, little information is available regarding a formal curriculum for RCA in the published literature.[15,16] Kung et al[17] developed and implemented a case-based teaching method involving a 1-hour presentation on RCA for radiology residents. In addition, Lambton and Mahlmeister[18] paired a simulation activity with RCA training for nursing students. While these 2 studies put forward the concept of a simulation-based method to teach RCA, both describe a more superficial overview of the RCA process rather than a structured curriculum.

This work is unique in that we report on our development and execution of an interactive, simulation-based method to teach RCA in a short session. The purpose of this report is to share the curriculum and results of using RCA simulations for training, and clinicians’ preparedness to conduct and teach RCA after the training. Furthermore, this study provides educational materials and details on the curriculum to teach RCA using a low-technology simulation method.

Methods

This project was reviewed and approved by the Research and Development Committee, White River Junction, VA Medical Center. Participants were US-based clinicians from various medical specialties in the Department of Veterans Affairs (VA) Chief Resident in Quality and Safety (CRQS) training program. The CRQS program was developed for residents to gain experience teaching QI/PS concepts and leading related projects.[19,20]

Conceptual model

The overall course design was informed by adult learning theory (Figure 1). This conceptual model is based on the work of Kolb et al, which suggests that learning is best accomplished through concrete experience, followed by reflection and discussion. Participants are exposed to new information or have a concrete experience, actively practice what was learned with example cases, reflect on the experience and ask questions, and internalize or learn the abstract concepts through group discussion.[21,22]

Figure 1.

Conceptual model of optimal learning through simulation, adapted from Kolb et al.[21,22]

Learning objectives

The course learning objectives were developed through consensus discussion among patient safety experts. These objectives were based on the course’s goal to develop clinicians’ ability to actively engage in patient safety activities such as RCA. The learning objectives included (1) basic structure of an RCA (eg, main components and how it is done), (2) how to do an initial understanding and final event flow diagram, (3) how to collect information for RCA (eg, interviews), (4) how to do cause and effect diagramming to obtain actionable items (eg, fishbone diagram), and (5) how to develop actions and outcomes. We developed course materials, including presentations, a role-playing case, and an elaborate RCA case. Presented information may be found in Table 1, and case summaries may be found in Table 2. The full version of these materials is available online and may be modified to meet user needs.

Table 1.

RCA presentation content and take-home message for each learning objective.[23]

Learning objective	PowerPoint content	Take-home message
1. Basic structure of an RCA	Introduction to RCA Purpose of an RCA In complex systems such as health care, reporting medical errors and evaluating their causes is necessary for improving safety. RCA is a structured and standardized method for investigating the causes of medical errors by focusing on the systematic and organizational factors that led to the adverse event. RCAs are useful for determining what happened, why it happened, and what can be done to prevent it from happening again. When to conduct an RCA RCAs should be done following a known adverse event, close call or near miss adverse event, or an event that resulted in an unexpected negative outcome. The Joint Commission sentinel event policy also requires RCAs following sentinel events such as wrong-site surgery. In the VA, RCAs are completed locally by hospital staff and are sent to the NCPS and cataloged. Following an adverse event, the patient safety manager at each VA facility determines whether to commission an RCA team by using a matrix that incorporates the severity, frequency, and vulnerability to recurrence. The patient safety manager is trained in RCA and patient safety protocols and procedures, and facilitates teams to conduct the RCA. When NOT to conduct an RCA RCAs should not be done when the event is the result of an illegal act of someone in the hospital (eg, angel of death or taking medications on purpose), when the event resulted from an intentionally unsafe act (eg, wheelchair races), when illegal drug or alcohol use is involved (eg, nursing assistant came to work intoxicated), or when patient abuse is suspected. Legal aspects of RCA Information from RCA cannot be obtained from lawsuits, and RCA data should be de-identified during the process. In the VHA, the information cannot be discussed under 5705 protections. Furthermore, RCA cannot be used for personnel action. Staff cannot be punished, fired, or reprimanded based on what is learned in RCA.	• The goal is to explore and identify root causes and systems issues rather than blame individuals.
2. How to do process flow diagramming	Process flow diagramming Purpose of a process flow diagram A process flow diagram may be used to gain an initial understanding of what is known and what is not known about the event, and develop a plan to further investigate the event. Process flow diagram rules Process flow diagramming is based on the sequence of events, and should describe the steps and timeline of the adverse event. It would include actions leading up to, during, and after the case.	• Think about what actually happened and use the questions that arise to inform the interviews. The goal is to determine the “holes in the story” and to end up with a final understanding of the flow diagram. This goal makes it different from a flow diagram used for quality improvement purposes, which is a sequencing of events. • It might be easy for 1 person to lead this process, but it is essential that all members participate to get a factual and holistic understanding of what happened.
3. RCA data collection: interviews	Interviews Once an initial understanding process flow diagram is created, missing elements of the event and additional questions may be clarified through staff interviews. The interview should begin with informing the subject of the RCA process and its protections. Interviewers should be clear about what they hope to learn from the interview and remain sensitive to the subject’s feelings.	• Everything that is documented in the medical record did not always happen, and everything that happened is not always documented in the medical record; although this is ideal, participants need to seek the truth of what actually happened. • To truly understand what went wrong, you have to go to the place of the incident and examine it. • People may be emotional during interviews, and it is the interviewer’s job to compassionately respond to these emotions while doing their best to obtain information. The interviewee needs to be assured that no repercussions will result from the interview, or they may not openly share about the case.
4. How to do cause and effect diagramming	Cause and effect diagramming The goal of a cause and effect diagram is to help determine root causes and contributing factors of the event. The cause and effect diagram should follow the 5 rules of causation. (1) Clearly show the cause and effect relationship (eg, residents are scheduled 80 hours per week; as a result, fatigued residents are more likely to misread instructions, leading to error), (2) use specific and accurate descriptors for what occurred rather than negative and vague words (eg, the pump manual had 8-point font and no illustrations; as a result, nursing staff rarely used it, increasing the likelihood of incorrect pump programming), (3) identify the preceding cause(s), not the human error (eg, due to the lack of automated software to check high risk medication dosage limits, there was increased likelihood of insulin overdose), (4) identify the preceding cause(s) of procedure violations (eg, noise and confusion in the preparation area, coupled with production pressures, increased the likelihood of missing steps in the CT scan protocol), and (5) failure to act is only causal when there is a preexisting duty to act (eg, the absence of an assignment for designated nurses to check for STAT orders every 30 minutes increased the likelihood that STAT orders would be missed or delayed). Develop a fishbone or cause and effect diagram by identifying the problem statement and the factors (eg, people, process/methods, equipment, materials, measurements, environment) that contributed to the problem.	• Identifying root causes are more akin to hypothesis of causes. When actions are implemented and evaluated, they provide information on if the root causes were on target. • Ask “why” multiple times because what appears to be the root cause may not always be the root cause. It may be a surface-level factor, which requires further investigation to arrive at the system-level issue.
5. Identifying actions and outcomes	Actions and outcomes When all necessary information is gathered and diagrams are finalized, potential actions and outcomes may be identified. Actions should be driven by root causes and may follow a hierarchy of actions by Bagian et al. This involves aiming for strong actions whenever possible. Stronger actions include architectural/physical plant changes, usability testing, or engineering control (forcing functions). Although these actions may be costly, they involve changes to the physical environment that remain constant over time and may be ideal to support lasting improvements. Intermediate actions may help streamline workflow and include checklists/cognitive aids, software enhancements/modifications, or implementing tools for enhanced communication and documentation. Finally, weaker actions involve double checks, warnings and labels, or training. Such actions may prove effective when coupled with stronger actions; however, they are considered weaker because they require reminders or re-training of staff. Outcomes should be driven by the event and potential solution (actions).	• Unless actions are implemented and outcomes are evaluated, similar adverse events may occur.

Abbreviations: CT, computed tomography; NCPS, National Center for Patient Safety; RCA, root cause analysis; VA, Department of Veterans Affairs; VHA, Veterans Health Administration.

Table 2.

Case summaries.

Case	Summary
Sticky eyeball case	The case of the sticky eyeball takes place in a crowded, busy ER at a teaching hospital. A 3-year-old child fell and struck his right supra-orbital ridge on the corner of a coffee table, which requires sutures. The mother requests liquid topical adhesive to prevent scar formation. The overwhelmed ER attending instructs his resident to perform the procedure (who has 2 previous, supervised experiences using the product), and asks an ER nurse to provide the resident with the necessary materials. The procedure is performed in a cramped room that was previously a janitor’s closet. The child began moving and crying during the procedure, and the resident dripped glue into the child’s eye. The ER attending wants to know how this could have happened.
Mr Smith fictitious RCA case	Mr Smith is a young veteran of the war in Iraq and Afghanistan. He presented to the outpatient clinic for pre-surgical labs in anticipation of his scheduled (right) ACL repair the following morning. Mr Smith was admitted overnight primarily due to transportation difficulty. The ACL repair was uneventful, and Mr Smith appeared stable following surgery. During post-surgical rounds, he mentioned that physical therapy aggravated an old high school football injury in his left knee. Dr Martin saw Mr Smith again on Friday afternoon and then handed off care to Dr Miller for the weekend. On Saturday, Mr Smith was discharged home and received transportation by his cousin. One week later, Mr Smith’s mother called the outpatient clinic to cancel his follow-up appointment. The clerk rescheduled for the same time slot the following week. Two days later, Mr Smith passed away at the university hospital in his hometown. Although his family declined an autopsy, he was found to have sepsis, which likely was the cause of death.

Abbreviations: ACL, anterior cruciate ligament; ER, emergency room; RCA, root cause analysis.

Course structure: 3 parts

The course is taught in a workshop format. It involves brief lectures, followed by exercises and discussions facilitated by faculty. Most exercises are done in small groups of 3 to 5 learners.

Part 1 (approximately 45 minutes)

The course begins with a presentation (10 minutes) introducing RCA concepts and how to construct a process flow diagram. Please see Supplemental Appendix A for the full presentation. Participants read and act out the role-play case (5 minutes; Supplemental Appendix B). They practice constructing an initial understanding process flow diagram with the given information, and are encouraged to reflect on the process (eg, what was the purpose, what was easy and/or hard) and ask questions of the course instructors (20 minutes). This exercise is followed by group discussion of RCA structure and process flow diagramming, and participants are instructed to make a list of questions about the case that they still want to know to complete the final process diagram (10 minutes).

Part 2 (approximately 45 minutes)

Part 2 of the course is parallel to Part 1. Several additional components of RCA are described that largely focus on the investigative aspects of RCA, including how to conduct interviews and perform site visits. Cause and effect diagrams as well as construction of actions and outcomes are also described. Participants use their unanswered questions from Part 1 to structure their interviews, and practice interviewing simulated providers. They use the information obtained from the interviews to complete their cause and effect diagram, and identify actions and outcomes (35 minutes). They are encouraged to reflect on the process (eg, did they have enough information to determine causality, did they go backward enough in the causal cascade, what was easy and/or hard about the process), as well as ask questions of the course instructors. This exercise is followed by group discussion (10 minutes). If pressed for time, faculty could proceed directly to the more elaborate RCA case to practice skills instead of conducting mock interviews on the role-play case. Please see Supplemental Appendix C for a sample agenda.

Part 3 (approximately 2 hours)

After the initial exposure and experience, participants practice again with a more complex RCA case. Participants apply the learned skills from Parts 1 and 2 to entirely new information, and they gain experience conducting an RCA from start to finish. They are first given a simulated medical record (Supplemental Appendix D) that includes an adverse event, and learners construct an initial flow diagram of the event (Supplemental Appendix E). Please see Supplemental Appendix F for the RCA medical unit diagram. Based on this initial understanding of the case, learner groups develop a list of people from the case that they would want to interview and draft a list of questions for each interview. The instructors coach the learner RCA teams on how to conduct interviews during RCA. Participants then conduct interviews with faculty who play the roles of the various staff involved in the case. The role-play simulation is directed by an interview guide for each simulated role (Supplemental Appendix G). These guides inform the individuals simulating the roles about what information they know, and how they should behave during the interview. After completing the interviews, each group completes a final understanding process map (Supplemental Appendix H), constructs a cause and effect diagram (Supplemental Appendix I), and identifies actions and outcomes. This section of the course lasts for approximately 1 hour 45 minutes.

Debriefing (15 minutes)

The course ends with a 15-minute debriefing exercise. Learners have an opportunity to discuss any lingering unclear points. The focus of the debriefing is a discussion of planned involvement with future RCAs at their institution.

Course evaluation

External evaluators administered a voluntary, non-incentivized, web-based survey to participants following RCA training in November 2016 (N = 114) using Qualtrics (Provo, UT, USA). The goal of this survey was to evaluate participants’ preparedness to conduct RCA and teach the key principles of RCA. The question types were Likert-type scale (1 = not at all, 5 = extremely), and we calculated the frequency as a basic descriptive statistics.

Results

Participants

A total of 93 individuals completed the survey of the 114 invited to participate, culminating an 82% response rate.

Preparedness to conduct and teach RCA

Nearly all respondents (99%, N = 92) reported feeling at least moderately to extremely prepared to conduct and teach RCA following training (Figure 2). The majority of respondents reported feeling very to extremely prepared to conduct and teach RCA (77%, N = 72).

Figure 2.

Reported preparedness to conduct and teach root cause analysis among 93 alumni of the Chief Resident in Quality and Safety program. Chief Resident in Quality and Safety alumni are US-based clinicians from various medical specialties. The data presented are Likert-type scale (1 = not at all, 5 = extremely) and were collected through a voluntary, non-incentivized, web-based survey administered in November 2016.

Participant feedback

We report a few antidotes from participants but did not receive enough open-ended responses to conduct any analyses.

“The content was very useful and practical, and designed to engage learners at all levels.”

“I enjoyed the interactive nature of the class, there was never an opportunity to check out. I felt each session was very hands-on, which helped me to remember major concepts and implement the concepts we were being taught. I also thought that the manner in which the information was presented gave me ideas for ways I could teach the residents at my program.”

“The hands-on, practical approach to the session in which we utilized real-world examples was useful and effective.”

“Some of the material was a little basic, and we could have benefited from more in-depth knowledge and discussion of the topic.”

Discussion

To achieve the goal of safer health care aligned with high-reliability principles, heath care workers must gain an understanding of event investigation tools such as RCA. It is highly desirable to have personnel specifically trained to participate in RCA teams, and we provide an easy to implement, low-technology, simulation-based RCA curriculum to aid in this process. We tested our curriculum on clinicians of the national CRQS program and evaluated their perceived sense of competency with RCA following training. Overall, physician attendees reported better confidence in their ability to conduct and teach RCA with the training they received. The novelty of this work is that it provides the materials to simulate an RCA in a relatively brief time period (4-5 hours). Studies have demonstrated that to maximize learning, participants need to experience, reflect, and engage in active experimentation to make abstract concepts meaningful.[12,21,24-27] Participants appeared to enjoy this hands-on, interactive, and practical aspect of the course. However, we suggest modifying the difficultly of the course content depending on participants’ experience with the topics to ensure they remain engaged. At the start of the course, we suggest asking participants whether they have participated in an RCA team or whether they have been asked to teach about RCA.

We acknowledge several limitations to this study. Our evaluation strategy was not mature, and we ideally would need to evaluate participants’ performance on a real RCA team following training to obtain a better performance assessment. We are unable to make any definitive conclusions on their knowledge gained because we did not have an RCA testing scenario, and we did not assess for a baseline knowledge score before participating in the course. Furthermore, clinicians’ experience with QI/PS, including RCA, may have differed prior to course involvement. This study may also be subject to sampling bias, because our participants are residents that took 1-year off to participate in the CRQS program, and were eager to learn QI/PS methodology and tools. Therefore, it is unclear how this method would work on mid-career providers or medical students, where QI/PS is not typically a core aspect of their training.

Experiential learning is not a new teaching modality, and this advanced training is commonly used for various educational purposes, including graduate medical education, clerkships, and residency.[12,19,24,25,27-31] However, we believe that application of these methods including the use of simulation to teach about content specifically related to quality and patient safety is novel. Students and clinicians master what they learn in the classroom by practicing with patient actors and real-life cases. Furthermore, this method is not specific to only clinicians and may be applied to any discipline or interdisciplinary team. Watts et al[26] report that experiential learning worked well when used for industrial engineers and related industries. Increasing emphasis has been placed on educating clinicians regarding ways to deliver safer, higher quality care, and some graduate medical programs are incorporating QI/PS into their curriculum.[12,24,25,29,31]

The Accreditation Council for Graduate Medical Education (ACGME) developed the Clinical Learning Environment Review (CLER) program to provide guidance for an optimal clinical learning environment.[11,32] The goal of CLER is to promote experiential learning and enhance clinicians’ competency in QI/PS practices.[11,32] Clinical Learning Environment Review focuses on the idea that QI/PS work should be similar to clinical care training, where the expectation is that clinicians should be directly involved in the work to gain proficiency with such skills.[11,32] As health care systems move toward meeting the goals of CLER, they should try to provide both basic and advanced education on tools for QI/PS, including RCA.[11,32,33] We provide an example of how this may be achieved. In addition, Boussat et al[33] developed a simple model for standardized RCA reporting that presents causes and corrective actions, and may be used during time-constraining circumstances. Overall, clinicians reported enjoying the course, and our hope is that it will enhance their capacity to meaningfully participate in RCA teams by the end of the training.

Conclusions

Clinicians are leaders in patient safety and integral to the success of health care improvement initiatives. However, to meaningfully participate in such practices, they need to be trained in QI/PS methodologies and tools, including RCA. We report success with RCA simulations and experiential learning to improve their competency in QI/PS practices. Further research is needed to understand how the training affects clinicians’ capacity to participate in actual RCA teams following training, as well as applicability to other disciplines and interdisciplinary teams.

Click here for additional data file.

Supplemental material, AppendixA_RCA_Powerpoint_xyz27040aa30c22c for Teaching Root Cause Analysis Using Simulation: Curriculum and Outcomes by Maya Aboumrad, Julia Neily and Bradley V Watts in Journal of Medical Education and Curricular Development

Click here for additional data file.

Supplemental material, AppendixB_DocUDrama_Sticky_Eyeball_xyz27040c674734e for Teaching Root Cause Analysis Using Simulation: Curriculum and Outcomes by Maya Aboumrad, Julia Neily and Bradley V Watts in Journal of Medical Education and Curricular Development

Click here for additional data file.

Supplemental material, AppendixC_RCA_Session_SampleAgenda_xyz27040c148b9c5 for Teaching Root Cause Analysis Using Simulation: Curriculum and Outcomes by Maya Aboumrad, Julia Neily and Bradley V Watts in Journal of Medical Education and Curricular Development

Click here for additional data file.

Supplemental material, AppendixD_RCA_Case_Medical_Chart_xyz27040b5af615f for Teaching Root Cause Analysis Using Simulation: Curriculum and Outcomes by Maya Aboumrad, Julia Neily and Bradley V Watts in Journal of Medical Education and Curricular Development

Click here for additional data file.

Supplemental material, AppendixF_RCA_Medical_Unit_Diagram__xyz270402ac53212 for Teaching Root Cause Analysis Using Simulation: Curriculum and Outcomes by Maya Aboumrad, Julia Neily and Bradley V Watts in Journal of Medical Education and Curricular Development

Click here for additional data file.

Supplemental material, AppendixG_RCA_Simulated_Interview_Key_xyz27040d8527b6c for Teaching Root Cause Analysis Using Simulation: Curriculum and Outcomes by Maya Aboumrad, Julia Neily and Bradley V Watts in Journal of Medical Education and Curricular Development

Click here for additional data file.

Supplemental material, AppendixH_Final_Flow_Diagram_xyz27040b1edcd2d for Teaching Root Cause Analysis Using Simulation: Curriculum and Outcomes by Maya Aboumrad, Julia Neily and Bradley V Watts in Journal of Medical Education and Curricular Development

Click here for additional data file.

Supplemental material, AppendixI_Cause_and_Effect_Diagram_Answer_Key_xyz270409a808698 for Teaching Root Cause Analysis Using Simulation: Curriculum and Outcomes by Maya Aboumrad, Julia Neily and Bradley V Watts in Journal of Medical Education and Curricular Development

Click here for additional data file.

Supplemental material, Appendix_E_Initial_Flow_Diagram_xyz27040ca1139cf for Teaching Root Cause Analysis Using Simulation: Curriculum and Outcomes by Maya Aboumrad, Julia Neily and Bradley V Watts in Journal of Medical Education and Curricular Development