An Integrated, Multimodal Resident Curriculum in Patient Safety and Quality Improvement.

Introduction: Patient safety and quality improvement are essential components of modern medicine. The traditional model of graduate medical education does not lend itself well to learning these disciplines. This curriculum encompasses these disciplines across multiple modalities and extends throughout residency.
Methods: The curriculum includes introductory presentations suitable for naive audiences. Following these is a structured rotation that provides the opportunity both to experience in-depth self-directed learning and to practice skills involved in quality and safety. This rotation includes existing online courses published elsewhere, reflective writing exercises based on self-directed learning, and practice cases. Finally, residents lead a morbidity, mortality, and improvement conference where adverse events are identified and reviewed, specific interventions and outcome objectives are selected, and action teams are identified.
Results: After two presentations on system issues and individual issues, responses to the prompt "This talk will aid in my professional development" were 4.75 and 4.59 out of 5, respectively. Eighty-three percent of residents agreed they had a better understanding of the concepts of patient safety and/or quality improvement than they did before the rotation. Audience members for the resident-led morbidity, mortality, and improvement conference agreed it would lead to a change in their own practice. Discussion: The contents of this longitudinal curriculum have been incorporated into the core requirements of our general pediatrics residency program and could reasonably be imported into any residency requiring a robust longitudinal experience in quality improvement and patient safety.

Educational Objectives

By the end of this curriculum, learners will be able to:

Discuss principal sources of individual and systems-based errors in health care and strategies to mitigate these errors.

Discuss common techniques for quality improvement in health care.

Propose, implement, and evaluate quality improvement interventions.

Perform a multifactorial review of practice and actual patient safety events in a blame-free manner.

Identify potential interventions to limit risk of future similar events and evaluate their likelihood of success.

Develop goals for action teams addressing these interventions.

Introduction

The disciplines of quality improvement and its counterpart, patient safety, are increasingly important in the health care arena. Patients expect to be well taken care of, payors expect high-quality outcomes with low rates of complications, and accrediting bodies expect/require that physicians have a working knowledge of these concepts in order to maintain certification.[1,2] Nevertheless, the traditional model of graduate medical education, with trainees rotating from one clinical service to the next, limits the ability to teach concepts and techniques that require a baseline longer than 1 calendar month. Additionally, a major component of patient safety involves creating and maintaining an institutional culture that values and rewards thinking that anticipates errors and acts to prevent them before they occur. Such a culture requires active ongoing maintenance.[3]

A needs assessment conducted in conjunction with our transition to the Accreditation Council for Graduate Medical Education's (ACGME's) Next Accreditation System (NAS) demonstrated no consistent mechanism for teaching residents the principles of quality improvement, no mechanism for demonstrating or fostering a safety culture, and no opportunity for teaching residents how to report a safety event. This curriculum was developed to address these needs longitudinally throughout the residency experience. Additionally, the ACGME requires resident assessment across several competencies (patient care, medical knowledge, systems-based practice, practice-based learning and improvement, professionalism, and interpersonal communication skills).[4] Within these competencies are specific milestones that permit evaluation of resident progress throughout a training program.[5] A number of these milestones map to knowledge, skills, behaviors, and attitudes addressed by a program of education in patient safety and quality improvement (Table 1).

Table 1.

Competencies and Milestones Addressed by the Curriculum

Competency	Milestone	Description	Focus
Systems-based practice	SBP 2	Advocate for quality patient care and optimal patient care systems	Emphasizes patient safety
	SBP 3	Work in interprofessional teams to enhance patient safety and improve patient care quality	Advocates for individual and community health
Problem-based learning and improvement	PBLI 1	Systematically analyze practice using quality improvement methods, and implement changes with the goal of practice improvement	Locates, appraises, and assimilates evidence from scientific studies related to patient health problems

Additionally, the ACGME's Clinical Learning Environment Review (CLER) program was introduced as a key component of the NAS in 2012.[6] Per the ACMGE, “the goal of CLER is to provide formative feedback to sponsoring institutions on the effectiveness of resident and fellow engagement in 6 focused areas.”[7] Two of these areas, patient safety and quality improvement, offer challenges for institutions where these activities often occur at a frequency and pace that do not lend themselves well to involvement by residents, who often change clinical assignments every month. This curriculum can aid programs by providing ongoing education and opportunities for engagement with institutional quality and safety activities by addressing several areas identified in the ACGME's CLER Pathways to Excellence (Table 2).[8]

Table 2.

Clinical Learning Environment Review Pathways Addressed by the Curriculum

Focus Area	Pathway	Description
Patient Safety	PS Pathway 1: Reporting of adverse events, close calls (near misses)	Residents, fellows, and faculty members know their roles and responsibilities in reporting patient safety events at the clinical site.
	PS Pathway 2: Education on patient safety	Residents/fellows are engaged in patient safety educational activities where the clinical site's systems-based challenges are presented, and techniques for designing and implementing system changes are discussed.
		Residents/fellows and faculty members receive education on the clinical site's proactive risk assessments (e.g., failure mode and effects analysis).
	PS Pathway 4: Resident/fellow experience in patient safety investigations and follow-up	Residents/fellows participate as team members in real or simulated interprofessional clinical site-sponsored patient safety investigations (such as root cause analyses or other activities that include analysis, as well as formulation and implementation of actions).
Health Care Quality	HQ Pathway 1: Education on quality improvement	Residents/fellows receive progressive education and training on quality improvement that involves experiential learning.
		Residents/fellows are engaged in periodic quality improvement educational activities in which systems-based challenges are highlighted and approaches to designing and implementing system changes are discussed.
		Residents/fellows and faculty members are familiar with the clinical site's priorities for quality improvement.
		Residents/fellows are engaged in periodic quality improvement educational activities in which systems-based challenges are highlighted and approaches to designing and implementing system changes are discussed.

A review of existing resources revealed several previously published tools. Walker, Rubio, Horstman, Trautner, and Stewart developed a workshop for training faculty to mentor residents to deliver a morbidity and mortality (M&M) conference focusing on systems errors, employing validated quality improvement techniques, and developing action items during the conference.[9] Similarly, Garcia and Goolsarran developed a format for M&M conferences that allows for small-group activities to develop error analysis and peer review.[10] Though valuable experiences, these conference formats do not provide for longitudinal education. Tapper, Sullivan, and Tess described a longitudinal-iterative approach to teaching quality improvement through aiding trainees in conducting a plan-do-study-act activity in the inpatient setting.[11] Their resource also includes two didactic sessions on the practical aspects of quality improvement but does not delve into the cognitive and systems sources of errors. Additionally, their resource does not include the revised M&M format, as is also the case in the longitudinal didactic and practical educational modules created by Stewart and colleagues.[12] The curriculum presented here provides a relatively novel, comprehensive program of didactic, self-directed, and practical education in quality improvement and patient safety that follows learners from relative naiveté to being able to implement and evaluate learned concepts and extends throughout the entirety of the residency experience.

Methods

Overview

This curriculum was implemented by the Department of Pediatrics at the Saint Louis University School of Medicine in 2015 and has been continuously operating since. Initial materials were based on existing literature in quality and safety, iteratively modified based on trainee feedback, and updated as novel materials were developed. The curriculum begins with intern orientation in the first week of postgraduate medical training and continues through the third year. A diagram outlining all three parts of this curriculum is included in the Figure. This curriculum has been taught to pediatrics residents but would be broadly applicable to nearly any specialty with some modifications. The Saint Louis University pediatrics residency is a midsized program, with 13 categorical pediatrics residents each year and two pediatrics/neurology combined residents in each of the first 2 years.

Figure.

Curriculum flow sheet. IHI, Institute for Healthcare Improvement.

The curriculum began in year 1 with two lectures that described the systems and cognitive sources of medical error and tools for their mitigation. These talks built on safety and quality information published widely in the aviation and aerospace industries and works on behavioral economics and decision making under stress, quality and process improvement in manufacturing and industry, and human cognition, as well as many excellent works on patient safety in health care.

In the second year, learners spent half of one rotation completing one of a selection of quality and safety activities, observing existing hospital quality and safety programs, and completing existing online modules on quality and safety published elsewhere. This provided learners with the opportunity both to observe the regular and lengthy progression of process improvement in real-world settings and to explore the deeper theoretical underpinnings of the tools, techniques, and culture required for their successful use. Learners were provided access to a library of works on patient safety and quality improvement for self-directed learning and were asked to complete a reflective writing exercise at the end of the rotation.

In the third year, each learner was responsible for identifying a real-world safety event, conducting an investigation into the factors contributing to the event, discussing the findings with the department, and leading the process of identifying an intervention likely to reduce the potential for recurrence. Learners also had to identify individuals to participate in putting the intervention into place and develop SMART (specific, measurable, achievable, realistic, time-limited) goals for the action team. This review process was based on the Learning From Defects tool.[13]

Administration

Phase I

The educational approach and the target audience vary depending on the phase of the curriculum but generally progress along Bloom's taxonomy.[14] Phase I consists of introductory lectures for relatively naive learners who nevertheless have basic knowledge of the multiple systems of care involved in modern health care and a familiarity with the environment of care and the existence of risk in health care. These lectures are included in Appendices A and C (with narration) or Appendices B and D (without narration). Speakers electing to use presentations without narration should review the slides to ensure proper functioning of embedded videos as not all local systems support all audiovisual codecs and formats. If delivered without narration by local instructors, this phase requires 1 hour for each of the two presentations, as well as suitable screen/projector and meeting room resources. These presentations have historically been incorporated into a regular noontime series of didactic teaching sessions for residents and given towards the beginning of the academic year. They would be equally suited for delivery during an academic half-day or as part of an intern orientation. It should be noted that because cognitive sources of medical errors are discussed in this phase and such discussion can easily (but inappropriately) devolve into blame laying, recriminations, or personal accusations, the blame-free, nonjudgmental nature of these presentations should be emphasized to and by senior trainees presenting Phase I.

Phase II

Phase II of the curriculum is monthlong rotation for PGY 2 residents with a more mature understanding of the potential sources of risk in health care and their personal role in errors and error mitigation. As written, this phase requires 5 half-days per week for learners to accommodate several local factors explained below. However, this phase could reasonably be adjusted to any schedule that provides roughly 60–80 hours of learning time (including attending hospital quality and safety-related committees, taking the online Institute for Healthcare Improvement [IHI] courses,[15] reading quality texts, and/or providing answers for practice cases). In general, Phase II requires approximately 4 hours of instructor time per month per supervised trainee. This time is divided between helping learners identify which selectable items appeal to them, selecting texts to be read for the reflective writing activity, distributing and reviewing practice cases and learner responses, and regularly checking in to verify progress in the IHI courses.

Phase II uses four separate modalities of learning to provide a broad exposure to core principles and practices of patient safety and quality improvement. Each modality is optional, and the rotation can be structured to meet individual learner interests as well as local resources. Instructors should meet with learners at the beginning of the rotation to review the rotation outline (Appendix E), which should be distributed to the learners. During the rotation, learners are given the opportunity to participate in the IHI's Open School Safety 101 and Quality 101 courses as a foundation for the rest of the rotation.[15] These self-paced learning modules provide a more in-depth explanation of some of the core concepts of quality and safety.

As part of the initial meeting, a hospital-specific schedule of local quality and safety teams may be distributed. Learners are encouraged to attend hospital committees that review provider-specific quality-of-care measures, lean-based process improvement projects, and patient safety event reports, as well as code blue, rapid response, and other patient safety events. This provides learners with the opportunity to see techniques and practices put into action that they may have read about or heard discussed in a lecture. It further has the advantage of allowing residents who encounter hospital process issues on a daily basis to see the institutional response to these concerns—a process that is often relatively occult to these individuals. This can factor significantly into the ACGME's CLER process. Note that our hospital peer review committee was excluded from this group because of local legislation that would result in loss of privileged status if trainees were permitted to attend the meeting. However, this could be an appropriate additional meeting for learners to attend, depending on the local legal environment.

Also in the initial Phase II meeting, learners are given the option to select one of a list of texts on physician cognition and bias, decision making and team communication, quality and process improvement, or similar topics (bibliography included in Appendix E). Learners who choose to participate in this activity are asked to integrate material reviewed in the text with their own personal practice of medicine in a reflective writing exercise to be completed by the end of the month. The intent of this exercise is to give learners the opportunity to explore how what they have learned directly affects them personally and professionally and to anticipate how they will incorporate these principles and practices in their developing careers.

Phase II also includes a selection of practice cases that learners may, in consultation with supervisors or course directors, choose to complete. These cases provide learners with the opportunity to apply what they have learned in the rest of the rotation in a risk-free manner. The cases may be distributed all at once during the initial meeting. However, it may be useful to instead distribute one or two cases at the first meeting and the remainder midway through the rotation. This would allow the opportunity to check on learner progress and discuss any questions that might arise.

The initial meeting takes between 30 minutes and 1 hour, depending on whether materials have been previously distributed to learners. If desired, a mid-rotation meeting usually requires no more than 30 minutes. A further 1–2 hours are needed to review reflective writing exercises and practice case answers, discuss learner questions after the IHI courses, and complete evaluation materials. Since much of this rotation is self-directed, the learner time necessary is much greater than the instructor time. Many learners who completed this rotation described spending roughly 12 hours reading, in addition to time spent with hospital committees.

The Phase II rotation outline, bibliography of texts, committee schedule, practice cases with suggested answers, and educational objectives are included as Appendix E. This Appendix also features a previously published rubric used both to demonstrate to learners the goals of the reflective writing exercise and to guide evaluators in scoring these writing exercises.[16]

Phase III

Phase III is targeted towards PGY 3 residents with a near-professional awareness of the pervasive and multifactorial nature of sources of error. This phase asks learners to lead a review of a real-world safety event and provides experiential learning as they employ techniques covered in the previous phases. It has been our practice to identify safety events either from a learner's own recent experience or from the hospital event report database. Learners are asked to identify a particular problem, either in provider knowledge, skills, or behavior or in the systems of care delivery. After developing a preliminary outline of the problem, residents are asked to review the event with the aid of the Learning From Defects tool.[13] This review is discussed with a faculty mentor, with an eye towards importance to the hospital or training program, generalizability to other events, and freedom from blame. This review is generally conducted 2–3 weeks prior to the learner's scheduled presentation of the event review to a larger audience (in our experience, the Department of Pediatrics). This provides sufficient time for learners to develop their own presentation slide set to aid in reviewing before a large audience.

In developing topics or events to be reviewed during Phase III activities, learners can and should draw on information and experience gained in the previous phases. In particular, learners who participate with hospital quality and safety committees in Phase II should be aware of clinical site quality priorities. Events that reflect these priorities may be of particular benefit for review and can help residency programs contribute to high-importance processes.

In general, this phase requires about 12 hours of work on the part of the learner and 1–2 hours of review by instructors. The case is presented in an hour-long regularly scheduled conference. Follow-up after the conference is an important part of the curriculum as presented here but can be assigned to standing groups if learner time is not available. A tip sheet and time line for residents presenting their Phase III session are included as Appendix F.

Worth noting is that learners who complete Phase III successfully then are qualified to present the materials contained in Phase I. If this method is chosen, it would be appropriate to have a more experienced instructor available to whom more challenging questions from Phase I learners could be directed.

Results

In Phase I, all 45 residents in our program were asked to attend each session. To date, each of these lectures has been delivered four times, with 47% of the total 180 participants returning feedback forms. Each PGY 2 resident in our program was scheduled to participate in Phase II, with 42 having completed the rotation to date. Similarly, all PGY 3 residents were asked to participate in Phase III, with 22 sessions and 27 residents completing this part of the curriculum. All evaluation forms are located in Appendix G.

As may be expected, learner evaluation of different phases of the curriculum has varied. Evaluation of the talks given in Phase I has been generally quite positive. Of the 84 feedback surveys received regarding Session 1: Systems Factors, the mean response to “This session contributed meaningfully to my professional development” using a 5-point Likert-type scale with 5 = Strongly Agree was 4.73 (SD = 0.50). Of the 105 feedback surveys received regarding Session 2: Individual Factors, the mean response to “This session contributed meaningfully to my professional development” was 4.59 (SD = 0.55). Narrative feedback regarding the Phase I lecture sessions in response to “What skill did you learn from this that you will apply in the future?” is shown below.

“Being aware of my biases in order to catch myself before making errors that I would be prone to making. Will now know how to report an event/error.”

“Effective ways to discuss patient safety with our group. Not having to feel completely blamed & alone when errors are made.”

“We can't be lazy thinkers, we always have to keep an open mind to reduce cognitive bias.”

“Now I know how to report an event. Encourages me to critically think for each & every patient encounter.”

“Metacognition, working through why I'm thinking what I'm thinking.”

“Stop and think about what I'm doing & don't fall into pattern recognition. Also, trust ‘spidey-sense’ and check on/follow up on nagging suspicions.”

“More effective and open/thorough communication with the team to prevent preventable error.”

“Understanding of multiple processes that lead to errors.”

“By being able to identify system errors/problems, will be better able to prevent errors & poor outcomes.”

“How to think about medical errors using systems-based approach.”

“More effective and open/thorough communication with the team to prevent preventable error.”

Session 2

Specific narrative comments in response to the prompt “How could this session have been improved?” for both Sessions 1 and 2 nearly universally were related to the amount of material presented in a limited period of time.

To date, 42 PGY 2 residents have taken the Phase II rotation. Feedback regarding this phase has been somewhat more mixed. The IHI courses required a significant time commitment, with only half of learners able to complete them during the rotation due to time constraints. The rotation has since been adjusted to make the IHI Open School courses suggested but not required. A preponderance of residents (seven out of nine, 78%) who provided feedback indicated that they believed they had a better understanding of the concepts of patient safety and quality improvement than before the rotation. Narrative comments associated with Phase II by learning mode included the following:

Reflective writing exercise: “Helpful to reflect.”

Quality and safety readings: “It was nice to be able to read outside of medical ‘stuff’ and learn about patient safety using a different form than just lectures.”

Online quality and safety learning modules: “The QI [quality improvement] module was extremely helpful to get a framework of how QI projects are developed and run.”

Phase III was, in general, also well received. The mean response to the prompt “This session has increased my awareness of the potential for error in my own practice” was 4.60 (SD = 0.64). The mean response to the prompt “This session will lead to a change in my own practice” was 4.30 (SD = 0.79). Narrative comments regarding these sessions tended to favor open discussion that was free of blame, representation of a wide range of specialties, and the ability to hear senior providers discuss their thought processes regarding challenging clinical situations. Action items that arose out of Phase III discussions and went on to implementation included changing inpatient MRI scheduling, instituting an “anticoagulated patient” warning box in the electronic medical record, developing an educational module for pediatrics residents on neurosurgical emergencies likely to be encountered by general pediatricians, and changing preprocedure time-out policies.

In regard to the outcomes of the curriculum as a whole, with reference to the American Board of Pediatrics In-Training Exam, in 2016, all residents in our program answered all questions in the domain “recognize the common causes of adverse events in pediatric patients” correctly, and 96% of residents answered questions in the domain “understand and apply the definition of a preventable adverse event” correctly.

Discussion

This large, multiphase curriculum is the culmination of a 7-year process of continual revision and expansion of a coordinated program for resident education in quality improvement and patient safety. It began as a change to the standard M&M conference, which many felt was too punitive and confrontational to be of real benefit to modern learners. This expanded into first a single and then multiple sessions of didactic education on the sources of error and the culture of safety, including the importance of error reporting. The final addition was an entire required rotation devoted to a deeper understanding of factors affecting quality of care and sources of error. The final product is a multimodal, longitudinal program that helps to keep the importance of these two areas of medical practice in the forefront of thinking and decision making throughout the entire residency program.

There have been some challenges to introducing this curriculum. A significant time commitment is required on the part of those overseeing the Phase II rotation. Finding time for didactics that do not directly contribute to resident board scores and pass rates can be challenging in a busy residency program. Hospital quality and safety committees that do not have a history of working closely with residents may have concerns about the presence of the learners.

Given increased time and resources, the development of a more focused instructional tool on the basics of quality and safety to replace the existing online modules would be desirable. Providing learners with an understanding of the basic principles and practices of quality improvement and patient safety, as well as examples of their implementation in real-world settings, is an important step towards learners being able to independently employ these tools in their own careers. Unfortunately, learner feedback would suggest that the available self-directed learning resources still are not ideal.

Further development of a larger library of practice cases would be desirable. This would serve to expand the range of topics covered. In particular, a larger and more subtle range of cognitive errors and a more in-depth exploration of systems errors would be helpful.

Finally, the low number of residents providing feedback (particularly for Phase II) and the lack of specificity of that feedback make detailed analysis of learner impression of the curriculum difficult to determine. Currently, the option of requiring the provision of anonymous feedback about the rotation to the program director for later collation and distribution to curriculum faculty is under discussion.

Despite the above challenges, it is felt that the contents of this curriculum represent a relatively comprehensive survey of the quality and safety landscape for the novice learner. Learners completing the entire curriculum are asked to put what they have learned into practice, and relatively consistently, they have been able to suggest, develop, refine, and help implement meaningful and impactful changes in real-world hospital practice to improve patient safety. As stated previously, interventions implemented by those completing this module include a change to hospital-wide time-out procedures to document patient anticoagulation therapy, changes to hospital direct admission discharge and code blue procedures, and a restructuring of MRI-scheduling procedures to improve availability of scanner time for inpatients.

A. Session 1 Presentation With Narration.pptx

B. Session 1 Presentation Without Narration.pptx

C. Session 2 Presentation With Narration.pptx

D. Session 2 Presentation Without Narration.pptx

E. Phase II Materials.docx

F. Phase III Tip Sheet.docx

G. Evaluation Forms.docx

All appendices are peer reviewed as integral parts of the Original Publication.