An Inpatient Patient Safety Curriculum for Pediatric Residents.

Introduction: Patient safety is recognized as an important part of pediatric resident education. There is a lack of published safety curricula targeting pediatric residents. A local needs assessment showed that while residents felt safety was an important part of their current and future jobs, they did not feel prepared to apply safety principles to their future careers or participate in a root cause analysis (RCA).
Methods: This curriculum was delivered to senior-level pediatric and multiple-board residents during five monthly, hour-long, multidisciplinary sessions. Sessions covered systems-based thinking, terminology, the second victim phenomenon, RCA, and medication errors, while providing feedback on recent event reports filed by residents. Resident knowledge, attitudes, and reporting behavior were evaluated prior to and following the curriculum.
Results: Attendees showed statistically significant improved safety attitudes and preparedness to apply safety to their future endeavors; conversely, there were no significant changes in nonattendees. There were no significant changes in knowledge scores or event reporting. Answers to qualitative questions identified learning about the reporting process, RCAs, and follow-up on filed event reports as valuable parts of the curriculum. Residents desired more time to debrief about safety events. Discussion: The curriculum succeeded in engaging residents in patient safety and making them feel prepared for future practice. Residents showed a dissonance between their intentions to report and their actual reporting behaviors, the reasons for which require further exploration. Residents desired a forum to deal with the emotions involved in errors. This curriculum is easily transferable to other institutions with minor modifications.

Educational Objectives

By the completion of this curriculum, learners will be able to:

Define the terms medical error, near miss, sentinel event, preventable adverse event, and nonpreventable adverse event.

Report preparedness to apply principles of patient safety (such as systems-based thinking and human factors) to their future practice.

Participate in a root cause analysis.

File an adverse event report.

Introduction

Engaging residents in patient safety is widely recognized as an important part of graduate medical education. Residents are frontline providers and are frequently involved in or exposed to patient safety issues.[1,2] Educators hope that by teaching principles of patient safety during the critical period of residency training, residents will carry these skills forward with them for the rest of their careers. The American Board of Pediatrics (ABP) includes patient safety both in its Pediatric Milestones Project[3] and as content on its certification exams,[4,5] while the Accreditation Council for Graduate Medical Education (ACGME) has made resident engagement in patient safety a major outcome of its Clinical Learning Environment Review (CLER) program.[6] The CLER program aims to “encourage clinical sites to improve engagement of resident and fellow physicians in learning to provide safe, high quality patient care”[6] and requires trainees be provided with education about patient safety.

Despite this focus on the importance of training pediatric residents in patient safety, there is a lack of published curricula targeting pediatricians. The Institute for Healthcare Improvement (IHI)[7] and the World Health Organization[8] both have general curricula that cover the basics of patient safety but do not feature any content specific to pediatrics. Similarly, there are several high-quality patient safety curricula published on MedEdPORTAL[9-11] that do not feature cases or topics that are salient to pediatricians. Several curricula that are intended for pediatric residents have been published on MedEdPORTAL since the design and implementation of our project. Although the curricula are all of excellent quality, they have their limitations: One[12] involves more time and resources (including a monthlong safety and quality rotation that requires 60–80 hours of learner time) than would have been feasible at our program. The others[13,14] feature only one or two lectures on patient safety principles, with the rest of their content focusing on more in-depth quality improvement topics such as process improvement, plan-do-study-act cycles, six-sigma, and lean. We feel that our curriculum finds a nice middle ground between these existing curricula in terms of time commitment and depth of patient safety content.

A monthly patient safety conference previously existed at our institution, but it was informal and centered on discussions of recent adverse events or near misses that had occurred in the hospital. The content was not standardized, and sessions did not have set learning objectives. A needs assessment of our senior-level, postgraduate year 2 (PGY-2) or greater residents was carried out in December 2015, to which 45 residents responded (N = 75, response rate = 60%). Our results showed that 95% of respondents agreed that patient safety was an important part of their education, and 94% agreed that safety would be an important part of their future practice. Yet only 49% agreed that they felt prepared to apply patient safety principles to their future practice, and only 13% agreed that they felt comfortable performing a root cause analysis (RCA) to analyze a patient safety event occurring in the hospital.

We aimed to create a standardized patient safety curriculum focused on the inpatient setting for pediatric residents. In so doing, we hoped to provide pediatric residents with the foundational knowledge needed to be able to continue to apply patient safety principles for the rest of their careers.

Methods

Our participants were senior-level (PGY-2 to PGY-5) pediatric, internal medicine-pediatric, and triple-board (adult psychiatry, pediatrics, and pediatric psychiatry) residents (N =75) rotating at our hospital (a large academic pediatric referral center). For evaluation purposes, participants for our pilot curriculum were limited to senior-level residents. This was done because PGY-2 and above trainees had been exposed to the previous curriculum and thus could help us determine if the changes we made were improvements to their existing educational experience. Interns also likely had varying levels of existing patient safety knowledge from their medical school experiences, and we wanted to avoid this confounding our evaluation during the pilot phase of the curriculum. Our curriculum is suitable for all levels of learners, including medical students, interns, residents, fellows, or even faculty, without modification. Learners do not need to have any previous patient safety education or experience.

Setting

The 5-month-long pilot curriculum was presented as part of a monthly hour-long morning report session already reserved for covering patient safety topics. These sessions were part of the existing morning report series at our hospital, and while attendance was encouraged for residents who were on inpatient rotations onsite, it was not mandated.

Curricular Design

This curriculum was designed to feature the topics felt to be most essential to pediatric residents, with content adapted from the IHI.[7] Care was also taken to cover the content tested by the ABP on the General Pediatrics In-Training and Maintenance of Certification exams[4,5] and included as part of the ABP's pediatric milestones[3] and the ACGME's CLER program.[6] Table 1 illustrates each session's topic and learning objectives (mapped to the relevant ABP milestones and certification exam testing content, as well as the CLER Pathways to Excellence).

Table 1.

Outline of Learning Objectives for Each Session of the Patient Safety Curriculum for Pediatric Residents, With Corresponding Content Specifications From the ABPCE, ABPM SBP, and CLER PS

Session Topic/Learning Objective	Content Specifications
Overall curricular learning objectives
1. Define the terms medical error, near miss, sentinel event, preventable adverse event, and nonpreventable adverse event.	ABPCE 36 A
2. Report preparedness to apply principles of patient safety (such as systems-based thinking and human factors) to their future practice.	ABPCE 36 F, ABPM SBP6
3. Participate in a root cause analysis.	ABPCE 36 E2, CLER PS4
4. File an adverse event report.	ABPCE 36 C, CLER PS1
Session 1: Basic Principles of Patient Safety: Systems-Based Thinking, Safety Culture and Just Culture
1. Describe the contribution of adverse events to pediatric morbidity, mortality, and cost of care.	ABPCE 36 B1-2
2. Describe the characteristics of a culture of safety.	ABPCE 36 F1a, ABPM SBP6, CLER PS3
3. Describe the “Swiss cheese” model of errors.	ABPCE 36 E1, F1b, F1d; ABPM SBP6
4. Differentiate between unsafe systems and unsafe behaviors.	ABPCE 36 E1, ABPM SBP6
Session 2: Terminology and Types of Events
1. Define the different types of safety events tested by the ABPCE: medical error, near-miss event, sentinel event, preventable adverse event, nonpreventable adverse event.	ABPCE 36 A1-5
2. Give examples of each of the types of events listed in objective 1.	ABPCE 36 A1-5, 36 E1; ABPM SBP6
Session 3: The Reporting Process and the Second Victim
1. File an event report.	ABPCE 36 C4, ABPM SBP6, CLER PS1, PS2
2. Describe the process that occurs when an event report is filed at our hospital.	ABPCE 36 C1-2; ABPM SBP6; CLER PS1, PS2, PS4
3. Define the second victim phenomenon.	ABPCE 36 D3, CLER PS3
4. Describe techniques to help gain wisdom when an error or poor outcome occurs to one of our patients.	ABPCE 36 D3, CLER PS3
5. Describe resources for second victims at our institution.	ABPCE 36 D3; CLER PS2, PS3
Session 4: Root Cause Analysis
1. Describe the steps in carrying out a root cause analysis.	ABPCE 36 E2; ABPM SBP6; CLER PS2, PS4
2. Carry out the steps of a root cause analysis on a sample case.	ABPCE 36 E2; ABPM SBP6; CLER PS2, PS4
Session 5: Medication Safety
1. Describe the four steps in using medications.	ABPCE E7
2. Describe the medication usage process at our hospital.	ABPCE E8, CLER PS2
3. Identify evidence-based practices that can help improve the safety of medication usage in the hospital.	ABPCE 36 E3, E7-E9

Abbreviations: ABPCE, American Board of Pediatrics General Pediatrics Certification Exam Content Outline; ABPM SBP, American Board of Pediatrics Milestones Project Systems-Based Practice domain of competency; CLER PS, Accreditation Council for Graduate Medical Education's Clinical Learning Environment Review Patient Safety Pathway to Excellence.

The ABPCE objectives listed are based on the content outline for the test administered prior to August 31, 2017, which was when this curriculum was developed. The most recently updated ABPCE includes patient safety as one of the major content domains (Domain 25: Research Methods, Patient Safety, and Quality Improvement) but does not list more-specific subdomains, as in the previous version.

Each hour-long session was designed to include a maximum of 20–30 minutes of background didactic material, with the remaining time set aside for interactive group activities, which varied based on the session's topics and learning objectives. The curriculum was delivered by a chief resident for quality and safety (Catherine Polak), a medical education fellow (John Szymusiak), and a junior faculty member (Michael D. Fox) with an interest in patient safety and quality. The sessions were intended to be multidisciplinary, with faculty physician facilitators who had expertise in medical education attending sessions along with hospital-level safety officers, administrators, and other members of the multidisciplinary team, such as pharmacists and nursing. The goal of making the sessions multidisciplinary was to familiarize residents with the safety administrators and safety officers who would be following up with them on their reports. We also hoped to reinforce that patient safety falls upon all members of the multidisciplinary team and that to truly understand and improve our care systems we needed input from all team members. In addition, CLER Patient Safety Pathway 2 notes the importance of educating trainees about patient safety in a multidisciplinary setting.[6] While the sessions could be delivered by a single presenter without other facilitators or multidisciplinary team members, we feel that some of the richness of the sessions, with the diversity of viewpoints from different team representatives, would be lost. Facilitators do not need to have extensive patient safety knowledge or experience, although discussions will likely be more robust if facilitators have some background and practical experience with the topic.

The final 5 minutes of each session were dedicated to providing residents with feedback on event reports. A lack of feedback about event reporting is one of the most frequently cited barriers to reporting by residents.[1,2,15-17] Prior to each session, we reviewed a report from the patient safety officer detailing all of that month's filed event reports, looking for reports from residents that resulted in a significant investigation and led to a tangible change. At the conclusion of each session, the facilitator reviewed the circumstances of the chosen report, summarized the investigation process, and highlighted relevant findings, as well as any changes that came about as a result of the report. The patient safety officers also attended the sessions, frequently commenting on the investigation and changes that resulted and answering resident questions. Providing this feedback reinforced the importance and efficacy of reporting and allowed the hospital to spotlight changes to residents' day-to-day workflow that occurred because of resident reports. Providing feedback to trainees is also a part of the CLER Patient Safety Pathway 4.[6]

Following each session, a one-page summary of key points was sent out via email to all upper-level residents to reinforce key points and to allow any residents who could not attend a given session to be exposed to the content.

Curricular Implementation

The curriculum requires a space with room for a medium-to-large group (our sessions consisted of 10–20 trainees as well as five to six faculty members/multidisciplinary team members), a computer with Microsoft PowerPoint, and a projector and screen. To encourage discussion, it is preferable if the seating can be arranged in an open layout so that all participants can see each other. The PowerPoint files included here (Appendices A, C, E, G, & L) contain all of the content required for each session, with the exception of Session 4, which also requires printed handouts (Appendices H–J). All PowerPoints should be viewed in presenter mode with presenter notes visible as there are important facilitator prompts and instructions included in the notes. These notes serve to guide the facilitator through each session, highlighting verbal teaching points and recommended times to engage the audience or pause for a small-group activity or discussion. The notes for Session 4 also highlight when to distribute the handouts and how to instruct the learners in their use. Finally, the notes point out whether there are other recommended members of the multidisciplinary team to invite to the session. While we feel that all sessions benefit from having multidisciplinary representation, it is more crucial for some sessions than others.

Some portions of the sessions need to be customized to reflect local reporting practices, so the slides should be reviewed and updated by the facilitator prior to facilitating the actual session. Instances where customization is needed are pointed out in the PowerPoint presenter notes and by red, italicized text on the slides themselves. We also recommend that facilitators find local event reports that led to a system-level change to highlight in the final 5 minutes of each session. The slides detail the way we structured these event-reporting updates.

Table 2 summarizes the flow of each session, as well as the suggested amount of time to allot for each portion.

Table 2.

Outline of Each Session of the Patient Safety Curriculum for Pediatric Residents, With Suggested Time for Each Portion

Session	Suggested Time
1: Basic Principles of Patient Safety: Systems-Based Thinking, Safety Culture and Just Culture (Appendix A)
Didactic material (Slides 1–17)	25 minutes
Group facilitated discussion of examples of types of behaviors and how to respond to them (Slide 18)	25 minutes
Summary of Session 1 (Slides 19–21)	5 minutes
Event-reporting update (Slides 22–25)	5 minutes
2: Terminology and Types of Events (Appendix C)
Session 1 review (Slides 1–5)	5 minutes
Didactic material (Slides 6–22)	20 minutes
Case discussions (Slides 23–28)	25 minutes
Summary of Session 2 (Slides 29–31)	5 minutes
Event-reporting update (Slides 32–35)	5 minutes
3: The Reporting Process and the Second Victim Appendix E
Warm-up note card exercise (Slide 1)	Completed upon entering, prior to session start
Session 2 review (Slides 2–5)	5 minutes
Reporting process didactic and safety officer Q&A session (Slides 6–18)	20 minutes
Share second victim note cards (Slide 19)	5 minutes
Second victim didactic (Slides 20–32)	22 minutes
Summary of Session 3 (Slides 33–37)	3 minutes
Event-reporting update (Slides 38–41)	5 minutes
4: Root Cause Analysis Appendix G
Session 3 review (Slides 1–3)	3 minutes
Didactic material (Slides 4–14)	12 minutes
Workshop: introduction (Slides 15–20)	10 minutes
Workshop: small-group discussions (Slide 21)	15 minutes
Workshop: whole-group share and debrief (Slide 21)	12 minutes
Summary of Session 4 (Slides 22–23)	3 minutes
Event-reporting update (Slides 24–27)	5 minutes
5: Medication Safety Appendix L
Session 4 review (Slides 1–4)	5 minutes
Didactic material (Slides 5–18)	15 minutes
Group discussion with pharmacists (Slide 19)	23 minutes
Evidence-based strategies (Slide 20–23)	10 minutes
Summary (Slide 24)	2 minutes
Event-reporting update (Slides 25–28)	5 minutes

The summary documents (Appendices B, D, F, K, & M) should be sent out via email to all learners within a few days of delivery of the sessions. Several of the documents require updates to reflect local reporting practices—these areas are highlighted in red, italicized font.

For those who wish to assess learner attitudes about safety and usage of an event-reporting system before and after the curriculum, the precurricular evaluation Appendix N should be distributed to all learners (either via email or as a paper handout) prior to the first session. The postcurricular evaluation Appendix O can be distributed following the final session. More details about these evaluations are provided below.

While the sessions were designed to be delivered in sequence, each could easily be used as a stand-alone session with minor modifications by the facilitators (i.e., defining the difference between an adverse event and a near miss if the learners have not done the session on terminology).

Curricular Evaluation

Our evaluation was deemed exempt by the University of Pittsburgh Institutional Review Board.

Residents were administered a precurricular evaluation Appendix N prior to initiation of the curriculum, as well as a nearly identical postcurricular evaluation Appendix O the month after completion of the curriculum. The precurricular evaluation was carried out in December 2015, the curriculum was delivered from January to May 2016, and the postcurricular evaluation was carried out in June 2016. Surveys were administered via email using the Redcap system, and participation was incentivized by entry into a raffle for one of two prepaid Visa cards for each survey. Residents were assigned a deidentified study ID number to enable matching of individual pre- and postevaluations as well as to track attendance at the sessions.

The curricular evaluation began with several demographic questions as well as questions assessing the residents' self-reported use of the hospital's electronic event-reporting system and their attendance at safety rounds over the past 6 months. The residents' attitudes regarding patient safety, their preparedness to apply safety to their future practice, and their satisfaction with their safety education were assessed with nine questions using a 5-point Likert scale (1 = strongly disagree, 5 = strongly agree). The residents were also asked to respond to several qualitative questions asking what they liked about the curriculum, what could be improved, what was missing, and (on the postevaluation only) what changes they would make to their practice based on what they had learned from the curriculum.

The final portion of the evaluation was a knowledge assessment. Unfortunately, there is a lack of validated patient safety knowledge assessments in the literature; at the time of the curriculum's development, we could find only one such published assessment,[18] but it had not been validated in pediatricians and tested content that was not very relevant to our curriculum. As such, we developed our own assessment, consisting mainly of questions from the IHI's patient safety curriculum,[7] as well as several from Kerfoot, Conlin, Travison, and McMahon's assessment.[18] The questions chosen were mapped to the topics of our sessions and judged to have face validity by local content experts; however, further validation was beyond the scope of this study. The final knowledge assessment consisted of 20 multiple-choice questions. The knowledge assessment questions are not included in the pre- and postevaluations (Appendices N & O) here as they are copyrighted.

Statistical Analysis

Matched pre/post data were analyzed using the Wilcoxon matched-pairs signed rank test to look for changes from the pre- to postevaluation. Fisher's exact tests were used to look for differences in demographic information and to see if there were significant differences between the amount of change in pre- to postscores for attendees as compared to nonattendees. Statistical analyses were carried out using StataSE version 14.1 (StataCorp; College Station, Texas).

Qualitative data were reviewed by John Szymusiak, who analyzed responses to each question for common themes and tracked which themes were most frequently expressed.

Results

Of 75 eligible residents, 45 residents completed the precurricular evaluation (60%), and 43 completed the postcurricular evaluation (57%). Twenty-six residents completed both evaluations (35%), and analysis was limited to these residents. Fifteen of these residents (58%) had attended at least one of the sessions, whereas 11 (42%) had not attended any sessions (Figure). Attendance was assessed by sign-in at the sessions using the residents' study IDs, as well as by self-report, since not all residents signed into every session.

Figure.

Residents included in the final analysis of the patient safety curriculum for pediatric residents.

In general, attendance at the sessions was a challenge. Attendance ranged from seven to 20 residents at each session, with an average of 13.8 residents per session. Among attendees, the average attendance was 1.4 sessions, with a maximum of three sessions attended (two residents). The majority (87%) of attendees went to one or two sessions.

Demographic Information

The demographic information for the 26 residents who completed both evaluations is provided in Table 3. The only statistically significant (p < .05) difference between the groups was that more attendees were categorical pediatric residents, which is to be expected since these residents were rotating on pediatrics for the entire 5-month curriculum, whereas the multiple-board residents spent time off-site during the pilot period.

Table 3.

Demographic Characteristics of Those Completing Pre- and Postsurveys for the Patient Safety Curriculum for Pediatric Residents

Characteristic	Attended (N = 15)	Never Attended (N = 11)	pa
PGY level			.551
PGY-2	9 (60%)	6 (55%)
PGY-3	5 (33%)	2 (18%)
PGY-4	1 (7%)	2 (18%)
PGY-5	0 (0%)	1 (9%)
Program			.001
Categorical pediatric	13 (87%)	2 (18%)
Internal medicine	2 (13%)	6 (55%)
Triple board	0 (0%)	3 (27%)
Career plan			.205
Primary care	2 (13%)	0 (0%)
Subspecialty fellows	8 (53%)	5 (45%)
Hospitalist	1 (7%)	0 (0%)
Unsure	4 (27%)	3 (27%)
Other	0 (0%)	3 (27%)

Abbreviation: PGY, postgraduate year.

Fisher's exact test.

Quantitative Results

The results are summarized in Table 4. Attendees showed statistically significant improvements from pre- to postevaluation in their attitudes and preparedness to apply safety to their careers (p < .05). Nonattendees did not show any statistically significant changes from pre- to postevaluation, and neither group showed improvements in knowledge scores or self-reported reporting behaviors. A significantly higher percentage of attendees showed an improvement, pre- to postevaluation, in their knowledge of how to file event reports (p = .02) and in their comfort participating in RCAs (p = .02) compared to the percentage of nonattendees who showed improvement in these realms over the same time frame.

Table 4.

Change in Pediatric Residents' Attitudes and Knowledge Scores After the Curricular Intervention on Patient Safety

	Attended (N = 15)			Never Attended (N = 11)
	M ± SD		pb	M ± SD		pb
Survey Question	Pre	Post		Pre	Post		pc
Safety is important for resident education.	4.7 ± 0.5	4.6 ± 0.5	.65	4.5 ± 0.5	4.5 ± 0.5	>.99	>.99
Reporting is part of residents' duties.	4.6 ± 0.5	4.7 ± 0.5	.32	4.4 ± 0.5	4.4 ± 0.7	>.99	.61
I know how to file an event report.	4.6 ± 0.5	4.9 ± 0.4	.16	4.4 ± 0.9	4.1 ± 0.9	.16	.02d
I feel comfortable using a root cause analysis.	2.5 ± 0.7	3.5 ± 0.8	.004d	3.1 ± 0.9	3.2 ± 0.9	.56	.02d
Safety will be important for future practice.	4.5 ± 0.5	4.5 ± 0.6	.65	4.3 ± 0.6	4.3 ± 0.9	>.99	.61
I feel prepared to apply safety to future practice.	3.4 ± 0.8	3.9 ± 0.5	.03d	3.8 ± 0.9	3.7 ± 0.8	.56	.08
Safety rounds add to my understanding.	3.9 ± 0.7	4.3 ± 0.6	.03d	3.3 ± 1.3	3.1 ± 1.2	.41	.22
I enjoy safety rounds.	3.4 ± 0.9	4.1 ± 0.9	.003d	3.4 ± 1.4	3.2 ± 1.2	.32	.01d
My overall satisfaction with safety rounds	3.5 ± 0.5	4.3 ± 0.6	.002d	3.4 ± 1.1	3.5 ± 0.8	.56	.02d
Knowledge score	11.6 ± 1.9	11.8 ± 2.3	.817	13.7 ± 1.8	12.2 ± 1.7	.02d	.18

Scoring of questions about attitudes is based on a 5-point Likert scale (1 = strongly disagree, 5 = strongly agree); knowledge scores are on a scale of 1–20.

Pre vs. post; Wilcoxon matched-pairs signed rank test.

Percent increase in attended vs. not attended; Fisher's exact test.

Statistically significant at α = .05.

Qualitative Results

Residents listed the following as strengths of the curriculum: its focused learning objectives, the multidisciplinary nature of the sessions, the opportunity to gain insight into the reporting process and RCAs, the chance to receive follow-up on filed reports, and the summary emails. They suggested the curriculum could be improved by decreasing the amount of didactic material, including more real cases, and incorporating some aspects of the previous “unstructured format.” One resident wrote, “I do miss the more informal ‘what issues are you guys seeing on the floors right now’ aspect of senior safety rounds”; another expressed similar sentiments, stating that the “old structure is sometimes helpful to identify hospital safety issues.” Another resident missed having a dedicated venue to debrief on safety events in the hospital and said that “safety rounds can be cathartic if you have something you feel you need to share.”

When asked how safety rounds would change their practice, residents felt they were more likely to (1) file an event report, (2) think on a “system level” or from an “RCA-style” approach when analyzing events, (3) perform accurate medication reconciliations, and (4) explain their reasoning when ignoring pharmacy alerts in the computerized order entry system (as opposed to just selecting “other”).

Discussion

We designed, implemented, and evaluated an inpatient patient safety curriculum for pediatric residents. We found statistically significant improvements in residents' perceived preparedness to apply patient safety to their future endeavors, in their comfort applying RCAs to patient safety events, and in their curricular satisfaction. These changes occurred only in attendees, not in nonattendees, suggesting that they were not due to a maturation effect. Similarly, significantly more attendees showed increases in their knowledge of how to report and in their comfort using an RCA than did nonattendees. The curriculum was well received based on residents' survey responses, their qualitative answers, and the informal feedback received from both residents and participating faculty. We were also asked to present some of the content to another division's trainees.

We gleaned some interesting information from our qualitative data. One of the most frequently listed changes residents said they would make to their practice due to safety rounds was that they would file more event reports; however, we did not see any statistically significant increases from pre- to postevaluation among attendees (or nonattendees) when they were asked how often they had filed a report in the past 5 months. Unfortunately, given the anonymous nature of our evaluations, we could not look at actual reporting data to see if those who attended safety rounds had an increase in reports submitted using our institution's electronic reporting system. It is worth investigating further what residents perceive as barriers to reporting to try to identify what could potentially account for this difference between their intentions and actions.

In the qualitative data, multiple residents mentioned that they missed the old, unstructured format of previous safety rounds. They viewed these less formal sessions as opportunities to debrief events that had occurred in the hospital, as well as an opportunity outside of formal event reporting to bring safety events occurring in the hospital to the attention of administrators and hospital leadership. It is worth investigating if residents at other institutions also share these views and whether they would benefit from these opportunities.

Our study also presented challenges in implementing and evaluating the curriculum, as well as having certain other limitations. First, this was a single-center study with a modest sample size. Second, we faced a challenge in terms of attendance despite our curriculum being delivered during morning report, an hour of protected time (8:00–9:00 a.m.) prior to the start of rounds. Even though attendance at our sessions improved over the course of the 5 months, none of the residents were able to attend all five sessions, and most attended only one or two. A number of factors may explain this issue: Residents prioritize prerounding on their patients and patient care, especially when censuses are high during the winter months (which is when our curriculum was piloted); also, residents are constantly rotating onto different services, and when on outpatient or off-site rotations, they are unlikely to attend morning reports. This problem could be solved by trying other settings for curriculum delivery, such as noon conferences (once morning rounds are completed), multihour workshops with protected coverage so that all residents from a given class can attend, or evening sessions at the end of the workday (possibly with some type of incentive, like dinner, provided). Another option is videotaping the sessions and making them available for residents to view remotely at their convenience, although we feel that there is a lot to be gained from the interactivity of live attendance.

It is possible that there were some other underlying unmeasured differences between attendees and nonattendees in our study—either that nonattendees already had negative attitudes and experiences with patient safety or else that nonattendees had a lot of prior experience and knowledge about patient safety and did not feel that further training on the topic would be helpful. This could potentially affect the validity of our results, although the baseline attitudes on the preevaluation of attendees and nonattendees were quite similar, making this less likely.

Finding a valid knowledge assessment for our learners presented another challenge. We created a knowledge assessment mostly using questions from the IHI patient safety curriculum, and our assessment was felt to have face validity by local content experts; however, more formal validation of the assessment was beyond the scope of our project. Our attendees did not show any statistically significant changes in their knowledge scores. It is possible that our study, given the small sample size, was underpowered to detect a change in knowledge scores. We also feel that poor longitudinal attendance (as described above, most attendees went to only one or two sessions) and an overly challenging assessment likely contributed to the lack of significant improvement. The questions from the IHI's patient safety curriculum were designed for immediate recall (not spaced recall as much as 5 months afterward). Many of the questions also did not necessarily assess the key learning objectives of each session and were focused on less important details. This illustrates the need for the development of more validated patient safety knowledge assessments to assess both future curricula and interventions as well as residents' safety knowledge.

Based on our experience and data from this pilot curriculum, changes have been made to safety rounds. First, we have invited all residents and medical students to safety rounds, so that more learners can benefit from the curriculum. Due to logistical limitations, the sessions remain during morning report, but our hope is that over the course of their training, residents will gain exposure to the entirety of the curriculum since the five core sessions that make it up will be repeated each year. We have also instituted quarterly unscheduled sessions to provide the forum for free discussion and debriefing that residents reported missing. Finally, we hope to incorporate sessions on quality improvement and diagnostic errors, as well as opportunities for residents to carry out RCAs on real cases, into future versions of the curriculum.

We believe that our curriculum can be similarly successful at other institutions in engaging residents in patient safety efforts and increasing their preparedness to apply safety principles to their future practice. The resources provided here do not require faculty expertise in patient safety—a frequently cited barrier to teaching residents about safety[19]—and can be easily adapted to feature the reporting process of a given institution.

A. Safety Rounds Session 1.pptx

B. Session 1 Summary.docx

C. Safety Rounds Session 2.pptx

D. Session 2 Summary.docx

E. Safety Rounds Session 3.pptx

F. Session 3 Summary.docx

G. Safety Rounds Session 4.pptx

H. Session 4 RCA Case Description.docx

I. Session 4 RCA Preceptor Cheat Sheet.docx

J. Session 4 RCA Resident Worksheet.docx

K. Session 4 Summary.docx

L. Safety Rounds Session 5.pptx

M. Session 5 Summary.docx

N. Precurriculum Evaluation.docx

O. Postcurriulum Evaluation.docx

All appendices are peer reviewed as integral parts of the Original Publication.