Piloting a patient safety and quality improvement co-curriculum.

Background: Despite the push for resident and faculty involvement in patient safety (PS) and quality improvement (QI), there is limited literature describing programs that train them to conduct PS/QI projects. Objective: To determine the effectiveness of a co-learning PS/QI curriculum. Method: The authors implemented a co-learning (residents and faculty together) PS/QI curriculum within our general Internal Medicine program over 1 year. The curriculum consisted of two workshops, between-session guidance, and final presentation. The authors evaluated effectiveness by self-assessment of attitude, knowledge, and behavior change and PS/QI project completion.
Results: Thirty-eight of 32 (95%) resident and 8 faculty member participants attended the workshops and 27 of 40 (67%) completed the evaluation. Participants (87-96%) responded favorably regarding workshop effectiveness. The authors found significant improvement in 78% of items pertaining to PS/QI knowledge/skills, but no difference for attitudinal items. The final project evaluation participants rated project content as relevant to learning needs (75%); training as well-organized (75%); faculty mentorship for the project as supportive (75%); and the overall project as excellent or very good (71%).
Conclusion: The authors successfully demonstrated a framework for co-teaching faculty and residents to conduct PS/QI projects. Participants acquired necessary tools to practice in an ever-evolving clinical setting emphasizing a patient-centered and quality-focused environment.

Introduction

Two of the six key focus areas of the Accreditation Council of Graduate Medical Education (ACGME) Clinical Learning Environment Review program are patient safety (PS) and health care quality [1]. According to the Institute of Medicine (IOM), high-quality care should be safe, patient-centered, timely, efficient, and equitable [2].

The ACGME’s and IOM’s emphases on the importance of PS and quality improvement (QI) drive PS/QI’s incorporation into GME curricula [1]. In 2003, Ogrinc et al. performed a literature review and developed recommendations for teaching practice-based learning and QI. While the authors’ focus was on medical students rather than residents, their recommendations highlighted the importance of instruction on QI, change theory, and the link between quality and cost throughout the student’s training, as well as the need for well-defined and time-limited experiential activities imbedded within didactic instruction [3]. There is a paucity of literature describing curricular implementations to train faculty and residents how to effectively conduct PS/QI projects.

Wong et al. proposed a PS/QI curricular framework applied across multiple specialties in a GME setting that closely aligns with educational and clinical contexts, with a focus on patient-centered outcomes [4]. These authors emphasized three key principles to the curriculum implementation within their program:

Trainees and faculty learned together within the same sessions

Trainees were to work with faculty leaders in their chosen subspecialties; and

QI projects were to be aligned with departmental/institutional priorities and/or ongoing initiatives. Variability exists in the ability to prioritize these three key principles within other programs.

The purpose of this study was to determine the effectiveness of a structured PS/QI curriculum implemented in Carilion Clinic (CC) Internal Medicine (IM) residency program adapted from Wong et al’s multispecialty program described within the literature. We measured program effectiveness by using knowledge improvement and attitude change regarding their knowledge base.

Methods

In response to calls to better prepare physicians to practice in a more quality-focused and patient-centered environment, we developed and pilot-tested a co-learning (residents and faculty together) PS/QI curriculum as part of a longitudinal, quasi-experimental study to examine the curriculum’s impact on PS/QI within the CC IM residency program [1,2]. We examined participants’ reactions to the training and their knowledge and behavior changes. The study utilized a combination of didactic training and hands-on project activity. Participants engaged in hands-on activities such as guided completion of a QI Institutional Review Board (IRB) application and utilization of a DMAIC (an acronym for Define, Measure, Analyze, Improve and Control) improvement cycle-based template tool to outline their process.

The goals of our study were to educate residents and faculty about PS/QI together, to increase the knowledge base of participants, and to develop a systematic approach for starting QI/QA projects in a residency in which neither faculty nor residents have participated in that before.

Ethical approval

The CC IRB determined this study to be exempt from further review.

Participants

Our CC IM residents and faculty were the target participants for this pilot project. We intended outcomes of the project to help us to determine if we would offer the program to other departments throughout our teaching institution.

All 32 level 2 (16) and level 3 (16) IM residents participated. Eight IM physician leaders and eight exemplary senior residents were chosen to colead eight separate project teams. We implemented the curriculum throughout the 2015–2016 academic year and it consisted of two workshops (each 3 h in length), between-session guidance, and a final presentation with abstract and poster presentations.

Our planning group/leadership team consisted of the IM Program Director, one project manager, two faculty development specialists, two residents (one level 2, one level 3), one RN-level QI specialist, and one QI trainer (all authors to this article).

QI co-training curriculum

After searching the literature, we determined that Wong et al.’s [4] curricular model aligned best with our curricular intentions. We mirrored this model as closely as possible. However, differences in our environments made this challenging in some instances. Similar to them, we found value in involving both residents and faculty in the same training sessions (Wong et al.’s first of three key principles). The one exception to this is that we felt that our faculty leaders would also benefit from some basic QI leadership training. Therefore, before any co-training sessions began, two of our authors (DM and SW) conducted an 1-h training session for only the team leaders. The detailed goals for this training can be found within Table 1, but our overarching goal was to introduce leaders to basic tools for conducting and leading PS/QI projects.

Table 1.

Project timeline, tasks, goals and objectives, and evaluation methods.

Project timeline	Tasks	Goals and objectives	Evaluation
October 2014–July 2015	Planning and development of training, evaluation tools and process	Literature review on other GME QI projectsSelect project model
July 2015	Team leader training	Recognize the elements of your role as QI team leaderIdentify a QI projectIdentify your project teamLead and manage an effective team
August–September 2015	Introduction to QI	Introduction to QIDMAIC methodologyMeasurement in QISample size/pragmatic auditProcess tools (process mapping, cause-and-effect diagrams)Linking solutions to theories	Student self-assessed proficiency pretest administered prior to training
October 2015–January 2016	Track progress of resident workgroups
February 2016	Part II. QI training	Apply rapid change methodsConstruct and interpret run chartIdentify and leverage contextual factors for QI success
March–May 2016	Groups complete their QI project
May 2016	Groups present projects to QI panel of expertsAwards and recognition	Provide oral presentations	Core learning assessment – QIKATStudent/faculty self-assessed proficiency posttestStudent/faculty/course leadership satisfaction surveyDetermine time investment

Additionally, given that our residents were all general IM residents, Wong et al.’s second principle (matching subspecialty residents with faculty within their chosen subspecialty) was not applicable to our project. Their third principle focused on aligning projects with departmental and/or institutional quality priorities which we also found essential for implementation. Each of our team’s project ideas were vetted by the program director to ensure that they aligned with these priorities.

Our co-curriculum consisted of two 3-h workshops that were repeated to maximize participant ability to attend and one final project presentation session. The workshops were modeled after Wong’s content and were presented by our QI trainer. Session one provided an overview of the logistics of PS/QI and methods for project implementation. Session two focused on analyzing and reporting data. See Table 2 for the curricular intervention schedule and each session’s learning objectives which were based upon selection of Institute for Healthcare Improvement (IHI) modules as outlined in Table 3.

Table 2.

Curricular intervention schedule and learning objectives.

Month	Intervention
July	Team leader training (faculty team leaders only) learning objectives: Recognize the elements of your role as QI team leader Identify a QI project Identify your project team Lead and manage an effective team
August	Workshop I (residents and faculty)Objectives: List six key domains of healthcare quality Select ideal quality problems as targets for QI Apply basic QI skills, including audit, process tools and improvement methodology Distinguish between measurement for QI as compared to evaluative research Work collaboratively with other healthcare providers to carry out a QI initiative
September–January	TeamworkDetails
February	Workshop II (residents and faculty)Objectives: Apply rapid cycle change methods to a QI project Construct and interpret a run chart Identify and leverage contextual factors for QI success
March–May	Teamwork Data analysis Project completion and write-up
May	Final project presentation details Judges and awards

Table 3.

IHI open school mandatory courses.

Improvement Capability
QI 101: Fundamentals of Improvement
Lesson 1: Errors Can Happen Anywhere – and to Anyone
Lesson 2: Health Care Today
Lesson 3: The Institute of Medicine’s Aims for Improvement
Lesson 4: How to Get from Here to There: Changing Systems
QI 102: The Model for Improvement: Your Engine for Change
Lesson 1: An Overview of the Model for Improvement
Lesson 2: Setting an Aim
Lesson 3: Measuring for Improvement
Lesson 4: Developing Changes
Lesson 5: Testing Changes
QI 103: Measuring for Improvement
Lesson 1: Measurement Fundamentals
Lesson 2: Displaying Data
Lesson 3: Learning from Measures
QI 104: The Life Cycle of a Quality Improvement Project
Lesson 1: The Four Phases of a Quality Improvement Project
QI 106: Mastering PDSA Cycles and Run Charts
Lesson 1: Using a PDSA Template for Tests of Change
Lesson 3: Using a Run Chart Template to Display Data
QI 201: Guide to the IHI Open School Quality Improvement Practicum
Lesson 1: Putting Quality Improvement into Practice
Lesson 2: Starting Your Project
Lesson 3: Looking for Changes? Try Cause and Effect Diagrams
Lesson 4: Spell Improvement with P-D-S-A
Lesson 5: Data: Collect and Display
Lesson 6: Summarizing Your Project
Graduate Medical Education FOR FACULTY CLASS
GME 3: The Faculty Role: Understanding & Modeling Fundamentals of Quality & Safety
GME 4: The Role of Didactic Learning in Quality Improvement
GME 5: A Roadmap for Facilitating Experiential Learning in Quality Improvement
GME 7: Faculty Advisor Guide to the IHI Open School Quality Improvement Practicum

http://www.ihi.org/education/ihiopenschool/Courses/Pages/default.aspx

Prior to the first workshop session, each faculty member chose a PS/QI topic. Together with their resident co-leaders, they selected residents to make up their multilevel teams. Between each session, teams worked on aspects of their projects discussed within the preceding workshop. Our project manager was essential to keeping teams on task. She attended all workshops and team meetings, and requested regular status updates from each team. She also provided assistance with IRB approval and administrative duties.

Evaluation

To assess the effectiveness of the program, we used pre- and post-assessment. At the beginning and end of each workshop, residents and faculty were asked to evaluate the curriculum and self-assess their knowledge and potential for behavior change. Participants were reassessed about attitude and knowledge using the same instrument following the second workshop. The pre-assessment also included three questions related to their previous experience with PS/QI.

To measure behavior change, we collected data on participants’ involvement in PS/QI projects both before and after the curricular intervention.

Additionally, our final assessment form asked participants to indicate something they would commit to changing based on their involvement in the curriculum.

We report descriptive statistics for participants’ prior experience with PS/QI activities, feedback on the didactic sessions, and overall evaluation of the project. For pre- and posttest assessments, we used the independent samples T-test procedure to compare item means. To examine association between prior involvement in PS/QI activities and knowledge and attitudinal measures, we used Kendall’s tau correlation coefficient. Statistical significance for all analyses was set at p = 0.05.

Results

Attendance at the two required didactic sessions was good. Thirty-eight (95%) of the 32 resident and 8 faculty member participants were present for the two workshops. Twenty-seven of 40 participants (67%) completed the evaluation. The two residents who were unable to attend due to scheduling conflicts reviewed session videos. For session one (August 2015), 90% of those in attendance felt the content was relevant to their learning needs; 96% felt the educational objectives were accomplished; 96% felt the speaker was organized and effective; and 94% felt the overall session was either excellent or very good. For session two (February 2016), 87% of those in attendance felt the content was relevant to their learning needs; 94% felt the educational objectives were accomplished; 95% felt the speaker was organized and effective; and 92% felt the overall session was either excellent or very good.

Regarding instruction in PS/QI prior to participating in this project, 14 participants (52%) reported none; 7 (26%) reported prior instruction during residency training; 1 (4%) reported prior instruction during medical school; 4 (14%) reported prior instruction during both residency training and medical school; and 1 (4%) was unsure concerning this question.

Regarding whether they had been involved in or witnessed a medical error or adverse event during their training, 3 participants (11%) reported none; 7 (26%) reported yes, during residency training; 1 (4%) reported yes, during medical school; and 16 (59%) reported yes, during both residency training and medical school.

Regarding whether they had worked on a PS/QI project prior to participating in this project, 24 participants (89%) indicated no prior work; 2 (7%) indicated yes, during residency training; and 1 (4%) indicated yes, during both residency training and medical school.

Our pre- and post-assessment assessed both knowledge of PS/QI concepts and general attitudes toward concepts related to medical error, QI, and PS. The results of these assessments are presented in detail in Table 4. When comparing pre- and posttest results, we found no statistically significant differences on any of the six attitudinal items. However, we found statistically significant differences on seven of the nine items pertaining to conceptual knowledge of PS/QI and related skills.

Table 4.

Pre- and post-activity assessment of knowledge and attitudes regarding QA/QI.

	Pretest mean	Pretest % Strongly agree/agree	Posttest mean	Posttest % Strongly agree/agree	Significance
Attitudinal items
Making errors is inevitable in medicine	4.22	89	4.30	96	0.723
Competent doctors do not make errors resulting in patient harm	2.07	7	2.15	15	0.772
Only physicians can determine the main causes of a medical error	1.63	4	1.67	7	0.883
If there is no harm to a patient, there is no need to report an error	1.52	0	1.63	4	0.587
Learning how to cope with medical errors is essential in medical education	4.37	96	4.18	92	0.325
Learning about concepts in QI or patient safety helps me take better care of my patients	4.15	92	3.96	78	0.258
Knowledge items
Using the DMAIC model as a systematic framework for quality improvement	2.18	7	3.48	56	0.0001
Identifying interventions to improve patient safety or quality of care	3.07	41	3.55	67	0.056
Writing a clear problem statement (AIM)	2.81	26	3.63	66	0.001
Using measurements to improve your skills	3.11	37	3.63	66	0.020
Studying a process	3.18	45	3.78	74	0.010
Proposing and making change in a process system	3.15	37	3.63	70	0.037
Identifying and understanding adverse events	3.44	60	3.52	85	0.015
Identifying and understanding medical errors	3.52	67	3.85	81	0.078
Identifying best practices and comparing these to your local practice	3.22	41	3.63	63	0.035

Rating scale: 5 = strongly agree; 4 = agree; 3 = neutral; 2 = disagree; 1 = strongly disagree.

We examined the association between previous education about and involvement in PS/QI activities, previously witnessing a medical error or adverse event, and the attitudinal and knowledge items previously listed above and found a positive correlation. Table 5 presents significant correlations pertaining to these items.

Table 5.

Selected associations between prior education and experience with QA/QI and knowledge and attitudinal items.

Item type	Item	Post graduate year (PGY)	Prior teaching	Worked on prior QA/QI	Significance
Experience	Prior teaching
Experience	Witnessed error	−.316*			0.05
Experience	Worked on prior QA/QI	−.395**			0.01
Attitude	Only physicians can determine causes of medical error			.321*	0.05
Knowledge	Identifying interventions to improve patient safety/quality of care	.321*			0.05
Knowledge	Using measurements to improve your skills	.273*			0.05
Knowledge	Identifying and understanding medical errors	.304*			0.05
Knowledge	Identifying best practices and comparing these to your local practice	.285*	−.276*		0.01

*Denotes statistical significance at the 0.05 level.

**Denotes statistical significance at the 0.01 level.

Each of the eight teams completed a rapid cycle PS/QI project that aligned with institutional goals. These projects were presented at the end of the year as a poster and brief oral abstract presentation. The eight projects are outlined in Table 6. These projects demonstrated an immense behavior change for participants given majority reported having no previous experience participating in PS/QI projects. Project participants also were asked to complete an evaluation after the final poster/abstract presentation. A total of 24–40 (60%) participants completed this evaluation. Participants were asked to consider the knowledge gained as a result of participating in the project, the lessons they learned, and how they would apply their learning to their future medical careers. Seventy five percent felt that the overall project content was relevant to their learning needs; 71% felt that the educational objectives of the project were accomplished; 75% felt that the training was organized and effective; 75% felt that faculty mentorship for the project was supportive and effective; and 71% felt the overall project itself was either excellent or very good.

Table 6.

Year one project titles and aims.

Project title	Project aim
Impact of Distribution of Blood Glucose Logs upon Blood Glucose Recording Compliance	To assess if education and distribution of blood glucose logs increased compliance of returning completed blood glucose logs to appointments.
‘See Something, Say Something’ – Internal Medicine Resident Quality Event Reporting	To raise the number of events reported by internal medicine residents over a defined intervention period by 100% as compared to the prior year.
Reducing Catheter-Associated Urinary Tract Infections (CAUTI) with the Implementation of a CAUTI Bundle	To reduce Foley catheter utilization which contributes to overall reductions in the rates of CAUTI through a multifaceted approach, highlighted by daily physician to physician communication in combination with electronic alerts and monthly education.
Epic Optimization for the Internal Medicine Resident Rounding	To make pre-rounding faster and more efficient using a unique Internal Medicine Pre-Rounding Tab.
A Quality Improvement Project: Reducing Ready to Move Time	To reduce the average time, it takes the faculty medicine teams to place initial admission orders while maintaining a focus on education using education and awareness.
Morning Report Revitalization	To improve morning report quality for both presenter and audience with the goal of enhancing the teaching environment as part of ongoing resident education.
Blood Transfusion Practices as Quality Improvement: Implementation of an Education Initiative to Reduce Blood Product Utilization among Internal Medicine Residents	To evaluate the change in PRBC transfusion habits following a focused CME intervention among Internal Medicine residents at an academic tertiary referral center.
Make a List: The Impact of Short-Term Goals on Weight Loss	To determine if providing patients with a written summary of their short-term behavioral goals improved weight loss.

PRBC: packed red blood cell; CME: continuing medical education.

Discussion

It is inherently clear that PS/QI skills are important for all healthcare providers to acquire. With this in mind and in response to calls to action from the IOM and ACGME, our leadership team developed and implemented a co-learning curriculum to teach PS/QI based on the example described by Wong et al. Training residents and faculty in shared learning experiences and projects allowed for team members to develop shared understandings for immediate application and focusing projects on departmental and/or institutional priorities helped ensure the meaningfulness of each project.

With the assistance of a dedicated project manager, we were able to produce eight distinct team projects in roughly 10 months from conception to presentation which resulted in resoundingly positive feedback from the final evaluation survey on effectiveness and educational value.

Our findings regarding the lack of change in attitudes toward regarding QA/QI were somewhat surprising. Our interpretation is that our project participants had likely been exposed to more information about these topics than we originally thought, and that most physicians at this point in their training would have at least heard that QA/QI topics are important in medicine prior to this project. We are encouraged by our findings of significant change in most knowledge- and skill-based items, many of which reflect specific skills that are actually needed to carry out QA/QI projects such as using the DMAIC model as a systematic framework for QI, writing a clear problem statement (aim), using measurements to improve your skills, studying a process, proposing and making change in a process system, identifying and understanding adverse events, identifying best practices and comparing these to your local practice. This supports the idea that skills in QA/QI can be effectively taught during residency. Our findings also affirm Wong et al.’s conclusions in support of the co-learning curriculum model in order to foster an environment in which PS/QI methodology can be understood and applied in an academic setting.

Responses indicated a valuable learning experience for faculty and residents involved. Residents who participated in this pilot study at our institution indicated they both possess more knowledge of and are more likely to participate in future QI projects. This is a considerable accomplishment given that, pre-intervention, an overwhelming majority (89%) of respondents indicated that they had no prior QI project experience.

The findings of our pilot program for teaching support applying this model on a larger scale, both within our program and potentially in other teaching programs at our institution and elsewhere.

Conclusions

Through the adaptation of a previously discussed co-learning curriculum, we were able to demonstrate a successful framework for teaching faculty and residents how to become proficient in PS/QI projects. In doing so, we equipped them with necessary tools to practice in an ever-evolving clinical setting that places importance on a patient-centered and quality-focused environment.