Creating Change: An Experiential Quality Improvement and Patient Safety Curriculum for Medical Students.

Introduction: Medical students are the future drivers of change in health care. The AAMC encourages quality improvement and patient safety (QI/PS) education. Unfortunately, many schools do not have a formal QI/PS curriculum. To offer the patient-centered, safe, evidence-based, and high-value care patients deserve, students will be expected to have both knowledge of and experience in QI/PS. This extracurricular experiential QI/PS curriculum is designed to prepare medical students for this role.
Methods: The curriculum includes six monthly didactic and work-group sessions that cover QI/PS fundamentals and facilitate the design and implementation of student projects.
Results: Twenty-two medical students, with representation from academic years 1-4, completed the curriculum. The average Quality Improvement Knowledge Application Tool-Revised score increased from 5.61 to 7.75 (p < .01). Six projects were undertaken, with teams completing an average of 2.83 plan-do-study-act cycles. Projects decreased Clostridium difficile ordering, reduced discordance between documented and true intraoperative wound classification, and increased the quantity and quality of patient sleep. Responding "Agree" or "Strongly Agree," 80.9% of students felt their practice would change due to this experience, and 96.5% planned on participating in QI/PS in the future. Four students volunteered to continue as student leaders. Many students (96.5%) felt their experience was good or very good. Discussion: This ready-to-implement curriculum offers medical students an opportunity to obtain the knowledge and experience necessary to participate meaningfully in QI/PS now and throughout their careers.

Educational Objectives

By the end of the curriculum, students will be able to:

Demonstrate understanding of quality improvement and patient safety (QI/PS) fundamentals.

Apply QI/PS skills through the design, implementation, and evaluation of a multidisciplinary QI/PS project.

Advocate for a culture of safe, high-quality, high-value, patient-centered care through QI/PS.

Introduction

To affect positive change in health care, medical students need to develop a strong foundation in quality improvement and patient safety (QI/PS) during their undergraduate medical training. The AAMC encourages medical student QI/PS education.[1] Some undergraduate medical institutions have developed innovative QI/PS curricula.[2,3] Others, including our own, have faced challenges, including unclear best practices, limited mentorship capacity, insufficient curricular time, and an inability to foster learning opportunities that bridge education and clinical practice.

Prior to this intervention, exposure to QI/PS at the University of New Mexico (UNM) School of Medicine, over the course of 4 years, was limited to six Institute for Healthcare Improvement (IHI) Open School modules and 2 lecture hours.[4] Of curriculum participants, 85.7% (18 out of 21) responded “Strongly Disagree” or “Disagree” to the statement “My exposure to QI/PS during medical school is sufficient.” Design and implementation of this QI/PS curriculum were undertaken to address this deficiency. The curriculum employs an educational strategy grounded in known teaching and QI/PS best practices, including monthly didactic and work-group sessions preceded by independent study, immediate application of knowledge to the clinical setting through experiential learning, a multidisciplinary team setting, and consistent mentorship.[2,3,5-8]

A number of curricula focused on providing QI/PS education to trainees have been published by MedEdPORTAL.[9-12] Our curriculum is unique because students were eligible to receive elective or research credit upon successful completion and teams were responsible for the entire plan-do-study-act (PDSA) cycle, including problem identification, intervention design and implementation, evaluation, and analysis. Projects were implemented in the inpatient setting. Technology was utilized to facilitate data collection within the scope of student influence. In addition to measuring knowledge and attitudes, system measures and patient outcomes were used to evaluate curriculum success. Curriculum design and implementation were led by a resident physician, and no funding was required. To the best of our knowledge, there are no comprehensive QI/PS curricula for medical students published and immediately available for use outside of MedEdPORTAL.

This resource is intended to be used as a ready-to-implement QI/PS curriculum. It is adaptable for all learners and educators. No student prerequisite knowledge is necessary. It can be delivered by any educator interested in QI/PS education. This publication includes a syllabus; a schedule; recruitment materials; lecture presentations; session handouts; work-group session driving questions; a monthly team meeting guide; instructions for a project proposal, literature review, and project presentation; and both formative and summative surveys.

Methods

Recruitment and Project Identification

Medical students from academic years 1–4 were recruited via email (Appendix A) as well as a presentation at the institution's annual research day. Student participation was voluntary. Resident, attending, and nurse QI/PS leaders were recruited to serve as mentors through email (Appendix B) and face-to-face meetings.

A student information meeting was held to discuss curriculum goals, requirements, curriculum components, and scheduling (Appendix C). To successfully complete the curriculum, students were required to submit the pre- and postintervention knowledge assessment, attend at least five of the six didactic and work-group sessions, and attend at least four of the six independent team meetings. To be eligible to receive research or elective credit, students were required to complete the above as well as each of the requirements for the institution's mentored scholarly project. No learning contract was required.

Potential QI/PS projects were identified by participating residents, attendings, and nurses. Students were queried for additional project ideas, then were self-assigned to a project based on personal and professional interests. Each project team included students from two or more academic years. This helped to sustain the project throughput despite anticipated absences (e.g., holidays, vacations, USMLE Steps 1 and 2, difficult clinical rotations, etc.). Each team had at least one resident, physician, and nurse mentor. No student or mentor prerequisite knowledge was required.

Curriculum Overview

Six in-person didactic and work-group sessions were scheduled Appendix D. Sessions were held once per month. Scheduling conflicts were minimized by identifying institutional testing and vacation dates. Every student received a syllabus Appendix E and a copy of Fundamentals of Health Care Improvement.[13] Each session was approximately 2 hours long. The format of each session was similar, consisting of a 30- to 90-minute interactive lecture (Appendices F–K) supplemented by handouts (Appendices L–Q), followed by a 30-minute work-group session that provided an opportunity for teams to focus on their QI/PS projects, led by driving questions (Appendices R–W). At the end of each session, students were surveyed for feedback (Appendix X). Each month, teams met independently to facilitate project throughput. These meetings were aided by monthly team meeting guides (Appendix Y).

Session Details

Before Session 1, students completed institutional review board training. In preparation, students were encouraged to read chapters 1, 2, 4, and 6 from Fundamentals of Health Care Improvement.[13] At the beginning of Session 1, students completed the preintervention Quality Improvement Knowledge Application Tool–Revised (QIKAT-R; Appendix Z).[14,15]

The Session 1 lecture Appendix F, supplemented by the Session 1 handout Appendix L, included a course introduction, a brief historical overview of QI/PS, and a discussion about the significance of QI/PS to health care. Then, a combination of lecture and small-group activity was used to help students write an aim statement, perform a miniature root cause analysis, and identify process and outcome measures. The work-group session was guided by the Session 1 driving questions Appendix R. The project proposal (Appendix AA) and literature review (Appendix AB) assignments were discussed (due Session 2).

After Session 1, teams initiated prospective baseline data collection and met independently. Before Session 2, students were encouraged to revisit chapters 1, 2, 4 and 6 from Fundamentals of Health Care Improvement.[13] Weekly email communication was used to help teams achieve project milestones.

To improve data availability, teams were responsible for prospective surveying of patients, nurses, and physicians. This often involved anonymous, voluntary surveys. REDCap was used to improve ease of survey administration, increase data security, and decrease data-handling errors.[16] iPads were provided from the technology center and stored securely on campus for student use. Scripts were provided to ensure that surveys were being collected voluntarily and uniformly. Requests for retrospective data were made to the institution when necessary.

Starting at Session 2, two participants were recognized (receiving a $5 coffee gift certificate) every month for their exceptional contributions (e.g., identifying a project implementation barrier and a strategy to overcome it, scheduling and leading a stakeholder meeting, etc.). This was done to inspire and highlight best practices.

In addition, starting at Session 2, lectures included a review of curriculum feedback from the prior month. Resulting curricular changes were identified. This was done to encourage constructive feedback, as well as to role model cycles of PDSA.

The Session 2 lecture Appendix G, augmented by the Session 2 handout Appendix M, used small-group activities to explore process mapping, create a run chart, and identify rules to analyze run-chart data. The work-group session was guided by the Session 2 driving questions Appendix S. The project proposal and literature review were collected from each team. The project presentation (Appendix AC), an activity allowing teams to discuss their QI/PS project (including successes and challenges) with their peers and a panel of local QI/PS experts, was assigned (due Session 3).

Between Sessions 2 and 3, teams continued data collection, met, and implemented PDSA Cycle 1, if eligible (4 weeks of baseline data collection required). Before Session 3, students were encouraged to read Fundamentals of Health Care Improvement chapters 5, 6, and 7.[13] Escape Fire: Lessons for the Future of Health Care was also recommended as an optional text.[17]

The Session 3 lecture Appendix H, supplemented by the Session 3 handout Appendix N, focused on the project presentations. Students, mentors, and a panel of QI/PS experts were allotted time for discussion. The work-group session was guided by the Session 3 driving questions Appendix T.

Between this and every subsequent session, teams continued to collect data, meet independently, and work on their PDSA cycles (cycle numbers had begun to differ based on a range of factors, including variability of implementation ease).

The Session 4 lecture Appendix I, supplemented by the Session 4 handout Appendix O, offered an introduction to patient safety, including medical error and adverse events, system analysis, and just culture. It ended with a demonstration of the institution's incident-reporting system and a small-group activity titled “On the Receiving End of Error.” The Session 4 driving questions Appendix U facilitated the work-group session. Before Session 4, students were encouraged to read Fundamentals of Health Care Improvement chapters 3 and 8.[13]

The Session 5 lecture Appendix J, augmented by the Session 5 handout Appendix P, focused on theories of change and Kotter's eight-step process for leading change. The work-group session was guided by the Session 5 driving questions (Appendix V). Before Session 5, students were encouraged to read chapter 9 from Fundamentals of Health Care Improvement.[13]

Session 6 started with students completing the postintervention QIKAT-R, as well as a summative curriculum evaluation (Appendix AD). The lecture Appendix K, supplemented by the Session 6 handout Appendix Q, focused on QI/PS scholarship, including the importance of student authorship. An abstract-writing instructional component was followed by teams writing abstracts for their QI/PS projects. The work-group session was guided by the Session 6 driving questions (Appendix W).

Throughout the curriculum, students were encouraged to complete IHI Open School modules, which are free and open access for trainees.[4] A comprehensive list of references used is included (Appendix AE).

Statistical Analysis

A 95% confidence interval was developed for the average pre- (N = 24) and postintervention (N = 22) QIKAT-R scores using SAS version 9.4. Approval from the UNM Institutional Review Board was obtained.

Resource Requirements

On average, curriculum administration required 8 to 15 hours per week. Educators required between 2 and 6 hours to prepare for each didactic and work-group session. An additional 4 to 6 hours was spent meeting with teams each month. Every month, approximately 3 hours were spent in stakeholder meetings or on data collection. Correspondence required between 100 and 150 email threads per month. REDCap was updated, on average, 3 to 4 times per week. According to student feedback, participation required approximately 2 to 3 hours per week.

Results

Twenty-four medical students choose to participate in Cohort 1 (there are roughly 400 medical students at UNM). Of those, 22 (91.6%) completed the curriculum (Table 1). For those students who did not complete the curriculum, one felt the time commitment was too large, and a second stopped participating after a family emergency precipitated a leave of absence. Prior to starting the curriculum, 20 (83.3%) students reported having never received formal training in QI/PS. Twenty-three (95.8%) students reported having never previously participated in a PDSA cycle.

Table 1.

Curriculum Cohort by Academic Year

	Curriculum
Year	Start	Finish
1	10	10
2	4	3
3	7	7
4	3	2
Total	24	22

The curriculum was led by an internal medicine resident and an attending. Two additional internal medicine residents and attendings, as well as a general surgery resident and an attending, participated. Two nursing unit directors, a nurse practitioner, and a charge nurse participated. Only the curriculum leaders had formal QI/PS education, having attended the Society of Hospital Medicine's Quality and Safety Educators Academy.

Student participation was consistent (Table 2). Residents, on average, were able to attend approximately 50% of sessions. Session feedback was positive (Table 3). The average QIKAT-R score increased from 5.61 to 7.75 (p < .01). Six projects were undertaken, with teams undertaking an average of 2.83 PDSA cycles.

Table 2.

Student Session Attendance and Survey Participation

	No. out of Total (%)
Session	Attendance	Survey
1	24/24 (100)	21/24 (87.5)
2	24/24 (100)	17/24 (70.8)
3	21/24 (87.5)	15/21 (71.4)
4	24/24 (100)	20/24 (83.3)
5	19/22 (86.3)	13/19 (68.4)
6	21/22 (95.5)	15/21 (71.4)

Table 3.

Average Survey Responses by Session

	Ratings by Sessiona
Question	1	2	3	4	5	6
The stated objectives were clearly met.	4.57	4.53	4.53	4.75	4.15	4.30
The material was presented in an organized manner.	4.76	4.65	4.33	4.90	4.15	4.30
The group activities advanced my understanding of the material.	4.38	4.47	4.33	4.75	N/A	4.30
The presentation was engaging.	4.57	4.65	4.40	5.00	4.15	4.30
I feel comfortable moving forward.	4.48	4.65	4.13	4.90	4.15	4.30

Abbreviation: N/A, Not Applicable.

Five-point Likert scale (1 = Strongly Disagree, 2 = Disagree, 3 = Undecided, 4 = Agree, 5 = Strongly Agree).

One project resulted in an increase in the quantity and quality of patient sleep. Another led to a reduction in the discordance between documented and true intraoperative wound classification. A third resulted in a decrease in the number of Clostridium difficile orders. The remaining projects are ongoing and have had a null effect on targeted systems measures and patient outcomes. In 6 months, 2,015 surveys were administered, collecting a total of 38,604 data points.

Curriculum feedback was generally positive. Twenty of 21 (95.2%) students felt that their experience was good or very good. All felt that QI/PS is an important topic and that students should be required to learn about QI/PS. Asked whether they would recommend the curriculum to a friend, 95.2% (20 out of 21) responded “Agree” or “Strongly Agree.” Responding “Agree” or “Strongly Agree,” 81.0% (17 out of 21) felt their clinical practice would change based on this experience, 95.2% (20/21) felt comfortable participating in QI/PS work, and 95.2% (20 out of 21) planned on participating in QI/PS work in the future. Four students volunteered to participate in the next course as student leaders.

When asked to rate the time commitment, 85.7% (18 out of 21) responded “Just Right.” According to students, the five most effective curriculum components (in decreasing order of effectiveness), were writing a project proposal, monthly didactic lectures, monthly work-group sessions, monthly team meetings, and conducting a literature review.

When asked what was best, student responses included the following:

“Identify[ing] a real-life problem and collaborating with staff who were excited to work on a project like this.”

“Network[ing] with the different people who make up a hospital team (students, nurses, doctors, patients).”

“Learning about how difficult it is to change a system … even if the change appears to be minor.”

“This was a nice, broad [overview] of what QI/PS is … and I see much more utility in it than I ever did before.”

When asked what was worst, student responses included the following:

“Lectures were often very repetitive of the IHI modules, instead of supplementing them.”

“Sometimes data collection was overwhelming.”

“Having team members that did not participate much made it very difficult for our project to gain traction.”

“Scheduling of meetings.”

When asked what could be improved, student responses included the following:

“Involve students from other fields.”

“Make every team member accountable for their work to ensure everyone is participating equally.”

“Group size. When our group shrunk to 2 people (second years left for Step 1), we were able to get exponentially more done. Groups of 3 would be ideal.”

“More sessions where we look at our data.”

Discussion

Armstrong, Headrick, Madigosky, and Ogrinc identified the key ingredients of a successful QI/PS curriculum: experiential learning, aligning student efforts with institutional priorities, assessing a range of educational outcomes, and role modeling cycles of change in the educational process.[18] The original iteration of this curriculum employed three of these four elements. First, content facilitated stepwise project design and implementation, creating an environment where student knowledge and skill were reinforced in the clinical setting. Second, the curriculum moved beyond measuring learner attitudes, knowledge, and behavioral change by linking student efforts with improvements in systems and patient care. Third, role modeling cycles of change through curriculum feedback and modification, as well as recognizing strong individual performances, promoted iterative learning as best practice. The fourth ingredient, aligning student efforts with institutional priorities, has since been included to improve the student experience and the likelihood of systems and patient care improvements.

This curriculum can be scaled to fit any cohort size, expanded to include all learner types, and modified to address institutional deficits. Because interprofessional teams are crucial to QI/PS efforts, the curriculum can be further strengthened through the incorporation of nursing and paraprofessional health students. Subsequent cohorts have included nursing students, and plans are underway for future iterations to incorporate physical and occupational therapy, as well as respiratory and pharmacy, students.

Curriculum design and implementation have been marked by a variety of challenges. From engendering institutional support to recruiting faculty QI/PS mentors and building relationships with frontline staff, it has been important to identify shared goals, meet and exceed expectations, continuously demonstrate value, and focus on the quality and safety of patient care. Navigating institutional policies and rules and, at times, redefining processes constitute an ongoing effort that has been eased by administrative sponsorship and mutually beneficial relationships with key stakeholders. Additionally, we have benefited from the early identification of individuals with experience in education and curriculum development. This simplified a variety of early hurdles, including drafting a curriculum proposal, writing educational objectives, and integrating evidence-based teaching strategies. Project implementation, a continuing challenge, has been ameliorated by ensuring that all project elements remain within the scope of student influence (data collection and analysis, tests of change, etc.) and that project and institutional goals are aligned.

As this curriculum continues to grow, the most significant challenge remains the recruitment of resident, attending, and nursing QI/PS mentors. Similar to other institutions, UNM has limited QI/PS expertise, and those individuals acknowledging interest often have competing clinical, educational, and research responsibilities or report uncertainty due to a lack of experience. This challenge, a solution to which is likely key to curriculum sustainability, has begun to be addressed by making nursing students and resident physicians from the institution's hospitalist training track eligible to satisfy their QI/PS project requirement through curriculum participation and by recruiting select medical students who have completed the curriculum and demonstrated interest in QI/PS as student leaders. Primarily supported by the Division of Hospital Medicine at this time, efforts are ongoing to find additional faculty leaders throughout the institution, as well as to define incentives for their participation. The insufficient number of resident, physician, and nursing QI/PS mentors limits the number of students who can participate. Cohort 2 includes 28 medical students, including academic years 2–4 (first-year students were not recruited because the starts of the curriculum and the academic year coincided), three nursing students, four student leaders, 10 residents, eight attendings, and four nurses. Seven projects are ongoing. In the future, we hope to include at least 50 students per cohort.

Additional limitations exist. While an extracurricular setting was ideal for this curriculum pilot, failing to integrate QI/PS education within basic science and clinical skill learning opportunities could limit a student's ability to recognize QI/PS as an essential aspect of clinical practice. Additionally, known and unknown knowledge gaps, inexperience, and institution-specific factors continue to challenge our efforts to facilitate project design and implementation. Moreover, project gains can lapse after curriculum completion if advocates focus their attention on new challenges.

Compared to interventions relying on didactics alone or to remote learning through the IHI Open School, this curriculum creates a unique environment where students recognize the value of their QI/PS education through its application to the clinical setting. The curriculum is not intended to be a comprehensive exposure to QI/PS. Similar to other QI/PS curricula, students move from the novice to advanced beginner developmental level.[18] While the curriculum offers students sufficient knowledge and skill to participate in QI/PS efforts, higher developmental levels will necessitate additional training and/or experience. The traditional UNM School of Medicine curriculum satisfies AAMC learning objective one, with students learning “to critically evaluate the knowledge base supporting good patient care.”[1] The curriculum takes the additional step of achieving learning objectives two and three, pushing students to “[understand] the gap between prevailing practices and best practices, and the steps necessary to close that gap,” and then leading them to “[participate] in closing the gap between prevailing and best practices.”[1]

While the curriculum is suitable for all QI/PS educators independent of training level or past experience, countless opportunities exist for modification, adaptation, and refinement. In the future, teams will be limited to three or four members, reflecting student input regarding strengths and challenges of different group sizes. Based on feedback, the didactics will include additional activities and discussion opportunities, using this time to supplement the readings and highlight key points rather than repeat material. Furthermore, current and prior curriculum QI/PS project examples will be used to place material into context. Current student leaders have expressed an interest in teaching certain QI/PS topics, as well as developing a curriculum guide filled with institutional resources and helpful tips. Both will be employed. Additional readings will be used to augment the educational experience and allow students to explore specific topics of interest.

Regarding Session 1 and 2 organization, students felt that process mapping (currently in Session 2) was more important than root cause analysis (currently in Session 1) at the start of their QI/PS efforts and that root cause analysis (currently in Session 1) would be higher yield after completing a literature review (between Sessions 1 and 2) and just prior to finalizing PDSA Cycle 1 planning (Session 2). Thus, these Session 1 and 2 topics will be switched. Sessions 3 and 4 will also be switched to allow teams additional time to complete their initial PDSA cycles prior to the project presentation.

Curriculum additions will be made to address each of the IHI Open School essential knowledge domains.[19] Specifically, customer/beneficiary knowledge and social context/accountability need to be incorporated. In addition, topics will be added to proactively address questions raised by students. A discussion of effort versus yield will be added to Session 2 to aid teams in selecting interventions for their PDSA cycles. Session 4, the introduction to patient safety, will be made more comprehensive through the addition of a discussion on just culture and normalizing deviance. Session 5 will be enhanced with the addition of a change management case discussion (based on challenges with the implementation of prior curriculum projects).

Because project success was considered an important determinant of student experience, great effort was expended to facilitate project design and implementation. It was noted that projects with strong institutional support (e.g., quantity and quality of sleep, appropriate intraoperative wound classification) were demonstrably more successful than de novo projects. This likely reflects the multidisciplinary nature of QI/PS, as well as the synergistic effect of aligned grassroots and administrative efforts. We feel that future projects will be more successful and instructive if better aligned with institutional priorities. Thus, we plan to involve institutional QI/PS leadership teams in our future planning efforts.

A variety of future goals have been established. In the short term, we are looking to elicit feedback that will continue to refine the curriculum; develop a stable cadre of resident, attending, and nurse QI/PS mentors; nurture collaborations across educational silos to promote multidisciplinary learning; define faculty incentives to promote participation; and engage hospital leaders to align efforts. In the long term, the UNM School of Medicine hopes to grow the curriculum to accommodate all students, and we hope to make exposure to QI/PS more dynamic by incorporating QI/PS fundamentals into the formal curriculum.

A. Student Recruitment Email.docx

B. Physician and Stakeholder Recruitment Email.docx

C. Information Meeting.pptx

D. Schedule.xlsx

E. Syllabus.docx

F. Session 1 Lecture.pptx

G. Session 2 Lecture.pptx

H. Session 3 Lecture.pptx

I. Session 4 Lecture.pptx

J. Session 5 Lecture.pptx

K. Session 6 Lecture.pptx

L. Session 1 Handout.docx

M. Session 2 Handout.docx

N. Session 3 Handout.docx

O. Session 4 Handout.docx

P. Session 5 Handout.docx

Q. Session 6 Handout.docx

R. Session 1 Driving Questions.docx

S. Session 2 Driving Questions.docx

T. Session 3 Driving Questions.docx

U. Session 4 Driving Questions.docx

V. Session 5 Driving Questions.docx

W. Session 6 Driving Questions.docx

X. Session Surveys.docx

Y. Monthly Team Meeting Guide.docx

Z. QIKAT-R Measure.pdf

AA. Project Proposal Instructions.docx

AB. Literature Review Instructions.docx

AC. Project Presentation and Panel Discussion Instructions.docx

AD. Curriculum Survey.docx

AE. References.docx

All appendices are peer reviewed as integral parts of the Original Publication.