Patient Safety and Quality Improvement Education in Otolaryngology Residency: Preliminary Look at a Module-Based Approach.

Since publication of the Institute of Medicine's report To Err Is Human in 1999, patient safety and health care quality have become hot topics in the parlance of modern medical care. The Accreditation Council for Graduate Medical Education now requires integration of these topics into resident education, with evidence of trainee involvement in Patient Safety and Quality Improvement (PSQI) projects. Research in other disciplines indicates that interactive, experiential learning leads to the highest quality PSQI education. Otolaryngology as a field has been slow to adopt these changes into its residency curricula due to competing educational demands and lack of faculty expertise. The author reports preliminary experience with integration of an online module-based curriculum that addresses both of these issues.

The modern Patient Safety/Quality Improvement (PSQI) movement began with release of the Institute of Medicine (IOM) report To Err is Human.[1] Not long thereafter, the Accreditation Council for Graduate Medical Education began requiring PSQI education as part of its competency-based evaluation system, and the Clinical Learning Environment Review site visits now specifically evaluate institutional focus on PSQI.[2,3]

PSQI education in residency programs has been studied extensively in primary care disciplines such as internal and family medicine.[4] In otolaryngology, however, integration of PSQI has been limited; a recent survey of program directors showed that only 23% of respondents reported presence of a dedicated curriculum. Cited barriers to integrating such a curriculum were lack of faculty proficiency in PSQI topics and competing educational demands. Nonetheless, survey respondents acknowledged the importance of PSQI education.[5]

To this end, the Temple University otolaryngology residency program has developed a PSQI curriculum with the objective of integration into the standard educational schedule. By basing it on the Institute for Healthcare Improvement (IHI) Open School Basic Certificate curriculum, the hypothesis is that a formal curriculum can be implemented without prior faculty proficiency in PSQI and without a substantial increase in educational time commitment. Furthermore, it is hypothesized that residents will develop the skills and confidence necessary to carry out PSQI projects using standard methodology.[6]

Methods

Institutional review board exemption was obtained for this study. The IHI Open School’s newly updated Basic Certificate curriculum, consisting of 13 interactive online modules, was used as the basis of PSQI education (Appendix 1).[7] All residents of the Temple otolaryngology residency program were assigned to complete 1 to 2 modules per month, each requiring ~1 hour to complete. Module completion was confirmed by submission of a certificate to the program director. Residents completed anonymous online Likert scale–based surveys to assess each module; surveys were distributed using the Basic plan at www.SurveyMonkey.com.

Appendix 1.

Course List for Institute for Health Care Improvement Open School Basic Certificate Curriculum.[a]

Improvement Capability

QI 101: Introduction to Health Care Improvement

QI 102: How to Improve with the Model for Improvement

QI 103: Testing and Measuring Changes with PDSA (Plan-Do-Study-Act) Cycles

QI 104: Interpreting Data: Run Charts, Control Charts, and Other Measurement Tools

QI 105: Leading Quality Improvement

Patient Safety

PS 101: Introduction to Patient Safety

PS 102: From Error to Harm

PS 103: Human Factors and Safety

PS 104: Teamwork and Communication in a Culture of Safety

PS 105: Responding to Adverse Events

Leadership

L 101: Introduction to Health Care Leadership

Person- and Family-Centered Care

PFC 101: Introduction to Person- and Family-Centered Care

Triple Aim for Populations

TA 101: Introduction to the Triple Aim for Populations

Available at http://www.ihi.org/education/IHIOpenSchool/courses/Pages/default.aspx (Accessed on 10/26/16).

Modules were supplemented in the classroom setting with group discussions reviewing topics covered in each module, lectures by members of health system leadership with PSQI expertise (ie, chief quality officer), and self-directed workshops to develop PSQI projects. Projects were initiated by residents based on observations made during patient care and based on cases presented at Morbidity & Mortality conferences. The number of projects developed was recorded and compared with the number of projects in the previous year, before curriculum integration.

Finally, a validated instrument known as the Quality Improvement Confidence Instrument (QICI), a tool used to assess confidence in one’s ability to develop PSQI projects using standard methodology, was administered to all residents in person prior to initiation and upon completion of the first half of the PSQI curriculum.[8] Before-and-after QICIs were compared across 6 aspects of PSQI project implementation using a single-tailed t test, with P values <.05 considered significant. All calculations were made using Microsoft Excel version 2016 (Microsoft Corporation, Redmond, Washington).

Results

One hundred percent of the residents (n = 11) completed each online module in the assigned timeframe. Postmodule survey completion rates averaged 40% for these first 7 modules, with 1 survey administered per module (range of n = 3-5 participants, equaling individual survey completion rates ranging from 27%-45%).

Likert scale–based results from the surveys are illustrated in . Respondents were asked whether the online modules were “extremely,” “very,” “somewhat,” “not so,” or “not at all” worthwhile or appropriate in terms of the course material, online delivery format, ease of comprehension, and time needed for completion. Ninety-one percent felt that the course material was “somewhat” or “very” worthwhile. Ninety-four percent felt that the online delivery format was “somewhat,” “very,” or “extremely” appropriate, with 59% agreeing that it was “very” or “extremely” appropriate. Ninety-one percent felt that the course content was “somewhat,” “very,” or “extremely” easy to understand, with 70% agreeing that it was “very” or “extremely” easy to understand. Eighty-nine percent felt that the course length was “somewhat,” “very,” or “extremely” appropriate, with 60% agreeing that it was “very” or “extremely” appropriate.

Table 1.

Cumulative Postmodule Online Survey Results Demonstrating Resident Assessment of Online Module Content and Quality.[a]

Survey Questions	Extremely Worthwhile/Appropriate/Clear, %	Very Worthwhile/Appropriate/Clear, %	Somewhat Worthwhile/Appropriate/Clear, %	Not So Worthwhile/Appropriate/Clear, %	Not at All Worthwhile/Appropriate/Clear, %	Total, %
How worthwhile was the course material?	0	51	40	9	0	100
The information in this course was appropriate for online delivery.	15	44	35	6	0	100
Course content was clear and easy to understand.	11	59	21	9	0	100
Course length was appropriate for the topic.	9	51	29	9	2	100

The numbers indicate cumulative Likert scale–based responses for the first 7 surveys, one for each of the first 7 of 13 Open School Basic Certificate modules. Response rates for each module ranged from 27% (n = 3) to 45% (n = 5), with a mean response rate of 40% (n = 4.4).

Following implementation of the PSQI curriculum, the number of projects in the current academic year increased to a total of 4 ongoing projects involving 3 to 4 residents per project, with overall involvement of 100% of residents. This is an increase from zero resident projects during the prior academic year. These projects encompass 2 of the 6 domains of care, as defined by the IOM ( ).[6] One project is examining the role of bedside ultrasound implementation for workup of neck disease, thereby reducing examination wait times and potentially decreasing inpatient length of stay (relevant domain of PSQI: Efficiency). Another project is focusing on reducing medical error on call due to inadequate sign-out by standardizing the transition of care sign-out process (Safety). A third project aims to reduce the number of cases with airway-related complications and confusion for total laryngectomy (TL) patients who present to the emergency department (ED) by educating ED faculty and residents about TL anatomy (Safety). The fourth project is tracking rates of critical care patients who prevent airway stenosis before and after implementing cuff pressure monitoring protocols (Safety).

Table 2.

Patient Safety and Quality Improvement (PSQI) Projects Developed by Residents following Implementation of PSQI Curriculum.

Institute of Medicine Domains of Care	No. of Projects	Patient Safety and Quality Improvement Project Titles
Patient-Centeredness	0	NA
Effectiveness	0	NA
Survival	0	NA
Efficiency	1	Use of Bedside Ultrasound for Inpatients to Improve Efficiency and Reduce Length of Stay in Patients with Neck Disease
Safety	3	Standardizing Resident Sign-out During Transitions of Care to Reduce Medical Error: A Checklist ApproachImproving Management of Airway Emergencies for Total Laryngectomy Patients in the Emergency DepartmentReducing Rates of Tracheal Stenosis by Monitoring Endotracheal and Tracheotomy Tube Cuff Pressures in the ICU
Equity	0	NA

Abbreviations: ICU, intensive care unit; NA, not applicable.

QICI completion rates were 100%. Pre- and postcurriculum implementation QICI results are demonstrated in . Resident confidence in performing each step in a PSQI project was assessed on a scale of 1 (“not at all confident”) to 5 (“very confident”). In each of 6 identified steps, learners demonstrated significant improvement, with the 3 greatest improvements noted in “Choosing a Target” (mean difference [MD], 1.32; 95% confidence interval [CI], 0.67-1.97; P = .002), “Describing an Issue” (MD, 0.86; 95% CI, 0.29-1.43; P = .01), and “Defining the Problem” (MD, 0.85; 95% CI, 0.44-1.26; P = .002). Cumulative response data suggest that confidence in every step has improved significantly after completing the first half of the curriculum.

Table 3.

Average of Quality Improvement Confidence Instrument (QICI) Responses by Residents, before and after Completion of the First Half of the Patient Safety/Quality Improvement (PSQI) Curriculum.[a]

Steps of PSQI Project Development	Mean (SD) QICI Response Before PSQI Curriculum Implementation	Mean (SD) QICI Response After PSQI Curriculum Implementation	Mean Difference (95% Confidence Interval)	P Value
Describing an Issue	3.05 (0.80)	3.91 (0.37)	0.86 (0.29-1.43)	.01
Building a Team	3.07 (0.78)	3.80 (0.70)	0.73 (0.14-1.32)	.02
Defining the Problem	3.13 (0.59)	3.98 (0.39)	0.85 (0.44-1.26)	.002
Choosing a Target	2.27 (0.96)	3.59 (0.47)	1.32 (0.67-1.97)	.002
Testing the Change	3.08 (0.55)	3.90 (0.48)	0.82 (0.41-1.23)	.002
Extending Improvement Efforts	3.01 (0.62)	3.55 (0.44)	0.54 (0.05-1.03)	.03

1 = not at all confident; 2 = a little confident; 3 = neutral; 4 = confident; 5 = very confident. Response rate for this instrument was 100% (n = 11).

Discussion

The IHI Open School modules cover the fundamentals of patient safety, quality improvement, and patient-centered care. IHI curricula have been employed by training programs across various disciplines, including internal medicine, family medicine, and even otolaryngology.[9-11] Methods of incorporation range from PSQI integration into specific rotations to spreading out a curriculum across several years of residency. These programs found reasonable learner acceptance of the modules, quality-based changes in practice, improved PSQI knowledge on objective testing, and an increase in presentations at national and regional conferences.

In the Temple program, the curriculum is implemented for all postgraduate-year levels concurrently as a year-long course. Survey results indicate that over half of residents (51%-70%) found these modules to be “extremely” or “very” worthwhile from content, delivery, and time standpoints ( ). Certainly, there is a risk of bias in these results in that residents may want to submit responses that will please the course/program director. However, the surveys were administered online and intentionally conducted in an anonymous fashion to reduce the risk of such bias. Furthermore, from an anecdotal standpoint, I have observed that residents now frequently include PSQI lexicon when discussing the care of patients, suggesting that the culture of the program may be taking steps in a performance improvement direction.

Furthermore, preliminary data show statistically significant improvement in resident confidence in the various aspects of PSQI project development, based on results of the QICI ( ). Data from all 11 participants demonstrate an improvement in confidence in their ability to perform each of the 6 major steps of developing a PSQI project. For most steps, residents improved their confidence scores by nearly a point. The weakest improvement was noted in the final step of “Extending Improvement Efforts,” with an improvement of only 0.54 point (95% CI, 0.05-1.03; P = .03). This correlates with their lack of experience in this topic, as their group projects have not yet reached the stage of systemwide implementation.

One source of dissatisfaction in this study also serves as a limitation—the low postmodule survey response rate. Due to the anonymous, online nature of the surveys, I was unable to reinforce higher survey completion rates beyond providing verbal and electronic reminders. In further discussion with the residents, it seemed that some experienced technical difficulties in accessing the online surveys; I have since worked to rectify this and hope to see higher response rates with future surveys.

The primary barriers to PSQI education integration are believed to be lack of faculty expertise and the time commitment needed for a dedicated curriculum.[5] The need for faculty expertise as a prerequisite is minimized by use of online modules and recruiting quality experts from across the hospital and health system to participate in the course (Appendix 2). Otolaryngology faculty members may participate in parallel with residents to engage in discussion and project participation; our department faculty were invited to do so, but interest has been limited.

Appendix 2.

List of Lectures Provided by Outside Speakers, with Their Health System/Hospital Designations.[a]

Lecture Title	Speaker Title
Introduction to Quality Improvement	Visiting Professor
Fundamentals of Quality Improvement	Chief Medical Officer, Health System
Introduction to Patient Safety	Director of Performance Improvement
Fundamentals of Patient Safety	Chief Medical Officer, Hospital
Root Cause and Systems Analysis	Director of Risk Management
Communicating with Patients after Adverse Events	Director of Risk Management
Introduction to Quality Cost and Value in Health Care	Chief Quality Officer

Remainder of courses were taught by the author.

The time required for a dedicated PSQI curriculum can be lessened by completing modules outside of the standard didactic schedule and by spreading the 13 courses of the IHI curriculum over the course of several months or years. Time dedicated to individual study does not contribute to the 80-hour workweek limitations. Furthermore, in informal communication with the residents, the time needed to complete each module has in reality ranged from 15 to 30 minutes rather than the hour estimated for time completion as suggested by the IHI.

Overall, this preliminary study suggests that a PSQI curriculum can be integrated into the standard otolaryngology curriculum without establishing prior intradepartmental expertise in quality. Residents can develop their PSQI knowledge base by performing short, well-accepted online modules and use this knowledge to begin development of safety- and quality-related projects. Additional barriers to implementation of such a curriculum may include cost and motivation. Fortunately, IHI courses are available free of charge to academic institutions. Furthermore, upon completion of the 13 introductory modules, participants earn a Basic Certificate in Quality and Safety, which may motivate trainees who desire a career in health care leadership or academic medicine.

Conclusion

The IHI Open School modules are applicable to otolaryngology and are appropriate for PSQI beginners. Time commitment is limited, and they appear to be well accepted by residents. A major advantage of this program is that faculty PSQI expertise is not a prerequisite, thereby facilitating immediate curriculum integration. Long-term studies are needed to determine the impact of the IHI and other curricula on otolaryngology PSQI practices.

Author Contributions

Nausheen Jamal, study design, manuscript drafting, final approval, and accountable for all aspects of work.

Disclosures

Competing interests: None.

Sponsorships: None.

Funding source: None.