Introduction of a quality improvement curriculum in the Department of Internal Medicine, Lincoln Medical Center.

Community hospitals with limited resources struggle to engage physicians in Quality improvement initiatives. We introduced Quality Improvement (QI) curriculum for residents in response to ACGME requirements and surveyed the residents understanding of QI and their involvement in QI projects before and after the introduction of the curriculum. The current article describes our experiences with the process, the challenges and possible solutions to have a successful resident led QI initiative in a community hospital.
Methods: A formal QI curriculum was introduced in the Department of Internal Medicine from September to October 2015 using the Model for Improvement from Institute for Health care Improvement (IHI). Learners were expected to read the online modules, discuss in small group sessions and later encouraged to draft their QI projects using the Charter form and PDSA form available on the HI website. Online surveys were conducted a week prior and 3 months after completion of the curriculum
Results: 80% (100/117) of residents completed the pre-curriculum survey and 52% (61/117) completed the survey post curriculum. 96.7% of residents report that physicians should lead QI projects and training rather than the hospital administrators. Residents had 20% increase in understanding and confidence in leading quality improvement projects post curriculum once initiated. Most Residents (72%) feel QI should be taught during residency. Active involvement of residents with interest was seen after the initiation of Open School Institute of health improvement (IHI) curriculum as compared to Institutional led QI's. The resident interventions, pitfalls with change processes with an example of PDSA cycle are discussed.
Conclusion: A Dedicated QI curriculum is necessary to prepare the physicians deliver quality care in an increasing complex health care delivery system. The strength of the curriculum is the ease of understanding the material, easily available to all, and can be easily replicated in a Community Hospital program with limited resources. Participation in QI by residents may promote constructive competitiveness among related hospitals in public system to improve delivery of safe care. Abbreviations: ACGME: Accreditation Council for Graduate Medical Education; IHI: Institute of Healthcare Improvement; PDSA: Plan-Do-Study-Act; PGY: QI: Quality improvement.

Introduction

Quality improvement initiatives strengthen our ability to deliver the best outcomes for patients that are safe, effective, and equitable. Regulatory agencies including the Accreditation Council for Graduate Medical Education (ACGME) and Residency Review Committee (RRC) require that residents be involved in QI projects [1,2]. Clinical environmental learning review (CLER), an accreditation system by Graduate Medical Education is a type of institutional review involving site visits to institutions to review residents’ involvement in quality and patient safety initiatives [3]. Furthermore, employers are seeking physicians who have the knowledge and experience of quality improvement. We choose the ‘Model for Improvement’ by Institute of Healthcare Improvement (IHI) as the QI methodology as part of our curriculum to introduce and educate residents on how to set goals, establish measures, and select changes [4]. One of the unique features of this model is the cyclical nature of impacting and assessing change, most effectively accomplished through small and frequent Plan-Do-Study-Act cycles (PDSA) rather than large and slow adjustments, before changes are made system-wide [5]. We introduced a multifaceted Quality improvement curriculum for residents in response to ACGME requirements and to enhance residents’ skills for value based and ethical care in the clinical setting. We hoped that the curriculum will not only sensitize and teach the residents – most of them being International Medical Graduates – to the concept of quality improvement but also stimulate an interest in initiating and conducting quality improvement projects. The aim of this survey was to assess if mandating the Internal medicine residents complete the IHI curriculum will increase the involvement of the residents in QI projects or help them design and lead projects of their own. The current manuscript illustrates the establishment of resident driven QI projects and QI curriculum under the quality improvement initiatives in the Department of Internal Medicine.

Methods

Our Internal Medicine residency program has over 117 residents including preliminary and categorical residents (with one chief resident). Our hospital serves one of the most underprivileged communities in the south Bronx with many challenges to delivery of care. The QI didactic curriculum was implemented in September 2015, using the IHI open school relating to Quality Improvement. All residents and faculty were given access to the IHI Open School [3,4]. Six modules related to quality improvement were chosen and mandated to be completed by all residents, although they had access to the rest of the modules related to patient safety and high value care. Small groups of 15–20 residents were created with equal distribution of interns to third year residents. Each group covered one module per session and resident members from the group then presented and discussed the content of the modules. One core faculty member monitored the completion of the curriculum and moderated the session with the residents. Residents who did not attend the assigned groups were able to attend group sessions at other times. We explored residents’ attitudes regarding the value of the curriculum through a monkey survey prior and post implementation of the didactic schedule. Residents were asked to submit QI projects following the curriculum in accordance with the charter form (PDSA) available on the IHI website. Multidisciplinary work groups involving leadership, residents, and research personnel were created to assess improvement and the quality of ongoing projects. The residents categorical responses before and after the curriculum were analyzed using McNemar’s test of marginal homogeneity as the data does not belong to normal distribution. Data analysis was done using SPSS Version 17. The program was exempt from the Institutional Review Board as a part of Department of Medicine led quality of improvement initiative.

Results

A total of 85% (100/117) of residents completed the pre-curriculum survey and 52% (61/117) completed the survey post curriculum. About 48% of residents were PGY1 and 40% in PGY2/PGY3. Results of the residents’ responses on the questions investigating understanding and attitude were summarized (Table 1). A total of 96.7% of residents report that physicians should lead QI projects and training rather than the hospital administrators (p <0.01). There was a 20% (30 vs. 50.8) increase in understanding and confidence in quality improvement projects post curriculum. The various triggers and processes undertaken in 2015–16 are listed in Table 2. We observed 14% (p <0.008) increase in participation of residents after the curriculum. As compared to previous years, the number of resident-driven QI projects increased after the QI curriculum was implemented (11 vs. 4), Table 3. Very few residents were involved in QI projects before the start of curriculum. The resident-driven projects with baseline problems and suggested interventions are given in Table 3.

Table 1.

Survey responses pre and post curriculum

		Pre assessment % (n = 100)	Post assessment % (n = 61)	P value
1	What is your understanding of Quality improvement
	None /Minimal	49	31.1	0.04
	Knowledgeable/Very Knowledgeable	51	68.9
2	How confident do you feel in undertaking/Leading a QI project
	Not confident at all/Somewhat confident	70	49.2	0.048
	Confident/Very confident	30	50.8
3	To what extent do you think that physicians should undergo training on QI?
	None – should be done by administrators	6.06	3.3	0
	Partly/Definitely – Physicians should initiate and lead QI projects	93.93	96.7
4	How interested are you in initiating a QI Project?
	Not at all/Somewhat interested	45.53	49.2	0.331
	Interested/Very interested	54.54	50.8
5	Should the QI curriculum be taught in residency?
	Yes	62	72.1	0.008
	No/Don’t Know	38	27.9
6	Have you participated in a QI?
	Yes	34	47.5	0.05
	No	66	52.5

Table 2.

Triggers with process in resident driven initiatives.

Trigger	Processes
Poor patient satisfaction (HCAP Score)a	Morning conferences, one to feedback with residents and interns
Inappropriate imaging	Imaging appropriateness criteria (AUC)
Undetected delirium – patient at risk of in hospital complications	Institution of CAMb
Readmissions in alcoholic patients	CIWAc instead of AWATd, referral to Rehab
Inappropriate telemetry – higher cost, bed assignment	Telemetry utilization and guidelines revision
Increase in CHF Readmissions	Discharge planning and inpatient checklists
Direct and Indirect cost of Nebulizers	Replace MDIse with spacers
Delay in Tracheostomy and PEGf	Monitoring patients, To institute a protocol with GI and surgical colleagues
Readmissions to diabetes	Management of hyperglycemia
Increasing use of isolation and delayed removal from contact precautions	Institution protocol for MDROg and surveillance
Poor pain management	Revision of pain management protocol based on risk groups
Increasing efficiency of Rapid response and codes	Increase in simulation training sessions and scenarios
Inadequate patient safety	Practice based learning, Safety morning reports and seminars

aHCAHPS: hospital consumer assessment of healthcare providers and systems; bCAM: confusion assessment method; cCIWA: clinical institute withdrawal assessment for alcohol; dAWAT: acute alcohol withdrawal assessment; eMDI: metered dose inhalers; fPEG; percutaneous endoscopic gastrostomy;  gMDRO: Multidrug resistant organism. 

Table 3.

Examples of projects and suggested interventions by residents.

		Underlying problem	Interventions suggested by residents (PDSA cycle 1)
1	Physician-patient communication and patient satisfaction	Poor HCAP Scores	Morning lectures, Discharge script
2	Delirium and complications	Not identifying delirium – patient at risk of in-hospital complications e.g., falls, aspiration pneumonia.	Routine administration of CAM questionnaireIn progress
3	Imaging appropriateness	Inappropriate imaging and higher utilization of imaging – higher costs	Hospital-wide stewardship program, Lectures, email reminders.Radiology-Internal Medicine meeting
4	Surveillance and Contact isolation	Surveillance of all ICU and 9b units.The number of inpatient admissions greater than surveillance cultures	Identifying high risk patients at risk of MDRO transmissions
5	Alcohol withdrawal	Risk of alcohol withdrawal and delirium and prolonged stay	Short validated questionnaire to CIWA to replace AWAT. Prevent delirium tremens and prolonged stay.
6	Pain management	Pain management was inadequate in our survey (46 Spanish and 55% non-Spanish speaking)	Six questions to assess patient quality improvement programPatient expectations can improve pain
7	Telemetry utilization	The inappropriate use in 2013 was 33%.	Attending monitoring for appropriate use of telemetry reduces length of stay and cost.
8	Readmissions	The readmission rates vary from around 12–30%	Involvement of care management, Home health, Health home and PharmacyDischarge and admission process
9	Simulation training	Poor communication during rapid response codes	Increase in training sessions
10	Patient drug safety	Medical prescription errors.	Bi Monthly patient safety meetingsInvolvement of Pharmacy in QI initiatives
11	Accidental Extubationsa	Accidental Extubations (Just initiated)	Adherence to guidelines framed by ‘ICU team’

aNursing staff and residents. HCAHPS: Hospital consumer assessment of healthcare providers and systems; CAM: confusion assessment method;  MDRO: multidrug resistant organism; clinical institute withdrawal assessment for alcohol.

Discussion

As healthcare delivery becomes more complex, the need to educate the future clinicians in tools for successful implementation of safety measures and providing quality care has been felt by all institutions [6,7]. Governing bodies such as ACGME have created a reporting milestone for reporting the engagement of a physician in Quality improvement projects, highlighting the need for clinician educators to incorporate any of the teaching tools available into their residency curriculum [2]. The General Medical Council from the National Health Service in the UK recommends all doctors to take part in systems of quality improvement assurance and quality improvement [8]. A study from Dartmouth Medical School has corroborated the benefits of student involvement in clinical projects, such as an improved awareness of clinical issues and ability to provide novel approaches [9]. Our Internal Medicine residents are increasingly taught continuous quality improvement principles through QI curriculum and QI projects within the framework of Plan-Do-Act-Cycle and are given multiple opportunities in leadership roles, participate in patient care QI teams and committees for the past year. With their knowledge of how to design and run PDSA cycles, we hope that overall physician engagement will lead to better patient outcomes

Fewer residents completed the survey post curriculum as compared to pre curriculum (85 vs. 52, p = 0.04). Our low post curriculum survey rate might be due to the large number of residents in the program and this was the first year of our quality improvement initiative. Sepulveda et al. reports lower post-intervention survey (84 vs. 71%), in spite of having less residents [10]. Most of the studies [10,11] published involved fewer residents as compared to ours. The number of responses by residents were equally distributed in our groups irrespective of PGY level (48% vs. 40%) in contrast to a study by Smith et al. [12], where the response rate is reduced by 50% (40% in PGY1 vs. 24% in PGY2) by the second year. Continuous email reminders and resident enthusiasm towards QIs would have increased participation of PGY2s. The percentage of residents showing understanding and confidence in leading QI projects increased by 20% in six months. About 70% report that the QI curriculum should be taught in residency. The introduction of the IHI QI curriculum has been shown to make a positive impact on resident education of quality improvement in a surgical and dermatology residency program [11,13]. A total of 97% felt that physician educators should lead QI projects as opposed to hospital administrators. Initiatives were proposed to involve clinician administrators leading resident-driven QI projects [14]. Residents being the first line contact with the patient have more input on the failures of delivery of care than administrative personal. Residents were less likely to initiate a QI project post curriculum (54 vs. 52, p = 0.33) in contrast to leading a QI project (30 vs. 51). The former might be due to lack of the participation of faculty members skilled in QI projects as it sometimes can be resource intensive in a public hospital system like ours.

Sustained change in improvement of outcomes was documented for 6 out of 11 active projects started by residents (Ongoing PDSA cycle 2/3 data not shown). One such example of the major improvement was in patient communication and satisfaction (Table 4). Residents submitted the plan which was approved by our leadership and blinded surveys were done by residents not involved in patient care. Knowledge of the PDSA format allowed the residents to act on specific aspects of the patient experience with different PDSAs e.g., pain control, knowledge of medications, physicians’ attitude etc. and act on them.

Table 4.

PDSA cycle – patient-physician communication project.

	Baseline problemPoor Patient satisfaction scores		Pitfalls
Patient Surveys for this project by residents notinvolved in the project or by administrator fromInternal Medicine	PDSA cycle 1	Group discussions, Lectures during morning reportsRegular emails from preceptors about Patient satisfaction scores
	PDSA cycle 2	Resident to Intern feedback after the survey	Intern – resident might not meet the same day
	PDSA cycle 3	Lectures+ Resident-Intern feedback,+ Bedside checklist with intern at admission	Checklist are often ignored by interns/Residents
	PDSA cycle 4	Lectures + Resident-Intern feedback,+ Bedside checklist at admission through guest relations withoriginal package and patient informed+ Check list at discharge	Beginning of implementation

Pitfalls to resident-run QI projects are well known and understood by all programs [14], and are shown in Table 5. Integration of the Quality improvement curriculum in to resident training [15] may be stressful due to time constraints, but nevertheless, the majority of residents showed remarkable enthusiasm to write up projects based on the PDSA charter and start QI projects after the IHI open school model for improvement curriculum was implemented. Sustaining all the QIs will continue to be a challenge due to conflicting resident responsibilities, lack of leadership for QI, lack of elective time for internists and faculty in a community hospital, and poor recognition within the institution of the work and resident’s efforts. Our experience taught us that continuous workflow is critical to sustenance of the projects and any interruptions in leadership-preceptor-resident delayed the ongoing QI projects.

Table 5.

Pitfalls and possible solutions.

	Pitfalls	Change processes
1.	ICU rotation by residents/Vacation	Flexibility given
2.	Timing	Paging twice (Morning and 30 minutes before meeting)
3.	Compliance of meeting	Email reminders by program director
		Closed door meeting with leadership of internal medicine
4.	Unclear design of project	Residents were given a tour of Bellevue and Metropolitan hospitals to learn from their experience in execution in two projects
5.	Lack of interest by residents	Involvement of interns early in the residencyFaculty improvement program in QI.
		Resident Competitiveness – Involvement of 2/3 residents in same project
		Gift cards for ‘best suggestions’ for improvement.
		Third-year residents given opportunity to drive the resident project as they tend to be less interested in data collection after fellowship match.
6	Faculty inexperience	Resident-Faculty teams, Adoption of QUIC (Quality initiative collaborative) at our institution.
7	Low resident surveys	Anonymous paper surveys as opposed to bulk emails through the hospital-wide system

There is scant literature showing that resident-driven QI projects lead to better patient outcomes and this remains a challenge for forthcoming projects at our institution. Resident-adopted QIs showed sustained improvement in outcomes of papilloma vaccine response rates [16], obesity screening rates (not sustained at six months) [17], and pediatric immunization rates [18]. We are in the process of actively relating resident-initiated QI projects to patient outcomes. For example, Simulation training to video recordings of actual code encounter, routine CAM assessment to decrease number of falls or imaging appropriateness criteria based on guidelines to prevent inadvertent treatments and prolonging hospital stay.

Projects which are feasible and relevant to residents’ workflow and residents’ fellowship might improve buy-in and to completion. Involvement of medical students and fellows in the QI program might be an effective way to increase resident participation. To this purpose, our leadership adopted QUIC – ‘Quality improvement collaborative’ with early adopters in PGY1, PGY2, and PGY3 to speed up the quality improvement curriculum started one year ago, some of which are incomplete. Involvement of front runners of patient care like interns early in training i.e., within four months of joining the residency program might help them to run more PDSA cycles before they graduate, so they can see tangible improvements in patient outcomes and improve patient safety [19-21]. Integration and elective time for internists and faculty as part of the QUIC initiative might increase further resident participation. The strength of this project it is that it is done in a manner that can be easily replicated in other institutions and programs.

Limitations of study: There are no data linking successful involvement of resident’s involvement in QI projects to patient outcomes and we are in process of implementing some of our QI projects. Low numbers of residents participating in the post curriculum survey might have skewed some of the results. We acknowledge that PDSA is not a scientific method, but the open school IHI charter (PDSA) is widely used to report studies and analogous to SQUIRE guidelines [22].

Conclusions

Initiation of continuous quality improvement initiatives in a busy residency program of a public health system in a resource-limited setting is feasible only with the help of a dedicated curriculum. Long-term sustainability of our projects depends on the active participation of faculty members who are skilled in QI, development of Resident-Faculty teams, close assessment and tracking of projects. The resident-driven QI on patient outcomes is yet to be determined in our institution. Further studies are required to measure the direct residents’ participation to clinical outcomes. Going forward, we would like to integrate residents’ QI with institutional QI projects to ensure effective health care delivery.