Interventions to improve resident reporting of patient safety events: a quality improvement initiative.

BACKGROUND: Patient safety events (PSE) are opportunities to improve patient care but physicians rarely report them. In a previous study, residents identified knowledge regarding what constitutes a PSE, perceived lack of time, complexity of the reporting process, lack of feedback, and perceived failure to resolve the issue despite reporting to be barriers limiting their PSE reporting. The residency programs and system patient safety and quality improvement departments created targeted interventions to address identified barriers.
OBJECTIVE: Assess effectiveness of targeted interventions on improving PSE reporting rates amongst residents.
METHODS: As part of a multi-residency patient safety project, interventions were created to focus on the removal of barriers to reporting PSE identified previously. Post-interventions, an identical cross-sectional survey of the residents at the same two community teaching hospitals was conducted from Sept to Dec 2018 through an online questionnaire tool.
RESULTS: 78 out of 149 residents (52.3%) completed the survey. We found a significant improvement in the number of residents who endorsed reporting a PSE in the past 1 year (51.2% vs 23.5%, p = 0.001), as well as during the course of their training (52.6% vs 26.5%, P = 0.001). There was also a significant decrease in the number of residents who were unsure of how to report a PSE (p = 0.031) as well as those who viewed medical error as a sign of incompetence (p = 0.036).
CONCLUSION: Our study demonstrates that simplifying the PSE reporting process, improving knowledge and acceptance of patient safety/quality improvement principles and promotion of a just culture improves resident PSE reporting.

Introduction

Preventable harm in healthcare remains a major public health safety threat. A patient safety event (PSE) is defined as any process, act of omission, or commission that results in hazardous healthcare conditions and/or unintended harm to the patient [1]. Studies have indicated that up to 400,000 patient deaths per year in US hospitals can be attributed to medical errors, making medical errors the third most common cause of death in the USA [2].

Reporting of safety events by physicians, including resident physicians, remains less than expected. Milch et al. reported a low rate of 1.4% of PSE reports filed by physicians [3]. Schectman, et al., found that approximately two thirds of surveyed physicians have never reported a patient safety event, even though 60% of them were aware of 3 or more events in the preceding year[4].

The learning environment in which a resident trains has significant impact on future attitudes towards patient safety, practice patterns, and ability to utilize precepts of high-value cost-conscious care. These cultural effects have been shown to ‘imprint’ on the residents and to persist for at least 15 years [5]. Therefore, it is important to stress the attention needed to PSE reporting during residency training to change the culture long term.

During academic year 2016–2017, Rochester Regional Health conducted patient safety surveys in preparation for an ACGME site visit and found poor participation in PSE reporting by resident physicians. Resident physicians at Rochester General Hospital (RGH), Rochester, NY, a 528-bedded urban community teaching hospital, filed only 29 PSE reports in the preceding 3 years. Similarly, resident physicians and faculty at Unity Hospital (UH), Rochester, NY, a 351-bedded suburban community teaching hospital submitted only 22 out of 2133 PSE reports filed during academic year 2016–17 [6]. This led to a resident-led quality improvement project to assess our residents’ knowledge regarding PSE and to identify the barriers contributing to low PSE reporting. That study revealed that knowledge regarding what constitutes a PSE was the major barrier limiting the reporting. Similar to other studies, secondary barriers included time and complexity of the reporting process, lack of feedback, and perceived failure to resolve the issue despite reporting [6].

To improve resident safety event reporting in our system, a series of targeted interventions were developed to address the deficiencies noted in the previous study. This is the initial Plan – Do – Check- Act (PDCA) cycle of those interventions, which aimed to change resident behaviour and approach towards preventable harm by promoting education on safety events and making reporting easier and more efficient.

Methods and analysis

The targeted interventions were created via a partnership between the residency programs and the system patient safety and quality improvement departments. Analysis of the previous study indicated several areas in which maximal change could be affected - a less complex and time-consuming reporting process, education on events, and putting principles to practice in real-world events (e.g., morbidity & mortality conference, root cause analyses (RCAs)). These interventions and their rationale are summarized in Table 1.

Table 1.

Interventions made based on barriers identified on the previous study.

Intervention	Rationale	Timeline
Change PSE reporting system	Simplify reporting effort to increase participation
Enhance patient safety and quality improvement curriculum: 2 noon conferences run by patient safety and quality department Live demonstrations of patient safety event reporting system Mandatory on-line course on the science of patient safety and types of patient safety event reports	Improve knowledge and acceptance of patient safety and quality improvement principles and methodology	June-July, 2018Online courses to be completed within 4 weeks of assignment
Resident-specific monthly morbidity & mortality conference with focus on quality improvement methodology (e.g., fishbone analysis) as tool for analysis of care	Emphasize role of resident as front-line provider with responsibility to report PSE to prevent harmPromotion of ‘Just Culture’	Started in December, 2017
Semi-annual simulated root cause analyses run by patient safety department	Enhance knowledge of process	June-July, 2018
The interventions targeted all the residents who were part of the study.

The entire health system, encompassing all the residency programs that are part of the study, changed vendors to a simpler, faster reporting system (Riskonnect, Kennesaw, GA, USA). This was aimed at reducing the time and effort of the PSE reporting process. The new system is online and easily accessible from work or home computers. The reporting process was streamlined, requiring input via clicking sequentially narrowing event categories and entering text into free text boxes; it takes 3–5 minutes to enter most safety events. Residents who wish to remain anonymous have the option to submit the report as ‘residency program’ so it could be tracked by the Program Administration and feedback given to the residents regarding the outcome of the event.

Any system, howsoever simple it may be, would need a favorable user perception for gaining acceptance [7,8]. With this guiding principle, we made a concerted effort to not only train the residents on the importance of reporting patient safety events, but also on how to use the new system. In addition to compulsory online training modules, we also conducted sessions on training for the new system during noon conference where simulated PSE were filed to demonstrate the ease of the process. There was minimal expenditure incurred in these training activities.

Upon the completion of the interventions, a prospective cross-sectional survey was conducted at RGH and UH between Sept and Dec 2018, the same sites and residency programs as the previous study (May-Sept, 2017), only temporally separated by 15 months. Participants included all house staff currently undergoing post-graduate residency training at RGH and UH. Residents from Internal Medicine (58 residents and 5 chief residents), Radiology (16), Obstetrics and Gynecology (16), Dentistry (5) and Podiatry (9) residency programs at RGH and the Internal Medicine residency program at UH (41) were included in the study. During this time period, the final year of residents involved in previous study graduated and a new batch of 1st year residents joined their programs and became part of the current study. All residency programs across both hospitals are fully accredited and provide education and other clinical experiences as outlined by their respective accrediting bodies (ACGME, CODA, CPME).

For consistency, the survey tool used in the current study was identical to the one designed for the previous study [Supplementary 1]. It included possible barriers to reporting, expressed on a Likert scale as well as three clinical vignettes designed to assess the residents’ ability to identify safety events and classify them as near miss, adverse events or sentinel events and indicate whether they were reportable or not. The questions also assessed the respondent’s demographic data such as gender, speciality and current level of training and medical school background [6].

The online questionnaire was emailed to all the participants via their official work email addresses. Participation was anonymous, voluntary, and disclosing demographic details was optional. Three weekly reminders were emailed to all the potential respondents. This study was approved by the Institutional Review Board of Rochester Regional Health.

Perceptions of individual barriers to reporting PSE were expressed as seldom (0–25%), sometimes (26–50%), often (51–75%) or most of the time (76–100%) on a Likert scale. Mann-Whitney U test was used to analyze these ordinal data as well as to compare to responses from the previous survey. Chi-square test was used for comparing categorical data for this survey and comparison to the previous survey. Descriptive statistics including frequency distribution tables and cross-tabulations were calculated using SPSS v25.0.0.0 (International Business Machines, Armonk, NY, USA). Statistical significance was defined as p-value <0.05.

Results

A total of 78 residents completed the survey compared to 98 respondents for the previous survey, with an overall response rate of 52.3%(previous response rate of 67.6%). 39.7% of respondents were first year residents compared to 25.6% and 24.4% of respondents from second and third year of training, respectively.

A majority of respondents were training in internal medicine (85.9%), with lesser representation from obstetrics-gynecology (2.6%), dentistry (2.6%), podiatry (3.9%) and radiology (5.1%). Male and female distribution was 62.8% and 35.9%, respectively. 73.1% of respondents were non-US citizens from international medical schools, while US citizens from international medical schools and US citizens from American medical schools constituted 16.7% and 10.3% representation, respectively.

A significantly increased number of residents endorsed reporting a PSE during their training, compared to the previous survey (52.6% vs 26.5%; p = 0.001). Similarly, there was a significant increase in the number of residents that had submitted a PSE in the past 1 year (51.2% vs. 23.5%; p = 0.001 (Table 2). There were significant decreases in the number of respondents who were unsure of how to report a PSE (p = 0.031) and viewed medical error as a sign of incompetence (p = 0.036) (Table 3). There was no significant difference in the number of residents with respect to the other possible barriers assessed.

Table 2.

Comparison of the total number of respondents and the number of respondents who endorsed submitting a patient safety event during training as well as over the past 1 year across two studies before and after targeted interventions as above. There was a significant increase in the number of residents who had ever reported a PSE (p = 0.001) as well as a significant increase in the number of residents who had reported a PSE within the past 1 year (p = 0.037).

Change in resident reporting of patient safety events before and after interventions
	Current Survey(After intervention, Sep- Dec 2018)	Previous study(Prior to intervention, May-Sept 2017)
Total number of residents surveyed	149	145
Total number of respondents	78	98
Number of residents who endorsed reporting a PSE during residency training.	41 (52.6%)	26 (26.5%)
Number of residents who did not report a PSE in previous 1 year.	0 (0%)	3 (3.1%)
Number of residents who endorsed reporting 1–2 PSE in past 1 year.	32 (41.0%)	21 (21.4%)
Number of residents who endorsed reporting 3–4 PSE in past 1 year.	6 (7.7%)	1 (1.0%)
Number of residents who endorsed reporting >/ = 5 PSE in past 1 year.	2 (2.6%)	2 (2.0%)

Table 3.

Perceived barriers to safety event reporting. There was a significant decrease in the number of respondents who were unsure of how to report a PSE (p = 0.031) and those who viewed medical error as a sign of incompetence (p = 0.036).

	Likert Scale Responses
	Seldom (0–25%)		Sometimes (26–50%)		Often (51–75%)		Most of the time (76–100%)		Number of respondents for the barrier
Perceived Barriers to reporting	Previous Study	Current Study	Previous Study	Current Study	Previous Study	Current Study	Previous Study	Current Study	Previous Study	Current Study
1. Unsure how to submit a report	34 (37.0%)	33 (46.5%)	13 (14.1%)	22 (31.0%)	28 (30.4%)	6 (8.5%)	17 (18.5%)	10 (14.1%)	92	71
2. Time it takes to submit an event report	18 (20.5%)	15 (21.1%)	31 (35.2%)	24 (33.8%)	17 (19.3%)	18 (25.4%)	22 (25.0%)	14 (19.7%)	88	71
3. Unsure of what is considered a patient safety event	43 (48.3%)	34 (48.6%)	24 (27.0%)	20 (28.6%)	13 (14.6%)	10 (14.3%)	9 (10.1%)	6 (8.6%)	89	70
4. Fear of retribution to self	48 (53.9%)	42 (59.2%)	24 (27.0%)	21 (29.6%)	10 (11.2%)	5 (7.0%)	7 (7.9%)	3 (4.2%)	89	71
5. Fear of retribution to others	39 (43.8%)	26 (37.1%)	25 (28.1%)	22 (31.4%)	15 (16.9%)	15 (21.4%)	10 (11.2%)	7 (10.0%)	89	70
6. Fear of violating hierarchy	35 (40.2%)	36 (52.2%)	30 (34.5%)	22 (31.9%)	17 (19.5%)	7 (10.1%)	5 (5.7%)	4 (5.8%)	87	69
7. Lack of perceived change due to submitting a patient safety event	30 (33.7%)	22 (31.0%)	33 (37.1%)	27 (38.0%)	19 (21.3%)	12 (16.9%)	7 (7.9%)	10 (14.1%)	89	71
8. Medical error seen as a sign of incompetence	31 (35.2%)	34 (48.6%)	30 (34.1%)	24 (34.3%)	19 (21.6%)	9 (12.9%)	8 (9.1%)	3 (4.3%)	88	70
9. Increased scrutiny threatens medical autonomy	40 (45.5%)	33 (47.1%)	28 (31.8%)	29 (41.4%)	15 (17.0%)	4 (5.7%)	5 (5.7%)	4 (5.7%)	88	70
All respondents did not submit a response to all the barriers mentioned in the survey.

While 89.9% of residents were able to correctly identify a near miss, only 72.5% and 62.3% of respondents were able to identify an adverse event and sentinel event, respectively (Table 3). Most of the respondents (94.2%) would report a sentinel event, even if they did not correctly recognize it as a sentinel event. However, 17.4% of the respondents who answered the vignettes did not think an adverse event was reportable, while more than half (50.7%) did not think near misses were reportable. This represents a non-significant decrease in correctly identifying PSE types from the previous survey. There was a similar non-significant decrease in considering sentinel events reportable, but also a non-significant increase in considering adverse events and near misses reportable.

There was no significant difference in the level of reporting or knowledge based on resident demographics, or between residents training in different specialties and between residents at different levels of training.

Discussion

Previous studies looking at causes of underreporting of PSE by resident physicians found systemic barriers like lack of a reporting system, cumbersome reporting process, lack of anonymous reporting, and ‘human barriers’ such as limited amounts of time, fear of retribution and a personal knowledge gap regarding what qualifies as a reportable event as a common theme [9]. Our previous study indicated that a knowledge gap extending to both recognizing PSE and understanding the need to report them were the most important barriers to reporting. We also found that there was no difference in reporting pattern between residents who identified themselves as IMGs versus those that identified themselves as American citizens who trained in US medical schools or international medical schools. This prompted us to conclude that targeted interventions directed at potentially misguided cultural and educational expectations about IMG residents would likely be unsuccessful.

To create experiential learning opportunities, resident-driven morbidity and mortality reports and root cause analysis sessions were conducted. These learning events aimed at mitigating the ‘human factor barriers’ to reporting, reinforced that PSE reports were dealt with as opportunities to improve the system and patient care, and were not linked to punitive outcomes. We believe that this understanding may have helped residents be more forthcoming in reporting PSE. This is supported by the significant reduction in residents who felt that occurrence of PSE was a sign of medical incompetence. The residents were also given feedback on the outcomes of their PSE reports, directly or indirectly, with the goal of allaying concerns about the utility or change to which such reporting would lead.

Despite this improvement in the reporting process, there was no significant change in the performance of residents in correctly identifying or reporting the PSE scenarios presented in the clinical vignettes. There remains an opportunity to improve our curriculum content to help residents correctly recognize and report PSE scenarios. While some human factors were alleviated, improving resident knowledge while trying to sustain human factor gains will be the focus of future PDCA cycles.

Our study was limited by small sample size (149 residents) and including two teaching hospitals, which may not be representative of all residency programs. 52.3% of potential respondents completed the survey. The reasons for low participation rate may include lack of time to participate, timing of the survey, residents not checking their email or simple unwillingness to participate in a repeat survey. While this rate echoes previous studies, there is possible introduction of nonresponse bias despite the debate as to whether lower response rates affect the results [10,11]. Finally, both the current and previous studies looked at PSE reporting as endorsed by residents, which may not be accurate vis a vis actual event reporting.

Conclusion

Lack of knowledge still plays an important role in low resident PSE reporting rates. However, this first PDCA cycle is encouraging as indicated by the approximate doubling of respondents who have filed PSE and the percentage of residents who are unsure about reporting process and who view medical errors as a sign of incompetence decreasing significantly. In efforts to create a culture of reporting, our findings suggest that programs perform a self-assessment and invest their limited resources for training in case-based educational modules targeted to their individual programmatic barriers.

Click here for additional data file.