Teaching medical students to recognise and report errors.

Background: Medical student error reporting can potentially be increased through patient safety education, culture change and by teaching students how to report errors. There is scant literature on what kinds of errors students see during clinical rotations. The authors developed an intervention to better understand what kinds of errors students see and to train them to identify and report errors.
Methods: A safety curriculum was delivered during the Medicine clerkship for the academic year 2015-2016. Prior to the workshop, students completed a preintervention survey to determine whether they had reported a clinical error. Subsequently, they participated in an educational workshop. Facilitated discussions about conditions contributing to errors, types of errors, prevention of errors and importance of reporting followed. Students were required to submit a simulated error report about an error they personally observed. An end-of-year survey was sent to students who participated in the curriculum to determine clinical error reporting frequency.
Results: Students submitted 282 reports. Near miss errors were seen in 64% and adverse events in 36%. National Quality Forum serious events were reported in 14%, including one death. Recommendations to prevent similar events were weak (62%). Students correctly categorised 93% near miss, 88% adverse events, 67% diagnostic, 81% treatment and 78% preventative errors. On the preintervention survey, 8.5% stated they submitted an error report to their clinical site. On the end-of-year survey, 18% confirmed submitting a formal error report.
Conclusion: Training students to recognise and report errors can be successfully integrated into a clinical clerkship and impact clinical error reporting.

Introduction

Medical errors have been described as the third leading cause of death in the USA, which has been estimated to result in over 251 000 deaths annually.1–3 The true incidence of deaths related to errors is thought to be much higher.1 3 Due to lack of error reporting and inaccuracy of medical error detection, medical errors are not well captured.1 2

On the institutional level, hospitals are required to track serious adverse events (AEs), perform root cause analysis on sentinel events (‘unanticipated death or major permanent loss of function not related to the natural course of the patient’s illness or underlying condition’) and enforce patient safety measures to decrease such events in order to participate in Medicare.4 5 To achieve this goal, reporting systems are commonly used.4 These systems depend on voluntary reporting.4 6 Currently, most reporting is done by nursing (67.1%6–93.3%7) with a substantially lower contribution from physicians (2%8–23.1%6) and even lower by medical students (7%).9 10 This provides an opportunity to teach patient safety from the beginning of medical education as a way to decrease harm.11

Reporting can potentially be increased through patient safety education, culture change,12 13 highlighting the importance of medical error reporting,14 15 and teaching how to file incident reports. There is scant literature on what kinds of errors students see during clinical rotations and their reporting behaviours.9 10 Previous studies focused on acquisition of knowledge and attitudes of medical students related to patient safety12 13 16–19 with less focus on skill development such as error reporting. To fill this gap, we developed an intervention to better understand what kinds of errors students see and to train them to recognise and report errors. Here we describe the outcome of this intervention and the impact on clinical error reporting.

Methods

Wayne State University School of Medicine has a class of nearly 300 medical students. Students rotate on the Internal Medicine service in 8- week blocks during their junior year for their core clerkship in Internal Medicine. This provided an opportunity to educate students on quality improvement and patient safety (QIPS). A curriculum was developed using Agency for Healthcare Research and Quality (AHRQ) primers,20 Institute of Healthcare Improvement (IHI) modules and the Institute of Medicine’s reports21–23 on patient’s safety with the goal of providing a foundation in QIPS that included a focus on error reporting to improve health systems. The curriculum was developed in 2014, piloted in mid-academic year 2014–2015 and delivered to the all Internal Medicine clerkship students in academic year 2015–2016. The final curriculum consisted of two IHI modules (PS 201: Root Cause and Systems Analysis and ​PS 202: Building a Culture of Safety),24 25 review of cases on the AHRQ WebM&M Cases & Commentaries site26 and an internally developed a module on handoffs. Three safety assignments were assigned including observation of a sign out and completion of a sign out for review, a postdischarge phone using the AHRQ form, as well as submission of a near miss (NM) and AE report. Students attended a workshop about QIPS which occurred during the third or fourth week of the rotation. Education on error reporting was specifically delivered at the workshop and through the NM/AE reporting assignment.

Preintervention assessment

Students completed a written preintervention assessment/test (which was later administrated electronically) about their knowledge of patient safety topics. The assessment including short answer and multiple-choice questions. A question about whether they had ever reported a NM or AE at their clinical site prior to the workshop was included.

Interventions

Students participated in a 4-hour workshop which included (1) an overview of patient safety and medical errors addressing the scope of the problem, models of error and relevant definitions (errors, NMs, AEs, etc). Concepts related to error (human factors engineering, just culture, etc) were introduced (1 hour). This was followed by (2) viewing an episode of a popular television show, Grey’s Anatomy (I Saw What I Saw: season 6 and episode 6) depicting a series of errors culminating in a serious AE—death of a patient (45 min). (3) An overview of error analysis and the tools and processes use to analysis errors and AE was presented (15 min). Students then used these tools to analyse the events depicted on the television show (45 min). This was followed by facilitated discussions about conditions contributing to errors, types of errors, error prevention, interventions/actions and strength of actions to prevent errors (30 min). The programme culminated in discussions about the importance of reporting (15 min).

A postintervention assessment was administered to determine immediate recall and short-term learning. Students were also required to complete an assignment that included reading a statement from the WHO regarding the role of patient safety reporting in enhancing patient safety by learning from failures of the healthcare system.27 A list of National Quality Forum (NQF) serious reportable AEs—‘serious, largely preventable and harmful clinical events, designed to help the healthcare field assess, measure and report performance in providing safe care’ was provided to students as part of the assignment.28

Students were then required to report an error they personally observed during the rotation to an online learning-management system (Blackboard) that included a description of the event (what happened), patient harms, what could have happened, contributing factors and recommendations to prevent similar events in the future. They also had to categorise whether the event was a NM or AE; definitions of both were provided at the beginning of the assignment, the type of error (diagnostic, treatment, prevention or other) per Lucian Leape’s classification,29 whether a handoff was involved (defined by AHRQ as the process of transferring responsibility for care),30 if it prolonged hospital stay and whether the error was a NQF serious reportable event.28 Assignments were reviewed by the Clerkship Director. Students received individual feedback by email.

Analysis of assignments

The number of NM/AE and type of error (diagnostic, treatment, preventative and other) were analysed quantitatively. Assignments were reviewed independently by two physicians with training in QIPS. NE/AE were classified based on standard definitions. NM events we defined using the AHRQ definition, ‘any event that could have had adverse consequences but did not and was indistinguishable from fully fledged adverse events in all but outcome’.31 AEs were defined using the IHI definition, ‘unintended physical injury resulting from or contributed to by medical care (including the absence of indicated medical treatment) that requires additional monitoring, treatment or hospitalisation, or that results in death’.32 Conflicts regarding the identification and labelling of an event as an NM or AE and type of errors were reviewed with a third physician, discussed and resolved by consensus. Results were compared with student’s reports. Qualitative analysis was performed using methodology derived from Strauss A and Corbin J’s Basics of Qualitative Research.33 A master list of contributing factors was created from the reports. Inductive codes were formulated to identify contributing factors. Similar factors were grouped according to their respective codes. A total of 24 codes emerged during this process. Thirteen themes related to patient safety were revealed: medications, miscommunication, cognitive, nursing issues, failure to follow orders/protocols, EMR issues, overwork, transition of care, overnight care, poor chart review, falls, device failure and weekend delays. Reports were studied one more time to check the appropriateness of the codes. Discussion and consensus addressed disagreements in coding.

Student’s recommendations to prevent future events were reviewed and categorised into weak, intermediate and strong actions. The classification used was from the Root Cause Analysis Tools from the Veterans Administration National Center for Patient Safety.34

End of academic year survey

Students were surveyed at the conclusion of the year to determine if they had submitted a formal NM/AE report during their clinical rotations. The survey consisted of two questions. (1) Did you ever submit a formal NM or AE report to your hospital site (yes/no)? (2) If yes, on which rotation (multiple choice). The survey was sent three times between 7 July and 24 July 2016.

The project was reviewed by the Investigative Review Board (IRB) at Wayne State University and did not require IRB approval.

Results

A total of 282 students completed the curriculum between July 2015 and June 2016. Of 282 completing the curriculum, 280 students completed the preintervention survey; 24 (8.5%) indicated they submitted an error report to their clinical site prior to the workshop.

NM/AE reports were submitted by 282 students; 278 answered all questions and were included in this analysis. No student submitted more than one report. Overall, 179 (64%) students reported NM and 99 (36%) reported AE. There was one AE described that was associated with death of a patient. A total of 73 (26%) reports involved hand-offs. Prolonged hospital stay due to the error was observed in 106 (38%) reports. An NQF event was identified in 40 (14%). Types of error reported were primarily related to treatment or prevention (figure 1).

Figure 1

Type of error reported by 278 medical students for the academic year of 2015–2016 at Wayne State University, School of Medicine.

Reports provided enough detail to categorise and identify contributing factors. Some were more detailed than others providing rich descriptions of events. Some were less detailed, but all contained enough information to analyse. An example of NM and AE are provided (figures 2 and 3).

Description of a near miss event provided by a medical student for the academic year of 2015–2016 at Wayne State University, School of Medicine.

Description of an adverse event provided by a medical student for the academic year of 2015–2016 at Wayne State University, School of Medicine.

Students identified various contributing factors leading to NM and AE. Most identified multiple factors. Factors were reviewed and categorised. The most common errors involved medications (eg, missed medications, wrong route of administration, duration, dosing and failed medication reconciliation). Miscommunication and cognitive errors (eg, limited differential diagnosis, premature closure) were present in roughly one-third of all the cases. Students identified being overworked as a contributing factor to errors caused by healthcare providers and ancillary staff including physicians, nurses, pharmacists, transporters and patient sitters. Errors related to nursing (eg, understaffing, not following through orders, charting error) were seen in nearly 20% of reports. Errors specifically related to orders (eg, missed orders, following orders incorrectly) were seen in 16% (figure 4).

Figure 4

Common contributing factors reported by 278 medical students for the academic year of 2015–2016 at Wayne State University, School of Medicine.

Recommendations

Student recommendations to prevent similar events in the future were predominately weak in nature. Of 278, 172 (61.9%) were weak, 99 (35.6%) were intermediate and seven (2.5%) were strong in nature. The majority of the recommendations were concrete. Representative examples are shown in table 1.

Table 1

Examples of student’s recommendations categorised by strength for the academic year 2015–2016 at Wayne State University, School of Medicine using qualitative data analysis

Strength of the recommendation		Recommendation
Stronger	Standardisation of process	‘Reviewing of an X-ray could be standardised’.
	Simplifying process	‘Requesting new syringes with the appropriate measuring and needle combination to be available for (insulin) use so the transfer of insulin to an additional syringe (is) eliminated from the delivery process’.
Intermediate	Enhance communication	‘It is essential that there is proper written and verbal sign out of every patient. There should be a component of teach back between the new team taking over care and the team leaving for the evening’.
	Cogitative aid	‘Patient should have a Do Not Resuscitate (DNR) wristband that can be clearly seen’.
	Software enhancement	‘Have a warning/notification in the electronic medical record that prompts a physician completing discharge documentation that there are still blood culture results pending’.
	Checklists	‘Having a checklist to make sure each of the precautions have been addressed’.
	Staffing	‘Ensure that that there (are) an appropriate number of nurses staffed, otherwise nurses will feel pressured to rush their work’.
Weaker	Warning labels	‘Put contact precaution sign on the door’.
	Double checks	‘The order should be checked and double checked. The resident should be more careful when making orders’.
	Training and policy	‘A policy could be implemented that requires all personnel changing or inserting PICC lines to have completed a training session every 6 months… (otherwise) the PICC line (insertion) must be under the direct supervision of someone more experienced’.

PICC, peripherally inserted central catheter.

Student reporting accuracy

There were five student’s reports that did not qualify as a NM or AE. In one report, a student described a patient who presented to the Emergency Department with ‘stroke-like symptoms’. Patient was noted to have an abnormal urinalysis and was not treated for a urinary tract infection. Student identified this as a NM. However, on review, patient was asymptomatic (per student’s description) and thus had asymptomatic bacteriuria and did not merit treatment. We did not categorise this as a NM or AE. Other reports similarly did not describe events that met the definition of AE/NM.

Analysis of the remaining 273 reports showed that 162 of 175 (93%) and 86 of 98 (88%) were correctly categorised by students as NM and AE, respectively; 13 (7%) reports were reclassified from AE to NM and 12 (12%) had to be reclassified from NM to AE. Students correctly identified 31 (67%) diagnostic errors, 100 (81%) treatment errors, 43 (78%) preventative errors and 37 (77%) errors classified as other. Classification and categorisation of errors reported by medical students are shown in table 2.

Table 2

Classification and categorisation of errors reported by 278 medical students for the academic year 2015–2016 at Wayne State University, School of Medicine, using qualitative data analysis

		What students reported (n=278)*	Quality improvement physicians’ determination (n=273)†	Correctly classified	Reclassified
Types of event	Near miss	179	175	162 (93%)	13 (7%)
	Adverse event	99	98	86 (88%)	12 (12%)
Type of error	Diagnosis error	37	46	31 (67%)	15 (33%)
	Treatment errors	114	124	100 (81%)	24 (19%)
	Preventative error	67	55	43 (78%)	12 (22%)
	Other	60	48	37 (77%)	11 (23%)

After analysis, 273 reports qualified as near miss/adverse event

*Reports were not near miss/adverse event.

†Total cases reported by students.

End-of-year survey

The end-of-year survey was completed by 243 (86%) of students; 18% students recalled reporting a formal NM/AE during their clinical rotations (111% increase).

Discussion

Our study demonstrated that nearly all third-year medical students were able to successfully identify NM and AE during their Internal Medicine clerkship. Students observed a variety of errors with treatment errors being the most commonly observed. Students identified contributing factors with medication, miscommunication and cognitive errors being most predominant. The majority of the recommendations to prevent similar errors in the future were weak in nature. Our end-of-year survey indicated that, after participation in the curriculum and error reporting assignment, clinical error reporting by junior medical students more than doubled.

Previous reports suggest students observed medical errors.9 Madigosky et al reported that 76% of students observed a medical error.9 We required our students to report an error, likely making them more aware of NMs along with AEs. This may account for why essentially all our students observed errors.

There is a growing body of literature on educating students about patient safety and error reporting35 but studies on teaching medical students how to report medical errors is still limited. Hall et al delivered a 1-hour curriculum to junior medical students which included a reporting exercise based on a root cause analysis presented.36 They demonstrated increased student comfort in safety event analysis and increased commitment to error reporting; however, no clinical outcomes related to reporting were presented.

Clinical error reporting increased after our educational intervention with nearly one in five students reporting clinical errors. This is higher than what has been reported in the literature.9 Completion of a reporting exercise has been shown to overcome barriers to reporting.10 We believe that having our students report events they personally observed was an important aspect of the assignment. Describing what could have happened and making suggestions to prevent similar events allowed students to take ownership for improving the healthcare system. This will prepare students for their roles as interns and residents ready to contribute to a culture of safety as mandated by the Accreditation Council for Graduate Medical Education and underscored in the Clinical Learning Environment Review Pathways to Excellence.14 15 37

Our intervention can be used at other institutions. It can be adapted to the clinical rotation and site of training. Furthermore, it can be used during any year of medical school where students observe or provide care.

Limitations

Students were required to report a single event. Although they reported more NM than AE, they may have chosen more common events (NM) rather than more serious events (AE) to report. We do not know. Therefore, the proportion of NM to AE described by students may not reflect their actual occurrence or that NM are more likely. Unlike clinical error reports, we did not perform root cause analyses or further evaluate reports. In practice, safety reports are often submitted anonymously (personally communication with local safety officer) and lack detail. Yet, such reports provide valuable information. Although we analysed and categorised errors, we cannot verify the event or type of error. It is possible that our analysis would have changed if we had been able to review charts.

Our end-of-year survey was brief to ensure a high response rate. We did not ask detailed questions regarding the number of reports students submitted, whether the reports submitted were before or after the intervention. Additionally, the survey relies on recall of events and is subject to bias. Individuals exposed to the intervention later in their academic year had less opportunity to submit a formal error reports compared with those who participated earlier in the year. This may have resulted in a lower reporting rate by those students. Nevertheless, our preintervention reporting rate is consistent with previous published reports.

Finally, we cannot definitively demonstrate that the observed increase in reporting was due to our intervention. Our results could be confounded by external factors promoting increased awareness of medical errors and error reporting such as exposure to literature related to medical errors and increasing emphasis on reporting for resident physicians.37 However, the focus of the assignment was to have students be aware of NM/AE and report what they saw using standard reporting tools and have them practice reporting to overcome lack of experience reporting as a barrier to reporting. In this regard, we believe we were successful. Further research is needed to determine if these changes are sustainable.

Next steps

To further encourage clinical reporting, we now ask students to file a formal report on the NM/AE assignment submitted. We provide students with detailed instructions on how to report at each clinical site as this was identified by students as a barrier to reporting. In addition, we are working with our clinical sites to provide an option to identify as a ‘medical student’ when formally reporting an event. This will allow us to determine the number of reports submitted by students rather than relying on student’s recall. Based on additional student feedback, we now also send reminders to students to report events observed during the rotation. Finally, based on our results, we have further refined our curriculum regarding error prevention strategies to educate students about more impactful ways to prevent future errors.

Conclusion

To ensure a culture of patient safety, medical error reporting should begin as early as medical school. Training medical students to report errors can be successfully integrated into a clinical clerkship. Students who are mindful of patient safety can contribute to the culture of safety by reporting and improve healthcare system on a broader scale. We believe that our intervention can serve as a paradigm for other medical schools to adapt and further cultivate the curriculum to ensure better patient safety behaviours among the next generation of young doctors.