Zeev Friedman1, M Dylan Bould2, Clyde Matava3, Fahad Alam4. 1. Department of Anesthesia and Pain Management, Sinai Health System, University of Toronto, 600 University Avenue, Toronto, ON, M5G 1X5, Canada. zeev.friedman@uhn.ca. 2. Children's Hospital of Eastern Ontario, University of Ottawa, Ottawa, ON, Canada. 3. Hospital for Sick Children, University of Toronto, Toronto, ON, Canada. 4. Sunnybrook Health Science Centre, University of Toronto, Toronto, ON, Canada.
Abstract
PURPOSE: A growing body of evidence has shown that supervisors may "fail to fail" trainees even when they have judged their performance unsatisfactory. This has significant implications for the implementation of a nationwide competency-based education model of residency training. The objective of this study was to determine the incidence of "failing to fail" clearly underperforming residents. METHODS:Study participants were recruited via an email invitation sent to all departments of anesthesia at each of the hospitals affiliated with the University of Toronto. They were randomized into a high-stakes (assessment would affect the resident's progress) or low-stakes (assessment would not affect the resident's progress) group and asked to assess the performance (fail or pass grade) of a struggling resident. Participants assessed a video depicting an actor managing a scripted simulation scenario. It contained several critical clinical mistakes constituting a clear fail performance. The purpose of the study was only disclosed following the assessment. RESULTS: Of the 288 email invitations sent (144 in each group), 158 (54%) participants completed the study, with 93 in the high-stakes group and 65 in the low-stakes group. Twenty-eight participants (17.7%) failed to issue a failing grade, including 23.1% (15/65) in the high-stakes group and 13.9% (13/93) in the low-stakes group (P = 0.14). CONCLUSIONS: Though often discussed, this is the first study to quantitatively show that the "failing-to-fail" phenomenon likely occurs during residency training performance evaluations. Passing underperforming learners can potentially affect patient safety and result in severe personal consequences to the learner. The results indicate the need for better performance assessment training for faculty members.
RCT Entities:
PURPOSE: A growing body of evidence has shown that supervisors may "fail to fail" trainees even when they have judged their performance unsatisfactory. This has significant implications for the implementation of a nationwide competency-based education model of residency training. The objective of this study was to determine the incidence of "failing to fail" clearly underperforming residents. METHODS: Study participants were recruited via an email invitation sent to all departments of anesthesia at each of the hospitals affiliated with the University of Toronto. They were randomized into a high-stakes (assessment would affect the resident's progress) or low-stakes (assessment would not affect the resident's progress) group and asked to assess the performance (fail or pass grade) of a struggling resident. Participants assessed a video depicting an actor managing a scripted simulation scenario. It contained several critical clinical mistakes constituting a clear fail performance. The purpose of the study was only disclosed following the assessment. RESULTS: Of the 288 email invitations sent (144 in each group), 158 (54%) participants completed the study, with 93 in the high-stakes group and 65 in the low-stakes group. Twenty-eight participants (17.7%) failed to issue a failing grade, including 23.1% (15/65) in the high-stakes group and 13.9% (13/93) in the low-stakes group (P = 0.14). CONCLUSIONS: Though often discussed, this is the first study to quantitatively show that the "failing-to-fail" phenomenon likely occurs during residency training performance evaluations. Passing underperforming learners can potentially affect patient safety and result in severe personal consequences to the learner. The results indicate the need for better performance assessment training for faculty members.
Entities:
Keywords:
assessment; competence by design; failure-to-fail; residency program; simulation
Authors: Emily M Silver; Laura E Flores; Roshell Muir; Kathleen E Padilla; Manisha Salinas; Gianni R Lorello Journal: Can J Anaesth Date: 2021-05-24 Impact factor: 5.063