Anthony G Gallagher1, Patrick J Henn2, Paul C Neary3, Anthony J Senagore4, Peter W Marcello5, Brendan P Bunting6, Neal E Seymour7, Richard M Satava8. 1. ASSERT Centre, School of Medicine, University College Cork, Cork, Ireland. 2. School of Medicine, University College Cork, Cork, Ireland. 3. Department of Surgery, Tallaght Hospital, Trinity College, University of Dublin, Dublin, Ireland. 4. Department of Surgery, The University of Texas Medical Branch, Galveston, Texas, USA. 5. Department of Surgery, Lahey Clinic, Burlington, Massachusetts, USA. 6. Department of Psychology, University of Ulster, Derry, UK. 7. Department of Surgery, Tufts University School of Medicine, Springfield, Massachusetts, USA. 8. Department of Surgery, University of Washington Medical Center, Seattle, Washington, USA.
Abstract
BACKGROUND: Training in medicine must move to an outcome-based approach. A proficiency-based progression outcome approach to training relies on a quantitative estimation of experienced operator performance. We aimed to develop a method for dealing with atypical expert performances in the quantitative definition of surgical proficiency. METHODS: In study one, 100 experienced laparoscopic surgeons' performances on virtual reality and box-trainer simulators were assessed for two similar laparoscopic tasks. In study two, 15 experienced surgeons and 16 trainee colorectal surgeons performed one simulated hand-assisted laparoscopic colorectal procedure. Performance scores of experienced surgeons in both studies were standardized (i.e. Z-scores) using the mean and standard deviations (SDs). Performances >1.96 SDs from the mean were excluded in proficiency definitions. RESULTS: In study one, 1-5% of surgeons' performances were excluded having performed significantly below their colleagues. Excluded surgeons made significantly fewer correct incisions (mean = 7 (SD = 2) versus 19.42 (SD = 4.6), P < 0.0001) and a greater proportion of incorrect incisions (mean = 45.71 (SD = 10.48) versus 5.25 (SD = 6.6), P < 0.0001). In study two, one experienced colorectal surgeon performance was >4 SDs for time to complete the procedure and >6 SDs for path length. After their exclusions, experienced surgeons' performances were significantly better than trainees for path length: P = 0.031 and for time: P = 0.002. CONCLUSION: Objectively assessed atypical expert performances were few. Z-score standardization identified them and produced a more robust quantitative definition of proficiency.
BACKGROUND: Training in medicine must move to an outcome-based approach. A proficiency-based progression outcome approach to training relies on a quantitative estimation of experienced operator performance. We aimed to develop a method for dealing with atypical expert performances in the quantitative definition of surgical proficiency. METHODS: In study one, 100 experienced laparoscopic surgeons' performances on virtual reality and box-trainer simulators were assessed for two similar laparoscopic tasks. In study two, 15 experienced surgeons and 16 trainee colorectal surgeons performed one simulated hand-assisted laparoscopic colorectal procedure. Performance scores of experienced surgeons in both studies were standardized (i.e. Z-scores) using the mean and standard deviations (SDs). Performances >1.96 SDs from the mean were excluded in proficiency definitions. RESULTS: In study one, 1-5% of surgeons' performances were excluded having performed significantly below their colleagues. Excluded surgeons made significantly fewer correct incisions (mean = 7 (SD = 2) versus 19.42 (SD = 4.6), P < 0.0001) and a greater proportion of incorrect incisions (mean = 45.71 (SD = 10.48) versus 5.25 (SD = 6.6), P < 0.0001). In study two, one experienced colorectal surgeon performance was >4 SDs for time to complete the procedure and >6 SDs for path length. After their exclusions, experienced surgeons' performances were significantly better than trainees for path length: P = 0.031 and for time: P = 0.002. CONCLUSION: Objectively assessed atypical expert performances were few. Z-score standardization identified them and produced a more robust quantitative definition of proficiency.
Authors: Anthony G Gallagher; Martin Hart; David Cleary; Craig Hamilton; Kevin McGlinchey; Patrick Kiely; Brendan P Bunting Journal: PLoS One Date: 2020-05-12 Impact factor: 3.240
Authors: Alexandre Mottrie; Elio Mazzone; Peter Wiklund; Markus Graefen; Justin W Collins; Ruben De Groote; Paolo Dell'Oglio; Stefano Puliatti; Anthony G Gallagher Journal: BJU Int Date: 2020-12-20 Impact factor: 5.588