Ganesh Sankaranarayanan1, Lisa M Parker2, Aimal Khan3, James Dials4, Doga Demirel4, Tansel Halic5, Alyson Crawford6, Uwe Kruger7, Suvranu De7, James W Fleshman2. 1. Department of Surgery, UT Southwestern Medical Center, 5323 Harry Hines Blvd, Dallas, TX, 75390, USA. Ganesh.Sankaranarayanan@UTSouthwestern.edu. 2. Baylor University Medical Center, Dallas, TX, USA. 3. Vanderbilt University Medical Center, Nashville, TN, USA. 4. Florida Polytechnic University, Lakeland, FL, USA. 5. University of Central Arkansas, Conway, AK, USA. 6. Western Surgical Group, Reno, NV, USA. 7. Rensselaer Polytechnic Institute, Troy, NY, USA.
Abstract
BACKGROUND: Assessing performance automatically in a virtual reality trainer or from recorded videos is advantageous but needs validated objective metrics. The purpose of this study is to obtain expert consensus and validate task-specific metrics developed for assessing performance in double-layered end-to-end anastomosis. MATERIALS AND METHODS: Subjects were recruited into expert (PGY 4-5, colorectal surgery residents, and attendings) and novice (PGY 1-3) groups. Weighted average scores of experts for each metric item, completion time, and the total scores computed using global and task-specific metrics were computed for assessment. RESULTS: A total of 43 expert surgeons rated our task-specific metric items with weighted averages ranging from 3.33 to 4.5 on a 5-point Likert scale. A total of 20 subjects (10 novices and 10 experts) participated in validation study. The novice group completed the task significantly more slowly than the experienced group (37.67 ± 7.09 vs 25.47 ± 7.82 min, p = 0.001). In addition, both the global rating scale (23.47 ± 4.28 vs 28.3 ± 3.85, p = 0.016) and the task-specific metrics showed a significant difference in performance between the two groups (38.77 ± 2.83 vs 42.58 ± 4.56 p = 0.027) following partial least-squares (PLS) regression. Furthermore, PLS regression showed that only two metric items (Stay suture tension and Tool handling) could reliably differentiate the performance between the groups (20.41 ± 2.42 vs 24.28 ± 4.09 vs, p = 0.037). CONCLUSIONS: Our study shows that our task-specific metrics have significant discriminant validity and can be used to evaluate the technical skills for this procedure.
BACKGROUND: Assessing performance automatically in a virtual reality trainer or from recorded videos is advantageous but needs validated objective metrics. The purpose of this study is to obtain expert consensus and validate task-specific metrics developed for assessing performance in double-layered end-to-end anastomosis. MATERIALS AND METHODS: Subjects were recruited into expert (PGY 4-5, colorectal surgery residents, and attendings) and novice (PGY 1-3) groups. Weighted average scores of experts for each metric item, completion time, and the total scores computed using global and task-specific metrics were computed for assessment. RESULTS: A total of 43 expert surgeons rated our task-specific metric items with weighted averages ranging from 3.33 to 4.5 on a 5-point Likert scale. A total of 20 subjects (10 novices and 10 experts) participated in validation study. The novice group completed the task significantly more slowly than the experienced group (37.67 ± 7.09 vs 25.47 ± 7.82 min, p = 0.001). In addition, both the global rating scale (23.47 ± 4.28 vs 28.3 ± 3.85, p = 0.016) and the task-specific metrics showed a significant difference in performance between the two groups (38.77 ± 2.83 vs 42.58 ± 4.56 p = 0.027) following partial least-squares (PLS) regression. Furthermore, PLS regression showed that only two metric items (Stay suture tension and Tool handling) could reliably differentiate the performance between the groups (20.41 ± 2.42 vs 24.28 ± 4.09 vs, p = 0.037). CONCLUSIONS: Our study shows that our task-specific metrics have significant discriminant validity and can be used to evaluate the technical skills for this procedure.
Authors: Liane S Feldman; Sarah E Hagarty; Gabriela Ghitulescu; Donna Stanbridge; Gerald M Fried Journal: J Am Coll Surg Date: 2004-01 Impact factor: 6.113
Authors: Sandra L de Montbrun; Patricia L Roberts; Ann C Lowry; Glenn T Ault; Marcus J Burnstein; Peter A Cataldo; Eric J Dozois; Gary D Dunn; James Fleshman; Gerald A Isenberg; Najjia N Mahmoud; Richard K Reznick; Lisa Satterthwaite; David Schoetz; Judith L Trudel; Eric G Weiss; Steven D Wexner; Helen MacRae Journal: Ann Surg Date: 2013-12 Impact factor: 12.969
Authors: John D Birkmeyer; Jonathan F Finks; Amanda O'Reilly; Mary Oerline; Arthur M Carlin; Andre R Nunn; Justin Dimick; Mousumi Banerjee; Nancy J O Birkmeyer Journal: N Engl J Med Date: 2013-10-10 Impact factor: 91.245
Authors: Jad M Abdelsattar; Yazan N AlJamal; Raaj K Ruparel; Phillip G Rowse; Stephanie F Heller; David R Farley Journal: J Surg Educ Date: 2018-11-05 Impact factor: 2.891