Andrew J Hung1, Thomas Bottyan2, Thomas G Clifford2, Sarfaraz Serang3, Zein K Nakhoda2, Swar H Shah2, Hana Yokoi2, Monish Aron2, Inderbir S Gill2. 1. Center for Robotic Simulation and Education, USC Institute of Urology, Catherine and Joseph Aresty Department of Urology, Keck School of Medicine, University of Southern California, 1441 Eastlake Avenue Suite 7416, Los Angeles, CA, 90089, USA. Andrew.Hung@med.usc.edu. 2. Center for Robotic Simulation and Education, USC Institute of Urology, Catherine and Joseph Aresty Department of Urology, Keck School of Medicine, University of Southern California, 1441 Eastlake Avenue Suite 7416, Los Angeles, CA, 90089, USA. 3. Department of Psychology, University of Southern California, Los Angeles, CA, USA.
Abstract
PURPOSE: We evaluated feasibility and benefit of implementing structured learning in a robotics program. Furthermore, we assessed validity of a proficiency assessment tool for stepwise graduation. METHODS: Teaching cases included robotic radical prostatectomy and partial nephrectomy. Procedure steps were categorized: basic, intermediate, and advanced. An assessment tool ["proficiency score" (PS)] was developed to evaluate ability to safely and autonomously complete a step. Graduation required a passing PS (PS ≥ 3) on three consecutive attempts. PS and validated global evaluative assessment of robotic skills (GEARS) were evaluated for completed steps. Linear regression was utilized to determine postgraduate year/PS relationship (construct validity). Spearman's rank correlation coefficient measured correlation between PS and GEARS evaluations (concurrent validity). Intraclass correlation (ICC) evaluated PS agreement between evaluator classes. RESULTS: Twenty-one robotic trainees participated within the pilot program, completing a median of 14 (2-69) cases each. Twenty-three study evaluators scored 14 (1-60) cases. Over 4 months, 229/294 (78 %) cases were designated "teaching" cases. Residents completed 91 % of possible evaluations; faculty completed 78 %. Verbal and quantitative feedback received by trainees increased significantly (p = 0.002, p < 0.001, respectively). Average PS increased with PGY (post-graduate year) for basic and intermediate steps (regression slopes: 0.402 (p < 0.0001), 0.323 (p < 0.0001), respectively) (construct validation). Overall, PS correlated highly with GEARS (ρ = 0.81, p < 0.0001) (concurrent validity). ICC was 0.77 (95 % CI 0.61-0.88) for resident evaluations. CONCLUSION: Structured learning can be implemented in an academic robotic program with high levels of trainee and evaluator participation, encouraging both quantitative and verbal feedback. A proficiency assessment tool developed for step-specific proficiency has construct and concurrent validity.
PURPOSE: We evaluated feasibility and benefit of implementing structured learning in a robotics program. Furthermore, we assessed validity of a proficiency assessment tool for stepwise graduation. METHODS: Teaching cases included robotic radical prostatectomy and partial nephrectomy. Procedure steps were categorized: basic, intermediate, and advanced. An assessment tool ["proficiency score" (PS)] was developed to evaluate ability to safely and autonomously complete a step. Graduation required a passing PS (PS ≥ 3) on three consecutive attempts. PS and validated global evaluative assessment of robotic skills (GEARS) were evaluated for completed steps. Linear regression was utilized to determine postgraduate year/PS relationship (construct validity). Spearman's rank correlation coefficient measured correlation between PS and GEARS evaluations (concurrent validity). Intraclass correlation (ICC) evaluated PS agreement between evaluator classes. RESULTS: Twenty-one robotic trainees participated within the pilot program, completing a median of 14 (2-69) cases each. Twenty-three study evaluators scored 14 (1-60) cases. Over 4 months, 229/294 (78 %) cases were designated "teaching" cases. Residents completed 91 % of possible evaluations; faculty completed 78 %. Verbal and quantitative feedback received by trainees increased significantly (p = 0.002, p < 0.001, respectively). Average PS increased with PGY (post-graduate year) for basic and intermediate steps (regression slopes: 0.402 (p < 0.0001), 0.323 (p < 0.0001), respectively) (construct validation). Overall, PS correlated highly with GEARS (ρ = 0.81, p < 0.0001) (concurrent validity). ICC was 0.77 (95 % CI 0.61-0.88) for resident evaluations. CONCLUSION: Structured learning can be implemented in an academic robotic program with high levels of trainee and evaluator participation, encouraging both quantitative and verbal feedback. A proficiency assessment tool developed for step-specific proficiency has construct and concurrent validity.
Keywords:
Proficiency; Robotic curriculum; Robotic training
Authors: Carolyn E Kloek; Sheila Borboli-Gerogiannis; Kenneth Chang; Mark Kuperwaser; Lori R Newman; Anne Marie Lane; John I Loewenstein Journal: J Surg Educ Date: 2013-09-14 Impact factor: 2.891
Authors: Monty A Aghazadeh; Isuru S Jayaratna; Andrew J Hung; Michael M Pan; Mihir M Desai; Inderbir S Gill; Alvin C Goh Journal: Surg Endosc Date: 2015-01-22 Impact factor: 4.584
Authors: Rebecca A Fisher; Prokar Dasgupta; Alex Mottrie; Alessandro Volpe; Mohammed S Khan; Ben Challacombe; Kamran Ahmed Journal: Int J Surg Date: 2014-12-06 Impact factor: 6.071
Authors: Andrew J Hung; Jian Chen; Saum Ghodoussipour; Paul J Oh; Zequn Liu; Jessica Nguyen; Sanjay Purushotham; Inderbir S Gill; Yan Liu Journal: BJU Int Date: 2019-03-20 Impact factor: 5.588
Authors: Glen Denmer Santok; Ali Abdel Raheem; Lawrence Hc Kim; Kidon Chang; Byung Ha Chung; Young Deuk Choi; Koon Ho Rha Journal: Investig Clin Urol Date: 2016-11-28