Proficiency-based training for robotic surgery: construct validity, workload, and expert levels for nine inanimate exercises.

BACKGROUND: We previously developed nine inanimate training exercises as part of a comprehensive, proficiency-based robotic training curriculum that addressed 23 unique skills identified via task deconstruction of robotic operations. The purpose of this study was to evaluate construct validity, workload, and expert levels for the nine exercises.
METHODS: Expert robotic surgeons (n = 8, fellows and faculty) and novice trainees (n = 4, medical students) each performed three to five consecutive repetitions of nine previously reported exercises (five FLS models with or without modifications and four custom-made models). Each task was scored for time and accuracy using modified FLS metrics; task scores were normalized to a previously established (preliminary) proficiency level and a composite score equaled the sum of the nine normalized task scores. Questionnaires were administered regarding prior experience. After each exercise, participants completed a validated NASA-TLX Workload Scale to rate the mental, physical, temporal, performance, effort, and frustration levels of each task.
RESULTS: Experts had performed 119 (range = 15-600) robotic operations; novices had observed ≤ 1 robotic operation. For all nine tasks and the composite score, experts achieved significantly better performance than novices (932 ± 67 vs. 618 ± 111, respectively; P < 0.001). No significant differences in workload between experts and novices were detected (32.9 ± 3.5 vs. 32.0 ± 9.1, respectively; n.s.). Importantly, frustration ratings were relatively low for both groups (4.0 ± 0.7 vs. 3.8 ± 1.6, n.s.). The mean performance of the eight experts was deemed suitable as a revised proficiency level for each task.
CONCLUSION: Using objective performance metrics, all nine exercises demonstrated construct validity. Workload was similar between experts and novices and frustration was low for both groups. These data suggest that the nine structured exercises are suitable for proficiency-based robotic training.