Laura K Newcomb1, Megan S Bradley2, Tracy Truong3, Michelle Tang2, Bryan Comstock4, Yi-Ju Li3, Anthony G Visco2, Nazema Y Siddiqui2. 1. Department of Obstetrics and Gynecology, Duke University Medical Center, Durham, North Carolina. Electronic address: laura.newcomb@dm.duke.edu. 2. Department of Obstetrics and Gynecology, Duke University Medical Center, Durham, North Carolina. 3. Department of Biostatistics and Bioinformatics, Duke University School of Medicine, Durham, North Carolina. 4. Department of Biostatistics, University of Washington, Seattle, Washington.
Abstract
STUDY OBJECTIVE: To examine whether a set of virtual reality (VR) surgical simulation drills have correlative validity when compared with the validated Robotic Objective Structured Assessment of Technical Skills (R-OSATS) dry lab drills. DESIGN: A prospective methods comparison study (Canadian Task Force classification II-2). SETTING: A teaching hospital. PARTICIPANTS: Thirty current residents, fellows, and faculty from the Departments of Obstetrics and Gynecology, Urology, and General Surgery. INTERVENTIONS: Participants completed 5 VR drills on the da Vinci Skills Simulator and 5 dry lab drills. Participants were randomized to the order of completion. MEASUREMENTS AND MAIN RESULTS: VR drills were scored automatically by the simulator. Dry lab drills were recorded, reviewed by 3 blinded experts, and scored using the R-OSATS assessment tool. Spearman correlation coefficients were calculated comparing simulator scores and R-OSATS scores for the same surgeon. The correlation for overall summary scores between VR and dry lab drills was strong (r = 0.83; p < .01). Each of the 5 VR drills was also found to have a statistically significant correlation to its corresponding dry lab drill, with correlation coefficients ranging from r = 0.49 to 0.73 (p < .01 for all). The performance on VR drills also confirmed construct validity. Faculty and fellows had consistently higher overall scores than residents (median VR scores: 458 for faculty, 425 for fellows, 339 for residents; p < .01). CONCLUSION: We selected a core set of VR drills that reliably correlate with validated dry lab R-OSATS drills. Because dry lab drills require significant time and effort on the part of the trainees and the evaluators, this set of VR drills could serve as an ancillary method of determining trainee competence.
STUDY OBJECTIVE: To examine whether a set of virtual reality (VR) surgical simulation drills have correlative validity when compared with the validated Robotic Objective Structured Assessment of Technical Skills (R-OSATS) dry lab drills. DESIGN: A prospective methods comparison study (Canadian Task Force classification II-2). SETTING: A teaching hospital. PARTICIPANTS: Thirty current residents, fellows, and faculty from the Departments of Obstetrics and Gynecology, Urology, and General Surgery. INTERVENTIONS:Participants completed 5 VR drills on the da Vinci Skills Simulator and 5 dry lab drills. Participants were randomized to the order of completion. MEASUREMENTS AND MAIN RESULTS: VR drills were scored automatically by the simulator. Dry lab drills were recorded, reviewed by 3 blinded experts, and scored using the R-OSATS assessment tool. Spearman correlation coefficients were calculated comparing simulator scores and R-OSATS scores for the same surgeon. The correlation for overall summary scores between VR and dry lab drills was strong (r = 0.83; p < .01). Each of the 5 VR drills was also found to have a statistically significant correlation to its corresponding dry lab drill, with correlation coefficients ranging from r = 0.49 to 0.73 (p < .01 for all). The performance on VR drills also confirmed construct validity. Faculty and fellows had consistently higher overall scores than residents (median VR scores: 458 for faculty, 425 for fellows, 339 for residents; p < .01). CONCLUSION: We selected a core set of VR drills that reliably correlate with validated dry lab R-OSATS drills. Because dry lab drills require significant time and effort on the part of the trainees and the evaluators, this set of VR drills could serve as an ancillary method of determining trainee competence.
Authors: William B Lyman; Michael J Passeri; Keith Murphy; Imran A Siddiqui; Adeel S Khan; David A Iannitti; John B Martinie; Erin H Baker; Dionisios Vrochides Journal: Surg Endosc Date: 2020-06-16 Impact factor: 4.584