Virtual Reality vs. Dry-Lab Models: Comparing automated performance metrics and cognitive workload during robotic simulation training.

Background This study compares surgical performance during analogous vesico-urethral anastomosis (VUA) tasks in two robotic training environments, virtual reality (VR) and dry-lab (DL), in order to investigate transferability of skills assessment across the two platforms. Utilizing computer-generated performance metrics and pupillary data we evaluated the two environments' ability to distinguish surgical expertise and ultimately whether performance in the VR simulation correlates to performance on the live robot in the dry-lab. Materials and Methods Experts (≥ 300 cases) and trainees (<300) performed analogous VUAs during VR and dry-lab sessions on a da Vinci robotic console. 22 metrics were generated in each environment (kinematic metrics, tissue metrics, biometrics). The dry-lab included 18 previously validated automated performance metrics (APMs) (kinematics, events metrics) and were captured by an Intuitive systems data recorder. In both settings, Tobii Pro Glasses 2 recorded task-evoked pupillary response (reported as Index of Cognitive Activity [ICA]) to indicate cognitive workload, analyzed by EyeTracking Cognitive Workload Software. Pearson Correlation, Mann-Whitney and Independent t-tests were used for the comparative analyses. Results Our study included 6 experts (median caseload 1300 [interquartile range 400-3000]) and 11 trainees (25 [0-250]). 8/9 metrics directly comparable between VR and DL showed significant positive correlation (r≥0.554, p≤0.032). 5/22 VR metrics distinguished expertise including: task time (p=0.031), clutch usage (p=0.040), unnecessary needle piercings (p=0.026) and suspected injury to endopelvic fascia (p=0.040). This contrasts with 14/22 APMs in dry-lab (p≤0.038) including: linear velocities of all three instruments (p≤0.038) and dominant-hand instrument wrist articulation (p=0.013). Trainees experienced higher cognitive workload (ICA) in both environments when compared to experts (p<0.036). Conclusions A majority of performance metrics between VR and dry-lab exhibited moderate to strong correlations, showing transferability of skills across the platforms. Comparing training environments, APMs during dry-lab tasks are better able to distinguish expertise than VR-generated metrics.