A vision-based surgical tool tracking approach for untethered surgery simulation and training.

This paper presents progress in the development of an untethered surgery simulation and training system. A surgical trainee interacts with the simulation in the most natural way possible-using standard, untethered surgical tools and hands. This natural interface requires innovations in tracking, haptic feedback, and visualization. This article describes the approach used for tool tracking with color cameras. A novel tracking algorithm is used to identify the type of tool, its position, and its orientation. Integration of the vision system with the tissue deformation and visualization modules is also discussed.