Xiaoyu Tan1, Yonggu Lee2, Chin-Boon Chng2, Kah-Bin Lim2, Chee-Kong Chui2. 1. Department of Mechanical Engineering, National University of Singapore, Singapore, 117575, Singapore. xiaoyu_tan@u.nus.edu. 2. Department of Mechanical Engineering, National University of Singapore, Singapore, 117575, Singapore.
Abstract
PURPOSE: Flexible needle insertion is an important minimally invasive surgery approach for biopsy and radio-frequency ablation. This approach can minimize intraoperative trauma and improve postoperative recovery. We propose a new path planning framework using multi-goal deep reinforcement learning to overcome the difficulties in uncertain needle-tissue interactions and enhance the robustness of robot-assisted insertion process. METHODS: This framework utilizes a new algorithm called universal distributional Q-learning (UDQL) to learn a stable steering policy and perform risk management by visualizing the learned Q-value distribution. To further improve the robustness, universal value function approximation is leveraged in the training process of UDQL to maximize generalization and connect to diagnosis by adapting fast re-planning and transfer learning. RESULTS: Computer simulation and phantom experimental results show our proposed framework can securely steer flexible needles with high insertion accuracy and robustness. The framework also improves robustness by providing distribution information to clinicians for diagnosis and decision making during surgery. CONCLUSIONS: Compared with previous methods, the proposed framework can perform multi-target needle insertion through single insertion point qunder continuous state space model with higher accuracy and robustness.
PURPOSE: Flexible needle insertion is an important minimally invasive surgery approach for biopsy and radio-frequency ablation. This approach can minimize intraoperative trauma and improve postoperative recovery. We propose a new path planning framework using multi-goal deep reinforcement learning to overcome the difficulties in uncertain needle-tissue interactions and enhance the robustness of robot-assisted insertion process. METHODS: This framework utilizes a new algorithm called universal distributional Q-learning (UDQL) to learn a stable steering policy and perform risk management by visualizing the learned Q-value distribution. To further improve the robustness, universal value function approximation is leveraged in the training process of UDQL to maximize generalization and connect to diagnosis by adapting fast re-planning and transfer learning. RESULTS: Computer simulation and phantom experimental results show our proposed framework can securely steer flexible needles with high insertion accuracy and robustness. The framework also improves robustness by providing distribution information to clinicians for diagnosis and decision making during surgery. CONCLUSIONS: Compared with previous methods, the proposed framework can perform multi-target needle insertion through single insertion point qunder continuous state space model with higher accuracy and robustness.
Entities:
Keywords:
Deep learning; Deep reinforcement learning; Needle steering; Tool–tissue interaction; Uncertainty
Authors: Martijn G Schouten; Janneke Ansems; W Klaas Jan Renema; Dennis Bosboom; Tom W J Scheenen; Jurgen J Fütterer Journal: Med Phys Date: 2010-09 Impact factor: 4.071