Chunxia Qin1,2, Xingchen Ran3, Yiqun Wu4, Xiaojun Chen5. 1. School of Biomedical Engineering, Shanghai Jiao Tong University, Shanghai, China. 2. Room 805, School of Mechanical Engineering, Shanghai Jiao Tong University, Dongchuan Road 800, Minhang District, Shanghai, 200240, China. 3. College of Biomedical Engineering and Instrument Science, Zhejiang University, Zhejiang, China. 4. Shanghai Ninth People's Hospital Affiliated to Shanghai Jiao Tong University School of Medicine, Shanghai, China. 5. Room 805, School of Mechanical Engineering, Shanghai Jiao Tong University, Dongchuan Road 800, Minhang District, Shanghai, 200240, China. xiaojunchen@sjtu.edu.cn.
Abstract
PURPOSE: Image-guided surgical navigation system (SNS) has proved to be an increasingly important assistance tool for mini-invasive surgery. However, using standard devices such as keyboard and mouse as human-computer interaction (HCI) is a latent vector of infectious medium, causing risks to patients and surgeons. To solve the human-computer interaction problem, we proposed an optimized structure of LSTM based on a depth camera to recognize gestures and applied it to an in-house oral and maxillofacial surgical navigation system (Qin et al. in Int J Comput Assist Radiol Surg 14(2):281-289, 2019). METHODS: The proposed optimized structure of LSTM named multi-LSTM allows multiple input layers and takes into account the relationships between inputs. To combine the gesture recognition with the SNS, four left-hand signs waving along four directions were designed to correspond to four operations of the mouse, and the motion of right hand was used to control the movement of the cursor. Finally, a phantom study for zygomatic implant placement was conducted to evaluate the feasibility of multi-LSTM as HCI. RESULTS: 3D hand trajectories of both wrist and elbow from 10 participants were collected to train the recognition network. Then tenfold cross-validation was performed for judging signs, and the mean accuracy was 96% ± 3%. In the phantom study, four implants were successfully placed, and the average deviations of planned-placed implants were 1.22 mm and 1.70 mm for the entry and end points, respectively, while the angular deviation ranged from 0.4° to 2.9°. CONCLUSION: The results showed that this non-contact user interface based on multi-LSTM could be used as a promising tool to eliminate the disinfection problem in operation room and alleviate manipulation complexity of surgical navigation system.
PURPOSE: Image-guided surgical navigation system (SNS) has proved to be an increasingly important assistance tool for mini-invasive surgery. However, using standard devices such as keyboard and mouse as human-computer interaction (HCI) is a latent vector of infectious medium, causing risks to patients and surgeons. To solve the human-computer interaction problem, we proposed an optimized structure of LSTM based on a depth camera to recognize gestures and applied it to an in-house oral and maxillofacial surgical navigation system (Qin et al. in Int J Comput Assist Radiol Surg 14(2):281-289, 2019). METHODS: The proposed optimized structure of LSTM named multi-LSTM allows multiple input layers and takes into account the relationships between inputs. To combine the gesture recognition with the SNS, four left-hand signs waving along four directions were designed to correspond to four operations of the mouse, and the motion of right hand was used to control the movement of the cursor. Finally, a phantom study for zygomatic implant placement was conducted to evaluate the feasibility of multi-LSTM as HCI. RESULTS: 3D hand trajectories of both wrist and elbow from 10 participants were collected to train the recognition network. Then tenfold cross-validation was performed for judging signs, and the mean accuracy was 96% ± 3%. In the phantom study, four implants were successfully placed, and the average deviations of planned-placed implants were 1.22 mm and 1.70 mm for the entry and end points, respectively, while the angular deviation ranged from 0.4° to 2.9°. CONCLUSION: The results showed that this non-contact user interface based on multi-LSTM could be used as a promising tool to eliminate the disinfection problem in operation room and alleviate manipulation complexity of surgical navigation system.
Authors: Miguel Ángel Bautista; Antonio Hernández-Vela; Sergio Escalera; Laura Igual; Oriol Pujol; Josep Moya; Verónica Violant; María T Anguera Journal: IEEE Trans Cybern Date: 2016-01 Impact factor: 11.448
Authors: Carlos Aparicio; Carolina Manresa; Karen Francisco; Pedro Claros; Javier Alández; Oscar González-Martín; Tomas Albrektsson Journal: Periodontol 2000 Date: 2014-10 Impact factor: 7.589
Authors: Gerardo Pellegrino; Giuseppe Lizio; Francesco Basile; Luigi Vito Stefanelli; Claudio Marchetti; Pietro Felice Journal: Methods Protoc Date: 2020-11-05
Authors: Francesco Grecchi; Luigi V Stefanelli; Fabrizio Grivetto; Emma Grecchi; Rami Siev; Ziv Mazor; Massimo Del Fabbro; Nicola Pranno; Alessio Franchina; Vittorio Di Lucia; Francesca De Angelis; Funda Goker Journal: Int J Environ Res Public Health Date: 2021-06-07 Impact factor: 3.390