Yoon Jae Kim1, Hyung Seok Nam2, Woo Hyung Lee2,3, Han Gil Seo3,4, Ja-Ho Leigh5, Byung-Mo Oh3,4, Moon Suk Bang6,7, Sungwan Kim8,9. 1. Interdisciplinary Program for Bioengineering, Graduate School, Seoul National University, Seoul, 08826, South Korea. 2. Department of Biomedical Engineering, Seoul National University College of Medicine, Seoul, 03080, South Korea. 3. Department of Rehabilitation Medicine, Seoul National University College of Medicine, Seoul, 03080, South Korea. 4. Department of Rehabilitation Medicine, Seoul National University Hospital, Seoul, 03080, South Korea. 5. Department of Rehabilitation Medicine, Incheon St. Mary's Hospital, College of Medicine, The Catholic University of Korea, Incheon, 21431, South Korea. 6. Department of Rehabilitation Medicine, Seoul National University College of Medicine, Seoul, 03080, South Korea. msbang@snu.ac.kr. 7. Department of Rehabilitation Medicine, Seoul National University Hospital, Seoul, 03080, South Korea. msbang@snu.ac.kr. 8. Department of Biomedical Engineering, Seoul National University College of Medicine, Seoul, 03080, South Korea. sungwan@snu.ac.kr. 9. Institute of Medical and Biological Engineering, Medical Research Center, Seoul National University, Seoul, 03080, South Korea. sungwan@snu.ac.kr.
Abstract
BACKGROUND: While spontaneous robotic arm control using motor imagery has been reported, most previous successful cases have used invasive approaches with advantages in spatial resolution. However, still many researchers continue to investigate methods for robotic arm control with noninvasive neural signal. Most of noninvasive control of robotic arm utilizes P300, steady state visually evoked potential, N2pc, and mental tasks differentiation. Even though these approaches demonstrated successful accuracy, they are limited in time efficiency and user intuition, and mostly require visual stimulation. Ultimately, velocity vector construction using electroencephalography activated by motion-related motor imagery can be considered as a substitution. In this study, a vision-aided brain-machine interface training system for robotic arm control is proposed and developed. METHODS: The proposed system uses a Microsoft Kinect to detect and estimates the 3D positions of the possible target objects. The predicted velocity vector for robot arm input is compensated using the artificial potential to follow an intended one among the possible targets. Two participants with cervical spinal cord injury trained with the system to explore its possible effects. RESULTS: In a situation with four possible targets, the proposed system significantly improved the distance error to the intended target compared to the unintended ones (p < 0.0001). Functional magnetic resonance imaging after five sessions of observation-based training with the developed system showed brain activation patterns with tendency of focusing to ipsilateral primary motor and sensory cortex, posterior parietal cortex, and contralateral cerebellum. However, shared control with blending parameter α less than 1 was not successful and success rate for touching an instructed target was less than the chance level (= 50%). CONCLUSIONS: The pilot clinical study utilizing the training system suggested potential beneficial effects in characterizing the brain activation patterns.
BACKGROUND: While spontaneous robotic arm control using motor imagery has been reported, most previous successful cases have used invasive approaches with advantages in spatial resolution. However, still many researchers continue to investigate methods for robotic arm control with noninvasive neural signal. Most of noninvasive control of robotic arm utilizes P300, steady state visually evoked potential, N2pc, and mental tasks differentiation. Even though these approaches demonstrated successful accuracy, they are limited in time efficiency and user intuition, and mostly require visual stimulation. Ultimately, velocity vector construction using electroencephalography activated by motion-related motor imagery can be considered as a substitution. In this study, a vision-aided brain-machine interface training system for robotic arm control is proposed and developed. METHODS: The proposed system uses a Microsoft Kinect to detect and estimates the 3D positions of the possible target objects. The predicted velocity vector for robot arm input is compensated using the artificial potential to follow an intended one among the possible targets. Two participants with cervical spinal cord injury trained with the system to explore its possible effects. RESULTS: In a situation with four possible targets, the proposed system significantly improved the distance error to the intended target compared to the unintended ones (p < 0.0001). Functional magnetic resonance imaging after five sessions of observation-based training with the developed system showed brain activation patterns with tendency of focusing to ipsilateral primary motor and sensory cortex, posterior parietal cortex, and contralateral cerebellum. However, shared control with blending parameter α less than 1 was not successful and success rate for touching an instructed target was less than the chance level (= 50%). CONCLUSIONS: The pilot clinical study utilizing the training system suggested potential beneficial effects in characterizing the brain activation patterns.
Authors: Andrew Y Paek; Justin A Brantley; Akshay Sujatha Ravindran; Kevin Nathan; Yongtian He; David Eguren; Jesus G Cruz-Garza; Sho Nakagome; Dilranjan S Wickramasuriya; Jiajun Chang; Md Rashed-Al-Mahfuz; Md Rafiul Amin; Nikunj A Bhagat; Jose L Contreras-Vidal Journal: IEEE Open J Eng Med Biol Date: 2021-02-12