Hindsight Experience Replay Improves Reinforcement Learning for Control of a MIMO Musculoskeletal Model of the Human Arm.

Cervical spinal cord injuries often result in tetraplegia, causing decreased patient independence and quality of life. Functional electrical stimulation (FES), when combined with an appropriate controller, can be used to restore motor function by electrically stimulating the neuromuscular system. In previous work, FES controllers for a planar 2-segment, 6-muscle model of the human arm were trained for 15-30 minutes using reinforcement learning and were able to acquire targets that were 2.5-7.5 cm in radius. Here, we explore several enhancements to the reinforcement learning algorithm that allow FES controllers to learn to reach smaller targets in a larger workspace with as few as 0 patient-specific data points.