Restoration of reaching and grasping movements through brain-controlled muscle stimulation in a person with tetraplegia: a proof-of-concept demonstration.

BACKGROUND: People with chronic tetraplegia, due to high-cervical spinal cord injury, can regain limb movements through coordinated electrical stimulation of peripheral muscles and nerves, known as functional electrical stimulation (FES). Users typically command FES systems through other preserved, but unrelated and limited in number, volitional movements (eg, facial muscle activity, head movements, shoulder shrugs). We report the findings of an individual with traumatic high-cervical spinal cord injury who coordinated reaching and grasping movements using his own paralysed arm and hand, reanimated through implanted FES, and commanded using his own cortical signals through an intracortical brain-computer interface (iBCI).
METHODS: We recruited a participant into the BrainGate2 clinical trial, an ongoing study that obtains safety information regarding an intracortical neural interface device, and investigates the feasibility of people with tetraplegia controlling assistive devices using their cortical signals. Surgical procedures were performed at University Hospitals Cleveland Medical Center (Cleveland, OH, USA). Study procedures and data analyses were performed at Case Western Reserve University (Cleveland, OH, USA) and the US Department of Veterans Affairs, Louis Stokes Cleveland Veterans Affairs Medical Center (Cleveland, OH, USA). The study participant was a 53-year-old man with a spinal cord injury (cervical level 4, American Spinal Injury Association Impairment Scale category A). He received two intracortical microelectrode arrays in the hand area of his motor cortex, and 4 months and 9 months later received a total of 36 implanted percutaneous electrodes in his right upper and lower arm to electrically stimulate his hand, elbow, and shoulder muscles. The participant used a motorised mobile arm support for gravitational assistance and to provide humeral abduction and adduction under cortical control. We assessed the participant's ability to cortically command his paralysed arm to perform simple single-joint arm and hand movements and functionally meaningful multi-joint movements. We compared iBCI control of his paralysed arm with that of a virtual three-dimensional arm. This study is registered with ClinicalTrials.gov, number NCT00912041.
FINDINGS: The intracortical implant occurred on Dec 1, 2014, and we are continuing to study the participant. The last session included in this report was Nov 7, 2016. The point-to-point target acquisition sessions began on Oct 8, 2015 (311 days after implant). The participant successfully cortically commanded single-joint and coordinated multi-joint arm movements for point-to-point target acquisitions (80-100% accuracy), using first a virtual arm and second his own arm animated by FES. Using his paralysed arm, the participant volitionally performed self-paced reaches to drink a mug of coffee (successfully completing 11 of 12 attempts within a single session 463 days after implant) and feed himself (717 days after implant).
INTERPRETATION: To our knowledge, this is the first report of a combined implanted FES+iBCI neuroprosthesis for restoring both reaching and grasping movements to people with chronic tetraplegia due to spinal cord injury, and represents a major advance, with a clear translational path, for clinically viable neuroprostheses for restoration of reaching and grasping after paralysis. FUNDING: National Institutes of Health, Department of Veterans Affairs.

INTRODUCTION

High cervical spinal cord injury (SCI) resulting in tetraplegia prevents affected persons from performing reaching and grasping movements required for many activities-of-daily-living. Functional electrical stimulation (FES), in the absence of descending motor commands, applies spatiotemporal patterns of stimulation to peripheral nerves and muscles to reanimate paralyzed limbs for restoring lost functions. FES can be delivered through skin surface, intramuscular, or nerve cuff electrodes[1-3], and has successfully restored grasping to persons with mid- to low-level cervical SCI, who retain both volitional shoulder and elbow movements for commanding stimulation[4-6].

Restoring multi-joint reaching and grasping is more difficult in persons with high cervical SCI because the few available command options (sip-and-puff, eye tracking, retained head/neck movements) are unintuitive, scale poorly for commanding coordinated multi-joint movements, and interfere with intact head and face function. Intracortical brain-computer-interfaces (iBCIs) that directly map cortical activity to desired movement eschew the need for retained volitional movement, thereby potentially addressing these shortcomings. Intact non-human-primates (NHPs)[7-9] and humans with paralysis[10-13] have successfully used iBCIs to command cursor movements and reaching and grasping using robotic limbs. Temporarily paralyzed NHPs have used iBCIs to command implanted FES-actuated wrist and grasping movements[14,15]. A more recent study used an iBCI coupled with surface electrical stimulation to provide assistive hand grasping to an individual with C5/C6 SCI[16] who retained volitional shoulder and elbow function. However, the 25-year-old Freehand implanted FES system[4-6] already successfully restored hand grasping to persons who retained volitional arm function, without requiring an iBCI. We for the first time show an individual with chronic tetraplegia using an implanted FES system to make both reaching and grasping movements, intuitively and effectively commanded by an iBCI, with a translational path for future clinical viability.

METHODS

Cortical and FES Implications

The participant (ID# T8) enrolled into the BrainGate2 pilot clinical trial (ClinicalTrials.gov NCT00912041) and gave informed consent for study procedures as approved by the Institutional Review Boards of University Hospitals Cleveland Medical Center (Cleveland, OH) and Massachusetts General Hospital (Boston, MA). At the time of implant, he was a 53-year-old man who experienced traumatic high cervical SCI (C4, ASIA A) eight years before enrollment. On his dominant right side (contralateral to the intracortical implant), he retained limited and nonfunctional voluntary shoulder girdle motion, but no voluntary glenohumeral, elbow, or hand function, and no sensation below the shoulder. An implanted baclofen pump controlled spasticity of his dominant arm.

Two 96-channel microelectrode arrays (Blackrock Microsystems, Salt Lake City, Utah)[17] were implanted into the hand area on the precentral gyrus[18] of his motor cortex (Supplementary Figure 1). After four months of using the iBCI to command movements of a 3D virtual arm, the participant received, during two procedures (125 and 280 days-post-implant), 36 percutaneous muscle stimulating electrodes (Synapse Biomedical, Oberlin, OH)[19] in his right upper and lower arm, including four percutaneous anodic current return electrodes, to restore finger and thumb (for a lateral hand grasp[20]), wrist, elbow, and shoulder movements (Supplemental Table 3 lists implanted muscles). Starting 142 days-post-implant, all implanted muscles were exercised using cyclical electrical stimulation patterns to improve strength, range-of-motion, and fatigue resistance. Exercise occurred 18 out of 45 weeks, averaging 8 hours/week spread over 2–3 days. Over the course of the study, the participant had four minor (and no serious) device-related adverse events, all of which were quickly treated, resolved, and appropriately reported to the governing IRBs.

FES+iBCI System Architecture and Neural Decoding

Figure 1A illustrates the FES+iBCI system. The iBCI consists of the implanted recording microelectrode arrays, with a neural decoder that translated recorded cortical activity into command signals for controlling muscle stimulation to produce coordinated reaching and grasping movements. The FES system consists of an external stimulator that delivered charge-balanced, biphasic, constant-current stimulation through percutaneous electrodes to produce muscle contractions and subsequent limb movement. The stimulation had a fixed current amplitude (20 mA) and frequency (12.5 Hz) and a variable pulse duration of 0–200 μs. The current pulse duration (“pulse-width”) applied at a given electrode determined the strength of the muscle contraction. The participant used a Mobile Arm Support (MAS) (Focal MEDITECH, Tilburg, Netherlands) for support against gravity and motorized humeral ab/adduction (also under cortical command), since neither his residual shoulder motion nor deltoid stimulation provided adequate humeral ab/adduction. Instrumented goniometers (Biometrics Ltd.-US, Ladysmith, VA) on his elbow, wrist, and hand measured joint motions.

Figure 1

Overview of the FES+iBCI System. (A) Neural activity is recorded from two microelectrode arrays implanted in the motor cortex. The recorded activity is then decoded into command signals that control the stimulation of biceps, triceps, forearm, and hand muscles, as well as the actuation of a mobile arm support (MAS), to enable cortical control of whole arm movements. Muscle stimulation was performed through percutaneous intramuscular fine-wire electrodes, and instrumented goniometers (Biometrics Ltd.-US, Ladysmith, VA) quantified the resultant wrist, elbow, and hand aperture movements, while an orientation sensor quantified MAS movements. (B) Simulation patterns convert the decoded command signals into the appropriate pulse widths to apply to each individual FES electrode, enabling the participant to coordinate the action of multiple electrodes and muscles using only a single command. Example stimulation patterns for the elbow, wrist, and hand are shown; supplementary Figure 8 illustrates how a stimulation pattern controls the angle of a joint.

Neural decoders were calibrated daily at the beginning of each experimental session to translate cortical activity patterns into movement commands for a virtual reality arm or the FES-actuated arm. Daily re-calibration helped to account for day-to-day variability in the recorded activity[21]. The decoders used two neural features from each electrode of the intracortical arrays: 1) unsorted threshold crossing “firing” rates, determined by counting all action potentials in a 20ms time window that crossed a preset noise threshold, and 2) average spectral high frequency power (250–3000 Hz) in a 20ms time window. The decoders used a linear transformation function, similar to the Kalman filter used in recent iBCI applications[22], to map the neural features to three movement commands. For the virtual arm, decoded commands determined the instantaneous movement velocities for the virtual arm joints (shoulder, elbow, wrist, grasp). For the FES arm, decoded commands determined the change in the percent activation of stimulation patterns associated with elbow, wrist or hand movements (or determined the actuation of the MAS). The stimulation patterns made it easy for the participant to coordinate the activity of multiple electrodes in a graded fashion with only a few command signals[20] (see Supplemental Methods). Figure 1B illustrates example stimulation patterns designed such that increases/decreases in the percent activation of the pattern smoothly coordinated multiple electrodes to cause joint extension/flexion. The decoded command signal for controlling the MAS caused no movement if it was below a certain threshold, and otherwise caused ab/adduction at a constant rate.

Session Overview

Figure 2A illustrates the timeline of surgeries and sessions. As part of the BrainGate2 pilot clinical trial, the study participant performed various virtual reality (VR) arm control sessions not directly related to the present study. For this study, he performed two types of sessions: (1) sessions where he controlled both the VR arm and the FES system in order to compare performance between the two, and (2) sessions where he controlled the FES system to complete meaningful functional tasks. Data reported in this study are from 26 VR vs. FES comparison sessions and functional task sessions collected after the second FES implant surgery.

Figure 2

Session overview. (A) Timeline of implants and experimental sessions. All days are referenced to the day of cortical implant (December 1, 2014 – 0 Days Post Implant). (B) Example image from the virtual reality game. The virtual arm is opaque while the target arm configuration (wrist flexion in this case) is translucent. (C) During the VR vs. FES comparison sessions, the participant completed three different experimental conditions. Block diagrams of each condition and an example session timeline are shown. (D) Example raster plots showing the timing of threshold crossings (top rows) and the average threshold crossing rates (bottom row) of a single channel tuned to wrist flexion and extension during a single-joint wrist movement task. The dotted line at t=0 indicates the presentation of the target movement. This channel records more threshold crossings when flexion targets are presented and has similar tuning properties during all three experimental conditions.

Comparing Cortical Control of a Virtual Reality Arm vs FES-Actuated Arm

During the VR vs. FES comparison sessions, the participant cortically commanded single and multi-joint movements of both a 3D virtual arm and his FES arm to perform point-to-point target acquisitions. Since the virtual arm had no inertial or musculoskeletal dynamics, it was useful for demonstrating the quality of the decoded iBCI commands independently of any control difficulties added by the FES-actuated arm and the stimulation patterns.

Experimental sessions consisted of three conditions (Figure 2C). During the attempted movement condition (AM), the participant observed the virtual arm[23] make goal-directed, point-to-point movements while he was instructed to attempt those same movements. Neural activity evoked during these attempted movements was used for decoder initialization. During the virtual reality condition (VR), the participant used this initial decoder to cortically command the real-time velocities of the elbow, wrist, hand, and/or shoulder joints to perform single and multi-joint movements of the opaque virtual arm to a target configuration represented by a translucent arm (Figure 2B). Decoder parameters were updated after each VR block by re-calibrating the decoder with all available data, and then held fixed for the FES condition. During the FES condition, the participant performed single and multi-joint movements of his own FES-actuated arm while receiving visual feedback of the arm movements and the target location via the VR system. The virtual arm’s joint angles were set equal to the joint angles recorded by the instrumented goniometers and the participant looked at the computer monitor during this phase instead of his own FES-actuated arm. This experimental set up (Supplemental Videos 1–2) enabled precise visualization of the target arm configurations during point-to-point FES arm movements and facilitated comparison between VR and FES movements (since visual feedback of the target location and the arm location was identical). The decoder output for the untested joints was set to zero. Supplementary Table 1 summarizes all sessions and joints tested during each day.

Functional Task Demonstrations

To demonstrate the potential of the system to restore meaningful function, the participant completed both a coffee drinking and a self-feeding task. For these sessions, we calibrated the decoder while he observed and controlled FES-actuated movements instead of virtual reality movements. First, we initialized the decoder using neural data recorded while his arm was automatically driven by the FES system (i.e. computer-controlled) to make elbow, hand, and shoulder (MAS) movements. He was instructed to simultaneously attempt to make the observed arm motions. We then refined the decoder by using neural data recorded while he performed user-controlled single-joint FES-actuated movements, cued by audio commands instead of the VR game. After refinement, the neural decoder was held constant for the functional tasks. We found that this calibration scheme worked better than calibrating with VR data, potentially because the difference in visual feedback between VR and physical reality caused the neural activity to change.

RESULTS

Quality of Recorded Neural Activity

Neural activity was strongly related to the participant’s intended movement commands during attempted movement (AM), VR, and FES movement conditions. Figure 2D illustrates example neural activity that was strongly tuned to wrist flexion over wrist extension during each condition (substantially more threshold crossings were observed during attempted wrist flexion over wrist extension). Similar consistency of neural tuning between task conditions (AM vs VR vs FES) on some channels was also observed for elbow flexion/extension, humeral ab/adduction, and hand opening/closing (Supplementary Figure 3). Of the 192 electrodes, we identified a neural feature (threshold crossing or spectral high frequency power) that coded for hand opening/closing on 15±2 (mean±SD) electrodes, for elbow flexion/extension on 25±2 electrodes, for wrist flexion/extension on 25±4 electrodes, and for humeral ab/adduction on 27±20 electrodes (Supplemental Figure 8). Supplementary Figure 9 reports the number of isolatable single neurons recorded over time.

Quality of Virtual and FES Arm Movements

Across the VR vs. FES comparison sessions, the participant consistently achieved 80–100% success rates during single-joint movements of the elbow, wrist, hand, and MAS (humeral ab/adduction) to specified target positions (Figure 3, Supplementary Video 1). To successfully acquire a target, he had to maintain the specified joint angle within a certain tolerance around the target for 500ms without exceeding a maximum movement time of 8–12 seconds. The starting position of each movement was equal to the ending position of the previous movement.

Figure 3

Single-joint FES and mobile arm support (MAS) movements under real-time brain control. (A) (1st column) Restored arm and hand movements and achievable ranges of motion. Line drawings are made from actual photographs of restored movements, and show complete range of restored motion. (2nd–4th columns) Overlaid time series of joint motions towards each target (columns) during an example block of each movement (rows). Each line illustrates a single movement from the example FES block (blue) or virtual reality block (pink). Gray rectangles illustrate the target and the tolerance allowed for target acquisition. Target distances (from the flexion to extension target) and allowed tolerances (widths) were 43.4°±6.0° (elbow), 24°±3.4° (wrist), 35.8°±5.8° (hand), and 41.3°±5.1° (MAS). The participant was in full control of the joint at all times (the joint position was not reset after a target was acquired). Example blocks with high success rates were chosen for illustration. (B) Success rate and average movement time is summarized for each FES block (circles). Circles are different colors if they occurred on different days. Average virtual reality performance (blue dotted line) and chance performance (red dotted line) are shown for reference. Supplementary Table 4 gives a more detailed quantification with accompanying statistical tests.

For some joint movements (elbow flexion/extension, MAS ab/adduction), the participant acquired targets with his FES-actuated arm as quickly and as successfully as the virtual arm (Figure 3A). For other joint movements (wrist flexion/extension, hand opening/closing), he achieved high success rates but targets were acquired more slowly and speeds varied non-uniformly as a function of joint angle (Figure 3A). Results for two and three-joint movements are shown in Supplementary Figures 4–5 and Supplementary Video 2. On average, he achieved fewer target acquisitions during control of his own arm as compared to control of the virtual arm (Figure 3B). However, FES movements were still far more successful (more target acquisitions) than chance movements for both single and multi-joint movements, and far more successful than what he could achieve with residual voluntary shoulder movements alone (Figure 3B, Supplementary Figure 6, Supplementary Table 4).

Failed reaching attempts were categorized as due to: (1) muscle fatigue making it impossible to reach the target even at full stimulation, (2) control interface (FES and MAS) challenges making it difficult to accurately stop within the target region, or (3) failure to decode the correct command signal to move the joint towards the target. Supplementary Figure 7 illustrates each failure mode. For single-joint movements, ~80% of failed trials were due to control interface challenges, predominantly due to the inability to maintain a desired hand grasp posture (Supplementary Table 2). These trials occurred primarily when the decoded commands for the hand were mapped to highly nonlinear portions of the stimulation pattern. Some portions of the pattern contained “dead space” that did not move the hand very much, while other portions caused large, quick movements, resulting in target overshoots. We alleviated this problem in later blocks by using an automatic procedure that warped the stimulation pattern so that command signals were mapped linearly to equilibrium positions (Supplementary Figure 8). ~15% of failed trials were due to muscle fatigue, while ~5% (only two trials) were due to failure to decode an appropriate command signal.

For multi-joint movements, the dominant failure mode was also control interface challenges (contributed to 67% of failed trials), and was due primarily to MAS movements causing undesired motion of other joints. Interestingly, failure to decode an appropriate command signal for at least one of the joints was more common for multi-joint movements (contributed to 38% of failed trials). Decoding failure may have been more common in multi-joint trials because of the cognitive burden of controlling multiple joints with real dynamics and SCI-musculoskeletal limitations. Muscle fatigue was a reason for failure in ~12% of multi-joint (mostly elbow and wrist) trials.

Performance of Functional Reach-to-Grasp Tasks

The participant was able to successfully acquire a cup of coffee and take a drink (Figure 4A, Supplementary Video 4) and feed himself using the FES+iBCI system (Supplementary Video 5). The coffee drinking task required him to 1) extend his elbow, 2) open his hand, 3) grasp the cup securely, 4) flex his elbow to transport it close to his mouth, 5) take a drink using the straw, 6) extend his elbow to return the cup, and 7) release his grasp. He required between 20–40 seconds to complete the drinking task and was successful in 11 of 12 attempts made during the illustrated session (Figure 4B). During self-feeding, he consistently and repeatedly scooped forkfuls of mashed potatoes, and navigated his hand to his mouth to take several bites. When asked to describe how he commanded the FES arm movements, he replied, “It’s probably a good thing that I’m making it move without having to really concentrate hard at it. … I just think ‘out’ and … it just goes”.

Figure 4

The participant using the FES+iBCI system to take a drink of coffee. (A) He reached out to grasp the cup of coffee (left) and bringing it to his mouth to take a drink through a straw (right). Photos taken on trial day 392 (2015.12.28). (B) The length of time it took to complete each phase of the drinking task. Data is shown for 12 trials completed within a single experimental session; only one trial was failed when the cup was dropped. (C) Example time series of elbow and hand motion when the FES+iBCI system was turned on (left) and when the FES system was turned off (right). When the system was on, the decoded neural commands (blue) and the elbow and hand joint angles (orange) changed appropriately as the participant moved through the phases of the task, enabling him to take a drink of coffee. When the system was off, he could only make small, uncontrolled elbow jerks caused by his residual shoulder motion and could not move his hand at all. Data for panels B and C collected on trial day 463 (2016.03.08).

The participant was completely unable to perform meaningful movements with the FES system turned off (Figure 4C); his minor residual shoulder girdle motion only caused a small, uncontrolled elbow jerk and could not move his hand at all, despite his attempts to command the required arm movements. This indicates that no substantial motor recovery occurred due to the FES and/or iBCI (demonstrated more extensively in Supplementary Figure 6). Supplementary video 3 offers an additional qualitative comparison of the movements the participant could make with and without the FES+iBCI system, and also shows that he could move each joint individually with the FES+iBCI system while suppressing the motion of undesired joints. Finally, note that the participant required continuous visual feedback of his arm movements, as lack of proprioception prevented knowledge of arm position and hence an inability to perform meaningful movements without visual feedback.

DISCUSSION

FES+iBCI restoration of arm and hand functions, combined with a mobile arm support (MAS) (standard for persons with C4 SCI implanted with FES arm systems) under iBCI command, represents a neurotechnology-based circumvention of SCI, giving persons with chronic tetraplegia intuitive control over reaching and grasping movements using their paralyzed limbs. The study significantly extends previous iBCI research with individuals with paralysis controlling cursors or robotic limbs[10-13]. The movements afforded to the study participant (reaching out, grasping/scooping, reaching back to the face) allowed him to repeatedly take a drink of coffee and feed himself with his own arm and hand, solely of his own volition. These actions are representative of movements needed to perform a wide range of reaching tasks, suggesting that more functional activities are achievable with the current system.

FES movements were moderately slower and less accurate than the same movements of the VR arm under brain control (Figure 3, Supplementary Figure 4,5). This discrepancy may have been due to the difference in time spent practicing VR vs FES (65 vs. 15 hours), but may have also been caused by the more difficult control task presented by an FES-activated arm: dynamics due to arm mass, muscle contractile properties, interactions between joints, and MAS motor dynamics. We previously demonstrated that these control difficulties are addressable with a feedback control system[24,25] that converts higher level movement commands decoded from the iBCI (e.g. desired joint velocities) into the lower level muscle stimulations needed to smoothly achieve that movement. The feedback controller incorporates joint angle sensors that continuously sense any movement “error” or deviation from the desired movement, and recruits the appropriate muscles to reduce that error while taking into account the dynamics of the musculoskeletal limb (similar to how robotic arms are controlled). It should be noted that even without an implemented feedback controller, the participant was able to modulate his neural activity and use visual feedback alone to perform meaningful FES arm movements, even on day one.

The percutaneous, readily removable, FES electrodes provide proof-of-concept for fully-implanted FES systems. While this choice limited the number of joints that could be restored and their ranges-of-motion, future fully implantable FES systems can take advantage of enhanced electrode design and surgical placement (e.g., more precisely located intramuscular electrodes implanted via open surgery, or peripheral nerve cuff electrodes for more distributed motor unit recruitment[3]) and associated techniques (e.g., model-based optimization of muscle stimulation patterns, muscle tendon transfers to replace the functions of denervated muscles[6], and more extensive exercise programs) to restore motion more fully. The use of implanted FES is critical for clinical adoption of this technology. While some earlier studies have focused on the use of surface FES to restore only hand grasp to persons with lower-level SCI, either commanded by electroencephalography (EEG)[26,27] or iBCIs[16], surface FES systems do not have a history of wide spread and long term clinical adoption. In contrast, fully implanted systems for FES grasp restoration, specifically the Freehand[1,4], have a history of successful clinical adoption, likely due to the seamlessness of day-to-day setup and use, and their durability (<1% of electrodes fail over 3 years[1]).

While iBCI-commanded systems (including robotic arms) have not yet restored movements with the same speed and precision of able-bodied movements, the current level of gross movement they can restore is still enough to achieve clinically relevant functions (such as self-feeding). Enhanced speed, precision, and multifunctional control may be achievable through electrode technologies that record more neurons from distributed cortical networks, improved decoding algorithms, implantable FES technologies, and restored somatosensation. Somatosensation restored through intracortical stimulation[28] may also eventually allow users to make reach and grasp movements that are safer in the absence of constant visual feedback, due to sensory feedback of object properties (e.g. temperature) in the reachable workspace. Despite current limitations, iBCIs currently offer the best option for seamless clinical integration and greater functional performance, particularly over their non-invasive counterparts (e.g. continuous control of a high-dimensional robotic limb has been successfully demonstrated with iBCI systems[12] but never with EEG). Research advances in intracortical electrode biocompatibility[29] and fully-implanted brain recording interfaces[30] continue to increase the clinical viability of iBCI-commanded systems.

The present FES+iBCI system offers persons with chronic tetraplegia from SCI the possibility of regaining lost arm and hand function to perform activities-of-daily-living. Continued clinical translation of this technology will be aided by iBCI and FES technological advances resulting in smoother and more dexterous arm and hand movements. Future systems inspired by this work may provide full-time and more accurate control of the arm and hand, enabling restoration of a wider range of functional activities and resulting in increased independence and quality-of-life.