Extracting Attempted Hand Movements from EEGs in People with Complete Hand Paralysis Following Stroke.

This study examines the feasibility of using electroencephalograms (EEGs) to rapidly detect the intent to open one's hand in individuals with complete hand paralysis following a subcortical ischemic stroke. If detectable, this motor-planning activity could be used in real time to trigger a motorized hand exoskeleton or an electrical stimulation device that opens/closes the hand. While EEG-triggered movement-assist devices could restore function, they may also promote recovery by reinforcing the use of remaining cortical circuits. EEGs were recorded while participants were cued to either relax or attempt to extend their fingers. Linear-discriminant analysis was used to detect onset of finger-extension from the EEGs in a leave-one-trial-out cross-validation process. In each testing trial, the classifier was applied in pseudo-real-time starting from an initial hand-relaxed phase, through movement planning, and into the initial attempted-finger-extension phase (finger-extension phase estimated from typical time-to-movement-onset measured in the unaffected hand). The classifiers detected attempted-finger-extension at a significantly higher rate during both motor-planning and early attempted execution compared to rest. To reduce inappropriate triggering of a movement-assist device during rest, the classification threshold could be adjusted to require more certainty about one's intent to move before triggering a device. Additionally, a device could be set to activate only after multiple time samples in a row were classified as finger-extension events. These options resulted in some sessions with no false triggers while the person was resting, but moderate-to-high true trigger rates during attempted-movements.

Introduction

Each year, approximately 800,000 Americans experience a new or recurrent stroke. The middle cerebral artery stroke is the most common non-hemorrhagic stroke, which preferentially affects the upper limb (Dobkin, 2004). In spite of available rehabilitation therapies, many individuals are left with chronic hand paralysis even after the rest of the limb has regained significant function.

Neuromuscular stimulation systems and motorized exoskeletons are being developed that could restore function to individuals with chronic hand paralysis. However, these movement-assist devices need some command signal that tells the device what hand action the person is trying to make. If attempted hand movements could be detected directly from the brain, then a device could be designed to open/close the paralyzed hand based on the cortical signals produced naturally when the person attempts to move.

Using brain activity to trigger a movement-assist device may have some additional therapeutic benefits beyond simply restoring function. Repetitive, voluntary movements are thought to promote motor recovery through use-dependent plasticity. Using brain activity to control a movement-assist device would provide practice in generating motor-plans in the cortex. It also would re-link motor-planning activity in the cortex with execution of the motor-plan in the periphery potentially strengthening sensorimotor connections throughout the central and peripheral pathways.

Activity-dependent motor relearning is thought to occur through Hebbian mechanisms, which are strongly affected by the intensity and relative timing of activity in the pre- and post-synaptic neurons. Studies have shown that two key aspects of timing effect synaptic plasticity. First, “causal” timing (where the pre-synaptic neuron fires before the post-synaptic neuron) results in an increase in the strength of a synapse whereas “anti-causal” timing (post-fires before pre) reduces the strength of the synapse (Kandel, 2000; Bi and Poo, 2001). Triggering a movement-assist device with attempted-movements extracted from electroencephalograms (EEGs) would re-establish a causal link between cortical and peripheral neural activation.

Second, in Hebbian plasticity, the increase or decrease in synaptic strength is highly dependent on how closely in time the pre- and post-synaptic neurons fire. The closer in time the two neurons fire, the greater the effect this firing has on changing the strength of the synapse (Kandel, 2000; Bi and Poo, 2001). However, there are often delays when using an assistive device to generate movement. Extra processing time is needed for the device to interpret what the person wants to do and then implement the functions that will eventually generate the desired movement. Minimizing delays between motor-planning in the cortex and execution of that plan by a movement-assist device may be important for strengthening the relevant neural connections. In able-bodied individuals, characteristic changes in brain activity are seen prior to movement-onset throughout the parietal, supplementary motor, premotor, and primary motor cortices as the motor-plan is formed and then implemented (Pfurtscheller et al., 1997; Pfurtscheller and Silva, 1999). If the intended movement can be detected from these early phases of motor-plan formation, delays in activating an assistive device will be minimized thus potentially increasing therapeutic benefit.

This study evaluates the feasibility of using scalp EEGs to detect attempts to open the hand in people with complete chronic hand paralysis following stroke. While studies have already shown hand movement information can be gleaned from EEGs in individuals with partial hand paralysis (our own study; manuscript under review) or from the magnetoencephalograms (MEGs; Buch et al., 2008) and EEGs (Daly et al., 2009) of individuals with complete finger-extensor paralysis following stroke, this study specifically examines the feasibility of detecting attempted-finger-extension from the motor-planning phase prior to when movement would have normally started. For individuals with complete hand paralysis, being able to detect attempted-movement at any time point could be useful since other options for triggering a movement-assist device are limited. However, we chose to optimize our decoding methods in this study for early detection because of the potential therapeutic importance of minimizing the delay between motor-plan formation in the cortex and execution of that plan in the periphery by a movement-assist device.

Materials and Methods

Experimental design

Four individuals were enrolled in this study after having a subcortical ischemic stroke one or more years prior. All participants had chronic paralysis in one hand leaving them unable to generate any voluntary finger-extension. Table 1 indicates the time since the stroke, the hemisphere in which the stroke occurred, the number of testing sessions in which the person participated, and the scores of the wrist and finger portion of the Fugl-Meyer Motor Assessment, which assesses isolated and synergistic voluntary motor function (Gladstone et al., 2002). All participants were right-hand dominant prior to their stroke. Participants came in approximately once a week for up to 4 weeks to repeat the testing sessions. The study protocol was approved by MetroHealth Medical Center's Institutional Review Board. Informed consent was obtained from all participants prior to the study.

Table 1

Study participant demographics.

			Fugl-Meyer score
Subject no.	Time post-stroke	Hemisphere	Wrist	Hand	No. of sessions
1	2 years, 9 months	Right	4	4	4
2	7 years, 2 months	Right	0	1	2
3	1 year, 11 months	Left	1	2	4
4	1 year, 9 months	Right	1	1	4

Participants were seated in front of a computer screen with both forearms resting comfortably on a padded table. Visual cues to open or relax either the affected or unaffected hand were presented on the computer screen. Participants were instructed to attempt to extend the fingers of the affected hand throughout the hand-open cues even though they were unable to generate any physical motion. Each trial consisted of relaxing both hands for 5 s followed by a cue to continuously hold either their right or left hand-open for 5 s. Cues to open/relax the affected or the unaffected hand were presented in random order in blocks of up to 20 movement trials followed by a break.

The analysis reported here focused only on detecting the transition from the relaxed to the attempted-movement state. Complementary analysis is also needed for detecting the transition back from the attempted-movement to the relaxed state. However, those data are not reported here due to space limitations and due to complications in the data resulting from some participants frequently relaxing before they received the relax cue.

Data collection for this study was combined with data collection for a separate study evaluating the effect of different types of visual cues in stroke therapy. Therefore, the visual cue to extend or relax the fingers took two different forms – a simple static cue or one that required visual tracking (visual cue study is still ongoing; results are not reported here). Data from each type of cue was analyzed separately to ensure that potential differences in reaction times between cue types did not confound classifier results. Therefore, each testing session contributed two sets of outcome measures to the summary results reported below.

Data collection

Thirty-two gold cup EEG electrodes spanning the sensorimotor areas were attached to the scalp with conductive paste (locations approximately corresponding to a rectangular grid spanning F3 through CP4 in the 10-5 system; Oostenveld and Praamstra, 2001). EEGs were recorded with a Pentusa RX5 system (Tucker Davis Technologies, FL, USA). These data were collected at 610 Hz and band-pass filtered to 2–150 Hz with an additional notch filter at 60 Hz. Data were down sampled to 305 Hz for further processing. Power was calculated via fast Fourier transforms (FFT) every 100 ms using overlapping windows of data (window length was varied as part of the optimization process described below). Bend sensors on the index finger and/or electromyographic (EMG) sensors over extensor muscles on the unaffected limb where also collected and used to determine a “normal” time-to-movement-onset for each person.

Data analysis

Classifiers for early detection of attempted-finger-extension were developed in two phases. The goal of the first phase was to identify combinations of specific EEG features that modulate the most with finger-extension. The goal of the second phase was to maximize early detection of movement-onset during motor-planning using the best set of EEG features identified in phase one. These two phases used two distinct epochs of data as described below and illustrated in Figure 1.

Figure 1

Spectrogram from one channel of EEG showing epochs used in the different stages of analysis (red = higher power; blue = lower power). Triangles indicate initial presentation of the hand-open cue and an estimated movement-onset time (median time-to-movement-onset in the unaffected hand). The two gray boxes spanning the spectrogram indicate the two 1-s time segments used in Phase-I of the analysis [i.e., the relaxed (0) and attempted-finger-extension (1) epochs]. Analysis in Phase-II emphasized early detection of attempted-finger-extension during the “movement-preparation” epoch. The lower part of the figure shows how the assigned rest/attempted-finger-extension transition point (0-to-1) was systematically shifted across the movement-preparation epoch in Phase-II as part of the classifier optimization process.

Phase-I analysis: EEG feature identification

Phase-I analysis used data that were in either the relaxed epoch or the assumed attempted-finger-extension epoch to first identify which EEG features were significantly modulated (see Figure 1). EEG features were first evaluated individually using linear regression in a 10 × 10-fold cross-validation process. Features were identified that had significant modulation with the finger-extension vs. rest epochs at the 95% confidence level. EEG features tested included power bands in the 2-to-29-Hz range calculated from common-average referenced EEGs as well as from other linear combinations of the EEG channels determined by common spatial pattern analysis (Wang et al., 2004). To ensure EEG modulation was not due to inadvertent scalp EMGs, we verified that all EEG power data used for movement prediction had the expected decrease in power with attempted-finger-extension and increase in power with relaxation (and not the other way around).

After this initial screening, backward feature elimination (Guyon et al., 2002; Schroder et al., 2005) with linear regression and five-fold cross-validation was used to identify a subset of significant features that together convey the most robust information about the relaxed vs. attempted-finger-extension epochs.

Phase-II analysis: optimizing early detection

In phase-II, three key signal processing parameters were optimized using the best combinations of features identified in Phase-I. These key parameters were:

the window size used to calculate the FFT: 100, 300, 500, 700, and 900 ms tested,

the number of previously calculated power samples (updated every 100 ms) used to predict present hand state: 0 (current power calculations only), 2, 4, or 6 preceding power samples included,

the presumed transition point between the relaxed (0) and attempted-finger-extension state (1) used for training the classifier (see bottom of Figure 1 showing state labels transitioning from 0 to 1 at all possible time points in the motor-planning epoch).

Although data used to train the classifiers extended into the attempted-finger-extension epoch, classifiers were optimized, and assessed based on how well they detected attempted-movement strictly during the motor-planning epoch. The motor-planning epoch spanned the first time sample after initial cue presentation (to allow 100 ms for some minimum visual processing time) to the time sample before the typical movement-onset time estimated from the unaffected hand.

Performance assessment

Performance was assessed in a pseudo-real-time manner for each trial during a leave-one-trial-out cross-validation process. Performance was measured in terms of the false-positive rates (calculated as percentage of rest samples classified as finger-extensions) and true-positive rates (calculated as the percentage of motor-planning samples classified as finger-extensions). A one-sided Wilcoxon signed rank sum test was performed using true- and false-positive rates calculated from each trial to determine if the true-positive rates were significantly higher than the false-positive rates.

Controlling false-positive rates

Standard linear-discriminant analysis sets the threshold for classification in a way that maximizes the number of correctly classified time points. However, the default “optimal” classifier threshold may result in a higher false-positive rate than is appropriate for some device applications. Therefore, the classifier thresholds were adjusted for each session to produce controlled false-positive rates of 1, 5, 10, or 20% false-positive samples during the rest epochs prior to cue presentation. The corresponding true-positive rates were calculated at each new classification threshold.

Extending the analysis time window

In our previous study with stroke survivors who had some limited remaining hand function, movements were measured in both the affected and unaffected hand. The median time-to-movement-onset was up to 450 ms longer for the affected hand than the unaffected hand (manuscript under review). With the more severely paralyzed individuals in this study, the time needed to form and initiate a motor-plan will also likely take longer when attempting to move the paralyzed hand vs. when moving the unaffected hand. Since it is likely that the actual movement planning epoch took longer than the duration calculated from the unaffected hand, the above classifier optimization, and performance assessments were repeated with the estimated movement-onset time extended by an additional 0, 200, 400, and 600 ms. By expanding the assumed motor-planning epochs by 200, 400, or 600 ms, it is probable that some performance assessments included data samples from what would have been after the start of motor-execution had the person still been able to move. However, a clearly defined boundary between the motor-planning and the motor-execution phase of an attempted-movement cannot be accurately identified in this population because the movements cannot actually be measured. Therefore, the analysis done with no extension to the pre-movement analysis window serves as a control showing prediction accuracy prior to movement-onset without assumptions of longer motor-planning times for the affected hand.

Improving accuracy with consecutive-event triggers

One potential way to reduce unintended activation of a movement-assist device is to only trigger a device after a fixed number of consecutive data samples are classified as being in the finger-extension state. To evaluate this option, a second analysis was performed where the previously optimized classifier output was reanalyzed requiring between two and five consecutive-time samples to be classified as finger-extension in order to trigger a hypothetical movement-assist device.

While the performance measures in the previous analysis consisted of true- or false-positive “rates” (i.e., the percentage of time samples during movement planning vs. rest classified as a finger-extension state), a rate or “per-time-sample” performance measure is no longer appropriate when multiple consecutive finger-extension classifications are required to activate a single trigger. Therefore, the device-trigger-event resulting from multiple consecutive-classifications was considered an all-or-nothing event that either happened or did not happen during a given epoch within each trial. Only one trigger per epoch per trial was needed to deem that epoch as having a false trigger (rest epoch) or a true trigger (motor-planning epoch) in that trial.

The performance measures for this test were the fraction of trials with a true trigger in the motor-planning epoch compared with the fraction of trials with a false trigger in the rest epoch. For fair comparison of whether or not an all-or-nothing trigger event occurred in either phase of each trial, the length of the time window during rest over which a false trigger could occur was matched to the length of the time window in the motor-planning epoch in which a true trigger could occur. If the number of consecutive-classification events required to activate a hypothetical device exceeded the duration of the motor-planning phase in a given session, no true or false triggers were assigned.

Results

The specific frequency bands that were modulated with attempted-movements spanned the typical alpha and beta ranges in each person. However, unlike typical able-bodied EEG patterns, useful EEG features came from electrode locations on both hemispheres in approximately equal proportions. In almost 70% of the cases, optimization of the FFT window length resulted in 900 ms FFT calculation windows being used for classification. These longer overlapping FFT windows had the effect of both smoothing the power features over time and maintaining a high resolution in the frequency domain.

Figure 2A shows the adjusted true-positive rate (true-positive rate minus false-positive rate) at the default threshold and when the threshold was adjusted to achieve the different predefined false-positive rates. By design, the default threshold setting determined by linear-discriminant analysis maximizes the difference between the two classification states. However, the default threshold generally resulted in relatively high false-positive rates (22 ± 9%). Adjusting the classification thresholds to achieve a lower false-positive rate typically reduced the difference between the true- and false-positive rates. While each participant had a significant difference between their true- and the false-positive rate each day when the default threshold was used (P-values ranged from 8 × 10−6 to 4 × 10−13), shifting the classifier threshold to achieve the lower false-positive rates in some cases reduced the difference between the true- and false-positive rates to a non-significant level. Note in Figure 2, false-positive rates have already been subtracted out making the chance level zero in all cases. The increase in adjusted true-positive rates at higher false-positive thresholds indicating that allowing some false-positives can pay off by disproportionately increasing the adjusted true-positive rate.

Figure 2

Adjusted true-positive rate as a function of false-positive rate. Adjusted true-positive rate was calculated as the true-positive rates during the motor-planning epoch minus the false-positive rate during the relaxed epoch. Boxes indicate 25, 50, and 75% quartiles respectively. (A) Duration of the motor-planning epoch was estimated using the median time-to-movement-onset of the unaffected hand. (B) Estimate of the duration of the motor-planning epoch was expanded by 200, 400, or 600 ms (results for optimal expansion plotted).

More accurate classification could be achieved by expanding the duration of the estimated motor-planning window when both training and evaluating the classifiers. In Figure 2B the presumed time-to-movement-onset was extended by 0, 200, 400, or 600 ms depending on which resulted in the best adjusted true-positive rate for each individual session. Figure 3 shows how extending the presumed duration of the motor-planning phase on average increased the difference between the true- and false-positive rates even though a handful of individual sessions were optimized at time extensions less than 600 ms.

Figure 3

Mean adjusted true-positive rate as a function of the assumed duration of the motor-planning phase used for training/assessing the classifiers. Different lines indicate results when different threshold settings are used to generate the predefined false-positive rates shown. The false-positive rates have already been subtracted out from the actual true-positive rates to get the adjusted true-positive rates plotted here.

Figure 4 shows the effects of applying an additional requirement of detecting multiple consecutive finger-extension classification events before triggering a hypothetical movement-assist device. The X coordinate indicates the fraction of trials that had a false trigger during the relaxed epoch. The Y coordinate indicates the fraction of trials that had a true trigger during the motor-planning epoch. Color indicates the number of consecutive-events needed to trigger a device (2 = blue, 3 = green, 4 = red, and 5 = pink). Each data point within each color represents performance in a different session. Perfect performance would be in the upper left corner of each plot. Nearly all sessions and testing conditions had more trials with true triggers than false triggers.

Figure 4

Fraction of trials with a false trigger in the rest epoch (. a true trigger in the motor-planning epoch (. Original classification thresholds were set to achieve the false-positive rates indicated at the bottom of the figure. In the top row, the duration of the motor-planning phase was calculated using the median time-to-movement-onset calculated from the unaffected hand (no extension). In the bottom row, the assumed motor-planning phase was extended by 600 ms. Solid diagonal line in each plot indicates the ratio of the mean true-and false-positive rates calculated without regard to consecutive triggers.

The solid diagonal line in each plot in Figure 4 shows the original ratio of the mean true- vs. false-positive rates for each condition when consecutive triggers were not required (ratio calculated from same rate data used to make Figure 2; ratio scaled to match the different values along the axes). Dots above and to the left of the solid line indicate that requiring consecutive triggers resulted in an improvement in the overall ratio of true- vs. false-positive events.

Discussion

This study showed that attempted-finger-extension could be extracted from the EEGs of individuals with complete hand paralysis one or more years after a subcortical ischemic stroke. These decoded EEGs could potentially be used to trigger a movement-assist device to restore functional use of the hand. More importantly, attempts to open the hand could be detected during the motor-planning phase of movement before a movement would have typically started. This early detection could be used to rapidly trigger a movement-assist device, potentially restoring close-to-normal timing between motor-plan formation in the cortex and initiation of that plan in the periphery. Restoring tight causal timing throughout the central and peripheral sensorimotor pathways may promote Hebbian plasticity and help rebuild more-normal sensorimotor patterns.

At the default classifier threshold, significant prediction of attempted-finger-extension was seen prior to the assumed movement-onset time, which was initially estimated using the median movement-onset time of the unaffected hand. As expected, the likelihood of triggering a movement-assist device increased as the assumed movement-onset time was extended by 200, 400, or 600 ms to account for likely longer motor-planning times in the stroke-damaged cortex. In people with no ability to generate physical hand movement, the exact transition point between motor-planning and the start of what normally would be motor-execution is unclear. Therefore, it is unknown how much of that performance increase is due to the inclusion of additional motor-planning activity and how much is due to the inclusion of activity associated with attempted motor-execution. Regardless, these results suggest that, in trials where movement failed to be detected during early motor-planning, the device still has a good chance of be triggered during delayed motor-planning or initial attempted execution. Activating a movement-assist device with a 200-to-600-ms delay could still provide functional benefit in this population. However, the therapeutic benefits of the delayed movement attempts may not be as strong as those attempts that result in more rapid device activation.

The default classifier threshold from linear-discriminant analysis is designed to maximize classification accuracy and, therefore, resulted in the biggest differences between true- and false-positive classification rates. However, the default threshold also resulted in a fairly high false-positive rate. False-positives could be problematic in systems designed to fully open the hand as soon as the classifier detects attempted-finger-extension. This problem could be resolved by shifting the threshold to reduce unintentional device activation, but doing so also reduced the difference between the true- and false-positive rates.

For some “graded” device applications, preventing false-positives may not be as important as maximizing classification accuracy. For example, a neuromuscular stimulation system could be set up so that each time sample classified as finger-extension would incrementally increase the current to the extensor muscles, and each time sample classified as rest would incrementally decrease the stimulation current. In this scenario, using classifier thresholds with the highest overall accuracy would produce the most appropriate hand responses. Misclassifications in any individual time sample would only have a small incremental effect and would be quickly overwritten by the larger number of correctly classified time samples.

False detections of attempted-finger-extension during rest would be more problematic for devices designed to completely open the hand in an all-or-nothing manner as soon as a single time sample is classified as a finger-extension event. With these devices, inappropriate hand opening could be avoided by shifting the threshold to reduce false triggers as well as by requiring multiple consecutive-time samples to be classified as finger-extension events. Figure 4 shows many combinations of threshold settings and consecutive-classification requirements where there were no false triggers during rest in any trial but a potentially useful number of attempted-movements that were successfully triggered (i.e., dots located along the Y axes in Figure 4). For example, in the no-time-extension condition (upper row in Figure 4), setting the threshold for a 1 or 5% false-positive rate and requiring two or three consecutive triggers resulted in many sessions where there were no trials with false triggers and up to half of the trials had true triggers (see green and blue dots along the Y axis).

The number of successful movement attempts substantially increased when the time window allowed for classification was expanded by up to 600 ms (bottom row of Figure 4). For example, setting a threshold to reduce the false-classification rate to 1%, and requiring between three and five consecutive triggers resulted in no false triggers and a large range of correctly triggered trials including some sessions with up to 100% correct triggers (see green, red, and pink dots along the Y axis in bottom left plot in Figure 4). Even at the default classification threshold (LDA) which resulted in the highest false-positive rates, false triggers could be completely eliminated in many cases by requiring four or five consecutive-classification events (see red and pink dots along the upper half of the Y axis in the bottom right plot of Figure 4).

Requiring multiple consecutive-classification events to trigger a device fairly consistently improved the ratio of true events to false events as can be seen by the majority of the dots in Figure 4 being above and to the left of the solid diagonal line. However, device activation will inevitably be slower as more consecutive-time points are used to build certainty about triggering a device. Online tests are needed to determine the relative importance of speed vs. reliability for promoting therapeutic benefits.

High detection reliability may not be essential for therapeutic benefit. During actual online use of an EEG-triggered movement-assist device, the user will receive immediate feedback of the success or failure of each attempted-movement. This feedback from failed attempts could help the user learn to generate stronger, more-consistent cortical activation patterns with practice. The cortical patterns required to trigger the device could also be gradually shaped over time to require the user to make more-normal EEG patterns as they progress with their therapy.

This study shows that early detection of attempted-movements is possible in individuals with complete paralysis of finger-extensor muscles after stroke. These detectable modulations can provide a starting point from which to develop a therapeutic retraining paradigm using EEG-triggered assisted movements. Again, online studies are now needed to fully explore how best to utilize EEG-triggered assisted movements in stroke rehabilitation.

Conflict of Interest Statement

The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.