KITE-BCI: A brain-computer interface system for functional electrical stimulation therapy.

CONTEXT/
OBJECTIVE: Integrating brain-computer interface (BCI) technology with functional electrical stimulation therapy (FEST) is an emerging strategy for upper limb motor rehabilitation after spinal cord injury (SCI). Despite promising results, the combined use of these technologies (BCI-FEST) in clinical practice is minimal. To address this issue, we developed KITE-BCI, a BCI system specifically designed for clinical application and integration with dynamic FEST. In this paper, we report its technical features and performance. In addition, we discuss the differences in distributions of the BCI- and therapist-triggered stimulation latencies.
DESIGN: Two single-arm 40-session interventional studies to test the feasibility of BCI-controlled FEST for upper limb motor rehabilitation in individuals with cervical SCI.
SETTING: Rehabilitation programs within the University and Lyndhurst Centres of the Toronto Rehabilitation Institute - University Health Network, Toronto, Canada. PARTICIPANTS: Five individuals with sub-acute (< 6 months post-injury) SCI at the C4-C5 level, AIS B-D, and three individuals with chronic (> 24 months post-injury) SCI at C4 level, AIS B-C. OUTCOME MEASURES: We measured BCI setup duration, and to characterize the performance of KITE-BCI, we recorded BCI sensitivity, defined as the percentage of successful BCI activations out of the total number of cued movements.
RESULTS: The overall BCI sensitivities were 74.46% and 79.08% for the sub-acute and chronic groups, respectively. The average KITE-BCI setup duration across the two studies was 11 min and 13 s.
CONCLUSION: KITE-BCI demonstrates a clinically viable single-channel BCI system for integration with FEST resulting in a versatile technology-enhanced upper limb motor rehabilitation strategy after SCI.

Introduction

Since the emergence of brain-computer interfacing (BCI) in the 1970s,[1] researchers have explored its use as an assistive technology for facilitating communication,[2] and controlling computers[3] or devices to restore movement of a paralyzed limb.[4] This last application has motivated the adoption of BCI technology within the field of neurorehabilitation.[5,6] In the context of rehabilitation of voluntary movement, a BCI can be coupled with a movement-assisting device (e.g. robot or electrical stimulator) and used as a short-term intervention to promote recovery of motor function after stroke or spinal cord injury (SCI). For example, BCI has been incorporated into functional electrical stimulation therapy (FEST) for upper limb rehabilitation in individuals with severe motor impairments — a population who may not respond to best-practice therapies.[7]

FEST uses functional electrical stimulation (FES), a technology that generates coordinated muscle contractions using electrical pulses to artificially produce functional movements.[8] In conventional FEST, a therapist activates the stimulation using a switch. During therapy, the individual is cued to attempt a movement; and after several seconds, the therapist triggers the stimulation. The individual is also instructed to continue attempting the movement, even when the stimulation is active. Such an approach suggests that the patient with impairment is voluntarily attempting to move, which is followed and accompanied by the afferent feedback of a successfully executed FES-assisted movement. The cortical motor activity paired with the afferent feedback is the hypothesized mechanism facilitating functional improvements resulting from FEST.[9] The repeated motor command-somatosensory feedback coupling, present throughout the duration of therapy, is believed to induce activity-based plasticity in the central nervous system (CNS), which underlies motor learning in both intact and injured nervous systems.[5]

During BCI-FEST, in comparison, the stimulation is triggered when the BCI detects an imagined or attempted movement. Using the BCI provides greater confidence that the patient is voluntarily generating motor commands during therapy, which is reflected in the electroencephalography (EEG) activity used to determine when to trigger the stimulation. In such a way, BCI can engage the CNS into re-learning motor functions.[5]

Multiple integrated BCI-FEST systems have been proposed.[10-14] The encouraging results from studies by Tabernig et al.[10] or Osuagwu et al.[11] suggest that BCI-FEST resulted in clinically relevant recovery more effectively than therapy with randomly or automatically activated FES. However, despite these results, the use of BCI-FEST technology in clinical practice is minimal. Letourneau et al.[15] conducted a Canadian national survey on physicians’ perceptions of BCI and its potential to help individuals with a severe disability. Even with the estimated thousands of patients that could potentially benefit from BCI, few neurologists and physiatrists reported being aware of this technology.

We believe that there are two obstacles preventing the adoption of BCI-FEST technology in clinical practice. The first issue relates to the time allotted for each patient in rehabilitation. In standard clinical practice, the length of a therapy session, including the setup of any equipment used, is often limited to one hour. Consequently, the setup itself rarely exceeds 15 min,[16,17] and a lengthier setup is a significant limitation for clinical application.

The second issue relates to the adaptability of the intervention to the patients’ needs. While some patients may need to focus on grasping, others might need to concentrate on reaching, or both; or start the intervention practicing reaching and transition towards grasping as the therapy progresses. A system that cannot be easily customized to facilitate multiple functional movements may limit the number of patients who could benefit from it by not offering the necessary versatility reflective of the individual and progressive nature of rehabilitation.

To address the issues described above, we designed and developed a BCI system with a quick setup, and integrated it with a FEST system for upper limb rehabilitation. The BCI requires a single configuration session, and the stimulation can be customized to assist in multiple types of grasping, reaching, or the combination of both motions. The design was informed by engineers and therapists with two decades of clinical work in upper limb rehabilitation. The integrated system supports a wide range of movements that can be tailored to patients’ rehabilitation goals and progress throughout the therapeutic intervention.

Our early work included two case studies in which we evaluated the feasibility and safety of delivering the BCI-FEST intervention in a clinical environment.[13,18] In our first proof-of-concept study, two experimenters delivered 40-hours BCI-FEST to an individual who had sustained a stroke six years prior to the study. In the following work, we tested the feasibility of having a therapist and a BCI operator deliver 80 h of therapy, again, to an individual with chronic stroke (six years after the injury). The BCI-FEST intervention was safe, and we successfully delivered 73 out of 80 sessions using the BCI system (the remaining seven sessions used FEST alone). Most importantly, both individuals showed clinically meaningful improvements in their arm and hand function following BCI-FEST. Details of the two case studies can be found in Marquez-Chin et al.[13] and Jovanovic et al.,[18] respectively.

In this report, we describe our approach to an integrated BCI-FEST system, with a focus on the technical features that allow the clinical application of the system. It consists of the latest iteration of our BCI, which we call KITE-BCI, and a versatile FEST. We report on the BCI setup duration and system performance from two interventional studies for upper limb rehabilitation after SCI. Finally, we discuss the differences in stimulation latencies between the BCI-triggered and therapist-triggered stimulation.

Materials and methods

KITE-BCI

Concept

FEST has been extensively and successfully used by our group in the last two decades. The KITE-BCI system, and its predecessor,[13,18] is a BCI designed to detect movement attempts and trigger stimulation during FEST. The system uses a single EEG channel, selected out of eight possible locations determined during the initial configuration. The system’s algorithm for detecting attempted movements is inspired by the methods for detecting event-related desynchronization (ERD). ERD is a decrease of power in a user-specific frequency band of electrical brain activity which is produced by execution, imagination, and preparation (i.e. attempt) of movement.[19]

Hardware

The EEG activity is recorded using gold-plated 10 mm cup-shaped reusable EEG electrodes (Genuine Grass® Reusable Cup EEG Electrodes, Natus, Pleasanton, CA, USA), an amplifier, and a data acquisition system (DAQ). The DAQ is connected to a personal computer, which serves as the processing unit that detects an attempted movement and triggers the FES. In our studies, we have used stand-alone biopotential amplifiers (QP511/IP511 A.C. Amplifier, Grass Telefactor, West Warwick, RI, USA) at x10,000 amplification and National Instruments (Austin, TX, USA) DAQ systems (DAQPad-6015/USB-6363), removing the need for a dedicated EEG system and simplifying the instrumentation requirements. The output signal is generated by the DAQ and connected to the stimulator via a custom-made relay and an industry-grade isolator (Eurotherm-Action Instruments). The isolator limits the current flow and ensures the safety of the BCI users.

The FES is delivered using a COMPEX Motion (COMPEX, Ecublens, Switzerland) FES system,[20] a programmable stimulator with four channels that uses transcutaneous self-adhesive electrodes (5 × 5 cm, 5 × 10, and 3 cm round ValuTrode (X) Cloth electrodes; Axelgaard Manufacturing Co., Ltd., Fallbrook, CA, USA). The FES-assisted movements are programmed into protocols consisting of multiple stages, where each stage corresponds to a movement phase. The programmable nature of the COMPEX Motion allows us to customize the facilitated movements according to individual patients’ needs and their progress during therapy.

Stimulation pulses were asymmetrical and bi-phasic, with a frequency of 40 Hz and 250 µs pulse width. The 40 Hz pulse frequency allows us to neutralize the FES artifact as the BCI processing for detecting movement attempts is focused on alpha (8-12 Hz) and beta (12-30 Hz) EEG bands.

The integrated BCI-FEST system is equipped with a manual switch for the therapist in the form of a hand button or a foot pedal, accompanied by a yellow LED used to easily indicate the state of the BCI system. The decisions to add the switch and the LED were informed by the feedback from collaborating clinicians. These two features have been critical for the delivery of therapy and the clinical viability of the present BCI-FEST system.

Software

The software, developed in LabView (National Instruments, Austin, TX, USA), processes the EEG signal in real-time and detects movement attempts. The EEG activity is acquired at 200 Hz (with 20 samples block size) and processed to generate a single-value BCI output every 100 milliseconds (i.e. 200 Hz / 20 samples). Processing includes band-pass filtering using a third order Butterworth filter (according to each participant’s frequency band displaying a decrease in power), squaring the signal, calculating the root mean square, and applying a moving average filter of one second in length (i.e. 10 samples). The BCI output is then compared against a Power threshold to detect the decrease in power. At the same time, we measure the time during which the BCI output remains below the Power threshold and compare that duration against a Time threshold. A BCI activation, or the successful detection of an attempted movement, only occurs when the BCI output is sustained below the Power threshold for the duration equal to or longer than the Time threshold. Both the Power threshold and Time threshold can be adjusted during operation using a graphical user interface (GUI).

While we use EEG signals in this study, the abovementioned BCI approach was initially developed using intracranial electrocorticographic signals, based on a simple signal processing technique that allowed for the rapid implementation of a BCI system.[21]

The KITE-BCI is an asynchronous BCI system, which continuously detects movement attempts. This design was adopted because rehabilitation is a dynamic process, in which the occurrence and length of the active and rest periods of the therapy are not predefined. Therefore, to ensure smooth delivery of therapy and avoid the disruptions caused by any unwanted BCI activations, the system has armed and unarmed states. While there is always a physical connection between the BCI and FES systems, the BCI states enable (or disable) the connection via software. When unarmed, a BCI activation cannot trigger the stimulation. In contrast, when the system is armed, a BCI activation triggers the stimulation and the system reverts automatically to an unarmed state to avoid subsequent unintended commands to the FES system. The therapist uses the manual switch to arm the BCI system when they give a cue to the patient to start attempting a movement.

The KITE-BCI can also record raw EEG data, estimated power of the band-limited EEG signal, and BCI events. The raw EEG signal and estimate power are recorded at 200 Hz, while the BCI events are recorded at the same frequency as the BCI output update rate, which is 10 Hz. The recording feature makes it possible to characterize the BCI performance and measure stimulation latencies, discussed at greater length below.

Personnel requirements

In its present form, the delivery of BCI-FEST requires a registered therapist trained in FEST and a BCI operator trained in using KITE-BCI. At the beginning of each session, the therapist sets up the FES electrodes and adjusts the stimulation parameters, while the BCI operator simultaneously sets up the BCI system.

During a session, the therapist is equipped with a manual switch which they use to control the BCI-FEST system. Additionally, the therapist guides the patients to ensure the quality and correctness of the practiced movements. The BCI operator's role is to monitor the state of the BCI system presented on the GUI and, if needed, adjust the processing parameters.

KITE-BCI operation

Before starting an intervention, a single configuration session is necessary for determining the location of the EEG channel and the frequency band that exhibits a power decrease during movement attempts. The configuration process has been previously described by Marquez-Chin et al.[13] In summary, participants are seated in front of a monitor and cued to attempt 80–160 hand movements (ready-set-go-stop), while we record EEG signals from eight locations: F3, F4, Fz, C3, C1, C4, C2, Cz. We also record the instants in which cues are presented to the participants using a photoresistor circuit mounted on the screen, which generates pulses according to presented cues (see Supplementary material).

To analyze the recorded EEG activity, we first apply a filter bank between 3 and 32 Hz (overlapping 2 Hz windows with 1 Hz steps). Then, using the Hilbert transform, we extract the envelope of the filtered signal, which we square to estimate power values.[22] Next, we extract the 12-second epochs centered around the go cue (i.e. 8 s before and 4 s after the go cue) for each 2 Hz frequency band, in all available trials. The first two seconds of each epoch are used to calculate the baseline power as that interval corresponds to participants sitting still without attempting any movement. After calculating the change in power relative to the baseline, we average the power values across the epochs and create a single temporospectral map for each electrode location. Finally, the electrode location and the frequency bands displaying the relative decrease of power are selected by visual inspection.[23] The described configuration process is repeated for each limb when therapy targets both left and right upper limbs. The selected electrode locations and frequency bands are used for the remainder of the study (i.e. entire intervention). While the configuration process uses eight EEG channels, the BCI-FEST intervention is conducted using a single channel.

The events that take place during a single FES protocol are listed below. The conceptual illustration of the BCI-FEST intervention and the sequence diagram are shown in Figure 1.

Figure 1

Top: The conceptual illustration of the BCI-FEST intervention, displaying the EEG electrodes placed on the participant’s scalp and the FES electrode placed on the participant’s arms and hands. The figure also shows an LED used to inform the therapist on the state of the BCI system. The LED is placed on the shoulder allowing the participant to focus on the practiced movements and not the operation of the BCI. The therapist standing across from the participant is guiding the movement and has access to the integrated system via the manual switch placed on the table. Bottom: Diagram displaying the sequence of events in a single BCI-FEST movement phase.

Step I) At the start of an FES protocol, therapists cue the patient to attempt the first phase of the movement. At the same time, they press the manual switch. This initial switch activation arms the BCI system and turns the feedback LED ON.

Step II) When the BCI detects a movement intent it triggers the stimulation, disarming the system and turning OFF the LED. If the BCI is not activated, the therapist can press the switch a second time, triggering the stimulation, and resetting the BCI state and LED. During BCI-FEST, as in conventional FES therapy, patients continue to attempt movements while being assisted by stimulation.

Step III) Steps I and II are repeated until the last switch activation, which is always triggered by the therapist, as it turns off the stimulation and is followed by rest and not another movement. This is referred to as the rest trigger. Afterwards, the system is ready for the next movement iteration.

Each FES protocol used with KITE-BCI, which corresponds to a functional movement, consists of at least two phases – activating agonist, then antagonist muscle (e.g. biceps and triceps brachii). The protocols that combined reaching and grasping often involved at least three phases: arm reaching, followed by opening and/or closing the hand, and finally retrieving the arm.

Interventional studies with BCI-FEST in SCI

We conducted two interventional studies to assess the feasibility and efficacy of BCI-FEST for upper limb rehabilitation for individuals with sub-acute (less than six months after injury)[24] and chronic (at least 24 months after injury) SCI. Participants of both studies were adults with traumatic incomplete SCI at the C4-C7 level and AIS B-D. In both studies, participants received up to 40 sessions. The sub-acute participants received up to five sessions weekly, while chronic participants received up to three sessions weekly. Each session included practicing FES-assisted movements selected based on the participant's goals. The same BCI-FEST system was used to support grasping, reaching, or both types of movements simultaneously. Additionally, except in cases of two participants (one in each study), the presented integrated BCI-FEST system was used for rehabilitation of both upper limbs.

We used the data from these studies to evaluate the clinical usability of KITE-BCI by measuring setup duration and system performance.

Outcome measures

BCI setup duration

The duration of the BCI setup was defined as the time required to mark electrode location(s), prepare the skin to decrease impedance, attach the electrodes, and ensure the quality of their connection. We then averaged the durations across sessions (per participant) and participants.

BCI performance

We calculated the BCI Sensitivity to assess the performance of the KITE-BCI system during therapy sessions. We defined this value as the number of successful BCI activations divided by the total number of cued movements. The successful BCI activations were defined as the activations following the therapist's cues to attempt a movement. Calculation of the BCI sensitivity did not include rest triggers. Rest triggers were identified as stimulation triggered by the therapist that occurred less than two seconds following the arming of the system.

Results

We recruited five participants for the sub-acute study, who completed a total of 149 one-hour therapy sessions. For the chronic study, we recruited three participants, who completed a total of 120 sessions. A detailed breakdown of the number of completed sessions for each participant, their demographic and SCI information are presented in Table 1.

Table 1

Participant's demographic and neurological data, and the number of sessions completed in the intervention.

Participant ID	Sex	Age	Cause of Injury	Level of injury	Time since injury	AIS	Completed sessions
							L	R	B	T
Subacute group
AAOF	M	66	fall	C4	35 days	C	6	2	4	12
AAOG	F	72	fall	C4	70 days	C	6	16	18	40
AAOH	M	37	motor vehicle accident	C4	53 days	B	12	16	12	40
AAOI	M	58	bike accident	C4	50 days	D	29	0	29	29
AAOJ	F	26	fall	C5	149 days	B	5	9	14	28
Chronic group
AAOA	M	31	diving	C4	42 months	B	0	40	0	40
AAOB	M	53	motor vehicle accident	C4	24 months	C	20	20	0	40
AAOC	F	31	motor vehicle accident	C4	68 months	C	12	22	6	40

Note: AIS, American Spinal Injury Association Impairment Scale; L, left side; R, right side; B, both sides; T, the total number of completed sessions.

BCI Performance Results

BCI configuration

We completed a single two-hour BCI configuration session for each participant in both studies. The selected electrode locations, frequency bands and the number of trials included in the configuration analysis for each participant are presented in Table 2. For participant AAOI from the sub-acute group, the configuration was done only for the left hand as only that side was targeted during therapy. Similarly, for participant AAOA from the chronic group, the configuration was done only for the right side. For everyone else (i.e. the remaining six participants) the configuration was performed for both hands.

Table 2.

The BCI configuration data.

Participant ID	Hand	Electrode location	Frequency band [Hz]	Number of trials
Subacute group
AAOF	Left	C2	8–12	80
	Right	C1	8–12	80
AAOG	Left	C4	9–13	100
	Right	C1	10–14	120
AAOH	Left	C4	12–16	80
	Right	C3	9–13	120
AAOI	Left	Cz	12–16	140
	Right	N/A	N/A	N/A
AAOJ	Left	C4	9–13	120
	Right	C3	9–13	120
Chronic group
AAOA	Left	N/A	N/A	N/A
	Right	C1	23.3–25	160
AAOB	Left	C2	14–18	120
	Right	C1	9–12	140
AAOC	Left	C4	18–25	120
	Right	C1	16–21	120

Note: BCI, brain-computer interface; N/A, not applicable.

Figures 2 and 3 present the corresponding ERD maps, which were used for the selection of the frequency bands. We focused on the area immediately after the movement/attempt starts (represented by the vertical line at time 0s), looking for areas displaying a relative decrease in power, indicated by cooler (i.e. blue) colors.

Figure 2

Maps displaying the relative change in power (between −1 and 1) generated after BCI configuration session for sub-acute participants. The horizontal axis represents time, while the vertical axis represents frequencies. The GO cue is at 0 s, as indicated with the vertical line. The selected frequency bands are shown between the two horizontal purple lines. Refer to Table 2 for specific frequency ranges used.

Figure 3

Maps displaying the relative change in power (between −1 and 1) generated after BCI configuration session for chronic participants. The horizontal axis represents time, while the vertical axis represents frequencies. The GO cue is at 0, as indicated with the vertical line. The selected frequency bands are shown between the two horizontal lines. Refer to Table 2 for specific frequency ranges used.

To quantify the impact of the single-channel design on the BCI setup, we recorded the setup duration in 223 (out of 269) sessions. The average BCI setup duration in both studies ranged between 10:01 (minutes:seconds) and 12:11. The average duration of the BCI setup across participants was 11 min and 13 s. The setup duration for each participant is included in Table 3.

Table 3

The BCI setup duration and performance data.

Participant ID	Hand	BCI Sensitivity	BCI Setup Duration
		[%] (n = total number of expected BCI triggers)	mm:ss
Sub-acute group
AAOF	Left	78.0 (n = 445)	11:35
	Right	74.4 (n = 550)
AAOG	Left	71.3 (n = 1374)	11:28
	Right	69.5 (n = 1809)
AAOH	Left	76.9 (n = 1020)	10:01
	Right	74.8 (n = 1363)
AAOI	Left	72.2 (n = 2036)	10:12
	Right	N/A
AAOJ	Left	75.2 (n = 718)	12:11
	Right	80.2 (n = 1005)
Chronic group
AAOA	Left	N/A	11:53
	Right	83.2 (n = 2564)
AAOB	Left	81.8 (n = 566)	10:21
	Right	81.5 (n = 1663)
AAOC	Left	75.6 (n = 1146)	12:07
	Right	70.0 (n = 1392)

Note: BCI, brain-computer interface; N/A, not applicable.

The BCI setup included setting up EEG electrodes until they were connected to the amplifier. It did not include the setup of the FES electrodes, which was conducted simultaneously by the therapist. The time taken for the placement of FES electrodes was not measured as it was regularly completed more quickly than the BCI setup.

BCI Performance Results

Across the two studies, the BCI sensitivity ranged from 69.5% to 83.2%. The detailed breakdown of individual BCI performance measures is listed in Table 3.

Discussion

This report presented KITE-BCI, a novel BCI system for clinical application in upper limb motor rehabilitation. We evaluated KITE-BCI performance in two single-arm interventional studies involving individuals with cervical SCI. Detailed clinical results from the study with sub-acute participants have been reported elsewhere.[24] In general, most participants experienced improvements in upper limb motor function. Three out of five participants demonstrated clinically significant improvements on self-reported assessments of independence, used as primary outcome measures. Additionally, four out of five participants showed significant improvements on observational assessments of upper limb motor function, used as secondary outcome measures.

The KITE-BCI is characterized by three features that make it and the integrated BCI-FEST system viable for clinical application. First, the single-channel design has allowed for a shorter BCI setup duration. The average BCI setup duration in both studies ranged between 10 and 12 min. These values are comparable to the preparation time of other technology-enhanced therapies, such as conventional FEST[25,26] or BCI-based robot-assisted therapy.[17] The second feature is a single BCI configuration session, which can be used for all the different FES-assisted movements practiced during the entire intervention (up to 40 one-hour therapy sessions). This removes the need for multiple training sessions and enables a quicker start to the intervention. The final feature is the versatility of the integrated BCI-FEST system to support a variety of upper limb movements. This feature is driven by the goal to continue building on the already existing customization of the FEST upper limb intervention developed by our group over the last two decades.[27] The BCI system was designed to detect movement attempts during FEST intervention and to be compatible with multi-channel FES devices, which can be programmed to support a wide range of functional movements, including simultaneous reaching and grasping motions.

The KITE-BCI system was implemented as a detector of the intention to move, not a classifier. A BCI system that can identify intentions to perform specific movements would greatly benefit an FES system that adjusts in real-time. Another advantage would be the ability to disregard attempts to perform incorrect movements during therapy. While, technically, it may be possible to implement a classifier of various types of reaching and/or grasping,[28-30] such a system would likely require more frequent re-calibration, reducing its application in a clinical environment. The re-calibration issue would be particularly noticeable given that the practiced tasks during therapy are often not predefined but determined by the therapist according to the baseline level and the evolution of the individual’s upper limb motor function throughout the intervention. Reliable real-time classification of various upper limb movements using non-invasive methods, such as EEG, is one of the most important challenges in the BCI field. For the presented BCI-FEST system, the role of identifying that the correct movement is attempted is fulfilled by the therapist, as is the case with conventional FEST and other interventions. The therapist’s presence is important as they also guide the participants to perform physiologically correct movements supported by the FES.

A clinically viable BCI classifier of arm and hand movements could be explored in future work. Future studies could also investigate other ways in which existing BCI systems can be used to complement therapy. In addition to using a system to control an external device, a BCI system could also be used to monitor and guide the progression of the therapy. For example, the variation in the adjustable parameters of a BCI detector, such as the described Power threshold of the KITE-BCI, could potentially be used to indicate a patient’s effort for a given task. Under such implementation, the Power threshold could be used to quantify and modify (increase or decrease) the difficulty of the exercise. Furthermore, the effects of injury level and chronicity on BCI sensitivity, and subsequently effectiveness of the intervention, could be investigated to identify the optimal timing for delivery of BCI-FEST intervention in individuals living with tetraplegia and further increase the clinical impact of these technologies.

KITE-BCI Performance

While the single-channel design and single configuration approach of the KITE-BCI demonstrated its clinical feasibility, the system's performance had not been compromised. The overall BCI sensitivities, or the percentage of cued movement attempts successfully triggered by the BCI, were 74.46% and 79.08% for the studies with sub-acute and chronic participants, respectively. Both values were greater than 70%, which is often referred to as the threshold for an effective BCI system (for communication).[31] Additionally, in case of missed BCI detection, the therapist could trigger the stimulation manually and ensure that patients always received the stimulation.

Regarding the BCI performance metrics, we only measured BCI sensitivity because KITE-BCI was primarily designed to support the FEST intervention. In the implemented BCI operation mode during therapy only true positives and false negatives were clearly defined. In technical terms, KITE-BCI could be described as an asynchronous system as it continuously monitors brain activity and detects attempted movements as sustained decrease of power. However, the implementation of the armed and unarmed BCI states makes the KITE-BCI a hybrid system – a BCI that is both asynchronous and synchronous. True positives were defined as the BCI activations after the therapist had cued the patient to start attempting a movement (i.e. armed the BCI). False negatives were the instances when the therapist triggered the stimulation manually after the patient has been cued to move, but the BCI did not detect an attempted movement.

Outside of the periods when patients are attempting movements as part of therapy, the BCI continues to operate in an asynchronous mode. The two-state approach was adopted because the BCI was developed around the existing FEST method that our group had used successfully in the past. Therefore, due to the therapeutic process in which the BCI-FEST system was used, where the patients continued to move after the stimulation triggers, it was not feasible to clearly define false positives. The BCI activations taking place when the patient moves (as part of the rehabilitation process), indicates that the system is working as designed and responding to movement, rather than producing false activations. Similarly, during the period between repetitions of movements, patients often conversed with the therapist, or they were engaged in other activities such as passive stretching. The therapy in a clinical environment did not allow for a predetermined period of stillness, during which we could monitor true negatives.

While we were unable to measure other aspects of BCI performance, we observed a higher density of BCI activations following, rather than preceding, stimulation triggers. We generated a raster plot and a histogram of BCI activations centered around stimulation triggers facilitated by the BCI – see Figure 4. To generate the graphs shown in Figure 4, we used the BCI data from a single participant (AAOI), and we randomly selected 500 trials in which the BCI was used to trigger the stimulation. In each trial, we recorded all BCI activations in 30 s proceeding the stimulation trigger and 30 s following the stimulation trigger. The periods preceding stimulation triggers include instances when the stimulation was off (taking a break in between movement repetition leading to Step I of the Sequence of Events) or on (Step II of the Sequence of Events) – in other words, on average, the patients had less movement during these periods. In contrast, the periods following the stimulation triggers only included instances when the stimulation was on (Step II), when on average patients had more movement. As indicated on the histogram (Figure 4 – bottom), we observed a greater count of BCI activations after the stimulation triggers (time = 0s) than before. These observations suggest that KITE-BCI generates more activations when patients are attempting or executing movements compared to when they are taking a break in between movement repetitions.

Figure 4

Top: The scatter plot of BCI activations surrounding 500 randomly selected stimulation triggers activated by the BCI, for participant AAOI. The vertical axis indicates trial number, and each black dot represents a BCI activation in the unarmed state which did not trigger the stimulation. The lighter (green) dots clustered at 0 represent BCI activations that triggered the stimulation. Bottom: The corresponding histogram plot. The horizontal axis which applies to both graphs indicates time preceding, and time following stimulation triggers, in seconds.

Stimulation Latencies

In this study, we also attempted to gain a better understanding of the differences in stimulation latencies between the BCI- and therapist-triggered stimulation. We recorded the time when the BCI was armed (i.e. the therapist’s cue), then when the stimulation was triggered, and finally, whether it was triggered by the BCI or the therapist. We used these time-points to calculate the differences between the stimulation triggers and arming of the BCI-, which we defined as stimulation latencies. We separated the BCI and therapist-triggered stimulation latencies and calculated the respective means, standard deviations, ranges, and median values.

Due to the experimental design in which the therapists were instructed to only trigger the stimulation after the BCI missed to detect an attempted movement, we were not surprised to observe that the BCI-triggered stimulation had shorter latencies than the therapist-triggered stimulation (see Table 4). However, we also generated overlapping histograms of stimulation latencies in two conditions to compare the distributions. As shown in Figure 5, we observed differences in distributions of latencies between BCI-triggered and therapist-triggered stimulations – and the differences were consistent across the two studies. While the BCI-triggered latencies were clustered close to zero, the therapist-triggered latencies were spread around the mean.

Table 4

Stimulation latencies.

Group	BCI-triggered[seconds]				Therapist-triggered [seconds]
	N	Range	M ± SD	Median	N	Range	M ± SD	Median
Sub-acute group	7613	0.1–39.9	3.7 ± 3.9	2.4	2707	2.0 - 95.5	9.7 ± 5.6	8.7
Chronic group	5791	0.1–50.8	3.7 ± 4.1	2.4	1540	2.0–122.7	10.3 ± 6.8	9.2

Note: M, mean; SD, standard deviation.

Figure 5

The histogram of stimulation latencies (a time difference between the therapist’s cue and the stimulation trigger) lower than 30 s, in the study with the sub-acute participants on the left side, and in the study with chronic participants on the right side. Each bar represents a count of stimulation latencies within half-second time windows, from 0 to 30 s.

The experimental design can account for the differences in mean values. However, it does not entirely account for the differences in the distribution shapes. The findings suggest that in the case of therapist-triggered stimulation in conventional FEST, the peak of the distribution would be shifted towards zero due to a shorter mean latency. However, we argue that the distribution shape would remain similarly wide because the therapist-triggered stimulation is dependent on the therapist’s perception of patient’s efforts. In contrast, the BCI is programmed to directly respond based on the patient’s focused attempt to move, and more consistently deliver stimulation at shorter latencies. The timing between the movement-related cortical activity and stimulation-induced afferent feedback was shown to play a critical role in inducing excitability of cortical projections to a muscle[32] – which is one of the hypothesized mechanisms underlying functional recovery following neuromodulation-based interventions such as BCI-FEST.[33]

We acknowledge the limitations associated with the experimental design and the presented comparison of stimulation latencies. However, it nonetheless provides insights into the distribution of stimulation activations in the BCI condition and sets the framework for future controlled comparison of BCI-FEST and conventional FEST. Further studies investigating the differences between BCI- and therapist-triggered stimulation latencies are warranted.

Conclusions

The KITE-BCI can be set up in less than 15 min and has the versatility to support the progression of FEST intervention, which evolves according to the changes that the patient experiences. Additionally, this system does not require continuous training and maintains the patient’s focus on practicing movements. Most importantly, KITE-BCI integration with FEST produces clinically meaningful improvements, making it a promising step towards a clinically viable BCI-FEST system.