Optimization of machine learning method combined with brain-computer interface rehabilitation system.

[Purpose] Stroke patients are unable to move on their own and must be rehabilitated to allow the nervous system to trigger and restore its function. Traditional practice is to use electrode caps to extract brain wave features and combine them with assistive devices. However, there are problems that the electrode cap is not easy to wear, and the potential recognition is not good, and different extraction methods will affect the accuracy of the Brain-Computer Interfaces (BCI), which still has room for improvement. [Participants and Methods] The brainwave headphones used in this experiment do not must a conductive gel to get a good EEG for neural induction and drive the upper limb rehabilitation robot. Next, 8 stroke patients and 200 normal participants were invited for a 4-week rehabilitation training. The effectiveness of the training was determined using Fast Fourier Transform (FFT), Magnitude squared coherence (MSC) feature extraction methods, and five machine learning techniques that induced flicker frequencies.
[Results] The results show that the optimal steady-state visual evoked flicker frequency is 6 Hz, and the identification rate of FFT is about 5.2% higher than that of the MSC method. Using an optimized model for different feature extraction methods can improve the recognition rate by 1.3%-9.1%.
[Conclusion] The images based on Fugl-Meyer Assessment (FMA), Modified Ashworth Scale (MAS) index improvement, and functional Magnetic Resonance Imaging (fMRI) show that the sensory region of brain movement has become a concentrated activation phenomenon. Besides strengthening the feature extraction method also lets the elbow has an obvious recovery effect. 2022©by the Society of Physical Therapy Science. Published by IPEC Inc.

INTRODUCTION

The rehabilitation mechanism for stroke patients is to perform movements on the opposite side of the damaged brain, with different pathways passing commands to motor neurons, which in turn activates the adjacent areas of the brain that have been damaged.

Steady-state visual evoked potentials (SSVEP) is the most extensive clinical treatment mode in used brain-computer interface rehabilitation aids1, 2). The combination of signature signals and physiological states in brain waves must be converted into signature signals with accurate measurement data.

Currently, machine learning methods can identify the frequency of specific visual stimuli from the signals of steady-state visual evoked potentials, but the shortcomings only simplify the more general recognition and decision process3, 4). Thus, it is necessary to optimize the feature extraction method and improve the identification results through integration and decision-making.

In a review of the literature on brain-machine interfaces, most use EEG (α and β wave)-based systems to control upper limb rehabilitation aids. Besides training hand rotation, grip strength, and elbow imagination stretching, but also improve the effectiveness of treatment5,6,7,8), which can stimulate the brain’s neurotransmitter unit, thereby restoring its muscle and central nervous system connection. In about9,10,11,12,13,14,15), the robot is driven by brain waves and trained to analyze multi-channel signals to control the robot.

Besides, brain waves can operate virtual reality rehabilitation game16), can also control the direction of wheelchair travel17,18,19), but the above applications are using electrode caps to extract brain waves, and the use of the scalp also needs to be injected with conductive gel, resulting in wear inconvenience.

In this work, we use a wearable MindWave Mobile 2, which has the advantage of enhancing local electroencephalograms, wireless transmission, front-end brain-machine interface conversion, and making operation easier without the need for conductive gels.

The next step is to construct a steady-state vision induction system to combine the upper limb rehabilitation robot. Its structure is a two-dimensional plane of motion, the motor is mounted on the base to drive the link, and simulates the therapist for the shoulder and elbow movement trajectory and power to mix control can enhance the stimulation of brain neurons so that muscles and nerves to restore the link.

Also, we have improved the feature extraction method and added a modular machine learning architecture as a model of the input layer. The next step is to integrate and judge through the decision-making level in an attempt to improve the problem of poor potential recognition. In this study, compared with previous solutions9,10,11,12,13,14,15), besides Fugl-Meyer Assessment (FMA) clinical indicators, Modified Ashworth Scale (MAS) was added to analyze the rehabilitation status of patients. It strengthens the accuracy of the Brain-Computer Interfaces (BCI) system and helps the muscles connect with the central nervous system. Finally, functional Magnetic Resonance Imaging (fMRI) was used to observe the degree of activation of brain cells, and to verify that the system has a considerable rehabilitation effect.

PARTICIPANTS AND METHODS

According to the clinical rehabilitation research of stroke patients20), imaginative movements are often used to activate brain motor areas for nerve induction, which is better than traditional passive rehabilitation21). Thus, we use Neurosky’s mind wave mobile 2 combined with upper limb rehabilitation aids. The feature of this method is to improve the control power, and also to enhance the brainwave signal clarity22,23,24,25,26) so that the rehabilitation robot can be controlled. The sensor is sampled at 512 Hz and is controlled using the mass frequency variation of the θ value of the forehead (Fp1) and the grounding device clip on the right earlobe (A1). Continue with Fast Fourier Transform (FFT) and Magnitude squared coherence (MSC) and compare the performance differences between the two.

The main structure uses a motor-driven linkage to perform two-dimensional planar movements, which can also follow the trajectory set by the therapist, as shown in Fig. 1. We also give an FMA assessment before and after the imaginary training, which focuses on the mobility of the shoulder and elbow joints.

Fig. 1.

BCI training interface.

In the visual evoked potential system, the computer can produce visual stimulus event situations and meditation. With color (red, white, and blue) changes across the sphere to simulate the upper limb imagine the operation trajectory of the movement. This is followed by the signal analysis of the original brain wave to drive the rehab robot.

The first step is to FFT the induced brain wave signal, and obtain the corresponding comparable spectral analysis and phase-frequency components. Such as Eq. (1) is shown in the sampling rate, calculation length of the data by fast Fourier transform, and the resulting logarithmic spectrum.

(1)

The second step is to use the hompathic frequency function. It can be used to evaluate the brainwave signal and the reference signal. Based on the analysis of the linear correlation on the specific frequency, the time domain signal into frequency domain signal will be to compute the correlation. Will preserve the phase characteristics of signal and compare, see Eq. (2).

(2)

Saa is the power spectral density of the brain wave signal, Sbb is the power spectral density of the reference signal, and Sab is the power spectral density between the brain wave signal and the reference signal. The two groups of data are divided into n segments for homology analysis. In frequency domain conversion, the phase feature is not retained, so the reference signal is composed of the sine wave, see Eq. (3).

(3)

Besides, the modular architecture consists of an input and decision layer and consists of a multi-layered perceptron. The number of input layers is determined by the feature extraction method. The number of neurons in the input and output layers of a single module can be adjusted by the number of input parameters and the final classification result. The number of hidden layers and the number of neural elements in each hidden layer can be fine-tuned according to the characteristics of different input data. Figure 2 is a modular architecture.

Fig. 2.

Modular architecture diagram.

Next, the input layer of the system is corresponding to n feature extraction methods. m is the total number of input parameters, and each input parameter is connected to the corresponding input layer, and then the results of each input layer are extracted to connect to the decision layer for integration. The p-signature frequency of the visual evoked potentials was measured to be successfully corresponded to. It also improves poor potential recognition and accuracy.

The experiment recruited 200 normal participants and 8 stroke patients. It also uses sitting in front of the rehab robot and visually induced stimulation through a computer screen (including 5 stimulation frequencies: 6 Hz, 7 Hz, 8 Hz, 9 Hz, and 10 Hz). The sampling rate is set to 100 Hz. Each participant had a visual stimulation of 5 frequencies for 30 s. The NeuroSky brainwave headset is also used to capture forehead (Fp1) and right earlobe (A1) signals for control signatures and data collection of visual evoked potentials. The patient’s hand is fixed on the rehabilitation robot, and the variable resistance is fixed on the rod connecting the elbow joint. It can record the resistance value and angle change of the passive joint movement. To avoid the patient’s hand moving too fast, there will be muscle antagonistic tension. We will set the training speed at 5 m/s.

Besides, the system takes the signal source every 1 s as a set of input parameters. Thus, a group of frequencies of each participant will get 20 input parameters, and the spectrum parameters are obtained by using a fast Fourier transform and 256 points. In the conversion, Hamming window function is used, and the window size is set to 30, and the overlap rate is 20. When BCI imagining movement starts, the patient starts imagining movement according to the visual instructions on the screen. The single training time is 20 s. During the imagining period, if the ball with flashing color (red, white, and blue) moves in the target area for 4 times, the robot will move and drive back to the origin. But, if it does not reach 4 times in 20 s, it means failure. BCI imagining training lasts for 4 weeks, 4 times a week each time, about 40–60 minutes of training, 5 times each time, and record the elbow angle change and torque. The experiment was conducted in accordance with Helsinki’s latest statement and received verbal consent from the participants prior to the start of the experiment.

To master the effectiveness of each BCI imagination training, the number and time of imagination success are analyzed. The effectiveness of the stroke recovery index FMA and spastic strength indicator MAS for 4 weeks was also evaluated. At the same time, set the image accuracy and imagine the success time ability indicators. AR represents the accuracy of the same side hand, and the NR as the number of imaginary successes and the total number of training sessions per week (1 record per successful drive robot), see Eq. (4).

(4)

RESULTS

In BCI Imagination training, we used the same equipment to collect data on visually induced power generation in 200 participants and eight stroke patients. The training effectiveness was assessed by comparing the characteristic extraction effectiveness, modularity effectiveness, FMA, and MAS after 4 weeks of training on the same side hand.

First, the participants were evaluated for extracting brainwave signals, using FFT and MSC feature extraction methods and combining five visual stimulation frequencies, as in Fig. 3.

Fig. 3.

Identification rate of five stimulation frequencies.

As can be seen from the figure above, the recognition rate decreases as the flicker frequency increases. Therefore, the identification rate of this experiment is better at 6 Hz. The average recognition rates of these two feature extraction methods were FFT (65.78%) and MSC (60.58%). Among them, the MSC identification rate decline is more serious.

For the modular model, the input layer and decision layer are constructed to experiment with two characteristic extraction methods (FFT and MSC), as well as two machine learning modules Neural Network (NN) and restricted Boltzmann machine (RBM) to evaluate the effectiveness and test the number of different neurons.

According to Fig. 4, when the number of neurons is equal to 5, the recognition rate of the four methods is close to saturation. Therefore, set 5 as the number of neurons in the input layer, as in Fig. 5. Then compare the performance differences between direct recognition and machine learning.

Fig. 4.

Neuron number assessment.

Fig. 5.

Evaluation of decision-making effectiveness.

As shown in Table 1, the two feature extraction methods can improve the recognition rate through NN and RBM machine learning methods. The performance of FFT using RBM is higher (12.33%), while MSC uses RBM (8.49%). Overall, the recognition rate of FFT using RBM is the highest (75.54%).

Table 1.

Machine learning performance evaluation

Method	FFT	MSC
Direct identification	63.21%	56.43%
NN	70.55%	60.25%
RBM	75.54%	64.92%

FFT: Fast Fourier Transform; MSC: Magnitude squared coherence; NN: Neural Network; RBM: Restricted Boltzmann machine.

The next step is to evaluate the performance of the modular model. First, only the decision-making layer is used to compare with the modular model proposed in this study, including NN and RBM, as in Fig. 6.

Fig. 6.

Input layer module and parameter optimization.

As can be seen from the figure above, all four methods are saturated when the number of neurons is close to 5. Therefore, this data is used to compare the effectiveness of decision-making levels.

When we integrate the modular architecture and optimize the input parameters, as can be seen from Table 2, the effect is better than direct integration. The two feature extraction methods use the corresponding machine learning modules to optimize the input layer and compare the number of neurons at the decision layer.

Table 2.

Impact and differences at decision-making levels

Input layer	Decision-making layer	Recognition rate
RBM	RBM	88.7%
NONE	RBM	83.9%
NN	NN	83.8%
NONE	NN	81.6%

RBM: Restricted Boltzmann machine; NN: Neural Network.

As can be seen from Fig. 6, the two methods in the number of neurons are equal to 5, its recognition rate is close to saturation. Therefore, this data is used to compare the performance of the input layer.

As can be seen from Table 3, the two feature extraction methods combined with the machine learning model as an input layer. The difference compared to not using optimization is 1.3%−9.1%.

Table 3.

Differences in input layer modules

Input layer	Decision-making layer	Recognition rate
N (F), R (FM)	RBM	90.0%
RBM	RBM	88.7%
N (F), R (FM)	NN	92.9%
NN	NN	83.8%

RBM: Restricted Boltzmann machine; NN: Neural Network.

According to the experimental plan, we used the optimal flicker frequency (6 Hz) and FFT signal acquisition method to train 200 participants (N1-N200) and eight stroke patients (S1-S8) in BCI control for 4 weeks, with 4 sessions per week, 5 sessions per hand (ipsilateral), and simultaneous pre-processing of the acquired raw brainwave signals to recognize the imaginary hand movements.

In the first week of the N1-N200 experiment, the inertial side hand drove the robot in an average of 10 s. The Y-axis movement was about 35 cm and the average energy ratio of Fp1 was 1.2. By the fourth week, the time taken improved to 6.1 s. The Y-axis movement was about 40 cm and the average energy ratio of Fp1 was 1.9. This shows that the time required to imagine success decreased compared with the initial period, and the Y-axis movement distance and energy ratio both increased.

After 4 weeks of training, the number of seconds decreased to 9 seconds, the Y-axis movement was 32 cm, and the average energy ratio of Fp1 was 1.5. The data showed that the system could help the patient to perform the imaginary movements more and bring about good rehabilitation results, shown in Table 4.

Table 4.

Average training results for weeks 1–4

No. of participants	Data	WK-1	WK-2	WK-3	WK-4	Mean
N1-N200	Fp1	1.2	1.5	1.7	1.9	1.575
	Time (s)	10	8.5	7.3	6.1	7.975
	y position (cm)	35	37	38	40	37.5
S1-S8	Fp1	1.1	1.3	1.4	1.5	1.325
	Time (s)	14	13	11	9	11.75
	y position (cm)	28	30	31	32	30.25

WK: week.

Stroke patients are given an FMA assessment before and after each BCI imagination training session, with the project focusing on upper limb movements (total score of 66) and more elbow movements represented by FMAprox (total score of 42), scored on a scale of 0, 1 and 2. MAS is evaluated 40 minutes after each training session, with scores of 0, 1, 2, 3, 4, and 5.

The results of the BCI pre and post-test assessments of S1-S8 at week 1 and week 4, where the FMA assessment increased by 6 points and the FMAprox increased by 4 points. The MAS section also decreased from 2 points before training to 1 point, representing a significant effect after training, shown in Table 5.

Table 5.

FMA assessment scale before and after training.

Data	Before training	After training
FMA	25/66	31/66
FMAprox	21/42	25/42
MAS	2	1

FMA: Fugl-Meyer Assessment; MAS: Modified Ashworth Scale.

Patients with a hemorrhagic stroke use fMRI to detect changes in local blood flow caused by nerve activity in the brain. Figure 7 (a) shows the black area (premotor cortex) where the bleeding position before training is present. After four weeks of BCI rehabilitation training, the activation site during side movement is concentrated on both sides of the frontal motor cortex (premotor cortex), and there is a significant reduction in the activation position during movement, as in Fig. 7 (b). Thus, the fMRI experiment has shown that this BCI rehabilitation training is effective in reshaping the brains of patients.

Fig. 7.

Brain activation area. (a) pre-training, (b) post-training.

DISCUSSION

The goal of this study, besides improving the inconvenience of traditional electrode caps, is to provide rehabilitation exercises for patients with impaired spinal cord or central nervous system who are unable to exercise. Combining upper limb rehabilitation robots with visual stimulation evoked potentials can increase the willingness of self-training and reduce the manpower burden of the therapist.

The optimal visual evoked flicker frequency of this system is 6 Hz, and the feature extraction method is enhanced by modular machine learning technology. The results show that the recognition rate of FFT is higher than that of the MSC method is about 5.2%. For different feature extraction methods, using the optimized model can improve the recognition rate of 1.3%−9.1%. Besides, according to the FMA clinical indicators of stroke patients, the score of shoulder and elbow function is increased by 5 points, while the score of MAS is reduced to 1 point. It was learned that the patient after four weeks of rehabilitation training and modular model performance evaluation, besides the fMRI experimental results showed a significant reduction in the brain activation position trend, but also confirmed that the enhanced feature extraction method can make the shoulder elbow has a significant recovery effect.

Finally, due to the different upper limb extension conditions and restrictions of ordinary individuals and patients, and the stretching trajectory of the robot is too monotonous. Thus, it may be necessary to design training steps separately to increase the effectiveness of training. In the future, virtual reality will be designed to help imagine movement, and it is expected to enhance the stimulation of the cerebral cortex to enhance the training effect27, 28). The authors declare no conflicts of interest.

Conflict of interest

None.