A Study of Various Feature Extraction Methods on a Motor Imagery Based Brain Computer Interface System.

INTRODUCTION: Brain Computer Interface (BCI) systems based on Movement Imagination (MI) are widely used in recent decades. Separate feature extraction methods are employed in the MI data sets and classified in Virtual Reality (VR) environments for real-time applications.
METHODS: This study applied wide variety of features on the recorded data using Linear Discriminant Analysis (LDA) classifier to select the best feature sets in the offline mode. The data set was recorded in 3-class tasks of the left hand, the right hand, and the foot motor imagery.
RESULTS: The experimental results showed that Auto-Regressive (AR), Mean Absolute Value (MAV), and Band Power (BP) features have higher accuracy values,75% more than those for the other features. DISCUSSION: These features were selected for the designed real-time navigation. The corresponding results revealed the subject-specific nature of the MI-based BCI system; however, the Power Spectral Density (PSD) based α-BP feature had the highest averaged accuracy.

Introduction

Brain Computer Interface (BCI) is a new communication channel between brain and computer. Although this communication is available for everyone, the disabled and paralyzed patients use it the most. Using this interface, the brain activities are recorded from the scalp in a specified physiological state. Then, they are translated to appropriate external commands in a computer. Different modalities of Electroencephalography (EEG), Magneto-encephalography (MEG), and functional Magnetic Resonance Imaging (fMRI) are used in BCI, which the former is more applicable due to its lower experimental cost and a spontaneous and evoked signals are used in EEG-based BCIs. Usually, 5 different methods are defined in the context of the BCI using the above signals. The mental task, the Movement Imagination (MI) and the Slow Cortical Potential (SCP) methods employ the spontaneous signals, whereas the visual evoked potential and the P300 methods employ the evoked ones (Chin et al., 2010; Resalat, S. N. et al., 2012). MI, also known as motor imagery, is one of the most enjoyable methods of the BCI systems because of its voluntarily generated signals by the subject without external stimulation, even when it is accompanied with the virtual reality. Virtual Reality (VR) is a graphical interface (medium), which is defined in terms of special collection on technological hardware, including computer, head-mounted monitors, usually headphones, and motion sensing electrodes. Most current VR environments are visual experiences, but some are based on the sound simulations through the headphones. VR is often used to describe a wide variety of applications commonly associated with immersive, highly visual, and 3D environments (Jingkun et al., 2010). All in all, VR in MI-based BCI could be a graphical interface designed in a computer to not only attract the subjects but also send feedback signals to notify them of their recent MI whether it is performed correctly or not.

For an individual, performing motor imagery consists of imagining the movement on the limbs. Therefore, the resulting signals contain specific temporal and spectral features, which make the movement easy to recognize. For example, the left hand imagination decreases the power of the β rhythm of the EEG over the right motor cortex. Therefore, detecting this decrement could be translated into left hand imagination. This phenomenon appears when the subject imagines the right hand movement. In MI-based BCI, once the MI is detected, the corresponding command will get associated and executed (Lecuyer et al., 2008). An MI task is analyzed by the frequency component of the EEG signals (Ming et al., 2009). A VR-based MI is performed by the entropy feature and, an MI-based BCI is administered by the use of EEG powers in α and β rhythms (Leeb R. et al., 2007).

To analyze and detect desired information in EEG signals, different features and classifiers are used. For decoding and classification of objects through task-oriented EEG signals, Taghizadeh-Sarabi et al. used 3 different wavelets namely Daubechies 4, Haar, and Symlet2 for feature extraction method. Selected features were classified by the one-against-one Support Vector Machine (SVM) multi-class classifier. The results indicated that symlet 2 had higher performance than the two other methods (Taghizadeh-Sarabi et al., 2015). In another study, Daliri proposed a new kernel approach based on the Earth Mover’s Distance (EMD) for EEG signal classification. The results showed that the new kernel method was very effective, and could classify the data with higher accuracy than traditional methods (Daliri, 2013). There are many other studies in the field of BCI while selecting the best feature and classifier is the main purpose of them (Hashemi et al., 2013; Resalat, S. N. et al., 2015).

In this work, the performance of several widely used features were examined and compared to specify the best features. The effective features over different feature sets such as statistical features, the frequency amplitudes and components, entropy and band powers, auto-regressive, moving average, discrete cosine, and sine transform were investigated using a VR environment in an MI-based BCI system. In addition, this work, we revealed a military VR for paralyzed veterans due to lack of such VR.

In section two, the materials and the methods used in this study are introduced. Section three presents the experimental results and in section four we discuss the results and make a conclusion of this entire study.

Methods

In this research, we implemented a thorough study of different feature sets, which are mostly used in MI-based BCI systems, with the Linear Discriminant Analysis (LDA) classifier. LDA was selected due to its fastest response time among the other available classifiers (Resalat S. N. et al., 2015; Scherer et al., 2008).

Electrode montage and the subjects

The participants were selected from university students studying postgraduate degree, master of science (MSc); they were in good health condition with no history of diseases. Five male right-handed subjects with the mean age of 25.6±2.3 years participated in the experiments. The study were carried out in the Biomedical Engineering Laboratory of the Science and Research, Branch of the Azad University. All of the subjects were informed about the experiments and signed the consent form. Three channels were positioned at C3, Cz and C4 according to the international 10–20 system for EEG recording and the ground electrode was placed on the right ear lobe. EEG signal acquisition was performed by ProComp Infiniti™ device. The impedance of the electrodes, which were less than 1KΩ, was intentionally measured to ensure the well-recorded data (Ron-Angevin et al., 2005). The sampling frequency of the EEG recording system was 256 Hz. Figure 1 (a) displays the positions of the positive to the negative electrodes.

Figure 1.

(a) The position of the electrodes in the experimental setup; (b) The proposed BCI design for navigation in the virtual environment; (c) Timing of each trial in offline processing.

Experimental groups

The MI-based BCI system consisted of two network computers; one computer was dedicated for EEG signal acquisition and processing, and another one for rendering the virtual reality environment. Two computers of the workstations were connected via TCP/IP protocol. The scheme of the BCI was illustrated in Figure 1 (b). Three different movement imagination tasks of the left hand, right hand, and feet were introduced to the subjects. It should be noted that all subjects were familiar with the tasks due to their previous experiences. These familiarity reduced the training time of the experiments in offline processing, therefore the navigation took place in real-time application.

To acquire the EEG-based MI signals, designing of an experimental protocol was necessary. Usually this step includes several runs and each run includes several trials. A trial timing of a typical experimental protocol with 3 different cues is illustrated in Figure 1 (c). In this protocol, a fixation cross was displayed to the subject indicating the resting period. Next, a beep sound was played to alert the subject. Then, a red cue was displayed to the subject indicating the type of MI the subject should perform.

For example, in Figure 1 (c), a red cue toward right, left, and down asks the subject to perform left hand, right hand, and foot motor imagery, respectively. It should be noted that, each trial lasted for 10 seconds and 36 number of trials, producing 6-minute data, was considered 1 run. Finally, each subject performed 6 runs and accomplished the proposed experiments within 3 sessions. In order to avoid the subject’s bias toward the cues and trials, the type of movement imagination was selected randomly. Thus, with the help of a designed protocol, related MI signals were acquired for further processing and feature extraction. The experimental protocol of trials was simulated with virtual reality toolbox of MATLAB and was shown to the subject in VR workstation.

Signal analysis

The signal analysis consisted of two phases of feature extraction and classification, which are described as follows:

From each trail, four 2-s segments using rectangular window with no overlap were taken out (Figure 1) and 12 different features were applied to each 2-s segmented trial of the recorded EEG, which are described as follows:

Mean Absolute Value (MAV): This feature computes the absolute value of all values in a specified window and then determines the mean of the resultant values, which is calculated as follows:

Where, n is the number of samples in each segment.

Fast Fourier Transform (FFT): The sorted FFT amplitude and the sorted FFT component of the segmented signal are defined as features.

The Auto-Regressive (AR) Model: Forward-backward Autoregressive (AR) model of order m is developed and the resulting coefficients are used as signal features. The AR Model is as follows:

In Equation 2, α s are the AR coefficients, m is the model order and E is an additive white noise with a zero mean and finite variance (Pineda et al., 2003).

Moving Average (MA) Model: In time series analysis, the moving average (MA) model is a common approach for modeling univariate time series models as:

Where, μ is the mean of the series, the θ1, …, θq are the model parameters and the ε(t), …, ε(t-q) are white noise error terms. The value of q is the model order.

The Auto-Regressive-Moving-Average (ARMA) model: ARMA (m, q) refers to the model with m autoregressive and q moving-average terms. This model contains the AR(m) and MA(q) models:

Alpha-Band Power (α-BP): This feature is computed using Short Time Fourier Transform (STFT), Power Spectral Density (PSD), and FFT in a frequency bandwidth of 8 Hz to 12 Hz.

Entropy: the concept of entropy is known from Shannon’s theory. It is a measure of the uncertainty (randomness) of the feature vectors. It is defined as follows:

Where, p(x) is the estimate distribution of the features that exhibit the highest possible randomness.

Discrete Cosine Transform (DCT): Given N input samples x(0), x(1), …, x(N-1) in a segmented signal, the corresponding DCT is computed as follows: where = if k=0, otherwise

Discrete Sine Transform (DST): The DST feature of a segmented signal is defined as follows:

The Linear Discriminant Analysis (LDA) classifier is used to classify the whole data set for all subjects. It uses a hyperplane to separate the different classes. For a two-class problem, the class of a feature vector depends on which side of the hyperplane the vector is. LDA presumes a normal distribution of the data, with equal covariance matrices for both classes. The separating hyperplane is obtained by seeking the projection that maximizes the distance between the two classes’ means and minimizes the inter-class variance (Leeb R. et al., 2007). The accuracy of LDA is calculated with the 10×10 fold cross-validation strategy. This strategy divides the data into 10 folds while considering 9 of them for training and the other one for testing the classifier. The outputs of the testing phases are averaged within 10 iterations so that in each of them, the testing fold varies. Finally, this procedure repeats for 10 epochs and the accuracy is averaged within these epochs.

Results

In this section, the results of the various feature extraction methods described in the previous section are presented. Results are reported in terms of the accuracy of the LDA classifier, with the 10×10 fold cross-validation strategy. As Figure 2 shows, the MAV (the statistical), AR, α- BP-PSD, and α-BP-FFT features outperform other feature extraction methods and have accuracies above 75%. Therefore, 4 different virtual environment navigation based on these 4 kinds of feature extraction methods were performed. The subjects were asked to navigate the same route in the virtual environment 4 times while the feature extraction method in each experiment was different. The movement imagination of the left hand, right hand, and foot corresponds to moving left, right, and forward in the virtual environment, respectively. This correspondence exists unless the subject reaches to a place, which the enemies can be seen. In this situation, the foot motor imagery corresponds to shooting.

Figure 2.

The accuracy of different feature extraction methods for LDA classifier obtained with 10×10 fold cross-validation strategy; X is the index of feature extraction method, Y is the accuracy, L and U are the lower limit and the upper limit of the accuracy defined by the standard deviation, respectively.

Different snapshots of the simulated virtual environment are shown in Figure 3. There are some main positions in the virtual environment, based on which the EEG signals of the subject will be classified. A sample position can be seen in the Figure 3 (a). As it can be seen, there are also two arrows at the bottom of the screen to show feedback to the subject. If the output of the classifier was true, the green arrow would start to fill by green color, and if the output of the classifier was false, the red arrow would start to fill by red color. If the green arrow filled, the subject would start to move as shown in Figure 3 (b). The new main position is shown in Figure 3 (c). Results of the navigation of the subjects in the virtual environment are tabulated in the Table 1. All subjects were able to navigate in the virtual environment, and as can be seen in the Table 1, for different subjects, different feature extraction methods work well, i.e. the feature extraction methods are subject-specific.

Figure 3.

Different snapshots of the virtual environment, in the (a) beginning of the main position, (b) end of the main position, (c) beginning of the second main position.

Table 1.

Classification results for different feature extraction methods and subjects.

Subjects	Feature extraction methods
	MAV	AR	α-BP-PSD	α-BP-FFT
S1	67	74.2	70	70.7
S2	73.4	65	71.2	69
S3	68.3	70.5	74.6	72.5
S4	70.2	67.3	75.7	74.5
S5	63.4	64.8	67.3	64
Ave.	68.5	68.4	71.8	70.1

Discussion

There are many various feature extraction methods that could be used in MI based BCI. In this work, the performance of the most widely used methods were examined and compared with each other to specify appropriate methods. To do so, different feature extraction methods such as MAV, sorted FFT amplitude and component, AR, MA, ARMA, α-BP, DCT, and DST using the LDA classifier on an MI-based BCI system were developed. It should be noted that our preliminary study revealed that MAV feature had higher accuracy rate compare to the other statistical features such as mean, standard deviation, second, third, and fourth cumulants and moments. This is the same for 7 sorted frequency components and 10 sorted frequency amplitudes among the other combinations. Moreover, AR(5), MA(10), and ARMA(1, 10) had the highest accuracy over the other orders in each corresponding method. Finally, the first 8DST and the first 6DCT coefficients outperformed the other number of selected coefficients. As it can be seen in Figure 2, AR with the model order of 5 outperformed the other available features. This amount of accuracy is almost high enough in a 3-class classification. Beside the AR order, the MAV, α-BP using PSD, and FFT had also accuracies higher than 75%.

Therefore, these 4 feature sets were selected for the real-time application, which its environment is displayed by the images of Figure 3. These figures, from the left, guide the subjects to navigate to the right or to the left using the arrows as the feedback sign. When the target was selected, the foot motor imagery was considered for shooting. The same 5 subjects in offline analysis carried this real-time application and their corresponding accuracies are shown in Table 1. As it can be seen, in real-time application, the overall accuracy was lower than that in the offline mode. This could be due to the graphical VR that distracted the full attention of the subjects and the lower length of the segments in the online approach, which was 2 seconds for each segment.

However, in real-time application, the α-BP feature using PSD could be selected as the best feature in MI-based application. Moreover, the accuracy of the subjects varies among the features. For example, the accuracy of the S1 was the highest using the AR method while the accuracy of the S2 was the best with MAV feature. Therefore, the accuracy of the MI-based BCIs is subject-specific. In this study, a virtual environment was simulated to show the subject a virtual reality-based feedback while performing 3 movement imageries of left hand, right hand, and feet. The results indicated that the subjects were able to navigate into the virtual environment.