Enhanced inter-subject brain computer interface with associative sensorimotor oscillations.

Electroencephalography (EEG) captures electrophysiological signatures of cortical events from the scalp with high-dimensional electrode montages. Usually, excessive sources produce outliers and potentially affect the actual event related sources. Besides, EEG manifests inherent inter-subject variability of the brain dynamics, at the resting state and/or under the performance of task(s), caused probably due to the instantaneous fluctuation of psychophysiological states. A wavelet coherence (WC) analysis for optimally selecting associative inter-subject channels is proposed here and is being used to boost performances of motor imagery (MI)-based inter-subject brain computer interface (BCI). The underlying hypothesis is that optimally associative inter-subject channels can reduce the effects of outliers and, thus, eliminate dissimilar cortical patterns. The proposed approach has been tested on the dataset IVa from BCI competition III, including EEG data acquired from five healthy subjects who were given visual cues to perform 280 trials of MI for the right hand and right foot. Experimental results have shown increased classification accuracy (81.79%) using the WC-based selected 16 channels compared to the one (56.79%) achieved using all the available 118 channels. The associative channels lie mostly around the sensorimotor regions of the brain, reinforced by the previous literature, describing spatial brain dynamics during sensorimotor oscillations. Apparently, the proposed approach paves the way for optimised EEG channel selection that could boost further the efficiency and real-time performance of BCI systems.

Introduction

As an unconventional communication pathway, brain computer interface (BCI) enables us to communicate with a computer or with other external devices without any muscular stimulation. Although the primitive goal of developing BCI was to assist physically disabled people experiencing motor function abnormalities, recent technological advancements augment BCI in many other applications, including lie detection [1], brain fingerprinting [2], mood assessment [3] and gaming [4]. Most of the proposed BCIs are subject-specific and require time-consuming, sometimes frustrating calibration sessions. Thus, inter-subject BCIs are desired; yet development of such BCIs come across challenges including the inherent variabilities in brain dynamics across subjects due to the diversity in individual brain growth [5]. Moreover, multichannel electroencephalogram (EEG) that captures the electrical activity of the brain, suffers from the effect of outliers due to excessive channels, causing, at the same time, high computational burden to the BCI system. However, developing an inter-subject BCI with subjects who share associative neural oscillations for particular cognitive task, seems more feasible. Previous studies addressed inter-subject association of neural dynamics during natural vision [6] and natural music listening [7]. During motor imagery (MI)-based inter-subject BCI, it is important to measure sensorimotor synchronisation across subjects. MI is the kinesthetic imagination of a motor task, which shares equivalent sensorimotor oscillations corresponding to actual motor execution [8]. Movement-related cortical potential-based BCI without subject-specific training has been proposed in [9]. In [10], an inter-subject BCI has been developed for modifying mental states that can be used for treating major depressive disorder. Rana et al. [11] have developed a real-time toolbox for implementing inter- and intra-subject BCI using functional magnetic resonance imaging. An online inter-subject BCI with P300 speller paradigm has shown the deviation of inter-subject evoked potentials [12]. Learning from subspaces that have been estimated via common spatial pattern (CSP) applied on inter-subject/session data of similar characteristics, i.e. non-stationarities, can enhance the performance of BCI [13, 14]. In these experiments, selecting suitable subjects is critical due to the fact that the brain dynamics significantly vary across subjects. However, these methods perform well, specifically in the context of small training trials available from the target subject [15, 16]. Another study has proposed ensemble of classifiers, which can be used for single-trial classification without explicitly being trained [17]. In [18], sparse common spatial pattern is proposed as a novel method for optimal channel selection technique within a constraint of optimal classification accuracy. In this Letter, a coherence analysis in time–frequency (T–F) space is proposed to select the set of EEG channels, who have relatively high normalised coherence power. The underlying hypothesis is that highly coherent and common inter-subject EEG channels can improve BCI performances, by reducing the effect of outliers from undesired channels. To achieve this, the wavelet coherence (WC) has been adopted as a means to measure coherence between two time series in T–F domain, since it has successfully being used as a tool of measuring couplings between brain regions using sensory-evoked potentials [19] and to the associativity assessment of inter-personal brain activity [20].

Methods

At first, the available multichannel EEG data (Section 2.1) were preprocessed using CSP [21]. Then, wavelet decomposition (up to the third level) was applied, using the Daubechies three-sample filter (db3). At each decomposition level, subband energy and subband entropy were used as features for the classification of the MI types, which was realised via a two-layer feed-forward neural network [22] and used to classify MIs.

WC was estimated to discern if the coherent power reveals common inter-subject regions, where the sensorimotor oscillations from two subjects co-vary. Based on the WC power (WCP), some channels were selected assuming that those channels will enhance the T–F synchronisation of two subjects. Then, a novel inter-subject BCI framework was proposed with a view to validate whether selected channels can improve the classification performances. The motivation behind was to eliminate undesired channels which introduce outliers. On the contrary, selected channels enhance sensorimotor coherence; thus, the pairwise inter-subject BCI performances.

Dataset and experimental settings

The multichannel EEG raw signals used for the validation of the proposed hypothesis are those included in the dataset IVa from BCI Competition III (http://www.bbci.de/competition/iii/desc_IVa.html). One hundred and eighteen channels in total (extended 10/20 system) were employed to capture the multichannel EEG signals (Fig. 1). During the recording session, the sampling rate was set at 1kHz, which was later downsampled to 100 Hz. While sitting on a comfortable chair with arms resting on armrests, subjects were asked to perform each one of the three MIs, i.e. left hand, right hand (RH) and right foot (RF); the publicly provided data, however, refer to two MI classes only, i.e. RH and RF. Single-trial EEG data were captured from five healthy individuals, by providing them class-specific visual cues before the kinesthetic imagination of the motor tasks. The data consist of 280 trials (140 trials for each MI class) for each subject and each trial has 3.5 s of EEG recordings for any of the two MIs. It should be noted that 2.5 s of data for each trial after 1 s of the corresponding visual cues were chosen for analysis.

Fig. 1

Electrode montages (extended 10/20 system) and selected inter-subject associative channels for subject air al-aw

The proposed BCI framework is studied in pairwise manner, i.e. signals from one subject are used to train and validate the algorithm whereas signals from another subject are used to test the algorithm. Thus, there are total of 560 trials for experimenting inter-subject BCI. The first 180 trials are used to train the classifier, whereas the consecutive 100 trials are used to validate it, so to avoid over-fitting. Finally, the rest 280 trials from another subject are used to test the classifier.

Common spatial pattern

In 2000, Ramoser et al. proposed CSP, as a spatial filtering method, for classifying two classes of single-trial MIs for the first instance [21]. Usually, CSP estimates projection of multichannel EEG signal, so that the difference between two classes is maximised. Suppose and represent the two MIs, i.e. RH and RF, respectively, which have been considered in this experiment. Then, CSP approximates a weight matrix that can be used to filter simultaneously recorded multichannel data. Estimating requires simultaneous diagonalisation of two covariance matrices for two classes as follows where and represent average covariance matrices over the training trials, whereas and are the diagonal matrices that must satisfy , where is the identity matrix.

Finally, the original EEG signals can be projected as where E and represent the EEG signal and spatially filtered EEG signal, respectively, both having N number of channel components and each component constitutes of P number of samples per trial. While the first components of attribute to maximal discriminative features for and minimal for , the last components of attribute to minimal discriminative features for and maximal for . More details on CSP can be found in [21].

Wavelet coherence and channel selection

Suppose x(t) represents a time-domain signal, where . Then, the continuous wavelet transform is defined as [23] where * represents the complex conjugate and is the wavelet basis function (mother wavelet). Usually, is dilated by a factor a, a > 0, whereas translated by a factor b, and these time and scale parameters (a, b) are continuous. In this study, the complex Morlet wavelet was adopted as the mother wavelet given by [23] where is a bandwidth parameter and is the wavelet centre frequency. As a Gaussian-windowed complex sinusoid, the adopted complex Morlet wavelet with its second-order exponential decay results in very good time localisation during the wavelet transform [23]. Additionally, this wavelet basis function gives information about both amplitude and phase, thus it is better used for capturing coherence between oscillatory harmonics in T–F space [23]. The values of and were set to 1 and 0.9549 Hz, respectively.

is the cross wavelet transform (XWT) which finds the T–F regions where two signals co-vary, but have high power in contrast to WC. Suppose and are two time signals (where n = 1, 2, …, N), then the XWT is defined as where s represents scale that is used to stretch the mother wavelet along time.

WC reveals associativity of two time-domain signals in T–F spaces. This similarity measure finds the T–F locations where the signals significantly co-vary, but does not necessarily have high power. The WC is defined as follows [24] where S is a smoothing operator and defined as and represent smoothing along scale/frequency and time axes, respectively. Fig. 2 shows the calculation of WC between two electrophysiological time series.

Fig. 2

Calculation of WC between inter-subject sensorimotor oscillations (i.e. regarding RH MI task) for two different channels corresponding to subject pair al-aw

a The channel FC5 has the highest value

b The channel Fpz has the lowest value. The arrows in T–F WC plot indicate the phase relationship of two time series having WCP > 0.9. The range of has only been considered significant while measuring inter-subject sensorimotor coherence

The single-trial EEG signals from two subjects have been considered as two sets. From each set, the trials are further categorised according to classes, i.e. RH or RF. Consider S1 and S2 as two individuals, each having total M trials and M/2 trials for each class. The WC has been applied to measure similarities of M/2 trials from S1 with M/2 trials from S2 one by one. The WCP is normalised to 1 and thus the average WCP ranges from 0 to 1. The accumulated WCP over trials are averaged and denoted as , given by where . The in the range from 0.6 to 0.9 has been considered significant and used to measure associativity of inter-subject channels, i.e. . The implicit assumption is that very high coherence power (>0.9) might represent resting EEG dynamics, irrespective of any information regarding cortical events.

Finally, the estimated is used to rank the channels in descending order. Eight channels were selected for each class, accumulating to 16 channels for two classes. If any channel falls within the selection criteria for both classes, we have selected another channel with higher either for RH or RF, thus limiting the number of selected channels to 16. The selected channels are specified in Table 1.

Table 1

Selected associative inter-subject channels for different subject pairs along with corresponding WCPN

S1–S2	Selected channels
aa-al	RH→C1(0.207) CCP3(0.203) P1(0.201) FFC8(0.201) FFC7(0.201) F7(0.200) FT10(0.200) Pz(0.200) PCP1(0.199)RF→CCP8(0.210) C6(0.207) O1(0.204) T8(0.201) Oz(0.199) C1(0.199) FT10(0.199) PCP8(0.199) PO4(0.199)
aa-av	RH→P7(0.203) PPO7(0.202) P6(0.201) CP6(0.201) F6(0.201) Fz(0.201) PCP7(0.200) P9(0.200)RF→CP3(0.203) FT8(0.201) FC2(0.201) CFC1(0.200) C6(0.200) OI2(0.200) FC4(0.200) CCP6(0.199)
aa-aw	RH→CFC3(0.195) FT10(0.194) CFC6(0.194) CCP5(0.193) CCP8(0.193) CCP1(0.192) C6(0.192) PCP1(0.191)RF→CP4(0.210) C1(0.207) PCP1(0.207) PCP2(0.207) PCP4(0.207) CP6(0.207) FFC2(0.207) Fz(0.205) PCP3(0.205)
aa-ay	RH→I2(0.202) PO7(0.202) I1(0.201) P1(0.201) P9(0.201) PO2(0.200) P10(0.199) P2(0.199) PCP2(0.199)RF→CP1(0.206) CPz(0.206) CCP2(0.204) Cz(0.203) T7(0.203) CCP1(0.202) P9(0.202) P1(0.202) PCP1(0.202)
al-av	RH→Cz(0.196) CFC8(0.195) FC6(0.195) TP9(0.194) TP10(0.192) P10(0.192) PPO8(0.192) C4(0.192)RF→PCP3(0.212) CP1(0.212) P3(0.211) O1(0.210) OI1(0.209) P1(0.208) P5(0.207) P3P4(0.207)
al-aw	RH→FC5(0.206) FFC5(0.205) PCP4(0.204) C5(0.203) CP2(0.203) PCP5(0.202) CP3(0.202) FC3(0.202) P4(0.201)RF→P1(0.218) PCP3(0.215) CP3(0.213) CP2(0.212) PO3(0.212) P3(0.212) CP1(0.211) P5(0.209) PCP2(0.209)
al-ay	RH→Oz(0.219) PO1(0.217) OPO1(0.215) PO2(0.214) PPO1(0.213) PO3(0.213) PPO2(0.212) OI2(0.212)RF→O1(0.209) PPO6(0.206) PCP3(0.205) CP3(0.205) I1(0.205) PCP5(0.204) OI1(0.204) CP5(0.203)
av-aw	RH→P5(0.197) PCP7(0.197) CCP5(0.196) P9(0.195) CP1(0.195) FT7(0.195) PPO5(0.195) PCP5(0.194) P3(0.194)RF→CCP5(0.209) C3(0.204) CFC6(0.204) C1(0.204) FC6(0.202) CP1(0.202) CCP3(0.201) FC3(0.201) CP3(0.200)
av-ay	RH→I2(0.198) OPO1(0.196) I1(0.196) FAF5(0.194) PPO8(0.194) FT8(0.193) FFC7(0.193) OI1(0.193)RF→CCP8(0.205) T8(0.204) CP5(0.204) C6(0.202) FAF6(0.202) CFC6(0.202) CCP6(0.201) PCP8(0.201)
aw-ay	RH→PCP1(0.209) CCP3(0.207) PCP3(0.207) Pz(0.207) CP3(0.206) P1(0.206) CPz(0.206) P5(0.202)RF→T7(0.213) CFC7(0.210) OPO1(0.206) CCP7(0.204) P7(0.203) Oz(0.203) FT9(0.203) FT7(0.202)

Results and discussions

Table 1 summarises the sets of mostly coherent channels for all subject pairs. The channels are reported according to values in descending order. Each set consists of sixteen different channels from different areas of the brain, where the inter-subject sensorimotor oscillations are mostly associative in T–F space. The common channels which fall within selection criteria for both RH and RF are indicated as bold. Fig. 1 shows the selected channels (electrode montages) for subject pair al-aw, most of which lie around sensorimotor regions of the brain. Sensorimotor regions are highly responsive during MIs [25] and share associative inter-subject information.

Table 2 describes the classification accuracies achieved from inter-subject experiments. In Case I, available 118 channels are used to classify the MIs while only 16 channels, reported in Table 1, are used in Case II. Let's consider, and represent classification accuracies achieved from Case I and Case II, respectively. The trials from subject S1 are used to train and validate the classifier while the trials from S2 are used to test the classifier. indicates the increased classification performances of inter-subject BCI by using associative channels only. However, the maximum classification accuracy achieved for subject pair aw-al is . But, the classification accuracy for this subject pair in Case I is . Interestingly, evinces the potential applicability of the proposed channel selection method. Also, the classification performances are not symmetric, i.e. interchanging of training and testing trials significantly influences the performances. For example, and etc. The achieved classification accuracy is significantly lower than . Such asymmetric Acc occurs, due to the fact that data-driven spatial filtering methods are prone to be overfitted while estimating unreliable parameters [26]. Although there are significant observations in which , some do not necessarily show any promising performance in any of the case. The cases in which are italicised in Table 2.

Table 2

Classification accuracies (%): inter-subject BCI (The cases in which Acc (S1 → S2) < Acc (S1 → S2) are italicised.)

S1–S2	Case I	Case II	S1–S2	Case I	Case II
aa-al	60.71	76.79	al-aa	56.43	57.14
aa-av	56.79	57.86	av-aa	53.57	61.07
aa-aw	57.86	65.36	aw-aa	62.86	51.43
aa-ay	58.21	51.43	ay-aa	50.36	49.64
al-av	49.64	47.14	av-al	70.71	54.29
al-aw	69.64	73.93	aw-al	56.79	81.79
al-ay	63.57	76.79	ay-al	67.50	67.50
av-aw	55.71	52.14	aw-av	53.57	50.36
av-ay	62.14	57.50	ay-av	51.79	53.21
aw-ay	63.21	53.57	ay-aw	52.50	64.29
mean	59.75	61.25	mean	57.61	59.07

Many time variant psychophysiological factors including attention, memory load, spontaneous cognitive processes etc. [27] and user's basic characteristics, such as lifestyle, gender, age etc. influence the individual brain dynamics over time [28], thus affect BCI performance. This inherent fluctuation of individual brain dynamics causes inter-subject variability that poses difficulties while developing BCI without subject-specific calibration. In EEG-based BCI, undesired channels additionally degrade performances by producing outliers [18]. Fig. 2 shows both highly associative and dissociative inter-subject sensorimotor oscillations and their corresponding WC. In this Letter, a novel T–F approach has been proposed to sort out associative sensorimotor inter-subject channels while most of the literatures have addressed the adaptation of machine learning algorithms for compensating variabilities [13-17]. Results show that selecting inter-subject coherent channels can significantly increase performances, thus implicating future development of efficient inter-subject BCI paradigm. An interesting aspect that arises from the proposed approach is the possibility to generalise it towards finding common spatial components (i.e. a subspace of the data) instead of selecting individual electrodes; yet, this is left as a future endeavour.

Conclusions

Inter-subject BCI, without time-consuming, sometimes frustrating calibration session, seems more convenient to users. Nevertheless, inherent instantaneous variability in EEG signals poses significant challenges. Outliers that have been generated within insignificant channels may be due to diversity in psychophysiological or other factors, limit the development of BCI in subject independent context. Interestingly, the only channels which share similar sensorimotor dynamics can be employed for improving BCI performance. This study delineates a novel method for selecting inter-subject associative channels based on WC and show how BCI performances can be improved in subject independent settings.