The emotion recognition system based on autoregressive model and sequential forward feature selection of electroencephalogram signals.

Electroencephalogram (EEG) is one of the useful biological signals to distinguish different brain diseases and mental states. In recent years, detecting different emotional states from biological signals has been merged more attention by researchers and several feature extraction methods and classifiers are suggested to recognize emotions from EEG signals. In this research, we introduce an emotion recognition system using autoregressive (AR) model, sequential forward feature selection (SFS) and K-nearest neighbor (KNN) classifier using EEG signals during emotional audio-visual inductions. The main purpose of this paper is to investigate the performance of AR features in the classification of emotional states. To achieve this goal, a distinguished AR method (Burg's method) based on Levinson-Durbin's recursive algorithm is used and AR coefficients are extracted as feature vectors. In the next step, two different feature selection methods based on SFS algorithm and Davies-Bouldin index are used in order to decrease the complexity of computing and redundancy of features; then, three different classifiers include KNN, quadratic discriminant analysis and linear discriminant analysis are used to discriminate two and three different classes of valence and arousal levels. The proposed method is evaluated with EEG signals of available database for emotion analysis using physiological signals, which are recorded from 32 participants during 40 1 min audio visual inductions. According to the results, AR features are efficient to recognize emotional states from EEG signals, and KNN performs better than two other classifiers in discriminating of both two and three valence/arousal classes. The results also show that SFS method improves accuracies by almost 10-15% as compared to Davies-Bouldin based feature selection. The best accuracies are %72.33 and %74.20 for two classes of valence and arousal and %61.10 and %65.16 for three classes, respectively.

INTRODUCTION

Considering the importance of emotions in managing daily life of an individual, the need for designing brain computer interface (BCI) systems which can explore brain signals and detect user's emotional states for people with disabilities is growing. In such systems, revealing some of emotions like fear and stress can play a vital role for these patients in dangerous situations. Emotions affect cognition, perception, memory, attention, and learning process. People need to express their emotions to communicate with others; also daily activities, are entirely influenced by the individual's feelings.[12] In order to evaluate electroencephalogram (EEG) signals correlated to emotions, different categorizations of emotional states have been proposed that two of them are more common: Discrete model, consists of six basic emotions suggested by Ekman et al. and two-dimensional continues model suggested by Russell.[34] In Ekman model, six discrete emotions acceptable for all nationalities and cultures, are include: Happiness, sadness, fear, anger, surprise, and disgust. In this model, emotions do not have any coherence to each other and belong to distinct emotional classes. In Russell continues model, two-dimensional model is defined for continuous representation of two axes: Valence and arousal. Valence axis ranges from pleasant to unpleasant, while arousal axis ranges from exciting to calm and all emotions are distributed in two-dimensional spaces corresponding to their degree of valence and arousal.

In neurophysiologic researches, biological signals has shown a relationship between emotions and physiological activities.[56] Several studies have been accomplished by approach of emotion recognition using different physiological signals such as heart rate, skin conductance, respiration rate, electrocardiogram, and electromyogram.[78910] Furthermore, many systems have been proposed for emotion recognition using facial expressions and speech signals.[1112] However, in recent years, most authors are focused on EEG signals for designing the emotion recognition systems. Murugappan et al.[13] proposed a method using statistic features from EEG frequency bands and wavelet transform to classify five discrete emotions (fear, disgust, and neutral, happy, and surprise), and they achieved average accuracy of %79.17. In another work of this author,[14] different discrete emotions were classified by energy-based wavelet features and the average accuracy of %83.26 was obtained. Chung and Yoon.[15] suggested emotion recognition system based on power spectral density (PSD) features; they classified two and three emotional classes of valence and arousal classes and achieved the average accuracy of 66.6% and 53.4% for two and three classes of valence levels, respectively. Soleymani et al.[16] suggested a system using EEG, pupillary response and gaze distance for classification of three classes of valence and arousal dimension based on PSD features. They obtained the best accuracies of 76.4 and 68.5 for two and three classes of valence and arousal classes.

Bastos Filho et al.[17] proposed an emotion recognition system based on three feature extraction methods from EEG signals to discriminate three emotional states: Stress, calm and normal state. Extracted features were PSD, statistical and high order crossing (HOC), and the best accuracy of %70.1 was achieved using PSD features. Furthermore, other feature extraction methods based on statistic, nonlinear and energy logarithmic features are introduced in different publications; the summary of some results and applied feature extraction methods of recently works are shown in Table 1.

Table 1

Some classification results of publications related to emotion recognition

As it has been demonstrated, several feature extraction methods are suggested by different researchers in order to classification emotional classes, but there have not been any studies completed using feature extraction based on AR model. The goal of this study is evaluating the performance of AR features in the classification of two and three emotional states over valence/arousal model. To perform this approach, different AR orders of Burg's method based on Levinson-Durbin's recursive algorithm were extracted and AR coefficients were used as features vectors; afterward, two feature selection methods based on sequential forward feature selection (SFS) algorithm and Davies–Bouldin index were used in order to decrease redundancy of features and computation time. Then, selected features were given to K-nearest neighbor (KNN), quadratic discriminant analysis (QDA) and linear discriminant analysis (LDA) classifier to classify different emotional classes. The procedure of suggested emotion recognition system is as shown in Figure 1.

Figure 1

Block diagram of the experimental process

The organization and structure of the study is defined as follows: Materials and methods section is consisting of the research methodology by describing data acquirement, preprocessing, AR feature extraction, SFS method and classification process; in the next section, the research results are presented, and conclusion of the study is represented in discussions and conclusions.

MATERIALS AND METHODS

Data Input

In this research, EEG signals of a publicly available dataset for emotion analysis using physiological signals (DEAP) is used which is collected by Koelstra et al.[28] This database includes EEG and peripheral signals from 32 subjects: 16 women and 16 men with the average age of 26.9. EEG signals were recorded according to 10-20 standard system from 32 position include: Fp1, AF3, F3, F7, FC5, FC1, C3, T7, CP5, CP1, P3, P7, PO3, O1, Oz, Pz, Fp2, AF4, Fz, F4, F8, FC6, FC2, Cz, C4, T8, CP6, CP2, P4, P8, PO4, and O2. During the experiments, EEG signals were down sampled (128 Hz) and filtered (between 4.0 and 45 Hz), also eye artifacts were eliminated by blind source separation technique. Koelstra et al.[28] provided the preprocessed data and the raw data for each subject. In this study, the preprocessed data were applied to evaluate the proposed method. In the experiments, 41 min music video inductions were represented for each participants and degree of valence and arousal was ranged by using self-assessment manikins (SAM) questionnaire.[29] SAM is a distinguished questionnaire that visualizes the degree of valence and arousal dimensions by manikins and participants should choice one number from 1 to 9 written below the manikins as shown in Figure 2.

Figure 2

Self-assessment manikins scales of valence (above) and arousal (bellow)

In this study, emotional states are divided into two and three classes of valence and arousal dimensions according to the participant's SAM ratings. Valence and arousal level subdivisions are as shown in Table 2.[15]

Table 2

SAM ratings for affective class

Feature Extraction

Autoregressive model

The AR model has high ability in representing and modeling the characteristics and information inside a signal. AR model is frequently used in different approaches toward processing EEG signals such as: BCI designs,[303132] classification of schizophrenic patients,[33] estimation of hypnosis levels,[34] determination of sleep stages,[35] analysis of anesthesia[36] and classification of epilepsy diagnosis.[37] In AR model, each sample is obtained from the summation of previous weighted samples according to Eq. 1. The model order is determined by the number of weights, which are called AR coefficients.

Where, P is the model order and AR coefficients are denoted as ai (i = 1,…,p). In this paper, AR coefficients are obtained by applying Burg method.[38] In Burg method, AR reflection coefficients are estimated by minimizing the sum of forward and backward forecasted errors.

kp is The pth reflection coefficient which is a criterion of the correlation between x(t) and x(t − p). By applying the Levinson–Durbin recursion algorithm, these reflection coefficients ki can be converted, into AR parameters according to Eq. 2:

In the pth level of Burg method, after estimating previous reflection coefficients k1,… kp−1 through a recursive process, kp reflection coefficient is determined.

At each level, reflection coefficient is calculated as bellow:

where, ef,p−1 and eb,p−1 are forward and backward forecasted errors for (p − 1)th order of the model.[39] In the present work, AR coefficients from different orders of Burg's method based on Levinson–Durbin recursion algorithm were extracted as feature vectors, and the results of classification accuracies were compared.

Feature selection

In order to decrease complexity of computing and redundancy of features, different feature selection methods have been proposed. In this study, we used two approaches of feature subset selection: Scalar feature selection based on Davies–Bouldin index and vector feature selection based on SFS.

Feature selection based on the Davies–Bouldin index

In this case, feature selection is performed based on the values of Davies–Bouldin index.[40] The principle of this measure is based on two basis parts of data clustering: Minimizing inter-class distance (the distance among all data in a class), and maximizing intra-class distance (the distance between classes).

Mathematically, the Davies–Bouldin index is given as follow:

Where, diam (Ci) is the maximum distance between all pairs of samples in class i, d (i, j) is the distance between the center of class i and class j and M is the number of classes. Lower values of DBI index indicate less cluster overlap and thus higher class separation, while higher values show lower class discrimination.

In the experiment performed here, at first, Davies–Bouldin index for each feature was computed; then, features were ranked in descending order of criterion values. Finally, the features with the lowest ranking were selected.

Feature selection based on SFS method

SFS algorithm is one of the simplest feature subset selection methods. To achieve the best feature set, this algorithm is subsequently added to the first set of features which is initially empty. According to Figure 3, at first feature set A is considered empty and does not include any feature. Then, this algorithm seeks one of the features has the most influence in improving the fitness and adds the feature with the highest fitness x*; next, it seeks for the second feature that combination of it with the first selected feature results in the best. This procedure continues until adding a new feature does not increase the performance. Finally, A is considered the best feature set. Here, classification accuracy is considered as the fitness of a feature set.[41]

Figure 3

Procedure of feature selection using sequential forward feature selection method

Classification

K-nearest neighbor

K-nearest neighbor is a simple classifier that has been utilized in many pattern recognition applications. In this classifier, the class label of a new test sample is determined with respect to the labels of the nearest training samples. k closest training samples to a new test sample are determined and the label of a test sample is specified according the most repeatedly labels of these k closest samples. The number of the nearest neighbors (K) is required to be determined for the classification process. In this study, different K values were inspected and the k value with the best classification accuracy was selected.

Linear discriminant analysis

Linear discriminant analysis is one of the most distinguished classifiers in statistic, machine learning and pattern recognition. This classifier discovers a linear combination of features that separates or determines two or more classes of events or objects. This classifier finds a one-dimensional subspace in which the classes are commonly well separated by a linear separating hyperplane. The discriminant function is defined as follow:

Where, x is the set of measurements, k is the class of data, π is the prior probability and Σk is the covariance matrix.[4243]

Quadratic discriminant analysis

Quadratic discriminant analysis classifier is a generalized version of LDA classifier. In this classifier, unlike LDA, covariance of each class is not considered identical. Further, the surface that separates the subspaces will be conical. The discriminant function is given by:[43]

And the discriminant rule is as follow:

dk (x) = min1≤k≤Kdk (x)

EXPRIMENTAL RESULTS

In this paper, 32 channels of EEG signals from 32 participants during watching emotional inductions were used to evaluate the proposed methodology. To ensure the assumption of stationary, EEG signal of each channel was divided into1 s windows and AR coefficients were extracted for each window. The classification results based on different orders of AR model, different feature selection methods and different classifiers were compared. Two different feature subset selection methods were used for decreasing the redundancy of features. In the first method, Davies–Bouldin index was computed for each feature and features with the smaller values were selected. Then, the performance of these selected features was evaluated using classification accuracies of different classifiers. In the second method, SFS algorithm was applied. In this method, the best subset of features with the best classification accuracy was selected. In this study, three classifiers include KNN, QDA and LDA were used to classify two and three classes of valence and arousal levels. All the data were divided into test and training set and the leave-one-out cross validation was used to validate the performance of classification results. In this cross validation method, feature vectors of one participant were used as the test data and the feature vectors of others were used for training the model. This process is repeated until all participants are used as the test data; finally, the average of all participants’ classification accuracies was considered.

The classification results of two and three classes of valence and arousal using Davies–Bouldin and SFS feature selection methods through different orders of AR model and different classifiers are shown in Tables 3–6.

Table 3

Classification accuracy of two classes of valence and arousal using different orders of Burg's method and Davies–Bouldin based feature selection method

Table 6

Classification accuracy of three classes of valence and arusal using different orders of Burg's method and SFS feature selection method

Comparison Between Classifiers

As we said earlier, three classifiers were used for evaluating the performance of proposed method. According to Tables 3–6, nearly similar results have been obtained using different AR model orders; but, the best classification accuracies for two valence and arousal classes using Davies–Bouldin feature selection were %58.66 and %59.22 for KNN classifier, %58.26 and %57.42 for QDA classifier and %55.23 and %55.98 for LDA classifier by model order P = 9,10, respectively. Furthermore, the best classification accuracies for two valence and arousal classes using SFS method were %72.33 and %74.20 for KNN classifier, %70.35 and 69.26 for QDA classifier and %63.22 and %65.54 for LDA classifier by model order P = 10. According to the results of Tables 5 and 6, the best classification accuracies of three classes of valence and arousal using Davies–Bouldin feature selection method were %51.17 and %53.67 for KNN classifier,%51.87 and %52.14 for QDA classifier and %48.28 and %49.63 for LDA classifier by model order P = 8,10. Furthermore, the best classification accuracies for three classes of valence and arousal using SFS method were %61.10 and %65.16 for KNN classifier, %57.42 and %57.18 for QDA classifier and %51.20 and %52.36 for LDA classifier by using model order P = 8. The results show that the best classification accuracies for both feature selection methods are obtained using KNN classifier; while, the lowest classification accuracies are belonged to LDA classifier.

Table 5

Classification accuracy of three classes of valence and arousal using different orders of Burg's method and Davies–Bouldin based feature selection method

Comparison between feature selection methods

In this research, we used two feature selection methods; scalar feature selection based on Davies–Bouldin index and vector feature selection based on sequential forward selection (SFS) algorithm. According to the results of Tables 3–6, the best accuracies for two classes of valence and arousal using Davies–Bouldin feature selection are %58.66 and %59.22, and for three classes are %51.17 and %53.67; while, the best accuracies using SFS method are %72.33 and %74.20, for two classes and %61.10 and %65.16 for three classes, respectively.

The results show that feature selection based on SFS method has improved the classification accuracies by almost 10% to 15% as compared to Davies–Bouldin based feature selection. Scalar feature selection based on Davies–Bouldin index has lower complexity and computing time than SFS method; but, the obtained results are not significant. It seems the reason is ignoring features correlations in procedure of feature subset selection; because scalar feature selection methods treat features individually and ignore the feature associations, while SFS method considers correlations between features in selecting the best feature subset.

DISCUSSIONS AND CONCLUSIONS

As we mentioned earlier, there have not been any studies in previous works completed on emotion classification using AR features. Because of comprehensive ability of AR model to discover the characteristics of the signals, we decided to evaluate the performance of this kind of features in recognizing of emotions. In this study, we examined two different feature selection methods based on Davies–Bouldin index and SFS algorithm, and the classification results of three different classifiers (KNN, QDA and LDA) were compared through both feature selection methods. In order to estimate AR parameters, several methods such as Yull–Walker, Burg, and Covariance methods are proposed. In this study, the Burg's method based on Levinson–Durbin recursion was used because of its higher ability of minimizing both forward and backward forecasted errors compared to other methods. Selecting the order of the model is an important issue to model the signal; hence, different model orders were examined and the classification results were compared. We used SFS and Davies–Bouldin based feature selection methods because of their low complexity and simple procedure to remove the redundant features. comparison of classification accuracies of Tables 3–6 show that SFS method perform better than feature selection based on Davies–Bouldin index; also KNN classification results are better than other classifiers.

According to our proposed system, AR coefficients are efficient in discrimination of two and three valence/arousal classes of emotions, and in case of two classes, the proposed technique shows the better classification accuracy than three classes. We evaluated our method with EEG signals of available database for emotion analysis (DEAP). Until now, limited articles have been published using DEAP database; Koelstra et al.[28] proposed a system based on power spectral features from EEG signals, Fisher criterion for feature selection and naÏve Bayse classifier; They achieved the average accuracies of %57.6 and %62 for two classes of valence and arousal using this database. In another study using DEAP database, Yoon and Chung[44] designed an emotion recognition system based on Fast Fourier transform feature extraction, Pearson correlation coefficient for feature selection and Bayes classifier. They obtained the average accuracies of %70.9 and %70.1 for two classes of valence and arousal, and %55.4 and %55.2 for three classes. Bastos Filho et al.[17] proposed an emotion classification method to classify three emotional states: Stress, calm and normal using DEAP database. They used PSD, statistical and HOC features, and the best accuracy of %70.1 was achieved using PSD features. Chung and Yoon[15] proposed an emotion recognition method using Bayes classifier based on a weighed-log-posterior probability function and power spectral features using this database and the best accuracies of 66.6% and 53.4% were obtained for two and three classes of valence dimension, respectively.

Compared with previous studies using DEAP database, our new proposed method has shown higher classification accuracy. Almost in all these studies, feature extraction method is based on power spectral features. Our research showed that AR features are efficient and have similar classification accuracies to power spectral features in distinguishing affective emotional states. our suggested method based on AR features, SFS method and KNN classifier has improved the classification accuracy rate in the classification of valence/arousal classes by almost %2–%4 as compared to the best reported classification accuracies using DEAP database; until now, the highest achieved accuracies using DEAP database for two classes of valence/arousal space are %70.9 and %70.1[44] whereas, the classification accuracies of our proposed method are %74.20 and %72.33. Furthermore, in comparison with other new studies with other databases,[1621] our new proposed method has demonstrated higher classification accuracy with lower computational complexity. However, for the real situation, the classification accuracy must be improved higher. In the future, it is purposed to develop a system with higher classification accuracy and investigate another feature extraction, feature selection and classification methods to improve the performance and classification accuracy rate.

BIOGRAPHIES

Sepideh Hatamikia was born in 1989. She received the B.Sc. degree in biomedical engineering from Islamic Azad University, Dezful branch and the M.Sc. degree in Biomedical Engineering, Science and Research Branch, Islamic Azad University, Tehran, Iran in 2011 and 2013, respectively. Her research interests include biomedical signal processing, pattern recognition, analysis of brain signals and EEG-based brain-computer interface.

E-mail: shatamikia11@gmail.com

Keivan Maghooli has received his B.Sc. in electronic engineering from the Shahid Beheshti University, Tehran, Iran, M.Sc. in biomedical engineering from the Tarbiat Modaress University, Tehran, Iran, and Ph.D. in biomedical engineering from the Research and Science branch, Azad University, Tehran, Iran, majoring in Signal Processing and Artificial Intelligence. He has been with the Biomedical Faculty at Research and Science branch, Azad University, Tehran, Iran, since 2000, where he is currently an Assistant Professor and Head of Bioelectric Department.

E-mail: k_maghooli@srbiau.ac.ir

Ali Motie Nasrabadi received a BS degree in Electronic Engineering in 1994 and his MS and PhD degrees in Biomedical Engineering in 1999 and 2004 respectively, from Amirkabir University of Technology, Tehran, Iran. Since 2005, he has joined to Shahed university and now he is an associate professor in the Biomedical Engineering Department at Shahed University, in Tehran, Iran. His current research interests are in the field of Biomedical Signal Processing, Nonlinear Time Series Analysis and Evolutionary Algorithms. Particular applications include: EEG Signal Processing in Mental Task Activities, Hypnosis, BCI and Epileptic Seizure Prediction.

E-mail: nasrabadi@shahed.ac.ir

Table 4

Classification accuracy of two classes of valence and arousal using different orders of Burg's method and SFS feature selection method