A deep neural network for the classification of epileptic seizures using hierarchical attention mechanism.

Electroencephalogram (EEG) is a common diagnostic tool for measuring the seizure activity of the brain. There are many deep learning techniques introduced to analyze EEG. These methods show phenomenal results, although they are limited to computational complexity. Our objective was to develop a novel algorithm that gives maximum classification accuracy with a minor computational complexity. In this view, we have introduced a novel convolutional architecture with an integration of a hierarchical attention mechanism. The model comprises three parts: Feature extraction layer, which uses to extract the convoluted feature map; hierarchical attention layer, which is used to obtain weighted hierarchical feature map; classification layer, which uses weighted features for classification of healthy and seizure subjects. The proposed model can extract significant information from the EEG signal to classify seizure subjects, and it is compared with a few existing deep convolutional algorithms through experimentation. The experimental outcomes show that the proposed model has higher accuracy with less computational time.

Introduction

Epilepsy is a common neurological disorder that causes unprovoked, recurrent seizures. There is no remedy for epilepsy; however, uncertainty can be managed with detection and medications. The recurrent seizures may damage the neural structure and incidentally cause physical injuries such as accidents, fractures, and even death. Hence, accurate detection of seizures is the desired task to safeguard and improve the quality lifespan of epilepsy patients (Tang et al. 2020). Many earlier studies (Lee et al. 2014; Nicolaou and Georgiou 2012) are committed to electroencephalogram (EEG)-based seizure detection, which is a popular biomarker to study the neural activity of the brain. Identifying seizure activity in EEG signals is a challenging task due to its dynamic motion, viewpoint variations and computational complexity. Most current seizure detection methods consist of two main steps: feature extraction and classification. The traditional seizure detection methods are used different feature extraction methods before the classification process. A separate feature extraction method requires more attention in feature selection, and it is a less effective, more time-consuming process in the analysis of large medical datasets. Recently, deep learning algorithms are playing a key role in biomedical image and signal processing applications due to their automatic feature extraction process. The convolutional neural network (CNN) is a commonly applied deep learning architecture in image and video processing applications (Zhang et al. 2019; Ding and Tao 2017; Yang et al. 2020; Yonekura et al. 2017; Lee and Kwon 2017; Li et al. 2021; Kang et al. 2020). It has got more attention and become a powerful tool in the applications of image processing, where the input is generally two-dimensional (2-D) data. Thodoroff et al. (2016) introduced a recurrent convolutional neural network for seizure detection, in which the input EEG signals are converted as 2-D images and processed. In another work (Yuan et al. 2018), a multi-view learning model with an autoencoder architecture for the detection of seizures. Here, the CHB-MIT database is used, and a seizure detection module is projected by adopting a channel-wise contest method in the learning phase of the neural network. Further, Liu et al. (2020) introduced a novel deep convolutional long short-term memory (C-LSTM) model for seizure detection. Similarly, many application areas such as medical image analysis (Nardelli et al. 2018), industrial automation (Wang et al. 2020), multimedia applications (Jin et al. 2019) were used 2-D convolutional neural networks. The earlier studies (Kiranyaz et al. 2015; Wu et al. 2018) show that CNN is capable to analyze 1-D data. However, to date, few studies have applied one-dimensional CNN algorithms to signal processing applications. The authors (Kiranyaz et al. 2015) considered the CNN for the study of 1-D signals and they are designed a 1-D CNN model for the classification of irregularity in ECG signals. The MIT-BIH arrhythmia database was used to validate the network model. In the authors have developed a 1-D CNN-SVM model to analyze human knee movement mechanomyography signals. Most recently, Bhagya and Suchetha (2020) introduced a 1-D CNN with a deformable learning mechanism to analyze abnormal capnographic signals and the authors attained an average prediction accuracy of 92.9%. Even though the CNN architecture functions massively well, its operational performance can be additionally improved by making some changes in the original architecture. In this work, the CNN is integrated with an attention mechanism for enhancing the prediction probability of the proposed architecture. Attention is one of the most powerful concepts in deep learning, where it used different positions of a single sequence to compute a representation of the sequence (Vaswani et al. 2017). It is a mechanism that lets the neural network focuses attention on some region of the input when it is producing an output. The attention mechanisms are primarily developed to enhance the performance of encoder–decoder-based neural networks. The deep learning-based attention mechanisms are mainly implemented in the applications of Natural Language Processing (NLP), later its usage is extended to image and video processing applications. Bahdanau et al. (2014) presented an attention-based recurrent neural network (RNN) for language translation application. The authors have highlighted the importance of attention in various stages of the translation process. Zhai et al. (2019) introduced a dual self-attention pyramid network to integrate local channel features for optical flow valuation in video processing. The authors focused on obtaining significant features through an adaptive integration of local features with their total dependencies. In another video processing application (Jang et al. 2018), a hierarchical attention method with the combination of bi-directional long short-term memory (LSTM) is used for Dialog state tracking. Similar to NLP and video processing applications, a few biomedical image processing applications are also integrated attention mechanisms with deep learning methods. Veasey et al. (2020) explains a convolutional attention network to diagnose lung cancer where the input is a CT scan image. In their method, each 2-D slice convolutional features are weighted dynamically by the attention mechanism to focus on the most significant features, and it is performed well in multi-scale classification with a minimal learning rate. Similarly, a Prior-Attention method (Wang et al. 2020) is introduced for detecting COVID-19 in CT chest Images. The prior-attention learning block is used to locate lesion areas more accurately, which enhances the classification performance of the network in COVID-19 tasks. In another work (Zhang et al. 2020), an attention-based adversarial training method is proposed to design a patient-independent seizure detection method. The attention weights are learned automatically from the individual EEG channels. This method outperforms the existing methods with less testing latency.

Few more recent epilepsy seizure detection studies are focused on customized feature selection (Jiang and Zhao 2020), multivariate scale features (Furui et al. 2020) and multi-feature fusion (Radman et al. 2020) methods. Our proposed method differs from those works by focusing on hierarchical attention-based 1-D CNN for appropriate learning and classification. In this work, we adopt a robust hierarchical attention mechanism with the combination of CNN to focus on salient context features of the data. Thus, the outcomes of the proposed methodology are listed as follows: Further, the work is structured as follows. The overall workflow of the proposed methodology is discussed in Sect. 2. The competence of the proposed technique is evaluated and discussed with relevant performance metrics in Sect. 3. The work is concluded in Sect. 4.

Known that each single-channel EEG signal is collected from different parts of the brain and each channel will have a variation in the data. Therefore, parallel feature extraction for every two adjacent channels is performed with two separate convolution layers to obtain a multi-channel fusion feature map.

A filter-based feature selection process is applied to select the most significant and relevant features from a huge set of features, which results in faster training, reduces the over-fitting and improves the prediction rate

An effective attention strategy was implemented and applied to the fusion feature map to obtain the attention feature map.

The proposed method is capable to model robust and salient feature representation of raw EEG signal. It has achieved the best classification accuracy in epilepsy seizure prediction with less computational time.

Proposed methodology

The proposed convolutional model is developed by appending the hierarchical attention block in the traditional CNN architecture. The proposed model consists of three main operational layers. They are feature extraction, attention and classification layers. In the feature extraction layer, two parallel convolution blocks are fed with two individual EEG channels to extract the feature map. Each convolutional block contains three convolution and three pooling layers to extract the lower-dimensional feature map. Then extracted feature maps are given to the attention layer, where the output is a hierarchical weighted attention feature set. Then these weighted features are fed into a fully connected layer for classification. In this section, the proposed architecture of the hierarchical attention-based CNN is presented and the main contribution of this work is a novel attention mechanism. Figure 1 shows the architecture of the proposed methodology.

Fig. 1

Overall architecture of proposed methodology

Feature extraction layer

The most important function in the feature extraction layer is the convolution process. It is a process of changing one function and winning the sum of dot products. The convolution operation is performed between the kernel and input signal, which results in the convoluted feature map. Sequentially, the extracted feature map is given to the pooling layer to downsample and reduces the dimensionality of the feature map.

Let a be the input vector of length n and k be the kernel of length m. Thus, the convolution function is:where is the length of the output.

The obtained feature map is downsampled using the pooling layer. There are two popular pooling techniques, such as mean pooling and max pooling. The proposed approach uses the mean pooling operation as it takes into account all the input values. The input feature map is divided into distinct pooling segments. The mean value of each segment is measured as follows:where n is length of each segment. and N is number of segments.

In forward propagation, the convolved output of the previous layer is input to the present layer l, and it is mathematically represented in Eq. (3), in which each l involves in a signal feature.where is the bias of the kth signal, is the input of kth feature signal, is the weight of the kernel at layer from jth feature signal to kth signal at layer l, and is the jth output feature signal at layer . A significant feature set is obtained from the input signal with a sequence of convolution and pooling operations.

Attention layer

The attention layer consists of two stages: feature selection and attention weighted inputs. The outcome of attention layer is a hierarchical attention feature map, where the feature variables are added with some attention weights. The weighted attention features will enhance the system performance.

The deep learning models are used to extract the features directly from the raw data, which may contain some irrelevant information, which leads to a high dimensional feature map. Hence the feature selection becomes more important in deep learning applications because few features may be irrelevant and having less significance to the dependent variable. These redundant inclusions will affect system performance in terms of complexity and less reliable predictions.

The EEG signal is nonlinear data. The seizure EEG signal is almost similar to a healthy EEG signal except for some particular time intervals. It means that the healthy EEG signal will exhibit as seizure EEG with high and rapid change in amplitude when the seizure activity occurs. In deep learning techniques like CNN, the raw EEG is directly given for feature extraction without separating seizure intervals. Therefore, the resultant feature map is a mixture of healthy and seizure characteristic features. This point out that the two separate feature maps generated from healthy and seizure EEG, which contains some similar relevant features. So, it is especially important to apply a feature selection technique for the final feature map to improve the learning rate of the classification layer. Feature selection is a method of selecting the most significant features from a huge set of features, which results in faster training, reduces the over-fitting and improves the prediction rate. The Pearson correlation coefficient is used in the proposed work for the feature selection process.

Pearson correlation coefficient

Correlation is an important property of the data which is used to measure the linear relationship among two variables. The aim is to find the features which are highly correlated with the target. The Pearson correlation coefficient is a popular and widely used method to measure the correlation of numerous data variables. It is the covariance of the two variables divided by the product of their standard deviations. It can be represented as:where cov is the covariance, and denotes the standard deviation of p and q respectively. Equation (4) is defined in terms of mean as:where is the correlation coefficient, n is the sample size, and represents the individual sample points, and denotes the mean of p and q, respectively.

The p and q are considered as feature maps and respectively. The Pearson correlation coefficient is used to measure the strength and direction of the linear relationship between p and q. The correlation coefficient lies in between (− 1 1) if the two features are linearly dependent. The correlation coefficient nearing 1 indicates a positive correlation and nearing − 1 indicates a negative correlation. If the features are uncorrelated, then the correlation coefficient is considered as 0. This means that the higher the absolute value of the correlation coefficient, the greater the correlation, and vice versa. Therefore, the feature which is higher than the threshold value (let 0.5) is selected. Finally, the hierarchical features are obtained.

Hierarchical attention mechanism

Attention is a selective method, and it will increase the comprehensibility of the network by focusing on a specific region of the data. The recent works (Zhai et al. 2019; Veasey et al. 2020) are integrated the attention mechanism with CNN to extract the feature map and enhance the performance of the network by considering the channel and spatial information. In this paper, the proposed hierarchical attention mechanism is different from the literature. In the proposed method, the attention weights are obtained from hieratically selected features. Further, the network incorporates the local interpretation into the weighted interpretation through attention weight, which is attained by matching the local representation with the intermediate representation.

Attention weights

Attention weights emphasizes prominent input regions and significant features to safeguard only relevant formations specific to the real task. Each weight vector learns to focus on a division of target structures. The weight vector encloses contextual information to minimize lower-level responses. Each local feature interpretation is defined as and intermediate interpretation is defined as the attention weight of each feature is defined as:where is the similarity between local and intermediate representation. It can be obtained aswhere tansig(.) is used to measure the similarity between and . After obtaining the attention weight, the weight vector is calculated aswhere represents the attention weighted features. is a feature variable and is a weight vector. N represents the number of feature variables. The overall attention mechanism is shown in Fig. 2.

Fig. 2

Proposed hierarchical attention mechanism

Classification layer

The obtained weighted feature map of attention layer is classified in the classification layer. A fully connected MLP layer is used in the conventional CNN architecture to classify the features (Yíldírím et al. 2020). In sophisticated computer vision applications, the MLP is considered inadequate in performance due to its high growth rate of a single-layer neural network and redundancy. From the literature, it is shown that the conventional CNN with the combination of support vector machine (SVM), improves the classification performance of the network (Navaneeth and Suchetha 2019). Hence, the SVM is used in the proposed model to classify the feature vector. The SVM is a well-known supervised classification algorithm. The SVM model can be written as:The input features are classified by using the below criteriawhere indicates the class objective, represents the feature value, and represents the weight and bias, respectively. The optimal hyperplane is obtained by updating weight and bias values. They are computed as:For the classification of nonlinear data, a proper kernel function has to be applied. In proposed method, a Gaussian kernel is adopted. Then the classification function of SVM is written as:where k stand for the kernel function.

Results and discussion

The efficacy and performance estimation of the proposed attention methodology is validated with the standard classification parameters. In addition to this, the traditional 1-D CNN-MLP algorithm and attention-based 1-D CNN-MLP algorithm are implemented and compared with the proposed hierarchical attention-based 1-D CNN-SVM algorithm.

Dataset

The effectiveness of the proposed method is analyzed with the EEG signals for the classification of healthy and seizure signals. In this work, the data set is taken from Bonn University, Germany (Anand and Selvakumari 2019), which is open source and publicly available. The Bonn database consists of various groups of EEG signals from A to E. Each of these groups is recorded in different conditions of the subject. Datasets A and B are considered healthy, and they are recorded when the person is in awake and relaxed conditions respectively. The datasets C and D are recorded in the inter-ictal period. The last group E is recorded during the ictal period of the person. All these signals are recorded for the duration of 23.6s using a 10-20 standard electrode system at a sampling rate of 173.61Hz. Each group in the Bonn database contains 100 distinct single channel EEG epochs with a sample length of 4097.

Classification task

The Bonn database contains five different subsets. These subsets are divided into three classes as healthy, ictal (seizure) and inter-ictal to form different classification cases. In the classification task three different cases are considered in the study by their wide usage in the literature. (Riaz et al. 2015; Tzallas et al. 2009). The different classification cases are listed in Table 1.

Table 1

Different cases for classification

Cases	Grouping	Classes
Case I	Set A	Healthy
	Set E	Ictal
Case II	Set A	Healthy
	Set C	Inter-Ictal
Case III	Set ABCD	Healthy
	Set E	Ictal

In case I, dataset A is considered as a healthy class and dataset E is stated as seizure class. Case II is formulated with the datasets A & C in such a way to classify the healthy class and inter-ictal class respectively. In case III, the four datasets from A to D are put together as a healthy class and E is listed as seizure class.

Performance evaluation

The architecture of CNN-SVM with hieratical attention is trained and tested on a publicly available Bonn reference seizure database. The seizure EEG signals are classified as healthy, Ictal and Inter-Ictal classes. The well-known standard classification measures such as accuracy, sensitivity, specificity, F-measure, and MCC are considered for the performance evaluation of the proposed algorithm. Let consider and are the true positive and true negative of the samples, respectively. Similarly, the and are the false positive and false negative of the samples, respectively.

The exactness of a classification method can be measured by using a statistical measure known as accuracy. It can be calculated as the ratio of truly classified samples to the total number of classified samples. The mathematical expression of accuracy is:The sensitivity and specificity are the measures of true positive rate and true negative rate of samples, respectively. Sensitivity represents the percentage of truly predicted healthy samples and specificity represents the percentage of truly predicted seizure samples.The tests accuracy of a binary classifier is measured using the F-measure value, and it is varies in between zero to one.The MCC is an eminence measure of a binary classifier, and it is considered as the best measure over F-measure and accuracy because it denotes all four categories of confusion matrix with a single value.The weighted features are fed into the fully connected layer for classification. The fully connected layer has the SVM, which is the best binary classifier for classification. To validate the classification performance, a k-fold cross-validation strategy is applied. First, the features are divided into k equivalent subsets. From the k subsets, an independent subset is locked in as the testing information for approval, and the rest of the () are used for training in each k fold. The same classification method is applied to different classification cases and the performance parameters of different convolutional algorithms are listed in Table 2.

Table 2

The classification performance of different convolutional algorithms

Cases	Parameters	Various convolutional algorithms
		Conventional CNN	Attention-based CNN	Proposed hierarchical Attention-based CNN
Case I	Accuracy (%)	96.19	97.2826	98.33
	Sensitivity	0.9474	0.9780	0.9800
	Specificity	0.9775	0.9677	0.9700
	Precision	0.9783	0.9674	0.9780
	F-measure	0.9626	0.9727	0.9800
	MCC	0.9244	0.9457	0.9600
Case II	Accuracy (%)	95.11	95.11	95.56
	Sensitivity	0.9368	0.9462	0.9667
	Specificity	0.9663	0.9560	0.9444
	Precision	0.9674	0.9565	0.9457
	F-measure	0.9519	0.9514	0.9560
	MCC	0.9027	0.9022	0.9113
Case III	Accuracy (%)	95.65	96.20	97.21
	Sensitivity	0.9375	0.9570	0.9775
	Specificity	0.9773	0.9670	0.9667
	Precision	0.9783	0.9674	0.9667
	F-measure	0.9574	0.9622	0.9721
	MCC	0.9139	0.9240	0.9442

The proposed method has achieved the finest classification results with less computational complexity by the selection of hierarchical weighted features. The classification performance in an average of different techniques are shown in Table 3. The performance of the proposed technique is compared with the existing methods and shown in Table 4. The hierarchical attention-based deep learning with SVM classification has achieved best accuracy by comparing with traditional methods.

Table 3

The average classification performance of different algorithms

Parameters	Various convolutional algorithms
	Conventional CNN	Attention-based CNN	Proposed hierarchical Attention-based CNN
Accuracy (%)	95.65	96.20	97.03
Sensitivity	0.9406	0.9604	0.9747
Specificity	0.9737	0.9636	0.9604
Precision	0.9746	0.9638	0.9634
F-measure	0.9573	0.9621	0.9694
MCC	0.9137	0.9240	0.9385

Table 4

Comparative study of proposed method with traditional methods

Author	Method	Cases	Accuracy (%)
Lee et al. (2014)	WT, phase-space reconstruction and ED + NEWFM	A E	98.17
Nicolaou and Georgiou (2012)	Permutation entropy + SVM classifier	A E	93.55
Riaz et al. (2015)	Time-frequency features in EMD + SVM	A E	99
		D E	93
		AB CD E	96
		A D E	85
		ABCD E	83
Lin et al. (2016)	Stacked sparse autoencoder + softmax classifier	A E	95.5
		A D	86.42
Chandaka et al. (2009)	Cross-correlation-aided SVM classifier	A E	95.96
Proposed	Hierarchical attention-based deep learning + SVM	A E	98.33
		A C	95.56
		ABCD E	97.21

Receiver operating characteristic curve

The diagnostic potential of a binary classifier is illustrated as a graphical plot which is known as the ROC curve. The ROC curve is plotted in between true-positive rate (TPR) and false-positive rate (FPR). The flawless classification represents at TPR = 1 and FPR = 0, while the poorest at TPR = 1 and FPR = 0. Hence, the larger ROC area indicates the highest classification accuracy. Figure 3 represents the ROC curve for the two classes such as healthy and ictal. The area under the curve (AUC) will be used to compute the classification performance. The higher AUC indicates the better performance of the classifier.

Fig. 3

Roc curve between healthy and ictal class

Computational complexity

In real-time applications, the run time plays a key role in reducing the computational complexity of the system. The deep learning algorithms are gained more reputation due to their automatic feature extraction process, which influences the overall execution time of the system. This section compares the computational time of the proposed method with other deep learning algorithms. In the proposed algorithm, the average execution time required to extract feature attributes from the original signal is about 0.1307 s. The feature extraction time of the deep learning algorithm is less when compared to the other traditional feature extraction techniques. Similarly, the classification time of the proposed hierarchical attention method is about 0.2354 s when compared to that of conventional CNN method through back-propagation. The maximum run time for the proposed attention method is 0.3661 s, which is less when compared to a conventional CNN of 0.4783 s. Figure 4 compares the execution time of few convolutional techniques and it is perceived that the overall execution time of the proposed method is considerably low.

Fig. 4

Computational time comparison of various convolutional neural network approaches

Conclusion

A deep learning model of seizure classification is presented in this paper. This study involves the implementation of a hierarchical Attention-based convolutional neural network for the classification of raw EEG signals. The convolution operation is used to obtain the feature map from the raw EEG signal. The obtained feature map is given to the attention layer, where the features are weighted hierarchically using the attention mechanism. In the attention layer, the features are arranged by adding the weights according to the level of significance. Further, these weighted features are classified using the SVM classifier. In this work, the raw seizure EEG signals have been effectively classified with the proposed 1-D CNN technique and the performance parameters are computed. Further, these performance metrics are compared with other convolutional methods. It has been observed that the proposed method significantly reduces the computational complexity and improves the performance of the classifier.