Comparative analysis between convolutional neural network learned and engineered features: A case study on cardiac arrhythmia detection.

Background: Atrial fibrillation (AF) is one of the most common cardiovascular problems, and its asymptomatic tendency makes AF detection challenging. Machine and deep learning methods are commonly used in AF detection. Objective: The purpose of this study was to evaluate the information provided by convolutional neural network (CNN) and random forest (RF) machine learning models for AF classification.
Methods: We manually extracted 166 time-frequency domains and linear and nonlinear features to classify single-lead electrocardiograms (ECGs) as normal, AF, other, or noisy sinus rhythms. We selected a subset of 56 robust features using a genetic algorithm that was used in the RF model. In a separate study, a 1-dimensional, 12-layer CNN was designed on the raw ECG rhythms. Four features from the output layer and 128 features from the fully connected layer of CNN were explored independently for classification. The models were trained and internally validated on 8,528 ECGs and externally validated on a hidden dataset containing 3,658 ECGs. Next,we analyzed the correlation between engineered and CNN-learned features.
Results: An RF classifier trained with 56-engineered features resulted in an F1 score of 0.91, 0.78, and 0.72 for normal, AF, and other rhythms, respectively. However, an ensemble of support vector machine and the CNN model resulted in an F1 score of 0.92, 0.87, and 0.80, respectively.
Conclusion: We explored various features and machine learning models to identify AF rhythms using short (9-61 seconds) single-lead ECG recordings. Our results showed that the proposed CNN model abstracted distinctive features for AF classification.

This study presents an exhaustive analysis of time–frequency, linear, and nonlinear engineered features that can be used for cardiac arrhythmia classification.

An optimized 1-dimensional,12-layer convolutional neural network (CNN) model is also designed for the classification of short, single-lead electrocardiographic recordings.

The CNN-derived features in the proposed support vector machine classifier improved the arrhythmia classification performance.

This study provides an explicit comparison between engineered and CNN-abstracted features for arrhythmia detection.

Background

Atrial fibrillation (AF) is the most common cardiac arrhythmia, affecting 1%–2% of the world’s population, and is associated with significant mortality and morbidity., AF episodes are commonly identified using 12-lead electrocardiography (ECG) characterized by atrial activity and ventricular rate. Artificial intelligence–based methods have been used recently to detect AF from ECG recordings. Most classic machine learning methods are based on feature engineering in which features are manually extracted and later processed by predictive models. In contrast, convolutional neural networks (CNNs) have been built with an unsupervised feature extraction mechanism to classify signals. Despite CNN showing extraordinary performance in the AF classification problem, little is known about the information extracted by CNN for classification.

Recently, many studies have proposed various algorithms for automated diagnosis of cardiac abnormalities.3, 4, 5, 6, 7 Yu and Chen developed a wavelet transform and neural network–based arrhythmia detection method having accuracy of 99.65%. Ghorbani Afkhami et al used statistical and mixture modeling features of ECG signals to achieve an overall accuracy of 99% for identifying various types of arrhythmias. Asgari et al used support vector machine (SVM) for automatic AF detection that eliminates the need for P- or R-peak detection. The model achieved sensitivity of 97% and specificity of 97.1%. However, the results in these studies were obtained from training the machine learning model on a very small number of ECG records from a Massachusetts Institute of Technology–Boston’s Beth Israel Hospital AF and arrhythmia database, so the proposed techniques may not perform well on a larger cohort. Although a myriad of AF detection techniques have been reported in the literature, most of the studies have limited applicability due to use of a small and relatively clean dataset; the lack of an out-of-sample validation set to ensure broad generalizability of the approach; dependency on fixed-duration or multiple-lead ECG recordings; and computationally complex algorithms with longer training time.

To address these issues, we designed two experiments: the first, we implemented feature engineering–based classifiers such as random forest, SVM, linear discriminant analysis (LDA), quadratic discriminant analysis (QDA), decision trees, and k-nearest neighbor (KNN); and the second, we explored CNN-derived features from the raw ECG data, which expands on our previous work in the area. The objective of this study was to optimize the proposed algorithm in each experiment and to compare the performance of algorithms. Our effort was motivated by the PhysioNet Computing in Cardiology Challenge 2017, which provided the ECG dataset with a variety of arrhythmias (Figure 1).

Figure 1

Representative electrocardiographic recordings of 10-second duration for different classes of cardiac rhythms. AF = atrial fibrillation.

To the best of our knowledge, this is the first study to analyze the interrelationships between CNN and engineered features for arrhythmia classification. Some key contributions of this study are as follows:

The study provides an explicit comparison of information obtained by engineered and CNN-abstracted features for AF classification.

Various novel predictors for AF are implemented that derive morphologic, temporal, and spectral information from a single-channel ECG signal, which can be as short as 9 seconds.

A CNN-SVM integrated model is implemented that surpassed the classification performance of state-of-the-art models.

The results of this study can aid in the interpretability of the deep learning model while allowing for the discovery of potentially newer non-QRS features useful for classification of arrhythmias.

Methods

ECG dataset and preprocessing operations performed to clean data, a methodology for feature transformation, selection, and classification tools used in this study for arrhythmia detection are described. We designed random forest and CNN-based classification models and analyzed the performance of these models in different scenarios. This study uses publicly available de-identified data.

Experimental dataset and preprocessing

We used a training dataset of 8,528 ECG recordings collected using single-channel ECG device (AliveCor, Mountainview, CA). The data containing cardiac rhythms of 4 classes were sampled at 300 Hz and had a bandwidth of 0.5 Hz to 40 Hz. A hidden test dataset of 3,658 recordings of similar lengths was used to access the performance of the built classifiers. Access to this test dataset was not provided to the public. Before feature extraction, we performed a data quality check on ECG recordings and corrected inverted ECGs identified using the percentile-based statistical algorithm. To reduce computational time and complexity, we down-sampled all ECG recordings to 200 Hz.

Feature engineering

Time–domain features

We explored 56 time–domain features, including some novel and standard measures of the heart rate variability (HRV) signal that are recommended by the Task Force of the European Society of Cardiology and the North American Society of Pacing and Electrophysiology. We calculated 8 descriptive measures—mean, standard deviation (SD), kurtosis, skewness, range, median, median absolute deviation, and mean absolute deviation—each from preprocessed ECG and RR intervals calculated using the Pan-Tompkins algorithm. We also calculated descriptive measures, except for median absolute deviation and mean absolute deviation, from first- and second-order RR intervals along with their coefficient of correlations. We generated a unique set of features by defining 10 quantiles on RR interval series from each class and identified the mean of RR intervals in the lower 5% and upper 5% range of RR intervals. We applied the Kolmogorov-Smirnov test to compare the RR interval series with a reference RR interval database created by considering 50 examples from each class. We then computed 8 features using the mean and median of the p value by comparing RR interval series with the reference database.

Recent work on AF detection showed that P-wave indices, such as PR interval, P-wave morphology similarity, etc, can indicate abnormal atrial activities., Combining information from atrial and ventricular responses of the cardiac cycle can enhance AF detection efficacy. Therefore, we identified PR regions in the entire ECG recordings and calculated 7 descriptive measures from each PR region. The mean of descriptive measures from the recordings is then used as features in our classification models. To identify the PR region approximately, we extracted 50 samples before the R peak identified using the Pan-Tompkins algorithm. We also calculated the mean duration of the QT interval in ECG recordings. To obtain an approximate QT region, we extracted 15 samples before the R peak to identify Q onset and 225 samples after the R peak to account for the T wave.

HRV measures can noninvasively evaluate sympathovagal balance., We derived 2 unique HRV measures: (1) correlation coefficient and (2) percent of variance explained by the first principal component by analyzing RR interval and 1 sample delayed RR interval series obtained from each ECG recording. Mathematically, the correlation coefficient was calculated as given in Equation 1:

Another novel feature was obtained by calculating the mean absolute deviation of the derivative of the heart rate signal. We calculated the percent of RR intervals lying between 0.6 and 1.2 seconds. We also plotted the histogram of RR and delta RR series to provide a visual measure of parasympathetic nervous system activity. To quantify the measure, we identified the length of elements at the origin. This robust feature provides information on ectopic beats in each ECG rhythm that otherwise may go unobserved. Based on our experiments, domain knowledge, and literature review, we extracted a set of 56 time-domain–based features for classifying ECG rhythms.

Frequency–domain features

Spectral analysis has been widely used for characterizing ECG signals to fetch local atrial activity., Typical time–domain techniques fail to accurately reflect the cardiac response, especially when the ECG signal is affected by noise and excessive amplitude variations. We computed the power spectral density (PSD) of ECG recordings using the Welch's averaged periodogram method with a hamming window function. To extract fiducial points from PSD, we computed ratios of PSD using Equation 2. Similarly, psdratio2 was computed by dividing PSDs between frequencies 1–40 Hz and 0–40 Hz.

We also analyzed the spectral content of RR interval series. As the RR series is nonuniformly sampled, we computed PSD using the Lomb-Scargle periodogram. Three main power spectral components—very low frequency (VLF; ≤0.04 Hz), low frequency (LF; 0.04–0.15 Hz), and high frequency (HF; 0.15–0.40 Hz)—are known to demonstrate changes in automatic modulations of heart. Therefore, we calculated power in each of these components along with the spectral power ratio to gauge sympathetic and parasympathetic activities based on ECG signal. In addition to these features, we attempted to estimate PSD using wavelet coefficients. We extracted 48 unique features by decomposing ECG signals with “db4” Daebauchies wavelet at 7th scale. PSD calculations for 7 detail signals and 6th approximate signal were then made using the Welch method by applying a hamming window of 256 samples. Furthermore, we calculated mean PSD for both approximate and detail signals in 6 frequency bands: 1–3, 3–5, 5–9, 9–17,17–33, and 33–65 Hz. A total of 54 features based on spectral analysis were used in the classification models.

Nonlinear approach

Nonlinear complexity indices can characterize underlying mechanisms in the nonstationary ECG rhythms. Sample entropy (SampEn) is heuristically interpreted as a measure to assess complexity in the ECG recordings., In this study, we calculated SampEn from every 1-second epoch of the recording in a sliding nonoverlapping window manner. We computed SampEn as given in Equation 3:where the maximum epoch length to be compared, m = 3, tolerance window, r = 0.25 × SD of the epoch, and length of the ECG recording, N = 200. is the probability that 2 sequences will match for m points, whereas is the probability that the 2 sequences will match for m+1 points. We then calculated 6 descriptive measures from the SampEn series obtained from ECG recording.

To reduce the effect of noise and improve the signal-to-noise ratio in the ECG signal, we decomposed signal with Symlet wavelet at fifth scale. We constructed a feature set consisting of variance, mean, and variance of the autocorrelation of third, fourth, and fifth detail coefficients.

We calculated the SampEn of all 5 detail coefficients using parameters m = 2 and r = 0.25 × SD of series of detail coefficients.

Linear approach

In our previous studies, we developed a method known as probabilistic symbolic pattern recognition (PSPR) to quantify the underlying changes in morphology of ECG recordings., In this study, PSPR symbolically discretizes ECG recordings (down-sampled at 8 Hz) using 5 symbols (a, b, c, d, e) with thresholds defined by quantile length of 4. The pattern transition probabilities (PTP) of symbols in all discretized series then were calculated. We also applied PSPR on a reference database consisting of 100 normal ECG recordings and 25 paroxysmal AF episodes from the PhysioNet 2001 challenge. The similarity between PTPs of given series PTPik and reference series PTPjk were then calculated as given in Equation 4:

A set of 4 PSPR features was derived by calculating pattern transition similarity (PTS) for 6th and 7th symbol PTP using reference normal and paroxysmal atrial fibrillation (PAF) series. Furthermore, we calculated 4 descriptive measures—mean, maximum, minimum, and SD from 7th approximate and 7 detail wavelet coefficients obtained after decomposing ECG recordings with “db4” Daebauchies wavelet. This resulted in a set of 32 wavelet descriptive-based features, which provided information on time–frequency scale.

Feature selection using genetic algorithm

To identify redundant features that do not improve classification performance, we used a genetic algorithm (GA)–based stochastic feature selection technique. To optimize the GA-based algorithm, we used a population size of 100 and generation size of 50, and evaluated a fitness function, which computes an average F1 score of 3 main cardiac rhythms: normal, AF, and other. We trained a random forest classifier with 170 decision trees for every iteration.

CNN architecture for feature extraction

Deep learning techniques have been widely used in the last decade in the field of translational bioinformatics, medical informatics, and medical imaging. A few studies have also reported classifying ECG signals using the CNN model and other deep learning–based approaches., In this study, we designed a 1-dimensional, 12-layer CNN architecture to learn the structure of ECG rhythms. This design is an improved version of our 13-layer CNN, which required more training time and had greater architectural complexity. CNN can automatically exploit spatial or time relationships in data without requiring the domain knowledge.

To find an optimized CNN architecture, we investigated various structural and learning parameters, such as number of hidden layers, feature maps, kernel size, stride, and regularization coefficient. The architecture shown in Figure 2 resulted in the best performance, so we report results from this architecture. We also tried various preprocessing techniques such as normalizing raw ECG data with z-score, min-max, and used the first derivative of ECG instead of raw ECG input to CNN. We zero-padded ECG signals to analyze the consistent data duration of 60 seconds.

Figure 2

One-dimensional, 12-layer convolutional neural network architecture designed to distinguish 4 classes of cardiac rhythms using single-lead electrocardiography. AF = atrial fibrillation; BN = batch normalization; CL = convolution layer; FC = fully connected layer.

In the proposed CNN architecture, we used 12 convolution layers with a filter size of 1 × 5 and each layer followed by the batch normalization layer, activation layer, and dropout layers. We introduced nonlinearity in the model by using a ReLU activation function. To increase the generalizability of the model, we introduced regularization by using dropout layers and L2 regularization. We penalized the squared magnitude of weight by using L2 regularization factor: λ =1 × e-4. Furthermore, we applied max-pooling layers to control overfitting.

Result

Machine learning for arrhythmia classification

We extracted a set of 166 features characterizing the morphologic, spectral, complexity, and temporal dynamics of ECG signals. Figure 3 shows box plots of 4 representative features used in the final classification model to discriminate 4 classes of cardiac rhythms. We analyzed 4 groups of features as specified in Table 1, individually and collectively to gauge their contribution to classify cardiac rhythms. We also explored various conventional classifiers such as LDA, SVM, QDA, decision trees, KNN classifier, and random forest using 4 sets of features and compared their classification performance in terms of F1 score.

Figure 3

Box plots of 4 representative features extracted for classification of cardiac rhythms. AF = atrial fibrillation; HF = high frequency; LF = low frequency; PSD = power spectral density.

Table 1

Details of 166 features extracted with feature engineering

Type	Description	No. of features
Group 1: Time domain	Descriptive measures of PR interval, duration of QT interval, RR intervals, first- and second-order RR intervals	56
	Quantile based on RR intervals
	KS test based on RR intervals
	Heart rate variability measures
Group 2: Frequency domain	PSD of wavelet coefficients	54
	VLF power
	LF power
	HF power
	PSD ratios
Group 3: Nonlinear	Descriptive measures of sample entropy computed on ECG and wavelet coefficients	20
Group 4: Linear	PSPR features from ECG recording sampled at 8 Hz	36
	Descriptive measures of wavelet coefficients

ECG = electrocardiography; HF = high frequency; KS = Kolmogorov-Smirnov; LF = low frequency; PSD = power spectral density; PSPR = probabilistic symbolic pattern recognition; VLF = very low frequency.

We grouped extracted features into 4 categories (Table 1). Each group of features was analyzed with 6 classifiers using 5-fold cross-validation technique. Table 2 shows the comparison of average F1 score obtained from different classifiers for each group of features. With group 1 features, a maximum score of 0.73 was obtained with the random forest classifier. Interestingly, group 3 features provided a better classification response than groups 2 and 4, suggesting that analyzing the signal complexity of ECG provides more discriminative information than spectral and linear analysis methods for arrhythmia detection. Overall, LDA and QDA classifiers did not perform well, which may be due to issues of class imbalance and the features not being linearly separable. By far an ensemble of decision trees (170 in this study) performed superior compared to the other mentioned classifiers for identifying most of the rhythms. Of note, the extracted time–domain features provide the most meaningful information for discriminating all cardiac rhythms.

Table 2

Performance of various classifiers while using different categories of features for classification

Classifier	Average F1 score
	Group 1: Time domain	Group 2: Frequency domain	Group 4: Nonlinear	Group 4: Linear
SVM	0.71	0.38	0.40	0.38
LDA	0.19	0.30	0.41	0.26
KNN	0.71	0.26	0.38	0.22
QDA	0.20	0.32	0.42	0.19
Decision trees	0.66	0.39	0.40	0.30
Random forest	0.73	0.46	0.46	0.36

KNN = k-nearest neighbor; LDA = linear discriminant analysis; QDA = quadratic discriminant analysis; SVM = support vector machine.

To improve classification efficacy, we first considered evaluating the entire set of 166 features. Because random forest performed better than other classifiers, we used random forest as a representative machine learning model for the rest of the analysis. We then conducted a similar analysis using GA-selected features. The GA algorithm resulted in a subset of 56 features that were used in the final classification model, yielding an F1 score of 0.89, 0.76, and 0.71 for normal, AF, and other rhythms, respectively, on a cross-validated training dataset. The confusion matrix on this dataset is given in Table 3.

Table 3

Confusion matrix obtained from the cross-validated training dataset using random forest model

Normal	4740	18	288	30
AF	56	530	151	21
Other	706	73	1604	32
Noise	85	10	41	143
	Normal	AF	Other	Noise

AF = atrial fibrillation.

In contrast, on the hidden test dataset of 3,658 ECG recordings, the F1 scores were 0.91, 0.78, and 0.72, respectively. We believe that lack of information on the actual category of other rhythms makes the identification of appropriate features more challenging, thereby reducing classification accuracy.

CNN architecture for arrhythmia classification

A typical CNN model negates the need to manually extract features and therefore has less computational burden. The proposed architecture only requires 1-dimensional convolution operations such as multiplications and additions, which makes it computationally efficient and a suitable choice for real-time cardiac monitoring and anomaly detection applications using commodity Graphics processing units (GPUs).

We provided preprocessed 60-second-long ECG signals sampled at 200 Hz as an input to the 12-layer CNN. We divided the training database of 8,528 ECG recordings into 90% for training and 10% for validating the CNN model. Based on our experiments, we optimized CNN using a stochastic gradient descent with momentum approach. To validate the model at regular intervals, we also used an early stopping criterion, which stops training when the validation loss stops decreasing for 30 epochs. In the final CNN architecture, we used a maximum of 60 epochs for training, a minibatch with 20 observations at each iteration, and a Softmax function as the output unit activation function. Other technical specifications for training the CNN architecture are tabulated as kernel size of convolutional layers of 5, initial learning rate of 0.0001, stride in convolutional layer of 1, stride in pooling layer of 2, L2 regularization of 1 × e–4, momentum of 0.9, minibatch size of 20, and maximum epochs of 60.

We have designed an optimized CNN architecture that has a fast learning speed and gives high accuracy to classify 4 categories of cardiac rhythms. Because the focus of this study was to emphasize improving the classification efficacy for usable ECG recordings containing normal, AF, and other rhythms, we report results from these 3 classes. With CNN-Softmax, we obtained an F1 score of 0.90, 0.85, and 0.78 for identifying normal, AF, and other cardiac rhythms, respectively. We designed the CNN architecture using MATLAB (Version 2017b; MathWorks, Natick, MA).

Ensemble of SVM and CNN for arrhythmia classification

Many recent studies have shown that an SVM classifier trained with CNN-extracted features outperforms a typical CNN-Softmax classifier in various classification problems.30, 31, 32 Therefore, in this study we investigated the performance of a hybrid model of SVM and CNN for cardiac rhythm classification. We used flattened features from the first fully connected layer of the trained CNN model (see the previous section: CNN architecture for arrhythmia classification) in an SVM model for classification. A set of 128 features was extracted using network activations. The hybrid approach resulted in a superior performance, yielding an F1 score of 0.92, 0.87, and 0.80 for identifying normal, AF, and other cardiac rhythms, respectively.

The training and validation loss of the CNN architecture depicting the summation of errors made for each example in the training and validation sets is shown in Supplemental Figure 1. Figure 4 shows the comparison of F1 scores for each rhythm using 3 classification approaches. The reported results from the random forest classifier were provided to us by the PhysioNet 2017 challenge organizers after evaluating our model on hidden 3,658 ECG recordings. However, organizers could not evaluate our CNN-based models on the hidden set due to MATLAB platform’s incompatibility issue. Hence, we report CNN results obtained from the validation dataset.

Figure 4

F1 scores of 3 main cardiac rhythms obtained using feature engineering–based random forest (RF) (on the hidden test dataset), convolutional neural network (CNN), and an ensemble of support vector machine (SVM) and CNN classifiers (on the validation dataset). AF = atrial fibrillation.

Applications of CNN in AF detection have been increasing. Despite CNN resulting in high classification accuracy, one criticism of the CNN approach is that the results are not intuitive. In contrast, feature engineering–based machine learning methods allow identification of significant predictor variables. Here we aimed to uncover the types of ECG morphologies captured by CNN. To do so, we analyzed the correlations between CNN-learned and engineered features used in the random forest model. Figure 5 shows the heat map of the correlation matrix obtained by combining 166 manually extracted and 128 CNN-extracted features. Most of the features in SampEn (group 3) and PSD obtained from wavelet coefficients (group 2) showed high correlation (p-value <.001). Interestingly, most of the CNN features showed high Pearson correlation (p-value <.001; 2-tailed t test at α = 0.01) with 4 time–domain features: correlation coefficients from RR and delayed RR intervals; percentage of RR intervals lying between 0.6 and 1.2 seconds; total variance explained by first principal component computed from RR and delayed RR interval series; and number of datapoints centric at origin in the grid of RR interval and first-order difference of RR intervals. CNN features were comparatively more correlated, likely due to the convolution operations.

Figure 5

Heat map of the correlation matrix showing normalized Pearson correlation between a set of 166 manually extracted and convolutional neural network (CNN)–extracted features. Red box highlights CNN features that are more correlated with time–domain features.

The correlation between features may be problematic in predictive modeling, especially in linear models because collinearity causes singularity. However, nonlinear models are comparatively more robust for collinearity. This may be one reason why the CNN-SVM model performed better even though the CNN features were highly correlated.

Conclusion

In this exploratory study, we developed and evaluated 2 artificial intelligence models: a random forest using engineered ECG features and a CNN. Our results showed that both approaches yield high accuracy in classifying ECG recordings into correct cardiac abnormality class, yet CNN performs slightly better in terms of F1 score. Our results also showed that most of the features extracted within the CNN architecture are not correlated with engineered features. We interpret this as evidence that feature abstraction in the CNN algorithm generates novel predictors carrying out additional discriminating information to engineered features considered in our work. Furthermore, most CNN features are statistically significantly correlated with each other. This suggests that there is room to develop novel CNN architecture designs promoting correlation-based dimension reduction while not losing performance.

Funding Sources

The author has no funding sources to disclose.

Disclosures

The author has no conflicts of interest to disclose.