An automated COVID-19 detection based on fused dynamic exemplar pyramid feature extraction and hybrid feature selection using deep learning.

The new coronavirus disease known as COVID-19 is currently a pandemic that is spread out the whole world. Several methods have been presented to detect COVID-19 disease. Computer vision methods have been widely utilized to detect COVID-19 by using chest X-ray and computed tomography (CT) images. This work introduces a model for the automatic detection of COVID-19 using CT images. A novel handcrafted feature generation technique and a hybrid feature selector are used together to achieve better performance. The primary goal of the proposed framework is to achieve a higher classification accuracy than convolutional neural networks (CNN) using handcrafted features of the CT images. In the proposed framework, there are four fundamental phases, which are preprocessing, fused dynamic sized exemplars based pyramid feature generation, ReliefF, and iterative neighborhood component analysis based feature selection and deep neural network classifier. In the preprocessing phase, CT images are converted into 2D matrices and resized to 256 × 256 sized images. The proposed feature generation network uses dynamic-sized exemplars and pyramid structures together. Two basic feature generation functions are used to extract statistical and textural features. The selected most informative features are forwarded to artificial neural networks (ANN) and deep neural network (DNN) for classification. ANN and DNN models achieved 94.10% and 95.84% classification accuracies respectively. The proposed fused feature generator and iterative hybrid feature selector achieved the best success rate, according to the results obtained by using CT images.

Introduction

Background

Coronaviruses are ribonucleic acid (RNA) viruses that can infect many species of animals and people. Until severe acute respiratory failure syndrome (SARS) appeared, it was known as the cause of 15–30% of the common cold [1]. The common cold is usually associated with mild and self-limiting infections. New types of coronavirus began to appear in 2002. The disease manifested itself in the form of a respiratory infection, which is more severe than normal cold symptoms. The first of these is severe acute respiratory failure syndrome (SARS), which is thought to be transmitted from bats to musk cats and then from cats to people in the Guangdong region of China in February 2003. The infection has affected 8000 people in the world and approximately 800 deaths have been reported [2]. A new case report for SARSCoV was not produced after 2004. The second new coronavirus infection was the Middle East respiratory syndrome (MERS) seen in Saudi Arabia in 2012 and thought to be transmitted from camel to human. The MERS-CoV epidemic that continued until 2018 was detected in 2229 cases and 791 of the cases resulted in death [3].

In 2019, new coronavirus disease (COVID-19) appeared as an infectious disease caused by severe acute respiratory syndrome coronavirus (SARS-CoV-2) and was first reported in the Wuhan province of China [4]. The virus has spread out worldwide and has been declared a pandemic by the World Health Organization (WHO) [5]. Although the most important clinical symptoms are fever and cough, symptoms such as fatigue, headache and shortness of breath can also be seen. However, diagnostic tests are needed because all these symptoms are not specific to the disease and the disease can progress rapidly to severe pneumonia [6]. Although the real-time reverse transcription-polymerase chain reaction (RT-PCR) test for viral nucleic acids in the diagnosis of COVID-19 is the gold standard, computed tomography (CT) has become more and more important in the diagnosis [7].

Motivation

Early detection and treatment of COVID-19 is crucial. Compared to RT-PCR tests, thoracic (lung area) CT imaging is known to be a more reliable, prompt and practical method to diagnose and evaluate COVID-19, especially in the outbreak area [8]. Studies have demonstrated that the sensitivity of CT images achieves higher accuracy [[9], [10], [11], [12], [13], [14]]. According to recent research, sensitivity of the PCR (capacity to detect patients) remains only 71% for COVID-19 infection [15]. However, CT images can be used as a problem-solving approach in patients who are negative for RT-PCR but not diagnosed precisely.

Numerous studies presented in the literature on COVID-19 disease detection. Ozturk et al. [9] proposed DarkCovidNet model for classifying COVID-19, healthy and pneumonia disease. They used X-ray images and achieved 87.02% accuracy rate. Hemdan et al. [10] proposed a deep learning model, which is called COVIDX-Net. They used 25 COVID-19, and 25 healthy images. They obtained 90% success rate. Wang et al. [11] collected 1119 CT images for COVID-19 diagnosis. They proposed a new architecture called M-inception by modifying the classical inception network. In the proposed M-inception method, the features were reduced before the classification layer. In the experiments, 89.5% accuracy with 88% specificity and 87% sensitivity was achieved. Zhao and colleagues [12] proposed a method using transfer learning and data augmentation together with deep learning. In their study, they achieved an accuracy of 84.7% by using 275 CT images in total. Al-Karawi et al. [13] improved a Fast Fourier Transform-Gabor scheme, which predicted in almost real-time, the state of the patient with an average accuracy of 95.37%. They selected randomly 150 images, which were consisting of 75 COVID-19 and 75 non-COVID-19 images. 60% of the total images were used for training, the remaining 40% of the images were used for testing. In the classification step, SVM classifier was used. Loey and colleagues [14] presented a deep transfer learning model with classical data augmentation and conditional generative adversarial network (CGAN). VGGNet16, VGGNet19, AlexNet, GoogleNet, and ResNet50 were used for detecting the COVID-19 infected patient. ResNet50 was found as the most appropriate classifier for detecting the COVID-19 from the chest CT images. They obtained 82.91% testing accuracy by using the classical data augmentation and CGAN.

Contributions

The novelty of this study is to develop a new hybrid and simple handcrafted feature generation model and to achieve high performance by using fewer features. Hence, the contributions of the proposed approach are as follows:

A novel effective feature generation model is developed. This model uses dynamic-sized exemplars to generate more comprehensive features with the pyramid model. The main aim of the presented feature generator is to extract local and global features in detail.

The main problem of the most feature selectors like ReliefF and NCA is to select optimal features automatically. The RFINCA feature selector addresses this issue and uses the efficiency of both feature selectors.

A robust COVID-19 detection framework was developed, and results revealed that the proposed handcrafted features based automated COVID-19 classification method attained higher accuracies than deep learning models (See Table 5).

Table 5

Comparison of the proposed approach with the previous studies.

Study	Dataset reference	Proposed methods	Accuracy (%)
[25]	777 COVID-19708 Healthy (Private Dataset)	DRE-Net	86
[26]	313 COVID-19229 Healthy (Private Dataset)	UNetþ3D DeepNetwork	90.8
[27]	219 COVID-19175 Healthy 224 Pneumonia (Private Dataset)	ResNet þ LocationAttention	86.7
[11]	325 COVID-19740 Healty (Private Dataset)	M-Inception	89.5
[12]	349 COVID-19, 397 Healty CT images [28]	DenseNet	84.7
[13]	275 COVID-19195 Healty (Private Dataset)	FFT-Gabor	95.37
[14]	349 COVID-19, 397 Healty CT images [28]	CGAN	82.91
[29]	496 COVID-191385 others (Private Dataset + [44,45])	CNN	94.98
[30]	100 normal98 lung cancer397 other (Private Dataset + [46,47])	DL multitask	93
[31]	564 COVID-19660 non-COVID-19 [51]	VGG16 based lesion-attention DNN	88.6
[32]	313 COVID-19229 without COVID-19 (Private)	UNet	90.1
[33]	413 COVID-19439 non-COVID-19 [49,50]	ResNet-50 + 2D CNN	93.02
[34]	460 COVID-19397 non-COVID-19 [51,52]	SqueezeNet	83
[35]	1029 COVID-191695 non-COVID-19 (Private)	AH-Net + DenseNet121	90.8
[36]	53 COVID-1997 other (Private)	SVM	98.71
[37]	51 COVID-1955 control(Private)	UNet++	98.85
[38]	230 COVID-19130 normal (Private)	AD3D-MIL	97.9
[39]	521 COVID-19397 normal76 bac. pneu. (Private)	DLShuffleNet V2	85.40
[40]	COVID-19/other pneu./healthy. (Private)	3D UNet-based network	94
[41]	284 COVID-19,281 CAP, 293 SPT,306 HC	CCSHNet	97.04
[42]	219 COVID-19, 1345 pneumonia and 1341 normal images [50]	mAlexNet + BiLSTM	98.70
[43]	361 COVID-19,1341 Normal and 1345 Pneumonia	InstaCovNet-19	99.08
Proposed method	349 COVID-19, 397 Healty CT images [28]	FDEPFGN and RFINCA	95.84

Organization

Information about the used CT image corpus is provided in section 2. The proposed image classification framework is explained in Section 3. Section 4 presents a complete experimental study of the proposed CT based COVID-19 detection framework. The overall discussion is given in section 5. Lastly, the conclusion part is given in section 6.

Material

Chest CT scan is an effective method for the diagnosis of COVID-19. The radiological knowledge of physicians does not always provide an exact diagnosis. At the same time, it takes a longer time to get results with the nose, throat swabs, and blood tests. Therefore, chest CT images have an important role in the management and follow-up of patients who are hospitalized. The dataset [12] used in this study is publicly available. This data set was confirmed by senior radiologists from Tongji Hospital in Wuhan, China, who diagnosed and treated a large number of COVID-19 patients during the emergence of this disease between January and April. The dataset contains 349 COVID-19, 397 healthy CT images. Fig. 1 shows the COVID-19 and healthy chest CT images.

Fig. 1

Pictorial demonstration of the CT image dataset.

The proposed image classification method

The main goal of this work is to present a novel handcrafted features based CT image classification approach for COVID-19 detection. To achieve this goal, this work presents a novel residual dynamic-sized exemplars based pyramid model. The used feature generation functions are local binary pattern (LBP) and statistical features [[16], [17], [18]]. By using these functions and dynamically sized exemplars-based pyramid model together, a novel feature extraction network is developed. The proposed method is called fused dynamic sized exemplars based pyramid feature generation network (FDEPFGN). After the feature generation process, the generated features are utilized as an input of the ReliefF that selects positive weighted features and eliminates negative weighted features. In RFINCA [19] feature selector phase, the selected positive weighted features by ReliefF are forwarded to iterative NCA to select the optimum number of best features automatically. The selected features are then used as an input of the classifier. The graphical representation of this framework is shown in Fig. 2 .

Fig. 2

Graphical representation of the proposed FRDEPFGN and RFINCA based CT image classification framework.

Fig. 2 denotes the graphical explanation of the developed CT image classification framework. This model is an iterative model, where firstly, the used input (CT) RGB image is transformed into a gray-level image. Three types of the exemplars are used for feature generation and the sizes of these exemplars are 16 × 16, 32 × 32, and 64 × 64. This model generates features using three levels. For the level creation, bilinear interpolation is used. At each level, bilinear interpolation halves the size of the image. The extracted features in each level from the exemplars are concatenated and the informative ones are selected by RFINCA. These informative features are classified by using DNN and ANN classifiers.

Although, Fig. 2 explains the proposed FRDEPFGN and RFINCA based CT image classification method graphically, in order to explain the proposed approach as a whole in a clear way, the pseudo-code of the proposed FRDEPFGN and RFINCA based CT image classification approach is given in Fig. 3 . Lines 01–19 express preprocessing, feature generation, and feature concatenation steps of the proposed approach. Actually, these lines (Lines 01–19) define the FRDEPFGN network. Line 20 shows the RFINCA feature selector and Line 21 illustrates classification phases. LBP generates 59 features and the statistical feature generation function (SF) extracts 19 features. Therefore, LBP and SF generate 78 features together as can be seen in Line 05 and Line 12. The details of the proposed framework are explained in the subsections.

Fig. 3

Pseudo code of the proposed RDEPFGN and RFINCA based CT image classification method.

Preprocessing

The first phase of the proposed FRDEPFGN and RFINCA based CT image classification framework is preprocessing. In this phase, RGB to gray conversion is carried out to obtain a 2D image. Then, the obtained CT image is resized as 256 × 256 dimensions. The first and second steps of our method are given below.

Transform CT image to 2D image.

Resize the 2D CT image to 256 × 256 sized image.

Feature generation

Feature generation is one of the crucial phases of pattern recognition. A novel model, which is called as FRDEPFGN, is proposed to generate features. The proposed FRDEPFGN is inspired from deep networks. Especially, inception network uses dynamic sized convolution operator. Therefore, FRDEPFGN uses dynamic sized exemplars. FRDEPFGN uses two feature generation functions, which are explained in the subsections.

Statistical feature generation

Statistical feature generation has been widely used in the literature and it is one of the first known handcrafted feature generation methods. Numerous statistical moments can be used in the statistical feature generation, such as maximum, minimum, average, energy, skewness, kurtosis, variance, and standard deviation. These methods work effectively and the computational complexity is low. In this work, 19 statistical functions are used for statistical feature generation. The mathematical definitions of the used statistical function are given below. where is CT image. By using the defined 19 statistics above, the used SF is created. Therefore, SF extracts 19 features from an image or an exemplar.

Textural feature generation

LBP, which is one of the widely preferred feature generators, is used as a second feature generation function. There are many LBP like feature extractor in the literature. Since the time complexity of the LBP is , it extracts distinctive features and has a simple structure. Therefore, it can be programmed easily and has been used in many study areas [20]. LBP is applied to generate textural features of the CT images and the steps are as follows:

Divide image into overlapping windows with size of 3 × 3.

Generate bits using signum, center pixel and neighborhood pixels together.

Calculate decimal map value by using bits extracted.

Generate histogram and obtain feature vector.

These steps are defined for LBP feature generation function. A numerical example of LBP is shown in Fig. 4 graphically.

Fig. 4

Graphical representation of LBP feature generation.

LBP divides the image into 3 × 3 sized overlapping blocks to generate features. In Fig. 4, the center pixel value is 52. This pixel is compared to its neighborhood pixels to generate binary features using the signum function. The generated bits are then converted into a decimal value.

Description of the proposed fused residual dynamic exemplar pyramid feature generation network

Steps of the proposed FRDEPFGN method are given in this section. where expresses th exemplar of the th type. Three types of exemplar are used in this work and they are categorized according to size.

Extract textural and statistical features from image by using LBP and SF function.

Divide CT image into exemplars () with size of 16 × 16, 32 × 32 and 64 × 64.

Generate features using three types of exemplars.

Concatenate generated features.

Decompose CT image by using bilinear interpolation. In this step, W x H sized images are resized to W/2 x H/2 sized images. By using this step, a multileveled feature generation method is obtained.

Repeat Step 4, Step 5, Step 6, Step 7 three times.

Concatenate all extracted features.

RFINCA feature selection

RFINCA aims to use the effectiveness of the ReliefF and NCA selectors together and selects the optimum number of features automatically. Because ReliefF and NCA cannot select the optimum number of features automatically alone. In the feature extraction phase, the proposed FRDEPFGN generates 34,632 features in total, since the used feature extraction functions are run 444 times in the proposed approach. In order to select the most informative features among these 34632 features, RFINCA, which is the combination of ReliefF and NCA, is used as a feature selector. ReliefF and NCA are distance-based feature selectors where ReliefF can generate negative weights, on the other hand, NCA generates non-negative weights. Generally, negative weighted features can be assigned as redundant features. Therefore, in the first step, ReliefF is applied to all features in order to calculate negative weighted features and eliminate them. In the second step, iterative NCA is applied to the selected positive weighted features [21].

To select an optimal number of features, the error calculation of the k-nearest neighbor (k-NN) classifier is used. Feature range from 20 to 1000 was also used to decrease the complexity of the RFINCA. To better explain the proposed RFINCA, the flow diagram is shown in Fig. 5 [19].

Fig. 5

Flow diagram of the RFINCA.

According to this diagram (see Fig. 5), ReliefF selects positively weighted features from the extracted features. This step is called redundant feature elimination. The selected positive weighted features are utilized as an input of the NCA, then NCA weights are calculated. By using these weights, an iterative feature selection is realized. The error of the selected feature vector is calculated by using k-NN (or any other classifier can be used). The minimum error valued feature vector is selected as an optimal feature vector.

In order to clearly explain the steps of the RFINCA, the pseudo code of the procedure is also given in Fig. 6 .

Fig. 6

Graphical representation of the RFINCA procedure.

Lines 01–03 define min-max normalization. Since both NCA and ReliefF are distance-based feature selectors, normalization is applied to obtain effective results. Lines 04–11 express the ReliefF phase and Lines 12–18 shows iterative NCA. The optimal number of features () selection is denoted in Lines 20–22. The 10th step of this work is given as follows:

Select the optimal/informative features by using RFINCA.

RFINCA selects 989 features from the extracted 34,632 features.

Classification

In the last step, deep neural network (DNN) [[22], [23], [24]] is used with 10-fold cross-validation. DNN is the enhanced type of the artificial neural network (ANN), and has 2 or more hidden layers. The employed DNN is backward network and utilizes scaled conjugate gradient (SCG) for learning, since it needs gradient computing of functions. SCG algorithm utilizes steepest descent direction. During the implementation of the DNN approach, first weights are randomly allocated and (input of hidden layers) are computed using Eq. (32).where is allocated weights, expresses inputs and is activation function. Then, the weights are recalculated by employing the backpropagation technique. In this step SCG that is a steepest optimization method is utilized and it employs orthogonal vectors to minimize the error. The mathematical notation of SCG is given in Eqs. (33), (34), (35)).

Herein, is multiplier, orthogonal vector and is input. By employing this optimization approach, weights are recalculated. The selected 989 features are employed as an input to the SCG based 3 hidden layered DNN to assess the performance of the proposed feature extraction and selection framework. There is no regular approach presently for constructing an optimal neural network with an appropriate number of layers and number of neurons in each layer. For this reason, we empirically construct the DNN by carrying out different experiments. We have manually adjusted a DNN in every experiment by tuning the number of hidden layers, the number of nodes creating the layer for each hidden layer, the number of learning steps, learning rate, momentum, and the activation function. We employed an SCG optimization technique for the backpropagation algorithm and set the learning rate to 0.7, momentum to 0.3, and batch size to 100. We have calculated the classification accuracy, using 10-fold cross-validation for every manual formation to look for the remaining DNN hyperparameters. This search procedure is repeated for different sizes of the hidden layer representations. After this exhausting manual process, the best performance in the classification is achieved with a DNN consisting of 3 hidden layers with 270, 150, and 50 nodes respectively. In the ANN implementation, a similar process is carried out, and the best performance is achieved with a hidden layer of 250 neurons. The scaled conjugate gradient is selected as an optimizer in this work. The tangent sigmoid is selected as an activation function. Moreover, batch normalization is employed in the model.

Classify the selected features by using ANN or DNN classifier with 10-fold cross-validation. Training/validation curve for ANN and DNN are shown in Fig. 7, Fig. 8 respectively.

Fig. 7

Training/validation curve for ANN.

Fig. 8

Training/validation curve for DNN.

Experimental results

The CT images are utilized as an input of the proposed framework. The details of the used dataset are provided in Section 2. These images include two categories, which are COVID-19 and healthy CT images. Ten-fold cross-validation is selected as validation and testing methodology. This method was carried out on a personal computer (PC) using MATLAB (2018a) programming environment. The used PC has simple configurations (i5 1.8 GHz CPU 8 GB RAM). This method does not require the use of a graphical processing unit or a parallel computing structure. In the classification phase, ANN and DNN classifiers are used. The obtained results are listed in Table 1 according to the folds.

Table 1

The calculated accuracy rates (%) of the ANN and DNN according to the folds.

Fold	ANN	DNN
1	82.43	83.78
2	88.0	90.67
3	92.0	92.0
4	96.0	100.0
5	96.0	98.67
6	97.33	98.67
7	94.67	97.33
8	97.30	98.65
9	98.65	98.65
10	98.65	100.0
Average	94.10	95.84

Per Table 1, ANN and DNN achieve 94.10% and 95.84% classification accuracies respectively. To give details of these results, confusion matrices of the used classifiers are shown in Table 2 and Table 3 . Recall, precision, and accuracy parameters can be extracted from these matrices. By using recall and precision, F1-score can be calculated. Since F1-score is the harmonic mean of the precision and recall and mathematical expression of it is given in Eq. (32). Geometric mean and balanced accuracy can also be calculated using Eqs. (33), (34)).

Table 2

Confusion matrix of the ANN classifier.

Actual classes	Predicted classes		Recall
	COVID-19	Healthy
COVID-19	326	23	0.9341
Healthy	21	376	0.9471
Precision	0.9395	0.9423	0.9410

Table 3

Confusion matrix of the DNN classifier.

Actual classes	Predicted classes		Recall
	COVID-19	Healthy
COVID-19	330	19	0.9456
Healthy	12	385	0.9698
Precision	0.9649	0.9530	0.9584

The calculated F1-score and geometric mean and accuracy are also listed in Table 4 .

Table 4

The calculated F1-score and geometric mean and accuracy of the ANN and DNN classifiers.

Classifier	Class	F1-score	Geometric mean	Accuracy
ANN	COVID-19	93.68	–	–
	Healthy	94.47	–	–
	Average	94.08	94.06	94.06
DNN	COVID-19	95.52	–	–
	Healthy	96.13	–	–
	Average	95.83	95.76	95.77

Discussions

This work presents a novel CT based COVID-19 detection approach by using a new handcrafted feature generation network and a new hybrid and iterative feature selector. The proposed feature generator is inspired by the inception network and residual network. Therefore, a novel residual dynamic exemplar pyramid model is used to generate the features comprehensively. 16 × 16, 32 × 32, and 64 × 64 sized exemplars are used and the pyramid model is constructed by using bilinear interpolation. RFINCA selects the most informative or discriminative features and automatically determines the optimum features. Feature generation and feature selection are two fundamental problems in pattern recognition and classification. These problems need to be resolved to obtain high classification accuracy. This study resolved these issues using the proposed FRDEPFGN and RFINCA approaches.

The proposed framework achieves high classification accuracies (See Table 1). To indicate the success of the proposed framework, comparisons with the state-of-the-art methods are listed in Table 5.

X-ray images are generally preferred for COVID-19 diagnosis studies. But, in this study, high performance (95.84%) is obtained using chest CT images. As it can be seen from Table 5, one of the highest performance is the proposed approach with an accuracy of 95.84%. Although the obtained results are satisfactory, we believe that higher accuracy can be achieved by employing more CT images.

Moreover, the developed model uses handcrafted features, and in the classification phase, DNN classifier is used. The primary aim of this model is to reach high accuracies using handcrafted features. Since the number of samples is less, the transfer learning based models must be utilized in this case. But the accuracy of the transfer learning based models [14,33] are lower than the proposed approach. Since, the proposed model utilizes the handcrafted and simple feature generation methods, the computational cost of the proposed model is less than the transfer learning based models. Furthermore, if we compare the proposed model performance with the state-of-the-art for COVID-19 detection using CT images, most of the state-of-the-art models have achieved lower accuracy than the proposed approach (See Table 5).

The advantages of the developed FDEPFGN and RFINCA based chest CT image classification approach are given below.

A novel dynamic exemplar pyramid feature extraction method is developed.

A novel iterative feature selection method is proposed.

ANN and DNN classification results show that the developed framework successfully extracts and selects features.

The proposed method is a cognitive method, since there is no need to set millions of parameters as deep learning networks.

Conclusions

This paper presents a hybrid FDEPFGN and RFINCA framework for COVID-19 detection from a limited number of chest CT images. Our model is based on generating features with FDEPFGN and selecting informative features with RFINCA. DNN and ANN are employed to test the model. Evaluation of the proposed method is measured with different performance metrics such as precision, geometric mean, F1 score, and accuracy. Overall, our experiments show that the developed FRDEPFGN and RFINCA based CT image classification framework can achieve a better performance than 16 state-of-the-art approaches (See Table 5). The future work may contain the following aspects: (i) Expand the size of the dataset and test our proposed model on a larger dataset. (ii) Try to use some pretrained deep learning architectures. (iii) Some advanced or hybrid fusion rules will be tested by using pretrained deep learning architectures. (iv) Our proposed method can be embedded into other automated healthcare systems.

Funding

There is no funding source for this article.

CRediT authorship contribution statement

Fatih Ozyurt: Conceptualization, Data curation, Formal analysis, Visualization, Writing – original draft. Turker Tuncer: Methodology, Resources, Validation, Visualization, Writing – original draft. Abdulhamit Subasi: Conceptualization, Methodology, Supervision, Writing – review & editing.

Declaration of competing interest

The authors declare that they have no known competing financial interests or personal relationships that could have appeared to influence the work reported in this paper.