Automatic Diagnosis of Coronavirus (COVID-19) Using Shape and Texture Characteristics Extracted From X-Ray and CT-Scan Images.

Automatic diagnosis of coronavirus (COVID-19) is studied in this research. Deep learning methods especially convolutional neural networks (CNNs) have shown great success in COVID-19 diagnosis in recent works. But they are efficient when the depth of network is high enough. However, the use of a deep network requires a sufficiently large training set, which is not available in practice. From the other hand, the use of a shallow CNN may not provide superior results because it is not able to rich feature extraction due to lacking enough convolutional layers. To deal with this difficulty, the contextual features reduced by convolutional filters (CFRCF) is proposed in this work. CFRCF extracts shape and textural features as contextual feature maps from the chest X-ray radiographs and abdominal computed tomography (CT) images. Morphological operators, Gabor filter banks and attribute filters are used for contextual feature extraction. Then, two convolutional filters are applied to the contextual feature cube to extract the nonlinear sub-features and hidden relationships among the contextual features. Finally, a fully connected layer is used to produce a reduced feature vector which is fed to a classifier. Support vector machine and random forest are used as classifier. The experimental results show the superior performance of the proposed method from the recognition accuracy and running time point of view using limited training samples. More than 76% and 94% overall classification accuracy is obtained by the proposed method in CT scan and X-ray images datasets, respectively.

Introduction

A novel coronavirus called COVID-19 has been emerged since December 2019 in Wuhan, China [1]. Rapidly, it has been declared as an epidemic disease with many other cases across the world [2]. By 30 September 2020, a total of 33.7 million/1.01 million patients have been diagnosed/died with COVID-19 infection. Because of limited number of available COVID-19 test kits and also limited ability and capacity of hospital staff, intelligent and automatic diagnosis of coronavirus is a vital and challenging task. The best tools for COVID-19 infection diagnosis are chest X-ray radiographs and abdominal computed tomography (CT) images [3].

So far, several studies have been worked on the available X-ray and CT images to automatically diagnose the coronavirus (COVID-19). Most of them have been concentrated on deep learning methods especially convolutional neural networks (CNNs). Due to success of deep neural networks for image classification in various applications, they have been attend in many research works [[4], [5], [6]]. Deep learning is also used in various medical applications. A method for white blood cells classification is proposed in [7] which uses the best selected features achieved by a fitness function and extreme learning machine (ELM). Brain tumor classification is done using ELM in [8]. The discrete cosine transform is used for feature extraction. Then, an ELM based approach is used to select the best features. After feature fusion, ELM is used for final classification. Deep learning is used for feature fusion in an automatic system for stomach infection recognition [9]. Attention to this point is necessary that deep neural networks such as CNNs need large scale labeled images for training. But, due to limited available medical images, transfer learning is used in almost all deep learning methods in medicine field. In the transfer learning mechanism, the knowledge obtained from other tasks such as recognition of various objects is transferred to a specific domain. Three deep networks such as ResNet50 trained on the ImageNet dataset have been analyzed on chest X-ray images in [10] and considerable results have been reported. A developed version of CNN called decompose, transfer and compose (DeTraC) has been adopted for COVID-19 diagnosis through chest X-ray image classification in [11].

Selection and extraction of appropriate features has high importance in any pattern recognition problem. A deep learning architecture is proposed in [12] for multi-layer deep features fusion where it does feature extraction based on transfer learning. In [13], both hand-crafted and CNN deep features are used for classification of gastric infections. The extracted features are fused and the best features are selected using a genetic algorithm. The modified mask recurrent CNN based ulcer segmentation is proposed for diagnosis of gastrointestinal diseases such as ulcer. The grasshopper optimization along with minimum distance fitness function is used to optimize the achieved deep features [14]. A Newton-Raphson feature selection method is combined with a deep learning model for skin cancer recognition in [15]. Conventionally, relatively deep CNNs have shown great performance because they do feature extraction layer by layer. A brief review of recently published works about diagnosis of COVID-19 through applying deep learning networks on radiology images is represented in Table 1 . More abstract sub-features and details are extracted from deep layers which simplify discrimination between different classes. But, deep networks have hyperparameters and as said before, they need a high number of training samples to properly be learned. From the other hand, training a deep network has high computational burden.

Table 1

Deep learning networks for COVID-19 diagnosis using radiology images.

Ref (year)	Image type	Model	Description
[33] (2020)	X-ray	Hybrid deep learning	Visual geometry group based neural network (VGG) is combined with spatial transformer network (STN) and CNN
[34] (2020)	CT	Multi-scale CNN (MSCNN)	MSCNN is used to learn scaled-invariant patterns based on multi-scale spatial pyramid decomposition.
[35] (2020)	X-ray	Integrated stacked deep convolutional network	The pre-trained models such as ReNet101 and MobileNet are used to compensate for limited training data
[36] (2021)	X-ray	Deep features and SVM	VGG19, AlexNet, ResNet and GoogleNet are used for deep feature extraction. Then, metaheuristic algorithms are used for feature selection. Finally, SVM is used for classification.
[37] (2021)	CT	Dual-branch combination network (DCN)	DCN simultaneously achieves individual-level lesion segmentation and classification.
[38] (2020)	X-ray	Deep learning based CNN called as nCOVnet	A CNN model with 24 layers is introduced where the VGG16 is used as the base model and five custom layers are used as the head model.
[39] (2020)	X-ray	SqueezeNet	Deep SqueezeNet is used with Bayes optimization and a detailed augmentation.
[40] (2020)	X-ray	Deep transfer learning	Transfer learning is used to train ResNet18, ResNet50, DenseNet-121 and SqueezeNet.
[41] (2020)	X-ray	DarkCovidNet	DarkCovidNet inspired by DarkNet contains 17 layers with different filtering on each layer.
[42] (2020)	X-ray	CVDNet, a deep CNN	CVDNet is based on residual neural network constructed by two parallel levels to capture local and global features.
[43] (2020)	X-ray	A Siamese neural network	Contrastive learning is integrated with a pre-trained ConvNet encoder to achieve unbiased feature representation. A Siamese network is learned for final classification.
[44] (2021)	X-ray	ResNet based deep learning	The framework is composed of two deep learning models. The first model is for discrimination of COVID-19 from other infections. The second model is for localization that assigns the recognized X-ray into left lung, right lung or bipulmonary.
[45] (2020)	X-ray	Convolutional CapsNet	Convolutional CapsNet uses the capsule networks for binary and multi-class classification.
[46] (2020)	X-ray	CNN with gravitational search optimization (GSA)	The DenseNet121 is used as the considered CNN architecture where hyperparameters are set by GSA.
[47] (2020)	CT	CNN based transfer learning-Bidirectional ling short-term memory (BiLSTM)	A hybrid structure containing AlexNet architecture and transfer learning is proposed. The BiLSTM is also used to take into account the temporal properties.
[48] (2020)	X-ray	Combined CNN-LSTM	CNN is used for feature extraction and LSTM is used for detection.
[49] (2020)	X-ray	Concatenation of Xception and ResNet50V2	Multiple features are extracted by two robust networks: Xception and ResNet50V2
[50] (2020)	X-ray	COVIDX-Net	COVIDX-Net assesses seven different deep networks including VGG19 and the second version of Google MobileNet.
[51] (2020)	X-ray	DWT + CNN	The discrete wavelet transform (DWT) and CNN are used for feature extraction, minimum redundancy and maximum relevance (mRMR) is used for feature selection and random forest-based bagging approach is used for classification.
[52] (2021)	X-ray, CT	CMT-CNN	Contrastive multi-task CNN (CMT-CNN) encourages local aggregation with a contrastive loss.
[53] (2021)	X-ray	CNN based models: a comprehensive study	Eight pre-trained CNN models such as VGG16, AlexNet and GoogleNet are assessed.
[54] (2021)	CT	CCSHNet	CCSHNet uses a novel transfer learning and determines the best two pre-trained models. It uses a discriminant correlation analysis based fusion method.

In addition to deep neural networks, which do feature extraction and classification in an end-to-end and unified framework, there are other pattern recognition methods which do feature extraction and classification individually. For example, the work in [16] uses feature extraction methods such as Gray level co-occurrence matrix (GLCM) [17] and local directional pattern (LDP) for feature extraction, and support vector machine (SVM) for classification.

There are various transforms for feature extraction. The chest X-ray and CT scan images of COVID-19 patients have different shape, geometrical structures and texture with respect to normal persons or people with other types of diseases. So, extraction of shape and textural features can be useful for coronavirus image classification. The morphological filters, which have been shown great success in contextual feature extraction in various fields [18,19], have been used to provide morphological profile (MPs). The extended version of morphological filters, known as attribute filters with flexibility in selection of attributes to be analyzed, have been used to provide the extended multi-attribute profile (EMAP) [20]. Another powerful texture descriptor is Gabor filter with capability of texture analysis in different scales and directions [21,22].

In this work, MP, Gabor filters and EMAP are used for contextual feature extraction. But, the direct use of these feature cubes is not so appropriate because 1- the obtained feature cubes have high dimensionality and 2- there are correlation, nonlinear characteristics and hidden relationships among the contextual feature maps which should be extracted. To this end, two convolutional filters are applied to the obtained contextual feature cube to extract more abstract sub-feature maps containing delicate shape and textural details for discrimination between COVID and NanCOVID images. Then, the feature maps are flattened and a fully connected layer is used to extract the final reduced feature vector. The extracted feature vector is given to a classifier to find the class label. SVM [23,24] and random forest (RF) [25] are used as classifiers.

The proposed method called contextual features reduced by convolutional filters (CFRCF) is assessed on two different datasets: chest X-ray radiographs and abdominal CT scan images. Various measures are computed to provide a comprehensive comparison. CFRCF is compared with different cases when MP, Gabor filter bank, EMAP and local binary pattern (LBP) [26,27] are used for feature extraction and SVM and RF are used as classifier. The experimental results show the superior performance of the proposed methods for discrimination between COVID-19 and normal (or other diseases). Some contributions of the CFRCF method are represented in the following:

The use of shape and textural features extracted by MP, Gabor and EMAP transforms simplifies discrimination between COVID and NanCOVID images.

Convolutional filters extract more abstract and detailed contextual features which improve diagnosis accuracy. In addition, convolutional operators extract the hidden relationship among the extracted contextual feature maps.

FC layer is used for feature reduction and discarding redundant information.

The proposed CFRCF method can achieve good classification results using limited training samples because of some reasons:

For initial feature extraction, the unsupervised contextual feature extraction methods (EMP, Gabor and EMAP) are used without any requirement to the labeled samples.

A simple structure containing just two convolutional layers is suggested for feature extraction using CNN. Therefore, the number of trainable parameters is not large, and so the overfitting problem is relatively avoided.

In the classification phase, two classifiers with low sensitivity to the number of training samples (SVM and RF) are used, which are relatively efficient even with small training sets.

This paper is organized as follows: the proposed method is described in section 2. Different contextual feature transformations containing morphological profiles, Gabor filters and attribute profiles are reviewed, and then, the use of convolutional filters for nonlinear extraction of hidden features are described. Datasets, evaluation measures, parameter settings and experimental results are reported in section 3. Finally section 4 concludes the paper.

Contextual features reduced by convolutional filters

Although several deep learning based methods such as CNN model have been proposed recently for COVID-19 diagnosis, but, they are often efficient when a high number of training samples is available; or when different methods such as transfer learning, data augmentation or semi-supervised approaches are used to deal with this difficulty. This work proposes a simple shallow network for COVID-19 diagnosis, which utilizes the advantages of shape and textural feature extractors to compensate for lack of the convolutional layers. From the other hand, most of the CNN based methods use a CNN model for both feature extraction and classification in a unified framework, which is not so applicable when limited number of training samples is available. So, in contrast to conventional CNN based methods, the proposed method uses two convolutional layers for nonlinear extraction of hidden sub-features and a fully connected layer for feature reduction. The extracted features are then given to a classifier with low sensitivity to the number of training samples (SVM or RF). The proposed framework not only provides high accurate results using small training sets but also runs very fast in real applications.

To discriminate between CT scan image (or X-ray image) of a COVID-19 patient and a NanCOVID-19 person (a normal person or a person stricken to other disease), noting to shape features, geometrical structures and textural differences is important. The contextual information of an image can be extracted by applying the contextual image transformations. These filters are usually mappings that transform an image pixel as a function of gray levels of a set of neighboring pixels. To extract shape features, morphological filters are appropriate tools. In addition, attribute filters have more flexibility in shape feature extraction with defining any arbitrary attributes. To extract textural features in different scales and directions, Gabor filter bank is an appropriate choice. So, morphological profile (MP), Gabor filters and extended multi-attribute profile (EMAP) are used for contextual feature extraction in this study. The extracted features can be given to a classifier such as SVM for classification. But, the extracted feature cubes (MP, Gabor and EMAP) have high volume which require to feature reduction before giving to the classifier. Conventionally, the principal component analysis (PCA) transform [28] is used for feature reduction. The first principal component (PC1) containing the most energy of the feature cube is a two dimensional image. The PC1 is reshaped to a form a feature vector. But, the use of PCA for feature reduction has a main disadvantage. Some informative features which contains important details for discrimination between COVID and NanCOVID classes may be removed with discarding the PCs with small variances.

To deal with this difficulty, contextual features reduced by convolutional filters (CFRCF) is proposed in this work. According to the CFRCF method, a convolutional neural network (CNN) is used for feature reduction. To this end, at first a CNN containing two convolutional (Conv.) layers and two fully connected (FC) layers is trained to separate COVID and NanCOVID images. Then, the trained network is used for feature reduction where its input is the contextual feature cube extracted by MP, Gabor or EMAP. Two convolutional layers consecutively extract sub contextual features. The convolutional operators can extract more abstract and detailed contextual features and also extract correlation and hidden relationships among the contextual features. The feature cube extracted by the second convolutional layer is flattened and given to a FC with neurons. By applying an activation operator in the output of this layer, in the trained CNN, a feature vector is extracted which can be given to classifier for diagnosis. With adding a FC with neurons, the feature cube achieved in the second convolutional layer is reduced to a feature vector. With this feature reduction, not only, we deal with the overfitting problem, but also, remove the non-informative and redundant features which may degrade the classification accuracy. SVM and RF are used here because they are powerful classifiers with relatively little sensitivity to the training set size. Fig. 1 shows the general block diagram of CFRCF and Fig. 2 illustrates the suggested structure of CNN in CFRCF. In the following, the used contextual feature extraction methods and the proposed structure of CNN for feature reduction are described with more details.

Fig. 1

General block diagram of the proposed CFRCF method.

Fig. 2

Suggested structure of CNN in CFRCF.

Morphological profile

A morphological filter by reconstruction reduces the image complexity by attenuating unimportant details while preserves the geometrical structures of the regions. Result of the transform depends on interaction of structures present in the image with neighborhood region of the filter. The closing/opening operator by reconstruction suppresses darker/brighter regions with respect to gray level values of adjacent areas, which are smaller than the used structuring element (SE) as a moving window. The other structures remain unchanged. SE by specifying the neighborhood shape and size determines how amount contextual relations are involved in the image analysis. The opening and closing operators are applied on each image channel individually. The outputs are stacked together to form a morphological profile (MP). For a given image, the MP is constructed consisting of feature maps where is the number of applied opening (and closing) filters.

Gabor filter bank

Gabor filters attract considerable attention for texture description. The Gabor filters result in joint localization in both original (space) and transform (spatial-frequency) domains. The Gabor functions are inspired from the human visual system, and so, they have high capability in texture interpretations. A filter bank containing Gabor filters with varied scales and directions is used to acquire localization characteristics in both spatial and frequency domains. The result of convolution of these filters with the given image is a Gabor feature cube. A Gabor filter is composed of a sinusoidal function which is modulated by an envelope with Gaussian shape. The impulse response of a Gabor filter is defined by [29]:where is the imaginary unit, i.e., , is centered at frequency and is the Gaussian envelope:

A Gabor filter bank containing scales and directions are applied to the given image. The result is a Gabor feature cube with feature maps.

Attribute profiles

Attribute filters as an extension of morphological filters are flexible in definition of attributes for contextual information modelling. An Attribute profile is obtained by applying a set of attribute filters to a gray level image. These filters, defined in the mathematical morphology framework, merge connected components at different levels. The attribute filter removes the connected components that do not satisfy a given criterion. The criterion evaluates the considered attribute extracted from the image regions. To this end, the attribute computed for a given connected region , , is compared to a threshold . If , the region remains unchanged; otherwise, it is set to the gray level of adjacent region with nearer value. Merging the region with the adjacent region with lower/greater gray level is known as thinning/thickening. An attribute profile (AP) generated from an image with a given attribute is acquired by applying a set of attribute thinning and thickening filters by a sequence of thresholds as follows:where / denote the thickening/thinning transformations. AP of each image channel is obtained. The obtained APs are stacked to provide the extended multi-attribute profile (EMAP). EMAP is usually obtained with considering different attributes not a single one.

Convolutional neural network

Convolutional filters due to their ability in feature extraction from the neighborhood regions are among the best well known operators for nonlinear feature extraction. CNNs utilize the local connections to hierarchically extract contextual information layer by layer by applying convolutional filters. Moreover, by utilizing the shared weights, they significantly reduce the number of parameters. A CNN structure is proposed for feature extraction in this work. This CNN accepts the contextual feature cube extracted by MP, Gabor or EMAP as input. At first, the whole CNN is trained to learn weights. Then, the output of the first fully connected (FC) layer is activated and used as the extracted feature vector with reduced dimensionality. The following structure is suggested in this work:

Two dimensional convolutional (Conv.) layer containing 16 kernels with the size of .

BatchNormalization (BN) layer for normalizing the feature cube across a mini-batch.

Rectified Linear Unit (ReLU) layer which applies a threshold operation to each element of the extracted feature cube to set any value less than zero to zero.

Conv. layer containing 32 kernels with the size of

BN layer

ReLU layer

FC layer with neurons to produce a feature vector.

FC layer with neurons corresponding to two classes of COVID and NanCOVID.

Softmax layer which applies a Softmax function to the final FC neurons to compute the conditional probability of the image given class COVID or NanCOVID.

Output of step 7 in the trained CNN is the reduced feature vector which given to a classifier (SVM or RF) for classification. According to experiments, an appropriate value for the parameter is 100. So, a 100 1 feature vector is achieved from each medical image.

A two dimensional convolutional operator in the convolutional layer is defined as follows [30]:where is the activation function; is the value of neuron at position for feature map and layer ; is the index of feature map in th layer connected to th feature map; is the weight of position (, which is connected to the feature map ; is the height/width of the convolution kernel and denotes the bias of feature map in layer .

The use of BN and ReLU layers help to avoid overfitting problem. ReLU as a nonlinear operation is used in hidden layers to improve the performance. It returns zero if the input of a neuron be negative. Adam optimizer is used to compile the model. The output of the second convolutional layers, i.e., the extracted sub feature maps, when input of CNN is MP, Gabor and EMAP, are shown for CT dataset in Fig. 3, Fig. 4, Fig. 5 and for X-ray dataset in Fig. 6, Fig. 7, Fig. 8 , respectively.

Fig. 3

Extracted contextual feature maps for CT dataset when MP is used as input of the convolutional filters.

Fig. 4

Extracted contextual feature maps for CT dataset when Gabor feature cube is used as input of the convolutional filters.

Fig. 5

Extracted contextual feature maps for CT dataset when EMAP is used as input of the convolutional filters.

Fig. 6

Extracted contextual feature maps for X-ray dataset when MP is used as input of the convolutional filters.

Fig. 7

Extracted contextual feature maps for X-ray dataset when Gabor feature cube is used as input of the convolutional filters.

Fig. 8

Extracted contextual feature maps for X-ray dataset when EMAP is used as input of the convolutional filters.

Experiments

Datasets and evaluation measures

Two public datasets are used for doing experiments: abdominal CT scan images and chest X-ray images. The COVID-19 CT image dataset is publicly available on https://github.com/UCSD-AI4H/COVID-CT [31] accessed in 8 April 2020. It contains 349 CT samples of clinical findings of COVID-19 and 397 CT images of NanCOVID. The X-ray dataset is publically available on https://github.com/ieee8023/covid-chestxray-dataset [32] accessed in 18 April 2020. It contains 187 and 73 chest X-ray images of COVID-19 patients and NanCOVID ones, respectively. All images are resized to 150 150 pixels in both datasets.

Due to limited number of samples in X-ray dataset, the enhanced contrast of X-ray images obtained by histogram equalization are obtained and added to the original X-ray images for data augmentation. In other words, the number of samples in X-ray dataset becomes twice. Table 2 represents the number of samples in each dataset and also the number of augmented samples in X-ray dataset. In CT scan dataset, 70% of data samples are used for training and 30% for testing and in X-ray dataset 90% of data is used for training and 10% for the testing. Also, in both datasets, 10% of training samples are considered as validation samples and the remainder of samples are used as training ones.

Table 2

The number of samples in each class of datasets.

Dataset	Class	No. of samples	No. of augmented samples
CT	COVID	349	---
	NanCOVID	397	---
X-ray	COVID	187	374
	NanCOVID	73	146

The following evaluation measures are used to assess the performance of different feature extraction methods: the classification accuracy of each class (COVID accuracy and Nan COVID accuracy), average accuracy of two classes, overall accuracy and F-measure. In addition, the running times are reported to compare the computational complexity of methods.

Parameter settings

The experiments are done using a laptop Inter Core i7 CPU, with 2.8 GHz processor and 16GB RAM in the MATLAB 2018b software. To construct the MP, opening filters and closing filters are applied to each CT scan (or X-ray image) to form a MP with size of from each gray level medical image where and are the height and width of image and . Because the available images are RGB, a MP is constructed for each channel R, G and B. Then, the MPs of all channels are stacked such that a MP with size of is constructed for a RGB image. The morphological filters with structural element (SE) of ‘disk’ shape are used in the experiments.

The Gabor filter banks for both datasets consist of 18 filters containing scales and directions with window length of 30. In other words, the output of the above filter bank on each gray level image is a Gabor feature cube with size where is the number of Gabor features. For a RGB image, the Gabor feature cube is constructed from each RGB channel and then, the feature cubes are stacked to form the final Gabor feature cube for the RGB image.

To construct EMAP, two following attributes are used for applying attribute filters on each image: ‘a’ (the regions’ area) and ‘d’ (diagonal of box bounding the region). The following values are considered as thresholds of attribute and thresholds of attribute for CT scan dataset: and the following thresholds are used as for X-ray dataset:

All threshold are empirically chosen. For each gray level image, attribute filters with attribute are applied where is the number of the considered thresholds for attribute . In addition, attribute filters with attribute are applied where is the number of considered thresholds for attribute . In other words, the extended multi-attribute profile (EMAP) for each gray level image is a cube. Similar to MP and Gabor feature cubes for RGB image, the EMAP feature cube is obtained for each RGB channel. Then, the attribute profiles are stacked together to form the final EMAP feature cube for each RGB medical image. To implement LBP as a competitor, 16 neighbors inside a circle pattern with radius size of 5 are used.

Due to limited number of available medical images, almost all deep neural networks such as CNN, used in medical applications, have utilized the transfer learning. But, in this work, CNN is implemented without transfer learning to show its performance without dependency to other types of datasets and pre-trained networks. The Adam optimizer with initial learning rate of 0.001, mini-batch size of 128 and 100 epochs is used to train the CNN model. Other parameters are considered as the default values defined in MATLAB 2018b. For example, the factor of weight decay (L2 regularization) is set as 0.0001.The training and validation data are also shuffled once before training.

Classification results

The experimental results for CT and X-ray datasets are reported in Table 3, Table 4 , respectively. The first and second ranks for each evaluation measure is bolded with red and blue colors, respectively. The proposed CFRCF method in different cases are assessed. MP, Gabor and EMAP are used for initial contextual feature extraction and SVM and RF are used for classification. These six different cases in CFRCF are compared with the conventional approaches where SVM and RF are used for classification of MP, Gabor, EMAP and LBP features. To have a fair comparison, the results of the CNN alone with the same settings are also reported.

Table 3

Comparison results for CT image dataset.

Table 4

Comparison results for X-ray image dataset.

Generally, the use of a deep CNN can obtain superior classification results because CNN has high potential in extraction of deep features through applying the convolutional filters in consecutive layers. But, in cases that a low number of training samples is available, and so, the use of a shallow CNN is applicable, the shallow CNN model may not achieve high accurate classification results because it lacks rich feature extraction due to removing the deep layers. So, the use of appropriate feature extraction methods before feeding data to the CNN model can be very useful.

Generally, in both datasets, CFRCF methods have significantly better performance than the conventional MP, Gabor and EMAP methods. This conclusion is expected because CFRCF uses a trained CNN for sub feature extraction and dimensionality reduction of the contextual feature cubes. The constructed contextual feature cubes obtained by MP, Gabor filters and EMAP not only have high dimensionality, which directly feeding them to the classifier is not so appropriate, but also they contain high redundant information. In addition, there is hidden and nonlinear relationship among the contextual feature maps. By applying two consecutive convolutional filters continued by ReLU activation functions, the nonlinear sub features are extracted. Then, by applying a FC, a reduced feature vector is achieved. The achieved feature vector is finally given to a SVM or RF classifier which results in superior classification results.

The following conclusions can be found from the achieved results for CT scan dataset (see Table 3):

The highest average accuracy is achieved by Gabor-RF (CFRCF).

CNN ranks first from the overall classification accuracy point of view. But it cannot diagnosis the COVID cases with a high accuracy. In addition, the F-measure obtained by the CNN is not so satisfactory.

The highest COVID accuracy is obtained by MP-RF (conventional). MP-RF (CFRCF) ranks second.

Although the highest NanCOVID accuracy is obtained by LBP-SVM. But, it is not a valid result. Because LBP-SVM obtains 0% COVID accuracy. In other words, all images are labeled as NanCOVID by LBP-SVM. MP-SVM (conventional) ranks second.

MP-RF (conventional) and MP-RF (CFRCF) rank first and second, respectively in terms of F-measure.

Although Gabor-RF (CFRCF) ranks first from the average accuracy, but its COVID accuracy is 68.81% which has relatively significant difference with respect to MP-RF (CFRCF) that is 75.23%. Although MP-RF (CFRCF) ranks second in terms of several measures, but it has a bit difference compared to the first rank in each measure.

In comparison between MP-RF (CFRCF) and MP-RF (conventional) from the running time (in the testing phase) point of view, CFRCF is run in 0.43 s while the conventional method is run in 3.04 s. In conventional methods, volume of the input data fed into the classifier is relatively high. So, they are run slower than the CFRCF methods which do feature reduction through a trained CNN.

The following results are concluded from the experiments on X-ray dataset (see Table 4):

The preferred choice in X-ray dataset is Gabor-SVM (CFRCF) because it ranks first from the overall accuracy and F-measure, ranks second in terms of average accuracy, NanCOVID accuracy, and ranks third in terms of COVID accuracy with a bit difference with respect to the second rank.

About COVID accuracy, although LBP-SVM shows 100% COVID accuracy, but, it results in 0% NanCOVID accuracy, and so, it fails to work. In addition, although LBP-RF and MP-SVM (conventional) achieve 100% COVID accuracy, but they obtain a little NanCOVID accuracy that implies they falsely labels most of images as COVID.

CNN ranks second from the COVID classification accuracy but if cannot provide a comparable average accuracy or overall accuracy with respect to the first ranks methods.

After Gabor-SVM (CFRCF), Gabor-SVM (conventional) can be an appropriate choice. From the running time in testing phase, Gabor-SVM (CFRCF) is much faster than the Gabor-SVM (conventional), i.e., 0.02 s versus 0.33 s.

Generally, MP-RF (CFRCF) in CT scan dataset and Gabor-SVM (CFRCF) in X-ray dataset are the best candidates. It implies that morphological, shape and structural characteristics (extracted by morphological filters) in CT scan dataset and textural features (extracted by Gabor filters) in X-ray dataset are the most important features in discrimination between COVID and NanCOVID cases. Also, the proposed CFRCF methods are run faster than their associated competitors because they reduce the data dimensionality through crossing data from two convolutional layers and then a fully connected layer.

The proposed CFRCF method is also superior with respect to some state-of-the-art methods. For example, Concatenation of Xception and ResNet50V2 [49], which diagnoses COVID-19 using X-ray images, achieves 80.53% COVID accuracy and 91.4% overall accuracy while the proposed CFRCF method (Gabor-SVM) results in 97.50% COVID accuracy and 94.23 overall accuracy using X-ray images. The proposed CFRCF method is compared with some state-of-the-art COVID-19 detection methods in Table 5 . Generally, more efficient neural networks, usually deeper and more complicated ones, lead to extraction of more abstract features with more details, and so, improve the classification accuracy. According to the represented results in Table 5, networks such as ResNet-50 and VGG19 outperform networks such as Inception and Xception. A simple CNN with two layers is used for feature extraction in the proposed CFRCF method. Despite the simplicity, CFRCF is efficient and relatively accurate.

Table 5

Comparison between the proposed model with some recent works for binary classification of COVID-19 (COVID-19 vs. Nan−COVID-19) using X-ray images.

Reference	Method	Number of X-ray images (C: COVID, N:Nan-COVID)	Accuracy
[38] (2020)	nCOVnet	C: 42 and N: 42	88.10
[50] (2020)	COVIDX-Net	C: 25 and N: 25	InceptionV3: 50 MobileNetV2: 60 ResNetV2: 70 InceptionResNetV2: 80
			Xception: 80 DenseNet201: 90 VGG19: 90
[51] (2020)	DWT + CNN	C: 237 and N: 1206	Xception: 0.9046 Inception: 0.9084 DenseNet201: 0.9241 MobileNet v2: 0.9413
			VGG19: 0.9627 ResNet50: 0.9945
[52] (2021)	CMT-CNN	C: 231 and N: 4007 + 1583	VGG-19: 93.42 ResNet-50: 95.66 EfficientNet: 97.23
Proposed	CFRCF	C: 187 and N: 73 (Augmented: C: 374 and N: 146)	94.23

It is expected that the use of more efficient and deeper network can increase the classification accuracy of the proposed model. However, attention to this point is notable that applying deeper and more complicated network containing more learnable parameters requires more number of training samples to learn. The use of semi-supervised approaches and transfer learning strategy beside the data augmentation can be used to handle the small sample size problem. But, all these strategies increase the total computational burden.

The CFRCF method has two main advantages: 1- it reduces the features dimensionality and 2- it produces features with high energy and low correlation. These advantages lead to improvement of the classification accuracy. The original features extracted by MP, Gabor and EMAP are compared with the features extracted by the MP (CFRCF), Gabor (CFRCF) and EMAP (CFRCF). The results are shown in Table 6 . The features are compared in terms of dimensionality, mean of variance and mean of correlation. As found from this table, the high number of original features extracted by MP, Gabor and EMAP are reduced to 100 using CFRCF. The CFRCF features have much more variance than the original features which means that the produced features significantly contain higher energy. In addition, the mean of correlation among the extracted features is significantly less than the mean of correlation among the original features. It means that CFRCF removes redundant features and produces features with much less overlapping in the feature space. Fig. 9, Fig. 10 show the correlation among the features in CT and X-ray datasets, respectively. In each figure, the first three subplots are related to the original MP features (100 features, 1000 features and 10,000 features) and the fourth subplot is related to the 100 CFRCF features.

Table 6

Comparison among original features and CFRCF features.

Dataset	features	Number of features	Mean of variance	Mean of correlation
CT	MP	150×150×15 = 337,500	0.10	0.51
	MP (CFRCF)	100.00	29.75	0.01
	Gabor	150×150×54 = 1,215,000	3.03	0.58
	Gabor (CFRCF)	100.00	51.58	0.04
	EMAP	150×150×66 = 1,485,000	0.14	0.79
	EMAP (CFRCF)	100.00	27.06	0.01
X-ray	MP	150×150×15 = 337,500	0.04	0.46
	MP (CFRCF)	100.00	49.59	0.00
	Gabor	150×150×54 = 1,215,000	0.31	0.39
	Gabor (CFRCF)	100.00	48.02	0.01
	EMAP	150×150×66 = 1,485,000	0.08	0.15
	EMAP (CFRCF)	100.00	28.90	0.00

Fig. 9

Correlation among the original MP features (100, 1000 and 10,000 features) and the correlation among 100 MP features extracted by CFRCF (the right bottom subplot) in CT dataset.

Fig. 10

Correlation among the original MP features (100, 1000 and 10,000 features) and the correlation among 100 MP features extracted by CFRCF (the right bottom subplot) in X-ray dataset.

Conclusion

The contextual features reduced by convolutional filters (CFRCF) is proposed for automatic diagnosis of COVID-19 infection using the chest CT scan or X-ray images. The proposed CFRCF method is based on contextual feature extraction from the medical images. Shape and geometrical characteristics are extracted by morphological filters through MP construction and also by flexible attribute filters through EMAP construction. The textural characteristics containing various scales and directions are extracted by applying the Gabor filter banks. Due to high dimensionality of the constructed feature cubes and also existence of redundant information in them, a trained CNN is used for nonlinear sub feature extraction and dimensionality reduction. To this end, after applying two consecutive convolutional filter, a fully connected layer is used to result a reduced feature vector. The achieved feature vector is given to the SVM or RF classifier to find the class label of image (COVID or NanCOVID). According to the experimental results, the shape and structural characteristics (extracted by morphological filters) and textural features (extracted by Gabor filters) processed by convolutional operators provide superior recognition accuracy. In addition, although the running time of them in the training phase is relatively high, but they are implemented very fast in the testing phase especially when SVM is used as classifier in the final stage. As an idea for doing future works, the performance of the proposed CFRCF method can be improved by increasing the depth of CNN. However, due to increasing the number of hyper parameters in deep networks and to deal with overfitting problem, the use of transfer learning and improved data augmentation approaches is unavoidable.

CRediT authorship contribution statement

Maryam Imani: Conceptualization, Methodology, Software, Validation, Formal analysis, Investigation, Writing - review & editing.

Declaration of Competing Interest

The authors declare no conflict of interest.