Deep learning and evolutionary intelligence with fusion-based feature extraction for detection of COVID-19 from chest X-ray images.

In recent times, COVID-19 infection gets increased exponentially with the existence of a restricted number of rapid testing kits. Several studies have reported the COVID-19 diagnosis model from chest X-ray images. But the diagnosis of COVID-19 patients from chest X-ray images is a tedious process as the bilateral modifications are considered an ill-posed problem. This paper presents a new metaheuristic-based fusion model for COVID-19 diagnosis using chest X-ray images. The proposed model comprises different preprocessing, feature extraction, and classification processes. Initially, the Weiner filtering (WF) technique is used for the preprocessing of images. Then, the fusion-based feature extraction process takes place by the incorporation of gray level co-occurrence matrix (GLCM), gray level run length matrix (GLRM), and local binary patterns (LBP). Afterward, the salp swarm algorithm (SSA) selected the optimal feature subset. Finally, an artificial neural network (ANN) is applied as a classification process to classify infected and healthy patients. The proposed model's performance has been assessed using the Chest X-ray image dataset, and the results are examined under diverse aspects. The obtained results confirmed the presented model's superior performance over the state of art methods.

Introduction

Nowadays, COVID-19 disease has been a fatal and dangerous one globally [1]. The novel corona-virus initially affects the throat and gradually moves in humans' respiratory tract. It completely spreads over the lungs, leading to high fever, throat pain, and many other consequences named ‘Pneumonia’. The architecture of COVID-19 is depicted in Fig. 1 [2]. If the novel coronavirus enters into the body, lungs are occupied with fluid, and get infected, which becomes patches named ‘Ground-Glass Opacity (GGO)’. It is highly tough to predict the diseases due to the limited healthcare facilities and clinics [3-10]. It is essential to discover an effective remedy for the disease-analyzing process.

Fig. 1

Structure of Coronavirus (COVID-19)

Even under a complicated situation, massive efforts have been taken to find an efficient approach to predict the existence of COVID-19. The better way to control the disease is social distancing, quarantine, and using sanitizers for hands and surrounding places, which many countries have followed. Unfortunately, the lockdown affects a country's GDP, and it also affected the people psychologically and physically. As the COVID-19 is an air-based disease, it can easily be spread by nature ailments like group interactions, crowds, and places like malls and theaters are some of the sources in the disease spreading. Even though the measures are insisted regularly, peoples are affected by COVID-19. The highly affected countries such as the USA, Italy, and Spain reached a higher death count than China. A scientist defined that “This virus is as economically contagious as it is medically contagious”. Hence, it is compulsory to develop a healthcare prediction model using artificial intelligence (AI), which is applicable for predicting the cases rapidly and precisely to eliminate a higher mortality rate.

Here, Deep Learning (DL) method is applied because of the massive benefits. It has changed many applications [11], specifically in the clinical sector, image database like retinal image, chest X-ray, and brain MRI, which offers challenging outcomes with better accuracy. Previously, an X-ray is a scanning machine that provides cheaper and rapid outcomes while screening internal organs and bones. Understanding diverse X-ray images are highly carried out physically by professional radiologists. It is more helpful for developing countries where X-ray service is accessible; however, more experts are not available. Also, DL aims to deploy a deep neural network (DNN) termed as ‘nCOVnet’, which examines the lung’s X-ray image and predicts corona's presence. Initially, computer-aided clinical predictive methods were applied to diagnose different kinds of diseases. Using these approaches, medical professionals comprehend and apply clinical findings as well. The identical studies regarding clinical detection for COVID-19 [12, 13] have restricted characteristics.

This paper devises an effective and intelligent healthcare system using a fusion model for COVID-19 diagnosis. The presented technique comprises different preprocessing, feature extraction, and classification processes. Preprocessing of images is done using the WF technique. Subsequently, the fusion-based feature extraction process is carried out by incorporating GLCM, GLRM, and LBP. Afterward, the optimal feature subset is selected by SSA. Lastly, ANN is applied as a classification process to classify the infected and healthy persons. The ANN is preferred due to the following characteristics: it requires less formal statistical training, the ability to implicitly detect complex nonlinear relationships between dependent and independent variables, and the ability to detect all possible interactions between predictor variables. The proposed technique has been used with the Chest Xray image database’s help, and the results are examined under diverse dimensions.

Related works

Machine learning (ML) models are applied to detect coronavirus’s disease level. Various ML algorithms such as logistic regression (LR), k-nearest neighborhood (KNN), two diverse decision tree (DT), random forest (RF) as well as support vector machines (SVM) classification models can be employed for COVID-19 diagnosis. The working function of these classifiers is estimated with accuracy values. The SVM classifier provides optimal accuracy. Besides, [14] have used the ML classification method for detecting COVID-19. Medical information was collected, and clinical findings are assumed as well as the performance of the ML classifier is determined with respect to different measures. In Ref. [14], 5 distinct classifiers have been utilized, such as SVM, RF, neural networks (NN), LR, and gradient boosted trees (GBT). The maximum AUC values can be attained using SVM and RF classification methodologies. In Ref. [15], a clinical predictive method for COVID-19 has been presented using the data accumulated from the hospitals. Several research works used diverse ML models like RF, NN, LR, SVM, GBT and assessed the classification models' function using sensitivity, specificity, and AUC scores. In this metric, an optimal function can be achieved using XGB with the best AUC score.

Researchers have developed a diverse range of methods to analyze medical data to enhance the medical sciences. For this purpose, Several classification approaches were used effectively to obtain related features from image databases. The COVID19 disease affects the human lungs, which can be identified by examining the lung X-ray. A successful CNN approach has been used with a convolutional neural network (CNN) to predict the Pneumonia case from chest X-ray [16-20]. The training process take place on a set of images offering the front view of lung X-ray images [21]. An automatic COVID-19 detection model was proposed by Shorfuzzaman and Hossain et al. [22], which uses Simese network classifiers with a CNN encoder for the prediction with maximum accuracy.

Singh and Gupta [23] utilized a DL model with rectified linear unit (ReLU) activation function for the detection of lung cancer. Esteva et al. [24] concentrated on the categorization of skin lesions. The model undergoes sampling and is divided into two classes: general cancer and malicious cancer. Liu et al. [25] developed a model for classifying the chest's tuberculosis. The given X-ray image database is inappropriate. Thus, it lacks in assuming region-level data at data preprocessing.

Butt et al. [26] employed ResNet-23 and conventional ResNet-18, which is applicable to obtain better accuracy by training the CT Scan images. Therefore, the maximum time is required, which is not accessible in many clinics. Fanelli and Piazza [27] applied a commonly accessible database collected to predict the movements of COVID-19 under various cases. Using mean-field kinetics of pandemic spreading, it detects the existence of COVID-19, and lockdown imposed by the government has changed the lifestyle of many peoples. Apostolopoulos and Mpesiana [28] used artificial intelligence (AI) with diverse transfer learning and reached better outcomes using VGG19 with optimal accuracy. The applied database was collected from Cohen’s GitHub, but it eliminated the data leakage, which happens because of the maximum images about a similar patient.

Li et al. [29] employed AI with 3D DL approach to predict COVID-19 patients on a database with higher CT Scans of many patients. An automatic method was presented in [30] under DL and AI applications, especially LSTM networks [6], to detect the trends and feasible cessation time of COVID-19 in diverse countries. The approach utilized in time-series data for predicting Canada's future virus trends and various infection clusters have existed recently. Though several models are available in the literature, there is still a need to design a fusion-based COVID-19 diagnosis model to achieve an improved detection rate.

The proposed method

The workflow involved in the presented methodology is shown in Fig. 2. As depicted, the figure indicates that the proposed model involves different processes, namely preprocessing, feature extraction, and classification.

Fig. 2

Flowchart of Proposed FM-ANN model

Preprocessing using Weiner filtering method

Noise elimination is defined as an image preprocessing model used to enhance the image's properties interrupted by noise. In particular, wiener filtering is applied in which denoising is performed according to the image’s noise content. It is used due to the following benefits: it exploits signals effectively, controls the output error, and is easy to implement. Consider the corrupted image, which is calculated by, , the noise variance of the complete image has been showcased by, the local mean is provided by, over a pixel window as well as local variance from a window is depicted by, . Followed by, a feasible path in denoising an image is represented as:

Then, if the noise variance over an image is 0,

When global noise variance is minimal, and local variance is maximum than global variance becomes 1; so that, if

The maximum local variance shows the existence of an edge from the image window. In case of the same local and global variances, the function can be expressed as:

The above analogies simplify a mean where output is a mean value of a window over a pixel without any anomalies. It is an inherent functionality of WF. The filter applies window size as input and determines the remaining objectives [31]. The attained simulation outcome projected the work in noise elimination from Chest X-ray images using WF. The key objective is to learn the impact of WF noise elimination on edges.

Fusion of feature extraction

In this section, the fusion of feature extraction takes place using the hybridization of GLCM, GLRM and LBP features. Besides, the optimal set of features is then extracted using SSA.

GLCM features

The GLCM is defined as a statistics-based model that assumes the spatial connection over pixel [32]. It is a histogram that determines the significance of pixel divided by a specific distance that exists in the image. Suppose be the image of size , with gray levels, and be 2 pixels with gray level intensity and , correspondingly. While taking in direction and in direction, the connecting line has direction that is same as . The generalized ‐occurrence matrix is depicted as:

In this model, is defined as a provided condition, like . shows the value of components in ‐occurrence matrix and implies the overall count of pixels. Generally, and have been applied for estimations. The 5 diverse texture features are described under the application of ‐occurrence matrix as provided:where implies mean value of ‐occurrence matrix

GLRM features

The GLRLM provides the dimension of homogeneous runs for all gray levels. It is determined for 4 various directions, and each one has 11 texture indices obtained from the matrices. The component of GLRLM links homogeneous runs of pixels with intensity from an image named as GLRLM. Assuming the GLRM matrix with 11 features as mentioned above is attained. For all images, 44 features features directions have been accomplished [33].

Where implies a GLRM matrix, shows a gray-level measure, depicts run length, and denotes the total of every value in GLRM matrix.

LBP features

The LBP is defined as a statistical model applied in image processing that accomplishes efficient features from images [34]. It is highly used in the computer vision domain. This operator develops a binary value for all pixels under the comparison of the center pixel as well as surrounding pixels around . LBP measures are accomplished by binarizing the variations among the neighbors of pixels in all images using a function (Eq. (21)).

In Eq. 21, denotes the radius and species of distance from neighboring pixels to middle pixel, while refers to the count of nearby pixel values. Uniform patterns are defined as binary LBP code of 0‐1 or 1‐0, and the count of transition is 2. Uniform patterns represent simple textures like spot, edge, and corner. It is also composed of uniform patterns.

Optimal feature selection using SSA

SSA

SSA is defined as a random population‐relied method recommended by [35]. It is used for accelerating the swarming process of salps while foraging in the ocean. In the deeper ocean, the salps model, a swarm, named as salp chain. In the SSA approach, the dominant is a salp in front of the chain, and balance salps are termed, followers. The salps location can be saved in a 2D matrix named as . Moreover, the food source is referred to as in search space as the swarm destination. The numerical approach for SSA is provided in the following: the dominant salp will modify the location under the application of given function:

The coefficient is a necessary attribute in SSA as it offers better management among exploration and exploitation phases. To modify the position of followers, given functions are applied:where where . Due to the time in optimization, the crisis among iterations are 1, and assuming , which has been determined by:

A brief stepwise definition of this approach is given in the following:

Upload the parameters of SSA like count of salps , count of iteration (A), best salp position and optimal fitness value .

Upload a population of salp's position arbitrarily.

Estimate the fitness of all salps.

Fix count of iteration to

Upgrade r1.

For all salps,

If , upgrade the place of leading salp by Eq. (22).

Else, upgrade the place of follower salp by Eq. (25).

Determine the fitness of all salps.

Upgrade when the attained solution is better than the existing solution.

Increase a to 1.

Follow Steps 5–7 till is met.

Provide the optimal solution and fitness value .

Solution representation

In feature selection (FS) issues, each solution is restricted from values. For the SSA approach, which is applied with FS problem, a binary version must be deployed. Here, a solution is described as a 1D vector, where the length of a vector depends upon the features in the actual dataset [36]. The cell in a vector contains the measure of “1” or “0”. Value “1” represents the corresponding feature as selected; else, the measure is depicted as “0”. Equation (26) is applied for mapping continuous measures into binary ones.where represents the discrete form of solution vector X and Implies continuous locations of search agent at dimension

Fitness function

The FS is labeled as a multi‐objective optimization issue in which two conflicting objectives should be obtained; minimum selected features and higher classification accuracy. The best solution can be identified where a solution with a lower count of selected features as well as maximum classification accuracy. For effective management of chosen features as well as classification accuracy, the fitness function in Eq. (27) has been applied in ISSA, and these models apply to the search agents.where Err(D) demonstrates classifier error rate, and are constants for controlling classification accuracy and feature limitation, refers to the size of discovered feature subset, signifies overall count of features. in [0, 1] and , thus, Fig. 3 shows the flowchart of SSA for FS.

Fig. 3

SSA for feature selection

Classification using ANN

The ANN structure of multilayer perceptron (MLP) is composed of specification of several layers, the type of activation function for all units, and weights of connections among various units, and has to be implemented for developing neural structure [37, 38]. It offers a supervised feedback ANN structure in three layers: the input layer shows limited rule base measures as shown in Fig. 4.

Fig. 4

Structure of ANN

The structure of ANN involves input, hidden, and output layers. The first layer indicates the attributes of the data. The second hidden layer defines the attributes that are linearly unseparable and output layer offers the required outcome. The threshold node is moreover appended in the input layer specifying the weight function. The resulting function can be represented as follows:where is the linear amalgamation of input x1, x2,…, x, and the threshold θ, is the connection weights among input x and the neuron j, f is the jth neuron’s activation function, and m is the output. A sigmoid function is an ordinary choice of the activation function as given below.

For training MLP, the backpropagation learning concept has been employed, which is a gradient descent model for weight adaptation. Every weight vector (w) is initialized to an arbitrary value. The pattern is computed layer by layer till the output layer offers a response. ANN contains the capability to resolve the issues in tedious systems like image classification, and it helps to classify different classes of COVID-19.

Performance validation

In this section, the presented technique has been validated using the Chest X-ray dataset [39], which encompasses 27 images under normal class, 220 images under SARS class, and 17 images under Streptococcus. For experimentation, a tenfold cross-validation process is employed. The proposed model is implemented using Intel i5, 8th generation PC with 16 GB RAM, MSI L370 Apro, Nividia 1050 Ti4 GB. For experimentation, Python 3.6.5 is used along with pandas, sklearn, Keras, Matplotlib, TensorFlow, opencv, Pillow, seaborn, and pycm. The parameters involved are batch size: 128, learning rate: 0.001, epoch count: 500, and momentum: 0.2. Figures 5 and 6 illustrate some test images from binary and multiple classes of COVID-19 chest X-ray images. The sample screenshots attained at the time of the execution is given in Appendix.

Fig. 5

Sample binary class images

Fig. 6

Sample multi-class images

Table 1 and Fig. 7 illustrate the FM-ANN model’s results for the classification of binary class in terms of different aspects. Under the CV-1, the proposed FM-ANN model has resulted in a higher sensitivity of 94.38%, specificity of 95.30%, accuracy of 94.29%, and an F score of 95.09%, respectively. Likewise, under the CV-2, the presented FM-ANN approach has provided maximum sensitivity of 95.17%, specificity of 95.43%, accuracy of 95.64%, and an F score of 94.85% correspondingly. Followed by, under the CV-3, the projected FM-ANN method has provided maximum sensitivity of 94.87%, specificity of 96.76%, accuracy of 96.40%, and an F score of 95.32%, respectively. Moreover, under the CV-4, the developed FM-ANN technology has provided high sensitivity of 94.32%, specificity of 95.89%, accuracy of 95.88%, and an F score of 96.83% correspondingly. Moreover, under the CV-5, the proposed FM-ANN approach has obtained maximum sensitivity of 96.74%, specificity of 95.30%, accuracy of 94.29%, and an F score of 95.09%, respectively.

Table 1

Results analysis of FM-ANN technique on binary class

Cross-validation	Sensitivity	Specificity	Accuracy	F score
fold 1	94.38	95.30	94.29	95.09
Fold 2	95.17	95.43	95.64	94.85
Fold 3	94.87	96.76	96.40	95.32
Fold 4	94.32	95.89	95.88	96.83
Fold 5	96.74	96.48	97.36	94.35
Average	95.10	95.97	95.91	95.29

Fig. 7

Binary class analysis of FM-ANN model

Similarly, the presented FM-ANN approach’s CV average analysis has provided a sensitivity of 95.10%, specificity of 95.97%, accuracy of 95.91%, and an F score of 95.29% correspondingly.

Table 2 and Fig. 8 demonstrate the FM-ANN approach’s results for Multi-class classification using diverse aspects. Under the CV-1, the applied FM-ANN technology has provided maximum sensitivity of 94.72%, specificity of 95.11%, the accuracy of 95.32%, and an F score of 94.20% correspondingly. Likewise, under the CV-2, the developed FM-ANN technology has shown maximum sensitivity of 95.91%, specificity of 95.73%, accuracy of 94.34%, and an F score of 95.63%, respectively.

Table 2

Results analysis of FM-ANN technique on multi-class

Cross-validation	Sensitivity	Specificity	Accuracy	F score
Fold 1	94.72	95.11	95.32	94.20
Fold 2	95.91	95.73	94.34	95.63
Fold 3	96.73	96.46	96.70	94.09
Fold 4	94.98	95.85	96.74	95.87
Fold 5	95.90	96.54	95.38	96.97
Average	95.65	95.94	95.70	95.35

Fig. 8

Multi-class analysis of FM-ANN model

Followed by, under the CV-3, the implied FM-ANN technique has offered greater sensitivity of 96.73%, specificity of 96.46%, accuracy of 96.70%, and F score of 94.09%, respectively. Moreover, under the CV-4, the presented FM-ANN model has depicted high sensitivity of 94.98%, specificity of 95.85%, accuracy of 96.74%, and F score of 95.87%, respectively. Furthermore, under the CV-5, the projected FM-ANN approach has offered maximum sensitivity of 95.90%, specificity of 96.54%, accuracy of 95.38%, and F score of 96.97%, respectively. In line with this, the CV average analysis of deployed FM-ANN technology has generated a sensitivity of 95.65%, specificity of 95.94%, accuracy of 95.70%, and an F score of 95.35% correspondingly.

Figure 9 examines the classification performance of the FM-ANN model with existing methods in terms of sensitivity. It is noted that the DT and CNN models have led to the lowest sensitivity values of 87% and 87.73%, respectively. Besides, it is noticed that the ANFIS and KNN models have resulted in slightly higher and closer sensitivity values of 88.48% and 89%. Similarly, the DTL and CoroNet models have appeared to even better classifiers with a sensitivity of 89.61% and 90%, respectively.

Fig. 9

Comparative analysis of the proposed model in terms of sensitivity and specificity

On continuing with, the XGBoost, RNN and CNNLSTM models have reached to certainly higher 92%, 92.04%, and 92.14%, respectively. Along with that, the MLP and LR models have obtained an identical sensitivity value of 93%. The LSTM and ANN models have reached moderate sensitivity values of 93.42% and 93.78%. Next, the CNNRNN model has shown a somewhat acceptable sensitivity of 94.23%. At last, the proposed FM-ANN model has achieved outstanding results by offering maximum sensitivity of 95.1% and 95.65% on binary and multiple classes, respectively. The figure investigates the classification function of the FM-ANN method with respect to specificity. It is clear that the CNN and MLP methodologies resulted in lower specificity values of 86.97% and 87.23% correspondingly. On the other hand, the ANFIS and DT approaches have provided better and nearby specificity values of 87.74% and 88.93%. Along with that, the LR and XGBoost frameworks have showcased considerable classifiers with the specificity of 90.34% and 90.44% correspondingly. On continuing with, the KNN, RNN and ANN schemes have accomplished better 90.65%, 90.87% and 91.76%, respectively. Likewise, the CNNLSTM and DTL approaches have attained best results with specificity values of 91.98% and 92.03%, respectively. Then, the CoroNet model has showcased considerable specificity of 92.14%. Next, the LSTM and CNNRNN frameworks have accomplished gradual specificity values of 92.64% and 92.67%. Eventually, the projected FM-ANN method has reached optimal results by providing higher specificity of 95.97% and 95.94% on binary and several classes correspondingly.

Figure 10 investigates the FM-ANN method's classification function with respect to accuracy. It is pointed out that the CNNLSTM and RNN approaches have resulted in the minimum and identical accuracy value of 84.16%. Followed by, it is clear that the CNNRNN and ANN methodologies have provided moderate and closer accuracy values of 85.66% and 86%. Likewise, the LSTM and DT frameworks have depicted even better classifiers with 86.66% and 86.71% accuracy. In line with this, the CNN, ANFIS, and KNN approaches have attained gradual 87.36%, 88.11% and 88.91%, respectively. On continuing with, the CoroNet and DTL methodologies have accomplished considerable accuracy values of 90.21% and 90.75%, respectively. Besides, the XGBoost and LR technologies have attained reasonable accuracy values of 91.57% and 92.12%. Then, the MLP technology has illustrated better accuracy of 93.13%. Consequently, the presented FM-ANN approach has attained maximum results by providing a higher accuracy of 95.91% and 95.70% on binary and multiple classes.

Fig. 10

Comparative analysis of the proposed model in terms of accuracy and F score

The figure investigates the classification function of the FM-ANN technique using F score. DT and KNN schemes have resulted in lower F score values of 87% and 89%. On the other side, it is apparent that the ANFIS and CNN methodologies have accomplished better and closer F score values of 89.04% and 89.65%. Likewise, the CNNLSTM and ANN frameworks have led to even better classifiers with the F score of 90.01% and 90.43%, respectively. In line with this, the RNN and CoroNet approaches have reached moderate results with F score values of 90.61% and 91%, respectively. Similarly, the CNNRNN, ANN, and LSTM frameworks have attained considerable 91.20%, 91.34%, and 91.89%, respectively. Besides, the XGBoost and LR methodologies have accomplished a better and closer F score value of 92%. Followed by, the MLP method has showcased a better F score of 93%. Finally, the proposed FM-ANN technique has accomplished optimal results by providing a high F score of 95.29% and 95.35% on binary and multiple classes correspondingly.

From the detailed experimental validation, it is ensured that the FM-ANN model has resulted in effective diagnostic performance over the compared methods by providing a maximum accuracy of 95.1% and 95.65% on binary and multiple classes. The improved diagnostic performance is due to the fusion of three feature extraction models and the application of the SSA for the parameter tuning of the ANN.

Conclusion

This paper has developed an efficient fusion model for intelligent COVID-19 diagnosis using chest X-ray images. Initially, the preprocessing of images takes place using WF technique. Subsequently, the fusion-based feature extraction process is carried out by incorporating GLCM, GLRM, and LBP. Afterward, the optimal feature subset is selected by SSA. Finally, ANN is applied as a classification process to classify the infected and healthy patients. The proposed model's performance has been assessed using the Chest X-ray image dataset. The experimental values verified that the proposed FM-ANN model had achieved outstanding results by offering a maximum accuracy of 95.1% and 95.65% on binary and multiple classes. As a part of the future scope, the performance can be further enhanced using segmentation techniques.