An Automated Lightweight Deep Neural Network for Diagnosis of COVID-19 from Chest X-ray Images.

Coronavirus (COVID-19) is an epidemic that is rapidly spreading and causing a severe healthcare crisis resulting in more than 40 million confirmed cases across the globe. There are many intensive studies on AI-based technique, which is time consuming and troublesome by considering heavyweight models in terms of more training parameters and memory cost, which leads to higher time complexity. To improve diagnosis, this paper is aimed to design and establish a unique lightweight deep learning-based approach to perform multi-class classification (normal, COVID-19, and pneumonia) and binary class classification (normal and COVID-19) on X-ray radiographs of chest. This proposed CNN scheme includes the combination of three CBR blocks (convolutional batch normalization ReLu) with learnable parameters and one global average pooling (GP) layer and fully connected layer. The overall accuracy of the proposed model achieved 98.33% and finally compared with the pre-trained transfer learning model (DenseNet-121, ResNet-101, VGG-19, and XceptionNet) and recent state-of-the-art model. For validation of the proposed model, several parameters are considered such as learning rate, batch size, number of epochs, and different optimizers. Apart from this, several other performance measures like tenfold cross-validation, confusion matrix, evaluation metrics, sarea under the receiver operating characteristics, kappa score and Mathew's correlation, and Grad-CAM heat map have been used to assess the efficacy of the proposed model. The outcome of this proposed model is more robust, and it may be useful for radiologists for faster diagnostics of COVID-19. © King Fahd University of Petroleum & Minerals 2021.

Introduction

The current occurrence of the novel coronavirus named as SARS-CoV-2 (coronavirus 19) was epi-centered in the Wuhan city, Hubei Province of South China. The committee of the UN agency declared an emergency as a world health crisis on January 30, 2020, in support of growing case notification rates in China and international regions. The rate of registered cases is growing every day and tracked real-time on the Johns Hopkins University Web site [1] and other alternative forums. China bore the big burden of mortality and morbidity up to the middle of the 2020 Gregorian calendar, whereas the prevalence of pandemic in different Asian countries, North America, and Europe remained low formally. Coronaviruses are engulfed, positive fiber large ribonucleic acid (RNA) viruses that infect human beings, however, additionally a good range of animals. Coronaviruses were initial delineate in 1966 by Tyrell and Bynoe, when the viruses were cultivated by the UN agency from inmates suffering with symptoms of common cold [2]. Coronaviruses got their name basically due to the resemblance of a solar corona depending on their morphology as spherical virions with a core–shell. There exist four subfamilies of coronaviruses namely alpha (α), beta (β), gamma (γ), and delta (δ). The α- and β-coronaviruses have emerged from mammals, exclusively from bats, whereas the coronaviruses γ and δ have emerged from birds as well as pigs. The size of the order differs between 26 to 32 KB. Beta-coronavirus amid all seven subtypes of coronaviruses may infect humans and might cause serious infections and fatalities, while α-coronaviruses generate either symptomatic or asymptomatic infections [3]. Coronavirus succeeded in creating its evolution from animals to humans from the Huanan wholesale seafood merchandise in China. The early scientific sign of the COVID-19 used in recognizing case discovery was the signs of pneumonia. The observations made until now recommend a mean and median incubation duration of 5 and 3 days within the range of 0–24 days, respectively. The quantity of those infected people by COVID-19 who remains asymptomatic during the virus has not been exactly determined until now, whereas the clinical indications of the infection in symptomatic patients generally begins after less than a week that includes fever, fatigue, cough, and other signs of upper respiratory tract diseases. The disease can advance to brutal disease with shortness of breath as well as severe chest signs consequent to viral pneumonia in around 75% of inmates. Pneumonia mostly transpires in the second or third week of an indicative infection. Major symptoms of viral pneumonia contain reduced oxygen saturation, blood gas variations, alterations perceptible through chest X-rays, and other imaging methods, with ground glass irregularities, interlobular connection, alveolar exudates, and patchy consolidation ultimately representing deterioration. So, humans got a major impact due to COVID-19. The COVID-19 pandemic has directed a striking fall of human life globally and poses a new challenge to public health, food products, and also to labor all over the world. The socioeconomic disturbance caused by the current pandemic is distressing. Many people are in danger of dropping into severe poverty and presently 69 million approximately might increase up to 132 million by the year ending [4]. Corporate sectors and staff are notably vulnerable as a result of the bulk lack of social safety and quality healthcare facilities and have misled the approach to productive assets. While not the means that to earn income throughout lockdowns, several are incapable to feed their families and themselves. Border shutting, trade limitations, and confinement methods are stopping farmers from accessing markets, including for getting inputs and commerce their manufacture, and agricultural staff from gathering crops, therefore troublemaking domestic and international food provide chains and decreasing access to healthy, secure, and numerous diets. The epidemic has destroyed jobs and placed uncountable livelihoods in danger.

Pandemics and epidemics have been frightening the human race. SARS, H1N1, Ebola, and many more have shown their result in the past, but with every such outburst, novel ways have been discovered to fight and control such unanticipated diseases that can gradually kill millions of individuals. Technology may not stop the arrival of the epidemics, but it can help stop the spread, instruct, notify, and allow those on the ground to be conscious of the state, and perceptibly reduce the impact. Wrong information about the number of sufferers, diagnosis and dealing options, vaccines, drugs, government policies, etc., generates more fright and unease among the population. The consequence can be extensive chaos, fright buying, hoarding of necessary commodities, charge rise, fighting on the streets, prejudice, scheme theories, and many more. To decrease misinformation, a few companies like YouTube, Facebook, and Google are untiringly working to direct people to the correct, demonstrable data such as that available by the World Health Organization (WHO) or local and government authorities. By assembling exact information for every individual, a clear scenario can be built and people can be knowledgeable about the accurate steps to take. When a drug- or vaccine-related query rises to stop the spread of the outbreak, the world is anxious to discover ways to slow down the extent of the virus and to recover an efficient treatment. Artificial intelligence (AI) presents an essential role in advising components of a vaccine by considering viral protein methods and serving health researchers search for tons of related research documents at an exceptional rapidity. Some social platforms have combined with WHO on Pandemic to assist maintaining their user well-informed with accurate, timely realities from the WHO, where individuals will be capable to ask queries and get responses. Likewise, Bigdata Analytics is showing its efficacy in recognizing the infected persons by connecting, tracking, and contacting them. Likewise, many technologies and methods have showcased their efficacy while dealing with COVID-19.

Shortness of breath, cold, cough, fever are the major significant signs in infected persons for the analysis of COVID-19. These signs may confirm carrier features by not being noticed in contaminated persons. For analysis of COVID-19, two prevailing techniques are available. One technique is a viral check-up via a nasopharyngeal swab to examine the existence of fragments of viral RNA. The collected samples are then experienced by utilizing the technique of real-time reverse transcription-polymerase chain reaction (rRT-PCR) [5]. In several conditions, a sample of swab collected from the nasal region may also be utilized. The results of rRT-PCR are normally accessible either in a few hours or within two days. Another technique is analysis of chest radiographs. Among the available medical radiographs, the chest X-ray is one of the imaging strategies,which can present the utmost sensitivity. As pathological tests executed in laboratories are enchanting more time, a quick and exact analysis is essential for an efficient struggle against the pandemic. Due to this reason, various specialists began to utilize radiological imaging techniques. The major benefits of analyzing medical radiographs are that they can assist in accelerating or screening the speed of analysis, particularly to overcome the deficiencies of RT-PCR. These processes are handled by using X-ray or computed tomography (CT) imaging methods. Cases of COVID-19 have related characteristics in X-ray images in the early as well as late stages. It demonstrates a round and innermost dispersion from within the image. So, radiological imaging presents the recognition of doubtful cases with an accuracy of 90%. Similarly, the specialists of radiology identified that the symptoms of COVID-19 as observed with CT imaging had their features that are varied from those features of the CT radiographs representing other viral pneumonia like influenza-A viral pneumonia. Therefore, medical doctors prefer to restore rRT-PCR with lung CT radiographs as one of the early analytic measures for identifying this novel kind of coronavirus, with the intent of instantly inhibiting spread of infection. However, the major blockage that radiologists face in examining radiography images is the visual scanning of small facts. Besides, a huge amount of CT images has to be assessed in a very short time thus raising the possibility of misclassifications. With the quick progress of computer knowledge, the digital image processing technology has been broadly used in the medicinal field, including image improvement, renovation, and organ segmentation by giving support for successive medical diagnoses. Deep learning (DL) methodologies, such as the convolutional neural network (CNN) with its vigorous capability of non-linear modeling, is utilized widely in the processing of medical images. The architecture of CNN is the most well recognized DL advance with greater achievement in the field of medical imaging. CNN has been utilized for both classifications as well as segmentation issues of CT images, respectively. The major success of CNN is its capability to learn characteristics mechanically from specific domain images, unlike traditional machine learning (ML) techniques. The well-liked approach for training CNN is to transmit learned facts from a pre-trained system that completed one task into another new task. This technique is simple and faster to apply with no need for an enormous annotated dataset for training. So, most of the researchers are showing interest to use this strategy, particularly with medical imaging. Consequently, numerous pre-trained techniques have won several international image categorization competitions such as VGGNet, ResNet, NASNet, MobileNet, Inception, and Xception.

Presently, several radiology images have been extensively utilized for the detection of COVD-19. Abraham and Madhu [6] suggested a method that utilizes amalgamation of features retrieved from multi-CNN using feature selection approach based on correlation and BayesNet classifier for the screening of COVID-19 quickly from chest X-ray radiographs. Hu et al. [7] have designed a weakly controlled DL model for quick and completely automated identification and classification of COVID-19 with retrospectively extorted CT images from multi-centers as well as multi-scanners. It was found that the proposed model can differentiate between non-pneumonia (NP) and community-acquired pneumonia (CAP) patients. The findings of the projected model have outperformed better accuracy and precision for the classification. Jaiswal et al. [8] has used pre-trained DL methods as an automatic tool for the recognition and analysis of COVID-19infection. The authors have proposed a DenseNet201-based deep transfer learning (DTL) approach to categorize the inmates as COVID positive or negative. The projected technique has been used to extract characteristics by utilizing its individual learned weights on the ImageNet dataset CNN framework. Horry et al. [9] has illustrated the usage of transfer learning from DL models to perform recognition of COVID-19 with three mostly used medical radiographs such as CT scan, Ultrasound, and X-ray for the chosen VGG19 model, which obtained 86% precision for X-ray, 84% for CT scans, and 100% for ultrasound. Aslan et al. [10] has used X-ray images of the chest to simply identify COVID-19. To evaluate the region of the lung for the identification of COVID-19, ANN-based segmentation was applied to the raw images. The projected model included the architecture of the pre-trained AlexNet and found a better accuracy rate for identifying COVID-19. Nour et al. [11] has proposed an intelligence diagnosis model depending on CNN together with rich filters, convolution, abstraction, and weight sharing features. Then, the deep features obtained from the deep layers of the CNN has been forwarded as input to the machine learning models for improving the diagnosis of COVID-19 infection. Further, Bayesian optimization algorithm has been used for optimizing the hyperparameters of the machine learning models. This automated COVID-19 analytical system developed to contribute to medical trials has obtained an accuracy rate of 98.97%, 99.75% as specificity, 95.75% of F1-score, and a sensitivity of 89.39%. Panwar et al. [12] has performed various experiments on the classification of binary image for the identification of positive as well as negative COVID-19 patients using Grad-CAM-based color visualization approach. It was found considering both CT and CXR scan images and attained an accuracy rate of 95.61% while identifying COVID-19 cases which is faster than the conventional RT-PCR approach. Shalbaf et al. [13] has proposed a novel ensemble advance that synergistically incorporates transfer learning approaches for diagnosis of COVID-19 and presented perceptive results on an openly obtainable medical radiograph dataset. A framework depending on the fusion of graph convolutional network (GCN) and CNN known as FGCNet is developed by Wang et al. [14] for the adequate categorization of the COVID-19 images. In this model, a CNN model with several features such as batch normalization, dropout, rank-based average pooling and multi-way data augmentation was developed to obtain individual image-level representation (IIR). The CNN model cannot extract relation-aware representation (RAR) features. Therefore, GCN was developed to determine the connectivity analysis and to retrieve the RAR features. In addition, it is used to integrate the features of CNN with the features of GCN to enhance the classifier performance. Silva et al. [15] has proposed a deep learning model known as EfficientCovidNet together with voting-based approach and cross-dataset analysis for efficiently detecting the COVID-19 sequences. Voting-based approach is used for categorizing images of a given patient into groups, while the cross-dataset analysis is utilized to validate the robustness of the architectures. A framework depending on three cascade phases of the categorization of COVID-19-infected CT images has been developed by Kenawy et al. [16]. Initially, a CNN model known as AlexNet is used for obtaining features automatically from the trained CT images. In the next phase, valuable features are extracted by applying a feature selection algorithm which makes use of stochastic fractal search (SFS) and guided whale optimization algorithm (Guided WOA) techniques. For balancing the selected features in the second phase, LSH-SMOTE (locality sensitive hashing synthetic minority oversampling technique) has been applied. Finally, in the third phase, the selected valuable features are classified using a voting classifier algorithm. Although the model has been successfully implemented, the fine tuning of the CNN parameters has not been performed which can further enhance the overall classification accuracy of the model. To categorize the COVID-infected images, a novel approach based on multi-objective differential evolution (MODE) and CNN has been devised by Singh et al. [17]. For the classification of the COVID-19-infected cases, a multi-objective fitness function is used. Then, the MODE algorithm is used for tuning the hyperparameters of CNN for improving the overall accuracy of the system. From the recent literature study, it has been seen that the most of the researchers uses chest X-ray images for diagnosis of COVID-19 cases. Recently, it has been found that medical imaging techniques such as X-ray and computed tomography (CT) play a crucial role in testing COVID-19 cases [18-22]. Meanwhile, the coronavirus usually causes infection in the lungs, therefore the both chest X-ray and CT images have been widely considered [23] by radiologists. Finally, there exist various recent studies on the detection of COVID-19 that used several DL models with CT images [24-27].

The following are some of the limitations observed while performing systematic study of the COVID-19 detection using DL approaches. Firstly, most of the studies have not been clinically confirmed by the expert radiologist. Secondly, the developed approaches may not work correctly for other types of medical images. Thirdly, most of the studies have been performed with smaller datasets due to the unavailability of larger and labeled datasets publicly. Lastly, most of the studies have not concentrated on the development of economically feasible ways to combat this disease. These challenges need to be addressed to further develop more accurate and economical feasible models for the identification of the COVID-19 disease using DL approaches. In connection to this, this paper is aimed to design and promote a unique lightweight-based deep learning (DL) automated approach called LW-CBRGPNet to perform multi-class classification model (Normal, COVID-19, and Pneumonia) and binary class classification model (COVID-19, Normal) on chest X-ray images. This proposed CNN scheme includes the combination of three CBR blocks (convolutional, batch normalization, Relu) with learnable parameters of global average pooling (GP) layer, fully connected (FC) layer. Apart from the design part, the proposed model also investigated various hyperparameters in order to validate the proposed scheme like batch size, number of epochs, learning rate, different optimizer technique, misclassification rate, etc. In continuous to validation, several other performance measures like tenfold cross-validation, confusion matrix, evaluation metrics, AUROC, kappa score and Mathew’s correlation, and Grad-CAM heat map has been used to assess the efficiency of the proposed model. Finally, the proposed scheme compared with some recent state-of-the-art techniques along with four popular pre-trained models such as ResNet-101 [28], VGG-19 [29], DenseNet-121 [30], and XceptionNet [31]. The experimental result obtained from the proposed scheme irrespective of different hyperparameters outperforms better when compared with the state-of-the-art CNN models and four pre-trained models. The proposed model is lightweight as it is composed of three convolutional blocks (each block carries one convolution layer followed by batch normalization and ReLU layer), alongside one GAP followed by one FC layer. The major contributions of this research study can be summarized as given below:

A lightweight DL-based methodology has been adopted that achieves a balance between parameters used for training and accuracy, making it different from heavier models that are compared in the literature.

The proposed lightweight model addresses AI-based high-level abstraction from the chest X-ray radiographs which avoids the requirement of handcrafted feature extraction process.

The recent research trends often consider less number of FC layers, as it increases the computation cost of the model because FC layer is able to accommodate large numbers of parameters in context to other layers. Inspired by this literature, it was tried to accommodate only one FC layer in the proposed scheme which helps in knowingly dropping the learnable parameters.

To measure the effectiveness of the proposed scheme, investigation was also done on various hyperparameters such as batch size, number of epochs, learning rate, different optimizer technique, misclassification rate, etc.

The validation of the proposed scheme has been achieved by analyzing various parameters such as tenfold cross-validation, accuracy, Kappa score, AUROC and Matthew’s correlation, Gradient-based class activation map (Grad-CAM).

Finally, a comparative analysis on four pre-trained transfer learning models along with recent state-of-the-art techniques has been studied over the same chest X-ray images.

The rest of this paper is organized as follows: In Sect. 2, the sample dataset has been presented along with details of the proposed architecture for COVID-19 detection. The obtained empirical results and comparative analysis of the results have been presented in Sect. 3. Finally, the paper ends with concluding remarks in Sect. 4.

Material and Methods

This section represents the detailed description of the dataset employed for this experiments study along with the proposed LW-CBRGPNet model for multi-class and binary class classification model and also presents the architectural overview of four pre-trained models considered for this study.

Dataset Used

In this section, a dataset consisting of three classes of chest X-ray images (normal, COVID-19, and pneumonia) have been considered for accomplishing the complete set of experimental analysis. Since COVID-19 is a novel infection, the total number of available infected images is relatively small. Therefore, in this study a total 750 number of chest X-ray samples from each class were collected from Figshare repository [32]. This dataset mainly considers the COVID-19-infected image from various open source sources and has been updated in regular intervals. For the model building process, a total number of 2250 frontal-view chest X-ray radiographs were considered in all three classes (normal: 750 numbers, COVID-19: 750 numbers, and pneumonia: 750 numbers) to maintain the data balance. In the final step of analysis, the tenfold cross-validation was performed to evaluate the proposed scheme. The detailed specification and size of images contained in the dataset is depicted in Table 1. The sample of chest X-ray radiographs from each class is presented in Fig. 1.

Table 1

Dataset description used in this study

Class	Number of images	Size
COVID-19	750	224 × 224
Normal	750
Pneumonia	750
Total	2250

Fig. 1

Samples of frontal-view chest X-ray images from the dataset: a Normal case, b COVID-19 case, and c pneumonia

Preprocessing

The preprocessing stage is one of the most crucial aspect of the proposed model, because the classification accuracy is directly dependent upon the preprocessing stage. The detailed description of preprocessing techniques being used in proposed scheme is described as follows:

Data Augmentation

The CNN model will be performed more effectively only if the model is trained by a large dataset to reduce the overfitting problem. However, the majority of biomedical image dataset suffer data scarcity problems with limited annotation and randomness. Therefore, data augmentation is the most commonly used technique to enhance the efficiency of CNN models [33]. Anyhow, the dataset considered for this study is a balanced dataset, i.e., equal samples chosen for each class as depicted in Sect. 2.1. Therefore, the key concern of this study to implement data augmentation over training dataset to enhance the randomness among chest X-ray images by using four different spatial transformations such as (i) flipping (horizontal direction), (ii) rotation (rotate angle 15 degree clockwise), (iii) zooming (10%), (iv) lighting (20%). In this case, 2250 images were considered and mini-batches operations were performed randomly to enhance both the diversity and randomness among all images during the training phase without increasing the number of images.

Image Normalization

The process of normalization is also an essential stage used in CNN models to preserve numerical stability. This model also ensures faster learning and obtains a stable gradient map in the image space [34]. For this reason, the preprocessing experimentation is performed using min–max normalization process by normalizing the input image pixel in the range between 0 and 1. The images used in the considered datasets are color RGB images and the rescaling was achieved by multiplying 1/255 with the pixel values.

Background of CBR Block

Deep learning-based CNN models in the field of AI have brought a revolution due to their capability of self-learning and higher classification accuracy in multidisciplinary engineering research domain. However, the recent research study in biomedical image processing has been achieved by implementing CNN model either from beginning, or by utilizing transfer learning (pre-trained model) to achieve the best accuracy. Though the standard architecture of pre-trained models is initially designed for ImageNet classification problems with some pertained weight [35], it comprises a large number of parameters and more complex hidden representation [36]. Hence, there is a need to design a lighter weight DL-based CNN model in biomedical image analysis in order to gain higher efficiency in comparison with the standard ImageNet models. Therefore, this paper presents the lightweight CNN model for identification of COVID-19 infection from chest X-ray radiographs. This proposed model is designed by a composition of fewer convolution layers, batch normalization layer, and rectified linear unit (ReLU) activation layer, accompanied by global average pooling layer, and softmax layer. The detailed description of each layer is clearly discussed in subsequent paragraphs.

Convolutional Layer (CONV)

A convolution layer made up of a set of filters that are used to convolve the input image, thereby generating several feature maps with respect to each filter [33]. In this layer, four major hyperparameters are used to estimate output volume dimension such as no of filters (N), spatial size of filter (F), padding (P), and stride (S). Mathematically, if the input volume dimension as , and output volume dimension as , then each terminology will be evaluated as shown in Eq. 1.

Batch Normalization Layer

Batch normalization is a process in which it enhances the stability of the neural network by normalizing the activation of the previous layer throughout the training phase using both mean and variance over mini-batch operation [37]. Batch Normalization reduces the amount of Covariance shift in hidden values, having a regularization affection, thereby preventing overfitting issue. This layer is usually used in between convolutional layer and ReLU layer to enhance the learning rate. In mathematically, if is the value of over a mini-batch with learning parameter and , then batch normalization of mean and variance over mini-batch along with normalized factor, scale and shift of will be evaluated as given in Eqs. (2–5):

ReLU Layer

The output from one hidden layer is fed to another layer by applying an activation function to establish the linear transformation using weights and biases. ReLU is the most common activation function used in DL models as it is computationally efficient than other common activation functions by converting negative input value into zero and keeping positive value the same. The ReLU layer can be defined mathematically as shown in Eq. 6.

Pooling Layer

The pooling layer used in CNN model for downsampling the spatial size of feature map to ensure the reduction of overfitting issues by reducing both the learnable parameters and computational cost as well. This layer is often positioned in between two successive convolution layers and operates individually on each depth slice of the input, and uses MAX operation to perform spatial resizing. This pooling layer categorized into three types: (i) max-pooling, (ii) average pooling, and (iii) sum pooling. Among all categories, most of the recent study uses max-pooling concepts that witness in many state-of-the-art literatures [38]. Therefore, in this current study, max-pooling (2D) has been used for 2D spatial data, and it takes the maximum value over the window described by pooling size for every magnitude on the feature axis. For each dimension, the window is moved by strides. The detailed graphical presentation of max-pooling layer achieved in this study is clearly depicted in Fig. 2.

Fig. 2

Graphical presentation of max-pooling layer used in this study

Global Average Pooling Layer

The concept of global average pooling (GAP) is designed to replace fully connected layers in traditional CNN models. Instead of including fully connected layers on top of the feature maps, it considers the average of each feature map and the resulting vector is given directly to the softmax layer. GAP is more innate to convolution architecture implementing similarity among feature maps and categories. As no parameter is needed to optimize GAP, it prevents overfitting in this layer. Furthermore, GAP is more potent for spatial resizing of input as it sums out the spatial information. The reduction of spatial dimension in GAP is similar to max-pooling layer. However, dimensionality reduction in GAP is of extreme type. For example, consider a tensor with h × w × d dimension reduced to 1 × 1 × d dimension. GAP layer accomplishes this by reducing each h × w feature map to a single number by simply considering the average of all h, w values.

Fully Connected Layers

The features extracted by convolution layers after flattening are fed forward to the fully connected layers (FC layers). FC layers are generally the final layers of any model that are responsible for performing classification on the flattened extracted features. The properties of FC layers are similar to classic ANN (artificial neural network) where weights of the neuron are identified by back propagation. Between the FC layers, dropout layer is also used to manage the overfitting issue that may arise during the training phase. The final FC layer uses a softmax activation function where the number of classes used in classification is specified.

Proposed Network Model

The proposed LW-CBRGPNet model is graphically presented in Fig. 3 which consists of an arrangement of three 2D convolution layers supervened by 2D batch normalization layer and Relu activation function. The combined arrangement of these three layers results in a CBR block. Apart from three CBR blocks, three 2D maxpool layers used in between two consecutive CBR blocks, supervened by Global average pooling layer to handle the overfitting problem. Finally, a standalone softmax layer is introduced at the last stage of architecture for output vector as final class level prediction in terms of both binary class (normal, COVID-19), and multi-class problem (normal, COVID-19, and pneumonia). The main purpose of this lightweight custom CNN implementation is to reduce the computational cost and learning parameter, and also able to increase the learning speed when compared with existing state-of-the-art CNN techniques and pre-trained transfer learning CNN model.

Fig. 3

Illustration of proposed LW-CBRGPNet model

The initial convolutional layer takes a chest X-ray radiograph as input of size 224 × 224 × 3 and convolved it by using 96 kernels of 5 × 5 × 3 size and stride 2 along with both batch normalization and ReLU activation function (i.e., first CBR block), and produce the feature map of size 110 × 110 × 96. After immediate first CBR block, the max-pooling is employed with kernel size 3 × 3 and a stride 3 in order to produce an output volume (lower-dimensional) of size 96 × 36 × 36. The final output volume of the first CBR block followed by max-pooling layer is convolved with 256 kernels of size 3 × 3 × 96 followed by same batch normalization, ReLU, and max-pooling layer (i.e., second CBR block). Similar to the first and second CBR block, the third CBR block is performed with input kernel size of 3 × 3 × 256, and produces a feature map of size 12 × 12 × 256. Then, the output volume of the third CBR block along with the max-pooling layer forwarded to the global average pooling layer. Finally, a softmax layer is employed at the end of the architecture to produce a classification result. The detailed information regarding trainable parameters of each layer in the proposed network model is summarized in Table 2.

Table 2

Illustration of proposed LW-CBRGPNet model architecture

Layer type	Output shape	Parameters	Stride	Padding	Kernel size	Dropout	Filter
Input	(3,224,224)	0	–	–	–	0	–
Conv2d	(96,110,110)	7296	2	0	5 × 5	0	96
BatchNorm2d	(96,110,110)	192	–	–	–	0	–
ReLU	(96,110,110)	0	–	–	–	0	–
MaxPool2d	(96,36,36)	0	3	0	3 × 3	0	–
Conv2d	(256,38,38)	221,440	1	2	3 × 3	0	256
BatchNorm2d	(256,38,38)	512	–	–	–	0	–
ReLU	(256,38,38)	0	–	–	–	0	–
MaxPool2d	(256,12,12)	0	3	0	3 × 3	0	–
Conv2d	(128,12,12)	295,040	1	1	3 × 3	0	128
BatchNorm2d	(128,12,12)	256	–	–	–	0	–
ReLU	(128,12,12)	0	–	–	–	0	–
MaxPool2d	(128,4,4)	0	3	0	3 × 3	0
Global Average Pooling	(128,1,1)	0	–	–	6 × 6	0	–
softmax	3	387	–	–	–	0	3

Pre-trained Transfer Learning for COVID-19 Diagnosis

This section presents the quick overview of four popular pre-trained transfer learning (TL) model considered for this study. Initially, the concept of TL is designed for ImageNet classification problems where it can deal with millions of images with 1000 prediction classes. Since, COVID-19 image dataset is available in limited numbers, in order to implement these four TL models for COVID-19 image analysis, only fine-tuning of the last layer of the network with the same initial pre-trained weight. Subsequently, the evaluation of four TL models such as ResNet-101 [28], VGG-19 [29], DenseNet-121 [30], and XceptionNet [31]. ResNet-101 is a deep residual network that comprises of 101 layer that start with a standalone convolution layer supervened by 33 residual block and one FC layer. VGG-19 is a deeper CNN-based architecture which is made up using 19 layers including 16 convolution layers (i.e., 5 convolution block) and 3 FC layer. DenseNet-121 also consists of 121 layers without any batch normalization layer, in addition five additional layers are used to include the initial 7X7 convolutional layer along with three transitional layers and one FC layer. Here each layer gets extra input from all of its preceding layers and transfers on its own feature map to all the subsequent layers. XceptionNet is a depth wise separable convolution layer consisting of 72 layers that start with two convolution layers followed by depth wise separable convolution layer along with four convolution layers and one FC layer.

Experimental Setup and Result Analysis

In this segment, a complete set of experimental analysis along with all the performance measure considered for evaluating the effectiveness of proposed methodology has been presented. This experiment has been performed using three standard dataset of total 2250 frontal-view chest X-ray images (normal: 750 numbers, COVID-19: 750 numbers, and pneumonia: 750 numbers) considered from Figshare repository [31]. First, all the considered images were rescaled into 224 × 224. The entire devolvement of proposed scheme along with all pre-trained CNN model were simulated in Pytroch toolbox. Initially, the list of validation experiments analysis were performed in order to evaluate the efficiency of suggested LW-CBRGPNet scheme by utilizing tenfold cross-validation, confusion matrix, evaluation matrices, area under the ROC curve (AUROC), kappa score and Mathew correlation, and Grad-CAM heat map. However, the overall accuracy of both proposed scheme and pre-trained model was verified by using tenfold cross-validation, where nine parts deal with training the model and remaining one part is used for testing the model as depicted in Fig. 4. This process is performed for all ten parts and the efficiency of proposed scheme was evaluated by considering the average of all tenfold. The evaluation metrics were evaluated using different metrics such as true positives (TP), false negative (FN), true negative (TN), and false positive (FP). The corresponding mathematical formulation of each metrics are illustrated in Eqs. (7–11). Similar to evaluation metrics, kappa score also performed to measure inter-rater reliability among data items as shown in Eq. (12), where indicate the relative observed agreement, and refers to hypothetical probability. Matthew correlation Coefficient (MCC) is also considered to ensure the more validation of proposed models using four categories of confusion matrix (TP, FN, TN, and FP) using Eq. (13). It is a correlation coefficient value in the range of − 1 to + 1, where perfect, average random and inverse prediction is represented by using coefficients of + 1, 0 and – 1, respectively. Finally, Grad-CAM heat map is also introduced to interpret visually which part of images is more effective for feature map extraction using the gradient information fed into the final convolution layer of the CNN model [33]. In second phase of experimental setup, two different scenarios were performed, where first scenarios deal with multi-class classification, i.e., three class level prediction (normal, COVID-19, and pneumonia), where second scenario perform binary classification i.e., two class level prediction (normal and COVID-19). The batch size, number of epochs, and learning rate have been chosen empirically as illustrated in Table 3. Apart from this, the proposed LW-CBRGPNet is compared with recent CNN-based state-of-the-art approaches such as DarkCovidNet [39], DeepBayes-SqueexNet [40], Concatenation of Xception and ResNet50V2 [41], Tailored DCNN CovidNet [42], fuzzy color and stacking approach [43], CapsNet [44], and deep 3D multiple instance learning [45]. Furthermore, the comparison analysis has been presented using four standard pre-trained transfer learning CNN model such as ResNet101 [28], VGG-19 [29], DenseNet-121 [30], and XceptionNet [31].

Fig. 4

Graphical illustration of tenfold cross-validation in both Training and Testing Data

Table 3

Training parameters used in proposed LW-CBRGPNet model

Training parameter	Value
Learning Rate	0.0001
Batch Size	16
Optimizer	Adam
Loss function	Categorical Cross-entropy
Epochs	50
Flipping	Horizontal
Zoom range	15%
Rotation	10 degree
Lighting	20%
Re-scale	1/255

Results and Analysis

In this set of experimental analysis, a series of analysis to validate the suggested scheme was carried out. Initially, tenfold cross-validation was accomplished with the ratio of 10% for testing the data, where 90% used for training data as shown in Fig. 4. The overall accuracy and COVID-19 accuracy along with precision, sensitivity, F1 score of tenfold cross-validation is tabulated in Tables 4 and 5 respectively for both multi-class and binary class prediction, and the overlapped confusion matrix of class level prediction is depicted in Fig. 5. The kappa score and Mathew correlation matrices of the proposed model were evaluated as 97.57% and 96.67%, respectively. The both obtained matrices reach approximately to 1, which indicates the proposed model behaves properly to predict the accurate class. Figure 6 illustrates the graphical presentation of both kappa score and Mathew correlation matrices with respect to multi-class overall accuracy in different epoch’s values. The AUROC curve of the proposed model along with four pre-trained models is depicted in Fig. 7.

Table 4

Multi-class classification result using tenfold cross-validation

Folds	Precision	Sensitivity	Specificity	F1-Score	Overall accuracy	COVID-19 accuracy
1	96.91	96.92	98.44	96.89	97.82	98.22
2	97.39	97.35	98.65	97.36	98.5	99.11
3	96.92	96.93	98.45	96.89	97.94	98.22
4	97.76	97.81	98.88	97.76	98.83	99.56
5	98.16	98.28	99.34	98.2	98.8	99.56
6	98.71	98.64	99.10	98.66	99.11	99.11
7	96.83	96.92	98.70	96.9	98.02	98.67
8	98.23	98.23	98.89	98.52	98.93	99.11
9	95.56	95.65	97.77	95.57	97.54	97.78
10	96.46	96.42	98.22	96.47	97.83	98.22
Average	97.29	97.31	98.64	97.32	98.33	98.75

Table 5

Binary class classification result using tenfold cross-validation

Folds	Precision	Sensitivity	Specificity	F1-Score	Overall accuracy	COVID-19 accuracy
1	100	98.65	100	99.32	99.32	100
2	100	97.3	100	98.65	98.66	100
3	97.37	98.66	97.3	98.01	97.99	96.55
4	100	100	100	100	100	100
5	98.65	98.67	98.65	98.65	98.66	98.7
6	98.59	97.22	98.61	97.9	97.92	100
7	98.7	100	98.68	99.35	99.34	100
8	100	98.67	100	99.33	99.32	100
9	95.95	100	96.05	97.93	97.96	98.7
10	100	98.7	100	99.35	99.39	100
Average	98.93	98.78	98.93	98.85	98.86	99.39

Fig. 5

Confusion matrix obtained from proposed LW-CBRGPNet model in both multi-class and binary prediction level

Fig. 6

Illustration of Kappa score, Mathews correlation, and accuracy plot of proposed LW-CNRNet model

Fig. 7

AUROC curve of proposed LW-CBRGPNet model along with four pre-trained CNN model

Finally, Grad-CAM heat map is also introduced to interpret visually which part of images is more effective for feature map extraction using the gradient information fed into the last convolution layer of the CNN model [33]. This process is achieved by sliding a small window of particular size SXS in original chest X-ray images and then checking the occupied area by using a trained model to find out whether this particular area is affected by COVID-19 infection or not. This process is continued to cover all regions of the input image by sliding the window with a particular size of stride N to generate heat map, so that the radiologist can easily determine the area of interest where COVID-19 affected. Figure 8 illustrated the example images of heat maps in COVID-19 infected images. The training and validation loss across at different epochs is illustrated in Fig. 9.

Fig. 8

Sample COVID-19 infected region detected by heatmap analysis

Fig. 9

Loss convergence plot obtained for proposed LW-CBRGPNet between number of epoch and loss

Proposed Model Comparison with Different Optimization techniques

In this experimental setup, various optimization techniques were explored to attain the superior classification accuracy. Initially, the training phase was performed by empirically selecting Adam [48] optimizer. To evaluate the effectiveness of the proposed scheme using Adam optimizer a comparison with three popular optimization techniques namely RMSProp [46], AdaDelta [47], and AdamW [49] was evaluated. The detailed classification results using different optimizers are enumerated in Table 6. From these findings, it can be noticed that the classification accuracy of proposed schemes using Adam optimizer outperform better as compared to other considered optimizers. Hence, the rest of the experimental analysis was carried out using Adam optimizer.

Table 6

Comparison of classification performance (in %) among different optimizers

Model	Optimizer	Precision	Sensitivity	Specificity	F1-Score	Accuracy
Proposed LW-CBRGPNet	RMSProp	96.05	96.22	96.15	95.96	97.17
	AdaDelta	95.39	95.55	96.96	95.3	96.77
	AdamW	95.86	96.05	97.74	95.79	97.07
	Adam	97.29	97.32	98.64	97.32	98.33

Optimal Learning Rate Selection

The optimal learning rate is decided based on the minimum loss. Figure 10 shows the graphical representation of learning rate selection between learning rates and the validation loss. The red dot indicates the optimal learning rate selection where loss decreases by reaching global minima.

Fig. 10

Plot between learning rate and loss obtained in proposed LW-CBRGPNet model

Results Comparison of Different Batch Sizes

In this section, evaluation of different batch size experiments has been performed to show the impact of batch size on testing accuracy. Table 7 shows the test accuracy of the proposed scheme when trained using different batch sizes such as 8, 16 and 32. From this experiment, it can be observed that a higher and stable testing performance is obtained by using batch size 16, and hence, a batch size of 16 has been chosen in the study.

Table 7

Testing accuracy (in %) obtained by the proposed LW-CBRGPNet model with different batch sizes

Model	Batch size
	8	16	32
Proposed LW-CBRGP-Net	97.27	98.33	97.76

Misclassification Results Analysis

The misclassification samples predicted by proposed LW-CBRGPNet are depicted in Fig. 11. The misclassification possibly occurred due to the similar imaging features among three class datasets consisting of normal, COVID-19, and pneumonia infection cases.

Fig. 11

Illustration of Misclassification results obtained by proposed LW-CBRGPNet model

Comparison with Pre-trained Transfer Learning Models

In this section, the comparison study has been performed on four popular pre-trained transfer learning models along with a proposed model to provide a comprehensive view for diagnosis of COVID-19 infection. The detailed description of each pre-trained model considered in this study is illustrated in Sect. 2.5, and detailed results of this experimental analysis is tabulated in Table 8. As like to earlier experiment analysis, here also the same three sets of X-ray images were deployed. The result obtained from this analysis is more promising and outperforms better as compared to four pre-trained models such as ResNet-101 [28], VGG-19 [29], DenseNet [30], and XcptionNet [31]. However, the proposed model is more effective over the pre-trained model, as it requires less number of learning parameters, faster training speed, and computational cost effectiveness because of lesser Convolutional layer being used as compared to pre-trained model. It is worth noting that all pre-trained transfer learning models are originally designed for ImageNet [33] classification problems for a large amount of labeled dataset. Therefore, fine-tuned has been in operation on the last layer by employing a fully connected (FC) layer to implement in chest X-ray images to achieve both binary class (Normal, COVID-19) problem and multi-class problem (Normal, COVID-19, and pneumonia). The FC layer of all these models is trained, whereas the remaining layers are initialized with pre-trained weights. The detailed explanation of these each model along with parameter specification are discussed in Table. 3.

Table 8

Comparison of classification results (in %) of the proposed model with Pre-trained CNN models

Pre-trained model	Precision	Sensitivity	Specificity	Accuracy	F1-score	AUROC
ResNet-101 [28]	97.46	97.42	98.61	98.1	97.37	97.04
VGG-19 [29]	96.26	96.29	98.09	97.45	96.22	96.02
DenseNet-121 [30]	96.16	96.28	98.09	97.45	96.2	95.09
XceptionNet [31]	95.43	95.53	97.73	96.96	95.46	95.10
Proposed LW-CBRGPNet	97.29	97.31	98.64	98.33	97.32	98.24

Comparison Analysis with State-of-the-Art Methods

In this study, a comparative analysis of proposed scheme along with alternative state-of-art techniques and four pre-trained models has been made in both multi-class classification and binary class classification on chest X-ray images. The detailed results of both class level predictions are illustrated in Table 9. From this result analysis, it can be observed that the suggested framework attains better overall accuracy in comparison to state-of-art techniques. Furthermore, a separate comparison of the COVID-19 class performance of the proposed approach has been made with the existing schemes as tabulated in Table 10. The research in the literature [33] and [40] has attained 100% COVID-19 class sensitivity for the use of less number of sample images. This proposed result again shows superior performance as context to the existing models.

Table 9

Comparing the efficiency of proposed model with current state-of-the-art deep learning COVID-19 identification approaches (in %) using chest X-ray images

References	Method	Number of chest X-ray samples	Binary class accuracy (%)	Multi-class Accuracy (%)
Nayak et al. [33]	ResNet34	Total:286 (COVID:143, Normal:143)	98.33	–
Ozturk et al. [39]	Dark CovidNet	Total:1127 (COVID:127, Normal:500, and Pneumonia:500)	98.08	87.02
Ucara et al. [40]	Deep Bayes-SqueezeNet	Total:5949 (COVID:76, Normal:1583, and Pneumonia:4290)	–	98.30
Rahimzadeh et al. [41]	Concatenation of Xception and ResNet50V2	Total:15,085 (COVID:180, Normal:6054, and Pneumonia:8851)	–	91.40
Wang et al. [42]	Tailored DCNN (Covid-Net)	Total:13,975 (COVID:398, Normal:8066, and Pneumonia:5538)	–	93.30
Togaçar et al. [43]	Fuzzy color and Stacking Approach	Total:458 (COVID:295, Normal:65, and Pneumonia:98)	–	97.06
Toramana et al. [44]	Convolutional CapsNet	Total:3150 (COVID:1050, Normal:1050, and Pneumonia:1050)	97.24	84.22
Han et al. [45]	Deep 3D Multiple Instance Learning	Total:460 (COVID:230, Normal:100, and Pneumonia:130)	97.90	94.30
Proposed	LW-CBRGPNet	Total:2250 (COVID:750, Normal:750, and Pneumonia:750)	98.86	98.33

Table 10

Comparison of COVID-19 class performance with other state-of-the-art approaches

References	COVID-19class sensitivity (%)	Specificity (%)	Precision (%)	Class
Nayak et al. [33]	100	96.67	96.77	Binary class
Ozturk et al. [39]	90.65	95.30	98.03	Multi-class
Ucara et al. [40]	100	99.10	98.30	Multi-class
Rahimzadeh et al. [41]	99.50	99.56	90.83	Multi-class
Wang et al. [42]	91.00	–	–	Multi-class
Togaçar et al. [43]	99.32	99.37	99.66	Multi-class
Toramana et al. [44]	84.22	97.04	97.06	Multi-class
Han et al. [45]	–	–	95.90	Multi-class
Proposed	98.75	98.64	97.29	Multi-class

Research Discussion

An automated deep learning-based LW-CBRGPNet framework is suggested in this paper for the diagnosis of COVID-19 infection with respect to pneumonia and normal cases by utilizing a dataset consisting of chest X-ray images. A Total of 2250 numbers of frontal-view of X-ray radiographs (normal: 750 numbers, COVID-19: 750 numbers, and pneumonia: 750 numbers) were used in this study to maintain the data balance. In context to COVID-19 diagnosis, several deep learning-based research outcomes have been reported for diagnosis of COVID-19 cases. However, most existing approaches are limited in terms of accuracy, utilization of limited unbalanced datasets, more computational cost and more memory utilization. Therefore, the proposed model is designed to address these issues, which is not only capable of learning feature maps from input images automatically but it also considers less learning parameters during the training phase. To evaluate the efficacy of the proposed scheme, various extensive experiments were accomplished by considering the various hyperparameters such as learning rate, different batch size, misclassification rate, and optimizer. Furthermore, several performance metrics were performed to evaluate the proposed scheme like tenfold cross-validation, confusion matrix, evaluation metrics, AUROC, kappa score and Mathew’s correlation, and Grad-CAM heat map. Finally, the comparative analysis was made in counter to four popular pre-trained models along with recent state-of-the-art techniques in both binary class and multi-class classification. Apart from this, a separate COVID-19 class performance along with precision and sensitivity has been presented as compared to existing techniques. The outcome of the proposed scheme from every set of experimental analysis outperforms better in every aspect by achieving better accuracy as context to state-of-art techniques. This proposed model is more robust, lightweight, and it can be suitable for radiologists for early diagnostics of COVID-19 cases thereby, reducing diagnostic error. The major advantages and disadvantages of proposed scheme as follows:

The proposed lightweight CNN architecture was designed by using only three CBR blocks accompanied by max-pooling layer, global average pooling layer and FC layer to handle overfitting issues.

The model is capable enough to learn the feature map automatically during the training phase. Hence, it does not require a handheld feature extraction process. However, the proposed architecture is more scalable and lightweight, and hence, it can be easily integrated with any smart devices for easy diagnosis.

The computational cost of the proposed model is very low when compared with other state-of-art techniques and pre-trained models, as a global average pooling layer was used followed by one softmax layer.

The proposed framework requires very less parameters during training the model as compared to pre-trained CNN model. Hence, it can be easily accommodated with any smart devices for early diagnosis.

The limitation of this study is that it is validated using a limited amount of dataset, and the input image size differs in both size and number of channels. Despite its different size, it is a challenge for the proposed model.

Conclusion

In this article, an automated and novel LW-CBRGPNet multi-class classification model for early and accurate diagnostics of COVID-19 patients utilizing chest X-ray radiographs is proposed. The proposed scheme is achieved based on a deep learning model, particularly CNN architecture, and training from scratch without performing handheld feature extraction which is completely opposite to a pre-trained model. The complete set of experimental analysis was carried out using publicly available chest X-ray radiograph dataset. As a result of this study, the proposed model provides an accuracy of 98.33%. Furthermore, the validation of the proposed scheme has been made by considering several hyperparameters and several comparisons were studied by considering four pre-trained transfer learning models along with recent state-of-the-art techniques. The outcome of the proposed scheme performs better in all aspects of experimental analysis for detecting COVID-19 infection. This proposed model is more robust, lightweight, and it can be suitable for radiologists for early diagnostics of COVID-19 cases thereby, reducing diagnostic error.