Concat_CNN: A Model to Detect COVID-19 from Chest X-ray Images with Deep Learning.

COVID-19 is creating havoc on the lives of human beings all around the world. It continues to affect the normal lives of people. As number of cases are high, a cost effective and fast system is required to detect COVID-19 at appropriate time to provide the necessary healthcare. Chest X-rays have emerged as an easiest way to detect COVID-19 in no time as RT-PCR takes time to detect the infection. In this paper we propose a concatenation-based CNN model that will detect COVID-19 from chest X-rays. We have developed a multiclass classification problem which can detect and classify a chest X-ray image as either COVID + ve, or viral pneumonia, or normal. We have used chest X-rays collected from different open sources. To maintain class balancing, we took 500 images of COVID, 500 normal images, and 500 pneumonia images. We divided our dataset in training, validation, and test set in 70:10:20 ratio respectively. We used four CNNs as feature extractors from the images and concatenated their feature maps to get better efficiency of the network. After training our model for 5 folds, we have obtained around 96.31% accuracy, 95.8% precision, 92.99% recall, and 98.02% AUC. We have compared our work with state-of-the-art pretrained transfer learning algorithms and other state-of-the-art CNN models referred in different research papers. The proposed model (Concat_CNN) exhibits better accuracy than the state-of-the-art models. We hope our proposed model will help to classify chest X-rays effectively and help medical professionals with their treatment.

Introduction

Coronaviruses are a large group virus which cause respiratory distress. COVID-19 or Coronavirus 2 is a new strain of this virus group that was previously not found in human body. This new strain of virus is named as ‘COVID-19’ virus as the first case of this virus was found in 2019. According to WHO’s confirmed report, first case of novel coronavirus (n-COV) was detected in Wuhan City, the largest metropolitan area of Hubei province in China in December 2019 [1]. COVID-19 virus is a single strand RNA that causes moderate to severe respiratory infection. In case of mild to moderate infection, infected person will recover within 14 days. As SARS-COV-2 is contagious and can spread rapidly among proximity of infected person, around 195 million people have been infected and around 4.1 million people died due to COVID-19 as of July 2021 [2]. Since the inception of the disease, be it developed countries like USA, China, Italy, England etc. or developing countries or underdeveloped countries, all are suffering due to huge case load and going through shortage of healthcare equipment, facilities, and lack of proper diagnosis approach with effective efficiency. SARS-COV-2 spreads rapidly through droplet from mouth, nose from an infected person to an uninfected person. Four variants of this virus—alpha, beta, gamma, delta are marked as variants of concern, whereas Eta, Iota, Kappa, Lambda falls under variant of interest [3].

Clinical Detection of COVID-19 largely depends on a genetic test known as Reverse Transcription Polymerase Chain Reaction (RT-PCR). Nasopharyngeal swabs are taken from infected patient for diagnosis purpose and RNA of the virus is extracted [4]. It takes around 2 h for detection with long turnaround time. So, it is costly, time consuming and complicated. If the swabs are not collected properly its accuracy rate becomes exceptionally low and might show negative result though the person is positive [5]. Rapid antigen testing is an easy-to-use process with less infrastructure requirement and equipment. It checks the presence of one or more antibodies in the infected person’s body as a response against the virus. Though it is cost friendly, easy to perform, quick response, and able to detect active SARS-COV-2 virus, its sensitivity and specificity is much lower than regular RT-PCR test [6]. The best solution is use of radiographic images like chest X-rays and chest CT-Scans. The advantage of using radiographic images is its wide availability all over the world. It is cost effective, and diagnosis is done at the time of testing. The main concern while using chest X-ray images is availability of trained doctors or healthcare professional. Research on infected chest X-ray images has found that some unknown features like vague darkening spots or ground glass opacity helps in COVID-19 detection [7]. However, analyzing every chest X-ray individually at the time of pandemic will delay the detection process thereby delaying the treatment procedure. Moreover, in rural places, availability of efficient radiologist is a problem for timely intervention and detection. So, an automatic COVID detection system or rather a computer aided COVID detection system is an important requirement at present scenario.

While the pandemic is still creating havoc on mankind, different researchers have come up with different ideas and proposed different systems for early detection and prevention of diseases. Among them applications of deep learning algorithms have shown promising results. In healthcare sector, deep learning [8] has shown tremendous development in automatic disease prediction such as tuberculosis detection [9, 10], cancer detection [11, 12], tumor detection [13, 14], bone fracture detection [15, 16], genome sequence analysis [17-19] etc. In their study, Abbas et al. [20], proposed a model named DeTraC that classifies chest X-ray images. The model is based on deep CNN models. Class decomposition layer was used to simplify the dataset structure. ResNet pretrained model was used to pretrain the model and then class-composition layer was used for parameter fine tuning. The dataset size was 185 images where 105 images are virus infected and 80 are normal chest X-ray images. The authors have achieved 95.12% accuracy, 97.91% sensitivity and 91.87% specificity. Das et al. [21], had proposed an ensemble of multiple state of the art CNN models like InceptionV3, ResNet50V2, and DenseNet201. All these models are trained individually and then training results are combined using weighted average ensemble technique. The dataset containing 538 COVID + images and 468 COVID − images is divided into train, test and validation set. The accuracy of the proposed model is 91.62%. Wang et al. [22] proposed a custom deep learning architecture named COVIDNet that classifies images into three categories: COVID infected patients, healthy patients, and pneumonia patients. Their model is designed using a lightweight projection-expansion-projection-extension (PEPX) pattern which obtained a classification accuracy of 94%. Apostolopoulos et al. [23] proposed COVID-19 identification system using five pretrained deep learning models. Transfer learning models VGG19, Inception, Xception, MobileNet, and Inception_ResNet_V2 are used for classification. They achieved accuracy around 96.78%. Their binary classification results are better than multiclass classification. A hybrid model combining Xception and ResNet50_V2 model was proposed by Rahimzadeh et al. [24]. They used a dataset of 6054 X-ray images and achieved an accuracy of 91.4%. Fu et al. [25] explored two characteristics of SARS-COV-2-dynamic viral shedding and serological characteristics. They identified the risk factors associated with poor recovery and lung lesion resolutions. They studied the relationship between pro-inflammatory response, viral shedding, and lung lesion evolution. According to their study, 58% of the infected patients will have poor recovery and prolonged interval of viral shedding. In another study, Cheng et al. [26], compared the performances of different ensemble CNN models like majority voting, unweighted average, and Bayes Optimal Classifier. Afsar et al. [27], have proposed a model named COVID-CAAPS which is based on capsule network to detect COVID infection. They combined different convolution and capsule layers to address the problem of class-imbalance. With smaller number of trainable parameters, they obtained an accuracy of 95.7%, sensitivity of 90% and specificity of 95.8%. A model named COVID-RENet [28] was proposed by Cohen et al. which extracts edge and region-based features using CNN. Later SVM was used to perform classification and improve accuracy. They have implemented a fivefold cross validation for on the COVID-19 dataset.

Different researchers have proposed different models for COVID detection using deep learning approaches. Most of the works are based on CNN as it shows higher accuracy for classification due to its hierarchical structure.

We have implemented a concatenation based convolutional neural network model for COVID-19 detection from chest X-rays and compared our results with state-of-the-art benchmarked transfer learning algorithms and other state-of-the-art works done by different researchers. This paper is organized as follows-

“Data Description” of the paper discusses the details of the data used for implementation. It focuses about the type of data available, data preprocessing steps, and the tools and environment we used for our model implementation. “Proposed Model” presents the detailed architecture of our proposed model. It also provides the training details and parameter fine tuning details. “Implementation Results” shows the implementation results using different performance metrics and comparisons are shown with state-of-the art transfer learning models and other state-of-the-art other research works. Finally, “Conclusion” concludes the implementation and discussion on the outcome of the proposed model.

Data Description

In this section details of the data source, dataset description, image preprocessing techniques are discussed.

Data Collection

For performing our proposed research work, we collected chest X-ray images from different online open sources. These dataset sources contain chest X-ray images having different infection manifestations including COVID-19 viral infection. These images are collected from all over the world and contain different chest X-ray images like viral pneumonia, bacterial pneumonia, normal chest X-ray images. Chest X-ray images of Viral infections like COVID-19, SARS, MERS-CoV, varicella, influenza, herpes and bacterial infections like streptococcus, staphylococcus, Klebsiella, Escherichia coli, Mycoplasma are included in these datasets with detailed medical information. Different views of these images, like: anterior–posterior view (AP), lateral view (L), and posterior anterior (PA) view are available. As our focus is to view the condition of the lungs for any infection, posterior-anterior (PA) views suit best. For this purpose, we have considered images having PA views only and discarded other views. Even though there are many viral and bacterial categories of images, for our experiment purpose we have considered three classes—COVID-19 positive, Normal chest X-ray, and viral pneumonia. Normal chest X-ray (CXR) images are considered as class 0, COVID-19 positive as class 1, and viral pneumonia as class 2. To maintain class balancing, we have created our dataset with equal number of images in each class, i.e., 500 COVID Chest X-ray images, 500 normal chest X-ray images, and 500 chest X-ray images of viral pneumonia. We have obtained our images from the following open sources:

150 images from GitHub which has images from all three classes i.e., 50 images from each class (https://github.com/ieee8023/covid-chestxray-dataset).

55 COVID images from GitHub (https://github.com/Figure-1-COVID-chestxray-dataset).

250 COVID images, 300 normal images and 270 Pneumonia images (https://www.kaggle.com/tawsifurrahman/covid19-radiography-database).

95 COVID images and 75 normal images(https://www.kaggle.com/nabeelsajid917/covid-19-x-ray-10000-images).

75 normal images and 230 pneumonia images (https://www.kaggle.com/paulimothymooney/chest-xray-pneumonia).

Data Preprocessing

As the images are collected from different sources, images are of different shapes and their brightness and contrast are different. So, the images are first resized to 224 × 224 shape. To improve the contrast of all images, we implemented an image processing technique Contrast Limited Adaptive Histogram Equalization technique (CLAHE) [39]. Here the entire image is divided into fixed number of tiles, contrast of each tile is enhanced, and then bilinear interpolation is used to remove the artificial boundaries within the images. CLAHE is used to enhance the lungs region of each image for better understanding. The images are next normalized by a factor of 1/255. The images are then shuffled for better distribution and the whole dataset is divided into train, validation, and test set in a ratio of 70:10:20 respectively (Table 1). In Table 2, a clear insight of image data distribution in provided.

Table 1

Dataset description

Type of images	No. of images
Healthy or normal (class 0)	500
COVID-19 (class 1)	500
Non-COVID-19 viral pneumonia (class 2)	500

Table 2

Image distribution for training, validation, and testing dataset 

Dataset	Healthy (class 0)	COVID + ve (Class 1)	Non-COVID viral pneumonia (class 2)	Total
Training	360	360	360	1080
Validation	40	40	40	120
Testing	100	100	100	300

Tools Used for Implementation

We have used Google Colab GPU (Tesla K80), Python 3.7. We used TensorFlow 2.7.0 and Keras 2.7.0 libraries for our model implementation on Google COLAB platform (Table 2).

Proposed Model

We have implemented a concatenation approach of convolutional neural network for COVID-19 detection using chest X-rays. Concatenation of CNNs’ helps to combine the features extracted by CNNs’ to enhance the classification process more accurate. Four different CNNs with different convolutional layer architecture are implemented as feature extractors from the training dataset. This section discusses the proposed concatenated CNN architecture, necessary optimization process, and parameter fine tuning stages.

CNNs as Feature Extractor

In this work four CNN architectures are used as feature extractors. These CNNs have been used to learn good representations of the images like edges, shapes, rotation, spatial representation. These models extract relevant features from the chest X-ray images to obtain a proper knowledge about normal, COVID infected and pneumonia characteristics. We have used Scikit-Learn, Keras, TensorFlow, OpenCV python libraries for implementing different functionalities on the model. Below image shows the architecture illustration of our COVID detection model.

In our proposed model in Fig. 1, we have illustrated four different CNN architectures. The preprocessed training images of size 224 × 224 pixel are provided as input to all four CNNs. Feature maps are generated from the last feature extraction layer of all extractors. As all CNNs extract same size of feature maps, we use a concatenation function to generate an enhanced quality of semantic features. A concatenated convolutional neural network (Concat_CNN) is proposed by using all extracted feature maps concatenation and connecting this concatenation layer to a fully connected layer with 32 neurons in association with ReLU Activation function which is further connected to a classifier layer. Our proposed model is implemented as a multi-class classification problem as we have categorized our data in three classes as COVID-19, normal, and pneumonia. Each feature extractor CNN is composed of two convolutional layers with varying filter size of 3 × 3. Each convolution layer is followed by a batch normalization layer, maxpooling layer with pool size (2 × 2) followed by a dropout rate at 0.5. ReLU activation function is used in each convolution layer. Rectified activation function is one of the most popular activation functions which directly maps the inputs to the outputs if it is positive, else it maps zero. It performs better than vanishing gradient and works faster. The equation of the Relu function can be defined as:

Fig. 1 

Proposed Concat_CNN architecture

Keras concatenation function concatenate is used to merge all feature extractors output. An interpretation layer or a fully connected layer with 32 neurons is further added followed by the final output layer. Activation function Softmax is used in the output layer as it acts best in multiclass classification problem. The equation of Softmax function can be defined as:

In each convolutional layer, a kernel initializer ‘he_uniform’ is used.

Kernel initializers are used to stabilize the weight initialization and distribution for neural network in each stage of iteration. Below tables provide an insight of the CNNs used as the feature extractors (Tables 3, 4, 5, 6).

Table 3

Details of all layers in feature extractor 1

Layer	Filter size	Number of filters	Pool size	Stride	Padding	Dropout	Activation function
Conv2D	3 × 3	32	–	1	Same	–	Relu
Batch Normalization	–	–	–	–	–	–	–
Maxpooling2D	–	–	2 × 2	2	-	–	–
Dropout	–	–	–	–	–	0.5	–
Conv2D	3 × 3	32	–	1	Same	–	Relu
Batch Normalization	–	–	–	–	–	–	–
Maxpooling2D	–	–	2 × 2	2	–	–	–
Dropout	–	–	–	–	–	0.5	–
Flatten	–	–	–	–	–	–	–

Table 4

Details of all layers in feature extractor 2

Layer	Filter size	Number of filters	Pool size	Stride	Padding	Dropout	Activation function
Conv2D	3 × 3	64	-	1	Same	–	Relu
Batch Normalization	–	–	–	–	–	–	–
Maxpooling2D	–	–	2 × 2	2	–	–	–
Dropout	–	–	–	–	–	0.5	–
Conv2D	3 × 3	64	-	1	Same	–	Relu
Batch Normalization	–	–	–	–	–	–	–
Maxpooling2D	–	–	2 × 2	2	–	–	–
Dropout	–	–	–	–	–	0.5	–
Flatten	–	–	–	–	–	–	–

Table 5

Details of all layers in feature extractor 3

Layer	Filter size	Number of filters	Pool size	Stride	Padding	Dropout	Activation function
Conv2D	3 × 3	32	–	1	Same	–	Relu
Batch Normalization	–	–	–	–	–	–	–
Maxpooling2D	–	–	2 × 2	2	–	–	–
Dropout	–	–	–	–	–	0.5	–
Conv2D	3 × 3	64	–	1	Same	-	Relu
Batch Normalization	–	–	–	–	–	–	–
Maxpooling2D	–	–	2 × 2	2	–	–	–
Dropout	–	–	–	–	–	0.5	-
Flatten	–	–	–	–	–	–	–

Table 6

Details of all layers in feature extractor 4

Layer	Filter size	Number of filters	Pool size	Stride	Padding	Dropout	Activation function
Conv2D	3 × 3	32	–	1	Same	–	Relu
Batch Normalization	–	–	–	–	–	–	–
Maxpooling2D	–	–	2 × 2	2	–	–	–
Dropout	–	–	–	–	–	0.5	-
Conv2D	3 × 3	64	–	1	Same	-	Relu
Batch Normalization	–	–	–	–	–	–	–
Maxpooling2D	–	–	2 × 2	2	–	–	–
Dropout	–	–	–	–	–	0.5	-
Flatten	–	–	–	–	–	–	–

Training and Parameter Fine Tuning

The proposed model is compiled using Adam optimizer provided by Keras library with the parameters as learning rateα α = 0.0001. Categorical Crossentropy Loss function is used as the model is a multiclass classification problem. The model has been trained for 50 epochs with Early stopping callbacks with patience for 10 epochs. To train our model effectively, we have considered fivefold cross validation in addition to splitting of images into different sets. In each fold, a different set of images is used as training and testing set while keeping the validation image set fixed.

Implementation Results

Performance Metrics

We evaluate our proposed approach by considering the following four performance evaluation metrics—classification accuracy, precision, recall, AUC [35]. Where TP stands for True Positive, TN stands for True Negative, FP stands for False Positive, FN stands False Negative. COVID + cases that are correctly classified are called True positive cases in the confusion matrix. If COVID + cases are falsely classified as COVID − then these are called False Positive. Same wise, if COVID − cases are classified correctly then these are called True Negative and COVID − cases are falsely classified as COVID + are known as False Negative.

Performance Evaluation and Compare Benchmark

In our work, we have proposed a concatenation-based CNN (Concat_CNN) model for COVID-19 disease prediction. However, for comparison purpose we have compared our proposed model results with other contemporary works done for COVID-19 disease prediction using chest X-rays. We also compared our work with individual transfer learning algorithms performance. This provides a better understanding of our proposed model’s performance with other contemporary works and benchmark transfer learning algorithms. As our model is trained with fivefold cross validation and for 50 epochs with learning rate at 0.0001 and Adam optimizer, same training approach is adopted for all transfer learning algorithm implementation. We have considered VGG16 [31], InceptionV3 [34], ResNet50 [32], and DenseNet201 [33] transfer learning algorithms. All model evaluation results are saved as.h5 files.

Table 7 demonstrates an overview of accuracy, precision, recall and AUC of the benchmark transfer learning algorithms and our proposed model. It is clearly understood from the table that our proposed model exhibits better results in most of the folds in terms of accuracy, precision, recall and AUC. Average accuracy obtained by our model is 96.31%, average precision is 95.8%, average Recall is 92.99% and average AUC is 98.02%. It helps to prove the efficiency and robustness of the proposed approach.

Table 7

Summary of all performance comparison with transfer learning algorithms

Fold no	Model	Accuracy	Precision	Recall	AUC
1	VGG16	87.22%	89.55%	90.74%	91.38%
	InceptionV3	78.55%	89.73%	90.12%	92.34%
	Resnet50	80.11%	82.97%	87.99%	88.80%
	DenseNet121	79.89%	90.56%	92.66%	93.45%
	Proposed Approach	93.44%	91.69%	88.33%	96.56%
2	VGG16	89.99%	91.22%	91.86%	93.82%
	InceptionV3	93.44%	93.59%	94.66%	96.44%
	Resnet50	90.55%	91.82%	89.66%	94.77%
	DenseNet121	96.88%	94.88%	92.08%	95.67%
	Proposed Approach	95.22%	93.55%	92.00%	96.48%
3	VGG16	94.17%	95.58%	92.00%	94.10%
	InceptionV3	94.11%	95.89%	94.89%	95.54%
	Resnet50	95.33%	96.86%	92.45%	95.09%
	DenseNet121	96.33%	97.25%	94.33%	99.36%
	Proposed Approach	97.22%	97.57%	93.99%	95.8%
4	VGG16	96.77%	96.71%	96.00%	95.49%
	InceptionV3	95.34%	96.20%	95.00%	97.28%
	Resnet50	96.44%	96.98%	96.57%	96.77%
	DenseNet121	96.54%	95.45%	95.78%	98.68%
	Proposed Approach	97.66%	97.28%	95.66%	99.32%
5	VGG16	96.77%	97.67%	96.85%	99%
	InceptionV3	96.88%	96.00%	94.07%	98.56%
	Resnet50	97.24%	97.48%	95.00%	97.11%
	DenseNet121	97.55%	98.07%	98.33%	98.56%
	Proposed Approach	98.00%	98.95%	94.99%	99.15%
Average	VGG16	92.98%	94.14%	93.43%	94.75%
	InceptionV3	91.66%	94.28%	93.78%	96.03%
	Resnet50	91.93%	93.22%	92.33%	94.50%
	DenseNet121	93.43%	95.24%	94.63%	97.14%
	Proposed Approach	96.31%	95.8%	92.99%	98.02%

Figure 2 shows the confusion matrix of our proposed model obtained in all five folds. Each confusion matrix provides an overview of consistent performance of our model in all folds.

Fig. 2

Confusion Matrix of proposed Concat_CNN model for five folds

If we analyze the confusion matrix of all folds, our concatenated CNN model detects most of the cases correctly. From this confusion matrix, it is evident that the model clearly identified true positive and true negative values efficiently.

In Fig. 3, accuracy, loss, precision, recall and AUC graphs are shown for all five folds. In each fold all the metrics exhibit better results than the previous folds. The proposed model also gives consistent high precision which again proves the sturdiness of the model.

Fig. 3

Accuracy, loss, recall, precision and AUC graphs for all 5 fold validation

Also, to demonstrate our proposed concatenated CNN model’s efficacy towards detection of COVID cases based on chest X-rays, a comparison is made in Table 8 with some recent studies and their implementation details. Among the different research works, some have shown promising results whereas some have faced difficulty with a small dataset [37, 38]. Some have even proposed new methods which shows promising results [37, 38]. From the Table 8 it can be observed that our proposed Concat_CNN model has the capability to yield better results than state-of-the-art deep learning models [24, 29, 30, 36] and implementations.

Table 8

Comparative study of COVID detection using chest X-ray images

Author’s name	Dataset description	Model description	Training details	Accuracy	Precision	Recall
Tahmina Zebin [29]	802 CXR images	VGG16 (Augmentation) VGG16 (GAN Augmentation) ResNet50(GAN Augmentation) EfficientNetB0(GAN Augmented)	80% training, 20% testing	88%, 90%, 94.3% 96.8%	93%	90%
Abdullahi Umar Ibrahim [30]	COVID-19—371 Viral pneumonia—4237, Bacterial Pneumonia—4237, Healthy—2882	Pretrained AlexNet Model for two-way, three way and four-way classification	70% for training, and 30% for testing	Average 95.5% around all classes	Average 93.11% around all classes	Average 96.56% around all classes
Mohammad Rahimzadeh [24]	Normal—8851, pneumonia—6054, Covid—180	A concatenation network of ResNet50V2 and Xception (fivefold cross validation)	–	Overall accuracy 91.4% in five folds	–	–
Amit Kumar Das [36]	training images—771 (COVID—438, normal—333), testing images—118 (COVID—43, normal—75), validation—117 (COVID—57, normal—60)	An ensemble network of DenseNet201, ResNet50V2, InceptionV3 (fivefold cross validation)	–	Accuracy—91.62%		Average 95.09 for COVID + ve, average 88.33 for COVID − ve
Liinda Wang [37]	Total—13,975, COVID—5538, 8066	COVIDNet	Train: test—50:1	93.03%		COVID + —91%, COVID − —94%Z
Asif Iqbal Khan [38]	Total—594, COVID—284, normal—310	Proposed a new model CoroNet	–	95.3%	98.60%	97.5%

Conclusion

In our paper, we have implemented a concatenation-based CNN (Concat_CNN) model to classify chest X-ray images into three classes: normal, COVID, and pneumonia. We used four CNN networks as feature extractors and concatenated the feature map to get more accurate classification. We obtained our dataset from different open sources. We tried to maintain class balancing by choosing equal no. of images for all classes in training phase. Our proposed model has an average classification accuracy of 96.31% between five folds. We hope our model will be helpful for medical diagnosis as number of COVID cases are still significantly increasing. So, a computer based COVID detection system will accelerate the detection and classification and will further help to solve the case load problem of medical professionals.