ADL-CDF: A Deep Learning Framework for COVID-19 Detection from CT Scans Towards an Automated Clinical Decision Support System.

The emergence of deep learning has paved to solve many problems in the real world. COVID-19 pandemic, since the late 2019, has been affecting lives of people across the globe. Chest CT scan images are used to detect it and know its severity in patients. The problem with many existing solutions in COVID-19 detection using CT scan images is that inability to detect the infection when it is in initial stages. As the infection can exist on varied scales, there is need for more comprehensive approach that can ascertain the disease at all scales. Towards this end, we proposed a deep learning-based framework known as Automated Deep Learning-based COVID-19 Detection Framework (ADL-CDF). It does not need a human medical expert in diagnosis as it is capable of detecting automatically. The framework is assisted by two algorithms that involve image processing and deep learning. The first algorithm known as Region of Interest (ROI)-based Image Filtering (ROI-IF) which analyses given input CT scan images of a patient and discards the ones where ROI is missing. This algorithm minimizes time taken for processing besides reducing false positive rate. The second algorithm is known as Multi-Scale Feature Selection algorithm that fits into the deep learning framework's pipeline to leverage detection performance of the ADL-CDF. The proposed framework is evaluated against ResNet50V2 and Xception. Our empirical study revealed that our model outperforms the state of the art. © King Fahd University of Petroleum & Minerals 2022, Springer Nature or its licensor holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.

Introduction

Covid-19 pandemic has causes unprecedented number of causalities across the globe. It is caused by novel coronavirus as revealed by World Health Organization (WHO). The disease soon became a pandemic ever since it appeared in Wuhan city of China in the late 2019. It could rapidly spread to different countries in the span of few months and caused severe damage to health and lives of people. Since the disease affects lungs, most of the research found in the public domain is based on chest X-rays and CT scan images [1]. Many deep learning models found in [1-3] and [4], to mention few, explored X-ray images for disease diagnosis. Since X-ray is cost-effective, it is preferred in most of the healthcare units. However, it is essential sometimes to use CT scans for better diagnosis though it is costlier than X-ray. Therefore, it is indispensable to have research on CT imagery as well. As explored in [5-7] and [8], to mention few, CT scan images are preferred in order to have better diagnosis of the disease. The research found that image-based diagnosis or computer vision-based approach reflected the usage of many deep learning models.

Pre-trained deep learning models along with LightGBM are used in [1] while deep transfer learning approach is preferred in [2]. A novel deep learning model known as nCovnet is used in [3] and pre-trained CNN models such as ResNetXt, Xception and InceptionV3 are preferred in [4]. ResNet and its variants are widely used as studied in [4, 9, 10] and [11]. ResNet101 along with data augmentation technique is used in [10]. A pre-trained CNN model known as VGG-16 is used in [12] along with EfficientNet and DenseNet121. A deep learning model with Mix Match approach is used in [13] while an ensemble deep learning model is used in [14]. CNN with the notion of ROI is used in [6].

From the literature, several insights are ascertained. First, deep learning models work better for image-based applications. Second, CNN and its variants are widely used for Covid-19 detection. Third, ResNet50 is one of the most widely used model for Covid-19 diagnosis. Fourth, CNN models have the ability to extract required features that help in efficient classification. Another important insight is that the existing literature lacks methods for very early detection of Covid-19. The problem with many existing solutions in COVID-19 detection using CT scan images is that inability to detect the infection when it is in initial stages. As the infection can exist on varied scales, there is need for more comprehensive approach that can ascertain the disease at all scales. Our contributions to this paper are as follows.

We proposed a deep learning-based framework known as Automated Deep Learning-based COVID-19 Detection Framework (ADL-CDF).

We proposed two algorithms to realize the framework. The first algorithm known as Region of Interest (ROI)-based Image Filtering (ROI-IF) which analyses given input CT scan images of a patient and discards the ones where ROI is missing. This algorithm minimizes time taken for processing besides reducing false positive rate. The second algorithm is known as Multi-Scale Feature Selection (MSFS) algorithm that fits into the deep learning framework’s pipeline to leverage detection performance of the ADL-CDF.

A prototype is implemented to evaluate the performance of the framework and its underlying algorithms. The implemented methods and experimental results showed the novelty and utility of the proposed framework.

The remainder of the paper is structured as follows. Section 2 reviews the literature on existing methods for automatic detection of Covid-19 from lung imagery. Section 3 provides details of the proposed methods and materials including algorithms. Section 4 presents result of our empirical study. Section 5 concludes the paper and provides future scope of the research.

Related Work

This section reviews the literature on existing methods of Covid-19 detection with lung imagery. Kassani et al. [1] used many pre-trained deep learning models in the process of Covid-19 detection using lung X-Ray and CT scans. In fact, they used them for feature extraction and followed by ML models for classification. Panwar et al. [2] also used lung X-Ray and CT scans with deep learning classifiers. Panwar et al. [3] proposed a CNN-based deep model known as nCovnet with X-Rays for fast detection of Covid-19. Jain et al. [4] proposed a methodology for X-Ray-based Covid-19 detection using model such as Inception V3, ResNeXt and Xception. Desai et al. [15] explored the role of deep learning in medical imaging associated with Covid-19 diagnosis. Ismael and Sengur [9] investigated multi-resolution approaches for Covid-19 detection. They found that ResNet50 with SVM for feature selection and classification could provide better performance. Jain et al. [10] proposed a deep learning model with ResNet101 and data augmentation for Covid-19 detection. Castiglioni et al. [16] used CNN for exploring diagnosis of Covid-19 with chest X-ray imagery. Togacar et al. [17] used deep models such as SqueezeNet and MobileNetV2 along with SVM, fuzzy colour and stacking methods.

Minaeea et al. [11] employed deep models such as “ResNet18, ResNet50, SqueezeNet, and DenseNet-121” along with transfer learning for predicting Covid-19. Apostolopoulos et al. [18] proposed a method to extract biomarkers from Covid-19 X-ray imagery using deep learning approach. Brunese et al. [19] explored a methodology based on explainable deep learning for Covid-19 diagnosis. Nigam et al. [12] studied deep models such as “VGG16, DenseNet121, Xception, NASNet, and EfficientNet” for automatic detection of Covid-19. Calderon-Ramirezet al. [13] proposed a deep learning model with semi-supervised learning for automatic Covid-19 detection using X-Rays. Ozturk et al. [20] proposed a CNN-based model with 17 convolutional layers for Covid-19 diagnosis. Tang et al. [14] proposed an ensemble deep learning model for diagnosing Covid-19. Hassantabar et al. [21] proposed a Covid-19 detection method based on CNN model where fractal technique is used for feature extraction. Altan and Karasu [22] proposed a hybrid model by combining chaotic salp swarm algorithm, deep learning and 2D curvelet transform with X-ray images for recognition of Covid-19. Islam et al. [23] proposed a hybrid model known as CNN-LSTM with X-ray imagery. Vinod et al. [24]

Karakanis and Leontidis [25] proposed lightweight deep models based on CNN for disease prediction. Saha et al. [26] proposed a CNN-based model known as EMCNet and combined it with tree-based classifiers for Covid-19 diagnosis. Rasheed et al. [27] used CNN along with data augmentation techniques in order to have better possibilities for Covid-19 detection. They used Principal Component Analysis (PCA) to improve speed of the process. Ouchichaa et al. [28] proposed a deep model known as CVDNet based on CNN while Das et al. [29] used deep transfer learning for leveraging performance in the disease prediction.

Anwar and Zakir [5] used EfficientNet as deep learning architecture for Covid-19 detection using CT scan imagery. Ahmed et al. [6] focused on IoT and CNN model ROI in order to have a learning framework for Covid-19 assessment early. Sharma [7] used ResNet model for classification of Covid-19 cases using CT scan imagery. They used computer vision and machine learning modules from Microsoft Azure. Lalmuanawma et al. [30] explored AI and machine learning method used to detect SARS-CoV-2 pandemic. Vinod et al. [24] proposed a novel deep learning model known as deep Covix-Net for Covid-19 detection. Rehman et al. [31] investigated Covid-19 detection along with the possible adversarial situations.

Saygili [32] used ML and image processing to have a novel methodology for Covid-19 detection early. Dansana et al. [33] used CNN with CT scans for early diagnosis of Covid-19. Rehman et al. [34] explored different deep learning models for Covid-19 detection covering various methodologies. Tartaglione et al. [35] investigated the hurdles of small datasets used for deep learning to detect Covid-19. They proposed a methodology to overcome the small dataset issue. Gupta et al. [36] used several pre-trained deep models to investigate on discrimination of Covid-19 cases from other ones with chest X-ray. Silva et al. [37] used ensemble approach with voting with deep learning models for Covid-19 detection. They used the concept of cross-dataset analysis for effectiveness. The research contributions from [38-47] encapsulate different deep learning models such as CoroNet [38], fusion of CNN and SCM [39], comparative analysis of deep methods [40, 43], data augmentation [41], ML applications [42], deep learning methods [44, 46, 47] and advanced warning methods [45].

State of the art methods that used CT images for Covid-19 detection are found in [1, 2, 5, 7, 8, 24, 32, 34, 37] and [48]. In [1], different deep learning models are used to extract features from lung CT images and then ML classifiers are used for prediction of Covid-19. They found ResNet50 and LightGBM combination could yield highest accuracy. In [2] deep transfer learning algorithm is proposed colour visualization approach to detect Covid-19 faster. In [5] EfficientNet architecture is used for detection of Covid-19. In [7] 200 Covid-19 patients are investigated using ML techniques. Yesar and Ceylan [8] made a comparative study with ML and deep learning methods besides methods-based texture analysis. In [24], a Deep Covix-Net is the model proposed for automated diagnosis of Covid-19. In [32], several ML models are evaluated to be part of computer-aided detection of Covid-19. In [34] deep learning models are investigated and their comparison is made among them in terms of utility. In [37], a voting-based scheme is proposed to have efficient approach for Covid-19 detection. Kassania et al. [48] a ML-based methodology is proposed to detect Covid-19 automatically. From all these CT-scan-based approaches it is found that there is a common drawback. The problem with many existing solutions in COVID-19 detection using CT scan images is that inability to detect the infection when it is in initial stages. As the infection can exist on varied scales, there is need for more comprehensive approach that can ascertain the disease at all scales. Table 1 shows different existing methods that made use of CT scans and X-ray images.

Table 1

Shows the existing models for Covid-19 detection

Image data	References	Methods	Datasets
Chest X-Rays	Kassani et al. [1] Panwar et al. [2] Panwar et al. [3] Jain et al. [4] Ismael and Sengur [9] Jain et al. [10] Minaeea et al. [11] Nigam et al. [12] Calderon-Ramirez et al. [13] Tang et al. [14] Hassantabar et al. [21]	Pre-trained deep CNN models and LightGBM Deep transfer learning algorithm nCovnet Inception V3, Xception, and ResNeXt ResNet50 and SVM ResNet101 and Data augmentation ResNet18, ResNet50, SqueezeNet, and DenseNet-121 VGG16, DenseNet121, Xception, NASNet, and EfficientNet Deep Learning and Mix Match Ensemble deep learning model Fractal technique with CNN	[49, 50] [51–53] [54] [55] [56–58] Collected from online resources [59] Collected from private hospitals in Maharashtra [60–62] [63] [64, 65]
CT Scans	Anwar and Zakir [5] Ahmed et al. [6] Sharma [7] Yasar and Ceylan [8] Vinod et al. [24] Saygili [32] Dansana et al. [33]	EfficientNet IoT and CNN with ROI ResNet ML, DL and texture analysis Deep Covix-Net ML and image processing Decision Tree, Inception V2, VGG-19	[66] [48] Dataset collected from SAL hospital, Ahmedabad, India [67] [68] [69] Open source dataset

Materials and Methods

The CT scans dataset used in this study is collected from [70]. It has the data from 95 COVID-19 patients and 282 normal patients. It has 15,589 COVID-19 CT scan images and 48,260 normal CT scan images. The proposed methodology is outlined in Fig. 1.

Fig. 1

Overview of the proposed system

The Framework

It takes CT scan images of one patient as input and produces the output as COVID-19 or NORMAL. This process is automated to realize a Clinical Decision Support System (CDSS) without the need for human medical expert. The novelty in the proposed system is that it has ROI based approach to filter out CT scan images. Towards this end, an algorithm known as Region of Interest (ROI)-based Image Filtering (ROI-IF) is proposed. This algorithm does image processing to discard samples that do not have region of interest which is nothing but a properly visible lung region.

After elimination of samples that do not have ROI, the selected CT scan images are provided to the deep learning framework known as Automated Deep Learning based COVID-19 Detection Framework (ADL-CDF). It is presented in Fig. 2. Pre-trained deep learning networks are also used to have efficient detection of COVID-19.

Fig. 2

Automated Deep Learning based COVID-19 Detection Framework (ADL-CDF)

After filtering CT scan images of a patient using the proposed Region of Interest (ROI)-based Image Filtering algorithm, the selected images are given to the ResNet50V2 backbone of the pipeline shown in Fig. 2. ResNet50V2 is the residual network is a Convolutional Neural Network (CNN) which 50 layers deep. ResNet is known for its skip connections or shortcuts that lead to jumping over some layers. A single block of RestNet50 is shown in Fig. 3. A ResNet50V2 model has 5 stages and each one is associated with a residual block. Each block contains 3 layers with 1*1 and 3*3 convolutions. Unlike conventional neural networks where each layer feeds into the next layer, in residual block each layer feeds into next layer and also directly into the layers about 2 or 3 hops aware known as identity connections.

Fig. 3

A block (canonical form) of ResNet50V2

As shown in Fig. 3, the layer l-1 is skipped over-activation from l-2. Adding skip connections is meant for addressing two problems such as reduction of accuracy saturation and to get rid of the problem of vanishing gradients. The outcome of ResNet50V2 backbone is given to the multi-scale feature selection layer in the pipeline. This layer encapsulates the proposed algorithm known as Multi-Scale Feature Selection (MSFS). The algorithm produces features at five different scales. Against each feature scale, flattening is carried out for suitable representation of the image. It is followed by the process of classification of each feature scale with dropout regularization followed by 5 dense layers each has 2 neurons and ReLU as activation function. The outcome of dense layer with 10 neurons is concatenated finally softmax activation function is applied to have final classification into COVID-19 | NORMAL. Table 2 shows the notations used in the paper.

Table 2

Notations used in the proposed system

Notation	Description
X_l and X_l+1	Input and output of the l-th unit
F	Residual function
hX_landf(Y_l)	Identity mapping
f	ReLU function
W_l	Set of weights
L	Deeper unit
l	Shallower unit
λ_l	Modulating scalar
F^	Absorbs the scalars into the residual functions

The residual block or residual unit performs computations expressed in Eq. 1 and Eq. 2. Where is the input feature associated with the lth residual block. A set of weights of that block is denoted as . The residual function is denoted as .

With respect to identify mapping, from Eq. 1 and Eq. 2, it is possible to derive Eq. 3.

With recursive approach, for any shallower unit l and deeper unit L, it results in Eq. 4.

When the block propagation’s chain rule, as explored in [71] is considered, we will get Eq. 5 where loss function is denoted as .

When simple modification is made to identity shortcut of residual network, it results in Eq. 6 where modulating scalar is denoted as while f is assumed as identity.

When this formulation is recursively used, it results in the expression in Eq. 7 where the denotes residual function absorbing scalars.

When backpropagation is applied, similar to Eq. 5, its results in Eq. 8.

The first additive term, unlike in Eq. 5, is modulated. This result various kinds of shortcut connected with ResNet50V2.

Image Filtering Algorithm

Lung HRCT scan device produces many CT scan images. If all the images are used, it may not be efficient as it increases processing time and resource utilization. We used the notion of ROI to identify images that have visible lung portion and discard images that do not comply with ROI.

As presented in Algorithm 1, it takes set of chest CT scans of a patient denoted as D and threshold th for inclusion criteria as inputs. It finally produces the selected images after discarding unnecessary scan images. The output is denoted as D’. In Step 3 of the algorithm an ROI is set which is basis for identification of CT scan images that are useful. In the given representative ROI, maximum number of dark pixels and minimum number of dark pixels is counted as in Step 4 and Step 5. From this count a threshold is computed as in Step 6. Step 7 through Step 11, there is an iterative process to analyse each CT scan image and subject it to inclusion criterion and finally add the image to output vector if it satisfies the threshold. The final outcome D’ is used for further processing.

Multi-Scale Feature Selection

Our novelty in this paper is to make use of features at different scales so as to identify even tiny infections of COVID-1 Towards this end, we proposed an algorithm known as Multi-Scale Feature Selection (MSFS) which is based on the notion of ROI of different scales. This mechanism is derived from pyramidal feature hierarchy presented in [72].

As presented in Algorithm 2, it takes Trained ResNet50V2 network denoted as R as input and produces multi-scale features map which contains features of several scales to detect even minute infections of COVID-19. Step 2 and Step 3 initialize scales map and feature map respectively. Step 4 through Step 7, there is an iterative process to extract ROI for different scales that are compatible with trained ResNet50V2. This process results in the creation of a scales map from R. Then from Step 8 through Step 12, there is another iterative process to extract features for all the scales. The features are kept in the map F which is the final outcome of the algorithm. This F is further used in the pipeline of the proposed framework shown in Fig. 2.

Performance Evaluation

Based on confusion matrix, the evaluation of the proposed algorithm is compared with the state of the art. Table 3 shows different metrics used in the evaluation process.

Table 3

Performance metrics used for evaluation

Metric	Formula	Value range	Best value
Specificity	TNTN+FP(9)	[0; 1]	1
Sensitivity	TPTP+FN(10)	[0; 1]	1
Accuracy	TP+TNTP+TN+FP+FN(11)	[0; 1]	1
Precision	TPTP+FP(12)	[0; 1]	1

Based on the confusion matrix presented in Fig. 4, the confusion matrix shows the measures like “true positive (TP), false positive (FP), false negative (FN) and true negative (TN)”. These are determined by comparing prediction result of deep learning models when compared with the ground truth.

Fig. 4

Confusion matrix

From the confusion matrix, different measures are derived. They are known as specificity, sensitivity, accuracy and precision. These measures are used in this paper to evaluate the proposed deep learning framework for COVID-19 detection.

Results and Discussion

This section presents experimental results in terms of Covid-19 detection and performance of the proposed framework named Automated Deep Learning based COVID-19 Detection Framework (ADL-CDF). The dataset is collected from [70] for our empirical study. The environment used for the experiments is as shown in Table 4.

Table 4

Details of execution environment

Item	Description
Coding language	Python
Environment	Google Colab
Operating system	Linux
CPU	Intel Xeon (2 GHz)
GPU	Tesla P100
RAM	12 GB

The experimental results are evaluated using performance metrics presented in Table 4. We made experiments with our deep learning framework ADL-CDF and also ResNet50V2 and Xception. Xception and ResNet50V2 are chosen for comparison (Table 5). The rationale behind this is that both are proven to be good for CT scan-based COVID-19 detection. ADL-CDF, ResNet50V2 and Xception are trained using the parameters presented in Table 6.

Table 5

Hyper parameters used for RestNet50, Xception and the proposed ADL-CDF models

Hyper Parameter	Parameter value
	ResNet50V2	Xception	ADL-CDF
Batch size	14	14	14
Learning rate	1e-4	1e-4	1e-4
Epochs	50	50	50
Loss function	Categorical cross entropy	Categorical cross entropy	Categorical cross entropy
Rotation range	0–360 degrees	0–360 degrees	0–360 degrees
Horizontal/vertical flipping	Yes	Yes	Yes
Optimizer	Nadam	Nadam	Nadam
Zoom range	5%	5%	5%
Width/height shifting	5%	5%	5%

Table 6

Performance comparison of proposed and existing models

Covid-19 Detection Model	Performance (%)
	Sensitivity (Covid –ve)	Sensitivity (Covid + ve)	Specificity (Covid –ve)	Specificity (Covid + v2)	Accuracy (Covid –ve)	Accuracy (Covid + ve)	Overall Accuracy
ResNet50V2	97.49	97.99	97.99	97.49	97.52	97.52	97.52
Xception	96.47	98.02	98.02	96.47	96.55	96.55	96.55
Proposed model	98.70	94.96	94.96	98.70	98.49	98.49	98.49

Figure 5 shows the excerpt of samples taken from dataset. It provides CT scan samples Covid-19 and also non Covid-19 samples.

Fig. 5

An excerpt from the dataset collected from [70]

When patients’ CT scan images are subjected to testing, the proposed model is capable of detecting Covid-19 efficiently. In other words, the model is able to discriminate between normal and affected samples. Figure 6 shows some of the tested samples (before testing) and their corresponding result after testing with visualization of the affected areas. Unlike other lung images like MRI, CT images cannot be used with one of the images. In fact, CT scan provides a set of images that are to be used. Therefore, for Clinical analysis of these medical images, analysing one image is not sufficient. The set of CT scan images is subjected to our algorithm named Region of Interest (ROI)-based Image Filtering (ROI-IF). It finally produces the selected images after discarding unnecessary scan images based on ROI.

Fig. 6

Results of Covid-19 detection

As presented in Fig. 7, the results of experiments with the proposed model are provided in terms of training accuracy. The proposed model is compared with ResNet50V2V2 and Xception. The proposed model shows highest training accuracy against different epochs.

Fig. 7

Training accuracy of the proposed model

As presented in Fig. 8, the results of experiments with the proposed model are provided in terms of validation accuracy. The proposed model is compared with ResNet50V2V2 and Xception. The proposed model shows highest validation accuracy against many epochs.

Fig. 8

Validation accuracy of the proposed model

As presented in Table 6, the performance of the proposed model and the two existing models is provided with different performance metrics.

As presented in Fig. 9, the performance of the proposed model is provided in terms of sensitivity, specificity and accuracy for both Covid-19 cases and non-Covid-19 cases. The results are evaluated by comparing with other deep learning models such as Xception and ResNet50V2. With respect to the sensitivity for Normal case, ResNet50V2 showed 97.49%, Xception 96.47% and the proposed model 98.70%. Similarly, with respect to the sensitivity for Covid-19 case ResNet50V2 showed 97.99%, Xception 98.02% and the proposed model 94.96%. With respect to the specificity for Normal case ResNet50V2 showed 97.99%, Xception 98.02% and the proposed model 94.96%. Similarly, with respect to the specificity of Covid-19 case ResNet50V2 showed 97.49%, Xception 96.47% and the proposed model 98.70%. With respect to the accuracy for Normal case, ResNet50V2 showed 97.52%, Xception 96.55% and the proposed model 98.49%. Similarly, with respect to the accuracy for Covid-19 case, ResNet50V2 showed 97.52%, Xception 96.55% and the proposed model 98.49%. The overall accuracy of ResNet50V2 is 97.52% and Xception is 96.55% while the overall accuracy of the proposed model is 98.49% which is highest. The proposed framework ADL-CDF outperforms both the existing models.

Fig. 9

Performance evaluation of the proposed model against existing

Conclusion and Future Work

In this paper, we proposed a deep learning-based framework known as Automated Deep Learning-based COVID-19 Detection Framework (ADL-CDF). The framework automatically detects Covid-19 provided lung CT scan images. It does not need a human medical expert in diagnosis as it is capable of detecting automatically. The framework is assisted by two algorithms that involve image processing. The first algorithm known as Region of Interest (ROI)-based Image Filtering (ROI-IF) which analyses given input CT scan images of a patient and discards the ones where ROI is missing. This algorithm minimizes time taken for processing besides reducing false positive rate. The second algorithm is known as Multi-Scale Feature Selection (MSFS) algorithm that fits into the deep learning framework’s pipeline to leverage detection performance of the ADL-CDF. We trained three models such as the proposed ADL-CDF, ResNet50V2 and Xception. The proposed framework is evaluated against ResNet50V2 and Xception. Our empirical study revealed that our model outperforms the state of the art. The results revealed that the proposed model can be integrated with Clinical Decision Support System (CDSS) for automatic Covid-19 detection in healthcare units. In future we intend to improve our framework to work with other imaging techniques for automatic Covid-19 detection.