A deep learning semantic segmentation architecture for COVID-19 lesions discovery in limited chest CT datasets.

During the epidemic of COVID-19, Computed Tomography (CT) is used to help in the diagnosis of patients. Most current studies on this subject appear to be focused on broad and private annotated data which are impractical to access from an organization, particularly while radiologists are fighting the coronavirus disease. It is challenging to equate these techniques since they were built on separate datasets, educated on various training sets, and tested using different metrics. In this research, a deep learning semantic segmentation architecture for COVID-19 lesions detection in limited chest CT datasets will be presented. The proposed model architecture consists of the encoder and the decoder components. The encoder component contains three layers of convolution and pooling, while the decoder contains three layers of deconvolutional and upsampling. The dataset consists of 20 CT scans of lungs belongs to 20 patients from two sources of data. The total number of images in the dataset is 3520 CT scans with its labelled images. The dataset is split into 70% for the training phase and 30% for the testing phase. Images of the dataset are passed through the pre-processing phase to be resized and normalized. Five experimental trials are conducted through the research with different images selected for the training and the testing phases for every trial. The proposed model achieves 0.993 in the global accuracy, and 0.987, 0.799, 0.874 for weighted IoU, mean IoU and mean BF score accordingly. The performance metrics such as precision, sensitivity, specificity and F1 score strengthens the obtained results. The proposed model outperforms the related works which use the same dataset in terms of performance and IoU metrics.

INTRODUCTION

Based on the World Health Organization (WHO), the coronavirus (COVID‐19) pandemic puts healthcare services worldwide under immense and growing pressure (WHO) (Ghazy et al., 2020). Coronavirus Disease 2019 (COVID‐19) has become a global pandemic with an unprecedented rate of development and a mechanism of dissemination that is not well known. Governments have begun to focus on new space utilization techniques, psychological distancing, and supplies for medical workers and regular persons (Loey et al., 2021b). This COVID‐19 will induce a respiratory ailment to those that have it, and it is infectious, transmitted by droplets in the air. The infection is most commonly spread through close contact and through the inhalation of the nose and upper respiratory droplets which are delivered when a person coughs or sneezes. The ugly symptoms this infection produces can include coughing, breathing difficulties, and fever (Wang, Wang, et al., 2020). Trouble in breathing is a warning that you might be developing pneumonia which needs immediate health consideration. Efficient screening of contaminated patients is a crucial phase in the battle against COVID‐19 (Vandenberg et al., 2020).

Deep learning has demonstrated immense promise in many real‐life implementations in several different areas (Khalifa, Taha, et al., 2020). All these potential applications include object recognition. This new method of object recognition is based on deep learning algorithms. This type of deep learning has obtained impressive results in identifying an object in images (Loey et al., 2021a). Chest CTs also emerged in clinical practice as an invaluable method for the diagnosis of lung disease in COVID‐19 related individuals (Kwee & Kwee, 2020). When working with lung images, segmentation is necessary to analyse the lung images, for example, to detect disease. COVID‐19 identification is a difficult task that regularly involves taking a gander at patient clinical photos (Perumal et al., 2021). Via deep learning and semantic image segmentation, the main objective of this research is to identify and localize the COVID‐19 contaminated areas in the patient's lungs.

Every area of work involves the development of goals, which helps in solving the cases and problems facing the organization. The field of Artificial Intelligence involves both studying and constructing intelligent entities which include both machine learning and deep learning. Deep neural networks (DNNs) are composed of artificial neurons that are built up in layers of data processing (Loey et al., 2021a) (Khalifa et al., 2021). These networks are structured in a deep architecture referred to as a ‘deep neural network’, a collection of multiple layers, that translate information into decisions. A subtype is CNN, whose known uses are exemplified by images and videos (Khalifa, Loey, et al., 2020). While information interchange at times gets hard to manage, convolutional neural networks (CNNs) have been a subtype that has flourished and found popularity due to astounding breakthroughs in image and video processing (Loey et al., 2020).

Segmentation can be characterized as partitioning the image into homogeneous pixel groups. It can be performed in a highly sophisticated way, down to the level of a single pixel (Rogowska, 2009). Semantic pixel‐level segmentation is the task of clustering image parts together that belong to the same object class (Wang et al., 2018). Semantic image segmentation is the method of converting an input image into a raster so that any pixel is labelled to a certain significance entity type from a predefined class nomenclature (Kang et al., 2020). The appearance of items is learned using the labelled pictures. Methods typically fuse two sub‐tasks: an identification component that labels pixels with the class, and a regularization part that enforces other pixels in the neighbourhood to be in the same class. The recognizing aspect depends on the pixel or patch stage of the local appearance (Li et al., 2020). Various elements, including texture, colour statistics and SIFT, are used for defining the local appearance.

The main contributions of this research are: (1) proposing a new model for the detection of COVID‐19 lesions in CT Lung images, (2) measuring the accuracy and the performance metrics of the proposed model, and (3) comparing the proposed model with other related works according to predefined metrics. One of the technical novelties for the presented work is the architecture of the proposed model. This work contains the following sections. The second section addresses past writings on relevant research. Section 3 discusses the dataset. Section 4 the proposed model for detecting COVID‐19 infected regions. The article discusses the outcomes of the experiment and ends with potential work.

RELATED WORK

Semantic Segmentation refers to assigning a class label to each pixel in the given image. This is done through Object identification and class labelling to each pixel in the given image (Garcia‐Garcia et al., 2018). Learning and classifying models for vision and image processing have been applied in artificial intelligence, robotics, and computer vision, and have been used with effectiveness to identify objects and control segmentation in images. The semantic segmentation method is deeply used in most segmentation problems in general. It can be used in the autonomous car driving by segmenting the roads, signs, cars, and pedestrians. Also, it is heavily used in the medical domain such as tumour detection in CT and X‐rays. To gain a better understanding of infection rates and to track and rapidly detect infections, the AI's capabilities are being leveraged to address the Covid‐19 pandemic (Bhattacharya et al., 2021; González‐Crespo et al., 2020), including the detection of COVID‐19 in medical chest CT scan (Lassau et al., 2021). This section focuses on recent papers in terms of applying representative works related to semantic segmentation based on deep learning for CT image diagnosis.

In Oulefki et al. (2020), the writers demonstrated an algorithm for automatic COVID‐19 Lung infection segmentation using chest CT images. Depending on 3D U‐Net architecture, the proposed tool is capable of detecting irregular regions with low contrast between lesions and healthy tissues with an accuracy of 0.98 and a precision of 0.73. The authors in (Yang et al., 2020) developed and tested a 3D deep learning model which is called CODID‐SegNet, for using the COVID‐19 segmentation method on CT pictures of the chest. The planned network would consist of ASPs that contain features that are good at highlighting the COVID‐19 boundaries and positions. Yang, He, et al. (2020)) introduced the MultiResUNet model to slice COVID‐19 from CT pictures. They adjust certain convolutions to render activation functions simpler, incorporate regularization and dropout to manage to overfit. The average precision score for Right, left, disease, and background lung is 84.66, 87.04, 72.14 and 99.23 respectively. Outcomes of the segmentation study are analysed, and the required outcomes are achieved. It is critical for medical personnel to identify rapidly where the infection is located.

In Wang, Zhang, et al. (2020) introduced four transfer learning models based on U‐Net. Three freely accessible non‐COVID19 datasets were used in their research. The suggested hybrid‐encoder learning approach includes a dedicating encoder for the COVID 19 infection function and general lung lesion features. Results show that a Hybrid‐encoder state‐of‐the‐art neural network model achieves accuracy, sensitivity, F1‐score of 0.994, 0.682, 0.707, respectively, when trained with only minimal data. In Ma et al. (2020)), the authors create a benchmark dataset for COVID‐19 segmentation. Authors gave three datasets to segment lung and infection from COVID‐19. Also, the authors include the guidance of standard preparation, measurement measures, and the corresponding code. Through training on 40 baselines, researchers can save resources by preventing needing to train their techniques and can concentrate on designing their approaches. In Voulodimos et al. (2020)) introduced a segmentation model of pneumonia based on CT images. The proposed model based on U‐Nets as convolutional neural architecture based on a real dataset. The analysis revealed that Completely Convolutional Neural Networks correctly segment the entity groups in the specified dataset.

In Yang, Li, et al. (2020), the authors create a new open benchmark dataset of COVID‐19. The research data collection includes 349 CT photographs from 216 patients and 463 non‐CT pictures. They show the reliability of the dataset for the creation of an AI‐based diagnostic model of COVID‐19. They create a machine learning algorithm that achieves an accuracy score of .89, on a test collection of initial CT photos from hospitals. In Fan et al. (2020)) proposed COVID‐19 segmentation model for CT pictures. The proposed model classifies infected regions from chest CT images. A parallel partial decoder that includes high‐level features as features is used to generate a global map. They present a semi‐supervised segmentation system that is focused on a random propagation technique, which only uses a few labelled training pictures and utilizes heavily uncategorized data. There is tremendous scope for the method to be incorporated in the diagnosis of COVID‐19.

DATASET

Data were collected in this study from (Jun et al., n.d.). These CT scans are numbered with numbers from one to twenty. Left lung, right lung, and multiple‐sided disease were assessed by two radiologists who collaborated, with the help of an expert, a chest radiologist, to be present in this case. The dataset is considered a heterogeneous one as it includes both healthy and COVID‐19 and CT scans from two sources. The first source includes 10 patients from a single source of CT scans, while the second source includes 10 patients' CT scans from the radiopaedia website (Radiopaedia.Org, the Wiki‐Based Collaborative Radiology Resource, n.d.). The first source consists of 2581 images of CT scans with its labelled images. The second source (radiopaedia) consists of 939 images of CT scans with its labelled images. The total number of images from the two sources is 3520 CT scans images with its labelled images. Figure 1 illustrates samples from the two sources for the original dataset.

FIGURE 1

Dataset sample images from classified to it is the original source

THE PROPOSED MODEL ARCHITECTURE

In this part, a detailed overview of the introduced architecture for semantic segmentation will be presented. The proposed architecture flowchart is presented in Figure 2. The model passes through three phases. The first phase is the pre‐processing, while the second phase is the training. The third phase is the testing phase. In the pre‐processing phase, all images in the dataset were resized to 128 × 128 pixels to reduce the processing time in the training phase. Moreover, all the pictures were normalized to remove any anomalies in the dataset images. In the training phase, the images were split into two parts. The first part for training purposes with 70% of the original data, While the second part was 30% for testing purposes. The training part will be fed into the proposed model for semantic segmentation which is presented in Figure 3 and will be discussed next section. The testing phase is used for testing the accuracy of the introduced architecture over the test data.

FIGURE 2

The proposed model flowchart

FIGURE 3

The architecture of the proposed semantic segmentation model

In this semantic segmentation, the architecture has two separate components as presented in Figure 3. The first component is the encoder with three stages (). Every stage consist of one convolutional layer, one ReLU layer, and one max‐pooling layer with total nine layers. Encoder which is a down sampling operation is aimed at capturing semantic or context information. The input is fed up with the encoder part. The input image is an image of CT for the chest with size 128 × 128 pixels and normalized. The encoder's first convolutional layer has 64 filters followed by ReLU and max‐pooling layer. The output of the first max‐pooling layer is then fed into the second convolutional layer (contains 128 filters) and then it is followed by ReLU and the second max‐pooling layer. The output of the previous max pooling is fed into the third convolutional layer which contains 256 filters then followed by ReLU and max‐pooling layer.

After extracting the compressed features, the decoder returns the features to their original scale, thereby reducing the sum of knowledge loss due to compression. Decoder which is an up‐sampling operation is aimed at recovering spatial information. Common decoders include bilinear interpolation, deconvolution and dense up sampling convolution. In the proposed model, the deconvolution option has been used. The decoder function is the opposite function of the encoder. The second component is the encoder within 3 stages (). The layer of the encoder component consists of the deconvolutional layers and the SoftMax layer. The first deconvolutional layer in the encoder consists of 256 filters, while the second deconvolutional layer consists of 128 filters and the third deconvolutional layer consists of 64 filters. The SoftMax layer is the last in the decoder component followed by the pixel classification layer. The output of the decoder layer is a mask image of the COVID‐19 lesions in the lung as presented in Figure 2.

EXPERIMENTAL RESULTS

A commercial software program (MATLAB) was used to develop the proposed semantic segmentation model. The execution was particular to run in a central processing unit (CPU). All tests were conducted on a computer server with an Intel Xeon CPU, about 100 GB of RAM with Titan X GPU.

The experimental trial setups were designed to capture the accuracy of the quantification along with the planned model's performance specifications. The following setups were adopted during the experimental trials:

Two sources of data as illustrated in the dataset section and will be noted as DS1 for dataset source 1 and DS2 for dataset source 2. Moreover, DS3 is a merged source of DS1 and DS2.

The training phase will be experimented with 70% of the data, while 30% will be used in the testing phase.

The number of test images for DS1 is 774, for DS2 is 281 and for DS3 is 1055 images.

Five trials were conducted with different images randomly selected for every trial for the training/testing phases.

Along with the following parameters set up for the proposed semantic segmentation deep learning model:

Batch size: 64

Momentum: 0.9

Number of epochs: 300

Learning rate: 0.001

Optimizer: Adam

The following accuracy and performance metrics will be measured during the experimental trials and they are (Gaspari et al., 2020) (Goutte & Gaussier, 2005): where is the counter of correctly identified pixels in the class, is the counter of miss‐labelled of rest of pixels of the classes, is the counter of correctly identified pixels of the rest of the classes, and is the counter of miss‐labelled pixels in the class. Sensitivity, Specificity, Precision, Accuracy and F1 Score (Goutte & Gaussier, 2005) are introduced from Equation (2) to Equation (5).

Accuracy: the ratio between the total pixels correctly identified of a class over the number of pixels of that label as presented in Equation (1).

where C is the predicted mask and G is the ground truth mask.

Global Accuracy: the ratio between the total pixels of one class overall number of pixels.

Mean Accuracy: the average of the Accuracy values of all the classes

IoU: Intersection over‐union is an estimation metric that is used to determine the object detection precision on a given dataset as presented in Equation (6).

Mean IoU: IOU is a commonly used measure for semantic segmentation, which first counts the IOUs and then computes their average.

Weighted IoU: The individual Class IOU averages were calculated using the total number of pixels as a weight.

BF Score: acronym for Boundary F1 contour matching score, it indicates how close the predicted boundary of a class matches the ground truth boundary.

Mean BF Score: the average of the BFScore of all the classes.

Testing performance outcomes

According to the previous section metrics, testing performance is considered the most common metric in deep learning evaluation metrics. Those metrics will be measured for the different dataset sources DS1, DS2 and DS3. Table 1 presents the global accuracy, mean accuracy, sensitivity, specificity, precision, and F1 score for the different dataset sources.

TABLE 1

Accuracy results with its performance metrics

Dataset source	Global accuracy	Mean accuracy	Precision	Sensitivity	Specificity	F1 score
DS1	0.9932	0.9872	0.9937	0.9812	0.9817	0.9874
DS2	0.9842	0.9775	0.9842	0.9706	0.9706	0.9774
DS3	0.9930	0.9717	0.9932	0.9499	0.9499	0.9711

Table 1 illustrates that for DS1, the results for all metrics super pass the achieved results for DS2 and DS3. One of the reasons for better accuracy for DS1 due to the homogeneity of the sources according to the image's quality and its labelled masks. The DS2 is captured from the radiopaedia website, which by its definition contains images of different qualities. The achieved results are not the highest, as it achieves 0.9842 in global accuracy and 0.9775 in mean accuracy but still a promising result. For the combined dataset source (DS3). The proposed model achieved 0.9930 in global accuracy and close to DS1, and 0.9717 in mean accuracy.

Table 1 showed that the achieved results for DS1 were the highest ones with 0.9932, 0.9872, 0.9937, 0.9812, 0.9817 and 0.9874 for global accuracy, mean accuracy, precision, sensitivity, specificity, and F1 score accordingly. The achieved results due to the homogeneity and the quality of the images from dataset Source 1.

The achieved result in Table 1 was calculated from the confusion matrix for DS1, DS2 and DS3. Figure 4 presents the confusion matrices for the different dataset sources.

FIGURE 4

Confusion matrix for (a) DS1, (b) DS2, (c) DS3

One of the most common accuracies in deep learning is the accuracy for every class. Table 2 presents the accuracy for every class for the proposed model.

TABLE 2

Testing accuracy for the different class for dataset sources

Testing accuracy	DS1	DS2	DS3
COVID‐19 Class	0.9812	0.9706	0.9499
Background Class	0.9933	0.9844	0.9935

Table 2 illustrates that in the background class accuracy, DS3 achieves the highest accuracy with 0.9935. For COVID‐19 class accuracy, DS1 achieves the highest accuracy possible with 0.9812 due to the quality and homogeneity of the images in DS1. For background pixels, it is all black in all dataset sources and more data means higher accuracy in deep learning, and DS3 contains the highest number of images among the other dataset sources.

Weighted IoU, mean IoU and BF score outcomes

Weighted IoU, Mean IoU and Mean BF Score is the most commonly used parameters in object and region of interest detection. Table 3 presented the IoU metrics and Mean BF score for the proposed model for different dataset sources.

TABLE 3

IoU metrics and mean BF score for different dataset sources

Dataset source	Weighted IoU	Mean IoU	Mean BF score
DS1	0.9892	0.7807	0.8591
DS2	0.9759	0.7468	0.8042
DS3	0.9886	0.7990	0.8746

Table 3 shows that for DS1 and DS3 the weighted IoU is very close to each other with 0.9892 for DS1 and 0.9886 for DS3. In the mean IoU metric and the mean BF score, DS3 achieved the highest accuracy possible with 0.8746. The achieved results were due to the large number of images in DS3.

Figure 5 illustrates the mean IoU metric for the dataset sources. The figure shows the number of images for mean IoU achieved accuracy. For instance, in DS1, there are 300 images from the test images achieved mean IoU = 1. In DS2, there are 80 images from the test images achieved mean IoU = 1. In DS3, there are 160 images from the test images achieved mean IoU = 0.8.

FIGURE 5

Mean IoU for (a) DS1, (b) DS2, and (c) DS3

Results discussion and comparison with related works

From the previous subsection, we can deduce that the homogeneity of the images and their number affects the obtained results. The proposed architecture over DS3 achieved the highest metrics in Section 5.2 Weighted IoU, Mean IoU and Mean BF Score with 0.9886, 0.7990, 0.874, accordingly. Those metrics concerns with the model accuracy in detecting and localizing the COVID‐19 Lesions in CT lungs images. While in testing accuracy for classification of the pixels whether is to belong to COVID‐19 or background class the proposed model achieves the highest accuracy possible over DS1 and close result to DS3 according to global accuracy metric with 0.9932 in DS1 and 0.9930 in DS3. So, the research decision was to select the proposed model for DS3 as the main results of this research with 0.9930 global accuracies, 0.9886, 0.7990, 0.874 for Weighted IoU, Mean IoU and Mean BF Score.

Figure 6 illustrated the final image segmentation result for the proposed model. The images in the (d) label present the predicted segmentation of COVID‐19 lesions in CT lungs. The difference between the predicted mask and the original mask is considered small as it achieved according to the previous section around 0.9930 accuracies with weighted IoU equal to 09886.

FIGURE 6

The final image segmentation result for the proposed model was (a) the original CT image, (b) the original mask image, (c) the original mask image over the original CT image, and (d) produced mask of the proposed model over the original CT image

Table 4 presents a comparison of results with related works that used the same dataset. Table 4 shows clearly that the proposed model outperformed the related works in terms of accuracy, f1 score, precision, sensitivity and IoU. None of the related works reported the IoU instead they (Wang, Zhang, et al., 2020) (Ma et al., 2020) report Dice Similarity Coefficient which is very similar to IoU. The Dice Similarity Coefficient tends to measure the average performance, while the IoU scores the worst‐case performance.

TABLE 4

Comparison results with related works

	Description	ACC	F1 score	Precision	Sensitivity	Dice similarity coefficient/IoU
(Voulodimos et al., 2020)	FCN and Unet	0.97	0.65	0.95	0.55	–
(Wang, Wang, et al., 2020; Wang, Zhang, et al., 2020)	Pretrained Multi‐Lesions	0.99	0.71	–	0.68	0.707
(Ma et al., 2020)	40 pre‐trained baseline models	–	–	–	–	0.700
(Yang, He, et al., 2020; Yang, Li, et al., 2020)	contrastive self‐supervised learning with transfer learning	0.89	0.89	–	–	–
(Fan et al., 2020)	Deep Network (Inf‐Net)	–	–	0.50	0.87	0.59
(Fan et al., 2020)	Deep Network (Semi Inf‐Net)	–	–	0.51	0.86	0.59
Proposed Model		0.99	0.97	0.99	0.95	0.799

CONCLUSION

COVID‐19 pandemic is one of the devastating pandemics throughout modern history. Its effects did not reflect only on human lives but also on world economics, global trading and consumption patterns. Advances in the computer science field and in particular in deep learning helped in improving the medical field's precision and speed. In this article, a deep learning semantic segmentation model for COVID‐19 lesions detection in a limited chest CT dataset was presented. The proposed learning approach integrated functionality from non‐COVID19 databases successfully and achieves substantial progress over other learning solutions. Studies showed that trained with a minimal dataset, proposed model strategy focused on pre‐trained model achieves a mean Sensitivity, F1‐score, and Accuracy of 0.95, 0.97 and 0.99, respectively, with improved generalization and lower overfitting threats for segmenting COVID‐19 infection. The research offers information regarding the effects of transmitting expertise from non‐COVID19 lung lesions in the light of learning from several lung lesion datasets, contributing to successful models. These outcomes showed that transfer learning from other medical activities for CT image segmentation is feasible, and this can be applied to other tasks. One of the possible future works is the apply the proposed model in others segmentation application domains such as brain CT scans to measure its performance. Also, building a deeper semantic model with more layers and test its performance against the proposed model.

CONFLICT OF INTEREST

On behalf of all authors, the corresponding author states that there is no conflict of interest.