Review and classification of AI-enabled COVID-19 CT imaging models based on computer vision tasks.

This article presents a systematic overview of artificial intelligence (AI) and computer vision strategies for diagnosing the coronavirus disease of 2019 (COVID-19) using computerized tomography (CT) medical images. We analyzed the previous review works and found that all of them ignored classifying and categorizing COVID-19 literature based on computer vision tasks, such as classification, segmentation, and detection. Most of the COVID-19 CT diagnosis methods comprehensively use segmentation and classification tasks. Moreover, most of the review articles are diverse and cover CT as well as X-ray images. Therefore, we focused on the COVID-19 diagnostic methods based on CT images. Well-known search engines and databases such as Google, Google Scholar, Kaggle, Baidu, IEEE Xplore, Web of Science, PubMed, ScienceDirect, and Scopus were utilized to collect relevant studies. After deep analysis, we collected 114 studies and reported highly enriched information for each selected research. According to our analysis, AI and computer vision have substantial potential for rapid COVID-19 diagnosis as they could significantly assist in automating the diagnosis process. Accurate and efficient models will have real-time clinical implications, though further research is still required. Categorization of literature based on computer vision tasks could be helpful for future research; therefore, this review article will provide a good foundation for conducting such research.

Introduction

The global spread of the coronavirus disease of 2019 (COVID-19) pandemic has led to the necessity of developing machine-based tools to detect the disease in diagnostic imagery. The use of these tools is imperative to contain severe acute respiratory syndrome coronavirus 2 (SARS-CoV-2; the virus responsible for COVID-19), screen out large numbers of suspected and confirmed cases, and ease patient management for hospitals. Effective COVID-19 diagnosis depends on tests done in laboratories, such as reverse transcription-polymerase chain reaction (RT-PCR), which is currently considered to be the gold standard [[1], [2], [3], [4], [5], [6], [7]]. Such laboratory procedures are time-consuming, have low sensitivity [[8], [9], [10], [11], [12]], and can lead to a significant number of false-negative results [[13], [14], [15]]. Furthermore, the lack of equipment and stringent requirements for testing bound the prompt and accurate screening of potentially infected subjects [[16], [17], [18]]. Thus, non-laboratory assessments, such as computer-assisted imagery analysis of chest radiographs (X-ray) or computed tomographic (CT) scans, have been introduced to inspect the lung regions to diagnose COVID-19. Contrary to the conventional X-ray approach (which does not provide a significant amount of detail), chest CT scan technology can provide a more detailed view of the lung, soft tissue, and blood vessels [19].

Therefore, artificial intelligence's (AI) deep learning CT imaging models have been developed and deployed expansively to sense infected or suspected patients [20]. So far, various deep learning approaches [16,[21], [22], [23], [24], [25], [26], [27]] have been proposed to aid in the fight against the COVID-19 pandemic, though their development is still in its relative infancy. These techniques are just a few of the developed technologies we mentioned here. A typical COVID-19 AI-based diagnostic framework is depicted in Fig. 1 .

Fig. 1

AI-based COVID-19 diagnostic pipeline.

Conversely, the research community is continually seeking to develop existing diagnostic models by improving their accuracy and efficacy and has extensively relied on previously developed simulations to identify coronavirus pneumonia. Hence, it is essential to frame and categorize some of the most important models that could lead to further COVID-19 research expansion. Although the COVID-19 CT literature is rather extensive, it is relatively difficult to summarize such a large amount of research literature to provide a basis for further studies. Prior to our review, several review articles examined many computer-aided detection (CAD) models and studies related to COVID-19 identification.

Each review article has different perspectives. For example, Samuel et al. [28] provided an initial review covering the machine learning and AI-based methods applied for COVID-19 treatment, medication, screening, and prediction. Nguyen et al. [29] identified several zones—such as medical image processing, data analytics, text mining, natural language processing, the Internet of Things (IoT), computational biology, and medicine—where AI could play a more effective role. Hussain et al. [30] summarized the state-of-the-art AI applications into the hospital management and administration and further outlined and classified various AI techniques for clinical data analysis, such as neural systems, classical SVM, and edge computing.

Ozsahin et al. [31] categorized 30 studies based on the classification tasks. The selected studies were grouped into different classes such as COVID-19/normal, COVID-19/non-COVID-19, COVID-19/non-COVID-19 pneumonia, and severity. Shao et al. [32] investigated the sensitivity and utility of chest CT scans based on 20 articles for diagnosing COVID-19 and its potential application to surgical settings. The study acknowledged and addressed the interrogation of CT's sensitivity for the diagnosis of symptomatic and asymptomatic COVID-19-positive patients. According to their reported outcomes, chest CT sensitivity can range from 57% to 100% for symptomatic and 46%–100% for asymptomatic COVID-19 patients, whereas that of RT-PCR ranged from 39% to 89%. Recently, Islam et al. [33] presented a taxonomy of deep learning techniques by targeting CT scans and X-ray modalities. Their review highlighted the data partitioning techniques, various performance measures, and well-known datasets developed for COVID-19 diagnosis.

Readers can acquire substantial knowledge by examining these studies. However, the available studies do not provide a categorical procedure to classify the available literature based on computer vision tasks; thus, critical technical divisions are left out. For example, in computer vision, classification, segmentation, and detection are considered standard and significant tasks. In the classification task, the required job depends on the labels. On the other hand, detection can be achieved by marking a region of interest with a bounding box or localizing an area of interest. Segmentation is treated differently because it requires pixel-wise delineation of the desired object or region.

Our review article aims to present a realistic review of content based on computer vision tasks and catalog the responsive systems (empowered by AI) developed for COVID-19. A flow diagram of our review is depicted in Fig. 2 , which shows the major parts and a collection of selective information for each category based on the collected studies and models. Our review article's contributions include:

Fig. 2

Flow diagram of the proposed study.

1: A review and categorization of COVID-19 CT-based diagnosis methods in terms of computer vision tasks (i.e., classification, segmentation, and detection/localization).

2: A summary of the most important aspects of the selected studies’ frameworks, findings, and data structure.

3: A comprehensive list of open-source datasets and resources for COVID-19 CT research. Critical parameters and information are described and presented for each dataset/database.

For this purpose, Google Scholar, Baidu, and five reliable databases—IEEE Xplore, Web of Science, PubMed, ScienceDirect, Kaggle, and Scopus—were used to obtain pertinent studies on the given topics. The entire literature search process is shown in Fig. 3 (a), with approximately 500 searches for datasets, 200 searches for classification methods, 200 searches for segmentation-based approaches, and 300 searches for detection-based methods. Using various filtering and scanning schemes, 30 classification-based articles, 29 segmentation-based articles, 25 detection-based articles, and 29 datasets were chosen from roughly 1200 searches, as shown in Fig. 3 (b). Most of the collected literature was selected according to their citations.

Fig. 3

Literature selection and browsing.

The selected studies were analyzed with respect to their abstracts, methodologies, datasets, results, and conclusions. Moreover, most of the research articles were chosen based on citations to cover the baseline works. To the best of our knowledge, we are the first to categorize and organize COVID-19 CT-based diagnosis techniques based on the three basic computer vision tasks (classification, segmentation, and detection). The rest of the article is organized as follows: COVID-19 CT classification-based approaches are covered in Section 2, segmentation is covered in Section 3, and detection-localization-based methods are covered in Section 4. Section 5 offers descriptions of and information on the datasets, and Section 6 includes a discussion of the findings and future research prospects.

Classification-based methods

Image classification is the process of predicting a particular class, or mark, for anything identified by a collection of data points. Such classification is a subset of the classification problem in which a mark is assigned to an entire image. The marked image could be further classified as an object or entity that belongs to a specific category. With the rise of COVID-19 pneumonia, many researchers have taken advantage of CT imaging features and developed AI-based systems [19].

Most of the developed COVID-19 diagnosing approaches search for specific CT scan patterns and classify them as COVID, non-COVID, COVID-positive, COVID-negative, community-acquired pneumonia, bacterial pneumonia, other pneumonia, or healthy, normal, abnormal, healthy, infected, non-infected, and so on. In this section, we collected the eligible studies/models based on the classification tasks and arranged them according to the above classes. For instance, a baseline study [34] used the pre-trained DenseNet to classify cases as COVID or non-COVID. The authors adopted transfer learning and data augmentation for training new image data. The study uses a dataset that was made publicly available. Xuehai et al. [35] developed a model based on the Self-Trans approach. The authors anticipated self-supervised learning with a transfer learning strategy to reduce the risk of overfitting to learn dominant and unbiased features—the proposed framework classified chest CTs as COVID (+ve) and COVID (−ve). Moreover, the authors created a publicly available dataset containing hundreds of positive COVID-19 CT scans.

Aayush et al. [36] established a model to classify COVID-positive and COVID-negative cases. The proposed model utilized pre-trained DenseNet-201 [37] for the classification task. The authors' results show that DenseNet-201 performs better than VGG [38], Inception ResNetX [39,40], ResNet 152V2 [41], and other models. Wu et al. [42] extracted lung regions from CT slices by using a threshold segmentation method and further established a deep learning model for the classification of COVID (+ve) and COVID (n). The method used axial, coronal, and sagittal views of CT images to gather more information. Wang et al. [43] established a transfer learning neural network–based inception network to classify COVID-19-positive/-negative cases. They randomly selected regions of interest (ROIs) of the lung and trained their model to extract important features. The model's final phase made predictions by taking advantage of a fully connected network. Mishra et al. [44] proposed a system that combined the predictions of each of the individual deep convolutional neural network (CNN) models for improving their predictive performance. The proposed framework better overcomes false predictions compared to the single model and predicted COVID-19-positive/-negative classes.

Shah et al. [45] introduced a deep learning model termed CTnet-10 to classify COVID-19-positive/-negative cases. They found that VGG-19 has the highest accuracy, and their model performed well in terms of training convergence and inference time. Liu et al. [46] proposed a lesion-attention deep neural network (LA-DNN) to predict COVID-19-positive and -negative cases. The proposed technique made multi-label predictions based on five lesions of COVID if the CT was positive. The authors claimed that their model could achieve clinical standards and is capable of being applied in practice. Ning et al. [47] developed a system based on VGG-16 and used it to discriminate between negative, mild, and severe cases. They used lung extraction from CT and clinical features data to train two systems. The final output predictions were achieved through a penalized logistic regression algorithm. Pathak et al. [48] also developed a model based on an Image-Net pre-trained ResNet-32 [41] variant to classify cases as COVID-19 positive/negative. The results of their proposed method showed that their transfer learning–based model could achieve a better classification accuracy than the models that relied on supervised learning. Aniello et al. [49] used ADECO-CNN to classify cases as COVID + ve/−ve. A normalization method was utilized to eliminate image noise and improve image quality. Compared with other models (such as GoogleNet [40], VGG19, and ResNet), the method achieved a high accuracy of 99%. Dilbag et al. [50] assembled a deep transfer learning model based on densely connected convolutional networks (DCCNs), ResNet152V2, and VGG16 to classify the suspected subjects as COVID-19 (+), tuberculosis, pneumonia, or healthy.

Several models have also been developed to distinguish between COVID-19 pneumonia and other pneumonia types. Xu al [51]. classified influenza-A viral pneumonia (IAVP) and irrelevant infection from CTs of 618 patients. The proposed method used 3D ResNet-18 for segmenting the infectious areas and the location attention model for the classification task. In the final stage of their proposed algorithm, the authors used a noisy-OR Bayesian function to predict the infection type and overall confidence score for each CT case. To differentiate COVID-19 from community-acquired pneumonia (CAP), one system [52] used a dual-sampling attention network and VB-Net toolkit segmentation [53] for pneumonia infection regions to ensure that decisions were made based on infected areas. Wang et al. [54] proposed a novel multi-task prior-attention residual learning model to screen out COVID-19 and identify pneumonia types between COVID-19 and interstitial lung disease (ILD). The proposed model coupled two 3D-ResNets into a single model to perform the mentioned tasks. Polsinelli et al. [55] proposed a SqueezeNet model [56] to classify COVID-19, CAP, and healthy cases from CT images. The framework has a strong classification ability and is performed rapidly on a medium-speed computer without GPU acceleration. Perumal et al. [57] classified COVID-19 CT scans into CAP and normal CT images. For improved data analysis, the suggested system was compared against a variety of machine learning and deep learning classifiers. The proposed model achieved an accuracy of 96.69%, sensitivity of 96%, and specificity of 98%.

The method proposed by Yan et al. [58] takes advantage of multi-scale spatial pyramid (MSSP) decomposition [59] to classify COVID-19 and common pneumonia. The outputs of three CNNs were concatenated for better classification, and upon comparing them with radiologists’ opinions, the model achieved a better performance. Matsuyama et al. [60] proposed a ResNet-50-based model to classify COVID-19/non-COVID-19 pneumonia. They found that using the wavelet coefficients of the CT images as inputs for the fine-tuned CNN was much better than using the pixel values of the original image. Hu et al. [61] used a CNN with ShuffleNet V2 [62,63] as a backbone to efficiently distinguish COVID-19 patients from those non-infected or infected by other pneumonia types (bacterial pneumonia or SARS). Ibrahim et al. [64] introduced a VGG-19-based method to classify COVID-19, pneumonia, lung cancer, and normal cases. The model showed that the VGG-CNN has better predictive ability.

Rahimzadeh's [65] proposed network used ResNet50V2 as its backbone and classified the input CTs as COVID-19 or normal cases. The proposed method applied a feature pyramid network (FPN) [66] to investigate various resolutions of the input image, which remarkably increased the classification performance. Chen et al. [67] classified the input CTs as COVID-19 or healthy cases based on a few CT samples used for training purposes. Their proposed model used a self-supervised learning strategy where the pre-trained encoder can effectively capture the unknown image features. Shuyi et al. [68] developed a DenseNet model to classify COVID-19 patients and healthy persons. They found that the model performance is better than that of radiologists in finding the particularly small details of some lesions with the use of class activation maps [69]. Thus, the developed model can reduce the burden on radiologists. Alshazly et al. [70] proposed COVID-Nets to classify COVID-19 infections, non-COVID-19 viral infections, and healthy persons. The suggested model has two sub-systems: A (ResNet) and B (DenseNet). Images were provided as input to both systems, which determined the diagnosis results. The results of both systems were further fused and compared with the results of the separate systems; the joint decision-making results were better than those of the sub-systems. Zhu et al. [71] recycled the pre-trained ResNet-50, a feature extractor, and a classifier. The model was fine-tuned on the COVID-19 dataset using pre-trained weights from the ImageNet dataset to avoid overfitting, ultimately differentiating between COVID-19 and non-COVID-19 pneumonia. Tripti et al. [72] established a generative adversarial network (GAN) [84] and ResNet50-based model to classify COVID-19 and non-COVID-19 cases. They used a whale optimization algorithm (WOA) [85] to optimize the GAN parameters to generate more CT images and finally feed them into ResNet-50 for diagnosis. Bai et al. [22] classified CT scans into COVID-19 and non-COVID-19 by developing and incorporating EfficientNet into their designed architecture. Initially, the lungs were extracted to eliminate the non-lung areas from the CTs, and then each slice was stacked into three channels as the input to EfficientNet-B4, which completed the classification task. Next, Zhenxing et al. [78] proposed a model to diagnose and classify COVID-19 and non-COVID-19 cases. The proposed model evaluated COVID-19 severity by targeting 3D CT images and clinical symptom information. Sertan et al. [83] further proposed a model to classify the input chest CT volumes into COVID-19 and normal CT volumes. The suggested AI approach used the ResNet-50 architecture for COVID-19 prediction. Table 1 provides detailed selective information related to classification-based methods.

Table 1

COVID-19 classification methods and their selective information.

Source/Author	Performed Tasks	No. of CT Scans/Images/Slices/Patients	Framework/Approach	Classes	Performance
Zhao et al. [34]	COVID/non-COVID Classification	746 CT images	Pre-trained DenseNet	1. COVID-192. Non-COVID-19	On Test SetF1-score: 0.90, AUC: 0.98, ACC: 0.89
Xuehai et al. [35]	Classification of COVID +ve and COVID −ve	349 positive CT scans397 negative CT scans	Train DenseNet-169 by a method called Self-Trans	1. COVID-19 positive2. COVID-19 negative	On Test SetACC: 0.86, F1-score: 0.85, AUC: 0.94
Aayush et al. [36]	Classification of COVID +ve and COVID −ve	2492 CT scans, 68% for training, 17% for validation 15% for test	Pre-trained DenseNet201	1. COVID-19 positive2. COVID-19 negative	On Test SetAUC: 0.97, ACC: 0.998, SPE: 0.992F1-score: 0.998, Recall: 0.997, Prec: 0.999
Wu et al. [42]	Classification of COVID +ve and COVID −ve	368 CT scans of COVID-19 patients and 127 CT scans of patients of other pneumonia	Segmentation and ResNet-50	1. COVID-19 positive2. COVID-19 negative	On Test SetACC: 0.76, AUC: 0.819, SPE: 0.615, SEN: 0.811
Wang [43]	Classification of COVID +ve and COVID −ve	453 COVID CT images	Inception network	1. COVID-19 positive2. COVID-19 negative	On Test SetACC: 0.829, AUC: 0.90, SEN: 0.81, SPE: 0.84, F1-score: 0.77
Mishra et al. [44]	Classification of COVID +ve and COVID −ve	360 +ve scans397 –ve scans	VGG16, ResNet50, InceptionV3, ResNet-50, DenseNet121, DenseNet201	1. COVID-19 positive2. COVID-19 negative	On Test SetACC: 0.883, AUC: 0.883, F1-score: 0.867
Shah et al. [45]	Classification of COVID +ve and COVID −ve	349 COVID CT scans and 463 non-COVID CT scans	CTnet-10	1. COVID-19 positive2. COVID-19 negative	On Test SetACC: 0.821
Liu et al. [46]	Classification of COVID +ve and COVID −ve	Total 1224 patients564 COVID +ve CT scans660 COVID −ve CT scans	LA-DNN based VGG16	1. COVID-19 positive2. COVID-19 negative3. Csld, GGO4. InSepThi	On Test SetAUC: 0.94, SEN: 0.888, PRC: 0.879, ACC: 0.886
Ning et al. [47]	Classification of COVID +ve, COVID −ve, and non-informative CT	19685 CT slices	VGG16, DNN, penalized logistic regression algorithm	1. COVID-19 positive2. COVID-19 negative3. Non-informative CT	On HUST-19 SetAUC: 0.994, ACC: 0.946, SEN: 0.936, SPE: 0.946
Pathak et al. [48]	Classification of COVID +ve and COVID −ve	413 COVID +ve images439 images of normal or pneumonia infected patients	Pre-trained ResNet-32	1. COVID-19 positive2. COVID-19 negative	On Test SetACC: 0.93, SPE: 0.95, SEN: 0.91, PRC: 0.95
Aniello et al. [49]	Classification of COVID +ve and COVID −ve	2482 CT images1252 positive images1230 negative images	ADECO-CNN method	1. COVID-19 positive2. COVID-19 negative	On Test SetACC: 99.99%, SEN: 99.96%, PRC: 99.97%, Spec: 99.97%
Dilbag et al. [50]	Classification of COVID+ve, pneumonia, tuberculosis, and healthy	2373 COVID, 2890 pneumonia infected, 3193 tuberculosis, and 3038 healthy images	VGG16, DenseNet201, and ResNet152V2	1. COVID-19 positive2. Pneumonia3. Tuberculosis and healthy	On Test SetAUC: 98.29%, ACC: 98.94%, SEN: 98.84%, SPE: 98.83%, F1-score: 98.31%
Xu et al. [51]	Classification of COVID-19, IAV, and irrelevant to infection	349 COVID positive and 397 COVID negative CT scans	Res-Net18, location attention classification model and noisy-OR Bayesian function	1. COVID-192. IAV3. Irrelevant	On Test SetF1-score: 86.7%, PRC: 86.7%, ACC: 86.7%
Ouyang et al. [52]	Classification of COVID-19, CAP, and non-pneumonia	4982 CT scans from 3645 patients	3D ResNet34 and VB-Net	1. COVID-192. CAP and non-pneumonia	On Test SetAUC:0.944, ACC:0.875, SEN:0.869, SPE: 0.901, F1-score: 0.82
Wang et al. [54]	Classification of non-pneumonia, COVID-19, interstitial lung disease	Total 4657 CT scans936 Normal CT scans1315 COVID CT scans2406 ILD scans	3D-UNet3D-ResNet	1. Non-pneumonia2. COVID-193. Interstitial lung disease	On Test SetACC: 0.933, SPE: 0.955, SEN: 0.876
Polsinelli et al. [55]	Classification of COVID-19, CAP, and non-pneumonia	360 COVID-19 CT scans 397 CT of other illnesses and healthy scans	SqueezeNet	1. COVID-192. CAP3. Non-pneumonia	On Test SetACC: 0.853, SEN: 0.876, SPE: 0.820, PRC: 0.850, F1-score: 0.862
Perumal et al. [57]	Classification of CAP and normal patients	From different sources including coronacases [73] and radiopaedia [74]	Pre-trained models VGG-16 [38], Resnet-50 [41], InceptionV3 [75], and AlexNet [76]	1. CAP2. Normal	ACC: 96.69%, SEN: 96%, and SPE: 98%
Yan et al. [58]	Classification of COVID and common pneumonia	226 patients CT scans with COVID and 462 patients CT scans with common pneumonia	Multi-scale spatial pyramid (MSSP) decomposition	1. COVID-192. Common pneumonia	On Test SetACC: 0.962, SEN: 0.995, SPE: 0.956, AUC: 0.977
Matsuyama et al. [60]	Classification of COVID-pneumonia and non-COVID-19 pneumonia	720 CT images	ResNet-50	1. COVID-192. Non-COVID-19	On Test SetACC: 0.922, SEN: 0.904, SPE: 0.933,F1-score: 0.839
Hu et al. [61]	Classification of COVID-positive and COVID-negative	521 COVID-19 and 397 healthy subjects	ShuffleNet V2	1. COVID Infected2. Other pneumonia (SARS)	On Test SetAUC: 0.969, SEN: 0.902, SPE: 0.916, ACC: 0.912
Ibrahim et al. [64]	Classification of COVID-19, pneumonia, lung cancer, and normal	618 total CT patients224 samples with IAVP219 samples with COVID-19175 samples healthy	VGG-19	1. COVID-192. Pneumonia3. Lung cancer4. Normal	On Test SetACC: 98.05%, RC: 98.05%, PRC: 98.43%, SPE: 99.5%, AUC: 99.66%
Rahimzadeh et al. [65]	Classification of COVID-19 or normal CT	95 COVID CT scans282 Normal CT scans	ResNet-50 V2	1. COVID-192. Normal CT	On Test SetACC: 0.985%, SEN: 0.95
Chen et al. [67]	Classification of COVID-19 and healthy	216 Patients COVID +ve scans171 Persons COVID −ve scans	Pre-trained encoder and self-supervised strategy	1. COVID-192. Healthy	On Test SetACC: 0.868, Prec: 0.883, AUC: 0.931, RC: 0.872
Shuyi et al. [68]	Classification of COVID-positive and COVID-negative	Total 295 CT scans146 COVID +ve scans149 healthy CT scans	DenseNet	1. COVID-192. Healthy	On Test SetAUC: 0.98, ACC: 0.92, SEN: 0.97, SPE: 0.87, F1-score: 0.93
Alshazly et al. [70]	Classification of COVID-19, non-COVID-19 viral infections, and healthy	Total 4173 CT images2168 COVID -19, 758 healthy, and 1247 others’ images	COVID-Nets based ResNet and DensNet	1. COVID-192. Non-COVID-19 viral infections3. Healthy	On Test SetACC: 83.89%, PRC: 80.36%, SPE: 92%,SEN: 82%, F1-score: 81%
Zhu et al. [71]	Classification of COVID-19 and non-COVID-19	1357 COVID-19 positive and 1235 negative samples	Pre-train ResNet50	1. COVID-192. Non-COVID-19	On Test Set: ACC: 93%, SEN: 93%, SPE: 92%, AUC: 93%
Tripti et al. [72]	Classification of COVID-19 and non-COVID-19	1252 COVID-19 images1230 non-COVID-19 images	Generative adversarial network (GAN) and ResNet-50	1. COVID-192. Non-COVID-19	On Test setACC: 99.2%, SEN: 99.78%, SPE: 97.78%, F1-score: 98.79%
Bai et al. [22]	Classification of COVID-19 and non-COVID-19	Total: 1186 patients,521 patients with COVID-19, and 665 patients with non-COVID-19 pneumonia	Pre-trained EfficientNet-B3 [77]	1. COVID-192. Non-COVID-19	On Test setACC: 96%, SEN: 95%, SPE: 96%, AUC: 95%
Zhenxing et al. [78]	Classification of COVID-19 and non-COVID-19	209 COVID-19 patient scans and 207 normal patient scans	CNN, fusion of channel [79, 80] and self-attention [81, 82]	1. COVID-192. Non-COVID-19	On Test setDiagnosis Assessment: 98.28%Severity Assessment: 94.83%
Sertan et al. [83]	Classification of COVID-19 and normal	80 normal CT scans and 19 COVID-19 CT scans	ResNet-50	1. COVID-19 and normal2. Normal	On Test setAUC: 96%, ACC: 84%, SEN: 100%SPE: 80%

Segmentation-based techniques

Image segmentation involves partitioning a digital image into several segments (e.g., sets of pixels, also referred to as image objects). Segmentation aims to make an image more meaningful and easier to interpret by simplifying and changing its representation [86]. Image segmentation is an essential task in analyzing medical images and obtaining further diagnostic insights and includes measuring the area and volume of segmented structures. As a result, lung CT image segmentation is extensively used for COVID-19 detection and diagnosis.

It is crucial to segment pneumonia lesions from COVID-19 patients’ CT images. Therefore, several segmentation models and tools have been developed to diagnose COVID-19 by targeting lung lesions and different manifestations [21]. The segmented lesions can identify coronavirus, determine the severity of pneumonia, and ensure that patients follow up. For example, an AI system [87] was developed to diagnose COVID-19 and differentiate it from other common pneumonia types and normal controls. The proposed framework is comprised of two models: the lung-lesion segmentation model and a diagnosis prediction model. The proposed system successfully identified important clinical markers and correlated coronavirus lesion properties.

Most COVID-19 researchers have utilized segmentation by three means: 1) only segmenting the COVID-19 lung lesion or infection, 2) combining classification results with segmented lesions for joint diagnosis, or 3) segmenting COVID-19 lesions along with quantifying severity (less common). In the following sections, we arrange the collected literature in the same order.

COVID-19 lung lesion or infection segmentation

Most procedures have exploited segmentation for identifying COVID-19 lesions or infection regions. The segmented lesions can help determine the severity of pneumonia and may ensure that patients follow up. Wang et al. [88] performed COVID-19 pneumonia lesion segmentation by utilizing noisy training labels. Zhou et al. [89] used U-Net as the backbone and included an attention mechanism for segmenting the COVID-19 infection. The attention mechanism was added to the U-Net to improve feature representation. The focal Tversky loss was also employed to train the model to enhance small ROI segmentation performance.

To handle the diversity of COVID-19 infection in CT images, Yan et al. [90] created a novel system that includes a feature variation block in their framework that adjusts the global properties of the features for COVID-19 infection segmentation. In addition, they created a dataset (comprising 21,658 chest CT images from 861 patients with confirmed COVID-19 pneumonia) to train their proposed model—Voulodimos et al. [91] semantically segmented COVID-19 infection. In comparison to U-Nets, they first demonstrated that fully convolutional neural networks (FCNs) could segment accurately despite class imbalance and human-made annotation errors. They further devised a light U-Net model trained on a personal computer (PC) with a limited dataset to deal with class imbalance problems.

Yao et al. [92] exploited a label-free method for COVID-19 lesion segmentation to alleviate the scarcity of annotated images. To distinguish healthy tissue from probable COVID-19 lesions, the authors created a NormNet (voxel-level) anomaly modeling network. Their training technique requires only a large-scale healthy CT lung dataset (with no tagged COVID-19 lesions) and produces better results. Similarly, Laradji et al. [93] presented an algorithm that also solves the scarcity of labeled CT scans, performing as well as those with full supervision. Qiu et al. [94] built an extremely minimal network termed MiniSeg. The proposed network is a lightweight deep learning model optimized for efficient COVID-lesion segmentation. The model outperforms state-of-the-art image segmentation approaches not only in terms of efficiency but also accuracy. Chen et al. [95] suggested a technique to segment numerous COVID-19 infection locations by employing aggregated residual transformations to learn robust and expressive feature representation. A soft attention mechanism was further applied to efficiently distinguish a range of COVID-19 symptoms.

To accurately segment COVID-19 infected regions, and to solve the problem of the diversity of lesion shapes and areas, Pei et al. [96] presented an architecture referred to as the multi-point supervision network (MPS-Net). Multi-scale feature extraction and sieve connection were used to extract features of various sizes. A training system was also created to exploit the ground truth. Chen et al. [97] segmented the COVID-19 lung lesions by using the ROI to verify the applicability of the 3D network and remove the inappropriate background. Further, a 3D network was adopted to extract spatial features. A combination loss function was proposed for faster convergence. To segment COVID-19 CT irregularities, Paluru et al. [98] proposed a lightweight CNN model called Anam-Net. The authors claimed that Anam-Net has 7.8 times fewer parameters than the state-of-the-art UNet (or its variants), thus making it lightweight. Müller et al. [99] devised a model for segmenting the COVID-19 infectious regions that relied on the on-the-fly generation of unique and random image patches. The model was trained using several pre-processing methods and data augmentation technique.

Similarly, to locate COVID-19 multi-class lung infection areas, Fan et al. [16] created the Inf-Net network. The network improved the representation of lesion boundaries by employing reverse attention and boundary attention. The proposed network did not require a large amount of labeled data due to the semi-supervised structure.

Abdel et al. [100] developed a few-shot segmentation algorithm, FSS-2019-nCov. The study's primary goal was to produce accurate segmentation from a small amount of annotated lung CT data. The proposed FSS architecture allowed for learning from small support samples and improved query sample generalization. In addition, the Res2Net50-based encoder [101] resulted in improved network convergence. Using CT images, Saeedizadeh et al. [102] constructed a model to perform COVID-19 lesion segmentation. The U-Net architecture served as the fundamental basis of the method, and the model could recognize pathologic COVID-19 regions with a high degree of accuracy. Xie et al. [103] used relational two-stage CNNs to segment pulmonary lobes from CT images. They captured visual and geometric correlations between high-level convolution features, indicating object and object-part relationships. Elharrouss et al. [104] presented a multi-task deep learning model to segment COVID-19 infection. The model was trained by using two streams of inputs to conduct multi-class segmentation. Furthermore, the multi-input stream allowed the model to train on various features, improving the outcomes. Finally, to diagnose COVID-19 pneumonia, Li et al. [105] developed a semi-supervised COVID-19 infection segmentation method. Transformation equivalence and perturbation invariance were used to exploit unlabeled images. In addition, a dual-form uncertainty mechanism was included for reliability and robustness.

Combined classification and segmentation for joint diagnosis

To diagnose COVID-19 pneumonia, some segmentation-based techniques have been used along with the classification processes. For instance, the trained model of Amyar et al. [106] classifies input CTs as COVID/non-COVID, followed by a segmentation process. The devised network relies on a multi-task deep learning approach to identify COVID-19 patients and segment COVID-19 lesions from chest CT images. Gao et al. [107] built a dual-branch combination network (DCN) for COVID-19 diagnosis, achieving individual-level classification and lesion segmentation simultaneously. The authors further proposed a new lesion attention module to combine the intermediate segmentation results and focus the classification branch more intensely on the lesion areas. A slice probability mapping approach was also used to convert the classification results from the slice level to the individual level.

Another diagnostic model [108] combined classification results and segmented lesions for joint diagnosis. Moreover, the authors constructed an open access COVID-19 dataset with annotated CT scans. Dey et al. [109] used a machine learning–based pipeline for COVID-19 infection detection. Their pipeline entailed COVID-19 infection segmentation followed by feature extraction, selection, and fusion to classify infected regions.

Infection segmentation and quantification

Some methods have segmented lung infections and quantified infection severity. For example, the system designed by Shan et al. [53] can automatically segment and quantify infected areas in the lung. To segment COVID-19 infection areas in CT images, they proposed a “VB-Net” neural network. The V-Net [110] was combined with a bottleneck structure [41] in this enhanced 3D CNN. Chaganti et al. [111] presented a method for the segmentation and quantification of abnormal CT patterns related to COVID-19. Lungs, lobes, and COVID-19 lesions were first segmented before some constraints were enforced for the evaluation of COVID-19 infection. The quantification of the severity was determined through the combined scores derived from the imposed constraints. Similarly, Zhou et al. [112] considered the dynamic changes of actual patients’ data measured at different time points for COVID-19 CT quantification and segmentation of the infected regions. They divided the 3D segmentation problem into three 2D segmentations utilizing the symmetry of the lungs and other tissues to solve large scene–small object problems with little data. Oulefki et al. [21] designed a system that automatically segmented and measured COVID-19 lesions with 3D visualizations. The proposed system adopted a flexible and efficient end-to-end method. Their solution exceeded state-of-the-art segmentation techniques in terms of robustness and accuracy. He et al. [113] established a synergistic learning paradigm for COVID-19 severity evaluation with lung lobe segmentation and classification. A multi-task, multi-instance (M2UNet) deep network was built to assess the severity of COVID-19 infections and lung lobe segmentation. Selvaraj et al. [114] introduced a deep neural network (DNN) model trained on a small dataset and used a region-specific technique to choose features. The Zernike moment (ZM) and gray level co-occurrence matrix (GLCM) were used to extract unique shape and texture features as vectors that were used to perform segmentation and showed the severity of the COVID-19 infection. Table 2 provides detailed selective information related to the COVID-19 segmentation-based methods.

Table 2

COVID-19 segmentation methods and their selective information.

Source/Author	Performed Tasks	No of CT Scans/Images/Slices	Framework/Approach	Target ROI	Performance
Zhang et al. [87]	Distinguishing COVID-19 from common pneumonia (CP) and normal controls	For Lesion Segmentation: 4,695 CT imagesFor Classification model: 361,221 CT images	Segmentation models (U-net, DRUNET, FCN, SegNet) and 3D classification network (3D convolutional blocks)	Lung lesions	On Internal Validation SetACC: 92.49%, SEN: 94.93%, SPE: 91.13%AUROC: 0.97 (95% CI: 0.9665–0.9904)
Wang et al. [88]	COVID lesion segmentation	558 COVID-19 patients CT scans	CNN, self-ensemble CNNs [115, 116]	COVID-19 lesions	Dice (%): 80.29±11.14, RVE (%): 17.72±23.40, HD95(mm): 18.72±27.26
Zhou et al. [89]	COVID lesion segmentation	Dataset1: 100 axial CT imagesDataset2: A private dataset	U-Net and attention mechanism [117, 118]	COVID lesion regions	For Dataset1Dice: 69.1%, SEN: 81.1%, SPE: 97.2%
Yan et al. [90]	COVID infection segmentation	21658 chest CT images from 861 COVID-19 +ve patients	Encoder decoder framework and PASPP	Lung and COVID-19 infection region	For Lung: Dice: 0.987, SEN: 0.986, PRC: 0.990For COVID-19: Dice: 0.726, SEN: 0.751, PRC: 0.726
Voulodimos et al. [91]	COVID deep models’ analysis for infection segmentation	939 cross-sectional images from 10 axial volumetric CT scans	U-Nets and FCN	Lung and COVID-19 infection region	For U-Net Segmentation:PRC: 0.96, RC: 0.57, ACC: 0.97, F1-score: 0.65For FCN Segmentation:PRC: 0.86, RC: 0.50, ACC: 0.97, F1: 0.57
Yao et al. [92]	COVID lesion segmentation	For Lung Modeling: 453 CT Volumes from LUNA-16 [119]For Lesion Segmentation: 138 CT Volumes from Lung Segmentation 515 CT scans	3D UNet and MONAI [120]	Lung and COVID-19 lesion	Segmentation on Corona Cases Dataset [73]:PRC: 90.5%, SEN: 78.6%, SPE: 81.2%, AUC: 87.0%
Laradji et al. [93]	Improvement of COVID-19 lesion segmentation performance	Total Slices Infected SliceCOVID-A: 98 98COVID-B: 829 372COVID-C: 3520 1841	VGG-16 and FCN8 [121]	Infection region	Dice: 0.73, IOU: 0.57, PPV: 0.65, SEN: 0.82, SPE: 0.92
Qiu et al. [94]	COVID-19 segmentation	Medseg dataset [122], COVID-19 CT lung and infection segmentation dataset [123], and Mosmed dataset [124]	Encoder (Attentive Hierarchical Spatial Pyramid -AHSP) and Decoder (Feature Fusion Module-FFM) Network	COVID-19 infected regions	Segmentation On COVID-19 CT segmentation dataset mIoU: 84.49, SEN: 85.06, SPC: 99.05, DSC: 76.27 HD: 51.06
Chen et al. [95]	Multiple COVID-19 infection regions segmentation	SIRM dataset [125]	Residual Attention U-Net and ResNeXt block	COVID-19 infection	DSC: 0.94, ACC: 0.89, PRC: 0.95
Pei et al. [96]	COVID-19 lesion segmentation	Medseg dataset [122]	U-Net	COVID-19 lesion	Dice: 0.8325, SEN: 0.8406, SPE: 0.9988, IOU: 0.742
Chen et al. [97]	COVID-19 lung lesions segmentation	Public dataset [16]	Conditional Random Field CRF and 2.3D attention model	COVID-19 infection	Performance on Public DatasetACC: 93.92±3.87, PRC: 78.26±9.94%, RC: 96.39±2.92%, DSC: 75.22±11.25%
Paluru et al. [98]	COVID-19 abnormalities segmentation	Medseg dataset [122] and [123]	CNN and U-Net	Lung extraction and COVID-19 abnormality segmentation	Segmentation outputs for Experiment1Dice: 0.755, SEN: 0.900, SPE: 0.993, ACC: 0.991
Muller et al. [99]	COVID-19 infection segmentation	20 +ve COVID-19 CT scans	3D U-Net	Lungs and COVID-19 infected regions	For Lungs: DSC: 0.956, SEN: 0.956, SPE: 0.998For COVID Infection: DSC: 0.761, SEN: 0.730, SPE: 0.99
Fan et al. [16]	COVID-19 lung infection segmentation	100 labeled CT slices, 1600 unlabeled images	Reverse attention and explicit edge-attention mechanisms	COVID-19 infection area	For Inf-Net: Dice: 0.682, SEN: 0.692, SPE: 0.943, MAE: 0.082For Semi-Inf-Net: Dice: 0.739, SEN: 0.725, SPE: 0.960, MAE: 0.064
Abdel et al. [100]	COVID-19 infection segmentation	Medseg dataset [122]	Encoder (using Res2Net module) [101] and decoder network	COVID-19 lung infection	On Test SetDSC: 0.798, SEN: 0.803, SPE: 0.986, MAE: 0.065
Saeedizadeh et al. [102]	Segmentation of pathologic COVID-19 regions	Medseg dataset [122]	U-Net and 2D total variation [126]	COVID-19-specific pathologic regions, ground glass regions, and COVID consolidation regions	For pathologic regions: SEN: 0.808, SPE: 0.960, Dice: 0.801For Ground-Glass mask: SEN: 0.762, SPE: 0.979, Dice: 0.655For Consolidation mask: SEN: 0.558, SPE: 0.988, Dice: 0.537
Xie et al.[103]	COVID-19 pulmonary lobe segmentation	Dataset1 (COPD set): 5000 subjects [127]Dataset2(COVID-19 set): 470 subjects [128]	CNN and U-Net	Pulmonary lobe, lung	For Lobe segmentationIOU: 0.9197For Lung segmentationIOU: 0.9706
Elharrouss et al. [104]	COVID-19 lung infection segmentation	Medseg dataset [122]	Encoder-decoder and convolutional neural networks	COVID-19 lung infection	On Test SetDice: 0.786, SEN: 0.711, SPE: 0.993, PRC: 0/856,F1-score: 0.784, MAE: 0.076
Li et al. [105]	COVID-19 lesion segmentation	852 whole-volume chest CT scans	3D multi-decoder VNet [110]	COVID-19 lesion	DSC: 77.4%, Jaccard: 64.5%, ASD: 3.9mm
Amyar et al. [106]	1.COVID/non-COVID Classification2. Lesion segmentation3. Image reconstruction	Total 1369 patients, 449 COVID-19, 425 normal, 98 lungs cancers, and 397 of different kinds of pathology patients	U-Net and multi-task learning architecture	COVID-19 infected region	Experiment 1 SegmentationDice: 88.0%, ACC: 95.23%, SEN: 90.2%, SPE: 99.7%Experiment2 ClassificationACC: 94.67%, SEN: 0.96, SPE: 0.92, AUC: 0.97
Gao et al. [107]	Classification and COVID lesion segmentation	1918 CT scans from 1202 subjects	DCN, FCN, and U-Net	Lung and COVID-19 lesion	For Slice-level Classification: ACC: 95.99%, AA: 95.59%, SEN: 89.14%, SPE: 98.04%For Individual-level Classification: ACC: 96.74%, AA: 96.95%, SEN: 97.91%, SPE: 96.00%For Lung Segmentation: DSC: 99.11%, PRC: 99.33%, RC: 98.89%For Lesion Segmentation: DCS: 83.51%, PRC: 83.46%, RC: 83.55%
Wu et al. [108]	Classification and segmentation of COVID-19 chest CT	144167 chest CT images of 400 COVID-19 patients and 350 non-infected cases	Res2Net network [101], Activation Mapping [69], Image Mixing [129], VGG-16 [38]	Lung infection	For Classification: SEN: 95.0%, SPE: 93.0%For Segmentation: Dice: 78.5%
Dey et al. [109]	COVID-19 infection segmentation and segmented regions classification	Medseg dataset [122]	Morphological segmentation, KNN classifier, image thre-sholding, and fused feature vector	COVID-19 lung infection	On Test Set: ACC: 0.9108±0.0256, PRC: 0.8953±0.038, SEN: 0.8436±0.0243, SPE: 0.9451±0.0540, F1-Score: 0.8514±0.0246, AUC: 0.9224±0.0327
Shan et al. [53]	To quantify infection regions of interest	249 CT images for training300 CT images for Validation	VB-Net	COVID-19 infection regions	DSC: 91.6%±10.0%
Chaganti et al. [111]	Abnormalities quantification associated with COVID-19	9749 chest CT volumes	Dense UNet	Lung segmentation, lobe segmentation, and abnormality segmentation	Abnormality SegmentationPO (P < .001) = 0.92, PHO (P < .001) = 0.97LSS (P < .001) = 0.91, LHOS (P < .001) = 0.9
Zhou et al. [112]	Segmentation and quantification of the infection regions	201 anonymized CT scans from 140 COVID-19 patients and a private dataset	Data preprocessing technique with a decomposition of the 3D segmentation into three 2D ones	Lung lobes segmentation and infection segmentation	Segmentation on the Highest-quality Data SubsetDice: 0.802±0.072, RC: 0.794±0.068
Oulefki et al. [21]	COVID-19 Lung Infection segmentation and measurement	COVID-CT-Dataset [34]	Enhancement method, along with the segmentation and visualization	COVID lung infection	ACC: 0.98, SEN: 0.73, PRC: 0.73, Dice: 0.71, SPE: 0.99
He et al. [113]	COVID-19 severity assessment and Lung lobe Segmentation	666 chest CT scans of 242 COVID-19 patients	Multi-task multi-instance U-Net (M2UNet)	Lung lobe	On Test Set: ACC: 0.985±0.005, PRC: 0.975±0.022, RC: 0.952±0.011, F1-Score: 0.963±0.011, AUC: 0.991±0.010
Selvaraj et al. [114]	COVID-19 infection segmentation with severity illustration	80 CT cases	Deep neural network (DNN), Zernike moment (ZM) and gray level co-occurrence matrix (GLCM)	COVID-19 infected region	For Classification: AUC: 0.942, SEN: 0.767, SPE: 0.928ACC: 93.8%For Segmentation: SEN: 0.701, SPE: 0.942, Dice: 0.757, MAE: 0.082

Detection-based techniques

In the COVID-19 literature, the term “detection” is often used synonymously with “diagnosis.” However, in computer vision, detection is described differently because it is an operation that involves localizing an object. Localization could be achieved by placing a bounding box (such as a rectangle, square, or contour) or interactive colors (heat maps) depending on the image's object shape. Therefore, this section focuses on such methods tailored to detecting COVID-19 that automatically explore infected lung regions. However, we note that quite a small subset of research articles has considered computer vision–based detection for COVID-19 diagnosis. Thus, to cover more literature, we have included methods in which the ROIs or lesions are localized by contours, heat maps, bounding boxes, or interactive colors for diagnosis. These methods are hybrid (considering segmentation and classification practices) with visualized predictions (heats maps, contours, or bounding boxes).

For instance, Wang et al. [130] proposed a prognostic model and claimed that it could assist in medical resource optimization and COVID-19 prevention. The proposed system not only identified COVID-19 but also visualized suspicious infected lung areas using heat maps. Wang et al. [131] proposed a weakly supervised deep learning model to accomplish COVID-19 classification and lesion localization. Combining class activation mapping and 3D-linked components, the model yielded lesion localization results without annotations. Ahuja et al. [132] designed a three-phase COVID-19 CT detection model comprising three phases. In Phase 1, the decomposition of stationary wavelets was utilized for data augmentation. In Phase 2, a trained CNN model was employed for binary classification. Finally, in Phase 3, defects in CT scan images were located. He et al. [133] introduced a differentiable neural architecture search (DNAS) framework that relied on the class activation mapping (CAM) technique to interpret the final results. The model produced promising results on the three experimental datasets with a ten-fold reduced model size and higher accuracy. Polat et al. [134] presented a CNN model to detect and localize lesion patterns in COVID-19 CT images. The proposed method segmented 102 CT images and obtained 16,040 CT image segments, which were classified and labeled as either COVID-19-infected or healthy regions. Automatic localization of the COVID-19 pneumonia findings was achieved in the final stage.

Alom et al. [135] introduced an end-to-end system and applied classification and segmentation to identify COVID-19 infections. First, an inception recurrent residual convolutional neural network (IRRCNN) was used to classify X-ray and CT images as normal or COVID-19 cases, which was followed by segmentation (NABLA-3 network). Finally, the infected regions were localized by heat maps. Gozes et al. [24] designed a coronavirus-related abnormalities-based hybrid system that relies on commercial software and training the U-Net model on extensive CT data. The authors demonstrated that AI-based models could detect COVID-19 with high accuracy. Harmon et al. [136] trained a series of deep learning models on a diverse, multi-national cohort of 1280 patients to localize the parietal pleura/lung parenchyma. Lung segmentation was utilized for localizing the chest cavity regions. After the localization process, classification was applied for COVID-19 pneumonia. Hu et al. [137] employed a weakly supervised framework and achieved COVID-19 classification and lesion localization based on less labeled data. The proposed network used pre- and post-processing of the data for lung segmentation. The authors further used multi-scale learning followed by weakly supervised learning for lesion localization.

To identify the COVID-19 infected regions in high-resolution CT images, Chen et al. [23] built a system on the top of U-Net++ [138] that extracted valid areas from CT images and considered ResNet-50 as the backbone. The model generated and framed the suspicious lesion outputs using prediction boxes. The proposed method significantly decreased the reading times of radiologists by 65%. Pu et al. [139] focused on COVID-19 detection, quantification, and disease progression, accomplishing lung region segmentation and identification of infected regions. The authors interpreted the infected areas using heat maps, which they consider to be essential to assessing disease progression. Perumal et al. [140] applied the transfer learning technique to differentiate between various pulmonary diseases, including COVID-19 pneumonia. The proposed method included an image enhancement process and extracted Haralick [141] texture features. The extracted features were then fed into the various pre-defined CNN models. Finally, the infected region was identified using Grad-CAM [69].

To minimize the burden on physicians and improve diagnostic accuracy, Wang et al. [142] built and deployed a classification- and segmentation-based AI system capable of analyzing CT images and providing infection probability. To aid in the research, a dataset with contours and infection regions was also created. The proposed method included lung region extraction and lesion segmentation and classification to detect COVID-19 pneumonia. Similarly, Ni et al. [143] developed an abnormalities detection technique for COVID-19 patients. To classify and pinpoint possible abnormal regions in CT scans, their architecture retrieved the most prominent features, followed by a classifier and regressor. Afterward, the 3D U-Net [19] was used to classify voxels that reflected abnormalities in the detected regions. Compared to resident radiologists, the algorithm performed exceptionally well in detecting COVID-19 pneumonia. Zhang et al. [144] created the IAAS (Intelligent Assistant Analysis System), a real-time system to aid radiologists in COVID-19 assessment. A modified 3D CNN and a combined V-Net with bottleneck structures were used in the software, which performed the quantification and localization of COVID-19 lesions.

Alshazly et al. [145] modeled an AI-based framework with custom-sized CT images to automatically classify COVID-19 and non-COVID-19 cases. The model used InceptionV3 [75] for the classification task and the final predictions were visualized using the Grad-CAM [69] approach. As a result, the proposed system identified COVID-19 cases and accurately localized the COVID-19 infection-associated regions. Mobiny et al. [146] created a detail-oriented capsule networks (DECAPS) framework by boosting the COVID-19 classification accuracy. DECAPS employed a Peekaboo training procedure using a two-stage patch crop and drop strategy to generate activation maps. The activation maps focused on ROIs that combined coarse- and fine-grained data representations. The authors adopted a conditional GANs [147]-based data augmentation procedure for dealing with data scarcity. Javaheri et al. [148] introduced a highly accurate detection framework named CovidCTNet. The proposed method utilized small sample sizes of CT image data to identify COVID-19 and extract infected lung areas. Non-lung-related parts were removed with the BCDU-Net [149] and the COVID-19 infection areas were extracted and highlighted.

Gunraj et al. [25] developed COVIDNet-CT, a deep CNN architecture that uses a machine-driven design exploration technique to detect COVID-19 cases from chest CT scans. The critical visual factors associated with COVID-19 infection were explored by GSInquire [150]. Additionally, COVIDx-CT, a benchmark CT image dataset, was introduced for public research. The same author proposed another COVID-Net CT-2 [151] detection model based on enhanced deep CNNs, with critical areas highlighted in red. Two large and diverse benchmark datasets were used to train the network. Finally, the authors used explainability to examine the tailored algorithm's decision-making behavior to guarantee that decisions were made based on meaningful visual cues in CT images.

Jaiswal et al. [36] classified COVID-19 and non-COVID-19 cases in chest X-ray and CT images. The transfer learning approach was adapted on a Pruned EfficientNet [77] model to detect COVID-19 automatically. A local interpretable model-agnostic explanations (LIME) approach [152] was employed to interpret the predictions made by the model. Qiblawey et al. [153] suggested a cascaded technique for detecting, localizing, and quantifying COVID-19 infections from CT images. Ma et al. [154] and Ghavami et al. [155] also proposed COVID-19 detection models and included visual interpretations in their final outputs. A recent paper [156] proposed a simple yet efficient and pure detection network—the sequential region generation network (SRGNet)—to jointly detect and segment the COVID-19 lesion areas. The method utilizes supervised segmentation information to predict the COVID-19 infected areas. Table 3 provides detailed selective information related to the detection-based methods.

Table 3

COVID-19 detection-based methods and their selective information.

Source/Author	Performed Tasks	Dataset Information	Framework/Approach	Final Output	Performance
Wang et al. [130]	A DL system that automatically focuses on abnormal areas and identifying COVID-19 from other pneumonia	A total of 5372 patients with computed tomography (CT) images, including additional information	DenseNet121 [37] and FPN [66]	Heat maps visualization of suspicious lung areas	On COVID-19AUC: 0.87On other PneumoniaAUC: 0.88For Viral PneumoniaAUC: 0.86
Wang [131]	COVID-19 classification and lesion localization	For Training: 499 CT volumesFor Testing:131 CT volumes	3D CNN, U-Net, and DeCoV-Net	Visualizations and predictions of those regions where pneumonia occurs	ROC AUC: 0.959PR AUC: 0.976For ClassificationPPV:0.840 and NPV:0.982
Ahuja et al. [132]	COVID-19 detection with binary classification e.g., COVID and Non-COVID	349 positive CT images and 397 CT images of non-COVID patients	ResNet18, ResNet50, ResNet-101, and Squeeze-Net [56]	Positive cases abnormality localization	On TestingACC: 99.4%, SEN: 100%, SPE: 98.6%, AUC: 0.9965
He et al. [133]	COVID-19 classification and results interpretability	COVID-CT-set, CC-CCII, and Mosmed Data [65, 87, 124]	Differentiable neural architecture search (DNAS) framework, Gumbel Softmax technique [157] and class activation mapping (CAM) [158]	Visualization of discriminative lesion regions	For CovidNet3D-LACC: 96.88, PRC: 97.50, SEN: 92.86, F1-score: 0.9512
Polat et al. [134]	Detect and localize COVID related lesions patterns	COVID dataset [159] and SIRM [125]	CNN	Automatic localization of COVID-19 pneumonia findings	ACC: 0.903, SEN: 0.905, SPE: 0.903, PRC: 0.8551, and F1-score: 0.8714
Alom et al. [135]	COVID-19 detection and infected region localization	Total samples: 5,216Normal samples: 1,341Pneumonia samples: 3,875	IRRCNN [125], NABLA-N [135]	Infected region with heat maps and contours	For X-Ray: ACC: 84.67%For CT: ACC: 98.78%
Gozes et al. [24]	An automated tool for COVID detection, quantification, and tracking	Normal slices = 1036Abnormal slices n = 829	Resnet-50, Grad-CAM technique [69]	Opacities quantitative measurements and heat maps visualization	COVID and Non-COVID ClassificationACC: 0.996, AUC (95%CI: 0.989-1.00), and SEN: 98.2%, SPE: 92.2%
Stephanie [136]	Localization of parietal pleura-lung parenchyma followed by a classification task	In total, 2724 CT scans from 2617 patients	AH-Net [160], Densnet-121, Grad-CAM [69]	Attention/activation maps generated using Grad-CAM and utilized for classification of COVID-19	For COVID-19 pneumonia ClassificationACC: 90.8%, SEN: 84%, SPE: 93%
Hu et al. [137]	Detecting and classifying COVID-19 infection	Total 450 patient scans, 150 chest CT exams of COVID-19, CAP and NP patients with additional information, and lung segmentation dataset [161]	CNN and U-Net	Classification detection network with COVID-19 lesions localization maps	NP with AUC: 0.90±0.03CAP with AUC: 0.86±0.03COVID-19 with AUC: 0.92±0.02
Chen et al. [23]	COVID-19 pneumonia detection	46,096 anonymous images with additional clinical characteristics	ResNet-50 and UNet++ [138]	Predictions with bounding boxes	External Dataset Per Patient ResultsSEN: 98(%, SPE: 94 (%), ACC: 96%, PPV: 94.23%, NPV: 97.92%
Pu et al. [139]	Detect, quantify, and monitor COVID-19 progression	120 CT scans for training72 CT scans for test	U-Net and bidirectional elastic registration algorithm [162]	Heat map visualization of disease progression	DSCLung boundaries: 0.95 (CI 0.95–0.96)Main lung vessels: 0.79 (CI 0.77–0.81)Regions of disease: 0.81 (CI 0.76–0.86)
Perumal et al. [140]	Detection and classification of different types of pulmonary diseases, including COVID-19	202 CT scans from various online sources (GitHub, RSNA, and Google images)	Transfer learning, CNNs, and Haralick features [141]	Infected regions heat map visualization	For ClassificationACC: 93%
Wang [142]	Lesion segmentation and classification with visual predictions	Training CasesPositive: 718, Negative: 413Testing CasesPositive: 159, Negative: 128	FCN, V-Net, U-Net, 3D U-Net++, ResNet50	Visualized highlights of the lesion regions to the screening result	Classification on Test SetSEN: 0.974, SPE: 0.922Segmentation DiceFCN: 0.68, V-Net: 0.739, U-Net: 0.7423D U-Net++: 0.754
Ni et al. [143]	COVID-19 abnormalities detection, including voxel segmentation and lobe segmentation	14,435 participants with chest CT images with pathogen diagnosis	Deepwise & League of Ph.D. Technology Co. Ltd [163], 3D U-Net, and MVP-Net [164]	COVID-19 pneumonia abnormalities visualization	For per-patient: SEN: 1.00 of (95%(CI) 0.95, 1.00), F1-score: 0.97For per-lung lobe: SEN: 0.96 of (95% CI 0.94, 0.98), F1-score: 0.86
Zhang et al. [144]	localization and quantification of COVID-19 pneumonia	Multiple cohorts datasets (CT and RT-PCR)	Modified 3D CNN combined with V-Net [41, 110]	Visualization of bilateral lesions of COVID-19 patients	Anatomic distribution of infected bronchopulmonary segments (readers are referred to Table 3 in the article)
Alshazly et al. [145]	COVID-19 associated regions localizations and visual explanations	SARS-CoV-2 dataset [165] and COVID19-CT dataset [35]	Applied transfer learning with CNNs	Visualized clusters for CT images of COVID-19 from other lung diseases	On SARS-CoV-2 datasetACC: 99.4%, PRC: 99.6%,SEN: 99.8%, SPE: 99.6%, F1-score: 99.4%
Mobiny et al. [146]	COVID-19 classification and activation maps of regions of interests	COVID-CT-Dataset [34]	Capsule Networks (CapsNets) [166, 167] and GAN [168]	Activation maps of classified COVID-19 cases	For Classification: PRC: 0.843, RC: 0.915, SPE: 0.860, ACC: 0.876, F1-score: 0.871, AUC: 0.961
Javaheri et al. [148]	To detect and distinguish between COVD-19 and CAP	A cohort of multiple datasets	BCDU-Net [149] and CNN	Visualized Covid-19 infection areas	Overall ACC: 82%For Non-COVIDPRC: 78%, RC: 90%, F1-score: 83%For COVID-19PRC: 88%, RC: 74%, F1-score: 80%
COVIDNet-CT-1Gunraj et al. [25]	COVID-19 infection regions detection and analysis	104,009 CT images from 1,489 patients	Machine-driven design exploration algorithm (GSInquire) [150]	Visual infection analysis of predictions in terms of critical visual factors associated with COVID	Normal Non-COVID COVIDSEN: 100 99 97.3SPE: 99.5 99.2 99.9PPV: 99.4 98.4 99.7NPV: 100 99.5 99.3
COVID-Net CT-2Gunraj et al. [151]	COVID-19 infection regions detection and analysis	396,025 CT images from 8,246 patients	Machine-driven design exploration algorithm (GSInquire) [150] and Generative Synthesis [169]	Visual infection analysis of predictions in terms of critical visual factors associated with COVID	ACC: 98.1%/97.9%, SEN: 6.2%/95.7%, PPV: 96.7%/96.4%, SPE: 99%/98.9%, and NPV: 98.8%/98.7%
Jaiswal et al. [36]	COVID-19 classification and detection	COVID-CT-Dataset [34]	Pruned EfficientNet [77]	Visualized analysis for the explainability of the predictions	ACC: 0.85, PRC: 0.81, RC: 0.92, F1-score: 0.86, AUROC: 0.84
Qiblawey et al. [153]	Segment, detect, localize, and quantify COVID-19 infections	Medseg [122] and [123]	Encoder-Decoder CNNs, UNet, and Feature Pyramid Network (FPN)	Visualized detection of lung and infection regions	SEN for dataset 1110 subjects:Mild: 98.3%, Moderate: 71.2%, Severe: 77.8%, Critical: 100%For COVID-19 detection: SEN: 99.64% and SPE: 98.72%
Ma et al. [123]	Infection segmentation and visualized spatial distribution map	70 annotated COVID-19 cases	Region-scalable fitting (RSF) [170] model with U-Net	Visualized infection spatial distribution map (heat map)	Quantitative results on COVID-19 CT datasetDSC: 0.7081 ± 0.1908, NSD: 0.8015 ±0.1956, HD: 35.75 ± 47.53
Ghavami et al. [155]	COVID-19 classification and detection of the infected areas	3359 samples from 6 different medical centers	Convolutional Neural Networks (CNNs)	An interpretable COVID-19 detection system	For ClassificationSensitivities (SENs): 97.75% and 98.15%Specificities (SPEs): 87% and 81.03%
Wu et al. [156]	Segmentation and detection of COVID-19 infection	313,167 CT slices from 438 patients	CNN and context enhancement (CE)	A framework to jointly detect and segment the lesion areas of COVID19 from CT images	Pixel ACC: 83.2IoU: 71.1Dice: 83.0

Datasets

Datasets are indispensable for designing and training deep learning models. Specifically, AI diagnostic models must be developed using medical image datasets. Thus, datasets should be given equal importance to the model setups. Therefore, in our study, we selected 29 standard and publicly available chest CT datasets that can be used for COVID-19 diagnosis research. Fig. 4 shows that the most popular data repositories are individual web pages (33.3%) and databases such as Mendeley (33.3%), Kaggle (16.6%), and GitHub (16.6%). The datasets and their accessible links are provided in supplementary Table 1 at the end of the manuscript.

Fig. 4

COVID-19 CT datasets' composition and availability.

Large datasets with supplemental AI-based models

Some of the datasets are large and have been supplemented with AI models. For instance, CC-CCII [87] is one of the most extensive CT datasets, containing 6752 scan results for 4154 patients and a total of 617,775 slices in three categories: novel corona pneumonia (NCP), common pneumonia (CP), and normal control group (Normal). An AI-based model was developed and tested with this dataset to diagnose NCP and differentiate it from common pneumonia and normal controls. The system performance was evaluated by dice coefficient (DC) and pixel accuracy (PA) metrics by a five-fold cross-validation test. DeepLabv3 [171] has been adopted as the backbone for subsequent analysis for better segmentation performance. Wu et al. [108] constructed a large-scale COVID-19 classification and segmentation (COVID-CS) dataset with 144,167 chest CT images of 400 COVID-19 patients and 350 non-infected cases. A novel joint classification and segmentation (JCS) system was designed to perform real-time and explainable COVID-19 chest CT diagnosis by considering the constructed datasets. Another large dataset, COVID-Net CT-1 [25], with three case types—NCP, CP, and normal—was utilized with a neural network tailored for COVID-19 detection. For NCP and CP cases, the slices were marked as containing lung abnormalities. The dataset was further extended to a large-scale COVID-Net CT-2 [151] dataset. The authors designed a neural network tailored for COVID-19 detection as part of the open-source COVID-Net initiative. The performance validation showed that COVID-Net CT-2's decision-making behavior is consistent with radiologist interpretation by leveraging correct, clinically relevant critical factors.

He et al. [133] formed a large, clean, and segmented CT dataset named Clean-CC-CCII with three given classes—NCP, CP, and normal—by fixing errors and removing noise. Furthermore, the authors proposed an automated model for COVID-19 diagnosis. COVID-CT-set [65] is another collected dataset that consists of 48,260 CT images from 282 normal persons and 15,589 CT images from 95 patients with COVID-19 infections. The authors presented an accurate and high-speed model for COVID-19 diagnosis that achieved 98.49% accuracy on more than 7996 test images in a single image classification stage. At the patient identification phase, the system correctly identified almost 234 of 245 patients with high speed.

SARS-CoV-2 CT-scan [165] is another public CT dataset with 2482 CT scans: 1252 COVID-positive and 1230 non-COVID cases. This dataset aimed to support COVID-19 diagnosis research with AI-based methods. An xDNN was used, which scored 97.31% on the F1 scale. TCIA COVID-19, a dataset created by Harmon et al. [136], contains chest CT images from 632 individuals with COVID-19 infections. Clinical data, such as COVID-19-positive RT-PCRs, were added to the dataset. Yan et al. [58] collected 828 CT scans with abnormal findings from two hospitals and designed an AI-based system that used a multi-scale convolutional neural network (MSCNN). The experimental results indicated that the proposed model can detect COVID-19 and distinguish it from other types of pneumonia with a small amount of training data.

Small datasets with supplemental AI-based models

Yan et al. [90] developed a segmentation-based dataset consisting of 21,658 chest CT images from 861 patients with confirmed COVID-19 pneumonia. They also presented a method for segmenting COVID-19 infections. Wang et al. [142] grouped 1136 chest CT training cases (723 COVID-19-positive) from five hospitals to train an AI model to evaluate CT images and predict the risk of COVID-19 pneumonia infection. The test dataset, which included various lung illnesses, achieved a sensitivity of 0.974 and a specificity of 0.922. Wang et al. [43] created a dataset that included 1065 CT images of pathogen-confirmed COVID-19 individuals as well as those who had previously been diagnosed with normal viral pneumonia. The researchers adjusted the Inception transfer learning algorithm before testing it on their dataset. For comparison and modeling, Song et al. [172] gathered a chest CT scan dataset (Ncov2019) of 88 patients diagnosed with COVID-19, 101 patients infected with bacteria pneumonia, and 86 healthy people. To detect the patients with COVID-19, the authors also developed a deep learning–based CT diagnosis system (Deep Pneumonia). When applied to their CT dataset, the model correctly differentiated COVID-19 patients from others with an excellent AUC (area under the curve) and recall (sensitivity).

To classify COVID-19, the COVID-CT-MD [173] dataset was composed by radiologists who analyzed a subset of 55 COVID-19 and 25 CAP cases to identify and label slices with infection. The labeled subset of the data contains 4993 slices demonstrating infection and 18,416 slices without infection. Wang et al. [88] composed the two-part UESTC-COVID-19 dataset for pneumonia lesion segmentation: Part 1 includes 70 vol where lesion regions were annotated with some noise by non-experts, while Part 2 consists of 50 vol where lesions were annotated without any noise by experts. Additionally, a novel noise-robust framework was proposed to learn from the noisy labels in the segmentation task.

Datasets with no supplemental models

Some of the datasets/databases contain only chest CT scans or images. For example, there is a dataset named “2500 CT images of COVID-19 Lung” [174], which includes 2933 lung CT images of COVID-19 patients gathered from previous papers, reputable media reports, and public databases. Sixty-eight examples were collected from the sirm.org public database, and 101 cases were gathered from the GitHub public database. COVID-CT [34] is another open-source dataset consisting of 349 CT images from 216 COVID-19 patients and 463 CT images from non-COVID-19 patients. A senior radiologist confirmed the dataset's utility. The authors claimed that their dataset can be used to develop AI-based COVID-19 diagnosis models. Zaffino et al. [175] constructed an open-source lung CT dataset based on segmentation and classification, including 62 CT volumes from 50 COVID-19 patients. The CT volumes are provided with automatic threshold-based annotation obtained with a Gaussian mixture model (GMM) and scoring provided by an expert radiologist.

Jun et al. [123] constructed a COVID-19 CT lung and infection segmentation dataset containing 70 CT scans of patients diagnosed with COVID-19. The dataset has further segmented lung lesions provided by experts. Several other open-source datasets have been developed without AI models. However, these datasets have been provided with additional clinical information.

Datasets with supporting clinical information

Additional clinical information includes the patient's age, clinical history, RT-PCR test findings, etc. RICORD [176] is an excellent example of an open-source dataset with 240 thoracic CT scans and 1000 chest radiographs from four sites in different countries. With supporting clinical characteristics, RICORD is projected to be better for prediction models and able to explain long-term performance across demographics and healthcare systems. The RICORD dataset is divided into RICORD (a) and RICORD (b), as given in Table 4 . Similarly, Ning et al. [47] created the HUST-19 open-source hybrid dataset which includes chest CT scans, 130 clinical characteristics, and the laboratory-confirmed SARS-CoV-2 clinical status from 1521 individuals with pneumonia (including COVID-19 pneumonia). The database was intended to predict COVID-19 morbidity and death outcomes using a trained algorithm.

Table 4

Datasets and their selective information.

Dataset/Author’s Name	Location	Data Structure	Framework/Approach	Applications	Upshots/Results
CC-CCII, Zhang et al. [87]	China	A database of a total of 617,775 CT images from 3777 patients	U-net, DRUNET [183], FCN, SegNet and DeepLabv3	Prediction of progression to critical illness	AI system for diagnosing COVID-19 pneumonia using CT scans and evaluating drug treatment effects with CT quantification
Wu et al. [108]	China	144,167 COVID-19 CT images of 400 patients and 350 uninfected	CNNs, Activation Mapping [69], and Image Mixing [129]	Classification and segmentation of lung infection	For Classification: SEN: 95.0%, SPE: 93.0%For Segmentation: Dice: 78.5%
COVID-Net CT-1, Gunraj et al. [25]	Canada and China	104,009 CT images from 1489 patients	Machine-driven design exploration algorithm (GSInquire) [150]	To train and validate models for COVID-19 diagnosis from CT images	Normal: 99.5%, NCP: 99.2%, COVID-19: 99.9%
COVID-Net CT-2, Gunraj et al. [151]	Canada, China, and USA	396,025 CT images from 8246 patients	GSInquire) [150], Generative Synthesis [169]	To train and validate models for COVID-19 CT diagnosis	ACC: 98.1%/97.9%, SEN: 6.2%/95.7%, PPV: 96.7%/96.4%, SPE: 99%/98.9%, NPV: 98.8%/98.7%
HKBU_HPML_COVID-19, He et al. [133]	China	Total of 3993 CT scans having 131,517 NCP, 135,038 CP, and 73,635 normal slices	MNas3DNet41(AutoML) [9]	Useful for healthcare professionals to develop effective models	For NCP、CP and normalACC: 0.87, SEN: 0.861, SPE: 0.931
COVID-CT-set, M Rahimzadeh et al. [65]	Sari, Iran	15,589 CT images of 95 patients with COVID-19	ResNet50V2 network [41]	Useful for COVID-19 CT diagnosis	Correct identification of 234 patients from 245 patients
SARS-CoV-2 CT Soares et al. [165]	Sao Paulo, Brazil	Total 2482 CT scans with 1252 COVID-19 positive and 1230 non-infected CT scans	eXplainable Deep Learning approach (xDNN) [184]	COVID-19 identification through their composed dataset	A comprehensive dataset with a deep learning approach (xDNN) achieved a promising F1-score of 97.31%
TCIA, Harmon al. [136]	Worldwide (China, Japan, Italy, etc.)	In total, 2724 CT scans from 2617 patients	Grad-CAM [69], Lung segmentation model [160]	COVID-19 pneumonia detection from chest CT using multinational datasets	For COVID-19 pneumonia ClassificationACC: 90.8%, SEN: 84%, SPE: 93%
CT dataset, Yan et al. [58]	China, Brazil	416 COVID-19 positive CT scans and 412 common pneumonia CT scans	Multi-scale convolutional neural network	To assist radiologists and physicians to perform quick diagnoses and mitigate the heavy workload	SEN: 89.1%, SPE: 85.7%, ACC: 87.5%
Yan et al. [90]	China	21,658 annotated CT images from 861 COVID-19 patients	Encoder decoder framework and PASPP-ASPP [185]	To segment lungs and COVID-19 infected regions	Lungs: DSC: 0.987, SEN: 0.986, SPE: 0.990COVID: DSC: 0.726, SEN: 0.751, SPE: 0.726
Wang et al. [142]	China	1136 training cases from 723 COVID-19 patients	Used segmentation and classification baseline models	Deployment of a real-time AI-based COVID-19 diagnostic system	On Test Dataset: SEN: 0.974, SPE: 0.922
Pneumonia CT Wang et al. [43]	Tianjin, China	1065 CT images of pathogen-confirmed COVID-19 cases	Modified Inception transfer-learning model	To apply for COVID-19 CT images screening	On Internal Validation: ACC: 89.5% with a SPE: 0.88, and SEN: 0.87On External Testing: Total ACC: 79.3% with an SPE: 0.83 and SEN: 0.67
Song et al. [172]	China	777 NCP, 505 BP, and 708 normal CT images	ResNet-50, FPN attention model [66]	Identification of COVID-19 infected patients by CT images	NCP and BP: AUC: 0.95, ACC: 0.86, SEN: 0.96NCP and No normal: AUC: 0.99, ACC: 0.94, SEN: 0.93
COVID-CT-MD, Afshar et al. [173]	Canada	CT scans of 171 COVID-19 patients, 60 patients with CAP, and 76 normal patients	The slice and lobe labeling processes focus on regions with distinctive manifestations	Assist in the development of advanced Machine Learning (ML) and Deep Neural Network (DNN) based solutions	A dataset, with COVID-19 and CAP cases, further accompanied with lobe-level, slice-level, and patient-level labels to facilitate the COVID-19 research
UESTC-COVID-19 Dataset, Wang et al. [88]	China	CT scans (3D volumes) of 120 COVID-19 patients	CNN, Self-Ensemble CNNs, MAE loss [186], Dice loss [110]	COVID-19 pneumonia lesion segmentation from noisy labels	Average Dice: 80.72%, RVE: 15.96%HD95: 17.12 mm
2500 CT images of COVID-19 Lung [174]	USA, China, and Italy	2500 CT images	Only CT scans	For public research	A data collection from previous publications
COVID-CT datasetXingyi Yang et al. [34]	Wuhan, China	349 COVID-19 and 463 non-COVID-19 CT images	Multi-task learning and self-supervised learning	Helpful in developing AI-based diagnosis models of COVID-19	F1-score: 0.90, AUC: 0.98, ACC: 0.89
Zaffino et al. [175]	Italy	62 CT volumes of 50 COVID-19 patients	Gaussian mixture model (GMM)	Lung region segmentation and lung tissue classification	Mean ± standard deviation coefficient of determination (R2): 0.72 ± 0.03Pearson correlation coefficient (C): 0.89 ± 0.01
COVID-19 CT scansMa et al. [123]	Israel	70 annotated COVID-19 cases	left lung segmentation, right lung segmentation, and infection segmentation	A three-level segmentation benchmark set up to promote the studies of annotation-efficient deep learning methods	Achieved average DSC scores of 97.3%, 97.7%, and 67.3%, NSD scores of 90.6%, 91.4%, and 70.0% for left lung, right lung, and infection, respectively
MIDRC-RICORD-1a, Tsai et al. [176] (a)	USA	31,856 CT images of 110 patients	Only CT scans with annotations and supporting clinical variables	Annotation or data augmentation efforts and evaluation of the examinations for disease entities beyond COVID-19 pneumonia	Achieve the stated objectives for data complexity, heterogeneity, and high-quality expert annotations
MIDRC-RICORD-1b, Tsai al. [176] (b)	USA	21,220 CT images of 117 patients	Only CT scans with supporting clinical variables	The first multi-institutional, multi-national expert annotated COVID-19 imaging dataset	Achieve the stated objectives for data complexity, heterogeneity, and high-quality expert annotations
HUST-19, Ning et al. [47]	Wuhan, China	19,685 CT slices from 1521 Patients	CNN frameworks, Inception Net V3 [75] and ChexNet [187]	Useful for diagnosis and management of patients with COVID-19	Negative Receiver: AUC: 0.944Mild Receiver: AUC: 0.860Severe Receiver: AUC: 0.884
MosMedData, Morozov et al. [124]	Moscow, Russia	1110 anonymous patient’s Chest CT scans	No framework/architecture is applied	High-quality dataset with binary pixel masks depicting regions of interest, useful for development and validation	AI system for diagnosing COVID-19 with SEN: 90%, SPE: 96%, AUC: 0.96
BIMCV-COVID19+ Iglesia et al. [177]	Valencian Region	163 CT imaging studies	Entities are localized with anatomical labels and stored in a Medical Imaging Data Structure (MIDS) format	Useful for academic research and education	A COVID-19 images dataset of available in an open format
COVID-19: CASISTICA RADIOLOGICA ITALIANA [125]	Italy	115 COVID-19 positive cases	Clinical data, including PCR status	Detailed case analysis, including age, address, treatment, and so on, is provided	COVID-19 positive cases with CT images, patients’ age, clinical history, illness experience, and final diagnosis
Eurorad COVID-19 [178]	Worldwide (India, Maldives, Arabia, etc.)	50 COVID-19 positive cases	Clinical data, including PCR status	To provide a learning environment for radiologists, radiology residents, and students worldwide	The detailed information with patients’ age, clinical history, imaging findings, differential diagnosis list, and final diagnosis
COVID-19-AR Desai et al. [179]	USA	23 CT studies performed on a total of 105 patients	A DICOM-based de-identified data stored standard format	To contribute samples from rural populations to the global research community	8/23 (35%) CT were negative for airspace opacification
COVID-19-CT-scan-dataset Surabhi et al. [180]	Unspecified	17,099 CT images	Only CT scans	For public research	A CT data collection
Houssein et al. [181]	Egypt	5500 non-COVID-19 images and 4044 COVID-19 images	HQCNN [66]	To develop such a model to predict and help COVID-19 in the early stages	ACC: 99.0%SEN: 99.7%
Loey et al. [182]	Not applicable	Utilized CGAN network and constructed 4425 images from [123]	CGAN [60]	To be used for a more extensive area of research	ACC: 82.91%, SEN: 7.66%SPE: 87.62%

Morozov et al. [124] presented a large-scale MosMed dataset containing 1110 chest CT scans. The data are categorized as either “with symptoms of COVID-19 (CT1-CT4)” or “without signs of COVID-19 (CT-0).” The dataset has some additional markers, from CT-0 (normal and non-viral pneumonia CT) to CT-4 (diffuse hyaline opacity, lung parenchyma involvement more than 75%). These corresponding markers can be used to classify patients automatically and to point out suspicious locations for radiologists in CT scans. Iglesia et al. [177] developed the BIMCV-COVID19+ experimental dataset, which includes COVID-19 CT images and additional clinical information. The images are stored in high resolution and entities are localized with anatomical labels in a medical imaging data structure (MIDS) format. Moreover, the dataset provides extensive information such as the patients’ demographic information, the type of projection, and the acquisition parameters for the imaging study.

Several other datasets have also been composed with supporting clinical information and are available for public research. For instance, the COVID-19 Casistica Radiologica Italiana database [125] includes 115 COVID-19 positive cases with detailed information such as age, past medical history, course of disease, symptoms, and CT imaging diagnosis. This dataset is helpful for providing hands-on experience to radiologists. Fifty COVID-19-positive cases were included in the Eurorad COVID-19 [178] database, together with the patients’ ages, clinical history, imaging findings, discussion, differential diagnosis list, and final diagnosis. The authors demonstrated that CT imaging plays an essential role in the diagnosis, treatment, and prognosis of the infection course, allowing the physicians to determine an adequate therapy and response. Desai et al. [179] published a collection of radiographic and CT imaging studies for rural patients who had tested COVID-19 positive. Each patient is described by clinical data, including demographics, comorbidities, selected lab data, and critical radiology findings.

Data augmentation–based datasets

Data augmentation is the process of generating new data examples for training a model. Data augmentation is a useful technique for providing more information from less data. Some authors have adopted data augmentation techniques to obtain more diverse data. Throat et al. [180], for example, used an augmentation method to create a COVID-19 chest CT dataset with non-COVID-19 and COVID-19 CT images. The dataset was amplified by different augmentation techniques, generating approximately 17,099 CT images. Houssein et al. [181] generated a dataset containing non-COVID-19 and COVID-19 cases in X-ray and CT images. The dataset was augmented using different augmentation techniques, generating about 17,100 X-ray and CT images. Loey et al. [182] built a small dataset and used a deep transfer learning (DTL) model to classify the COVID-19 cases. For this purpose, the authors enriched the dataset by using classical data augmentation and CGAN. They then used a classifier to predict the classification outcomes: COVID-19 or non-COVID-19.

There may be several datasets missing from our review. However, we covered the most important, extensively used, and basic chest CT datasets utilized for COVID-19 research to the best of our knowledge. We curated the most crucial and detailed information from each dataset. The selective information about each dataset is arranged in Table 4.

Discussions and future perspectives

Several studies and research projects have employed AI and deep learning in medical imaging diagnosis, such as with CT scans. Most of the COVID-19 diagnosis research has utilized classification and segmentation techniques. In general, the majority of the classification-based methods have relied on pre-trained CNNs and transfer learning strategies. The pre-trained models have faster training convergence, particularly when there is a small amount of data. Moreover, such models are more efficient and avoid overfitting due to multi-training procedures.

However, the typical algorithmic flow of a neural network involves adding the pooling layer after the convolution layer. Thus, there are more chances of losing important features in up-sampling, down-sampling, or average sampling. To avoid information loss, building the residual blocks (ResNet101, DenseNet201, etc.) with deeper networks can deepen the complexity of the prediction networks and may evaluate more non-linear data. Adding or designing such types of systems could extract more feature information and make classifiers more accurate. Another solution could be federated networks such as multi-network-based systems, which can extract more features—for example, incorporating models including VGG-16 or VGG-19 with ResNets to jointly extract lung features from the input CT images and perform classification tasks.

Many COVID-19 predictive models are segmentation-based because segmentation provides high accuracy, is computationally cheap, and is simple to perform and comprehend. However, segmentation remains challenging for COVID-19 diagnosis because (1) no generally accepted solution, (2) there is a continually growing number of different ROIs, and (3) there are substantial variations in the properties of ROIs. Furthermore, the segmentation algorithms are exaggerated due to the intensity of homogeneity, artifacts, and closeness in the gray level of distinct soft tissues [100]. The works proposed for COVID-19 infection region segmentation have primarily relied on U-Net and a few of its variants (Residual U-Net, 3D U-Net, Dense U-Net, U-Net++, etc.) because U-Nets are particularly useful (i.e., much faster to train) and generate highly detailed segmentation maps using minimal training samples [188].

To improve the segmentation outcomes for COVID-19 diagnosis, the flexible U-Net-based architecture can be exploited along with transformers and attention mechanisms (TransUNet) [189]. TransUNet architecture establishes self-attention means to cover the loss of feature resolution caused by transformers. In addition, TransUNet uses a hybrid CNN-transformer architecture to take advantage of both comprehensive high-resolution spatial information from CNN features and the global context encoded by transformers. U-Nets can also be combined with GANs [84] to achieve the same accuracy and precision as manual annotations when segmenting images [190]. The GAN framework creates new data and includes two networks—a discriminator and a generator—that compete against one another to enhance their performance. In addition to the aforementioned architectures, various other network configurations, such as cascading two or more U-Nets [191], could help to improve segmentation outcomes [188]. There has been less attention given to COVID-19 lesion quantification-based analysis. Most of the existing algorithms have attempted to segment the infected region and utilize it for simple classification tasks. Hence, there is a need for quantification-based segmentation procedures to provide real-time aid to the COVID-19 clinical practice.

It was rather challenging to differentiate between COVID-19 classification-, segmentation-, and detection-based methods. Most CT and X-ray research on COVID-19 detection has not interpreted detection according to the computer vision definition. Thus, chaotic contents were found while searching such research articles. It is known that, from a medical perspective, detection has the same meaning as diagnosis. From the computer-vision perspective, detection is an operation that includes localizing an object by a boxplot of any shape. If the medical view is considered the only definition criterion, COVID-19 detection would be regarded as COVID-19 case classification and COVID-19 infection segmentation. If the computer-vision perspective is considered to be the only definition criterion, it remains similar to COVID-19 infection segmentation to a certain extent.

Less attention has been given to utilizing computer vision–based detection and recognition algorithms to diagnose COVID-19. As a result, we included those methods where ROIs or lesions were localized for diagnosis using bounding boxes, contours, heat maps, or interactive colors. These are hybrid approaches (i.e., they consider both segmentation and classification practices) with visible and interpretable predictions. Most of the cited methods in Section 4 are fundamentally segmentation-based with lesion localization, visualized marks, and colored outputs. We recommend that computer and medical technologists use important computer vision technical terms while performing AI-based COVID-19 diagnosis research. Otherwise, we may continue to face contradicting research results. Therefore, COVID-19 detection should be defined as a function different from classification and segmentation: for example, an infection localization and detection operation in interactive graphics with specific, measurable scales and decisions. There is a broad research gap in the utilization of state-of-the-art computer vision–based detection algorithms for CT-based COVID-19 diagnosis.

The COVID-19 AI-based predictive models produce relatively higher quantitative outcomes (SEN, SPE, ACC, etc.). The reason for this lies in the datasets and biased evaluations. For example, most COVID-19 datasets have two divisions (i.e., training and test data) rather than three (i.e., training, validation, and testing). Even though the datasets have been constructed with three subsets, it is likely that the test data partitions have been used for the training (e.g., in the form of cross-validation, which is somehow familiar to the trained model), which cannot be regarded as unbiased evaluation. Thus, holdout test sets (which should not be used as cross-validation while training the model) are essential and will reduce bias in the trained model. Furthermore, there is a shortage of COVID-19 medical images and annotations; we noted few datasets that were utilized for segmentation.

Deep learning–based models are prone to overfitting if they do not have many training samples. Popular data augmentation solutions, such as rotation, cropping, and scaling, are applied to address the data scarcity problem; as more training samples are introduced, the overfitting problem may be alleviated. The new images produced through data augmentation don't provide any additional substantial information. However, apart from its gains, it might have the reverse effect and may cause data leakage because the test set could contain samples that are similar to those in the training set. To address this challenge, GANs could be utilized to create a variety of images in addition to the data augmentation process [192]. Besides introducing larger datasets, the dataset's quality and efficient labeling strategy are also essential. In addition, datasets with additional information also have the potential to facilitate the COVID-19 AI diagnostic models and research.

In general, CT plays a prognostic role in COVID-19 diagnosis. When AI predictive models are applied with CT images, they can provide both rapid and accurate results. Some of the examined studies and experiments suggest that AI has promising diagnostic performance for COVID-19 and is vital in assisting doctors in making timely decisions on patient isolation and treatment.

Conclusion

The most accurate COVID-19 diagnoses are based on laboratory tests, such as RT-PCR. Patients infected or suspected of being infected with COVID-19 are typically admitted to the hospital for diagnostic procedures. These laboratory tests entail procedures that can be time-consuming, have low sensitivity, and have a high risk of false-negative results. Similarly, a shortage of equipment and strict testing standards have hampered the timely and accurate screening of suspected cases. Thus, non-laboratory examinations, such as computer-assisted imagery analysis of chest radiography (X-ray) or CT scans, are used to examine the lung regions to diagnose COVID-19. Specifically, CT image findings have been important for creating machine-based techniques to diagnose COVID-19. Implementing these methods is essential to contain the virus, screen out vast numbers of suspected and confirmed cases, and ease patient management in hospitals. Therefore, many chests CT–based AI and computer vision strategies have been developed for COVID-19 diagnosis. In this review study, we have reported and categorized the most well-known literature regarding the computer vision tasks of classification, segmentation, and detection. We also discussed the most well-known datasets utilized for COVID-19 CT research. According to our review study, AI- and deep learning–based models have been widely used to extract COVID-19-related infections from chest CT images. AI-enabled CT imaging methods can assist in automating the diagnosis process and reshaping the workflow while minimizing patient interaction and improving doctors' and radiologists’ practices. Most of the COVID-19 diagnosis methods utilized classification techniques, while many COVID-19 prognostic models are based on segmentation approaches. There has been less attention given to computer vision-based detection algorithms for detecting COVID-19. A large number of diagnostic models produce relatively higher quantitative (SEN, SPE, ACC, etc.) outcomes due to their biased evaluations. There is a need for holdout test sets to avoid bias in evaluating a trained model. Furthermore, there is a shortage of diverse and well-annotated datasets with additional clinical information. For instance, few datasets are available to train and test segmentation-based COVID-19 models. Ultimately, our review study supports the necessity for further research on the given topic in the future.

Declaration of competing interest

The authors whose names are listed above declare no conflict of interest.