Artificial intelligence (AI) for medical imaging to combat coronavirus disease (COVID-19): a detailed review with direction for future research.

Since early 2020, the whole world has been facing the deadly and highly contagious disease named coronavirus disease (COVID-19) and the World Health Organization declared the pandemic on 11 March 2020. Over 23 million positive cases of COVID-19 have been reported till late August 2020. Medical images such as chest X-rays and Computed Tomography scans are becoming one of the main leading clinical diagnosis tools in fighting against COVID-19, underpinned by Artificial Intelligence based techniques, resulting in rapid decision-making in saving lives. This article provides an extensive review of AI-based methods to assist medical practitioners with comprehensive knowledge of the efficient AI-based methods for efficient COVID-19 diagnosis. Nearly all the reported methods so far along with their pros and cons as well as recommendations for improvements are discussed, including image acquisition, segmentation, classification, and follow-up diagnosis phases developed between 2019 and 2020. AI and machine learning technologies have boosted the accuracy of Covid-19 diagnosis, and most of the widely used deep learning methods have been implemented and worked well with a small amount of data for COVID-19 diagnosis. This review presents a detailed mythological analysis for the evaluation of AI-based methods used in the process of detecting COVID-19 from medical images. However, due to the quick outbreak of Covid-19, there are not many ground-truth datasets available for the communities. It is necessary to combine clinical experts' observations and information from images to have a reliable and efficient COVID-19 diagnosis. This paper suggests that future research may focus on multi-modality based models as well as how to select the best model architecture where AI can introduce more intelligence to medical systems to capture the characteristics of diseases by learning from multi-modality data to obtain reliable results for COVID-19 diagnosis for timely treatment .

Introduction

Infectious disease was identified in December 2019 in Wuhan, China, and it is known as COVID-19. It is caused by severe acute respiratory syndrome coronavirus 2 (SARS-CoV-2), and it is an ongoing pandemic. The number of people infected with COVID-19 increases daily with a double ratio as compared with the number of infected cases the day before as shown in Fig. 1a (WHO 2020a). About 23,901,406 cases of COVID-19 have been reported in more than 210 countries (as shown in Fig. 1b) with 819,134 deaths (21% of the percentage as shown in Fig. 1c) up to 25 August 2020. Coronavirus disease is highly contagious and is becoming a challenge for public health how to control of this disease in developed countries, as identified cases are increasing day by day. The World Health Organization (WHO) announced the outbreak on 30 January 2020 as a Public Health Emergency of International Concern (PHEIC), and on 11 March 2020 declared it a pandemic (WHO 2020b).

Fig. 1

a:Current increase of daily cases of COVID-19 (WHO 2020a). b:Current Total cases of COVID-19 (WHO 2020a). c:Number of deaths due to COVID-19 (WHO 2020a)

It was breaking news that the infected cases can be confirmed via tests (Ai et al. 2020), called the Reverse Transcription Polymerase Chain Reaction (RT-PCR). However, there are many areas without the accessibility of such RT-PCR tests. Besides, this laboratory test also has high false-negative rates due to difficulty in quality control during sample preparation. All those led to a worse situation that the quick spread of pandemic around the world hit the record. Thus, appropriate rapid and accurate image processing tools, especially for X-ray and CT-based imaging tools, are of great help to the physician. For instance, in Italy, the United States, and China, the majority of COVID-19 cases have been identified through the manifestation characteristics in the computed tomography (CT) images (Kumar et al. 2020; Kaur et al. 2019). Otherwise, suspicious patients, even without common COVID-19 symptoms such as fever, cough, and shortness of breath, were hospitalized or quarantined until the outcome of laboratory tests were clear (Liang and Yu 2020). Moreover, due to the high false positives rate of RT-PCR test, many suspect patients had to be tested several times for confirmed positive or negative results. Therefore, the acquisition of images such as X-ray and CT scans chest images play an important role in limiting viral transmission and the appropriate stages of treatment in the fight against the COVID-19 (Liang and Yu 2020).

Scientists have made significant contributions to the campaign of fighting against COVID-19, there is growing research findings, with each passing day, such as research reports and publications reported by both industrial and academic researchers. According to the biomedical research search engine named PubMed, 775 published research articles contain the word “CORONAVIRUS” in 2019 (Economist 2020). In the first 80 days of 2020, the number of published articles has increased from 775 to 1245 on contagious viruses. Similarly, significant research activities were booming after the 2003 Severe Acute Respiratory Syndrome (SARS) epidemic and the Middle East Respiratory Syndrome (MERS) epidemic in 2012. Both SARS and MERS are caused by a coronavirus. The world-renowned economists called “the science of the time” mentioned that such a scientific contribution (as shown in the Fig. 2) to acute respiratory syndrome disease would dramatically increase the vaccine’s development pace (Economist 2020).

Fig. 2

Presentation of research articles on research related to Coronaviruses

The COVID-19 medical image diagnostic pipeline consists of three steps, as shown in Fig. 3. The first step is the pre-scan configuration. This step contains instructions and assistance from the healthcare worker to adjust the imaging facilities according to clinical protocols. The second step is the acquisition of images. It is to take CT or X-ray images that are acquired by holding a single breath. After getting the raw images, the third processing step is involving the intelligent system aim at diagnosing the disease condition and severity (Shi et al. 2020).

Fig. 3

The Pipeline of COVID-19 Medical Image Diagnostic

The introduction of Artificial Intelligence (AI) to the field of medical imaging research (Soomro et al. 2020) has been seen a promising role in the diagnosis and prediction of the disease (Dargan and Kumar 2020; Goceri and Songul 2020; Dargan et al. 2020). Compared to traditional image processing techniques, AI-based image analysis techniques provide more accurate, efficient, speedy, reliable and reproducible information about the diseases. AI-based techniques for the diagnosis of COVID-19 are based on image analysis, image segmentation of infected lung regions, and classifications for clinical evaluation. These AI-based techniques have shown great potential to be commercialized. In fact, few techniques have reached the market already (Bullock et al. 2020). The standard AI models for diagnosing COVID-19 are shown in Fig. 4.

Fig. 4

Standard Model of AI-system for Diagnostic of COVID-19

This research work contains a comprehensive review of the role of medical imaging with AI techniques against COVID-19. This article includes sections on the role of intelligent imaging to combat COVID-19, then a discussion of AI-based algorithms for COVID-19 and we also discuss the databases publicly available. Finally, we explore the challenging issues and future work in this area.

AI-system based CT and X-rays images analysis

Healthcare workers are at high risk while patients are being treated. There are still reported cases that they can still be infected even with Personal Protective Equipment (PPE). It is due to the high contagion of COVID-19 even before patients showing any symptoms of COVID-19. Therefore the scientific community has recommended imaging facilities and their appropriate systematic flow which are very important to reduce the risks and save lives of the paramedics and nurses especially (Apostolopoulos and Mpesiana 2020). Next, we are focusing on two different methods of acquiring medical images that can be used to support the diagnosis.

Traditional imaging system: CT and X-rays images analysis

Medical images from chest X-ray and CT are mainly used for screening purposes in order to diagnose COVID-19. They are very important for paramedics and specialists to remain out of contact with the virus. To this end, a safer and most reliable automatic image acquisition method is required. But the traditional image-taking process is based on contact between healthcare personnel and patients, because healthcare personnel helps patients to position themselves correctly to capture the image (Xie et al. 2020; Narin et al. 2020). During this process healthcare workers have close contact with patients and will lead to high viral exposure. The main goal is not to create patient-healthcare contact, we need a fully automatic AI-based imaging system to avoid patient-healthcare contact.

AI-based imaging system: CT and X-ray images

To solve the problem of correct positioning of the patient during image acquiring and the contact between the patient and the healthcare personnel, AI and its sub-field methods provide automatic screening of patients wherein the position of patients is recognized (Singh et al. 2014; Li et al. 2007; Achilles et al. 2016), as image view with the help of visual sensors such as Red, Green, and Blue (RGB) image, thermal image, and Time-of-Flight (TOF) pressure images (Casas et al. 2019).

Various scanning parameters can be easily managed using AI-integrated imaging sensors. For instance, an AI-imaging visual sensor can be used to scan the various position of a patient with a pre-defined start and end points model. This model is based on patient position data to determine the scanning range. The main scanning positions include the neck, shoulders, elbows, knees, and correct wrist configuration. 3D modeling mechanics of the human body have been used in the selection of appropriate points to avoid radiation exposure (Cao et al. 2017; Rhodin et al. 2018).

Other parameters, such as the lighting conditions adjusted to the optimal International Organization for Standardization (ISO) (Booij et al. 2019) and the inferred AI values, are also important ISO centering is an important parameter for aligning the patient’s body so that the center of the body overlaps with the ISO scanning center to produce a quality image. Good ISO centering is proven to reduce the radiation dose and produce a quality image (Booij et al. 2019). The excellent results can be achieved by developing sparse modeling of the 3D mesh in 3D space to represent the digitized human body. A lot of research work (Loper et al. 2015) has been carried out to model the human body using AI-based techniques. The most recent works are presented by Li et al. (2019), where the model needs to be trained only once and can be applied to all the entire key points of the subject and is based on multimodal inference. Such kind of AI mechanics made big contribution to the image workflow systems implemented during this COVID-19 pandemic. With the help of an AI-based imaging system, health workers are able to monitor patients without any contact as there is also a need to prevent the spread of the virus in healthcare workers.

AI in Diagnostic of COVID-19

During this ongoing pandemic, patients suspected of COVID- 19 must seek for an urgent diagnosis and prompt treatment for early containment. Manual analysis of the medical images obtained by radiologists is a time-consuming procedure and is prone to some human error. Moreover, COVID-19 is a new deadly, and challenging disease that has symptoms almost similar to those of other infectious diseases such as Severe Acute Respiratory Syndrome (SARS). More domain knowledge is needed from expert and experienced radiologists for the precise diagnosis of COVID-19. Therefore, well-trained AI models can relieve human labor by learning radiologists’ knowledge and building in the computational models to ensure accurate and fast diagnosis. Generally, AI-based methods for diagnosis of COVID-19 consists of image segmentation and classification tasks on the CT or X-ray chest images. Figure 5 illustrates the framework for the AI-based methods.

Fig. 5

AI for diagnostic of COVID-19

Image segmentation for diagnostic of COVID-19

Segmentation is a fundamental but critical step in identifying any abnormal regions in medical images. It is also an essential step in AI-based diagnostic methods for COVID-19. For CT or X-ray images, the region of interest for COVID-19 is primarily the lungs, lobes, bronchopulmonary segments, and infected region or lesions. The accurate segmentation is a must for diagnosis and recommendation for timely treatment. CT images are high-quality 3D images that offer more information than X-ray images. Many machine learning methods used multi-channel models to detect the lungs from CT images. On the contrary, the segmentation of chest X-ray images is more challenging due to the ribs projected onto the soft tissue and gives a low to varying contrast problem. Thus, researchers are likely working on the segmentation of CT images for COVID-19 diagnosis, while X-ray images are mostly for the classification application. The following sections describe the state-of-the-art segmentation methods based on CT images as well as the biological nature of the CT-image for COVID-19 diagnosis.

CT-image segmentation for diagnostic of COVID-19

The CT images are utilized for the diagnosis of COVID-19, and normal CT images are called CT-non-COVID-19 images, and infected CT images are called CT-COVID-19 images as shown in Fig. 6.

Fig. 6

Analysis of CT-COVID-19 Images and CT-Non-COVID-19 Images

It is observed that the CT images were classified with few structural parameters, and these parameters were ground-glass opacities, consolidation, cavitation, or nodular opacities. The distribution of these pulmonary/lung changes was identified according to changes in the appearance of the lungs or lesions. The advanced stage of the disease processes over time. The distribution of parameters such as time and frequency plays an important role in analyzing the lungs evolved. Figure 7 shows the early-phase CT images and the late-phase CT images from COVID-19. It is observed that the disease spreads because the dilated vessels in the affected area, as shown in the yellow circle, as well as the oxygen saturated (O2-saturation), become low from high, and these observations from the CT images are like indicating COVID-19 infection.

Fig. 7

COVID-19 infection observation from early and late phase.a represents early phase and b represents late phase

The distribution of these lungs is divided into upper zone predominance, lower zone predominance and non-zonal predominance.

There is a peripheral or central predominance.

Analysis of lobes.

Presence of pleural or pericardial effusion.

Presence of enlarged mediastinal or hilar lymph nodes.

The AI-based COVID-19 image segmentation and processing procedures are to find the region of interest (ROI), where are lungs and lesions in the segmentation of CT images for the COVID-19 analysis. ROI-based segmentation methods for the diagnosis of COVID-19 can be classified into two types. These are methods oriented towards the lung region, and methods oriented towards the lung lesion. The lung region-oriented method is based on the separation of pulmonary regions such as the whole lung or the lung lobes, and it is obtained by the segmentation of the background regions in CT or X-ray images, and this step is known as a pre-processing step for the analysis of COVID-19 images. For example, many methods are implemented based on the two-step process for diagnosing COVID-19 and in which the entire lung region is segmented using machine learning methods. The lung lesion oriented methods are based on the separation of artifacts such as metal in the lung from the lung regions. This is one of the challenging issues because the lesions or the presence of any artifacts can be of variable size (usually small) and of different shapes and textures. Identifying the location of lesion regions becomes a challenging task due to the small size of the shapes and textures. Table 1 summarises the state-of-the-art segmentation methods for diagnosing COVID-19 on the basis of CT images.

Table 1

Image segmentation ,methods for diagnostic COVID-19

Authors	Modality	Technique	ROI	Highlights
Zheng et al. (2020)	CT	CNN:U-Net	Lungs	Deep learning method
Cao et al. (2020)	CT	CNN:U-Net	Lungs and Lesions	Deep learning method
Huang et al. (2020)	CT	CNN:U-Net	Lungs and Lesions	Deep learning method
Qi et al. (2020)	CT	CNN:U-Net	Lungs and Lesions	Machine learning model
Gozes et al. (2020)	CT	CNN:U-Net	Lungs and Lesions	Deep learning method
Chen et al. (2020)	CT	CNN:U-Net++	Lesions	Deep learning method
Jin et al. (2020)	CT	CNN:U-Net++	Lungs and Lesions	Deep learning method
Shan et al. (2020)	CT	CNN:VB-Net	Lungs and Lesions	Deep learning method
Tang et al. (2020)	CT	Commercial Software	Lungs and Lesions	Machine learning method
Shen et al. (2020)	CT	Threshold-based region growing	Lesions	Computer-aided quantification method
Xu et al. (2020)	CT	classic ResNet	Lesions	Deep learning method
Shi et al. (2020)	CT	Size-aware random forest (iSARF).	Lesions	Machine learning method
Tang et al. (2020)	CT	A random forest (RF)	Lesion	Machine learning method
Chaganti et al. (2020)	CT	Multi-scale deep reinforcement learning.	lesions, lungs and lobes	Deep learning method
Gozes et al. (2020)	CT	ResNet50	Lesions	Deep learning model
Rajinikanth et al. (2020)	CT	Threshold filter	Lesions	Image processing model
Ozkaya et al. (2020)	CT	VGG-16, GoogleNet and ResNet-50	Lesion	Machine learning method
A et al. (2020)	CT	Transfer learning	lesions	Deep learning method
Chen et al. (2020)	CT	Modified U-Net	Lesions	Deep learning method
Wang et al. (2020)	CT	CNN:U-Net	lesion	Deep learning method
Song et al. (2020)	CT	DRE-Net	Lesion	Deep learning method
Wang et al. (2020)	CT	COVID-19Net	Lesion	Deep learning method
Li et al. (2020)	CT	COVNet, RestNet50	Lung	Deep learning method
Alshazly et al. (2020)	CT	t-SNE/Grad-CAM	Lung	Deep learning method
Mukherjee et al. (2020)	CT	CNN:Tailored DNN	Lung	Deep learning method
Li et al. (2020)	CT	Stacked Auto-encoder	Lung	Deep learning method
Kuchana et al. (2020)	CT	2D CNN:U-Net	Lung	Deep learning method

Zheng et al. (2020) proposed a weak method for the computerized supervised detection of COVID-19, and the method was validated on the CT image dataset. Their method was based on patient-level for detecting COVID-19 either COVID-positive or COVID-negative, and they were the first to introduce such detection of COVID-19 using the AI-Algorithm. They addressed one of the main issues that some COVID-19 infected areas were not observed by professional radiologists, but their proposed method, called Deep convolutional neural network (DCNN) 3D model based on deep learning, was able to identify these areas. Their method is based on the use of extensive data augmentation, which is one of those efficient solutions applied to increase performance in deep learning (Goceri 2019), on the training of CT volumes to obtain the trained data. Subsequently, they used weakly supervised learning for the detection of COVID-19 either COVID-Positive or COVID-Negative. For an appropriate detection process, in their deep convolutional neural Network to Detect COVID-19 (DeCoVNet) method, they used a global pooling layer and temporally global pooling layer to technically manage the problem of weekly supervised detection of COVID-19. In order to observe certain areas of the COVID CT image, a pre-trained UNet was used to provide the lung masks to guide the learning of DeCoVNet. They tested their proposed method on 42,290 CT image cases out of 14,851 patients, the Area under Curve (AUC) of Receiver operating characteristic (ROC) was 0.959. However, there are still some limitations in their method, the network design and the training process need to be improved on the 3D segmentation network and use of accurate ground truth annotated by the experts. The generality of this method should be validated using databases from different hospitals rather than one hospital database being used.

Cao et al. (2020) have proposed a method based on the longitudinal progression of COVID-19 detection using deep learning at the voxel level for the segmentation of pulmonary opacities. For the training process, different CT images from 10 positive COVID-19 cases were used. Manual lung and pulmonary segmentation by experts was made as a benchmark, after which, the U-Net architecture based on a convolutional neural network was used to predict manual segmentation. The model was validated by analyzing the contrasting evolution of two confirmed cases of COVID-19 from Wuhan in China, and the promising quantitative analysis of these cases shows the potential of the deep learning method. However, it is necessary to continue studying with more validation in order to prove the performance of this method.

The segment of the pulmonary region and ground-glass opacity for quantitative validation are proposed by Huang et al. (2020). Additionally, monitoring COVID-19 progress is based on a deep learning tool from medical images such as CT images. They validated their method on the basis of data from 126 patients and analyzed the percentage of CT lung opacification from 126 patients. They observed that quantification of pulmonary involvement could be used to show the progression of COVID-19 disease and that it was worth finding and useful for future studies. There are still limitations to their study as defined below. Qi et al. (2020) have proposed CT a radiomics model based on machine learning. Their models have been implemented and validated for predicting the stay of patients in hospitals associated with Severe acute respiratory syndrome coronavirus 2 (SARS-CoV-2) infection. CT radiomics contain lesions, frosted glass opacities, and consolidation. These features were extracted using the U-net based algorithm to predict the hospital stay of patients. Total 1218 features were validated by experts and fed into two machine learning models: logistic regression and random forest. Excellent results were obtained by using multicentric cohorts for training and inter-validation on a set of independent data for the COVID-19 dataset. The limitations of their study are: First, the small sample size is used, the less diverse the samples are. This gives the lowest percentage of short-term hospitalizations of patients. Second, they have a semi-automated segmentation of lesions and this has an impact on the selection of outcomes. A large prospective multicenter cohort is required for the proper setting and testing of CT radiomics models based on machine learning.

There are no series of patients out of three sets of CT scans, so they cannot analyze systemic changes for evaluation.

According to ongoing research on COVID-19, the systematic confirmation of pulmonary opacities has not been validated as being directly linked to COVID-19’s pathological effects.

There are so many commercially available softwares that perform a quantitative lung assessment based on opacification percentage without the supervision of radiologists, but their proposed tool requires the supervision of radiologists.

Gozes et al. (2020) have proposed a U-Net based deep learning model for identifying the abnormalities in lung CT images as well as applying a commercial software to analyse nodules or focal opacities in 3D space. The proposed system achieved 95% accuracy for the identification of COVID-19 on the Chinese control and infected patients database. For a precise measurement of the opacity of a coronavirus patient, the method of rapid evaluation of high volumes of CT screening or thoracic diagnosis was used in different ways in order to obtain good performance. Their method is one of the successful methods for initial screening for the disease and tracking the progression of COVID-19, but it is working in a semi-supervised way and requires more validation for identification at a later stage of the disease.

Chen et al. (2020) has proposed a segmentation model based on U-Net ++ for the detection of COVID-19 using chest CT images. The model performs the segmentation of lesions based on COVID-19 images. The final result is based on COVID-19 and not COVID-19 lesions. They obtained CT images of 51 COVID-19 patients and 55 patients with other diseases were obtained to form their proposed model. The performances evaluated and their model reached 95.2% (precision), 100% (sensitivity), and 93.6% (specificity). For further validation, another database containing 16 viral cases of pneumonia and 11 non-neuropathic patients was used, and the proposed model can identify viral patients and 9 non-neuropathic patients. One of the main limitations of their model is that it cannot perform well on a large database. A significant improvement in the training model is needed for their model to function properly in this ongoing epidemic.

Jin et al. (2020) has implemented an AI system for rapid diagnosis of COVID-19, which involves a two-step pipeline for screening COVID-19 patients based on lung lesion region segmentation of CT images. The entire lung region was firstly segmented using the 3D U-Net ++ based segmentation network, then separated lung lesions such as mental and motion artifacts from the lung regions are used for positive or negative label classification. Because the lesions could be small and textured, it would be a difficult task for small localized lesions, and it is necessary to detect these small lesions effectively in order to make the method more robust. The study used CT images of 1,136 cases, including 723 COVID-19 positive cases and 413 COVID-19 negative cases. They used 3D U-Net ++ for segmentation and the classification model based on ResNet50, and their sensitivity and specificity obtained by the combined models 3D U-Net ++ and ResNet50 are 97.4 and 92.2% respectively.

Likewise, Shan et al. (2020) have implemented a segmentation model based on U-Net++ for the detection of COVID-19 from CT images. Proper segmentation of the lungs provides quantitative information that is important in the fight against disease progression and the observation of changes in the length of COVID progression during the treatment process. They proposed a method called VB-Net for the extraction of lung and lung infection regions. Their method provides precise quantification data for medical studies, including a quantitative assessment of progression and full prediction, as well as visualization of lesion distribution using a percentage of infections. They have achieved effective performances with the integration of the human loop strategy in the formation of the segmentation network based on VB-Net, this integration with the radiologists in the formation of the network provided more precise results. Their work is the first to be based on the human loop strategy but there are a few limitations in their work as follows: Tang et al. (2020) have implemented a Random Forest (RF) model for COVID-19 detection, based on CT images of 176 patients. Their method can not only detect COVID-19 but also determine the severity of COVID-19.

Validation of CT databases was collected in a single hospital, their method may not work on all COVID-19 patients in other geographic areas. The generalization of a deep learning system is necessary for subsequent validation of several databases.

Their system is designed to quantify only infections, but may not be used to quantify another type of pneumonia, such as bacterial pneumonia.

Their method has given good performance and can be improved to quantify the severity of other pneumonia using transfer learning.

Shen et al. (2020) have proposed a quantitative CT analysis method for COVID-19 severity detection. Total and individual lung CT scores for COVID-19 severe cases are higher than for non-severe COVID-19 cases. Their method segments lung lesions of the COVID-19 severity using a computerized algorithm based on a correlation test between organs. Their method can easily be used to assess lesions such as CT signs, consolidation of ground-glass opacity (GGO), and overt fibrosis of the disease. They found that the proportion of consolidation increased with the average density of lesions, and the proportion of GGO decreased with the average density of lesions. The limitations of this work included the retrospective nature of the study, the selection bias for severe COVID-19 cases, the small sample size, and the evaluation bias based on the radiologist-defined CT scores. In the future, their work may be improved by examining the correlation between quantitative CT parameters and clinical symptoms and laboratory clues would support clinical decision-making.

Xu et al. (2020) observed that many patients had a positive lung imaging test result but had no symptoms. Even these patients are not diagnosed as suspected or confirmed cases. COVID-19 CT imaging shows many distinct manifestations according to previous studies. The manifestations contain focal ground glass shadows, multiple consolidation shadows, and many other factors. The main objective is the precise identification of COVID-19 based on the IA method. They proposed a deep learning method to distinguish COVID-19 from another viral pneumonia disease. They used classic ResNet for feature extraction and their experimental results showed that the proposed method could better distinguish cases of COVID-19 from other diseases. Their method can be improved with the current segmentation with a better training model and verified on a larger dataset.

Shi et al. (2020) have proposed a machine learning method for the extraction of particular characteristics of COVID-19, intending to provide a potential application for COVID-19 screening based on CT imaging. The results were a sensitivity of 90.7%, a specificity of 83.3%, and an accuracy of 87.9%. These results have the potential to be used in screening systems. Their method is based on a size-aware and location-specific random forest learning framework (iSARF). There are certain limitations to their study. First, no clinical features related to pneumonia were presented to study the correlation between symptoms and radiological findings. This limitation can be overcome because more clinical features could improve the radiological findings of COVID-19 pneumonia. The second limitation is that the only baseline CT results from COVID-19 patients obtained by clinicians and the radiologist was included rather than following CT scans procedures, especially monitoring of disease progression is necessary for future work. Another limitation, the severe symptoms of COVID- 19 and other pneumonia diseases were not included in their study, and it should be studied in the future for further study validation. One of the advantages of their proposed system is that it is integrated into the uClound online service platform and is available in more than 20 clinical facilities in China.

Tang et al. (2020) have developed a random forest (RF) based method to assess the severity of COVID-19 patients. Their evaluation based on sixty-three quantitative features that contain the volume/infection ratio of the whole lung as well as the volumes of ground-glass opacity (GGO) regions was subsequently calculated from the CT images, these sixty-three trained features used as inputs to the RF model. The result is based on a non-severe or severe degree assessment. The 30 quantitative features obtained the best performances, and their performances are validated and achieved performances with a True Positive Rate (TPR) of 0.933, a True Negative Rate (TNR) of 0.745, and an accuracy of 0.875 and an AUC of 0.91. The limitation of this study is that two types of COVID-19 severity (severe and not severe) were used instead of four types (mild, common, severe, and critical) since their database contains a limited number of patients with mild and critical types of COVID-19 and this is one of the reasons why they classify the database into severe and non-severe cases. Mild and common cases were considered to be non-severe and as serious and critical were combined as serious. But sometimes mild cases lead to serious stages as well. For effective handling of COVID-19, the method of validation must be based on four cases.

Chaganti et al. (2020) have implemented a method to automatically quantify regions with ground-glass opacity and consolidation in the chest scanner using a deep learning algorithm. Abnormalities extracted from their system are the core results of chest CT scans found in COVID-19. The output of the deep learning method is connected to the 3D contours of CT abnormalities and based on the measurement of two degrees of severity of the disease called high opacity percentage (Percentage of High Opacity (PHO): PO (Percentage of Opacity) and PHO) and pulmonary opacity score high (Lung high opacity score (LHOS): LHOS and Lung severity score (LSS)). The performance is evaluated based on 100 COVID-19 databases and 100 CT scan control from several institutions in Canada, Europe, and the United States. The ground truth of samples was implemented by calculating the same measurements from manual annotations of the lesions, lungs, and lobes. The Pearson correlation coefficient between the method prediction and the ground truth was calculated as follows: 0.95 for PO(P<0.001), 0.98 for PHO(P<0.001), 0.96 for LSS(P<.001), 0.96 for LHOS(P<.001). The proposed method has a few limitations. The system has been trained with particular abnormalities related to COVID-19 and the presence of other abnormalities in the lungs poses challenges for their algorithm. The method can be improved from training stages as the training process can be carried out with all types of anomalies. The other main limitation is that they evaluate and quantify only COVID-19 cases and COVID-19 cases and healthy cases, but they have not evaluated other lung diseases including pneumonia, making it difficult to distinguish between other diseases and COVID-19 and the impact of this evaluation process on the diagnostic ability of the algorithm.

Gozes et al. (2020) proposed a weakly supervised deep learning method based on chest images. Results of COVID-19 and non-COVID-19 cases were obtained from the Chinese control CT image database, showing patients infected with 0.94 AUC. The proposed method contained a series of steps. The first step is to locate the pulmonary region of interest in the CT images of the human chest. The second step is based on a 2-dimensional (2D) ROI to identify the ROIs of the lungs as COVID-19 and non-COVID-19. The last step is based on unsupervised clustering for the final identification of COVID-19. But there is a limitation in their work because their database was just formed on the normal region (Non-COVID-19) and the abnormal region (COVID-19) but they have not classified the other viral pneumonia, therefore this can lead to false-positive identification of COVID-19 cases.

Rajinikanth et al. (2020) implemented a method for the detection of COVID-19 from lung lesions using CT images based on traditional image processing techniques. Their work is based on a series of steps such as threshold filtering, image enhancing, image segmentation, and extraction of area features to identify the pixel ratio between the lung and infections for the identification of COVID-19 and its severity. Their method gives an average performance, but they used a very small database called the Radiopedia database, which makes it difficult for them to identify COVID-19 against other diseases.

Ozkaya et al. (2020) have proposed a method based on the fusion and classification of the characteristics of COVID-19 identified at its early stage. It starts with the process of learning 150 CT images. The pre-trained CNN networks were formed separately for subset-1 and subset-2. Models VGG-16, GoogleNet, and ResNet-50 were used as a pre-trained network. Later, the features fusion and ranking method, after having been transformed into a support vector machine (SVM), had identified the COVID-19 and evaluated the performances. The performances obtained on the subset-2 with an accuracy of 98.27%, a sensitivity of 98.93%, a specificity of 97.60%, an accuracy of 97.63%, an F1 score of 98.28%, and Matthews Correlation Coefficient (MCC) measurements of 96.54%. There are a few limitations to their method. The main limitation is that lack of validation on the large database, and the other limitation is that it has not distinguished between viral pneumonia and COVID-19, which greatly impacts the diagnostic ability of the method.

Zahangir Alom et al. (2020) have implemented a new deep learning method for the identification of COVID-19 pneumonia from CT images. The method has a series of steps. The first step is to train the database and then using the transfer learning method on this database for the retraining of COVID-19 samples. Their method is based on the end-to-end detection of COVID-19 from image processing tactics of locating infected regions using deep learning techniques. Their method gave an accuracy of 98.78% over CT images. There are some limitations to their method. First, their model must be trained and tested with more annotated COVID-19 samples for better generalisability to make the model more robust and precise. Second, the paucity of samples labeled for lung segmentation provides results with false-positive detection. To avoid false positives, refined masks with transfer learning approaches are needed.

Chen et al. (2020) have implemented a method based on a deep learning technique for the automatic segmentation of COVID-19 CT images. They proposed a modified U-Net model by exploiting the residual network to improve the segmentation of features. Then, an efficient attention mechanism was built into the decoding process to achieve high-quality multi-class segmentation results. Their method improved multi-class segmentation by more than 10% over U-Net and a set of baselines. More work needs to be done to improve their method as there are some limitations in their method. Their method would be generalized in a wider range of practical scenarios. This can be achieved by facilitating the diagnosis of more types of disease from CT images. Because COVID-19 is a new disease and there are a few samples for validation and a knowledge-based model must be implemented to manage COVID-19 ground truth and to provide ground truth images differentiated from other diseases.

Wang et al. (2020) analyzed CT images based on inception transfer learning techniques. It is based on a retrospective, multi-cohort diagnostic study using a modified model. The proposed method achieved an accuracy of 0.895. Their screening method achieved a high sensitivity of 0.88 and 0.83 on some internal and external CT image datasets, respectively. In severe cases, their method had a better performance of 0.825 accuracy for distinguishing COVID- 19 from non-COVID-19 and other typical viral pneumonia due to similar radiological features. There are some limitations to their study, although revised inception model has been used to represent and learn the predictable COVID-19 results from many different databases, it has a low signal-to-noise problem, and in particular positive COVID-19 CT images which are difficult to detect due to a large number of variable objects. Especially the area of the CT image outside the lungs which is not relevant for the diagnosis of pneumonia. Besides, a training database is also small because the performance of the system is validated as the database volume increases. The large number of databases related to the progression of the disease of all pathological stages of COVID-19 and it is necessary to optimize the diagnostic system.

Song et al. (2020) have implemented a deep learning-based pulmonary CT diagnostic system to detect patients with COVID-19. Their method extracted automatically the radiographic features of new pneumonia, in particular the ground-glass opacity (GGO) from a radiograph. They developed a DeepPneumonia model to help doctors detect pneumonia-causing COVID-19 and locate major lesions. Their method is based on three steps. First, the extraction of the infected areas of the lungs and they used filter banks for lung segmentation with the lung itself to avoid noises due to different lung contours. The second step is based on the Details Relation Extraction Neural Network (DRE-Net) to extract the top-k details in the CT images and obtain the image level predictions. The last step is based on image-level predictions which have been aggregated to obtain a patient-level diagnosis. Their model is based on 88 patients with COVID-19 and 100 patients with bacterial pneumonia that were collected from 777 image databases. They achieved an AUC of 0.95 on DRE-Net and this is better performance compared to the other models such as VGG16, DenseNet, and ResNet. But still, more work is required to study their model on a large database.

Wang et al. (2020) have proposed a method for a fully automatic DL system using raw CT images for COVID-19 diagnosis and prognostic analysis. Their deep learning system for lung feature detection takes a two-step transfer learning strategy. First, they collected 4106 lung cancer patients with both CT image and Epidermal Growth Factor Receptor Precursor (EGFR) gene sequencing. Training is performed with a CT- EGFR dataset, the DL system that has learned hierarchical lung characteristics that may reflect associations between the chest CT image and micro-level lung functional abnormality. The second step is based on the collection of a large multi-regional COVID-19 dataset (n = 1266) from 6 cities or provinces to train and validate the diagnostic and prognostic performance of the Deep Learning (DL) system. Their method has yielded better performance, but their method has certain limitations. Their mode of data collection did not include some cases, such as death or admission to an intensive care unit. The second limitation is that they did not categorise the database according to disease progress stages such as normal to severe COVID-19. The performance of this model can be improved by separating the appropriate data into groups for training.

Li et al. (2020) have proposed three-dimensional deep learning system called COVNet for the diagnosis of COVID19 based on lung CT scans. The COVNet system is based on RestNet50, which uses set CT slices as inputs for the production of features. The authors have easily extracted both 3D global and 2D local representational features. The authors have obtained a large collection of CT images, including 1,296 COVID-19 CT samples. More specifically, in this study, 1735 community-acquired pneumonia and 1326 non-pneumonia CT images were obtained to serve control images and to make sure a robust diagnosis, as some common imaging features cannot distinguish COVID-19 and certain types of lung disease. The developed COVNet system on independent test data set revealed a high sensitivity of 90 percent and a high accuracy of 96 percent when diagnosing COVID-19. The AUC value of COVID-19 was 0.96. This study has one major limitation, although a large collection of CT scans were used in this study, all the training set data was obtained from a single hospital.

Alshazly et al. (2020) implemented advanced deep network architecture and proposed a transfer learning technique using tailored inputs to each deep architecture to deliver the optimum performance. The authors carried out detailed sets of experiments on two CT image datasets, namely the SARS-CoV-2 CT-scan and the COVID19-CT. Using the t-SNE algorithm, the authors analyzed the learned characteristics and the resulting visualizations showed well-separated clusters for both non-COVID-19 and COVID-19 subjects. The authors also analyzed the obtained networks using the Grad-CAM algorithm to obtain high-resolution visualizations showing the discriminatory regions of anomalies in the CT images. Their best models achieve F1 score, specificity, sensitivity, precision, and an average accuracy of 92.5 percent, 92.2 percent, 93.7 percent, 91.3 percent, and 92.9 percent, for the COVID19-CTT dataset, and 99.4 percent, 99.6 percent, 99.8 percent 99.6 percent, and 99:4 percent, for the SARS-CoV-2 dataset.

Mukherjee et al. (2020) have developed a lightweight (9-layer) CNN configured DNN that can simultaneously test/train both Chest X-rays and CT scans. The developed system showed a false-negative rate value of 0.02, an AUC value of 0.98, with an overall accuracy value of 96.29 percent. Furthermore, the authors achieved far better results while integrating Chest X-rays and CT scans for the diagnosis of COVID-19 infected subjects than conventional DNNs, like ResNet, MobileNet, and InceptionV3.

Li et al. (2020) developed a rapid and precise stacked autoencoder detector system for the tracing of COVID-19 subjects based on lung CT scans. The proposed system was an automatic complete solution requiring no manual feature extraction. In this model, four auto-encoders stacked layers were created to extract the enhanced features of the CT scans. The stacked layers were linked to another dense layer and a classifier called softmax. The authors also developed a novel classification loss method by overlaying reconstruction losses to improve recognition performance. Their test statistics demonstrated excellent performance based on a limited number of the COVID-2019 CT scans, and the model obtained F1-score rates, recall, precision, and average accuracy, of 94.8 percent, 94.1 percent, 96.54 percent, and 94.7 percent, respectively.

Kuchana et al. (2020) have proposed a two-dimensional deep learning framework with a U-Net backbone for the segmentation to identify lung spaces. The authors performed two segmentation tasks, first task segmented anomalies present in chest CT scan that were specific to COVID-19 infection, and the second task segmented lung space from CT slice. Both segmentation tasks led to a Chest CT scan forecast along with a volumetric evaluation to provide statistical information of the COVID-19-related abnormalities. The author obtained a Mean-Intersection over Union (M-IoU) value of 84.7 percent and an F1 score of 97.32 percent. The authors also have an excellent framework on how this technique can be incorporated into hospitals to prevent the rise of COVID-19. This study used lung CT image collections from Kaggle and GitHub for semantic segmentation.

Chest X-ray-image based COVID-19 diagnostic

X-ray images are commonly used for lung classifications as normal lungs or lungs infected with pneumonia due that the X-ray images are of gray pixels and less sensitive than 3D chest CT images. So X-ray images are used firstly to examine normal to mild COVID-19 cases. In general, abnormal chest radiographs are identified in 70% of patients in the early phase of admission and they become 80 to 85% in patients after admission to the ICU. For illustrations, normal and COVID-19 x-ray images are given in Fig. 8. Radiological signs include airspace opacities, ground-glass opacity (GGO), and subsequent consolidation. The predominant bilateral, peripheral, and inferior zone distributions are mostly seen clearly around 90% of chest x-ray images and these observations provided an excellent opportunity to perform classification of the identified COVID-19 disease. AI methods are used to classify COVID-19 among other pneumonia diseases. Table 2 shows the list of proposed methods for the diagnosis of COVID-19 based on the X-ray imaging modality.

Fig. 8

Analysis of X-rays-Non-COVID-19 Images and CT-COVID-19 Images

Table 2

Image segmentation methods for diagnostic COVID-19 from X-ray images

Authors	Modality	Technique	ROI	Highlights
Ghoshal and Tucker (2020)	X-ray	Bayesian CNN (BCNN)	Lungs	Deep learning method
Narin et al. (2020)	X-ray	ResNet50	Lungs	Deep learning method
Zhang et al. (2020)	X-ray	ResNet	Lungs	Deep learning method
Wang et al. (2020)	X-ray	COVID-Net	Lungs	Deep CNN
Apostolopoulos and Mpesiana (2020)	X-ray	Transfer learning based Model.	Lungs	Deep learning method.
Chowdhury et al. (2020)	X-ray	Preprocessing and deep learning techniques.	Lungs	Deep learning method
Farooq and Hafeez (2020)	X-ray	COVID-ResNet	Lungs	Deep learning method
Khalifa et al. (2020)	X-ray	Generative adversarial networks(GAN)	Lungs	Deep learning method
Hall et al. (2020)	X-ray	ResNet-50	Lungs	Deep learning method
Afshar et al. (2020)	X-ray	Capsule Networks	Lungs	Deep learning method
Li et al. (2020)	X-ray	COVIDMobileXpert	Lungs	Knowledge Transfer and Distillation (KTD) framework
Hammoudi et al. (2020)	X-ray	Tailored deep learning models.	Lungs	Deep learning method
A et al. (2020)	X-ray	Multi-tasking deep learning method	Lungs	Deep learning method
Karim et al. (2020)	X-ray	Deep Neural Networks (DNN)	Lungs	Deep learning method
Luz et al. (2020)	X-ray	EfficientNet family Model.	Lungs	Machine learning
Tartaglione et al. (2020)	X-ray	Preprocessing and deep learning techniques.	Lungs	Deep learning method
Oh et al. (2020)	X-ray	Patch-based convolutional neural network method.	Lungs	Deep learning method
Mahdy et al. (2020)	X-ray	Multi-thresholding and Support vector machine (SVM).	Lungs	Machine learning
Ozturk et al. (2020)	X-ray	Shrunken Features	Lungs	Deep learning method
Taresh et al. (2020)	X-ray	Pre-trained DL Models	Lungs	Deep learning method
DeGrave et al. (2020)	X-ray	Explainable DL Model	Lungs	Deep learning method
Sharma et al. (2020)	X-ray	Pre-trained DL Models	Lungs	Deep learning method
Jain et al. (2020)	X-ray	ResNeXt DL Model	Lungs	Deep learning method
Jaiswal et al. (2020)	X-ray	Mask-RCNN Model	Lungs	Deep learning method

Ghoshal and Tucker (2020) have implemented the deep learning method based on Bayesian Convolutional Neural Networks (BCNN) that can estimate the uncertainty in deep learning solutions to improve the diagnostic performance of COVID-19 from the chest x-ray database. They observed that the uncertainty of the prediction was strongly correlated with the accuracy of the prediction. There were 70 chest x-ray images of COVID-19 patients selected from the publicly available database while non-COVID-19 images were from Kaggle’s chest x-ray images. The results of the experiment show that Bayesian inference improves the detection accuracy of the standard VGG16 model from 85.7 to 92.9%. The authors have used salience maps to explain the locations of ROI based on deep networks to improve the process of understanding deep learning that led to a great support decision-making process.

Narin et al. (2020) have developed three different learning models to identify COVID-19 infection from X-ray images. ResNet50, InceptionV3 and Inception-ResNetV2 were used for the detection of COVID-19. The authors used the COVID-19 dataset and Kaggle’s chest X-ray images (pneumonia), and the database contains 50 images of COVID-19 patients and 50 images of non-COVID- 19 chest X-ray images. The experimental results conclude that the ResNet50 model provided the highest classification performance with an accuracy of 98% compared to an accuracy of 97% by InceptionV3 and 87% by Inception-ResNetV2.

Zhang et al. (2020) have implemented a ResNet model-based deep learning to detect COVID-19 from X-ray images. The proposed model has two tasks. The first task is the classification between COVID-19 and non-COVID-19, and the second task is to detect the anomalies. Anomaly detection was done to optimize the COVID-19 score for classification. There were 70 patients with COVID-19 and 1,008 patients with non- COVID-19 pneumonia obtained from two different databases. The method obtained performance with a sensitivity and specificity of 96.0 and 70.7% and an AUC of 0.952. There are few limitations in their method, such as they missed 4% of COVID-19 cases and almost 30% of the false-positive rate. This can be improved by reducing the false-negative rate. Potential cases of COVID-19 need further study for early treatment and require more clinical diagnostic information. More clinical data are needed to further improve the performance of their model.

Wang et al. (2020) have implemented the deep convolutional neural network model named COVID-Net for the detection of COVID-19 from X-ray images. The two databases used contain 5,941 chest X-ray images of 1203 healthy people and 931 patients with bacterial pneumonia, 660 patients with viral pneumonia, and 45 patients with COVID-19. The proposed method provided a better performance with a test accuracy of 83.5%. There are a few limitations in this method that can be improved. COVID-Net has achieved a good sensitivity but it is important to check the missing cases of COVID-19. The second observation is that COVID-Net has achieved a high Positive Predictive Value (PPV) for COVID-19 of around 98.9%, which indicated some false positive COVID-19 cases. This ratio creates a deceptive and high Continuous Positive Airway Pressure (CPAP), many false positives increasing the burden on the healthcare system, and it needs more PCR testing as well as additional patient care. Moreover, training methodology also requires significant improvement to reduce false positives.

Apostolopoulos and Mpesiana (2020) have implemented the method using transfer learning for the detection of COVID-19 from an X-ray database. The dataset is a collection of 1427 X-ray images, 224 images of COVID-19, 700 images are common pneumonia and 504 are non-COVID-19 images. Their method achieved an overall accuracy of 97.82%. There are certain limitations in their work that can be overcome. First, the larger database is required for further validation. Secondly, it is important to implement a model that can distinguish between COVID-19 and other viral cases such as SARS.

Chowdhury et al. (2020) have implemented a deep learning-based method to detect COVID-19 using chest X-ray images. A publicly accessible database contains 190 images of COVID-19, 1345 images of viral pneumonia, and 1341 normal images of chest X-ray images. They pre-processed the sample through data augmentation. The imaging set was increased to contain 2,500 images of each category for training and validated on four different pre-trained deep CNNs. These networks were tested for two-class classification: normal and COVID- 19, and normal, viral and COVID-19. Their method provided good performance with accuracy, sensitivity, specificity, and precision on the two schemes of 98.3%, 96.7%, 100%, 100%, and 98.3%, 96.7%, 99%, 100% respectively. There is still room for improvement as the proposed methods were tested on a small database and more model validation will be required on a large database.

Farooq and Hafeez (2020) have implemented a deep learning model named COVID-ResNet for accurate detection of COVID-19. Their method applied a three-stage training on a pre-trained ResNet-50 network. Progressive resizing in this paper fine-tunes the network gradually at multi-scale and multi-resolution levels. Particularly, the Cyclical learning rate technique is used to select the best learning rate automatically. The method has achieved 96.23% accuracy on all classification classes such as normal pneumonia, COVID-19, and others. Their model can help with the early detection of COVID-19 and reduce the burden on the healthcare system. There is still a lot of room for improvement in their method. First, they applied data augmentation to increase the training set but their method has not been validated on a large database. Secondly, their method has not been generalized to detect the COVID-19 from other viral diseases.

Khalifa et al. (2020) have implemented generative adversarial networks (GAN) with a deep transfer learning refined on a limited data set to detect COVID-19. The role of GAN here is to improve the robustness of the model and address the overfitting issue while generating more images from the database. There were 5,863 X-ray images of normal and pneumonia cases used. They only used 10% of the data for training and 90% of the generated images by the GAN to verify the capacity and effectiveness of the proposed method. Other models such as AlexNet, GoogLeNet, Squeeznet, and ResNet18 were also selected to predict pneumonia from chest x-ray images. These models reflect the need to reduce complexity and reduce the amount of memory and time consumed. The combination of the GAN and deep transfer models improved the efficiency of the proposed method. Resnet-18 is the most suitable deep transfer model with 99% accuracy compared to other deep transfer models. The paper compared four deep models in detail. However, the method needs to be validated further on larger database as well as the classification of diseases in COVID-19, Non-COVID-19, and other viral diseases expected.

Hall et al. (2020) have proposed that the pre-trained deep CNN called ResNet-50 to be tuned to 102 cases of COVID-19 and 102 other cases of pneumonia with 10-fold cross-validation. The method worked well and provided an overall accuracy of 90.7% with the COVID-19 TPR of 0.83. They ensembled three pre-trained models named ResNET50, VGG16, and their proposed model for training on a balanced set of COVID-19 and pneumonia chest X-rays images. They tested these CNN classifiers on 33 unseen cases of COVID-19 and 208 pneumonia and achieved an accuracy of 94.4%. The method allowed more validations on large databases containing COVID-19 images and images of viral diseases.

Afshar et al. (2020) proposed a framework based on Capsule Networks instead of CNN Model to address small-sized training sample issue. They called their network COVID-CAPS. The lost function was modified to account for the problem of class imbalance. Their method has shown significant performance in diagnosing COVID-19 with smaller dataset. Initial results based on the X-ray database provided the best performance over CNN models with 95.7% accuracy, 90% sensitivity, 95.8% specificity, and area under the curve (AUC) of 0.97. Their model still needs to be adjusted for larger datasets to verify its efficiency and accuracy of being a less-parameter model.

Li et al. (2020) have implemented the method named COVIDMobileXpert. COVID-MobileXpert is a lightweight, deep neural network-based mobile app that captures noisy snapshots of chest X-ray images and screens for COVID-19. Their method is based on a three-player Knowledge Transfer and Distillation (KTD) framework, including a network of pre-trained Attending physicians (PAs) who extract essential features of lung diseases and a fine-tuned Resident Fellow (RF) network to learn the extracted essential features for discriminating COVID-19 from viral illnesses, as well as a fine-tuned lightweight medical students (MS) to screen COVID-19 on the device. To get more accurate results, they used noisy snapshots, chest X-ray images, and loss functions, and training Schemes for the MS network to learn the robust imaging features for accurate COVID-19 screening from mobile. There is still room for improvement in their method, the three-player KTD framework can be implemented as a generalized model with more training on other devices for medical imaging classification and segmentation applications for the diagnosis of other related diseases such as lung infections and musculoskeletal abnormalities.

Hammoudi et al. (2020) have implemented a set of tailored CNN-based models to detect cases of pneumonia infection such as COVID-19 and other viral diseases from chest x-ray images. They designed three sets of image categories that are normal cases, viral pneumonia cases, and bacterial causes, and processed as input and expected output for each category. The trained models used CNN like backbones such as ResNet34, ResNet50, and DenseNet169 and a classifier of a fully connected head and a single hidden layer. Then a simple health indicator averaging the influence of age, comorbidity, and infection rate indicator was proposed to evaluate the aggravation risk to COVID-19 pneumonia. The experimental results gave an average accuracy of 84%. Some limitations can still be observed and need to be improved. Their model can detect COVID-19 and other viral diseases on a large chest X-ray database. Moreover, the reliability of their proposed model must be cross verified with RT-PCR tests and clinical tests prior to development.

Zahangir Alom et al. (2020) have proposed a fast and efficient way to detect COVID-19 using a multi-tasking deep learning method from chest X-ray images. Inception Residual Recurrent Convolutional Neural Network with transfer learning tactics was used to train with samples to determine COVID-19 and normal images from chest X-ray images. The proposed model achieved 84.67% accuracy from X-ray images. There are some limitations in their study that need to be improved. First, the COVID-Det X-ray model needs to be trained and tested with a large database of COVID-19 samples. From our point of view, their model needs to be trained and tested with more samples to make the model more robust, accurate, and generalized.

DeepCOVIDExplainer, a model proposed by Karim et al. (2020), is based on an explicable method of deep neural networks (DNN) for the computerized detection of symptoms of COVID-19 from chest X-ray images. They used 16,995 chest X-ray images of 13,808 patients containing normal cases of pneumonia and COVID-19. Their method pre-processed chest X-ray images before augmentation and classified with a neural ensemble model, and followed by gradient guided class activation maps (Grad-CAM++) and layer-wise relevance propagation (LRP) to highlight class-discriminating regions in CXR images. The experimental results provided a positive predictive value (PPV) of 89.61% and recall of 83%. However, the over-fitting is a potential concern for this model due to the small training dataset. Second, their method has not been verified for diagnostic and localization details from radiologists. Moreover, other modalities like CT or other decisive factors should be considered to improve the detection accuracy.

Luz et al. (2020) have implemented a modification of the models of the EfficientNet family to improve the accuracy of the models and the sensitivity of COVID-19 detections. They used image preprocessing and data augmentation that contains several preprocessing tactics to remove noise and outliers. After the EfficientNet family, they used transfer learning to pre-trained EfficientNets on the ImageNet dataset because the ImageNet domain is much larger than chest x-rays and it improves COVID-19 image detections from the COVID-19 dataset. They improved the overall accuracy by 91.4%, the detection sensitivity of COVID-19 to 90%, and the positive prediction by 100% while containing about 30 times fewer parameters than the base model, 28 and 5 times fewer parameters than VGG16 and ResNet50 models.

Tartaglione et al. (2020) have implemented a method based on deep learning methods for the detection of COVID-19 from chest X-ray (CXR) images. Their proposed method is based on the preprocessing of the CXR images and lung segmentation, followed by a classification deep learning model obtained from transfer learning. They did intensive work on the proper choice of the architecture to-be-trained to handling removable biases in medical datasets, as well as addressed the issue of small COVID data and proved the importance of processing large databases through transfer learning. They used large publicly non-COVID CXR databases. These databases are being collected from major emergency hospitals in northern Italy during the peak of the COVID pandemic. Better generalization of the proposed model needs to be validated on more new datasets in the same domain.

Oh et al. (2020) addressed an important issue related to difficulty in the systematic collection of CXR data that commonly experienced before preparing deep neural network training. They implemented a patch-based convolutional neural network method with a relatively small number of trainable parameters for COVID-19 diagnosis. The proposed method consists of three steps. Firstly, CXR images were processed for data normalization purposes. Secondly, the lung areas were extracted by a segmentation network, which was an extended fully convolutional DenseNets. Lastly, a ResNet-18 based network was customized using local and global patch-by-patch training and inference to classify the corresponding labels, after which the final decision was made based on a majority voting. The model was stably trained with a small dataset initiating from ImageNet pre-trained parameters. The model was able to capture the distinguishing feature for COVID-19 through checking the global distributed localized intensity variation. The model was compared against COVIDNet. The results were examined through interpretable saliency maps that were strongly correlated with the radiological results. The proposed method is sensitive to the lung mask obtained from the segmentation network. It can be further improved by validated on larger databases.

Mahdy et al. (2020) have combined a multi-thresholding and Support Vector Machine (SVM) for the detection of COVID-19 from CXR images. Their method was conducted in four steps. The first step involved data pre-processing based on the median filter, the second step was to segment infected lungs using multi-threshold techniques, then the support vector machine was trained and tested. Lastly, the performance of the classification was measured. With a small database of COVID-19 CXR images, the proposed method achieved an average sensitivity of 95.76%, a specificity of 99.7%, and accuracy of 97.48%. However, such a high accuracy does not mean the proposed method is tangible for COVI-19 detection from CXR images. The proposed method is very time-consuming. A better model with a combination of deep learning should be sought for improved performance.

Ozturk et al. (2020) have implemented the method based on the classification of COVID-19 from CXR images using Shrunken Features. Due to the limited amount of samples available in the database, they used the hand-crafted method rather than deep learning methods. Their proposed method firstly extracted four hand-crafted feature vectors, including the grey level co-occurrence matrix(GLCM), local binary GLCM, grey level run length matrix, and segmentation-based fractal texture analysis(SFTA) features. Then the high dimensional feature vectors were reduced by applying a stacked auto-encoder and principle component analysis(PCA). The condensed feature vectors were used by SVM to generate the correct labels for unknown inputs. The proposed method achieved very promising results with an accuracy of 94.25%. The proposed method shows the good capability to deal with imbalanced sample data with help of replicating oversampling and augmentation. However, more intensive work needs to be done to deal with the inter-class imbalance issue. A larger database with balance and more labeled COVID-19 data is expected to make the CNN model work in a better way.

Taresh et al. (2020) performed an AI-based method-based case study for the detection of COVID-19 from chest x-ray images as they used the pre-trained deep learning models such as InceptionV3, Exception, InceptionResNetV2, MobileNet, VGG16, DenseNet169, NASNetLarge and DenseNet121. The main objective of validating the pre-trained models to select the best classification model for COVID-19. They used publicly available databases to validate their case study. The database contained 274 cases of COVID-19, 380 viral pneumonia, and 380 healthy cases of chest x-ray images. The confusion matrix is calculated from the pre-train models for the post-classification validation of the models. They used the open-source library named PyCM for the calculation of statistical parameters. Their case study proved that the VGG16 model performed much better than the other models in terms of overall scores of 99.69. There is still room to improve their case study, and their case study needs more detailed analysis, as such an effective pre-trained model has been trained on large databases and more training on deep learning gives a more useful generalization model for the detection of COVID-19.

DeGrave et al. (2020) proposed the robust COVID-19 deep learning systems, and their proposed model is based on deep learning systems to detect COVID-19 from chest x-ray images based on confounding factors rather than medical pathology. Because many systems seemed accurate, but they failed when tested in new hospitals. They sought to generalize the system for each database, and the proposed approach was performed on the training data of AI systems to introduce an ideal scenario for the AI to learn ideal shortcuts for accurate detection of the COVID-19. The results of their study provided explainable solutions to alleviate the learning of shortcuts in AI systems for detecting COVID-19 from chest x-ray images. They provided the solution to improve training data collection, as the data in which chest x-ray images are collected and processed to match the target population of a future AI system, and improved the choice of the prediction of tasks that involve more clinically. The authors confirmed that shortcut learning plays an ideal role for data collection, but the explainable AI solution provided external validation, and their solution can be adopted by AI methods for further validation and more precise detection of COVID-19.

Sharma et al. (2020) proposed an AI method to identify and monitor COVID-19 patients and recommended the appropriate time according to clinical management. Their model is based on deep learning models with trained chest x-ray images for the timely screening of COVID-19 patients. They used publicly available databases and these databases contain adult patients with COVID-19 for the development of AI-based classification models for COVID-19 and other diseases. major infectious diseases. They performed 25 types of augmentations on the databases to increase the size of the database and implement the generalized model. They used a transfer learning approach for training and testing their classification models. They performed well by combining two top-performing deep learning models, and each model is trained on 286 and rotated from an angle of 120 – 140, and thus increase performance, and gave the highest accuracy on normal images, COVID-19, non-COVID-19, pictures of pneumonia and tuberculosis. More validation is needed on their methods as they used a limited number of chest x-ray images for COVID-19 and use fewer images that had chances of overfitting classification models, and the model training process will be improved using a large database to develop more robust and suitable classification models.

Jain et al. (2020) proposed a deep learning-based model for the detection of COVID-19 from a PA view of chest x-ray images of patients with COVID-19. After categorizing the chest X-ray image databases, they used the data augmentation, then they used the Inception V3, Xception Net, and ResNeXt deep learning model and compared their performance. 6432 chest x-ray samples were collected in the Kaggle repository, of which 5467 were used for training and 965 for validation and comparison of the performance of DL models. The Xception model achieved the highest accuracy of 97.97% compared to other models. But the higher accuracy causes the overfitting, and it can be overcome by testing a larger number of chest x-ray image datasets.

Jaiswal et al. (2020) implemented a deep learning model for the identification and localization of COVID-19 from chest x-ray images. Their model is based on the Mask-RCNN model, and it is a deep neural network that incorporates global and local features of images for pixel-wise segmentation. They trained the RSNA Pneumonia Dataset to analyze the affected features and used post-processing steps that merge the bounding boxes (lung opacity) from multiple deep learning models to rank the features. Their study is very useful in identifying pneumonia and providing information on understanding the role of lung image size as it directly depends on the performance of the AI model. As they validated this distinction is quite subtle for pulmonary images for the presence or absence of pneumonia, and a large image may be more informative for a deeper understanding. On the other hand, the computational cost is one of the main problems in processing the large image size. They used thresholds in the background while training the network to process a larger image. Also, they used image augmentation, dropout, and L2 regularization to prevent overfitting, but still got weak results on the training set as per the required for testing the model. It is still possible to improve their model by adding new layers and adjusting the hyperparameters according to the model architecture.

COVID-19 computed tomography based testing and risk analysis

Complex radiological characteristics in COVID-19 chest CT scans have been recorded and summed up as four stages (Pan et al. 2020), namely early stage(0–4 days), progressive stage (5–8 days), peak stage (9–13 days), and absorption stage (> = 14 days). In short, the initial stage occurs about four days after the start of the primary symptoms. Ground-glass opacity (G-GO) can be identified under the pleura bilaterally or unilaterally within the lower lobes in the second stage. In the progressing stage, which occurs in about eight days, only diffuse G-GOs, complex patterns, and even accumulation can be observed in bilateral sub-lobes. In the apex stage, which occurs in about 13 days, dense accumulation is far more pronounced.

Identification of COVID-19 subjects from uninfected subjects

How to identify COVID-19 victims efficiently from non-COVID-19 subjects including specific patients of pneumonia and non-pneumonia etc. has been documented in a variety of reports. S. Jin et. al. studied 414 negative COVID-19 and 724 positive COVID-19 subjects based on chest CT images. They developed a scheme including a classification approach based on ResNet50 and a segmentation approach based on U-Net++ (Jin et al. 2020). Jin et al. (2020) examined chest CT images of 1385 negative and 496 positive cases individually. The authors proposed a 2D CNN scheme for the segmentation of lung images and then labeling pieces of positive COVID-19 patients. Empirical results indicate that the scheme being proposed obtained an AUC value of 0.97, sensitivity value 94%, and an accuracy value of 95.5% (Jin et al. 2020). Likewise, Zheng et al. have proposed a CNN U-Net+3D based scheme for training and testing 229 negative and 313 positive COVID-19 chest CT images respectively. The U-Net was employed for the segmentation of lung regions before sending to the CNN for forecasting the likelihood of COVID-19. The reported CNN U-Net+3D scheme obtained an AUC of 0.95, a specificity of 91.1%, and a sensitivity of 90.7% (Zheng et al. 2020). In another work (Chen et al. 2020), through the aid of AI tests, radiologists’ reading time has been reduced by 65%, with 93.6%, 100%, and 95.2% values of specificity, sensitivity, and accuracy respectively for COVID-19 classification. In Chen et al. (2020), authors used U-Net++ based segmentation scheme for segmenting COVID-19 associated lesions, the training data input was obtained from CT images from 50 COVID-19 subjects and 54 subjects with other ailments.

Identification of COVID-19 subjects from other lung fevers subjects

Due to the resembling radiological appearance of common lung fever (pneumonia), with COVID-19, their distinction is essential in supporting the evaluation process in medical care. Therefore, various techniques have been proposed for distinguishing between viral pneumonia and COVID-19. Shi et al. (2020) used chest CT images of 1030 subjects of viral pneumonia and 1657 subjects of COVID-19. In the early phase of processing, a VB-Net (Shan et al. 2020) was used to separate the image into 18 pulmonary, 5 lung lobes, and right/left lung segments. Several handcrafted features were extracted and employed for training based on the random forest algorithm. Experimental results depicted that the subject with minor infections had a small sensitivity to be recognized. Moreover, 87.9%, 83.3%, and 90.7% values of accuracy, specificity, and sensitivity were obtained. Likewise, L. Li et. al. employed a large number of Chest CT data, including 4355 images from 1326 non-pneumonia, 1735 viral pneumonia, and 1297 COVID-19 subjects. A ResNet50 scheme was used for 2D segmentation with collective weights and joined with max-pooling to differentiate COVID-19 from non-pneumonia and viral pneumonia. Experimental results achieved 0.96 of AUC, 96% of specificity, and 90% of sensitivity values respectively. Xu et al. (2020) employed A deep learning algorithm related to V-Net on chest CT data from 176 healthy subjects, 225 subjects with Influenza A, and 218 subjects with COVID-19. It was first employed to separate subject infection locations. The portion of the infected area was fed into the ResNet18 network along with the relative infection distance of features from the reference corner. This deep learning-based model obtained an accuracy of 87% (Xu et al. 2020). Song et al. (2020). proposed a deep learning CT diagnostic technique to recognize COVID-19 cases from subjects with pneumonia and subjects with good health. Training and testing data were based on chest CT images obtained from 87 healthy subjects, 100 good health subjects, and 88 with COVID-19 subjects. The system obtained excellent outcomes with an accuracy of 94% for pneumonia tracing and 86% for pneumonia classification (Wang et al. 2020). In another work, a CNN algorithm for the distinction of COVID-19 and common viral pneumonia was proposed (DeGrave et al. 2020). Slices were obtained from 3D CT images and used as input for developing the CNN model. The CT images were obtained from 56 common viral pneumonia subjects and 44 COVID-19 subjects. The model achieved 74%, 67%, and 73.1% values of sensitivity, specificity, and accuracy respectively.

Severity evaluation of COVID-19

In addition to early diagnosis, the severity evaluation is also crucial for patient care. Several studies have been carried out that focused on the severity evolution of COVID-19. VB-Net, a deep learning model was implemented to split the lung into anatomical sub-locations of segments and lobes, depending on which infection concentrations and area were determined for each anatomical sub-locations, and employed as quantitative attributes for training the random forest model. Findings demonstrated an accuracy value of 88%, with the true negative rate, and a true positive rate of 75% and 93% respectively (Shi et al. 2020). Likewise, Tang et al. (2020). presented a random forest model for the evaluation of severity (severe or non-serious) of COVID-19. Chest CT scans of 176 infected COVID-19 subjects were used (Tang et al. 2020). In summary, several reports for computed tomography-based COVID-19 assessment are presented, with interesting findings. For the future phase, COVID-19 testing work may promote prevention and assist radiologist’s judgment. Furthermore, the estimation of severity is of considerable significance, which may assist to measure the ICU case or clinical care planning decision, which needs further examination.

Overall analysis of AI-based methods for COVID-19 detection

The study of AI-based COVID-19 detection methods is important for potential automatic and precise clinical diagnosis and timely and appropriate treatment. However, implementing AI-based methods for COVID-19 at this stage is still not mature enough and facing some challenging issues. The AI-based methods on both CT images and CXR images are briefly explained in the Sects. 3 and 4 . Aiming at pre-diagnosing deadly and highly contagious COVID-19 as quickly and precisely as possible, monitoring the disease progressive development monitoring is important. But currently, limited research work has been done in this regard.

Given the quick outbreak of COVID-19, many methods are based on machine learning or deep learning models with limited ground truth databases. In our investigation, the CNN U-Net model and other DL models were very popular on the detection of COVID-19 from chest CT images or X-ray images, as summarised in Table 1. But detection of COVID-19 from x-ray images relies on few methods different from deep learning techniques such as Capsule Network and Shrunken features, even though these methods have limitations from the clinical perspective. How to transfer these working models into commercial software to fight COVID-19 became a high prioritized job for industrial companies all over the world. More importantly, the biological structure and feature of the lungs should be taken into account due that lung CT images or X-ray imaging modalities have low contrast infected regions and vary in both forms as well as the position of lesions in different patients. The main biological observation by AI-based methods is critical to properly handle the quantification and localization of the infectious region in the chest CT scan or chest x-ray. Nowadays, an intelligent imaging company named uAI has developed a deep learning-based software for the computerized segmentation of the infected regions with a coronavirus and detecting CT features for COVID-19 assessment through considering the distribution volume of infection in the lungs.

The above state of the art has shown the great efforts of researchers to come up with the diagnostic solution for COVID-19 based on AI methods. These studies are still at the initial stage but have made a big contribution to combating COVID-19 using modern technology. It provides a platform for researchers in this area that share and develop solutions that quickly diagnose the virus at an early stage and recommend early treatment as well as a follow-up treatment to prevent lung damage. These studies based on AI-based lung extraction, AI-based quantification, and AI-based assessment could be one of the most important tools for clinical monitoring methods based on the AI for the diagnosis of COVID-19.

Discussion and future work

AI has played an important role in the analysis and the diagnosis based on medical imaging, especially the detection of brain tumors, and eye diseases. At present, AI has played a vital role in the implementation of the COVID-19 imaging-based diagnostic system. Various image sources have been used to diagnose COVID-19 such as CT scan images and CXR images. Over a short period of six months, AI researchers have come up with many effective methods to diagnose COVID-19 images, but there is still room for development in the future. AI has proven to be a powerful tool for image acquisition to make scanning procedures efficient and safer by protecting medical personnel from COVID-19. The main objective of empowering AI for the diagnosis of COVID-19 from CXR and CT images is to facilitate better quality scanning and reduce radiation exposure to patients. It is very crucial to use the appropriate amount of radiation during the scan and to keep a minimum dose of imaging.

The current medical images have evidenced a negative radiological pattern in the early stages of the disease. The representation of the radiological sign in the early stage is very significant with an uncertain clinical diagnosis. The scarcity of available data may lead to overfitting of the results when applying AI methods on medical images for segmentation and classification tasks. For future studies and correct clinical evaluation, the quality of the samples with larger datasets needs to be developed. Therefore, more data sets should be generated and shared for proper medical assessments.

In this paper, we have investigated proposed AI methods for detecting COVID-19 from CT scans and X-ray images. Machine learning techniques and deep learning models, in particular, have become more effective methods for combating COVID-19. These proposed methods have the following limitations, and these limitations need to be carefully considered and addressed for future studies.

Imaging data such as CT images and CXR images are incomplete without proper labeling. Moreover, images with poor labels can not provide accurate lung segmentation and classification and can result in the wrong diagnosis of COVID-19.

Several reported methods are not validated on diverse COVID-19 databases. Other methods did not classify COVID-19 into proper and meaningful classes like COVID-19, Non- COVID-19, and viral diseases. It is very crucial to develop a system equipped with proper classification ability.

Many methods are only validated for CT images, but there are no proper classifications for CXR images. A multi-modality based system can take the best advantage of each type of modality, with the hope that a more efficient and generalized system can be developed for better medical clinic purpose.

Almost all of the proposed methods are weakly supervised methods as the manual labeling of imaging data is time-consuming and expensive. Unsupervised deep learning models and the transfer learning methods are required to conduct the imaging data.

Multidisciplinary research areas can be integrated, such as medical imaging, imaging fusion, natural language processing, and could be beneficial for the overall process of tracing COVID-19.

Concluding remarks

Coronavirus disease (COVID-19) has spread throughout the world so quickly. It influenced society significantly. Medical imaging has proven to be an important tool in observing the progress of the disease for early treatment. Over the past six months, AI techniques have been extensively booming and deployed for the speedy diagnosis of COVID-19. This review article presented the importance of AI methods with an intelligent imaging solution for fighting against COVID-19 as well as other viral diseases. CT and CXR images are two common imaging modalities that are used for the clinical diagnosis of COVID-19. We have comprehensively reviewed CT and CXR based AI-methods for the diagnosis of COVID-19.

AI and machine learning applications have boosted the accuracy of Covid-19 diagnosis from CXR and CT images, and most of the widely used deep learning methods have been implemented and worked well with a small amount of data for COVID-19 diagnosis. It is important to note that these images only provide partial information about patients infected with COVID-19. It is necessary to combine clinical experts’ observations and information from images to have a reliable and efficient COVID-19 diagnosis. For this proposition, future research may focus on multi-modality based models where AI can introduce more intelligence to medical systems to capture the characteristics of diseases by learning from multi-modality data to obtain reliable results for COVID-19 diagnosis and suggest timely treatment.