Deep Learning and Medical Image Processing for Coronavirus (COVID-19) Pandemic: A Survey.

Since December 2019, the coronavirus disease (COVID-19) outbreak has caused many death cases and affected all sectors of human life. With gradual progression of time, COVID-19 was declared by the world health organization (WHO) as an outbreak, which has imposed a heavy burden on almost all countries, especially ones with weaker health systems and ones with slow responses. In the field of healthcare, deep learning has been implemented in many applications, e.g., diabetic retinopathy detection, lung nodule classification, fetal localization, and thyroid diagnosis. Numerous sources of medical images (e.g., X-ray, CT, and MRI) make deep learning a great technique to combat the COVID-19 outbreak. Motivated by this fact, a large number of research works have been proposed and developed for the initial months of 2020. In this paper, we first focus on summarizing the state-of-the-art research works related to deep learning applications for COVID-19 medical image processing. Then, we provide an overview of deep learning and its applications to healthcare found in the last decade. Next, three use cases in China, Korea, and Canada are also presented to show deep learning applications for COVID-19 medical image processing. Finally, we discuss several challenges and issues related to deep learning implementations for COVID-19 medical image processing, which are expected to drive further studies in controlling the outbreak and controlling the crisis, which results in smart healthy cities.

Introduction

The Coronavirus disease (COVID-19) pandemic and its related efforts of containment have generated a worldwide health crisis impacting all sectors of human life. At its initial stage of inception, with the number of people affected by the disease being minimal, it did not reflect threats of such enormous capacity wherein the majority of the cases were resolved spontaneously. With gradual progression of time, COVID-19 was declared as an outbreak by the World Health Organization (WHO) with an extremely high-risk potential of affecting millions of lives in all countries, especially ones with weaker health systems. The virus is deadly due to two basic reasons- firstly, it is novel with no vaccines discovered, and secondly, it is easily transmitted through direct or indirect contact with the affected individual.

The statistics of COVID-19 reflect reasons for immense concern, with almost 44,748,380 being affected globally, with 1,179,035 patients losing the battle and succumbing to death as on October 29, 2020 (WORLDOMETER, 2020). To add more to the horrific statistical figure, United States of America (USA) being one of the leading flag bearers in healthcare advancement, record highest number of COVID-19 victims followed by Brazil, India, Russia, South Africa and the list goes on till 215 countries across the globe. The total number of COVID-19 diagnosed cases in the USA alone is 9,120,751, with 5,933,212 cases of recovery and 233,130 total deaths as on October 29, 2020 (WORLDOMETER, 2020). The number of new cases being reported every single day has been increasing at an accelerated rate, compelling governments, and administrative authorities across the globe to impose a non-compromising lockdown to ensure social distancing for the containment of the disease.

The global response to interrupt spreading of COVID-19 has been prompt and unanimous wherein the majority of the affected countries have sealed their borders barring traveling and transportation services. The WHO and Center for Disease Control (CDC) and Prevention have issued structured guidelines to be followed by general citizens, governments, national and international corporations to ensure complete containment of the disease, thereby breaking the chain leading to this pandemic. The global strategy for COVID-19 response as framed by the WHO include five steps: (1) mobilization of all sectors of human life to maintain hygiene and social distancing, (2) controlling of sporadic cases to prevent community spread, (3) suppressing community transmission by imposing relevant restrictions, (4) providing healthcare services to reduce mortality, and (5) development of vaccines and therapeutics for large scale administering. Fig. 1 shows the transmission of COVID-19.

Fig. 1

Transmission of COVID-19.

WHO and CDC have identified and enlisted symptoms that indicate plausible COVID-19 infection, and the symptoms include fever, dry cough, vomiting, diarrhoea and myalgia. The general public of all countries have been made conscious of the same to be responsive to seek treatment at the earliest in order to reduce morbidity rates. Governments have initiated to generously and enthusiastically invest in COVID-19 vaccine and related research. To add this initiative, an enormous amount of research and development activities are being directed pertaining to the COVID-19 pandemic. Machine learning (ML) and deep learning (DL) approaches have been a predominant choice for various disease detection (Zhang, Yang, Chen, & Li, 2018). The image processing technique has gained immense momentum in all sectors of healthcare, especially in cancer detection in smart cities (Khan, Asif, Ahmad, Alharbi, & Aljuaid, 2020). Hence these approaches have been a natural choice for COVID-19 research as well. The present study focuses on highlighting the contributions of DL and medical image processing techniques to combat the COVID-19 pandemic presenting an extensive review of the state-of-the-art frameworks developed by employing these technologies.

State-of-the-arts and contributions

In a desperate attempt to combat the COVID-19 pandemic, researches have been initiated on scientific studies in all directions, and DL integrated with medical image processing techniques have also been explored rigorously to find a definite solution (Hakak, Khan, Imran, Choo, & Shoaib, 2020; Iwendi et al., 2020). Numerous research publications have been published with similar objectives, as shown in Table. 1 . The uniqueness of the present work lies in its effort to emphasize significant DL and image processing techniques proposed in the detection of COVID-19 and also to highlight challenges associated with such implementations in order to open specific dimensions of future research, which are yet to be explored or thought of. The approaches discussed are elicited from various published articles of reputed publishers and thus help to enlist a serious set of recommendations for the research community and also administrative authorities to combat the disease.

Table 1

Deep learning implementations in COVID-19 datasets.

Ref.	Dataset	Methods Used	Evaluation Metrics	Research Challenges
Kermany et al. (2018)	Optical Coherence Tomography (OCT) image dataset	DL framework using transfer learning	Accuracy, Cross-Entropy Loss,True Positive Rate and False Positive Rate	Use of image dataset from varied sources to ensure generic usability of the proposed model not included in the study
Rajaraman, Candemir, Kim, Thoma, and Antani (2018)	Anteroposterior Chest X-ray dataset of children (1–5 years) from Guangzhou Women and Medical Center, China	Customized CNN model- VGG16	Accuracy, AUC, Precision, Recall, Specificity, F-Score and MCC	Random Sampling Process Used. Reliability of the model in highly non-linear locality pertaining to predictions not considered in the study
Wang, Kang, et al. (2020)	CT Images from Xi’an Jiaotong University First Affiliated Hospital, Nanchang University First Hospital, Xi’an No.8 Hospital of Xi’an Medical College	AI and DL based Framework	Accuracy, Specificity and Sensitivity	Analysis of the relationship between Hierarchical features of CT images and genetic, epidemological information not included in the study
Shan+ et al. (2020)	Chest CT scan Image dataset from Shanghai Public Health Clinical Center	Human-In-the-Loop Strategy, DL Based Segmentation Network - VB Net	Dice Similarity Index, Pearson CorrelationCoefficient, Time of Manual Contouring, Segmentation Accuracy	Quantification of Imaging Metrics and establishing correlation between syndromes, epdemicology and treatment responses not included in the study
Ghoshal and Tucker (2020)	X-ray Images of Posterior-Anterior (PA) part of the lungs from Dr. Joseph Cohen's GitHub repository augmented with Chest X-ray Images from the Kaggle Dataset	Dropweights based Bayesian Convolutional Network	Prediction Uncertainty and Accuracy	Evaluation of the results with traditional state-of-the-art models not performed. Consideration of ‘Omics’ dataset not included for better insights on image markers
Apostolopoulos and Mpesiana (2020)	X-ray Image dataset from GitHub and Cohen, Image dataset from Radiological Society of North America (RSNA), Radiopaedia, and Italian Society of Medical and Interventional Radiology (SIRM)	Transfer Learning based on CNN	Accuracy, Sensitivity and Specificity	More in-depth analysis using larger datasets and development of models capable of distinguishing between COVID-19 and other viral infectious diseases
Huang, Han, et al. (2020)	CT Image Dataset from Tongji Hospital, Wuhan, China	Convolutional Neural Network (CNN) Architecture	Opacification Percentage	A structured reasoning on the impact of COVID-19 viruses on the opacities not included
Hemdan, Shouman, and Karar (2020)	X-ray Image Dataset from Dr. Joseph Cohen and Dr. Adrian Rosebrock	COVIDX-Net comprising of Deep CNN Models	Accuracy, Precision, Recall and F1-Score	More in-depth analysis on larger datasets missing
Narin, Kaya, and Pamuk (2020)	X-ray Images from Dr. Joseph Cohen GitHub Repository	Deep CNN Models - ResNet50, InceptionV3 and Inception-ResNetV2	Accuracy, ROC and Confusion Matrices	Implementation of the CNN Models on larger datasets to enhance classification performance not considered

One of the major issues in COVID-19 research is the lack of reliable and adequate data. Due to the limitation in the number of tests conducted, multiple death cases and virus affected cases are being left out unreported. It is difficult to even comment if the failure factor in detecting COVID-19 infection is three, three hundred, or even more. Across the globe, none of the countries have been successful in providing reliable datasets pertaining to the existence of the virus in a representative sample of the mass population. But research and development activity cannot stop, and thus information fusion plays an extremely important role. The technical definition of information fusion, as per the text contents, mentions that “it is the process of combining and associating information from one or multiple sources to provide useful information for the detection, identification, and characterization of a particular entity”. In ML and DL applications, the availability of large-scale, high-quality datasets plays a major role in the accuracy of the results. Information fusion helps to integrate multiple datasets and use them in the DL models to achieve enhanced accuracy in predictions. As an example, computed tomography (CT) images from Xi’an Jiaotong University and Nanchang First Hospital and Xi’an No.8 Hospital have been integrated as part of Information fusion to be fed into the AI and DL models (Wang, Kang, et al., 2020). Similar information fusion has been observed in Ghoshal and Tucker (2020) where X-ray images of the lung from Dr. Joseph Cohen's GitHub repository have been augmented with Chest X-ray images available from the publicly available Kaggle repository. In Apostolopoulos and Mpesiana (2020), X-ray image datasets from GitHub, Cohen, Radiology Society of North America (RSNA), and Italian Society of Medical and Interventional Radiology (SIRM) were associated and used fed into the CNN for detecting COVID-19. In the later sections of the text, similar references can be visualized pertaining to applications of information fusion in order to fill the lag of data unavailability and still continue to generate predictions of enhanced quality.

It is important to understand that the pandemic is at its peak where existing medical facilities are overwhelmed. The emergency departments, intensive care facilities have been stretched beyond their regular capacity to serve the ever-growing population of patients. In such a crisis, the healthcare providers and also the patient family members need to make rapid decisions with minimal information. The phenotype of the COVID-19 disease starts with mild or no symptoms at all, yet rapidly changes its course to making patients extremely critical with even fatal outcomes suffering from multi-organ failures. The objective is to reduce such abrupt deteriorations, detect the disease at the earliest using the limited knowledge base and resources. The traditional lab-based RT-PCR (real-time polymerase chain reaction) test using the nose- throat swab has limited sensitivity and is also time-consuming. When the number of patients is huge, shortages of RT-PCR reagents and specialized laboratory resources for performing COVID screening tests are inevitable. The need for tools that help to augment the resources is thus an absolute necessity. ML and artificial intelligence (AI) are techniques that have the potential to enable accelerated decision making and improve patient-centered outcomes. Various studies have developed ML models to serve the same purpose using minimal resources resulting in significantly comparable accuracy (82–86%) and sensitivity (92–95%) as the gold standard test RT-PCR (Brinati et al., 2020). The use of AI and ML model could be coupled with radiological images leading with more accurate detection of the disease at an earlier ensuring availability of treatment. Models like DarkNet, detect and classify COVID-19 cases from pneumonia and serve the purpose for locations where there are shortages of radiologists due to an overwhelming number of patients (Ozturk et al., 2020).

Impact of DL based COVID-19 detection for sustainable cities

After conducting an extensive background study, it is evident that there is not many surveys conducted emphasizing the applications of DL frameworks and image processing in the prediction of COVID-19 cases. The present pandemic situation across the globe has impacted millions of lives. Thousands and thousands of people are getting affected by this highly contagious disease leading to questions on survival and sustainability of the human race (Megahed & Ghoneim, 2020). The only way to contain the disease is to detect the disease at its initiation, barring others from getting infected. This requires accelerated diagnosis without associated health hazards. The traditional approaches fail to provide the same due challenges pertinent to detection time, cleaning needs after each use of the diagnostic machinery and availability of resources. The use of ML approaches eliminates these issues and also detects faster. ML approaches, if used more predominantly, can lead to containment of the disease and reduce mortality.

The paper thus provides comprehensive information on various DL implementations in COVID-19 using real-time as well as publicly available image datasets. The unique contributions of our study are mentioned below:

The survey includes basic information on COVID-19 and its spread, which establishes the motivation and need for accelerated disease prediction ensuring containment of the disease in smart cities.

The role of DL applications in medical image processing is discussed in detail in support of its capability in COVID-19 predictions.

The recent works on DL and image processing implementations in COVID-19 are discussed explicitly.

The datasets, methodologies, evaluation metrics, research challenges, and the lessons learned are included from these state-of-the-art research works in addition to the future directions in controlling the pandemic in smart cities.

Paper organization

The rest of this work is organized as follows. Section 2 presents fundamental information on COVID-19, DL and expresses the general motivation towards the adoption of DL to process and analyze medical images from the existing literature. An overview of DL applications in medical image processing is presented in Section 3. Section 4 presents a focused review of potential DL implementations for medical image processing in COVID-19. Section 5 presents three use cases of plausible DL-based implementations for COVID-19 image processing. Section 6 summarizes the aforementioned reviews highlighting the lessons learned and enlisting the recommendations guiding towards the future direction of research. The paper is concluded in Section 7.

Background and motivations

This section presents the fundamentals of COVID-19, DL, and an overview of the adoption of DL to process and analyze medical images from the existing literature.

Overview and status of COVID-19 outbreak

At the outset, multiple pneumonia cases were being registered in the Wuhan city of the People's Republic of China (PRC) towards the end of 2019 (Pham, Nguyen, Huynh-The, Hwang, & Pathirana, 2020). The COVID-19 index case was identified in Wuhan, Hubei province, PRC in December 2019. The COVID-19 was identified and declared as an infectious virus initiated by acute respiratory syndrome coronavirus 2 (SARS CoV-2). The investigations revealed that the source of COVID-19 was likely from Huanan Seafood Market in Wuhan city, and eventually, the government of PRC officially declared an additional 27 cases by December 2019. Based on several experiments, the researchers observed that the infection was transmitted from wild bats (Andersen, Rambaut, Lipkin, Holmes, & Garry, 2020). This virus falls under the category of Beta-coronaviruses (beta-CoV), which consists of SARS coronavirus (SARS-CoV). The authors in Unhale et al. (2020) noted that the COVID-19 virus epidemic started in the spring carnival in PRC, where many people from across the globe traveled to participate in this event and gathering of this huge mass from several countries across the globe catalyzed the virus not only to spread within China but was carried across International boundaries to other countries.

The China country office, Regional Office for the Western Pacific, and Headquarters of WHO have been working rigorously in analyzing the effect of COVID-19 from the first week of January 2020 (WHO, 2019). In the last week of January, the officials of WHO announced the outbreak as Public Health Emergency of International Concern (PHEIC).

The authors in Kampf, Todt, Pfaender, and Steinmann (2020) observed that the spread of COVID-19 is minimal in the regions with high temperatures and humidity. The authors also highlighted the use of steam therapy for reducing the threat of coronavirus. The steam inhaled traverses in the respiratory glands to the alveoli and aids in improving the oxygen levels. The authors in Shereen, Khan, Kazmi, Bashir, and Siddique (2020) inferred that the parameters for growth or spread of the virus depend on the environmental conditions, hygiene, water droplets of respiration, and physical contact.

WHO has come up with an effective strategy on 3rd February 2020 to combat coronavirus (WHO, 2020), which provides guidelines and protocols for health workers, doctors, Government officials, etc. to combat the COVID-19 crisis and work efficiently as front line workers ensuring personal safety. One of the strategies would be to test all the suspected cases, isolate them, and find their contact and travel history. Another important strategy given by the WHO is to follow lockdown which can be an effective way for virus containment. The guideline report (W.H. Organization, 2020) released by WHO on 19th March 2020 mentions the fact that the infection prevention was based on the knowledge gained from the case history of patients who have suffered from severe acute respiratory syndrome (SARS). The report also focuses on following certain precautions such as usage of the medical mask by the public in COVID-19 affected areas, cleansing the hands with alcohol-based solutions or hand-wash, and the health care personnel are instructed not to touch the patients with bare hands. There is no specific medication available for COVID-19 as per WHO to date. United Nations Conference on Trade and Development (UNCTAD) has consolidated a list of best practices, guidelines (UNCTAD, 2020) that nations can follow to run the essential services which can help the growth of the economic condition during COVID-19. The world economic forum has proposed the use of digital Foreign Direct Investment (FDI) (WEFORUM, 2020) to accelerate the financial growth in developing countries.

Fundamentals of deep learning

DL and neural networks (NN) have gained immense momentum in present day scientific research since they have the capability to learn from the context (Alazab et al., 2020; Gadekallu, Khare, et al., 2020; Reddy, Parimala, Chowdhary, Hakak, & Khan, 2020; Schmidhuber, 2015). These two techniques have been widely used in various applications such as classification and prediction problems, image recognition, smart homes, self-driven cars, object recognition, etc due to their capability to adapt to multiple data types across different domains. Fig. 2 depicts various techniques used in DL. DL replicates the functioning of human brain in filtering information for accurate decision making. Similar to the human brain, DL trains a system to filter the inputs using different layers to aid the prediction and classification of data. These layers are like layered filters used by the neural networks in the brain where each layer acts as a feedback to the next layer. The feedback cycle continues until the precise output is obtained. The precise output is formed by assigning weights in each layer, and during training, these weights are adjusted to get the accurate output.

Fig. 2

Various techniques and applications of deep learning.

DL techniques can be categorized as supervised, semi-supervised, and unsupervised. In supervised learning, the model is trained with a known input-output pair. Each known value constitutes an input vector and the desired value, which is referred to as the supervisory signal. The method uses existing labels to predict the labels of the desired output. Classification methods use supervised learning (Patel et al., 2020) and can be applied to scenarios to identify faces, traffic symbols, recognizing spam in a given text, converting speech to text, etc.

Semi-supervised learning is an in-between technique of supervised and unsupervised ML methodologies. The training data in Semi-supervised learning consists of labeled and unlabelled values. Semi-supervised learning falls between unsupervised learning and supervised learning. The unlabeled data, when used in conjunction with a small amount of labeled data, can produce a considerable improvement in learning accuracy. There exist certain scientific assumptions related to DL techniques (Cheng, 2019). The first being, data in proximity to each other have the same label. Second is the cluster assumption, where the data in the cluster share the same label. The third being, the data is restricted to a limited dimension rather than the complete input space. Unsupervised learning deals with knowing the inter-relations among the elements of the data set and then classifying the data without using labels. Some of the algorithms following these techniques are clustering, anomaly detection, and NN. Clustering is the principle of identifying similar elements or anomalies in a data set (De Simone & Jacques, 2019). This anomaly detection of unsupervised learning is widely applied in security domains (Lima & Keegan, 2020).

Most of the DL techniques use Artificial Neural Network (ANN) for feature processing and extraction. Feedback technique is used for the learning mechanism (Shanmuganathan, 2016) where-in each level updates its input data to form a summarized representation. The term deep in DL technique refers to the number of layers required for the data to be transformed. A Credit Assignment Path (CAP) is used during this transformation process. In the case of a feed-forward NN, the depth of CAP is calculated by the number of hidden layers in addition to the number of output layers.

In the case of a Recurrent Neural Network (RNN), there might be more than one signal which traverses multiple times in a layer, and thus the CAP depth cannot be determined (Yu, Si, Hu, & Zhang, 2019). One of the predominantly used techniques of NN for image processing is CNN (Gadekallu, Rajput, et al., 2020; Huynh-The, Hua, Pham, & Kim, 2020; Rawat & Wang, 2017). In CNN, the feature extraction technique is automated and is performed during the training on the images making DL the most accurate method for image processing domains. RNN works similar to CNN, but the difference is that RNN is used for language computation. RNN uses the concept of feedback loops where the output of one layer is fed as the input of the next layer. RNN can be used for datasets which involve time-series (Che, Purushotham, Cho, Sontag, & Liu, 2018), text, financial data, audio, video, etc.

Generative Adversarial Networks (GANs) works on the concept of the generator network and the discriminator. The generator network produces fake data while the discriminator differentiates fake and real data. These two networks work towards improvising the training process, and thus GANs are mostly used in an application that requires the generation of images (Greenspan, Van Ginneken, & Summers, 2016) from the text. Google's inception network introduces inception block to compute convolutions and pooling operations that run simultaneously for the effective processing of complex tasks. This is an advanced level of DL used in automating the responsibility involved in image processing (Alom, Yakopcic, Nasrin, Taha, & Asari, 2019).

DL can be applied in varied domains, which involve the processing of a vast set of data. DL has great potential in smart cities as a huge amount of data will be generated in smart cities due to digitization (Bhattacharya, Somayaji, Gadekallu, Alazab, & Maddikunta, 2020; Habibzadeh, Nussbaum, Anjomshoa, Kantarci, & Soyata, 2019). The evaluation of DL techniques relies on two parameters: firstly, the enormous amount of data size to be processed and, secondly, the massive computational power. DL also aids in the faster analysis of complex medical images (Greenspan et al., 2016) for rendering an accurate diagnosis. DL is popularly implemented in the health care sector for broad data interpretations (Razzak, Naz, & Zaib, 2018), aiding early diagnosis of diseases, thereby reducing manual workload. The following section provides an overview of DL applications for medical image processing.

Overview of DL applications for medical image processing

Advances in medical science have significantly changed health care over the last few decades, allowing doctors to identify and treat diseases more effectively (Sahiner et al., 2019). But doctors, similar to any human beings, are also prone to errors. The scholarly credentials of a doctor lie not only in the individual's level of intelligence, but the way they treat the problems of patients and the associated type of health system that supports them (Lundervold & Lundervold, 2019). This combination caters to such wide variations in clinical outcomes, and ML, in this regard, is the best solution for improving the strength of a doctor in diagnosing and treating patients (Huang et al., 2019). The effectiveness of ML algorithms depends on the types of features extracted and data representation. ML algorithms primarily face two key challenges, one being efficiency in scanning all high-dimensional datasets and secondly training of the model to find the most appropriate task (Ghesu et al., 2016a; Krawczyk, Minku, Gama, Stefanowski, & Woźniak, 2017). DL has been one of the commonly used techniques that guarantees a higher degree of accuracy in terms of disease prediction and detection. Applications of DL techniques have introduced new breakthroughs in the field of healthcare, as shown in Fig. 3 . In the real world, numerous sources of medical data, including Magnetic Resonance Imaging (MRI), X-ray, Positron Emission Tomography (PET), Computerized Tomography scan (CT scan), have provided doctors vast volumes of information (Liu, Liu, & Wang, 2015; Rehman, Zia, et al., 2020; Shen, Wu, & Suk, 2017). CNN is one of the most preferred algorithm popularly used in image processing and analysis (Huynh-The et al., 2020). The authors in Litjens et al. (2017) reviewed various DL methods for medical image processing and have inferred the use of DL in object identification, image categorization, segmentation, etc. In the medical domain, DL for image processing is used in various departments such as ophthalmology, neurology, psychotherapy, cancer detection, and cardiology. The authors have also enlisted the unresolved research challenges in DL relevant to image analysis. In the current scenario of patients and medical stakeholders maintaining electronic records, AI has aided easing the medical image processing. The authors in Ker, Wang, Rao, and Lim (2017) reviewed various AI techniques that can be implemented for medical image analysis. The authors from diverse literature found that CNN has been widely used for this analysis, along with big data techniques for processing. The authors also highlighted the main challenges of the unavailability of high quality labeled data for better interpretation.

Fig. 3

Applications of DL in medical image processing.

Classification

Classification is often termed as Computer-Aided Diagnosis (CAD). Classification plays a significant role in medical image processing. During the classification processing phase, one or even more images are taken as input samples, and a single diagnosis factor is generated as an output which classifies the image (Gao, Li, Loomes, & Wang, 2017). In 1995, the authors in Lo et al. (1995) used DL to classify lung nodules. The detection procedure involves 55 chest X-ray images, two deep-neural hidden layers. Using this test, the radiologist noticed 82% of lung nodules. In Shen, Zhou, Yang, Yang, and Tian (2015), the author's used multi-scale DL approaches to identify lung nodules in CT images. The experimentation process comprises of three hidden layers, which take the CT images as input and provide a response to the output layer of the lung nodule. In Rajpurkar et al. (2017), the authors introduced a CheXNet DL model with 121 convolution levels, 1,12,120 chest X-ray images provided input dataset to diagnose 14 different forms of lung diseases. Using this examination, the radiologist states the CheXNet algorithm exceeds the range of F1-metric efficiency. In Pratt, Coenen, Broadbent, Harding, and Zheng (2016), the authors developed a model by training 10-layer CNN with three completely integrated layers on around 90,000 fundus images to diagnose Diabetic Retinopathy (DR). Experimental tests attain 95% sensitivity and 75% accuracy on 5000 testing images. Another related research in Abràmoff et al. (2016) employed IDx-DR version X2.1 to train 1.2 million DR images for identifying DR. Results indicate that the built design can achieve a 97% sensitivity and 30% increase in specificity. The work in Kawahara and Hamarneh (2016) proposed a multi-layer CNN for the classification of skin lesions. The multi-layer CNN is trained with a variety of high-resolution images. Results from the publicly available dataset of skin lesions reveal that the proposed model achieves a better accuracy rate than the other existing models.

Localization

In the classification, the images are fed to CNN, and the contents of the image are revealed. After the image classification is done, the next step in the detection of the disease is the image localization, which is responsible for placing the bounding box around the output position, which is called as classification with localization, the term localization here refers to figuring out the disease in the image. The localization of anatomy is a crucial pre-processing phase in a clinical diagnosis that enables the radiologist to recognize certain essential features. During recent years, several research works have been conducted using DL models to localize the disease. For example, in Roth et al. (2015), the authors presented a model for the classification of organs or body parts using a deep CNN. The CNN was trained with 4298 X-ray of 1675 patients to recognize the five organs of the body's legs, abdomen, neck, lungs, and liver. The experimental results produce a 5.9% classification loss and 0.998 Area under the curve (AUC). Another active research in Guo, Gao, and Shen (2015) introduced a model to locate T2 MR prostate images. Localizing accurate prostate has several obstacles, including thickness variations and inconsistent appearance. The authors used Stacked Sparse Auto-Encoder (SSAE) to train prostate images, and the trained features highlight the identity of the prostate in the image. The study was performed on the dataset containing 66 images of prostate MR, and the findings are positive, providing better performance than existing models. The work in Shin, Orton, Collins, Doran, and Leach (2012) trained Dynamic Contrast-Enhanced MRI (DCE-MRI) 78 images of patients’ livers and kidneys using a DL approach. Three different datasets were used for localizing multi-organ disease. The proposed model as a whole is more accurate for the localization of diseases in heterogeneous organs. In Payer, Štern, Bischof, and Urschler (2016), the authors used 3D CNN for landmarking in medical images. Spatial Configuration-Net (SCN) architecture was used to combine accurate response with landmark localization. Experimental evaluation of 3D image datasets using CNN and the SCN architecture provides higher accuracy. The work in Baumgartner et al. (2016) developed a model that helps to localize the fetal in the image. During this process, CNN was trained to recognize up to 12 scan planes and a network model designed to detect fetal accurately. Experimental tests achieved 69% precision, 80% recall, and 81% accuracy.

Detection

Creating accurate ML models capable of classifying, localizing, and detecting multiple objects in a single image remained a core challenge in computer vision (Duan et al., 2019). With recent advancements in DL and computer vision models, medical image detection applications are more comfortable to develop than ever before. Object detection allows for the recognition and localization of multiple objects within an image or video. Object detection is a computer vision technique that is used to identify instances of real-world objects. Object detection techniques train predictive models or use matching templates to locate and identify objects. Object detection algorithms use extracted features and learning algorithms to identify object type instances. Object detection is a key technology behind applications such as video surveillance, image retrieval system, and medical diagnostics (Albarqouni et al., 2016). The work in Ghesu et al. (2016b) proposed the Marginal Space DL model for object detection. Adaptive training patterns is used for achieving better performance in Deep NN layers. The approximate position, boundary delineation, incorporated with a DL model, find image outline segmentation. The experimental method includes 869 patients, 2891 aortic valve images, delivering 45.2% better performance compared to other previous models. Another work in Shin et al. (2016) introduced a novel model by training a CNN using triple cross-validation to detect interstitial lung disease, lymph nodes. The experimental results achieve AUC 0.95, the sensitivity of 85%. Due to the Graphics Processing Unit (GPU) memory restrictions, enhancing 2-D image detection to 3-D image detection is a severe challenge in image detection. In Peng and Schmid (2016), the authors used 2-D region proposal networks to capture solutions in 2-D objects and later use a different framework to integrate 2-D solutions into 3-D solutions.

Another interesting work in Liao, Liang, Li, Hu, and Song (2019) proposed a novel lung cancer detection model. This process encompasses two steps. Step one detects dubious pulmonary nodules using a 3-D NN. The second step encompasses cancer detection by collecting the finest five nodules and integration into the leaky noisy-OR gate. The results achieved 85.96% accuracy on training sets and 81.42% on test sets. In Xu et al. (2015), the authors proposed a DL model called the SSAE for the detection of nuclei in breast images. SSAE is trained for extracting high-level features, and the extracted features are used as inputs to the softmax classifier for nuclei detection. The experimental results showed that the proposed model achieved 84.49% F-Measurement and 78.83% precision. Similar research in Cruz-Roa, Ovalle, Madabhushi, and Osorio (2013) used CNN with SSAE to obtain high-level features. The softmax classifier was used for detecting cancer in the skin. The experiment carried on 1417 images and achieved 91.4% accuracy and 89.4% F-measure.

Segmentation

Image Segmentation in medical image processing plays a crucial role in disease diagnosis. Image segmentation divides a digital image into several fragments. Medical image segmentation aims to make digital images simpler and more comfortable to examine. The output of medical image segmentation is a collection of medical segments covering the whole medical image (Gu et al., 2019). Many inter-disciplinary techniques are currently being used for processing medical data for obtaining better accuracy in diagnosis. The authors in Guo, Li, Huang, Guo, and Li (2019) propose a supervised Artificial Neural Network (ANN) technique with the combination of cross-modality, which is applied in all the levels of ANN. Moreover, the authors also design an image segmentation method using CNN to depict the lesions of soft tissue from the images obtained by various inter-disciplinary techniques such as magnetic resonance, CT, and positron emission tomography. Proper mining and analysis of the organ feature are essential before applying image processing. This can help in scenarios where the images are unclear or erroneous during system malfunctions. The major challenge in existing methods for image processing is the lack of coordination between the training objectives and the dependencies of the output.

The authors in Oktay et al. (2017) propose a novel technique to overcome this challenge by implementing a generic training strategy that embeds prior knowledge into the CNNs using a regularisation model which is trained thoroughly. The identification of biomarkers in medical images helps in the diagnosis process. The supervised DL is mostly used for image processing but can lack accuracy since it needs extensive knowledge of the position of the organ and its characteristics. The biomarker discovery process can be achieved by detecting anomalous regions. This anomaly detection can give hidden information about the anatomical structure. The authors in Seeböck et al. (2019) use this idea to implement a Bayesian DL technique assuming that the epistemic uncertainties would relate to the anatomical eccentricities from the training dataset. The authors also use Bayesian U-Net to train on a scenario with weak labels of given anatomy by using different ANN methods. The MR scans of the prostate using segmentation can unfold valuable information for the detection of prostate cancer. But there are several challenges of using the segmentation of MR scans such as missing boundary region between the prostate and another organ, the multifaceted background, and a difference in feature of the prostate. The authors in Zhu, Du, and Yan (2019) design a novel technique called Boundary-Weighted Domain Adaptive Neural Network (BOWDA-Net) which eases the boundary detection in segmentation by implementing boundary-weighted segmentation loss. The authors also deploy a boundary-weighted transfer leaning method to solve the issue with small image datasets.

Registration

Image registration is a method for converting datasets to a single coordinate model. Image registration plays a vital role in the area of medical imaging, biological imaging. Registration is necessary to analyze or integrate data from several medical sources. Usually, a medical technician is supposed to display several images in different directions to reduce the visual contrast between images (Haskins, Kruger, & Yan, 2020). The medical technician is often expected to manually classify points in the image that have significant signal variations as part of a sizeable anatomical structure. Medical image registration saves a lot of time for doctors and physicists. To address the shortcoming of the manual registration process, DL implementations in the image registration process have improved the productivity of the image registration process (de Vos, Berendsen, Viergever, Staring, & Išgum, 2017).

More than half of cancer patients undergo radiation therapy, making it one of the most prevalent cancer treatments (Elmahdy et al., 2019). When the number of patients rises, more doctors and more patients will be assisted by medical image processing. Elastix is fully automated 3D deformable registration software, and its application can enhance the radiotherapy process in radiation oncology departments around the world. The proposed model provides a computationally, efficient global feature search. Previously, it was difficult to get 3D registration applications that could accommodate changes in alignment, translation, and variations in intensity accurately. The proposed model saves a lot of time for physicians and offers superior registration outcomes with clinical trust. Enhanced ability to combine diagnostic scans ensures that patients undergo fewer repeat scans, save time, and minimize exposure to radiation. The experimental findings of the Haukeland Medical Center cancer dataset obtained a success rate of 97% for the prostate, 93% for the seminal vesicles, and 87% for the lymph nodes. In Bai et al. (2013), the authors proposed a multi-atlas classifier to enhance the accuracy rate of the registration. The proposed model highlights two steps. In the first step, the patch-based label fusion method was formed in the bayesian model to extract multiple features from atlas classifiers. In the second step, using label data, better registration accuracy is achieved. The results conducted on cardiac MRI imaging dataset gained 0.92, 0.89, 0.82 dice score for the left ventricular cavity, right ventricular cavity, and myocardium.

In Chee and Wu (2018), the authors used a Self-supervised learning model to establish 3D-image registration. The goal of this model is to conduct image registration in minimal time to classify the internal areas of the brain and to incorporate information from different data types. The experimental results conducted on the axial view of brain scans achieved 100x faster run time. Another exciting work in Lv, Yang, Zhang, and Wang (2018), suggested a CNN image registration model for capturing abdominal images, to detect motion-free abdominal images. The experiments were conducted on ten different patients with a 1.5 T MRI scan. Moreover, the DNN model helped to achieve better registration results compared to other existing models.

Summary

In this section, we examined DL applications for medical image processing, such as classification, location, detection, segmentation, registration. In today's world of advances in DL, we have seen significant changes in health care over the past few years by providing new opportunities to improve the lives of the people. DL for the analysis of biological images has been used in the scientific domain to identify various diseases. We observe that DL applications dive into numerous applications for medical image processing. Previously, doctors have spent a long time looking at the reports, and most of the things have been done manually. DL has begun to improve the time-consuming process, providing better results, services, and sophisticated medical tools. DL applications reshape the healthcare industry by providing new possibilities by enhancing medical life for the people. We summarize the existing DL applications for medical image processing in Table 2 .

Table 2

Review of DL applications in medical image processing.

Ref.	Dataset	Topic	Methods used	Research challenges
Lo et al. (1995)	55 chest X-ray images	Lung Nodule Classification	Artificial CNN	Texture assessment approaches to identify disease trends
Shen et al. (2015)	Lung Image Database Consortium	Lung Nodule Classification	Multi-scale CNN	The initial training and testing set strikingly different
Rajpurkar et al. (2017)	1,12,120 X-ray images of 30,805 patients.	Pathological classification of pneumonia	CheXNet DL model	Considered F1-Score only as performance metrics
Pratt et al. (2016)	Kaggle dataset	Classification for DR	CNN	System failed to learn more complex features
Abràmoff et al. (2016)	Messidor-2 Â- ADCIS dataset	Classification for DR	CNN+ IDx-DR X2.1	Failed to substitute CNN-trained features
Kawahara and Hamarneh (2016)	Private Skin dataset	Classification of skin lesions	Multi-layer CNN	Excluded use of bigger skin dataset
Roth et al. (2015)	4298 X-ray of 1675 patients	Classification of organs	CNN	Predictions are sluggish, allowing implementation problems
Guo et al. (2015)	University of Chicago Hospital.	Localization of prostate	SSAE	Considered only 66 images of prostate
Shin et al. (2012)	DCE-MRI dataset	localize multi-organ disease	Single-Layer SSAE	Limited dataset, System failed to learn more complex features
Payer et al. (2016)	Private Dataset	Accurate response with landmark localization of the medical image	SCN Architecture	Strategies to minimize the complexity of the system are not included.
Baumgartner et al. (2016)	1003 pregnancy scan reports	Localize the fetal	CNN	All the performance metrics not evaluated with traditional models
Ghesu et al. (2016b)	869 patients, 2891 aortic valve images	Object detection	Marginal Space DL	Failed to address computational constraints
Shin et al. (2016)	905 images, 120 patients	Detect interstitial lung disease	Deep CNN	Failed to deal with theoretical work on cross-modality statistics
Liao et al. (2019)	Kaggle dataset	Lung cancer detection	3-D NN	Failed to detect high accuracy for small nodules
Xu et al. (2015)	Case Western Reserve University	Detection of nuclei in breast images	SSAE	Requires improvement in extraction of features
Cruz-Roa et al. (2013)	1417 skin images	Detect cancer in the skin	Softmax classifier	Excluded use of bigger skin dataset
Guo et al. (2019)	Soft Tissue Sarcoma dataset	Tumor segmentation	Deep CNN	Examined a single dataset on a single network.
Oktay et al. (2017)	UK Digital Heart Project dataset	Cardiac image segmentation	CNN	Low resolution slice
Seeböck et al. (2019)	226 images, 33 healthy volumes	Retinal anatomy segmentation	CNN	Requires improvement in learning process
Zhu et al. (2019)	81 prostate MR volumes	Prostate image segmentation	BOWDA-Net	Limited dataset, system failed to learn more complex features
Elmahdy et al. (2019)	Haukeland Medical Center cancer dataset	Cancer registration	Elastix automated 3D deformable registration software	Strategies to minimize the complexity of the system are not included.
Bai et al. (2013)	Short-axis cardiac magnetic resonance data set	Cardiac registration	Multi-atlas classifier	Requires better computational capability
Chee and Wu (2018)	Private MR brain image dataset	3D-image registration	Self-supervised learning model	Findings are confined to brain scans with axial vision
Lv et al. (2018)	27 healthy members	Detect motion-free abdominal images	CNN image registration model	Tiny dateset with minor lesions

Deep learning for medical image processing in COVID-19

This section discusses the potential of DL in medical image processing in order to combat the COVID-19 pandemic implementing four strategies. The strategies are outbreak prediction, the virus spread tracking, coronavirus diagnosis and treatment, vaccination, and drug discovery, as shown in Fig. 4 .

Fig. 4

Deep learning in medical image processing to fight COVID-19 pandemic.

X-ray is used to diagnose pneumonia and the basic stage of cancers. But CT scan is a more sophisticated technique that can be used to detect minute changes in the structure of internal organs, and it uses X-ray as well as computer vision technology for its results. X-ray fails to detect diagnosis related to soft tissues as it uses 2-D imaging, but on the other hand, CT scan uses 3-D computer vision technology as the scan takes multiple images from various angles of the body organ. Although both X-ray and CT scan capture images of internal body structures, in the case of traditional X-ray, the images tend to overlap. As an example, the ribs shadow the heart and the lungs, making the structure with major medical queries obscured, thereby failing to provide a more accurate diagnosis. On the contrary, in case of a CT scan these overlapping aspects are completely eradicated ensuring the internal anatomy is very prominent providing a clear understanding of the health condition.

The initial diagnosis of COVID-19 disease is usually based on basic symptoms of pneumonia, analysis of patient's travel history, and exposure to other COVID-19 patients. But the chest imaging plays a significant role in understanding the extent of infection and follow-up requirements. The indicators of COVID-19 cases typically have patchy or diffused asymmetric airspace opaqueness. The CT images, the indicators have revealed bilateral lung involvement. Patients with serious conditions in ICU have shown a consolidative pattern, whereas non-ICU patients have shown ground glass pattern in their reports. On the contrary, the chest images in SARS and MERS disease have revealed unilateral indicators. But in 15% of the cases, the initial X-ray and chest images have indicated normalcy for patients already infected by the disease. This emphasizes the need for further confirmation through physical tests or the use of DL based approaches, as discussed in Hosseiny, Kooraki, Gholamrezanezhad, Reddy, and Myers (2020).

In CT images, an X-ray rotates and captures images of a particular section, from varied angles. These images are stored in the computer and further analyzed to create a new image that eliminates all overlapping. These images help doctors understand internal structures with enhanced clarity getting the complete idea about size, structure, density, texture, and shape of the same. Thus CT scan is considered to be an effective diagnostic technique than X-ray. The chest CT or X-ray fails to differentiate between COVID-19 and other cold-related symptoms. The chest CT or X-ray mostly detects the presence of an infection, which could be the consequence of any other disease as well. Also, the COVID-19 disease is extremely contagious, and the uses of imaging equipment on multiple patients are extremely hazardous. The scan machines are highly sophisticated machinery, and cleaning these machines each and every time after single patient usage is impossible. Even if attempts are being made to clean, the rigorous probability of virus exiting on surfaces of the scan machine is extremely high. On the contrary, Swab tests are proven to be more prudent for COVID-19 detection and diagnosis rather than imaging techniques. Many COVID-19 patients have normal chest CT or X-ray but are later found to be COVID-19 positive. The traditional method RT-PCR has the capability to detect the disease with accuracy but has associated challenges of taking higher detection time and requirement of reagents. In a pandemic crisis with an ever-growing number of patients, the dire need is accelerated detection of the disease using minimal resources. To fulfill this need, ML algorithms based on image processing play a significant role in eliminating the overwhelming crowd of conducting the swab test and relevant others. Sample CT scan and X-ray images of COVID-19 patients are depicted in Fig. 5 .

Fig. 5

Sample X-ray and CT scan images of COVID-19 patients (Bernheim et al., 2020, Ozturk et al., 2020).

Outbreak prediction

The world faced an unprecedented global health crisis due to the outbreak of COVID-19 (Velavan & Meyer, 2020; Wu & McGoogan, 2020). The simple epidemiological and statistical models have attracted considerable attention from the authorities with regard to COVID-19 detection and predictions. It is also a known fact that governments and other legislative bodies of various countries have always depended on various outbreak prediction models that guide towards the implementation of new policies and determines the efficacy of earlier made decisions. In this present hour of crises, the authorities around the world are similarly emphasizing on implementation of different outbreak prediction models using COVID-19 data to make well-versed decisions. This would enable them to implement appropriate control measures (Ardabili et al., 2020) and develop protocols for COVID-19 containment, detection, and prediction. Some COVID-19 worldwide outbreak predictions are available at (OurworldinData, 2020; WHO, 2020a; WORLDOMETER, 2020). While the literature contains many attempts to resolve the COVID-19 outbreak and related concerns, there exists a dire need to strengthen the necessary capabilities of the traditional models predominantly to enhance the robustness of the predicted results.

Digital technologies have contributed immensely to resolve significant health care and related therapeutic concerns. The majority of these new technologies implement big data analytics, the Internet of Things (IoT) with 5G, blockchain technology, and AI (with ML and DL) (Ting, Carin, Dzau, & Wong, 2020). It is an established fact that DL has gained immense momentum in the field of ML with its implementations across all sectors of human life (Tajbakhsh et al., 2020). As an example, in the case of data-centric studies such as computer vision, DL methods have proved to be extremely successful in providing optimal solutions (Anwar et al., 2018; Liu et al., 2018).

DL methods have been used excessively in medical image processing and related studies (Lundervold & Lundervold, 2019; Suzuki, 2017). Since it is already a popular choice among researchers in the healthcare sector, it is naturally suggested as an appropriate method for modeling the present outbreak as well. A fully connected CNN for diagnosing of COVID-19 is depicted in Fig. 6 . The choice is primarily due to the dynamic nature of COVID-19 and the variability in its actions from nation to nation. For example, the study in Ardabili et al. (2020) has collected data pertinent to the COVID-19 pandemic in Italy. The implementation of ML and soft computing models in this work predicts the possibility of an outbreak providing an opportunity for the administration to plan accordingly for disease control and related economic arrangements. In the case of DL implementations, quality and size of data play a significant role in generating accurate results. The COVID-19, the outbreak started from Wuhan in China and the patients who were presumed to be affected by 2019-nCoV were admitted to a selected hospital in Wuhan to prepare an effective disease control strategy (Wu, Wu, Liu, & Yang, 2020). The data of these patients with laboratory-confirmed 2019-nCoV infection were prospectively collected and analyzed by the authors to assist in DL and ML related research activities (Huang, Wang, et al., 2020).

Fig. 6

A fully connected CNN for COVID-19 diagnosis.

In the case of other countries like Iran, China, Italy, and South Korea, the Google Trends Data is being used to collect coronavirus related information (Ayyoubzadeh, Ayyoubzadeh, Zahedi, Ahmadi, & Kalhori, 2020; Husnayain, Fuad, & Su, 2020; Strzelecki, 2020). The other data source for outbreak prediction involves fitting of the cumulative curve and relevant measurements from infected patients in Hubei, China, with an exponential curve (Remuzzi & Remuzzi, 2020) to visualize the geographical areas with a possible outbreak. DL thus enables the prediction of COVID-19 epidemics on a global scale. The accuracy could depend on the number of involved factors of COVID-19 cases which are categorized as: a) confirmed; b) active; c) recovered; d) deceased; e) every day reported; f) population; g) living conditions; and h) environments, etc. DL can also generate data-driven features and manage high-dimensional data, whereas ML typically relies on hand-crafted features and only match low-dimensional data (Cao et al., 2018). DL can, therefore, be established as more applicable in the field of genomic prediction like COVID-19. Similar implementations have been observed for the predictions of infectious disease, which also spread rapidly and are contagious (Chae, Kwon, & Lee, 2018).

The work in Liu et al. (2020) deployed ML methodology in predicting outbreak-related events of COVID-19 in various parts of the Chinese provinces using a clustering technique that allowed the exploitation of geo-spatial synchronicity. The DL techniques in medical imaging can also help in implementing pandemic modeling to interpret the cumulative numbers of infected people versus the number of recovered cases worldwide. The cases of COVID-19 range from asymptomatic to severe pneumonia till acute respiratory distress and multiple organ dysfunctions. The findings of COVID-19 patients analyzed at Anhui Medical University in China (Fu et al., 2020) revealed that the patients initially were detected with SARS-CoV-2 RNA with Reverse-Transcription Polymerase chain (RT-PCR). The results analyzed the genetic sequence and identified similarities with COVID-19 confirming cases of SARS-CoV-2. The Central Hospital of Wuhan examined some patients with conventional pathogen detection using chest CT scans (Zhou et al., 2020). The CT scan reports detected the existence of COVID-19 virus in the lungs (Bernheim et al., 2020; Zhang, Yang, et al., 2020), with additional claims of diagnosing COVID-19 faster than the current RT-PCR tests. It is important to mention that COVID-19 shared extremely similar imaging features with other types of pneumonia and hence difficult to decide on a conclusive diagnosis. With an intention to reduce this difficulty, the main objective of the study involved studying the existing DL algorithms, use significant features integrating medical image processing applications from CT scan images of COVID-19, and then reduce prediction errors to improve the accuracy of future estimations.

Virus spread tracking

In December 2019, when people were waiting for the New Year celebration of 2020, few cases of typical pneumonia caused by a novel coronavirus (2019-nCoV) (Wu, Leung, & Leung, 2020) were reported in Wuhan, China. The work in Rothan and Byrareddy (2020) revealed that a significant number of people were infected from the wet animal market in Wuhan city, considered the zoonotic origin of the COVID-19. Eventually, multiple cases got spread across China, and the world is giving it the status of a global outbreak (Surveillances, 2020). There have been attempts made to identify a host reservoir or intermediate carrier that initiated the spread of COVID-19 from animals to humans (Cascella, Rajnik, Cuomo, Dulebohn, & Di Napoli, 2020). The authors in Lu et al. (2020) considered two species of snakes as a possible reservoir of the COVID-19, whereas another study (Zhang, Zheng, et al., 2020) rejected the possibility. The work in Xu et al. (2020) found pangolins as a latent intermediate host of coronavirus. A study in Bassetti, Vena, and Giacobbe (2020) demonstrated that COVID-19 genomic sequence analysis had similarities with two severe bat-derived, acute respiratory syndromes (SARS)-like coronaviruses. Similarly, the study in Malik et al. (2020) claim bats, civet cats, and pangolins responsible for being the potential reason behind the SARS-CoV-2 infection in human.

Various applications have been developed using concepts of computer vision, ML, and image processing approach to monitor and control the spreading of COVID-19 disease. Computer vision, image processing, and ML-based devices are being used for inspection, identification, gauging, or guiding of COVID-19. As an example, protective gear, respirator, ventilators, automatic sanitizers are being used for treating patients and protecting healthcare professionals, ensuring virus containment as well. Thermal screening is being used for measuring the temperature of individuals as elevated body temperature is a primary symptom. Social distancing is being implemented strictly to ensure safe distancing from affected patients and vision-guided robots are being used in this regard. The Australian government has launched Draganfly, an unmanned aerial vehicle (UAV) company, for immediate deployment of drones to detect COVID-19 infections among people in remote locations. In collaboration with a firm DarwinAI, the University of Waterloo has established a deep CNN, COVID-Net, for the detection of COVID-19 cases from chest radiography images (VISION, 2020). Image processing and computer vision technologies are being used for the mass production of healthcare products and gears to be used by all stakeholders in the hospitals. These devices assist in avoiding the spreading of the virus by minimizing human contact. These technologies have proved their roles in diagnosing and reducing airborne virus particles, which has possibilities of infecting a large number of people.

The virus spread tracking system involves the use of data science that emphasizes on informing responses to the queries or issues relevant to the outbreak situation (Ji, Wang, Zhao, Zai, & Li, 2020). The key challenge of this evidence-based approach is to execute the model involving data collection, analysis and reporting in real-time. A report from the Ministry of Health in New Zealand (Kvalsvig, Barnard, Gray, Wilson, & Baker, 2020) suggested a detailed analysis that focused on the location of a typical person, time and other epidemiological parameters. It is possible to understand the impact and spread of COVID-19 based on gender, geographical region, travel history, age, and daily updates from any such surveillance data using DL and ML approaches.

The work in Pourghasemi et al. (2020), Saha, Gupta, and Patil (2020) used a geographic information systems (GIS) tool for tracking of infectious diseases. The authors in Boulos and Geraghty (2020) combined GIS-based ML algorithm and Support Vector Machine (SVM) for the risk measurement of COVID-19 outbreak cases in Fars Province, Iran. As far as data is concerned, the internet acts as a very useful source to gain a tremendous amount of information about the COVID-19 virus. Apart from that COVID-19 related information - the number of confirmed infectious cases, death tolls, and recoveries are also available at the Johns Hopkins University dashboard (C. for Systems Science J. H. U. Engineering (CSSE), 2020). Later, WHO also launched a COVID-10 dashboard (WHO, 2019), which operates on ArcGIS. HealthMap (Hossain & Househ, 2016) is also a dashboard that holds a collection of information from various sources. Aarogya Setu mobile app (G.o.I. NIC & MEITY, 2020) provides official data of COVID-19 cases in India.

DL applications have been used with medical image processing approaches for the development and validation of a model at Renmin Hospital of Wuhan University in China (Chen, Wu, et al., 2020). This model retrospectively collected 46,096 unidentified images of 106 hospitalized patients. The 106 admitted patients belonged to two categories wherein the first were COVID-19 infected patients, and the later had other diseases. The number of COVID-19 detected patients were 51. The team of medical practitioners evaluated and compared CT scan images of 21 COVID-19 pneumonia cases with the model developed at Renmin Hospital (Chen, Wu, et al., 2020). A DL-based system was designed to ensure an easy decision for doctors to detect COVID-19 instances of infected pneumonia early enough to control the epidemic.

Coronavirus diagnosis and treatment

Coronavirus is not a single virus but a group or family of multiple viruses. Once a patient is infected with coronavirus, the symptoms could be similar to normal cold infection or severe respiratory syndromes. As an example, Severe Acute Respiratory Syndrome (SARS-COV) and Middle East Respiratory Syndrome (MERS-COV) are some such severe infections (Wang, Wang, Ye, & Liu, 2020) having similar symptoms as COVID-19. Numerous people around the globe have been affected and hence countries have declared a national lockdown with millions of citizens being strictly quarantined (Das, Ghosh, Sen, & Mukhopadhyay, 2020; Hopman, Allegranzi, & Mehtar, 2020; Rahman, 2020; Singh & Adhikari, 2020). In such a crisis, outbreak prediction models and virus-spread tracking tools that involve DL and medical image processing have huge potential for COVID-19 diagnosis and treatment processes. These tools help in supporting doctors for the initial screening process and rapid detection for accurate diagnosis of the disease.

The role of technology is very important in the functioning of DL and medical image processing to combat COVID-19 ensuring faster and accurate patient diagnosis. The authors in Li et al. (2020) have discussed the potential role of AI (with ML and DL) in the diagnosis of COVID-19. Initially, tests were conducted taking throat swab and nasopharyngeal samples from a patient and the samples were used to collect RNA using specific chemical processes. This RNN mixed with a specific reverse transcriptase enzyme (i.e., RT-PCR) turns into two-stranded DNA. The enzyme causes the DNA to be synthesized into “primers”. The “primers” are then fused with a fluorescent dye. This combination signal becomes a viral DNA, which is finally referred as COVID-19 positive test of the patient (Molteni & Rogers, 2020).

The medical image processing techniques such as computerized tomography have always helped in fast and accurate diagnosis of diseases and it is no different in the case of COVID-19 as well. The sensitivity of a CT-based COVID-19 diagnosis is observed to be significantly 80%-90% better than RT-PCR while having 60%-70% specificity on the low side (Ai et al., 2020). (Bai et al., 2020). DL and medical image processing play an important role in differentiating between COVID-19 infected and non-infected patients. The COVID-19 symptoms closely match regular pneumonia. The hospitals in Spain consider this methodology as their default feature in a diagnostic pathway. However, other sources have identified X-ray as an alternative examination (Chen, Zhou, et al., 2020).

Interestingly, the work in Peng, Wang, Zhang, and C.C.C.U.S. Group (2020), Poggiali et al. (2020) presented a comparison between ultrasound and CT findings making ultrasound a more reliable approach in case of Pneumonia detection than chest X-ray. Another solution in Qin, Liu, Yen, and Lan (2020), Zou and Zhu (2020) mentioned the requirement of additional information in COVID-19 candidate diagnosis using PET-CT. Recently, the use of CT imagery with AI detection has helped in diagnosing COVID-19 cases with distinct manifestations. (Butt, Gill, Chun, & Babu, 2020). The work in Jin et al. (2020) proposed a detailed guidance report with useful tools to support COVID-19 diagnosis and care. The guideline consists of the methodology, epidemiological characteristics, population prevention, diagnosis, treatment of COVID-19 disease.

Standford University has provided data, models, tools, research studies, and funding opportunities for COVID-19 research. The research effort combined with COVID-19 datasets has helped to build comprehensive medical image processing and DL models for identification, virus diagnosis, treatment, and even potential vaccine development. Some available medical imaging datasets worth mentioning are - (a) Societa Italiana di Radiologia Medica (b) IEEE8023 chest x-ray and CT dataset on COVID-19 (c) CNN based Darwin AI and University of Waterloo (d) Centre for Mathematical Imaging in Healthcare provides AI support for COVID-19 diagnosis (e) RadiologyAi Consortium (CT scans of COVID-19 patients).

DL-based screening models have enabled to generate consistent and accurate outcomes by digitizing and standardizing the image data by integrating various medical image processing techniques. It has also been observed that sensitivity and precision in detecting COVID-19 using RT-PCR have a relatively low positive detection rate than the use of radiographic patterns on CT chest scans in the initial stages of the disease inception. Several CNN models have been explored theoretically by the authors in Butt et al. (2020) to distinguish CT samples with COVID-19, influenza viral pneumonia, or no-infection. The authors have also worked on existing 2D and 3D DL models integrating clinical comprehension to achieve AUC, sensitivity, and accuracy for coronavirus vs. non-coronavirus cases on the basis of thoracic CT findings.

Vaccine discovery and drug research

The WHO's Department of Research and Development has embarked on promoting the development of diagnostics, vaccines, and therapies for this novel coronavirus COVID-19 (WHO, 2020b). The COVID-19 infections require immediate identification to increase the chances of recovery among patients and providing opportunities to start treatment at the earliest. Diagnosis is thus a valuable step for understanding the number of COVID-19 affected people, and for segregating individuals who are resistant and potentially “protected” from infection. Developing an efficient and stable novel vaccine against this highly infectious COVID-19 disease is essential. Medical image processing and DL have the potential to contribute towards the COVID-19 pandemic for aiding in the discovery of vaccines and related drugs.

The ML/DL algorithms can be trained by massive datasets of chemical compounds. Some of the compounds can enhance human immunity, and some do not. In this way, ML/DL algorithms can learn the patterns of the compounds that can build immunity to a virus in a very quick time. The researchers can then use the ML-based algorithms to test whether their newly designed combination of compounds in the vaccines can be used as an antidote to a virus or not. In this way ML/DL algorithms play a very vital role in the process of discovering vaccines/drugs (Kannan, Subbaram, Ali, & Kannan, 2020).

Virus vaccines basically constitute of similar or part of the antigens that cause the disease. The vaccines, when introduced in the body, activates the immune system to generate specific antibodies for detecting and neutralizing the viruses. Viruses typically multiply quickly, and their antigens are vulnerable to mutations that prevent the recognition of these antibodies. The vaccine production effort to classify T-cell epitopes for the SARS-COV-2 virus is discussed in Qiao, Tran, Shan, Ghodsi, and Li (2020). The work in Tayebi (2020) used CNN algorithms as a DL approach for the prediction of cross-immunoreactivity (CR) in heterogeneous epitope vaccines.

RADLogics (RADLogics, 2020) developed AI-based systems and DL tools to be used for providing services in the hospitals. These tools work based on medical imaging like chest CT or X-ray scans for screening mild cases, triage new infections, and monitor advanced diseases to detect COVID-19 infections. DL applications based on medical imaging helps to identify various drug-manufacturing approaches to combat COVID-19. This breakthrough thus could set the stage for vaccines, or an effective antiviral. The work in Zhang, Saravanan, et al. (2020) suggests the use of the Deeper-Feature CNN (DFCNN) to identify potential drugs for 2019-nCoV. The study in Beck, Shin, Choi, Park, and Kang (2020) proposes a drug-target interactive DL model for the prediction of commercial antiviral drugs against COVID-19. In Ong, Wong, Huffman, and He (2020) a reverse vaccinology and ML-based approach for developing a vaccine against COVID-19 coronavirus is presented. The virus and proteins (spike, nucleocapsid, and membrane) are tested for the development of SARS and MERS vaccines. As a next step, the reverse vaccinology tool Vaxign and the ML application Vaxign-ML is used to predict candidates for COVID-19 vaccines. On 5 May 2020, Israel Institute for Biological Research (IIBR) recently claimed to evolve a monoclonal neutralizing antibody for coronavirus neutralization within the carrier body, which is definitely a ray of hope in the vaccine discovery of COVID-19 (ISRAEL, 2020).

Limitations of DL based image processing in COVID-19

Although millions of patients are getting infected by the disease, there still exists the absence of publicly available large datasets focusing especially on tests with missing infections. The accuracy of any DL model depends on the availability of data, and in the case of COVID-19, this is a major concern. The accuracy of the DL model becomes questionable due to the use of insufficient data. Also, accessibility to CT and X-ray images from wider demographics is a requirement that is difficult to collect from healthcare organizations. Also, the implementation of policy decisions based on DL based implementations requires access to behavioral attributes such as work, education, family background. All of this information is necessary to understand how people remain asymptomatic even after getting infected by the virus. These limitations stand predominant in achieving optimized scientific decisions. But as presented in the paper, various studies are being conducted relentlessly to eliminate such challenges and acquire more clarity in COVID-19 diagnosis (Vaid, Kalantar, & Bhandari, 2020).

After critically reviewing the evolving literature, it is evident that medical image processing and DL play a significant role in fighting the COVID-19 pandemic through a range of promising applications including outbreak prediction, the virus spread tracking, diagnosis/treatment of coronavirus, and discovery of vaccines/drugs. The comprehensive survey and selected references are summarized in Table 3 .

Table 3

Summary of deep learning for medical image processing in COVID-19.

Ref.	Dataset	Methods used	Evaluation metrics	Research challenges
Ozturk et al. (2020)	Chest X-ray	CNN, DarkNet, DarkCovidNet	98.08% accuracy	Use of a limited number of COVID- 19 X-ray images
Chae et al. (2018)	OLS, ARIMA	DNN, LSTM	Average performance by 24% and 19%, respectively.	Prediction of infectious disease.
Zhou et al. (2020)	CT scan Images	logistic regression model	89.47% sensitivity, 67.42% specificity	The laboratory testing methods are not uniform among different hospitals.
Zhang, Yang, et al. (2020)	Electronic medical records	Univariate and multivariate Cox regression analysis	Hazard ratio and confidence interval was used to detect the adverse outcome	Predicting the adverse outcome at the early stage of COVID-19.
Chen, Wu, et al. (2020)	CT scan images	DL-based model	100% sensitivity, 93.55% specificity, 95.24% accuracy.	Achieving consistent results between the expert and model.
Li et al. (2020)	chest CT exams	COVNet	90% sensitivity, 96% specificity	Unable to categorize the disease into different severity levels.
Ai et al. (2020)	chest CT	DL	97% sensitivity	The clinical and laboratory data were limited when regional hospitals were overloaded.
Butt et al. (2020)	CT chest images	CNN models	0.996 AUC, 98.2% sensitivity, 92.2% specificity	Achieving fast and reliable detection of COVID-19 from chest CT datasets.
Gozes et al. (2020)	CT scan	2D Slice Analysis, 3D Volume Analysis	98.2% sensitivity, 0.996 AUC, 92.2% specificity	Challenging to achieve high accuracy in detection of Coronavirus as well as quantification and tracking of disease burden.
Wang and Wong (2020)	Chest X-ray	COVID-Net	93.3% test accuracy	AI systems leveraging the more readily available and accessible CXR imaging modality.

The manual detection of cases of COVID-19 or non-COVID-19 pneumonia is a demanding and time-consuming process, as these cases are exponentially increasing. Indeed, the application of DL in medical image analysis effectively supports disease prediction of huge datasets obtained from available sources such as health organizations (e.g., WHO), healthcare institutes (e.g., China National Health Commission, Indian Medical Research Council). DL applications focus on medical imaging, which has emerged as a promising solution. DL applications are used to process and analyze medical imaging data to help radiologists and doctors enhance the accuracy of the diagnosis. DL from medical images have the potential to even identify possible targets for an appropriate COVID-19 vaccine. Multiple studies on COVID-19 have been conducted to emphasize on automated COVID-19 identification using DL systems using medical imaging datasets.

Deep learning for COVID-19 medical image processing: use cases

This section presents some use cases of DL for COVID-19 medical image processing.

Use case 1: automated detection and monitoring of COVID-19 patients in China using medical images based DL techniques

It is redundant to mention that COVID-19 cases spread at an alarming rate with dreadful effects on normal human lives, general public health and the global economy. At this stage, it is extremely crucial to develop auxiliary diagnostic systems to detect the disease in minimal time to ensure disease containment. Application of AI-based DL techniques integrated with radiology imaging techniques was the immediate need that was addressed in the study conducted in China in collaboration with the USA. The study revealed the fact that AI-based Computer Tomography images developed rapidly have high accuracy in the detection of COVID-19 positive cases. The datasets used in the study were collected from China as well as other countries with prominent COVID-19 cases. The study implemented a system that used 2D and 3D DL models, integrated them with existing AI models and ensured the inclusion of complete clinical knowledge. Multiple experiments were conducted to detect significant COVID-19 related CT features, thereby detect the disease. The progression of the disease in each patient was monitored using a 3D volume view, which generated a COVID-19 score. The accuracy of the results in classifying positive and negative COVID-19 cases were quite promising of about more than 99 percent AUC (Gozes et al., 2020).

Another significant study in the radiology department of Zhongnan highlights the use of AI-based DL software to detect visual symptoms pertaining to Pneumonia from CT scan images of the lungs of COVID-19 patients. The software has been immensely beneficial to assist the overworked healthcare professionals screen potential COVID-19 patients and forward them for further medical tests saving a lot of time. It is very difficult to differentiate between symptoms of normal Pneumonia and COVID-19 Pneumonia, which the software aided in identifying the typical or partial symptoms of more than 35,000 cases in 34 hospitals in China (WIRED, 2020).

Use case 2: automated detection and monitoring of COVID-19 patients in Canada using medical images based DL techniques

Researchers at the University of Waterloo have designed software that uses AI, and DL assisted X-ray screening method named COVID-Net to augment the polymerase chain reaction for SWAB tests conducted on COVID-19 patients. This augmentation with computational techniques – AI and DL have enhanced accuracy reducing the time of screening contributing effectively towards the containment of the disease. This screening tool was developed using almost 6000 image datasets of chest radiography images collected from 3000 patients (Ozturk et al., 2020; Wang & Wong, 2020).

The developing company of COVID-Net have taken the next step to develop COVID-RiskNet that aims at detecting the risk associated with the level of severity of the disease in an affected patient. The tool also suggests the plausible direction of treatment and segregates relatively better patients to be self-isolated from the severe cases needing inpatient medical care. It basically summarizes the status of severity and helps to prioritize the line of treatment accordingly.

The Toronto-based startup company – BlueDot developed a platform using AI, ML, and big data technologies to track and predict the outbreak of any infectious disease. This would set the alarm for the private sectors and government policymakers to implement their mitigation plans at the earliest, saving millions of lives. The platform was successful in alerting the administration on an unusual cluster of pneumonia patients being formed around a local market in Wuhan, China. This was the first reported and recognized information on COVID-19, and almost nine days later, the WHO made its official announcement. The software gathers data on 150 different types of diseases and syndromes around the world and updates the database every 15 minutes round the clock. The repository includes data from organizations like CDC or WHO and external healthcare sources, traveler history, human and animal population data, climate data and local information collected from journalists working in 1 million articles every single day. The analysts manually classify the collected data, develop specific taxonomy for efficient searching of keywords and then apply ML and natural language processing for training the model. The results generate the only minimal number of highly filtered cases to be further analyzed by humans for expert opinion and necessary action (DIGINOMICA, 2020).

Apart from the above-mentioned cases, DL models and genomic sequences of the COVID-19 virus extracted from patients is being used to detect the interaction effects of the viral genomic sequences. Neural network approaches is deployed to train large sequences from varied geographical locations to identify mutations appearing on the RNA sequences based on the sequences of other nucleotides in the genomic sequence. This profiling of viral evolution could help in identifying mid-level interactions and detect plausible viral genomes and mutations for specific proteins relevant to ensure faster treatment response towards fighting severity of the disease (Mila, 2020).

Use case 3: automated detection and monitoring of COVID-19 patients in South Korea using medical images based DL techniques

The Republic of Korea has been quite successful in the containment of COVID-19 without implementing the complete lockdown of its economy. However, public places involving the gathering of a large number of citizens have been locked down. The Division of Risk Assessment and International Cooperation at the Korean Disease Control and Prevention center have given the immense emphasis on adapting advanced ICT techniques to combat the spreading of COVID-19 and detection of the same (ITUNEWS, 2020). The company Seegene developed a COVID-19 detection kit at a very early stage using AI techniques which helped to perform widespread testing with a primary focus on high-risk groups such as those having underlying diseases, elderly citizens who share homes with multiple individuals in crowded city locations and also patients who return from international travels (Seegene, 2020).

The company VUNO developed an AI-based decision tool for chest X-ray images that used an algorithm capable of detecting abnormalities. The tool basically aided to classify and examine the intensive care patients using their X-ray images in less than three seconds (ITN, 2020). JLK Inspection developed a medical platform called AiHub for the diagnosis of COVID-19 using DL, AI and big data technology to examine anomalies in the lungs within seconds. The same company has developed a handheld chest X-ray camera which scans the chest in seconds and presents a heatmap visualization highlighting abnormal lesions (LABPULSE, 2020).

Lessons, challenges, and future directives

Although DL has gained immense momentum, popularity and has generated impressive results with simple 2D images, there exist limitations in achieving a similar level of performance in medical image processing. Research work in this regard is still-in-progress and some of the lessons learned are mentioned below:

One of the most inhibiting factors is the unavailability of large datasets with high-quality images for training. In this case, synthesizing of the data is a possible solution so that the data collected from varied sources could be integrated together

The majority of the state-of-the-art DL models are trained for 2D images. However, CT and MRI are usually 3D and hence add an additional dimension to the existing problem. Since the conventional DL models are not adjusted to this, experience plays a major role when DL models are implemented on these images

The non-standardized process of collecting image data is one of the major issues in medical image processing. It is important to understand that with the increase in data variety, the need of larger datasets arise to ensure the DL algorithm generates robust solutions. The best possible way to resolve this issue is the application of transfer learning, which makes pre-processing efficient and eliminates scanner and acquisition issues.

Challenges and issues

The challenges and issues pertaining to DL implementations for medical image processing for controlling COVID-19 pandemic in smart cities are enlisted below:

Privacy – Availability of COVID-19 high-quality images and larger datasets is a major challenge considering the privacy of patient data.

Variability in Outbreak Pattern – The outbreak of the data has followed complex pattern and extreme variation in behavior across various countries and hence reliability of the prediction diseases get added as an additional challenge.

Regulation and Transparency – Countries across the globe have adopted strict protocols in regulations to be complied pertinent to sharing of COVID-19 data, one of the major protocols clearly states that minimum data and specimens to be collected from patients in the minimum amount of time. Thus this makes it more difficult to analyze.

Variability in the testing process across various hospitals is also an important concern leading to non-uniformity in data labels.

The symptoms of Pneumonia and COVID-19 Pneumonia are very similar. Identification of an appropriate DL technique to exclusively and specifically detect COVID-19 with optimum accuracy still remains as a visible challenge.

Moreover, the coronavirus genome has been completely sequenced based on the data collected from thousands of patients suffering from the disease across the globe. This genome sequence has been extremely beneficial, especially due to the fact that the COVID-19 virus has a higher mutation rate. The present diagnostic tests help to identify specific genes from the virus, and the test accuracy depends on target areas of the relevant genomes. The effect of the mutation on the diagnostic tests is alarming and there exists a high possibility of generating a “false negative” for a patient actually suffering from the disease. These diagnostic tests provide their diagnosis based on the scrutiny of the coronavirus genes which often vary as the disease spreads from one human to another (Bos, Heijnen, Luytjes, & Spaan, 1995).

Future directives

The future directions in COVID-19 research lie in connecting hierarchical features of COVID-19 image datasets with other clinical information for conducting multi-omics modeling for enhanced prediction of the disease. Also, since the available datasets are of relatively smaller size, insufficient for yielding robust predictions, transfer learning is a future direction of research that could detect anomalies in smaller datasets and yield remarkable results. In COVID-19 diagnosis, detection of the disease in the earliest possible time is the major necessity. In this regard, the ML algorithm contributes to expediting the process using limited resources. Transfer learning supports the same objective, being an apt technique in COVID-19 detection, where time-to-delivery and availability of training data is a primary concern. This technique takes pre-trained models from academics, research Institutes, or open source communities and uses the same to perform ML tasks, thereby saving time and resources. It transfers the learned parameters or knowledge to various algorithms as their engineered features. DL yields good results when larger volumes of data are available, but in the case of transfer learning, the same can be achieved with limited labeled dataset (Zhuang et al., 2020). In this COVID-19 pandemic situation, availability of dataset, furthermore labeled ones, is an obvious challenge and hence transfer learning has immense potential to serve the purpose of COVID-19 detection. As an example, the study by Rehman, Naz, Khan, Zaib, and Razzak (2020) can be referred to where X-ray and CT images of COVID-19 cases were collected from the GitHub public repository. These images were then parsed to select the COVID-19 positive samples from these images. Apart from this, bacterial pneumonia, viral pneumonia and healthy image dataset were collected from the Kaggle Repository. When both of these data sets collected from different resources were combined, a database of 200 X-ray and CT images of COVID-19 cases, 200 cases of viral pneumonia, 200 cases of bacterial pneumonia and 200 cases of healthy subjects were formed. This pre-trained knowledge, when fed into the CNN architecture, delivered enhanced accuracy in COVID-19 detection results. Similar approaches were taken in Apostolopoulos and Mpesiana (2020) wherein a dataset of 1427 images with 224 COVID-19 cases were combined with 700 images of common bacterial pneumonia and 504 images of normal patients to form a data repository. This dataset, when fed into a DL model, detected COVID-19 with enhanced accuracy, sensitivity and specificity in comparison to the traditional approaches. Thus, creation of a centralized data repository for collecting COVID-19 patient data is an enormously important necessity to develop predictive, diagnostic and therapeutic strategies to combat the COVID-19 crises and similar pandemics of the future in smart healthy cities.

As reflected and reviewed in the existing studies, there are various DL implementations applied on different datasets utilizing different evaluation criteria where radiology imaging datasets have been found to be prevalent. But utilization of these implementations in real-world medical practice cases is a major concern that dictates the immediate need for benchmarking frameworks for the evaluation and comparison of the existing methodologies. These frameworks should enable the use of computational hardware related infrastructures considering similar patient records, data pre-processing methods and the evaluation criteria for the various AI methods ensuring data interpretability and transparency.

The inhabitants of this present-day world are much fortunate than the previous generation who have witnessed the Spanish flu pandemic in 1918 as we are immensely gifted with advanced technology. AI has been used extensively in all spheres of human lives. Since AI has touched all sectors of life, the same technology should be used and completely explored to combat the COVID-19 pandemic as well. As an example, the future of AI lies in the development of autonomous robots and machines for disinfection, healthcare work in hospitals, delivering medications and essentials to patients and also providing personal care for them. AI can be integrated with natural language processing (NLP) technologies to develop chatbots that can communicate remotely with patients and provide consultations during this crisis period.

Apart from the aforementioned benefits, the application of AI can play a significant role in eradicating fake news being spread in social media platforms. The use of AI can filter out information relevant to government policies, pandemic prevention protocols and the science behind the virus spreading and containment, thereby ensuring that authentic information alone reaches the common masses eliminating all possible chances of unnecessary panic creation. At this juncture of never-ending battle with the COVID-19 crisis, the only ray of hope is the development of a novel vaccine against the virus. AI has immense potential in this regard for investigating the genetic and protein structure of the virus to accelerate the process of drug discovery. Although this process is time-consuming and economically expensive using the traditional methods, with the use of AI and DL techniques, it would soon be possible to detect the most appropriate antibiotic from huge data set of hundred million molecules. This is definitely the most interesting and necessary future direction of research to win over the COVID-19 pandemic.

Conclusion

Among solutions to combat the COVID-19 pandemic, DL has been considered a great technique to provide intelligent solutions. Motivated by many applications of DL for medical image processing in the last decade, we have provided summarized recent efforts about the COVID-19 outbreak for smart, healthy cities. DL has been employed to achieve various solutions for COVID-19 disruption, including outbreak prediction, a virus spread tracking, diagnosis and treatment, vaccine discovery and drug research. Despite promising results, the successful use of DL to process COVID-19 medical images still requires considerable time and effort as well as close operation between different parties from government, industry, and academia. We have also enlisted a number of challenges and issues associated with existing studies such as data privacy, variability of outbreak pattern, regulation and transparency, and distinction between COVID-19 and non-COVID-19 symptoms. Finally, we have discussed a number of future directions of DL applications for COVID-19 medical image processing. We believe that the COVID-19 outbreak will be ending soon with help from DL and image processing techniques as well as many other technologies such as biomedicine, data science, and mobile communications. We also hope that our work is a good source of reference and can drive many novel studies on DL and medical image processing in the battle against the COVID-19 outbreak.

Declaration of Competing Interest

The authors report no declarations of interest.