| Literature DB >> 33935585 |
Francesco Piccialli1, Vincenzo Schiano di Cola2, Fabio Giampaolo1, Salvatore Cuomo1.
Abstract
The first few months of 2020 have profoundly changed the way we live our lives and carry out our daily activities. Although the widespread use of futuristic robotaxis and self-driving commercial vehicles has not yet become a reality, the COVID-19 pandemic has dramatically accelerated the adoption of Artificial Intelligence (AI) in different fields. We have witnessed the equivalent of two years of digital transformation compressed into just a few months. Whether it is in tracing epidemiological peaks or in transacting contactless payments, the impact of these developments has been almost immediate, and a window has opened up on what is to come. Here we analyze and discuss how AI can support us in facing the ongoing pandemic. Despite the numerous and undeniable contributions of AI, clinical trials and human skills are still required. Even if different strategies have been developed in different states worldwide, the fight against the pandemic seems to have found everywhere a valuable ally in AI, a global and open-source tool capable of providing assistance in this health emergency. A careful AI application would enable us to operate within this complex scenario involving healthcare, society and research.Entities:
Keywords: Artificial intelligence; COVID-19; Deep learning; Healthcare; Machine learning; Review; SARS-CoV-2; Survey
Year: 2021 PMID: 33935585 PMCID: PMC8072097 DOI: 10.1007/s10796-021-10131-x
Source DB: PubMed Journal: Inf Syst Front ISSN: 1387-3326 Impact factor: 5.261
Fig. 1A Worldwide map of scientific production on topics related to AI and the COVID-19 pandemic. Countries with a higher number of peer-reviewed publications are reported in blue; the darker the colour, the greater the number of papers. The links in red represent international collaborations
Schematic overview of information types, related sources, applications and AI methods
| Data type | Multimedia content | String patterns | Time series |
|---|---|---|---|
| Description | Image, Video, Audio, 3D Data Points | Texts, Genetic Sequences, Semantic Networks | Historical Data, Sensor Data, Scientific Data |
| Medical Images (CT, X-ray), Lung Ultrasound (LUS), Surveillance and Security Cameras, Drones, Acoustic Data, Smart Speakers, 3D Protein | Log Data, Documents, Scientific Literature, Genome Maps, Social Network Messages, Electronic Health Records | Clinical Values, Wearable Device, Internet of Things (IoT) Data (ventilators parameters, etc.), Statistical Parameters (deaths, ICU occupancy), Economic Indicators (GDP) | |
| Computer Vision, Motion Detection, Voice Recognition | NLP, Classifying Reviews | Sensor Data, Web Activity | |
| Medical Disease Detection and Diagnosis, Social Monitoring, Proteomics | Social Monitoring, Medical Treatments, Pathogenesis | Healthcare Optimization, Social Planning, Epidemic and Transmission Prediction | |
| Pneumonia Diagnosis, Mask Detection, Environment Recognition, Social Control, People Displacement Analysis, Disinfection Robots, Quarantine Checking, Cough Sound Recognition, Protein Folding | Medical Protocols Enhancing, Drug Development and Repurposing, Vaccine Development, Genomics | Outbreak and Transmission Prediction, Infection Tracking and Prediction | |
| DeCoVNet, FCNN, ResNet-18/34/50, SqueezeNet, DenseNet-161, VGG19, STN, CNN, SVM,RF, MLP, Grad-CAM U-Net, FCN SegNet, DeepLabv3 | ML-DSP, NLP, RNN, RF, LSTM, SDV, LDA | PNN+cf, BFGS-PNN, BFGS, MLP regressor, ANN, RF, LG, CD-Net, FFNN, LASSO, DeepFM, ARIMA-WBF, LSTM, PODA |
The data are divided into three groups, namely Multimedia Content, String Patterns and Time Series, the descriptions of which are provided on the top of the table. Next, for each group, typical sources and fields of application are reported with some explanatory examples. As for the application field we used the taxonomy presented in Chen et al. (2020). Finally, common AI techniques are reported for each column
Fig. 2A time perspective of a pandemic diffusion, with advances in society, medical centers and research institutes. This Figure shows how the collection methodology on the type itself of data evolves in time and interwines with various areas of society. Clinical data generates health records, which, grouped in time and by population leads to time series, predicted by forecasting models to anticipate the epidemic evolution. Whereas X-Ray and CT generated in hospitals can be used to train DL models for diagnosis purposes. Starting from the bottom, the first stage represents the moment where a pandemic has not yet occurred. In society, there are no specific involvements, and in research institutes laboratories study and test new viruses and diseases. In the second stage, monitoring and social surveillance are performed to prevent any virus from becoming endemic and impacting on society before the availability of a full study on a cure or a vaccine. At the same time, hospitals are generating medical data on infected patients, like chest CTs as in this COVID-19 pandemic. Neural Networks (NN) can start to analyze such images, improving their accuracy in predicting new cases. The third stage is the occurrence of a pandemic, in which period, time series data might be showing an exponential trend. Here, the time series start to become longer, and predictions in time can be made with ML models. At the fourth stage, there is a significant number of cases and substantial amount of data, making necessary a text analysis of papers with the help of an AI tool in order to process the huge number of articles about the virus. Finally, when the pandemic is over, and a vast amount of data is available, a retrospective analysis can be carried out
Summary of COVID-19 outbreak (stage 2) related literature
| Reference | Dataset | COVID19 Data | Time interval | AI/ML method | Performance | Relevance | Shortcoming |
|---|---|---|---|---|---|---|---|
| Randhawa et al. ( | National Center for Biotechnology Information (NCBI) database | before Jan 23, 2020 | Machine Learning with Digital Signal Processing (ML-DSP) approach, which uses 6 supervised learning approaches like Linear SVM and KNN, augmented by a decision tree approach to the machine learning component | Average ACC: 90.5 – 96.2 | i) Confirm the taxonomy of the COVID-19, and possible bat origin; ii) alignment-free methodology adopted to rapidly analyze large datasets. | ML-DSP is a black-box method that does not offer a (biological) explanation for its output and is not able to assign a taxon that it has not been trained on. | |
| Fong et al. ( | Archive of Chinese health authorities | Jan 21 – Feb 3, 2020 | polynomial neural network with corrective feedback (PNN+cf) | RMSE: 136.55 RMSE Lin Regressor: 520.16 RMSE ARIMA: 1016.27 | Data augmentation to the existing little data and fine-tuning the parameters of an individual forecasting model | Predicted result is very sensitive to the parameters used. Understand why algorithms incur very low or very high errors (for panel selection). | |
| Fong et al. ( | Chinese Center for Disease Control and Prevention | Jan 25 – Feb 25, 2020 | Broyden–Fletcher–Goldfarb–Shanno optimized polynomial neural network (BFGS-PNN), i.e. PNN enhanced with parameter optimization function. | RMSE: 62077.26 RMSE LinReg: 127693.55 | i) BFGS algorithm to optimize the parameters and network structure size using alliteratively hill-climbing technique. ii) Estimate the direct cost that is needed as an urgent part of national budget planning to control the COVID-19 epidemic | Compare and contrast the differences of other techniques and refine input for accuracy. | |
| Wang et al. ( | dataset of the radiology department of Huazhong University | Dec 13, 2019 – Feb 6, 2020 | Weakly supervised deep learning framework (DeCoVNet): UNet (pre) & three stages 3D Deep Network & UNet | (ROC) AUC: 0.959 | COVID-19 classification and lesion localization using 3D CT volumes | i) UNet model for lung segmentation did not utilize temporal information and it was trained using imperfect ground-truth masks ii) cross-center validations (more hospitals); iii) CT data of (CAP) not collected; iv) explainability | |
| Kang et al. ( | Chinese CDC | Jan 9 – Feb 14, 2020 | FCNN: Structured Latent Multi-View Representation Learning | ACC: 95.5% Sens: 96.6%, Spec: 93.2% | Classify COVID-19 vs CAP. Use of multi-view representation learning with multiple features, such as texture, surface, volume histogram, and intensity. | i) Diagnosis with more classes instead of only two types of disease (i.e. COVID-19 and CAP). ii) Clinical characteristics of patients can be useful for diagnosis. | |
| Xu et al. ( | 2 China Hospitals | Jan 19 – Feb 14, 2020 | Residual network (ResNet)-18 by concatenating the location-attention mechanism in the full-connection layer to improve the overall accuracy | binary ACC : 86.7% | Multi-center case study. Location-attention classification model | Only compared the CT manifestation of COVID-19 with that of IAVP. To combine the patient’s contact history, travel history, first symptoms, and laboratory examination. | |
| Mei et al. ( | 18 medical centers in China | Jan 17 – Mar 3, 2020 | SVM, random forest and MLP classifiers | (ROC) AUC: 0.92 Sens: 84.3 Spec: 82.8 | Compared performance to one fellowship-trained thoracic radiologist with 10 years of experience and one thoracic radiology fellow. | Explore various approaches, including 3D deep-learning models and develop the interpretability of CNN models. To validate the robustness of the models, is important to test the AI system in multiple centers. | |
| Liang et al. ( | NHC of the People’s Republic of China | before Jan, 2020 | three-layer feedforward neural network & LASSO Cox model | C-index: 0.894 (ROC) AUC: 0.911 | Deep Learning Survival Cox model had superior discriminating power compared with the classical Cox model, because it unravels the nonlinear relationships between complex clinical covariates and their hazards. | To extended deep learning model to integrate time-dependent covariates such as vital signs and high-dimensional features such a CT or X-ray images. | |
| Wang et al. ( | 5 hospitals; most from hospitals in Wuhan, others from hospitals in Beijing. | 20 Feb 2020 | i) classificaiton: ResNet-50, Inception networks, DPN-92, and Attention ResNet-50; ii) segmentation: fully convolutional networks (FCN-8s), U-Net, V-Net and 3D U-Net++. | best AUC: 0.991, with 3D Unet++ & ResNet-50 | Experience in building and deploying an AI system | i) Does not perform well when there were multiple types of lesions, or with significant metal or motion artifacts, ii) The system is too dependent on fully annotated CT images. |
The timings and data of the research papers analyzed are reported in the table. It can be observed that in this stage of the pandemic most of the works focus on the prediction of infection diffusion and early SARS-CoV-2 induced pneumonia diagnosis. Moreover, it is worth noticing that, since the first outbreaks occurred in China, a large number of the datasets used came from this country
Summary of selected research papers about COVID-19 and data used within those studies
| Reference | Dataset | COVID19 Data | Time interval | AI/ML method | Performance | Relevance | Shortcoming |
|---|---|---|---|---|---|---|---|
| Car et al. ( | JHU CSSE | Jan 22 – Mar 12, 2020 | MLP regressor. Limited-memory BFGS (Broyden–Fletcher–Goldfarb–Shanno algorithm) | Model of novel viral infections with geographical and time data as inputs. Average training time 2357 min on 16 48-thread HPC nodes, for 5-fold cross-validation and grid search of 5376 items. | Models can be compared with various infectious diseases. Other approaches should be applied to gain explainability. | ||
| Zhu et al. ( | Huami Wearable Device | Jul 1, 2017 – Apr 8, 2020 | Regression model combining sparse categorical features and dense numerical features (CDNet), that concatenates 2 subnetworks: CatNN and DenNN | Pearson’s coefficient | Prediction using dynamic physiological data may have an advantage in recognition of the outbreak of infection. | The validity of the statistical description depends on both the user scale and diversity. | |
| Ghamizi et al. ( | Google’s Mobility Reports | Jan 3 – Apr 29, 2020 | Feed-Forward Neural Network (FFNN) | FFNN provides accurate and interpretable predictions | better feature engineering or neural architecture search (with CNN or RNN) | ||
| Mackey et al. ( | Twitter and Instagram | Feb 5 – May 7, 2020 | NLP & RNN and LSTM | AUC: 94–99 (based on Li et al. ( | Identified over 1000 suspect selling posts | Multimodal methods that could analyze and distinguish both text and image have not been used. | |
| Murphy et al. ( | Netherlands Hospitals | Mar 4 – Apr 6, 2020 | CAD4COVID-Xray, based on CAD4TB v6 - a commercial deep learning system | AUC: 0.81 Spec: 78%; Sens: 75% | Performance compared against 6 independent readers | Need to take into account related patient details. | |
| Ls et al. ( | 2 hospitals in China | Jan 1 – Mar 18, 2020 | ResNet34 as a backbone model for multiple instance learning (MIL) framework training procedure | (ROC) AUC : 0.987 ACC: 97.4% on 5-fold cross-validation | Model can be employed as a tool for prognosis prediction. Validated a MIL-based predictive model using CT imaging. | i) Sample size was relatively small; ii)Lack of transparency and interpretability (like all DL models) | |
| Zhang et al. ( | Wuhan and Ecuador centers; Radiopaedia dataset | Jan 25 – Mar 25, 2020 | (i) segmentation networks: U-net, DRUNET, FCN SegNet, and DeepLabv3. (ii) Classification networks: ResNet-18 | ACC = 90.71% Sens = 92.50%, Spec = 90.00% | Performance comparable to that of practicing radiologists. | To refine the clinical prognostic model with varying risk thresholds associated with different clinical prognoses. | |
| Abdel-Basset et al. ( | Italian Society of Medical and Interventional Radiology | before Apr 11, 2020 | Few-shot segmentation (FSS) with four encoder blocks based on pre-trained Res2Net-50 | DSC: 0.798 Sens: 0.803, Spec: 0.986 | Model could outperform all approaches to multiple evaluation metrics | i) Comprehensive parameter improving to attain the highest results, ii) Predictions lack laborious uncertainty quantification, unable to achieve a very precise segmentation iii) Accountability and interpretability do need to be improved. | |
| Roy et al. ( | ICLUS-DB | Mar – Apr, 2020 | ConvNet similar to van Sloun and Demi ( | ACC: 96% binary Dice score: 0.75 | i) Fully-annotated dataset of LUS images, ii) Predicts the disease severity score associated with a input frame. | i) Leveraging the temporal structure between frames in a sequential model; ii) The data set should be wider and more balanced | |
| Banerjee et al. ( | Hospital in Brazil | Mar 28 – Apr 3, 2020 | i) ANN; ii) random forest (RF) and Lasso-elastic-net regularized generalized linear (glmnet); iii) simple logistic regression (LR) | (i) (ROC) AUC 0.95 ± 0.08 (ii) (ROC) AUC: 94% (iii) (ROC) AUC: 81% | Improve initial screening for patients with limited PCR-based diagnostic tools. | Random forests and glmnet offer a clearer overview of the most relevant factors, compared to ANN, as well as a better indicator on how a decision has been reached. | |
| Pan et al. ( | 2 isolation centers of Huazhong University of Science and Technology in Wuhan | Until Mar 31, 2020 | COVID-Lesion Net based on a combination of U-net and Fully convolutional networks | Dice coefficient: 82.08% 85.00% for the training | Deep learning-based quantification for COVID-19, quantification of the lung volume and the percent of the lung involvement. | i) Performance measured against no standard for the lesion area quantification for viral pneumonia, ii) Not multi-center training | |
| Ismael and Şengür ( | three different sources (Cohen, Kaggle, Radiology Assistant) | Mar 10, 2020 | deep features model (ResNet50) and SVM with Linear kernel | 94.7% accuracy other: 89.1%- 90.3% | Three CNN deep methods have been applied. In addition to different kernel functions, the deep features have been classified through SVM. | More testing needed. | |
| Lopez-Rincon et al. ( | NCBI database of genetic variation and NGDC (National Genomics Data Center) | Mar 15, 2020 | CNN | Accuracy of 98.73 | The network was able to systematically discover significant sequences to isolate the various virus classes. | Further testing is necessary |
During the first peak of the COVID-19 pandemic (stage 3), the principal affected countries were in Europe and America and therefore the databases generally come from these areas. The first examples of social network analysis are reported, with a limited number of instances. The temporal windows during which the data were gathered extend until April 2020. Despite the time interval reported for Mackey et al. (2020), the relationship with Stage 3 for the COVID-19 is due to the fact that in the USA, by that time, the pandemic phase was still in the first stages
Timing and data availability in research papers after the first peak of the COVID-19 pandemic (stage 4)
| Reference | Dataset | COVID19 Data | Time interval | AI/ML method | Performance | Relevance | Shortcoming |
|---|---|---|---|---|---|---|---|
| Doanvo et al. ( | CORD-19 dataset (Lu Wang et al. | before Jul 31, 2020 | NLP & SVD & LDA | N.A. | ML explores latent semantic information to recognize hidden patterns and does not rely on any a priori knowledge of topics. | LDA is an unsupervised probabilistic algorithm and lacks the quality of a supervised method. | |
| Ramchandani et al. ( | SafeGraph; Mapbox and The New York Times GitHub repository | Apr 5 – June 28, 2020 | deep learning model based on the high-level framework of DeepFM | average ACC: 63.7% | Method can derive embeddings from multivariate time series and multivariate spatial time series data by using both the temporal and spatial structure in a wide range of input features. | No suitable method for interpreting second-order interactions; higher-order interactions are only indirectly captured and cannot therefore be easily interpreted. | |
| Kim et al. ( | Google Search Trend; and datasets in | Mar 22 – May 5, 2020 | Two-level hierarchical architecture of Hi-COVIDNet model, which mainly consists of the country-level encoder and the continent-level encoder. | RMSE: 0.4045 RMSE (ARIMA):0.4931 RMSE (multiv. LSTM): 0.5188 | Exploit the geographic hierarchy as well as a hierarchical objective function to overcome a relatively short period of data collection for COVID-19. | Further testing is needed, on other country data. | |
| Minaee et al. ( | COVID-19 Image Data Collection (Cohen et al. | before May 3, 2020 | ResNet18, ResNet50, SqueezeNet, and DenseNet-161 | Model Spec: ResNet18: 90.7 | Made 5k images dataset publicly available. Used transfer learning, and fine-tuning on pre-trained convolutional models | Only benchmark for future works and comparisons.It is worth noting that, considering the amount of data labeled, the outcome of the work is still preliminary and a more definitive conclusion needs more tests on a larger dataset of the COVID-19 labeled X-ray images. | |
| Horry et al. ( | COVID-19 Image Data Collection (Cohen et al. | before May 11, 2020 | Model Selection with VGG19 and others | F1 score: a)0.84-0.89, b) 0.81-0.83 c) 0.96-1.00 | Provided a pre-processing pipeline aimed to remove the sampling bias and improve image quality. Showed pre-trained models tuned effectively for the Ultrasound image samples. | Needs great caution in the development of clinical diagnostic models using the available COVID-19 image dataset. To extend study with multimodal data fusion. A highly curated data set is not comparable to the available COVID-19 chest X-Ray dataset. | |
| Jin et al. ( | 3 centers in Wuhan; MosMedData, Tianchi-Alibaba, LIDC–IDRI databases | Feb 5 – Mar 29, 2020 | i) Segmentation module based on U-Net, ii) Deep network backbone ResNet152, iii) Guided Grad-CAM for attentional regions. | AUC: 0.9745 - 0.9885 Dice (segmentation): 92.55% | AI system outperforms all of radiologists. Unlike classical black-box deep learning approaches, it can decode effective representation of COVID-19 on CT imaging. | Guided Grad-CAM can only extract attention region rather than lesion segmentation. It is important to collect more data and build a wide data set with linked CT and clinical information to allow additional diagnostic analysis. | |
| Sadefo Kamdem et al. ( | Boursorama database; COVID-19 dataset | before Apr 24, 2020 | ARIMA-WBF model and LSTM model | ACC: 92.13% - 97.45% | Forecasting commodity prices and examining the effect of coronavirus on commodity price fluctuations. | Application of reinforcement learning. Not analyzed price overreaction behavior. | |
| Ou et al. ( | EIA weekly gasoline demand; datasets in | Feb 15 – June 5 2020 | PODA model, has 42 inputs, 2 layers, and 25 hidden nodes for each layer | RMSE: 6.2 - 65.2 | Framework to investigate and project motor gasoline demand based on COVID-19 pandemic impacts. | Model does not consider the dynamic effect of travel mobility and assumes that the demand for gasoline from light-duty vehicles and other sectors is constant throughout the pandemic. | |
| Wang et al. ( | RSNA Pneumonia Detection Challenge & Cohen Dataset | Jan 25 - May 1, 2020 | ResNet50 + feature pyramid network (FPN) | accuracy of 95.12% | i) Automatically identify COVID-19 patients with X-rays. ii) Automatic lung detection. iii) Propose CAD tool for assisting in the processing of large-scale chest X-ray data | Lack of interpretability and not addressed disease classification by severity. | |
| Gupta et al. ( | Five different virus diseases (i.e., Covid-19, Ebola, MERS, SARS, Swine flu) | Jan 22 - Oct 9, 2020 | Dense layers with LSTM | RMSE: 0.0766-0.0533 RMSE (SVM): 0.2801-0.4323 RMSE (DT): 0.1108-0.1223 | The proposed DL model has been compared to other popular prediction methods that indicate a lower RMSE. | For the model to work perfectly, the data must be accurate. |
The time intervals generally extend from March to October, 2020. There is a wide variety of data sets, with different kinds of diagnostics images and social data. In particular, images derive from both CT and X-Rays, and many researchers use the Cohen et al. (2020b) X-Ray Dataset. In this stage data are freely available and are generated not only by hospitals: a wide variety is of social data can be found, e.g. encompassing gasoline demand Ou et al. (2020) and numbers of incoming visitors to a country Kim et al. (2020) . The different time positioning of Jin et al. (2020) concerning the Stage 4 of the proposed temporal subdivision, can be found in the fact that the authors refers to their country situation, the China, where the pandemic was already in an advanced state with respect to the rest of the world
Fig. 3AI can be defined in terms of stages of observation and action. In the context of a pandemic, AI is applied in two main areas, namely medical research and the social context. Therefore, in order to study AI applied during a pandemic, we need to focus on four areas: disease detection (diagnosis), social dynamics observation (predictions), medical actions (treatments) and social management (tracing). Points plotted on the polar graph represent the papers in the literature. The positions deponent on how much each paper we analysed we considered belonging to each of area, within the 3 central stages of the pandemic
Fig. 4The ROC-AUC trend. Calculated by analysing best and worst classification ROC scores reported into COVID-19 related papers. For each phase, the trend of the best and worst values are calculated and mean trends fitted with a spline, and by showing the papers reported in our tables that reported AUC values