Tracking machine learning models for pandemic scenarios: a systematic review of machine learning models that predict local and global evolution of pandemics.

This systematic review aims to study and classify machine learning models that predict pandemics' evolution within affected regions or countries. The advantage of this systematic review is that it allows the health authorities to decide what prediction model fits best depending upon the region's criticality and optimize hospitals' approaches to preparing and anticipating patient care. We searched ACM Digital Library, Biomed Central, BioRxiv+MedRxiv, BMJ, Computers and Applied Sciences, IEEEXplore, JMIR Medical Informatics, Medline Daily Updates, Nature, Oxford Academic, PubMed, Sage Online, ScienceDirect, Scopus, SpringerLink, Web of Science, and Wiley Online Library between 1 January 2020 and 31 July 2022. We divided the interventions into similarities between cumulative COVID-19 real cases and machine learning prediction models' ability to track pandemics trending. We included 45 studies that rated low to high risk of bias. The standardized mean differences (SMD) for the two groups were 0.18, 95% CI, with interval of [0.01, 0.35], I 2 =0, and p value=0.04. We built a taxonomic analysis of the included studies and determined two domains: pandemics trending prediction models and geolocation tracking models. We performed the meta-analysis and data synthesis and got low publication bias because of missing results. The level of certainty varied from very low to high. By submitting the 45 studies on the risk of bias, the levels of certainty, the summary of findings, and the statistical analysis via the forest and funnel plots assessments, we could determine the satisfactory statistical significance homogeneity across the included studies to simulate the progress of the pandemics and help the healthcare authorities to take preventive decisions.

Introduction

The social motivation

Pandemic scenarios are always challenging, because they need massive, fast, and practical actions to save lives. The SARS-CoV-2 pandemics have become a unique event in the twenty-first century of humankind, surpassing all pandemic effects caused by the Spanish flu in 1918. All social complexities and gaps in a world where the current population walks toward the 8 billion inhabitants mark overcome all realities of a 1.5 billion crowded world from 1918. The world’s inhabitants from 1918 did not have effective vaccines or medications to fight against the H1N1 virus. Such a combination resulted in around 50 million deaths, and one-third of the world population was infected (Jordan 2019).

During COVID-19 pandemics, developed nations keep at least 50% of all global vaccine doses available and leave emerging countries behind, increasing the chances of the virus spreading across a population (Fox 2020).

The biological motivation

The chances of novel virus variants surging (Gallagher 2021) increase significantly when the vaccination process covers only a tiny parcel of the population. The natural virus variance mechanism is always fast enough to find more efficient ways to hijack human host cells and incubate and replicate faster. The result can be catastrophic and cause the pandemic scenario to go out of control of healthcare authorities (Shang et al. 2020). Once in this stage, it may become hard to retake control of the health situation in a region affected by an outbreak. That is why, virus contact tracing plays a critical role in reducing infections. However, virus contact tracing is an arduous manual task and is not enough when a novel virus jumps into humans (Buhat et al. 2021; Shufro 2020).

Both social and biological aspects have motivated the scientific community to find fast responses to protect the population since 2020. Applied computing has been collaborating with the health area by providing computational models that can contribute by attempting to predict the next steps ahead of the pandemic.

Ghaderzadeh and Asadi (2021) provide a systematic review of the current state of all models for detecting and diagnosing COVID-19 through radiology modalities and their processing based on deep learning (DP). Alyasseri et al. (2022) provided a comprehensive review of the most recent DL and machine learning (ML) techniques for COVID-19 diagnosis. Syeda et al. (2021) conducted a systematic review of the literature on the role of artificial intelligence (AI) as a technology to fight the COVID-19 crisis in the fields of epidemiology, diagnosis, and disease progression.

Objective

Indeed, all previous systematic reviews cited in Sect. 1.2 are critical guides to helping the health area to thrive during the pandemic. However, our systematic review presents an essential advantage to help health authorities to save essential time and be steps ahead of the disease. This systematic review aims to review ML model studies that track pandemic scenario trends within an affected region compared to the actual trends in the same region. The goal is to answer the question: how effectively do ML models behave compared to the reality in the regions affected by the pandemics? The results obtained in this systematic review will work as a reference for healthcare authorities to make fast decisions and position steps ahead of the pandemics, mitigate deaths, and prevent overcrowded hospital admissions. The scientific contribution of this systematic review is to use the lessons learned from the ML models to apply them appropriately in future pandemic scenarios and be preventively successful for population health safety.

The usage of ML models helps attenuate out-of-control pandemic scenarios once the prediction matches possible critical situations with a grade of certainty. To do that, we need to analyze the works published where authors proposed ML model predictions and how close the ML approached real scenarios in territories where COVID-19 caused damage.

To organize the results of this systematic review, we will use the Participants, Intervention, Comparison and Outcomes (PICO) approach (Moher et al. 2009; Liberati et al. 2009).

Materials and methods

We performed a systematic review of the articles that analyze COVID-19 pandemic scenarios and computational models used for intervention, prediction, and mitigation of the critical aspects of the virus spreading to optimize the conduction of the pandemic phase by sanitary authorities.

The systematic review uses Preferred Reporting Items for Systematic reviews and Meta-Analyses (PRISMA 2020 statement) to ensure the necessary transparency, protocol application, and guidelines (Moher et al. 2009; Liberati et al. 2009; Page et al. 2021).

This section discusses the research design and the steps used to achieve the goal of this systematic review. This paper addresses the eligibility criteria, the information sources, the research questions, the study selection, the data collection, and the article selection process.

Eligibility criteria

We have specified inclusion and exclusion criteria as synthesis to reach the necessary grouped studies.

The search takes place in July 2022 to identify meta-analyses focusing on computational models used for intervention, prediction, and mitigation of SARS-CoV-2 pandemic scenarios published between January 2020 and July 2022.

We have defined and refined the search string using the PICO approach (Booth et al. 2019). PICO covers PRISMA checklist topics, such as objectives, search questions, and eligibility criteria, by clustering participants, interventions, comparisons, and outcomes to facilitate our searches.We considered articles written in English only. We performed the search and retrieval in all databases on July 2022. We excluded theses, dissertations, opinions, criticism, protocols, books, posters, oral presentations, general surveys, and previous non-related systematic and literature reviews.

Participants: We included randomized controlled trials of the COVID-19-positive population within specific territories. We considered individual mobility within geolocation as a factor of virus transmissibility.

Intervention: we considered all interventions identified by the search where a pre-defined algorithm or plan aims to predict the following steps within the pandemic scenario and how such methods can improve critical scenarios. We do not consider analyses and models that do not offer options to sanitary authorities.

Comparison: We assess the studies where ML prediction models for pandemic scenarios can help sanitary authorities map and plan necessary steps to control, mitigate, or prevent the virus from spreading.

Outcomes: To be included, a trial had to use ML or mathematical models (MM) that consider the COVID-19-positive population, the territory in scope, the mobility of individuals (whenever present), geolocation information (or a possibility to consider such), the power of ML or MM to focus on helping sanitary authorities management to tackle the scenario and to take over the control over the situation again (examples: prediction of future virus waves, preventive actions like lockdown or restrictions to slow down the virus spreading, or what vaccines are more effective depending upon the virus variants detected in a studied territory). We exclude reports with analyses of ML models without aiming to help sanitary authorities (examples: surveys about methods in ML, MM focused only on simulation, but without a goal of efficient actions toward population).

We present the detailed inclusion and exclusion criteria established in this systematic review below:

Inclusion criteria: articles that use ML to attempt to build or identify a critical pandemics region due to a significant virus spreading; articles considering geolocation studies and implementations within a pandemic territory study; articles using reinforcement learning as an alternative to help mitigate the pandemics; articles that are focusing on pandemics prediction or evolution within a specific territory using ML; articles considering machine or specific deep learning prediction approaches intended to help sanitary authorities; articles concerning SARS-CoV-2 variants and spreading trends over regions; articles considering population mobility between locations affected by SARS-CoV-2.

Exclusion criteria: non-ML models, articles involving COVID-19 drug matching or finding; articles exploring ML closed to medical standpoints only, like medical imaging, X-ray, tomography to help identify COVID-19 presence; informative medical articles with superficial, generic, or overview medical interventions or analysis against SARS-CoV-2; articles exploring socioeconomic aspects, or blockchain methods during or after pandemics; reports about the usage of wearable IoT devices to help detect COVID-19 presence or exposure; articles focusing on population behavioral issues, mental health changes, or education impacts on the society during the pandemics; articles using algorithms to detect death tolls per se, without objectives to predict virus spreading or improvements in pandemic scenarios; articles about hospital capacity matching or optimization through ML algorithms (except when ML algorithms can also be used for epidemiological tracking); articles where bio environment aspects are involved as an effect of pandemics, like air pollution, climate change, or city water quality; general, direct, or indirect survey articles, which aim to cite or compare ML techniques as their focus.

Information sources

On 31 July 2022, we searched each of the databases listed in Table 1. We adapted the base search string from Table 2 to ensure that each database will return correct records to match our systematic review objectives (see Appendix A for details). We describe further information about the search strategy employed in this paper in Sect. 2.3. We queried the database by including the exact dates (01 January 2020 to 31 July 2022). However, some databases are limited to the publication year only. For these cases, we included 2020 to 2022 and retrieved the available records until 31 July 2022.

Table 1

All databases covered in this systematic review

Database	Coverage
ACM Digital Library	January 2020 to July 2022
BMJ Journals	January 2020 to July 2022
BioRxiv + MedRxiv	January 2020 to July 2022
EBSCOhost
Computers and Applied Sciences	January 2020 to July 2022
IEEE Xplore	January 2020 to July 2022
ElSevier
ScienceDirect	January 2020 to July 2022
Scopus	January 2020 to July 2022
Journal of Medical Internet Research
JMIR Medical Informatics	January 2020 to July 2022
Ovid
MEDLINE(R) Daily Update	January 2020 to July 2022
Oxford Academic (all journals)	January 2020 to July 2022
NCBI PubMed PMC	January 2020 to July 2022
Sage Journals Online	January 2020 to July 2022
Springer Nature
Biomed Central Journals	January 2020 to July 2022
Nature	January 2020 to July 2022
Springer Link	January 2020 to July 2022
Web of Science	January 2020 to July 2022
Wiley Online Library	January 2020 to July 2022

Table 2

Base search string we use during databases record retrieval

Search String
(COVID-19 OR SARS-Cov-2) AND (pandemic scenario) AND ((epidemiological model) OR (predictive model)) AND (machine learning)

Search strategy

To execute the search strategy, we need to define general and specific research questions to cover applications and results for the computational models and techniques found during the systematic review.

Here are the General Questions (GQ) addressed by this study:These are the Specific Questions (SQ) employed during this systematic review:We defined the following base search string based on general and specific questions to query all eligible and pertinent databases defined in Table 1.

GQ1. What were the primary studies regarding ML COVID-19 models to track pandemics’ evolution within a territory?

GQ2. What are the challenges in algorithms and computational methods to help sanitary authorities control pandemic scenarios?

SQ1. How do studies map COVID-19-positive population within the territory through ML models?

SQ2. What studies consider SARS-CoV-2 virus variants tied to COVID-19-positive population within a specific territory?

SQ3. What studies help healthcare authorities predict real-time or soft real-time pandemics evolution via ML models?

SQ4. What studies focus on building geolocation maps based on the COVID-19-positive population?

We detail the individual search strings for each queried database in Appendix A.

The search strategy development process considered candidate search terms identified by looking at words in those records’ titles, abstracts, and subject indexing. The databases strategy uses the Cochrane RCT filter reported in the Cochrane Handbook (Lasserson et al. 2019; Thomas et al. 2019).

Selection process

We combined two approaches to ensure accuracy during the selection process: assessment of records by more than one reviewer (Gartlehner et al. 2020; Waffenschmidt et al. 2019; Wang et al. 2020), and crowdsourcing (Noel-Storr 2019; Nama et al. 2019). In the case of crowdsourcing, we used the Mendeley Reference Manager (MRM) Tool as a central repository of records for teamwork. For this purpose, the tool has a specific feature named Private Group.

We did not use any ML approach to perform the selection process. Three researchers divided the screening tasks into abstracts, titles, keywords, and full-text articles. In case of inconsistencies or disagreements on such stages, the three researchers discussed until they had reached a consensus.

Data collection process

We have followed meticulous steps for the retrieval, selection, inclusion, and exclusion of records during this systematic review. To achieve the result, we executed the tasks detailed in the workflow from Fig. 1. We used the following tools to assist the data collection:

Fig. 1

Steps performed to execute the data collection process

Mendeley Reference Manager (MRM): the central repository of records collected through the search string from Table 2

Mendeley Desktop Tool (MDT): the tool that removes duplicated records

Mendeley Web Importer (MWB): the tool that imports records directly to MRM from the web search results pages for each database in Table 1

Data items

We organized the included studies in two domains and built a taxonomy to facilitate the analyses for the results section, according to Fig. 2.

Fig. 2

Taxonomic analysis and definition of (a) Domain 1: COVID-19 pandemics’ trending prediction and (b) Domain 2: COVID-19 geolocation tracking and prediction

All models and algorithms derived from ML techniques are eligible for inclusion. ML models focusing on predictions of pandemic spreading and geolocation will have preference during the inclusion process.

We collected data on:

The report: author, year, and source of publication.

The study: types of ML techniques used, sample applications

The participants: COVID-19-positive individuals and their geographical location, whenever available

The intervention: type, duration, timing, and mode of delivery of the employed ML model.

Risk of bias

We assessed the risk of bias (RoB) in the included studies by following the revised Cochrane risk-of-bias tool for randomized trials (RoB 2.0) (Higgins et al. 2019; Collaboration 2021). RoB 2.0 addresses five specific domains: Three review authors independently applied RoB 2.0 tool to each included study. They recorded supporting information and justifications for judgements of risk of bias for each domain (low, high, or some concerns).

Bias arising from the randomization process

Bias due to deviations from intended intervention

Bias due to missing outcome data

Bias in measurement of the outcome

Bias in selection of the reported result.

We discussed and resolved all discrepancies in judgments of RoB or justifications for judgments to reach a consensus among the three review authors. By following the guidelines from RoB 2.0 tool (Sect. 1.3.4) (Higgins et al. 2019), we derived an overall summary risk of bias judgement (low, high, or some concerns) for each specific outcome.

The review authors determined RoB for each study through the highest RoB level in any of the assessed domains.

For all studies with missing outcomes, we proceeded with instructions from Sects. 10.12.2 and 10.12.3 of the Cochrane Handbook for Systematic Reviews of Interventions (Higgins et al. 2019) and did not exclude the studies. Instead, we considered the effects of the missing outcome during the risk-of-bias determinations.

Effect measures

According to Cochrane Handbook for Systematic Reviews of Interventions (Higgins et al. 2019), our approach is to analyze the effect measures using the continuous data method when presenting the synthesis of the meta-analysis in this systematic review. We applied the effect size measures by first testing fixed-effect and random effects to determine the best statistical approach. We also adopted standardized means differences by summarizing the effect size measures within a forest plot.

Three review authors independently applied Cochrane Review Manager 5.4.1 tool (Collaboration 2020) to assess each included study and determine the effect size measure for all interventions and forest plot presentation. In case of any discrepancy or dual judgments, the three review authors resolved the conflict through discussion to reach a consensus.

Synthesis methods

To get the included studies that answer the researched questions, we categorized the research questions into the following main ML categories for the employed interventions: This approach helped us build a corresponding taxonomy map to facilitate the answers to the research questions framework and make the Grading of Recommendations Assessment, Development, and Evaluation (GRADE) summary of findings (Schunemann 2013).

COVID-19 pandemic trending prediction models

COVID-19 geolocation tracking and prediction models.

Within the GRADE summary findings, we could assess the risk of bias and level of certainty within the 95% confidence interval of grouped and pooled results.

We used the synthesis methods described in the Cochrane Handbook for Systematic Reviews of Interventions (Higgins et al. 2019). For this systematic review, we used the following synthesis meta-analysis aspects regards effect measures: According to Sect. 10.10.2 of Cochrane Handbook for Systematic Reviews of Interventions (Higgins et al. 2019), the simple usage of to determine heterogeneity has low power in the typical situation of a meta-analysis when studies have a small sample size or are few. Instead, we adopt the approach to measure heterogeneity for better precision.

We divided the meta-analysis into two comparison groups: actual COVID-19 cases’ detection versus ML COVID-19 cases’ prediction.

We got study heterogeneity in random-effects analysis. Section 10.10.4.1 of Cochrane Handbook for Systematic Reviews of Interventions (Higgins et al. 2019) states we should never choose between a fixed-effects and a random-effects meta-analysis based on the statistical test for heterogeneity only. However, we tested both fixed-effects and random-effects meta-analysis, and both approaches returned the same pooled results within the 95% confidence interval. Such developments and statistical significance led us to choose a fixed-effects meta-analysis approach.

We used standardized mean difference analysis to walk toward statistical homogeneity, because each study has a different number of observations within different periods.

As part of fixed-effect and standardized mean difference analysis, we applied the inverse variance statistical method by considering a 95% of confidence interval (CI).

We prepared the synthesis results within a forest plot. The forest plot determines important synthesis parameters by including studies via p value and the heterogeneity measurement. We applied the Cochrane Review Manager 5.4.1 tool (RevMan) to generate funnel and forest plots. RevMan includes RoB 1.0 assessment. However, we replaced it with the updated RoB 2.0 for risk-of-bias analysis (Collaboration 2021).

We built a funnel plot to detect possible publication bias during the analysis of the results (Sects. 13.3.5.2 and 13.3.5.3 of Cochrane Handbook for Systematic Reviews of Interventions (Higgins et al. 2019)). We also linked the included studies to the research questions framework (both general and specific questions). For all studies that are missing outcomes, we proceeded with instructions from Sects. 10.12.2 and 10.12.3 of the Cochrane Handbook for Systematic Reviews of Interventions (Higgins et al. 2019) and did not exclude the studies. Instead, the forest and funnel plots discarded the studies with missing numerical outcomes.

Since the meta-analysis resulted in fixed effects and the heterogeneity coefficient is . We agreed that we did not need to perform further meta-regression, sensitive, and other subgroup analyses, as they did not influence the final results of this systematic review.

Results

We now present the detailed results after applying the methods described in the previous sections. All results follow the PRISMA 2020 checklist (Higgins et al. 2019) and PRISMA protocol (Moher et al. 2015).

PRISMA workflow

The search strings initially returned 7031 records in database searching (see Table 3). We organized the results by following the PRISMA workflow according to Fig. 3.

Table 3

All databases used for this systematic review

Database	Records retrieved
The ACM Guide to Computing Literature	638
Biomed Central Journals	61
BioRxiv + MedRxiv	93
BMJ Journals	25
Computers and Applied Sciences (EBSCOhost)	965
IEEE Xplore	16
JMIR Medical Informatics	168
Medline Daily Updates (Ovid)	165
Nature	114
Oxford Academic	119
PubMed PMC	1795
Sage Journals Online	370
ScienceDirect (Elsevier)	1289
Scopus (Elsevier)	21
SpringerLink	458
Web of Science	17
Wiley Online Library	717
Total	7031

Fig. 3

PRISMA 2020 workflow

After removing duplicated records, we marked records as ineligible by automation and excluded records for other reasons. We screened 4,819 records for seeking and based on the abstracts and keywords, and we excluded 4,581 records else. Then, from the resultant 217 records available for retrieval, we excluded seven records we could not retrieve.

We then reviewed 210 full-text documents and reached 45 included studies, grouped into 30 new unique studies. Finally, we performed meta-analyses of the results.

Taxonomy of the included studies

We obtained the 30 new studies from the 45 reports by performing a detailed taxonomy by dividing them into two main clusters, as detailed in Sect. 2.9.

Figure 4 describes the entire taxonomic organization of all studies in this systematic review.

Fig. 4

Taxonomy of the studies included in this systematic review

Domain 1: COVID-19 pandemics trending prediction: 38 studies

Stochastic, principal component analysis (PCA), and logistic models: 7 studies

Reinforcement learning (RL): 3 studies

Recurrent neural networks (RNN) with long short-term memory (LSTM) architecture: 7 studies

Time-series: 8 studies

Gate recurrent unit (GRU): 2 studies

Convolutional neural network (CNN) with logistic regression: 1 study

Multilayer perceptron (MLP) neural network: 2 studies

Linear regression: 4 studies

Ensemble classifiers: 4 studies.

Domain 2: COVID-19 geolocation tracking and prediction: 7 studies

Ensemble regression learning: 1 study

RL: 2 studies

Fuzzy analytic hierarchy process (AHP) with Fuzzy technique for order of preference by similarity to ideal solution (TOPSIS): 1 study

AHP: 1 study

Random forest: 2 studies.

Answers to the research questions

The included studies addressed all research questions proposed by this paper, with different densities. We show the relationship between the included studies and the addressed questions in Table 4 and discuss them later in this section.

Table 4

Answers to research questions

Author, year	Country	Proposed methodology	Questions
Abbasimehr et al. (2022)	Iran, USA, Italy	Use time-series augmentation and Ensemble Bagging ML model to predict the incidence of COVID-19.	GQ1, GQ2, SQ1, SQ3
Abdallah et al. (2022)	Tunisia	Propose a new strategy based on reinforcement learning (RL) to study the dynamics of COVID-19 evolution	GQ1, GQ2, SQ1, SQ3
Ahouz and Golabpour (2021)	Iran	Predict the incidence of COVID-19 within two weeks to better manage the disease using the Least-Square Boosting Classification algorithm.	GQ1, SQ1, SQ4
Alali et al. (2022)	Saudi Arabia	Present an efficient machine learning method to forecast future trends of COVID-19 spread through Bayesian optimization	GQ1, GQ2, SQ1, SQ3
Alamrouni et al. (2022)	Saudi Arabia, Turkey, Nigeria, Cyprus	Predict the cumulative COVID-19 cases through ARIMA and ensemble ML models	GQ1, GQ2, SQ1, SQ3
Alassafi et al. (2022)	Saudi Arabia	Predict the cumulative COVID-19 positive cases by comparing RNN and LSTM neural networks’ efficiency	GQ1, GQ2, SQ3
Alanazi et al. (2020)	Saudi Arabia, Canada	Model for predicting COVID-19 using the SIR and ML for innovative health care.	GQ1, SQ3, SQ4
Amaral et al. (2021)	Brazil	A data-driven SIR model whose parameters are fully calibrated by temporal functions learned from individual regressors and trained on different data sources.	GQ1, SQ1
Barraza et al. (2020)	Argentina	Model the epidemic spread as births in a population, where every birth corresponds to a new infection case. The paper introduces a new Markovian stochastic model that can be described as a non-homogeneous Pure Birth process.	GQ1, GQ2
Basu and Campbell (2020)	USA	A prediction long short-term memory (LSTM) based model that can be trained on more than four months of cumulative COVID-19 cases and deaths.	GQ1, GQ2, SQ1
Bedi et al. (2021)	India	A modified SEIRD model is based on the SEIRD model coupled with a deep learning-based long short-term memory (LSTM) model for COVID-19 case prediction trending.	GQ1, SQ1, SQ3, SQ4
Bi et al. (2022)	USA	Present a GRU-based hybrid model to predict the spread of COVID-19 across USA	GQ1, GQ2, SQ1, SQ3
Bushira and Ongala (2021)	Namibia	Using geospatial technologies, identify and map COVID-19 risk zones and model future COVID-19 responses in Namibia.	GQ1, GQ2, SQ1, SQ4
Casini and Roccetti (2020)	Italy	Present three computational models of increasing complexity (linear, negative binomial regression, and cognitive) to identify the one that better correlates the relationship between Italian tourist flows during the summer of 2020 and the resurgence of COVID-19 cases across the country.	GQ1, SQ1
Chandra et al. (2022)	Australia, India	Employ LSTM models to forecast the spread of COVID-19 infections among selected states in India.	GQ1, GQ2, SQ1, SQ3
Chyon et al. (2022)	Bangladesh	Predict the spread of COVID-19 infections by employing ARIMA and time-series machine learning.	GQ1, SQ1, SQ3
de Araújo Morais and da Silva Gomes (2022)	Brazil	Employ ARIMA with neural network models to forecast the spread of COVID-19 infections	GQ1, SQ1, SQ3
Doornik et al. (2022)	UK	Build a short-term forecasting ML model for the Coronavirus pandemic	GQ1, GQ2, SQ3
Fang et al. (2022)	China	Propose a data-driven XGBoost model for the prediction of COVID-19	GQ1, GQ2, SQ1, SQ3
Garetto et al. (2021)	Italy	Compared to traditional compartmental models, a time-modulated Hawkes process to model the spread of COVID-19.	GQ1, GQ2
Giacopelli (2021)	Italy	A full-scale individual-based model of the COVID-19 outbreak to test various scenarios about the pandemic and achieve novel performance metrics.	GQ1, SQ1, SQ4
Haghighat (2021)	Iran	A combined multilayer perceptron (MLP) neural network and Markov chain (MC) model to predict two indicators of the number of discharged and death cases according to their relationship with the number of hospitalized patients.	GQ1, GQ2, SQ1, SQ3
Kolozsvári et al. (2021)	Hungary	Use the official epidemiological data to forecast the epidemic curves (daily new cases) of COVID-19 using Artificial Intelligence (AI)-based Recurrent Neural Networks (RNNs), then compare and validate the predicted models with the observed data.	GQ1, GQ2, SQ1, SQ3
Kou et al. (2021)	China	A multi-scale agent-based model (MSABM) for disease spread between cities and in a town. The model investigates the infectious disease propagation between cities and within a city using the knowledge from person-to-person transmission.	GQ1, GQ2, SQ1
Kuo and Fu (2021)	USA	Develop a COVID-19 case predicting model based on county-level data with ML techniques, and the next-1-day (N1D), 4-day (N4D), and 7-day (N7D) averages of daily cases and cumulative cases would be used as a response respectively.	GQ1, GQ2, SQ1, SQ4
Majhi et al. (2021)	USA, India	Build predictive models that can predict the number of positive cases with higher accuracy. Regression-based, Decision tree-based, and Random forest-based models have been built on the data from China to validate India’s sample.	GQ1, SQ3
Malakar (2021)	India	Model the COVID-19 vulnerability using an integrated fuzzy multi-criteria decision-making (MCDM) approach, namely fuzzy-analytical hierarchy process (AHP) and fuzzy-technique for order preference by similarity to ideal solution (TOPSIS) for West Bengal, India, through geographic information system (GIS).	GQ1, GQ2, SQ1, SQ4
Mallick et al. (2022)	India	Predict the COVID-19 trajectories of the infected population of Indian states using SEIR mathematical model and LASSO ML.	GQ1, SQ1, SQ3
Marzouk et al. (2021)	Egypt	Apply artificial intelligence-based models to predict the prevalence of the COVID-19 outbreak in Egypt. These models are long short-term memory networks (LSTM), convolutional neural networks, and multilayer perceptron neural networks.	GQ1, GQ2, SQ1, SQ3
Senthilkumar et al. (2021)	India, Australia, Pakistan, UK, USA, United Arab Emirates	A multimodel ML technique called ensemble learning, autoregressive, and moving regressive (EAMA). The multimodel forecasts COVID-19 parameters in the long term within India and globally.	GQ1, SQ1
Namasudra et al. (2021)	India	Present a novel Nonlinear Autoregressive (NAR) Neural Network Time-Series (NAR-NNTS) model to forecast COVID-19 cases. The authors train NAR-NNTS model with Scaled Conjugate Gradient (SCG), Levenberg–Marquardt (LM), and Bayesian Regularization (BR) training algorithms.	GQ1, GQ2, SQ1, SQ3
Nobi et al. (2021)	Bangladesh, South Korea	Apply principal component analysis (PCA) to the correlations for the changes of cumulative time-series of COVID-19 death and confirmed cases between different countries	GQ1, GQ2, SQ1, SQ3
Ohi et al. (2020)	Bangladesh, Saudi Arabia	Demonstrate what actions an agent (trained using reinforcement learning) may take in different possible pandemic scenarios depending on the spread of disease and economic factors.	GQ1, SQ1
Ozik et al. (2021)	USA, France	Present CityCOVID, which is a detailed agent-based model that enables the creation of efficient, urban-scale MPI-distributed agent-based models (ABMs).	GQ1, GQ2, SQ1, SQ3, SQ4
Pan et al. (2021)	Singapore, Taiwan, USA, Poland	A spatiotemporal analysis framework under the combination of an ensemble model (random forest regression) and a multi-objective optimization algorithm (NSGA-II).	GQ1, GQ2, SQ1
Pang et al. (2021)	China	Collect five indicators of the data of COVID-19 and its loss in Hubei Province. By applying PCA, the authors established the disaster loss index model of COVID-19 in Hubei Province.	GQ1, SQ1
Pourghasemi et al. (2020)	Iran, USA	Predicting mortality trends was built using regression modeling, spatial modeling, and risk mapping. The authors did the change detection using the random forest (RF) ML technique and the modeled risk map validation.	GQ1, GQ2, SQ1, SQ3, SQ4
Raheja et al. (2021)	India, UK	Present a novel approach (ML diffusion) prediction model for predicting the number of Coronavirus cases in four countries: India, France, China, and Nepal.	GQ1, GQ2, SQ1
Rashed and Hirata (2021)	Japan, Egypt	Analyze daily positive cases (DPC) data using a ML model to understand the effect of new viral variants on morbidity rates.	GQ2, SQ2
Sah et al. (2022)	Turkey, India, Saudi Arabia	Forecast COVID-19 Pandemic by combining ARIMA and LSTM-GRU Models in India	GQ1, SQ1, SQ3
Shoaib et al. (2021)	Taiwan, Pakistan, Thailand	Use the innovative design of the NAR-RBFs neural network paradigms designed to construct the SITR epidemic differential equation (DE) model to ascertain the different features of the spread of COVID-19.	GQ1, GQ2, SQ1
Swaraj et al. (2021)	India, Brazil	Present an ensemble model that integrates an autoregressive moving average model (ARIMA), and a nonlinear autoregressive neural network (NAR) to predict future cases of COVID-19.	GQ1, GQ2, SQ1
Ullah et al. (2022)	USA, Canada	Apply nonlinear modal regression and Monte Carlo algorithm to predict COVID-19 confirmed cases.	GQ2, SQ1, SQ3
Vasconcelos et al. (2021)	Brazil	A generalized logistic growth model with time-dependent parameters to describe the fatality curves of the COVID-19 disease. Such an approach applies to several countries that exhibit multiple waves of infections.	GQ1, GQ2, SQ1
Zivkovic et al. (2021)	Serbia, India, Vietnam, Turkey	Improve the current time-series prediction (forecasting) algorithms based on hybrid ML and nature-inspired algorithms to help predict future COVID-19 cases.	GQ1, GQ2, SQ1

We have the following density of research questions answered:Notice that SQ2 returned only one study (Rashed and Hirata 2021) from the search strategy executed between January 2020 and July 2022. One possible reason is that SARS-CoV-2 virus variants’ impact acknowledgment began around the third quarter of 2020, and that would explain the lack of ML models focusing on virus variants. However, we need to consider the virus variants prediction and spreading during epidemiological studies as essential variables in building prediction models.

GQ1: 29 studies (96%)

GQ2: 20 studies (69%)

SQ1: 23 studies (84%)

SQ2: 1 study (2%)

SQ3: 9 studies (53%)

SQ4: 9 studies (20%).

Some included studies have quick access to real-time COVID-19 confirmed cases data provided by their country governments or reliable institutions, facilitating predicting models more quickly. For instance, India’s studies frequently cited the same data provided by government authorities. USA studies relied on Johns Hopkins COVID-19 Map1. Other studies frequently use official data from World Health Organization (WHO) databases, which may depend upon countries’ slower case notifications. This scenario may stimulate some countries to step ahead in producing more studies about prediction models, while others do not have the same advantage.

Risk-of-bias results

We assessed the risk of bias using RoB 2.0 tool (Higgins et al. 2019; Collaboration 2021) (see Sect. 2.7 for the domains assessed in this systematic review).

In terms of the overall risk of bias, the following studies presented high risk (5 of 45 included studies) (Kuo and Fu 2021; Pan et al. 2021; Pang et al. 2021; Shoaib et al. 2021; Zivkovic et al. 2021).

The following studies presented some level of concern (13 of 45 included studies) (Abdallah et al. 2022; Alamrouni et al. 2022; Bi et al. 2022; Chyon et al. 2022; Majhi et al. 2021; Senthilkumar et al. 2021; Nobi et al. 2021; Ohi et al. 2020; Ozik et al. 2021; Raheja et al. 2021; Sah et al. 2022; Ullah et al. 2022; Vasconcelos et al. 2021).

We present the risk of bias for each study in Fig. 5. We used the RobVis R tool to generate the risk of bias reports (McGuinness 2019).

Fig. 5

Risk of bias from RoB 2.0 tool for each included study

We applied RoB 2.0 Tool over the two branches of our taxonomic analyses. That means we consider the risk of bias analysis over the context of each taxonomic context.

The studies from Kou et al. (2021) and Alanazi et al. (2020) show zero observations in the forest plot. However, they present outcome data according to their study contexts, which is why we rated their risk of bias low. The study from Haghighat 2021 (Haghighat 2021) shows zero observations in the forest plot, because it focuses on three categories only: hospitalized, deaths, and discharged COVID-19-positive patients. The total of cases is not the simple sum of such classes. However, the risk of bias is low, because the study shows the outcomes as expected (Fig. 6).

Fig. 6

Risk-of-bias summary from RoB 2.0 tool

Overall effect size results of the individual studies

We present the forest plot in Fig. 7 by including all the meta-analysis results from each included study in this systematic review.

Fig. 7

Forest plot for effect sizes. The figure displays the meta-analysis consisting of the summary statistics [standardized mean (SMD), standard deviation (SD), sample size, and weight proportion] for comparison between predicted cases by ML model vs. actual COVID-19 cases. It also displays the mean difference and its 95% confidence interval (CI) for the continuous outcome

We need the forest plot analysis to determine the accuracy of the ML prediction models to represent actual cumulative COVID-19 cases. Therefore, the more accurate the models are, the less risky they represent for public health authorities’ adoption.

The goal of a forest plot is to help physicians determine drug or treatment effectiveness within a population. Physicians are more interested in observing how many studies are far from the line of no effect and not passing through the diamond extension within the forest plot. Anything different from this approach means no statistical significance for physicians.

In our systematic review, we explore the exact opposite effect and interpretation. The studies closer to the line of no effect mean that they simulate real COVID-19 case scenarios more accurately. The same observation approach is valid for studies crossing the forest plot diamond. That is the statistical significance we aim for in this systematic review.

On the other hand, if actual case data reveal as too favored in the forest plot, it may indicate a risk of bias on missing or unclear outcomes presented by the study. The scenario is analyzed later in this review.

From Fig. 7, three case prediction studies are far from the line of no effect: Basu and Campbell (2020); Casini and Roccetti (2020) and (Kolozsvári et al. 2021). For the real case’s side of the forest plot, only (Namasudra et al. 2021) is a little far from the line of no effect.

Results of syntheses

The forest plot presented heterogeneity for fixed and random effect studies. We chose the fixed-effects analysis for the reasons explained in Sect. 2.9.

Another critical aspect to observe in the forest plot is the summary of results: SMD=0.18, with 95% CI limits between [0.01, 0.35]. Since the interval does not cross the line of no effect, it means that the difference found between the two groups was statistically significant. As we used SMD calculation, the overall impact returns Z statistic and the corresponding p value: Z = 2.02 and p value=0.04. Furthermore, since p value, the summary result is inside the statistical significance threshold established by medical and psychological research (Sect. 15.3.2 of Cochrane handbook (Higgins et al. 2019)). In this specific case, the overall effect means that the included ML prediction studies are significantly valid in simulating the trend of the real cumulative COVID-19 infection cases.

The forest plot (Fig. 7) presents zero observations for most geolocation studies due to focusing on the pandemic prediction simulation, not geolocation. However, the zero observations are discarded by RevMan 5.4.1 and do not influence the results of forest plot statistical analyses. The zero observations reflect the risk of bias within domain D3 (bias due to missing outcome data). We indicate it as No information statement for missing outcomes.

Risk of bias due to missing results

Based on the forest plot from Fig. 7, we flagged the following studies with No information in domain D3 within the risk-of-bias analysis. The referred studies do not match the forest plot proposed synthesis (predicted vs. the actual number of cumulative COVID-19-positive cases) (Abdallah et al. 2022; Alali et al. 2022; Alamrouni et al. 2022; Alanazi et al. 2020; Bushira and Ongala 2021; Bi et al. 2022; Chandra et al. 2022; Chyon et al. 2022; de Araújo Morais and da Silva Gomes 2022; Haghighat 2021; Kou et al. 2021; Kuo and Fu 2021; Malakar 2021; Mallick et al. 2022; Nobi et al. 2021; Ohi et al. 2020; Ozik et al. 2021; Pan et al. 2021; Pang et al. 2021; Rashed and Hirata 2021; Sah et al. 2022; Shoaib et al. 2021; Ullah et al. 2022).

All the above studies present zero number of patients and the impossibility of SMD calculations. However, they do not alter the overall pooled effects calculated within the forest plot. Because of the reasons already detailed in Sect. 2.7, we did not exclude these studies, as they provide essential methodologies for this systematic review.

Risk of publication bias

We then grouped all studies that produced results to generate the funnel plot (Fig. 8) and analyzed aspects related to publication bias.

Fig. 8

Funnel plot based on non-zero studies from forest plot (Fig. 7). The x-axis shows the SMD, while the y-axis shows the standard error (SE)

The funnel plot places most of the studies closer to the funnel’s center, with only one study (Basu and Campbell 2020) as an outlier within 95% CI. This aspect of the funnel plot indicates a low probability of publication bias. In other words, the interpretation is that the ML prediction studies can simulate real-case scenarios with a high chance of certainty, reinforcing the p value=0.04 calculated by the forest plot.

The GRADE summary of findings also includes the publication bias when presenting the pooled study results for the certainty of evidence.

Certainty of evidence

We presented the certainty of evidence assessment through the GRADE summary of finding tables (Schunemann 2013) divided into the two domains defined in Sect. 2.6. The GRADE summary of findings tables was created and assessed based on the taxonomy defined in Sect. 3.2.

A summary of findings for domains 1 and 2 revealed the following pooled studies and corresponding levels of certainty. Further details are available in GRADE summary of findings tables in Figs. 9 and 10.

Fig. 9

Question for domain 1 GRADE assessment: ML prediction compared to real cumulative cases for tracking COVID-19 pandemics’ future trending. Explanations: (a) Mohan et al. 2021 study should compare real and predicted cases; (b) Raheja et al. 2021 study presents an abnormal standard error and increases the risk of publication bias; (c) Marzouk et al. 2021 study should compare actual and predicted cases; (d) Basu et al. 2020 appear as an outlier in the funnel plot, indicating possible imprecision; (e) Vasconcelos et al. 2021 bring results, but presenting a low-graph resolution, which may cause imprecisions; (f) Pang et al. 2021 study should bring the predicted vs. real cases, as this is the goal of this study; (g) Shoaib et al. 2021 bring no outcomes; (h) Haghighat 2021 brings no outcomes; (i) Pan et al. 2020 graphs with predicted vs. real cases have low resolution, affecting exact values from the study; (j) Ohi et al. 2020 and Kou et al. 2021 do not bring results about predicted vs. real cumulative cases; (k, l) Sah 2022 and Bi 2022 should bring the predicted vs. real cases, as this is the goal of this study. There is only continuity from actual to predicted cases (Senthilkumar et al. 2021; Raheja et al. 2021; Marzouk et al. 2021; Basu and Campbell 2020; Vasconcelos et al. 2021; Pang et al. 2021; Shoaib et al. 2021; Haghighat 2021; Pan et al. 2021; Ohi et al. 2020; Kou et al. 2021; Sah et al. 2022; Bi et al. 2022)

Fig. 10

Question for domain 2 GRADE assessment: ML prediction compared to real geographic location for tracking COVID-19 pandemics critical geolocations: (a) Ozik et al. 2021 have no outcomes for cumulative COVID-19 cases’ prediction. Giacopelli 2021 brings very few observations; (b) Malakar et al. 2021 bring imprecise outcomes for cumulated COVID-19 cases during modeling; (c) Bushira et al. 2021 have no outcomes for cumulative COVID-19 cases prediction; (d) Kuo et al. 2021 have no outcomes for cumulative COVID-19 cases’ prediction (Ozik et al. 2021; Giacopelli 2021; Malakar 2021; Bushira and Ongala 2021; Kuo and Fu 2021)

Domain 1: COVID-19 pandemics’ trending prediction

Moderate level of certainty: Linear regression, LSTM RNN, stochastic and PCA logistic models, and ensemble classifiers’ family

Low level of certainty: CNN with logistic regression, time-series neural network, MLP, and GRU

Very low level of certainty: Reinforcement learning.

Domain 2: COVID-19 geolocation tracking and prediction

High level of certainty: Ensemble regression learning

Low level of certainty: Fuzzy AHP and TOPSIS, AHP, and random forest

Very low level of certainty: Reinforcement learning.

Limitations

The first limitation of this systematic review is that each study has access to heterogeneous databases, and there are no common databases where we could promote studies crossovers. The forest plot (Fig. 7) statistically normalizes the data to calculate a common SMD. However, we consider that studies crossover to common databases would bring a plus significance to this systematic review.

Some studies provided outcomes in the format of low-resolution graphics, which turned the task of numerical reading more difficult. In these cases, authors should provide the supplementary material with the numerical data from where they built the outcome graphics or a high-resolution graphics version. We needed to read the data more closely to avoid bias and take the best approximated numerical values.

As we mentioned in the risk-of-bias Sect. 3.7, some studies brought innovative ML methods to address COVID-19 prediction, but presented no results or numerical and graphical outcomes of the prediction results. Although Sect. 10.12.2 of Cochrane Handbook for Systematic Reviews of Interventions (Higgins et al. 2019) allows replacement data fulfillment for cases of lack of numerical outcomes, we did not adopt this approach to avoid any biased introduction to this systematic review. Instead, we decided to rely on the graphics data regardless they were not accurate. To preserve the original level of certainty, we considered either no data or approximated data (due to low-resolution graphics in some studies).

Conclusion

This article presented a systematic review highlighting the contribution of applied computing to the health area. There is a significant effort to model, predict, and mitigate the COVID-19 pandemic across the population. We conducted the systematic review based on the PRISMA protocol, where we could conduct the full review of 45 studies out of initially 7031 screened studies. We could then answer the general and specific questions proposed in this systematic review based on the PICO approach.

Compared to other systematic reviews cited in this study ((Ghaderzadeh and Asadi 2021; Alyasseri et al. 2022; Syeda et al. 2021), our systematic review has the advantage of providing a straightforward scientific contribution to joining all ML pandemic evolution models into a single study and comparing which model can be the best fit depending upon the affected region criticality.

Using Cochrane methodology and GRADE assessment for a summary of findings table allowed us to dive deep into the studies and determine their significance to the scientific contribution that can act as a speedy response to minimize the health disasters caused by the COVID-19 pandemics.

We observed essential ML approaches to predict the next steps of the pandemics regarding COVID-19-positive trends and geolocation. By conducting risk of bias, levels of certainty, a summary of findings, and statistical analysis via the forest and funnel plot assessments for the 45 studies, we could determine the statistical significance of these studies to simulate the progress of the pandemics. Despite some study limitations found during this systematic review, our final results corroborate the possibility of using ML prediction to serve the healthcare authorities for decision-making and preventive actions toward saving lives. ML and healthcare can offer valuable options to respond to and combat current and future pandemics.

The healthcare authorities can take immediate advantage of using the included studies from this systematic review and implement any of the corresponding ML models in the format of ML pipeline graphs or Kubernetes2 container resources already present in the cloud hyperscalers (like Google Cloud Platform3, SAP Data Intelligence Cloud4, Microsoft Azure5 or Amazon Web Services6). The containers will provide endpoints where users can enter big data for training and testing. Then, the trained ML container seamlessly becomes a production microservice available for final users.

Registration and protocol

This systematic review has not been registered in the PROSPERO database yet.

The citation of this systematic review and meta-analysis protocol is presented in Moher et al. (2015). We explain the details about differences in resultant statistical significance through this systematic review, but they do not cause any differences in the original PRISMA-P protocol.