A Scoping Review of Digital Twins in the Context of the Covid-19 Pandemic.

Background: Digital Twins (DTs), virtual copies of physical entities, are a promising tool to help manage and predict outbreaks of Covid-19. By providing a detailed model of each patient, DTs can be used to determine what method of care will be most effective for that individual. The improvement in patient experience and care delivery will help to reduce demand on healthcare services and to improve hospital management.
Objectives: The aim of this study is to address 2 research questions: (1) How effective are DTs in predicting and managing infectious diseases such as Covid-19? and (2) What are the prospects and challenges associated with the use of DTs in healthcare?
Methods: The review was structured according to the Preferred Reporting Items for Systematic Reviews and Meta-Analyses Extension for Scoping Reviews (PRISMA-ScR) framework. Titles and abstracts of references in PubMed, IEEE Xplore, Scopus, ScienceDirect and Google Scholar were searched using selected keywords (relating to digital twins, healthcare and Covid-19). The papers were screened in accordance with the inclusion and exclusion criteria so that all papers published in English relating to the use of digital twins in healthcare were included. A narrative synthesis was used to analyse the included papers.
Results: Eighteen papers met the inclusion criteria and were included in the review. None of the included papers examined the use of DTs in the context of Covid-19, or infectious disease outbreaks in general. Academic research about the applications, opportunities and challenges of DT technology in healthcare in general was found to be in early stages. Conclusions: The review identifies a need for further research into the use of DTs in healthcare, particularly in the context of infectious disease outbreaks. Based on frameworks identified during the review, this paper presents a preliminary conceptual framework for the use of DTs for hospital management during the Covid-19 outbreak to address this research gap.

Introduction

Background

The Covid-19 pandemic highlights the need for the ability to predict, and therefore better manage, infectious disease outbreaks. One potential technology that could provide a means of addressing this challenge and improving hospital management and quality of care is a Digital Twin system. Digital Twins (DTs) are virtual representations of physical systems, objects, processes, or services visualising the represented entity.[2,3] They learn from continuously updating real-time data and dynamically adapt their insights and recommendations to support improved decision-making. DTs are key to the industry 4.0 transformation, hence, research has been focussed in optimising manufacturing and production.[2-5] For example, using DTs in the offshore renewable sector[6,7] can reduce operational costs and maximise energy output due to the harshness of the environment where the devices operate. However, the application of DT technology to other sectors is growing, with the use of DTs in ‘smart cities’ offering huge potential.[7,8] The potential for DTs to improve healthcare systems is also being increasingly explored,[9,10] for example in personalised wellbeing specific to patient-centric dietary requirements as they can be used to model anything from an organ to a patient to a hospital.[12,13]

DT technology is emerging as a key tool to shift towards a more patient-centric and personalised system of healthcare.[9,11] Research on DTs in health and care is gaining momentum in areas such as personalised care, managing chronic conditions, management of severe traumas by integrating DTs with agents and Multi-Agent Systems and improving the management of public health emergencies. DTs are also being used to improve hospital management and demand, with the aim of improving patient experience and care delivery. Another area where this technology could potentially have a significant positive health impact is in the prediction and management of infectious disease outbreaks.

The novel coronavirus has exposed the unpreparedness of nearly every country for a global pandemic and has (as of December 15, 2021) resulted in over 271 million cases and nearly 5.3 million deaths. Due to the global spread of the disease, and the ubiquity of mobile apps and digital technologies, there is a large body of data available about the Covid-19 pandemic. DTs can collect data from a number of sources (such as IoT sensors, mobile apps and healthcare databases) in real time and analyse it to provide clear visualisations and insights. These insights could help hospital administrators and clinicians to better manage the availability of key resources (eg, hospital beds, ventilators, PPE, etc.) and the care of individual patients.

Rationale

There has been limited research on reviewing the applicability of DT in health and care. A search of PROSPERO (using several variations of the keywords ‘covid-19’ and ‘digital twins’) found no reviews registered about the application of DTs to the management of the Covid-19 pandemic. There are also few literature reviews on DTs in healthcare. A systematic review characterising DTs was identified, but although it included healthcare-related DTs, they were not a focus. Additionally, a narrative review was found that provided an overview of DTs in pharmaceutical manufacturing. No systematic reviews evaluating the potential and barriers of applying DTs in healthcare in general were found. However, a small number of companies in the healthcare industry are conducting research on the viability of DTs to optimise patient care and hospital management. The Covid-19 pandemic has highlighted the need to further explore the potential role of DTs in predicting and managing infectious disease outbreaks. The ability to effectively track, trace and isolate positive infectious disease cases will minimise public harm and disruption to the economy. Therefore, there is a need for an overview of the literature concerning the use of DTs in infectious disease outbreaks and the opportunities and challenges associated with their use in healthcare more broadly.

Objectives

The primary objective of this study is to summarise the state of the field of the application of DTs in healthcare, with a focus on the use and effectiveness of DTs for managing infectious disease outbreaks (specifically the Covid-19 pandemic). To achieve this objective, the review will address 2 main research questions:

How effective are DTs in predicting and managing infectious diseases such as Covid-19?

What are the prospects and challenges associated with DTs in healthcare?

Methods

The review was structured using the PICO template and the PRISMA-ScR (Preferred Reporting Items for Systematic Reviews and Meta-Analyses Protocols Extension for Scoping Reviews) guidelines. Appropriate Medical Subject Headings (MeSH) were selected for the search. The scoping review consists of a literature search, article selection, data extraction, data analysis and a discussion of the implications of the data for the research questions posed.

Eligibility criteria

The research question is based on the Population, Intervention, Comparison and Outcome (PICO) framework shown in Table 1.

Table 1.

PICO (Population, Intervention, Comparison and Outcome) criteria and definitions.

PICO criteria	Definition
Population	Healthcare service (eg, hospital bed management) and patients encountering a DT in a health setting.
Intervention	DTs using data collected from all sources eg, sensors, devices and applications.
Comparison	No specific comparison is required.
Outcomes	● The potential of DTs in the management of Covid-19
	● Identification of the applications of DTs in health and care and description the technology (eg, DT model)

Search strategy

A literature review was conducted on 4 academic databases: PubMed, IEEE Xplore, Scopus and ScienceDirect. An initial search was conducted on the 6th and 7th of August and the final search was conducted on the 12th of September, 2020. These 4 databases were chosen to cover the research areas in engineering and healthcare; IEEE Xplore and ScienceDirect comprehensively cover the engineering field, PubMed focuses on healthcare, and Scopus contains a large number of references in a variety of fields, and was included to capture anything missed by the more discipline-specific databases. To identify any relevant papers that are not included in those databases, Google Scholar was also searched. The Google search engine was used to identify grey literature and non-academic publications relevant to the subject and any relevant papers that may have been missed from earlier database searches.

The search terms were grouped into 3 themes and included with the structure: digital technology (MeSH OR Keywords) AND Application (MeSH OR Keywords) AND Covid-19 (MeSH OR Keywords). Keywords were searched for in the titles and abstracts of articles. Table 2 shows the search concept and keywords that were searched for this review. An example sample search string for the IEEE Xplore database is included in Appendix B.

Table 2.

Search terms.

Category	MeSH	Keywords (in title or abstract)
Digital technology	Capacity management OR patient centric OR outbreak prediction	digital twin
Application	Health management	‘health’ OR ‘care’ OR ‘healthcare’ OR ‘hospital’ OR ‘patient’
Covid-19	Covid-19	‘2019 novel coronavirus’ OR ‘2019-n-CoV’ OR COVID-19 OR coronavirus OR Covid

We further conducted search in the Journal of Medical Ethics to capture the papers that discussed the social and ethical challenges of DT. Discussions from these papers are captured in Discussions section later. We used the search terms social OR ethical challenges of DT. The search returned 13 papers.

Inclusion criteria

All papers discussing the use of digital twins in a healthcare setting were eligible for inclusion in the review. There was no limit imposed based on the year of publication because the application of DT in healthcare is relatively new; in initial searches, the earliest relevant paper found was published in 2018.

All papers discussing the ethical and social challenges of DT were included.

Exclusion criteria

Studies that focussed on the application of DTs in non-health settings and that were not published in English were excluded as the search results returned only 3 papers that were not in the English language and the source was not secure. Papers and other grey literature concerning DTs that were published by companies were also excluded. These papers were excluded because the research was focussed on developing DT-based solutions for specific healthcare problems and was not peer reviewed in open access.

All papers that did not explicitly mention DT were excluded from the discussion.

Screening and article selection

Journal articles, conference papers, non-academic papers and practitioner case reports retrieved by the search were downloaded and duplicates were manually removed. After duplicates were removed, the papers were screened in 3 consecutive phases – title, abstract and full text – based on the pre-specified inclusion and exclusion criteria. Details of the screening and selection process were recorded in a PRISMA flow diagram to ensure reproducibility (Figure 1).

Figure 1.

Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA) flow diagram of the search strategy.

Data extraction

Data was extracted from the papers selected for inclusion in the review by one reviewer. An initial review of the literature suggested items to be extracted (Table 3), but other data identified during the review were included if relevant.

Table 3.

Data extraction.

Article information	Data extracted
General study information	Title of publication
	Year of publication
	Authors
Characteristics of digital technology	Digital twin model
	Intended user/service

Data analysis and synthesis

It was not feasible to conduct a meta-analysis or statistical analysis due to the variety of source types and reported outcomes. A narrative synthesis was conducted on the extracted data and is summarised in the discussion to provide an overview of the current literature on DTs in healthcare settings, to draw conclusions about their prospects and limitations, and to explore the potential of DTs in managing and predicting infectious disease outbreaks.

Results

Included studies

A total of 603 references were retrieved from the databases. After removing duplicates and screening abstracts and full texts, 18 papers were eligible for the study based on the inclusion and exclusion criteria and were reviewed. A record of the screening process was captured in a PRISMA flow diagram (Figure 1) and the PRISMA-Scr checklist is presented in Appendix A.

Study characteristics

There was a large variety of types of papers included in the review: some presented a review of DTs as applied to the healthcare sector, some papers proposed a framework (without testing it on live data) supporting cloud-based data storage. A further few papers proposed a DT model based on live patient data for specific disease management. Therefore, the papers were split into 3 groups as review papers (4/18),[23-26] papers that presented a framework or model (8/18),[9,12,15,27-31] and papers that focussed on a specific disease (eg, diabetes or a procedure such as conducting heart surgery, 6/18).[14,32-36] The characteristics of the studies and the extracted data are summarised in Table 4.

Table 4.

Components of the review of papers.

Publication type	Author(s)	Category	Year of publication	Publisher	Digital Twin model	Intended user/service
Journal	Mohapatra and Bose 23	Review	2020	Springer Link Health Technol	A framework for digital twin implementation in the healthcare industry	Service improvement aimed at general public
Journal	Corral-Acero et al 24	Review	2020	European Heart Journal	Application of DT to accelerate cardiovascular research and enable the vision of precision medicine	Clinician and patient
Chapter	Bagaria et al 25	Review	2020	Springer Nature	DT for personal health and well-being	General public
Journal	Bruynseels et al 26	Review	2018	Frontiers in Genetics	Digital twins privacy in healthcare technologies and ethics of biomedical data	General public and clinician
Journal	Liu et al 28	Framework/Model	2019	IEEE Access	Cloud-Based Framework for the Elderly Healthcare Services Using Digital Twin	Elderly population
Journal	Rodríguez-Aguilar and Marmolejo-Saucedo 15	Framework/Model	2020	EAI Endorsed Transactions on Pervasive Health and Technology	DT of the public emergency system	General public
Journal	Laamartet al 29	Framework/ Model	2020	IEEE Access	QoE-based DT framework for health and well-being in smart cities	General public
Conference	Rivera et al 27	Framework/Model	2019	ACM Digital Library	A DT reference model in healthcare for decision-making processes when applying medical treatments to patients by healthcare professionals.	General public
Conference	Lutze 30	Framework/Model	2019	IEEE	A DT ehealth system focussing on information management	General public
PhD thesis	Albraikan 31	Framework/Model	2019	University of Ottawa, Canada	A DT model for emotional well-being	General public/patient with non-clinical mental health issues
Journal	Björnsson et al 9	Framework/Model	2019	Genome Med	Personalised medicine framework	General public
Journal	Croatti et al 12	Framework/Model	2020	Med Syst.	Agent-based DTs to manage severe traumas	Trauma patient/Clinician
Journal	Chakshu et al 32	Disease improvement	2019	Int J Numer Method Biomed Eng.	DT to detect the severity of carotid stenosis from head vibration	Clinician/patient
Preprint article	Rao and Mane 33	Disease improvement	2019	Persistent Systems Ltd.	Customised per-patient DT model in liver disease diagnosis using Domain Knowledge and Machine Learning	Clinician/patient
Journal	Grosman et al 34	Disease management	2020	Diabetes	DT programme to personalise MiniMed™ 670G settings in Type 1 diabetes patients	Clinician/patient
Journal	Walsh et al 14	Disease improvement	2020	bioRxiv	DT for multiple sclerosis using probabilistic neural networks	Clinician/patient
Conference	Martinez-Velazquez et al 36	Disease improvement	2019	IEEE	DT for cardiac patients	Clinician/patient
Conference	Mazumder et al 35	Disease improvement	2019	IEEE	DT for Cardio-vascular patients	Clinician/patient

Discussion

Principal findings

No peer reviewed papers were found that specifically addressed the potential of DTs to address the challenge posed by infectious disease outbreaks. Therefore, the review was unable to directly answer the first research question: ‘How effective are DTs in predicting and managing infectious diseases such as Covid-19?’ Potential benefits of DTs for healthcare in general – including increasing patient centricity and improving quality of care – were identified. However, the included studies also described key challenges associated with DTs in healthcare, including: interoperability, data processing, patient confidentiality and data security.

Prospects and challenges of DTs in healthcare

Recent research has proposed a potential role for DT technology in the management of hospital capacity by using data and simulations to offer decision support to help improve the management of patients during public health emergencies. DTs can help in developing patient-centric and personalised care by building a simulation of an individual based on their specific health data and insights from population health data, in improving the management of chronic conditions,[11,14] and in guiding complex medical procedures. Moreover, DTs also have potential to help improve mental health and well-being by identifying trigger factors and suggesting coping techniques aimed at improving the quality of life holistically.

Recently, there has been emerging discussions on the social and ethical challenges of DT. It has been argued that for an ethically acceptable form of digital twin there are 5 conditions that should be met to avoid mis-representation or bias.[37,38] This argument is expanded further to include the potential impact of the system on an individual so as not to inadvertently widen the gap in most vulnerable people and those with less cognitive resources.

Braun’s paper has given rise to discussion by a number of researchers. This includes the viewpoint that the discussion on ethics is too early as the technology is not fully developed, as well as concerns around access to the individual’s medical data by the company that developed the DT and hence could have control over.[41-43] This argument has been expanded to add concerns around ownership, representation, agency, in addition to control. Lupton takes issue to the terminology ‘Digital Twin’ to be used in a medical care setting for a simulation model. The discussion on ethics is only just starting highlighting the challenges associated and will shape how the technology is embedded in medical care in the future.

Another key challenge for DT technology that was identified during the review is establishing interoperability and trust between different data sources contributing to the DT (eg, high-quality clinical/medical data integration with limited quality biometric and behavioural data from the smart home and wearables). Another key challenge will be to find a way of balancing the cost and benefits of collecting and processing large amounts of data, while ensuring that data security and patient confidentiality are not compromised.

A small number of companies identified in the Google search have started to research the application of DT in managing the coronavirus pandemic.[47-49] No peer-reviewed papers by these companies were identified, however, the conceptual case studies presented on these companies’ websites suggest that DTs have the potential to accelerate the vaccine production process and improve hospital capacity, staffing and care delivery models, and patient management (such as critical-care beds availability, ventilator availability, etc). Therefore, DTs appear to have the potential to improve health and wellbeing of patients with novel diseases such as Covid-19 by using patient-specific simulations to trial new treatments and improve hospital management during crises. An established DT system could also potentially use data relating to patients’ symptoms and behaviour, such as the emotion-aware digital twin based on sensor data being developed by one of the papers, to support the management of infectious disease outbreaks. This work can be extended to take account of external factors such as location and number of positive cases from a large sample of the population, collected to predict the next outbreak/hotspot for Covid-19. These simulations could facilitate the prediction of potential risks and provide mechanisms for optimal decision making with regards to hospital preparedness and resource management.

Conceptual DT framework for Covid-19 prediction and hospital management

Limitations were identified in some of the models posited by the papers included in the review. For instance, one framework identified in the background research – which focussed on predicting the maintenance of complex equipment, and was not healthcare-specific – limited the development of the DT framework it proposed to five dimensions. This is a limitation for a DT framework in hospital management because it does not sufficiently cover the potential variables or indicators needed to quantify successful management. One of the healthcare-specific DT frameworks included in this review introduced the concept of feedback based on the user’s quality of experience (QoE), which will be an important aspect for a DT framework for hospital management. However, the framework used a very broad definition of health and well-being; data collected from wearable sensors and integrated with other healthcare data was used to suggest activity levels for the patient. The framework was simplistic and did not take into account the interaction of different variables, which would be a necessary component for a framework of hospital management. Although the literature on DTs is limited, both of the frameworks described were found to be useful for their specific applications of wind turbine lifecycle monitoring and increasing physical activity in participants.[3,29] However, they are not adequate frameworks for this particular context.

In the context of Covid-19, this paper proposes a framework for hospital management that ensures that all of the relevant dimensions can be added in the DT model and that there is a feedback loop in real-time, or near real-time, to ensure that the QoE is maintained at an acceptable level. Based on the frameworks described,[3,29] this paper proposes a patient-centric conceptual framework for Covid-19 hospital management based on the healthcare provider’s (the user’s) quality of experience (QoE). The five-dimensional architecture is extended to n-dimensions as shown in equation (1)

where PE are the physical entities, VE are the virtual equivalents, Ss are the services for PE and VE, DD is the DT data and CN are the connections. Additional dimensions (XX1. . .XXn) can be added based on a specific application (eg, number of available beds, number of infected patients, etc). The conceptual DT framework is presented in Figure 2.

Figure 2.

Conceptual patient-centric hospital management DT framework for Covid-19.

In the proposed conceptual framework (Figure 2), the PE is the hospital capacity and VE is the digital replica of PE. DD is collected from a number of sources such as wearable devices, medication and other patient-centric data. Ss and CN information is collected from the network, both fixed and wireless access. The DT framework is predicted in terms of the user’s QoE. Further, we extend the framework to include prediction of the patient’s risk factors of getting Covid-19 based on patient-centric data collected from DD and/or specific questionnaires developed as an app to collect risk factors such as occupation, housing, etc. There is a feedback loop to ensure that the user’s QoE is maintained at an acceptable level.

Limitations

The main limitation of this review is that, given the lack of relevant studies identified in the search, the first research question could not be adequately answered. Although an effort was made to apply the knowledge gained from the included studies to propose a conceptual framework for the use of DTs in hospital management, no conclusions could be drawn about its validity or potential usefulness. Another limitation of this review is that the number of studies identified as eligible for inclusion in the study was relatively small; while this likely reflects the early state of the body of literature regarding DTs in healthcare, it is also possible that relevant articles were missed due to the inclusion of Covid-19 specific keywords in the search string. Therefore, although it was not intended to be the primary focus of this review, the overview of DTs in healthcare in general is likely to be incomplete.

Future directions

As none of the included papers examined the use or effectiveness of DTs in relation to hospital management during the Covid-19 pandemic (or any infectious disease outbreaks), this is clearly a key area for future research. While there are a small number of companies in the healthcare sector that are investigating the potential of DT to provide solutions for the problem of hospital patient capacity and management, there appears to be a lack of academic research developing or evaluating these solutions.

With increased focus on patient-centric care, DTs offer the potential to better predict and manage infectious diseases such as Covid-19. Some patients who have recovered from Covid-19 suffer from long Covid, others have developed severe cardiovascular conditions or other serious diseases.[50,51] DTs could aide clinicians in predicting, based on risk factors, if a patient is likely to develop a serious health condition, or the probability that they will suffer from long Covid. This could help the healthcare sector to better manage the resources and provide support and monitoring to those who are at highest risk. There is an urgency to better understand these risk factors post-Covid and propose patient-centric solutions that will improve patients’ quality of life by giving them the appropriate supportive therapy.

Conclusions

The purpose of this scoping review was to examine the prospects and challenges associated with using Digital Twins in healthcare, with a particular focus on their potential to help predict and manage infectious disease outbreaks such as Covid-19. No studies were identified that used DTs for Covid-19 in particular (or infectious disease outbreaks in general), so the potential of DTs for hospital management during an outbreak was explored. This review extended the literature by providing an initial overview of the prospects and challenges of applying DT technology in the healthcare sector. There is a significant lack of research in the examination of DTs’ potential for improving hospital management, particularly in the context of infectious disease outbreaks, and research on the use and effectiveness of DTs in this context is urgently required.

Section	Item	PRISMA-ScR Checklist item	Reported on page #
Title
Title	1	Identify the report as a scoping review.	1
Abstract
Structured summary	2	Provide a structured summary that includes (as applicable): background, objectives, eligibility criteria, sources of evidence, charting methods, results, and conclusions that relate to the review questions and objectives.	1
Introduction
Rationale	3	Describe the rationale for the review in the context of what is already known. Explain why the review questions/objectives lend themselves to a scoping review approach.	2-3
Objectives	4	Provide an explicit statement of the questions and objectives being addressed with reference to their key elements (e.g., population or participants, concepts, and context) or other relevant key elements used to conceptualize the review questions and/or objectives.	3
Methods
Protocol and registration	5	Indicate whether a review protocol exists; state if and where it can be accessed (e.g., a Web address); and if available, provide registration information, including the registration number.	N/A
Eligibility criteria	6	Specify characteristics of the sources of evidence used as eligibility criteria (e.g., years considered, language, and publication status), and provide a rationale.	3-5
Information sources*	7	Describe all information sources in the search (e.g., databases with dates of coverage and contact with authors to identify additional sources), as well as the date the most recent search was executed.	4
Search	8	Present the full electronic search strategy for at least 1 database, including any limits used, such that it could be repeated.	15 (Appendix B)
Selection of sources of evidence †	9	State the process for selecting sources of evidence (i.e., screening and eligibility) included in the scoping review.	4-5
Data charting process ‡	10	Describe the methods of charting data from the included sources of evidence (e.g., calibrated forms or forms that have been tested by the team before their use, and whether data charting was done independently or in duplicate) and any processes for obtaining and confirming data from investigators.	5
Data items	11	List and define all variables for which data were sought and any assumptions and simplifications made.	5 (Table 3)
Critical appraisal of individual sources of evidence §	12	If done, provide a rationale for conducting a critical appraisal of included sources of evidence; describe the methods used and how this information was used in any data synthesis (if appropriate).	N/A
Synthesis of results	13	Describe the methods of handling and summarizing the data that were charted.	5
Results
Selection of sources of evidence	14	Give numbers of sources of evidence screened, assessed for eligibility, and included in the review, with reasons for exclusions at each stage, ideally using a flow diagram.	5-6
Characteristics of sources of evidence	15	For each source of evidence, present characteristics for which data were charted and provide the citations.	6-8
Critical appraisal within sources of evidence	16	If done, present data on critical appraisal of included sources of evidence (see item 12).	N/A
Results of individual sources of evidence	17	For each included source of evidence, present the relevant data that were charted that relate to the review questions and objectives.	6-8 (included in same table as characteristics)
Synthesis of results	18	Summarize and/or present the charting results as they relate to the review questions and objectives.	6
Discussion
Summary of evidence	19	Summarize the main results (including an overview of concepts, themes, and types of evidence available), link to the review questions and objectives, and consider the relevance to key groups.	8-9
Limitations	20	Discuss the limitations of the scoping review process.	10-11
Conclusions	21	Provide a general interpretation of the results with respect to the review questions and objectives, as well as potential implications and/or next steps.	11
Funding
Funding	22	Describe sources of funding for the included sources of evidence, as well as sources of funding for the scoping review. Describe the role of the funders of the scoping review.	11

Abbreviations: JBI, Joanna Briggs Institute; PRISMA-ScR, Preferred Reporting Items for Systematic reviews and Meta-Analyses extension for Scoping Reviews.

Where sources of evidence (see second footnote) are compiled from, such as bibliographic databases, social media platforms and Web sites.

A more inclusive/heterogeneous term used to account for the different types of evidence or data sources (eg, quantitative and/or qualitative research, expert opinion and policy documents) that may be eligible in a scoping review as opposed to only studies. This is not to be confused with information sources (see first footnote).

The frameworks by Arksey and O’Malley and Levac et al and the JBI guidance (4,5) refer to the process of data extraction in a scoping review as data charting.

The process of systematically examining research evidence to assess its validity, results, and relevance before using it to inform a decision. This term is used for items 12 and 19 instead of ‘risk of bias’ (which is more applicable to systematic reviews of interventions) to include and acknowledge the various sources of evidence that may be used in a scoping review (eg, quantitative and/or qualitative research, expert opinion, and policy document).

Source: Tricco et al.

Database	Search terms
IEEE Explore	(((‘Mesh_Terms’:Capacity management OR patient centric OR outbreak prediction) OR ‘Abstract’:digital twin) AND ((‘Mesh_Terms’:Health management) OR ‘Abstract’:”health” OR ‘care’ OR ‘healthcare’ OR ‘hospital’ OR ‘patient’) AND ((‘Mesh_Terms’:Covid-19) AND ‘Abstract’:”‘2019 novel coronavirus’ OR ‘2019-n-CoV’ OR COVID-19 OR coronavirus OR Covid))