The Clinical Information Systems Response to the COVID-19 Pandemic.

OBJECTIVE: The year 2020 was predominated by the coronavirus disease 2019 (COVID-19) pandemic. The objective of this article is to review the areas in which clinical information systems (CIS) can be and have been utilized to support and enhance the response of healthcare systems to pandemics, focusing on COVID-19.
METHODS: PubMed/MEDLINE, Google Scholar, the tables of contents of major informatics journals, and the bibliographies of articles were searched for studies pertaining to CIS, pandemics, and COVID-19 through October 2020. The most informative and detailed studies were highlighted, while many others were referenced.
RESULTS: CIS were heavily relied upon by health systems and governmental agencies worldwide in response to COVID-19. Technology-based screening tools were developed to assist rapid case identification and appropriate triaging. Clinical care was supported by utilizing the electronic health record (EHR) to onboard frontline providers to new protocols, offer clinical decision support, and improve systems for diagnostic testing. Telehealth became the most rapidly adopted medical trend in recent history and an essential strategy for allowing safe and effective access to medical care. Artificial intelligence and machine learning algorithms were developed to enhance screening, diagnostic imaging, and predictive analytics - though evidence of improved outcomes remains limited. Geographic information systems and big data enabled real-time dashboards vital for epidemic monitoring, hospital preparedness strategies, and health policy decision making. Digital contact tracing systems were implemented to assist a labor-intensive task with the aim of curbing transmission. Large scale data sharing, effective health information exchange, and interoperability of EHRs remain challenges for the informatics community with immense clinical and academic potential. CIS must be used in combination with engaged stakeholders and operational change management in order to meaningfully improve patient outcomes.
CONCLUSION: Managing a pandemic requires widespread, timely, and effective distribution of reliable information. In the past year, CIS and informaticists made prominent and influential contributions in the global response to the COVID-19 pandemic. IMIA and Thieme. This is an open access article published by Thieme under the terms of the Creative Commons Attribution-NonDerivative-NonCommercial License, permitting copying and reproduction so long as the original work is given appropriate credit. Contents may not be used for commercial purposes, or adapted, remixed, transformed or built upon. (https://creativecommons.org/licenses/by-nc-nd/4.0/).

1 Introduction

The magnitude of the coronavirus disease 2019 (COVID-19) pandemic has permeated every area of healthcare, academia, politics, and industry such that this survey must begin with a brief history. Initially identified in December 2019 following an outbreak of atypical pneumonia in Wuhan, China, COVID-19 was declared a global pandemic by the World Health Organization (WHO) on March 11 th , 2020 [ 1 , 2 ]. Through November 2020, more than 62 million cases of COVID-19 have been reported in 191 countries leading to more than 1,400,000 deaths [ 3 ]. The ensuing toll on the world's population is impossible to quantify. The global response has included exhaustive case identification efforts through screening and diagnostics, governmental responses and regulations to limit transmission, comprehensive preparations for expanding hospital capacity, and an unprecedented search for novel therapeutics and effective vaccination [ 4 5 6 ] The scientific community responded with a historic influx of research, with now over 280,000 scholarly articles pertaining to COVID-19 [ 7 ]. We are proud to report that clinical informaticists and the field of biomedical informatics are playing a prominent role in addressing COVID-19 [ 8 ]. Pandemic management occurred at multiple levels via international, national, regional, and local public and private institutions [ 9 10 11 ]. Though varied in detail, standard approaches emerged, and a universal theme arose – the need for timely and reliable information to enable critical decision making [ 12 , 13 ]. The novelty of the crisis and speed of change required continuous communication and distribution of data and knowledge to allow development, dissemination, and adoption of evidence-based practices [ 13 14 15 ]. Digital technologies have been employed globally to support and enable public-health responses to COVID-19 at an unprecedented scale [ 16 ].

Recent surveys in the IMIA Yearbook of Medical Informatics define Clinical Information Systems (CIS) as a “set of resources, techniques, devices, and methodologies used to support the needs of healthcare organizations” whose crucial role is “to capture, store, process, and transfer information to clinical decision makers” [ 17 , 18 ]. By integrating vital information in the daily practices of healthcare administrators and front-line providers, CIS can serve as a public health tool during acute crises [ 19 ]. The subsequent collection and storage of patient data and provider actions creates an invaluable source of data with myriad applications. Given the rapid rise of telehealth as a safe and effective mode of healthcare delivery in the era of physical distancing, it was an influential year for CIS [ 16 , 20 , 21 ]. An electronic health record (EHR) system is now utilized by > 95% of hospitals in the United States of America (USA) and > 80% of office-based practices in both the USA and the European Union [ 22 , 23 ]. Globally, 47% of countries have a national EHR and 70% of WHO member countries have an eHealth policy [ 24 ]. The prevalence of EHRs necessitates involvement of technology to facilitate both small- or large-scale clinical management strategies that reduce administrative burdens and enhance patient care [ 19 ]. The aim of this survey is to summarize pertinent articles specific to COVID-19 and the areas in which CIS have been applied during the pandemic to improve patient care. Given the volume and rapidly evolving nature of the literature, we do not attempt to be comprehensive. Rather, we highlight some interesting areas of research and operations to discuss opportunities and future directions.

2 Methods

We conducted a review of the literature limited from the year 2019 through the end of October 2020 utilizing PubMed/MEDLINE and Google Scholar databases by combining the search terms “clinical/health information systems”, “health/medical/clinical informatics”, “health information technology”, “digital health”, “electronic health/medical record”, “and health information exchange” with “coronavirus”, “COVID-19”, “severe acute respiratory syndrome 2 (SARS-CoV-2)”, “pandemic”, “pandemic/outbreak management”. We then searched the table of contents of major health informatics journals such as the Journal of the American Medical Informatics Association (JAMIA), the International Journal of Medical Informatics (IJMI), and Applied Clinical Informatics (ACI) for relevant original reports. Finally, we reviewed the reference section within each selected article to identify additional potentially relevant articles. The articles selected for detailed review are listed in Table 1 .

Table 1

Selected articles for detailed review.

Survey Section	Reference	Author(s)	Country	Journal	Key Contribution(s)
Antecedent Infectious Diseases, Health Information Systems	28	Zhao et al.	China	Telemed e-Health	Description of Chinese construct of regional and national informatics systems in response to SARS
	29	Chen et al.	Taiwan	Information Systems Research	Identifies framework for detecting emerging infectious disease with central and local coupling and decoupling circles
	30	Mandl et al.	USA	JAMIA	Syndromic surveillance systems can be used for detection of infectious diseases
	33	Advani et al.	USA	Studies in health technology and informatics	Created interactive dashboard for local municipalities across the US.
	34	Keck et al.	USA	JAMIA	Developed EHR-based influenza surveillance system for American Indian and Alaska Native populations
	37	Landman et al.	USA	Disaster Medicine and Public Health Preparedness	Used CDS to ensure screening for Ebola
	43	Borycki et al.	Canada, Australia, USA, Finland, Japan	Yearbook of Medical Informatics	Review of literature regarding health information technology used for patient-centered care during Ebola outbreak.
	44	Mandl	USA	JAMA	Viewpoint on EHR as public health tool for outbreaks of infectious disease
Clinical Information Systems within Health Systems	19	Reeves et al.	USA	JAMIA	Rapid implementation of EHR-based tools used to support clinical patient care during COVID-19 pandemic
	48	Grange et al.	USA	ACI	Rapid rollout of Information Technology Services to support clinical response to COVID-19.
	49	Lin et al.	USA	JAMIA	Informatics tools used for rapid onboarding of physician/staff and enhanced communication with families
	50	Milenkovic et al.	Serbia	IJMI	Created AI-driven patient triage and scheduling modules
	51	Yan et al.	China	JAMIA	Analyzed hospital webpages in China to determine themes of Health IT use in mainland China
	52	Ye et al.	China	JMIR	Provide broad framework of how information technology is used in mainland China
	53	Sylvestre et al.	France	JAMIA	Responding to pandemic without an EHR
	54	Kannampallil et al.	USA	JAMIA	Describe transition of informaticist from academic to operational leader and share lessons learned
Telehealth	55	Hong et al.	USA	JMIR Public Health and Surveillance	Correlation between COVID-19 cases and telehealth internet search volume; Survey of telehealth adoption and tele-ICU capabilities in US hospitals prior to COVID-19
	57	Bhaskar et al.	Australia, Canada, Kazakhstan, Trinidad and Tobago, USA, United Kingdom, Ireland, Israel, Philippines, India, and Poland	Frontiers in Public Health	Telemedicine status pre and post COVID-19 outbreak in countries around the world with recommendations for further development
	63	Mehrotra et al.	USA	NEJM Catalyst	Early account of rapid telehealth roll-out in US with heavy reliance on telephone encounters
	65	Mann et al.	USA	JAMIA	Description of rapid rollout of video visits in urgent care and ambulatory clinic settings at the onset of the pandemic in New York
	70	Monaghesh & Hajizadeh	Iran	BMC Public Health	Systematic review of telehealth utilization during COVID-19 outbreak from Dec 2019 to April 2020
	73	Wosik et al.	USA	JAMIA	Description of phases of telehealth transformation during COVID-19 in ambulatory and inpatient settings
	75	Vilendrer et al.	USA	JAMIA	Inpatient utilization of telemedicine across three different health systems in California in response to the pandemic
	74	Hron et al.	USA	ACI	Rapid implementation of an inpatient telehealth program in an academic children's hospital with significant utilization volumes
	76	Ong et al.	USA	ACI	Description of implementation of inpatient telehealth program in an academic medical center in response to COVID-19 with delineation of 13 use cases and 8 device options
	77	Jones et al.	USA	Diabetes Technology & Therapeutics	Demonstration of unchanged glycemic measures as outcome metric in pre-/post- intervention comparison of virtual diabetes program during COVID-19
	78	Wijesooriya et al.	Netherlands, USA	Ped Resp Reviews	Applications of telehealth in medical education and clinical research
	91	Nouri et al.	USA	NEJM Catalyst	Demonstration of statistically significant exacerbations of healthcare access inequity during COVID-19 pandemic
Case Identification, Remote Monitoring and Screening	14	Wang et al.	Taiwan	JAMA	Description of Taiwanese national response to pandemic utilizing technology for rapid case identification, contact tracing, and surveillance
	104	Judson et al.	USA	JAMIA	Created a self-triage and scheduling tool
	105	Judson et al.	USA	JAMIA	Digital chatbot used for daily screen of healthcare employees
	106	Annis et al.	USA	JAMIA	Remote patient monitoring system used to manage COVID-19 symptoms at home
	103	Ford et al.	USA	JAMIA	Used biomedical informatics tools for remote monitoring, biosensors, and dashboard
	107	Perlman et al.	USA, Israel, Great Britain	JMIR	AI drive self-assessment, symptom checker
	108	Perez-Alba et al.	Mexico	JAMIA	Onsite electronic self-administered triage tool
	109	Turer et al.	USA	JAMIA	Electronic personal protective equipment through use of telehealth
Diagnostic Testing	115	Weemaes et al.	Belgium	JAMIA	Laboratory information systems can help alleviate bottlenecks in COVID-19 diagnostic testing
Artificial Intelligence in Diagnostics and Predictive Analytics	123	Debnath et al.	USA	Bioelectronic Medicine	AI systems can augment clinical decisions in diagnosis and screening, risk stratification, prognosis, and allocation of resources
	111	Obeid et al.	USA	JAMIA	Natural language processing in a virtual care screening tool
	124	Zhang et al.	China	Cell	AI utilizing CT images can aid in diagnosis/prognosis
	125	Li et al.	China	Radiology	AI utilizing CT images can aid in diagnosis
	126	Murphy et al.	Netherlands	Radiology	AI for diagnostics in chest XR images
	127	Hurt et al.	USA	Journal of Thoracic Imaging	AI for diagnostic imaging implemented as cloud-based tool
	128	Carlile et al.	USA	JACEP Open	86% of ED physicians found an AI tool was easy to use in workflow and 20% reported algorithm impacted clinical decision making
	129	Goodman-Meza et al.	USA	Plos One	AI utilizing ancillary lab values can aid in diagnosis/screening
	116	Mei et al.	USA and China	Nature Medicine	AI as useful triage tool while definitive PCR tests result
	112	Liu et al.	China	JMIR	AI tool for CDS among general practitioners
	130	Liang et al.	China	Nature Com	Deep-learning neural network to predict critical illness of COVID-19 patients
	131	Jacob et al.	United Kingdom	European Respiratory Journal	National COVID-19 Chest imaging Database in the United Kingdom
	134	Wynants et al.	Europe	British Medical Journal	Systematic review found no AI models recommended for use in clinical practice
	135	Chen and See	Singapore	JMIR	Systematic review highlighting shortcomings without definitive conclusion
	136	Bakker et al.	Netherlands	JAMIA	Systematic review of health and economic impact of big data analytics found benefit no definitive benefit in current literature
Data Sharing and Interoperability	137	Plasek et al.	USA	JAMIA	Review of cross-border data sharing
	138	Zeng et al.	USA	Data Information Management	Overview of knowledge organization systems
	151	Dong et al.	USA	JAMIA	Normalization system for mapping heterogenous COVID-19 nomenclature to LOINC codes
	153	Garcia et al.	USA	JAMIA	Overview of CDC COVID-19 Information Management Repository
	155	Bruthans	Czech Republic	IJMI	Cross-border interoperability of electronic prescribing systems is limited
	156	O’Reilly-Shah	USA	Anesthesia & Analgesia	Commentary of shortcoming of data-sharing in the US
	157	Ienca and Vayena	Switzerland	Nature Medicine	Commentary of ethical and privacy concerns regarding broad data-sharing
	158	Lenert and McSwain	USA	JAMIA	Commentary of negative impact of data-sharing regulations
	159	Kandel et al.	Switzerland	The Lancet	76% of countries have robust COVID-19 detection capacity
	160	Holgrem et al.	USA	JAMIA	40% of US public health departments lack ability to electronically receive COVID-10 case reports
	161	Stenner et al.	USA	ACI	Commentary highlighting that more information is not always beneficial
	162	Foraker et al.	USA	Learning Health Systems	Framework to address infrastructure, duplication of data requests, and insufficient coordination by responsible entities
	163	Sim et al.	USA	Science	Commentary advising US NIH to lead efforts at mandating data-sharing in clinical trials
Epidemiologic Reporting	164	Dong et al.	USA	The Lancet Infectious Diseases	Developed first interactive web-based visual dashboard to track global cases of COVID-19
	168	Kamel Boulos et al.	China and USA	International Journal of Health Geographics	Commentary on geographical information systems utility during the pandemic
	169	Wissel et al.	USA	JAMIA	Developed interactive dashboard for local municipalities within the USA
	172	Thorlund et al.	Canada	The Lancet Digital Health	Created COVID-19 clinical trials registry visual dashboard
	173	Dixit et al.	USA	JAMIA	Outline important lessons regarding rapid implementation of local COVID-19 dashboards
Contact Tracing and Exposure Notification Systems	177	Ferretti et al.	United Kingdom	Science	Demonstrated ability to utilize algorithmic modeling in mobile applications to perform digital contact tracing
	183	Huang et al.	Singapore	JMIR mHealth Uhealth	Found wearable RTLS had significantly higher sensitivity for detecting patient contacts compared to Bluetooth contact tracing app
	184	Braithewaite et al.	United Kingdom	The Lancet	Systematic review of digital contact tracing found no objective evidence of effectiveness
	185	Anglemeyer et al.	Australia, New Zealand, United Kingdom	Cochrane Database Systematic Reviews	Cochrane review of digital contact tracing concluded solution is largely unproven
	188	Nayarn et al.	Singapore	JAMIA	Contact tracing augmented with technology reduced manual labor by 60% within hospital setting

Abbreviations: SARS = Severe Acute Respiratory Distress Syndrome; USA = United States of America; US = United States; JAMIA = Journal of the American Medical Informatics Association; EHR = electronic health record; CDS = clinical decision support; JAMA = Journal of the American Medical Association; COVID-19 = Coronavirus Disease 2019; AI = Artificial Intelligence; JMIR = Journal of Medical Internet Research; IT = Information Technology; ICU = intensive care unit; NEJM = New England Journal of Medicine; CT = computed tomography; XR = X-ray; JACEP = Journal of the American College of Emergency Physicians; ED = Emergency Department; PCR = polymerase chain reaction; LOINC = Logical Observation Identifiers Names and Codes; CDC = Centers for Disease Control and Prevention; NIH = National Institutes of Health; RTLS = real-time locating system.

3 Clinical Information Systems and Antecedent Infectious Disease Outbreaks

Prior to the emergence of the severe acute respiratory syndrome coronavirus 2 (SARS-CoV-2), several other epidemics and pandemics have occurred during the digital era, representing opportunities to use CIS in outbreak management. In 2003, following the outbreak of the first severe acute respiratory syndrome (SARS) in China, a web-based integrated database (SARSID) was used to collect hospital-based clinical data from SARS patient wards [ 25 ]. Similarly, global information systems such as the WHO's Global Outbreak Alert and Response Network were used to facilitate rapid sharing of new information pertaining to SARS [ 26 , 27 ]. Zhao et al. in 2010 described an attempt by the Chinese government to construct regional and national health systems utilizing information and computer technology to prepare for any subsequent outbreaks of emerging infectious diseases (EID) like SARS [ 28 ]. Public health authorities in the case of SARS recognized the ability of electronic systems to integrate health resources, collect patient data, and share medical information across regions in order to benefit public health [ 28 ]. Chen et al. published an elegant study in 2011 discussing the various information systems-related public health initiatives that were undertaken as a result of SARS [ 29 ]. These included national electronic disease surveillance systems, health information exchanges, and electronic health record alerting; all of which were intended to assist the management of EID [ 29 ]. In addition, the authors analyzed news coverage and contact tracing data from SARS patients in Taiwan to develop an alternative framework for outbreak management. They argued that most public health strategies typically emphasized strengthening central control and management. However, EID are often clinically ambiguous and effective control requires local ability to detect unusual cases. Public health information systems should therefore promote and facilitate local discussion, investigation, and recognition of important outliers and timely dissemination of information to improve detection of novel diseases. Such systems can be supported by well-designed CIS. Mandl et al. in 2004 described the opportunity to develop syndromic surveillance systems to detect discernable clinical case features of a given disease before confirming the diagnosis [ 30 ]. Initially intentioned for detection of a bioterrorist attack, the authors point out that such a system would also be useful for public health and detection of infectious diseases. Six years after SARS, the USA encountered the highly contagious strain of the influenza A virus, subtype H1N1 [ 31 ]. In June 2009 a pandemic was declared, and between 150,000 and 575,000 people died from the virus worldwide [ 31 ]. In response, a similar call was made for the design of a multinational informatics infrastructure with standardized data and indicators for information collection and disease surveillance [ 32 ]. In 2011, a group from Columbia University configured a widget-based system to provide clinical decision support (CDS) with automatic retrieval, alerts, and advice for primary care providers seeing patients with suspected H1N1 infection [ 33 ]. In 2014, Keck et al. published an interesting study in the JAMIA about using the EHR for influenza surveillance in the American Indian and Alaska Native (AI/AN) populations [ 34 ]. In the USA, the Indian Health Services (IHS) utilizes a nationalized EHR-based health information technology platform and provides healthcare to 2.56 million AI/AN citizens in multiple geographic locations across 37 states, including remote villages [ 35 ]. Within four weeks of the first recognized 2009 H1N1 cases in the USA, an EHR-based surveillance system, the IHS Influenza Awareness System, was created and implemented using algorithms based on International Classification of Diseases, Version 9 (ICD-9) codes, Current Procedural Terminology (CPT) codes, and other routinely collected clinical data. The software searched 162 databases across 343 health facilities and detected influenza-like illness visits with a sensitivity of 96.4% and a specificity of 97.8%. This system facilitated the timely and accurate detection of influenza-like illnesses and illustrated the potential capabilities of EHR-based public health surveillance. In March of 2014, an outbreak of Ebola Virus Disease emerged in West Africa and eventually led to more than 28,600 cases and 11,325 deaths over the course of two years [ 36 ]. Following this more recent epidemic, several authors highlighted the opportunity to utilize CIS to support local and public health management. Landman et al. described the use of CDS in their local hospital to require Ebola screening upon presentation to the emergency department [ 37 ]. Oza et al. built an interoperable EHR that included a tablet-based application designed to address the challenge of data collection in a highly infectious environment [ 38 ]. Mobile phone data was used to track the patterns of human mobility across West Africa, estimate the spread of disease, and was featured in a feasibility study of an electronic system for contact tracing [ 39 , 40 ]. The US Department of Defense created an Electronic Surveillance System for the Early Notification of Community-based Epidemics (ESSENCE) which screens the “reason for visit” in the EHR [ 41 ]. ESSENCE monitors the Military Health Systems EHR, provides alerts for potential EID, and allows epidemiologists and public health officers to investigate reportable disease events. The US Department of Commerce published a report on EHR design considerations in responding to highly infectious diseases, offering 18 measures to augment existing EHR functionality to increase public safety [ 42 ]. In the 2015 IMIA Yearbook, Borycki et al. provided a review of the literature describing how technology and EHRs can be used to support safe and patient-centered care regarding Ebola [ 43 ]. They outline the ability to use the internet, mobile applications and social media, EHRs, and CDS to support patient-centric care in times of emerging diseases and epidemics. They also detail the EHR design, human factors issues, policy and organizational issues, reporting, and quality improvement efforts utilizing health information technology (IT). Finally, Mandl wrote a viewpoint in the JAMIA about the potential to use the EHR as a public health tool across the USA in response to Ebola [ 44 ]. He highlighted many of the points described above and offered simple interventions to mitigate the socio-administrative and regulatory barriers that slow progress in health IT. Overall, past experience with EID demonstrated the potential to develop CIS that can surveil populations, rapidly identify new cases, offer immediate screening and CDS in order to limit transmission, gather large amounts of data, and enable broad communication. Unfortunately, huge populations, geographical variance, and disparity of health resources create significant barriers to widespread implementation of such systems. The global health informatics infrastructure remains highly fragmented. As of 2015, more than a decade following the outbreak of SARS, only 57.5% of hospitals in China participated in the regional medical consortium designed to enhance health information exchange (HIE) [ 45 ]. Even within countries with highly digitalized healthcare such as the USA, the infrastructure needed for effective HIE varies dramatically [ 46 ]. The full realization of CIS to mitigate the spread of infectious diseases was therefore not present when COVID-19 first emerged in Wuhan.

4 Clinical Informatics to Support the COVID-19 Response within Health Systems

Reeves et al. in JAMIA were the first group to publish a description of healthcare informatics supporting a rapid COVID-19 response at a health system [ 19 ]. Clinicians from the University of California, San Diego (USA) treated some of the earliest cases in the USA, including patients evacuated from Wuhan, China on February 7, 2020 [ 47 ]. Anticipating further infections, Reeves et al. detail the rapid configuration of EHR-based tools to support outbreak management. First, they report multi-modal electronic screening and triage processes to limit exposures to on-site personnel and patients. Second, standardized EHR-based note templates were created and used across multiple settings to help front-line care providers support hospital protocols by offering up-to-date information at the point of care. Third, CIS helped with appropriate testing, isolation, and ancillary orders. Fourth, an existing telehealth infrastructure was rapidly expanded, and providers were trained to transition away from in-person care when appropriate. Finally, EHR-based real time reports and an operational COVID-19 dashboard were created for use by the local incident command center in order to guide institutional decisions. A second early report written by Grange et al. in ACI detailed the rollout of IT services support for the clinical response at the University of Washington, one of the first endemic hotspots in the USA [ 48 ]. To enable rapid diagnostic testing, an in-house laboratory test was validated, unambiguously named, and embedded in a standardized set of orders that was made available to physicians and triage nurses. Their laboratory information systems were configured to interface with the EHR and external systems for public health reporting. CDS was used to provide direct access to informational hyperlinks, ICD, 10 th Revision (ICD-10) diagnoses, billing codes, and discharge instructions. Finally, telehealth was utilized in the intensive care setting to allow practitioners to perform bedside visits while conserving personal protective equipment.

Lin et al. in JAMIA described an interesting use of informatics at the University of Colorado (USA) to assist with the rapid onboarding of staff and physicians [ 49 ]. There was a need to educate providers with no recent experience in hospital medicine who were recruited to help manage a potential surge. A comprehensive training guide with a series of videos and resources was not only posted online, but it was also embedded into the EHR such that it could be accessed any time at the point of care. They also described efforts to improve communication with hospitalized patients’ family members not physically present at the bedside through use of tablets. Finally, nurses unable to work on the wards utilized secure messaging within the EHR to communicate with bedside nurses, physical therapists, chaplains, and social workers. This re-purposed workforce gathered information about hospitalized COVID-19 patients and relayed updates to family members. Milenkovic et al. in IJMI describe the adaptation of CIS at the University of Nis and the largest healthcare institution in the Republic of Serbia [ 50 ]. First, they created a patient triaging module utilizing deep neural networking based on questionnaires. They also developed an algorithmic scheduling model to group patients into high-risk categories that automatically scheduled into appropriate time slots. Prescriptions of chronic medications were extended to decrease the need for in-person visits. In addition, new reporting modules were created, and short message service (SMS), laboratory, and mail notification systems were expanded. Overall healthcare volume declined as the authors report effective social distancing was enabled through use of these various CIS modules. Yan et al. analyzed 368 webpages across 50 hospitals in mainland China and identified the following five focal themes of how IT was used to respond to the pandemic [ 51 ]: (1) popular medical science education; (2) digitalized hospital processes; (3) knowledge management for medical professionals; (4) telemedicine; and (5) new IT initiatives. Ye et al. in the Journal of Medical Internet Research (JMIR) Medical Informatics describe the framework of IT more broadly used to manage the pandemic throughout the country of China [ 52 ]. In this framework, government agencies, technology companies, medical facilities, and research institutions used technology including internet-based services, big data, predictive analytics, cloud computing, and AI for information delivery, screening, risk assessment, tracking, intelligent diagnosis, and telemedicine. Applied technology enabled detection, early response, intervention, and post-intervention processes to improve appropriate isolation and care. A common conclusion is that the EHR is an essential tool to support the clinical needs of a health system during a pandemic. However, EHR systems are not universally adopted. Faced with a different challenge, clinical informaticists at the Martinique University Hospital in France responded to the pandemic where they practiced without an EHR [ 53 ]. Sylvestre et al. describe in the JAMIA how after initially relying on classic outbreak monitoring with manual reporting, they developed simple technology-based tools to respond to rapidly evolving situations. Databases for triaged outpatient potential cases and hospitalized patients were built, and a web-application to enable monitoring, perform queries, and create real-time reports was created. Given globalization and large populations, technology and CIS were able to add significant benefit to otherwise manual processes. Kannampallil et al. from Washington University in St. Louis (USA) present their experience as informaticists engaging in an operational role during the pandemic and share four critical lessons applicable to all [ 54 ]: (1) work together, mitigate barriers, and fill data management gaps; (2) develop locally, share regionally and nationally; (3) adapt rather than build, and deliver knowledge at the right time, place, and format; and (4) support an evolving research enterprise.

Each of these authors report several challenges in completing extensive informatics build in a short period of time. One vital strategy also reported was the distinct benefit and need to include informaticists in the institutional incident command center structure to ensure proper prioritization and design of solutions to address health system needs. Many of the EHR-based tools mentioned above enable rapid process adaptation and can improve patient care. However, we found no objective evidence to show a benefit in medical outcomes with the use of these EHR-based tools. The literature in this area is largely descriptive due to the impracticality of well-controlled prospective or retrospective studies when rapidly responding to a global pandemic. Regardless, in the age of the EHR, it is clear that leveraging CIS and healthcare informatics expertise are important to a successful response and integral to the health system's response in addressing COVID-19. In addition to the EHR, one of the most important aspects of CIS throughout the pandemic has been the rapid and near universal expansion of telehealth services.

5 Telehealth

Over the last two decades, many healthcare systems have been slowly rolling out telehealth services. According to 2017 and 2018 American Hospital Association Survey data, more than 60% of hospitals reported adoption of some form of telehealth and more than 13% had tele-intensive care unit capabilities prior to the COVID-19 pandemic [ 55 ]. Prior to COVID-19, there was wide variation in deployment of telehealth infrastructure and utilization across the world and within individual countries, for example, in the USA there was slightly more utilization in rural areas [ 56 , 57 ]. Historically, the most commonly cited barriers to telehealth included poor reimbursement, lack of provider interest, regulatory barriers, lack of time or resources for training, insufficient telecommunications technology and infrastructure, privacy concerns, and patient preference for in-person care [ 55 , 58 59 60 ]. The COVID-19 pandemic provided sufficient incentives to simultaneously overcome many of these barriers in a very short time period. Given the concurrent needs to facilitate social distancing, minimize unnecessary infection exposures, and preserve personal protective equipment (PPE), patients and providers quickly reached to telehealth and related digital health solutions. As the number of COVID-19 patients increased in the early months of 2020, there was a tightly correlated spike in internet search volume for telehealth and telemedicine in the USA [ 55 , 61 ]. Partially based on previous experience with global emergencies, health care professionals quickly recognized the potential for telehealth to meet emerging healthcare needs during the pandemic [ 20 , 21 , 62 ]. Mehrotra et al. published one of the first accounts of rapid transition to telehealth within weeks of the public health emergency in four physician practices across the USA [ 63 ]. Of note, they highlighted heavy dependence on telephone over video visits because of technical, training, and workflow challenges. In contrast, as noted above, Reeves et al. also published one of the early reports of telehealth use in the pandemic as part of a broader clinical informatics response and highlighted the role of both existing telehealth infrastructure and redeployed clinical informatics team members to facilitate extremely rapid training and expansion of telehealth services [ 19 , 64 ]. Mann et al. published in JAMIA their experience at New York University, an early epicenter of the USA COVID-19 outbreak [ 65 ]. They noted initial greater than 2-fold growth of urgent care telehealth visits, quickly followed by greater than 40-fold growth in ambulatory visits, with consistently high patient satisfaction scores. As with the University of California, San Diego (UCSD, USA) group, these visits were primarily video-enabled on pre-existing telehealth infrastructure. In response to the pandemic, there was a rapid growth in applications and adoption of and use cases for telehealth in the ambulatory setting in almost every specialty from mental health to oncology to obstetrics [ 66 67 68 69 70 ]. There was also widespread extensions of telehealth utilization beyond evaluation and management by independent practitioners to provisions of a wide range of virtual therapy from physical therapy to occupational therapy to psychotherapy [ 66 , 71 , 72 ]. This extended spectrum of ambulatory telehealth use cases was partially enabled by the changes in legal and reimbursement policies discussed below. While telehealth adoption occurred first in the ambulatory setting following the COVID-19 outbreak, there quickly followed a spike in telehealth utilization in the inpatient setting as well [ 73 ]. Especially as shortages of PPE emerged, hospitals turned to novel applications of telehealth to minimize PPE requirements, enable social distancing, and minimize potential exposure to patients, families and staff. Inpatient telehealth applications included virtual multi-disciplinary rounds, telehealth consults both to isolated patients and from remotely located providers, communication/education with family members who were remote due to restricted hospital visitation policies, visual monitoring of patients in isolation, and virtually assisted patient resuscitation, amongst other novel applications [ 74 75 76 ]. Vilendrer et al. convened three very different health systems – an academic medical center, a teaching hospital, and a safety net county hospital – to share lessons learned [ 75 ]. The differences in patient demographics and available technology drove different use cases and design of solutions. However, all three organizations relied on established relationships with their vendor partners, pre-existing video conferencing solutions, and readily available consumer-grade hardware for rapid deployment. All three institutions identified ongoing gaps in translation services, hardware maintenance, and optimized security protocols. These solutions and implementation challenges are consistent with other case reports of rapid inpatient telehealth implementations [ 74 , 76 ]. While most of the published reports to date describe only process metrics and lessons learned from rapid implementation of inpatient telehealth, Jones et al. evaluated glycemic levels in patients before and after implantation of a virtual inpatient diabetes co-management service and found no clinically significant differences [ 77 ]. In addition to the pure clinical applications, there was early recognition that telehealth could be used to maintain associated clinical activities such as medical education and clinical research during pandemic conditions [ 78 ]. Direct observation and participation in clinical care is a critical component of medical education, yet in response to the need to minimize infection exposure and enable social distancing, many medical students and other trainees were initially excluded from the clinical setting or thrust into new modalities of care that disrupted traditional clinical supervision practices. Creative deployment of telehealth technologies such as virtual rounds [ 79 ] and proctored telehealth visits [ 80 ] enabled trainees to safely rejoin clinical conversations and participate in clinical care [ 81 ]. Similarly, where appropriate, telehealth has also been recommended and deployed to enable the continued conduct of clinical trials while minimizing risks to participants [ 82 ]. There are several factors which have contributed to the successful, rapid, widespread deployment of telehealth in the response to the public health emergency. Prior to the COVID-19 pandemic, regulatory issues and poor reimbursement had been repeatedly cited as barriers to deployment of telehealth services worldwide [ 59 , 60 , 83 ]. A strong incentive to focus resources on keeping patients and staff safe spurred governmental interventions to remove or decrease these barriers. In the USA, the Centers for Medicare and Medicaid Services rapidly expanded reimbursement for a broad range of telehealth services to beneficiaries and federal agencies encouraged and incentivized providers to implement telehealth services [ 84 85 86 ]. Importantly, these provisions extended to clinicians who may not independently bill for evaluation and management, including rehabilitation therapists and clinical psychologists [ 84 ]. Another critical policy change in the USA allowed temporary waivers for state-specific licensure [ 87 ]. Finally, enforcement of the Health Insurance Portability and Accountability Act was relaxed to allow broader access to communications technologies for providers and patients [ 88 ]. Similar regulatory relaxations and additional telehealth incentives were adopted in various countries and provinces around the world [ 60 , 83 ]. Most of the literature on telehealth deployment during the COVID-19 pandemic has focused on the potential and realized benefits, but there are important unintended consequences that deserve specific attention. While telehealth can improve healthcare access and ease the burdens of transportation, child care, time off work, etc. for families with limited resources, there has been growing recognition of the potential for telehealth to exacerbate disparities because of variable access to internet services and technology and varying levels of digital literacy [ 89 ]. Even as telehealth has been broadly deployed during the pandemic to reduce infectious exposure and reach patients whose travel was limited, there has been quick recognition of particularly vulnerable populations who are being disenfranchised by the evolving modes of healthcare delivery [ 90 , 91 ]. Nouri et al. evaluated the utilization of healthcare services in the University of California San Francisco primary care practices before and after wide-scale telehealth implementation at the beginning of the COVID-19 pandemic and showed statistically significant decreases in healthcare utilization by patients older than 65 years old, non-English speaking patients, and Medicare and Medicaid patients compared to their peers [ 91 ]. They outline a series of interventions to address barriers to telehealth adoption in vulnerable populations including interventions to increase digital literacy, provision of mobile devices, educating about free/low-cost broadband internet access, integration of interpreters, actively screening for barriers when scheduling visits, and advocating for policy and infrastructure development. As with all areas of health informatics innovation and research, clinical informaticists have a responsibility to critically assess and address inequities that are created or exacerbated as new telehealth technologies are deployed [ 92 ]. The practice of medicine has been irrevocably transformed by the surge of telehealth deployment during the COVID-19 pandemic. While it is certain that we will not revert completely to the pre-pandemic traditional practice of medicine, the precise landscape of post-pandemic telehealth remains to be seen. The recent investment in telehealth has helped establish long-lasting technical infrastructure to support telehealth going forward, but the policies that reduced telehealth barriers may easily be reversed. A wide range of permanent policy changes, infrastructure development, and outcomes research needs to be done to optimally integrate telemedicine and other digital health technologies into the future practice of medicine [ 20 , 21 ]. Furthermore, we need to better understand the effect of telehealth interventions on specific patient populations and to delineate which types of care are most amenable to this care modality. Prior to the pandemic, there were limited rigorous studies on patient outcomes associated with telehealth interventions [ 93 ]. The dramatic expansion of the telehealth implementations and use cases only exacerbates the need for rigorous research into the effects of this rapid transition in healthcare delivery. Clinical informaticists, who have a broad understanding of the people, process, and technical issues involved in ongoing telehealth optimization, are perfectly poised to lead the next phase of research and innovation.

6 Case Identification, Remote Monitoring, and Screening

Cohesive CIS across traditional healthcare, ambulatory, and community settings can create improved outbreak management and mitigation strategies at scale. As COVID-19 caused near universal travel restrictions, some nations were able to curb transmission by using technology to effectively screen individuals as they crossed borders [ 94 95 96 ]. Wang et al. in JAMA published the first and most robust report of electronically-assisted travel screening in Taiwan, a country of more than 23 million people [ 14 ]. A mere seven days after the Taiwan Centers for Disease Control (Taiwan CDC) activated their Central Epidemic Command Center, the National Health Insurance Administration and National Immigration Agency integrated patients’ travel history and patient identification cards into a unified database [ 14 ]. Government officials were then able to immediately identify those who: (1) were at risk due to recent travel in affected areas; (2) reported concerning symptoms during a clinical visit or online; and (3) were confirmed to be SARS-CoV-2 positive. They also utilized sensor data from mobile phones to identify those who may have been in contact with passengers from cruise ships known to have laboratory-confirmed outbreaks [ 97 ]. Furthermore, all hospitals, clinics, and pharmacies in Taiwan were granted access to patients’ travel histories to assist provider assessment and identification of additional potential cases. Following identification, individuals were automatically notified of risk status. Those entering the country classified as low risk were sent a health declaration border pass to personal mobile phones for expedient immigration clearance. Those identified as high risk, as well as domestically located patients with concerning symptomatology or other exposure, were notified to quarantine at home. The Taiwan CDC then surveilled and recorded daily health statuses of quarantined persons through the use of an electronic tracing system, termed the Infectious Disease Contact Tracing Platform and Management system [ 98 , 99 ]. In addition, the Taiwan CDC published a daily report of contact tracing data of individuals who were exposed to imported or indigenous confirmed COVID-19 cases [ 100 ]. The Taiwanese government has been heralded for these and other public health measures as a model example for transmission prevention. However, others express concern of violation of privacy rights [ 101 ]. As of November 28 th , 2020, Taiwan has recorded just 648 confirmed cases and 7 deaths due to COVID-19 [ 3 , 102 ]. For most countries, locally transmitted cases quickly became the predominant force driving new cases, and screening became the responsibility of local or regional public health officials and health systems [ 2 ]. Virtual screening systems were used to identify symptomatic patients who needed testing without requiring additional physical exposures [ 103 ]. Judson et al. at the University of California, San Francisco (UCSF) in the USA designed and deployed an electronic self-triage and self-scheduling tool to assist with efficient and appropriate dispositioning [ 104 ]. Accessed by patients through the EHR-tethered patient portal, the tool provided a series of questions about exposure, symptoms, and comorbidities. Branching logic determined the recommended visit type, allowed for direct scheduling, and provided advice and information. The tool was completed by asymptomatic (28%) and symptomatic (72%) patients who were triaged to emergent (24%), urgent (24%), non-urgent (12%), or self-care (40%). The sensitivity of detecting an emergency-level illness for patients who completed the self-triage tool was 87.5% (95% CI 72.0-79.5%) and the specificity for recommending self-care was 89.5% (95% CI 84.6-93.2%). Of the patients triaged to self-care, 61% had no further interaction with the healthcare system during the subsequent two days, suggesting the ability to reduce unnecessary encounters. Furthermore, the median time to having a scheduled appointment decreased by over two hours. Overall, the tool demonstrated an ability to improve triaging efficiency while decreasing unnecessary emergency department visits, thereby limiting unnecessary exposures. Judson et al. also created a chatbot-based employee screening tool at UCSF to facilitate front-entrance screening of healthcare workers [ 105 ]. Managing patients remotely was an important strategy to prevent overwhelming an overburdened system. Facing limitations in testing capacity, Annis et al. from the University of Minnesota (USA) configured and deployed an electronic patient education and a remote patient monitoring program for patients with diagnosed or presumed COVID-19 [ 106 ]. Utilizing interactive technology, the platform provided patients with disease information, reminders to physically distance and maintain hygiene, and daily symptom questionnaires to monitor progression. Patients had the ability to directly send messages to the care team and any potentially concerning results from the questionnaires were flagged for manual review. The overall activation rate of the program was 61.2% and the completion rate was 62%. 94% of patients who activated accounts checked in at least once and only 6.6% ultimately required an emergency department visit or hospital admission. The authors report the greatest benefit based on patient comments was a sense of safety from quick access to COVID-19 specific care. Ford et al. from the Medical University of South Carolina (USA) created several remote patient care tools via a COVID-19 registry in the EHR [ 103 ]. This included remote home monitoring, an integrated smartphone app with biosensors, and a dashboard to follow patient risk and disease progression. Perlman et al. describe a digital health application that offers self-assessment, an artificial intelligence-driven symptom checker, and communication with remote physicians if necessary [ 107 ]. For in-person encounters, electronic travel and symptom screening protocols embedded directly at the point of care helped to ensure compliance as patients accessed healthcare facilities [ 19 ]. Perez-Alba et al. utilized a quick response (QR) code posted on walk-in clinic walls to self-administer a questionnaire and provide information to staff in order to risk stratify, appropriately isolate, and prioritize patients without making direct contact [ 108 ]. In an effort to conserve PPE, Turer et al. from Vanderbilt University (USA) in JAMIA coined the term “electronic PPE” (ePPE) [ 109 ]. They define ePPE as the use of “telemedicine tools by on-site emergency providers to evaluate patients physically present in the (emergency department) to avoid physical proximity”. The use of ePPE allowed the safe triage of suspected patients. In addition to outlining their workflow with ePPE, the authors discuss legal implications as on-site physicians in the USA were previously required to perform medical screening exams that often involve a hands-on in person physical examination.

7 Diagnostic Testing

Accurate and timely diagnosis is key to pandemic management. The gold standard for diagnosis of infection with SARS-CoV-2 is reverse transcription polymerase chain reaction (RT-PCR) testing [ 110 ]. Unfortunately, there have been challenges with laboratory-based testing given the limited supply of test kits and a substantial burden on the normal processing of laboratories. With the struggle to increase testing capacity, many researchers have sought to use technology and CIS to help alleviate the situation by: improving screening and triaging to reduce unnecessary testing [ 111 , 112 ]; improving risk stratification to prioritize testing [ 113 , 114 ]; enhancing laboratory information systems (LIS) to reduce processing time [ 115 ]; and (4) supporting alternative mechanisms of diagnosis [ 116 ]. Weemaes et al. in JAMIA described the design, implementation, and requirements of the LIS necessary to support an exponential influx of SARS-CoV-2 testing at the Belgian National Reference Center for COVID-19 testing [ 115 ]. The authors detail how the LIS can help remove common bottlenecks across the following five phases of laboratory-based testing: (1) Pre-laboratory phase: computerized order entry and COVID-19 order sets aid in standardized and uniformed ordering; digital notifications are used to improve sample collection and shipping instructions; (2) Pre-analytical phase: computerized order entry alleviates burden on administrative staff for sample registration; automated scripted triaging enables quick and adaptable testing triaging; sample tracking enables easy retrieval for rapid testing, select sampling for studies, and individualized turnaround time estimates; (3) Analytical phase: bidirectional interfacing alleviates burden on laboratory technicians; (4) Post-analytical phase: statistical flagging of outliers for automated validation; automated fax and encrypted email enables automated reporting alleviating burden on administrative staff and call centers; and (5) Post-laboratory phase: automated email, text messages, sample tracking are used for epidemiological reporting and study; computerized order entry enables index patient tracking; searchable database allows for easily retrievable detailed information on each sample and daily summary statistics to assist in data-driven adjustments in crisis management.

8 Artificial Intelligence in Diagnosis and Predictive Analytics

At the onset of the outbreak, statistical modeling was heavily relied upon to forecast disease incidence, hospital censuses, ventilator requirements, and deaths [ 117 118 119 120 121 ]. These models were useful for policy makers and hospital administrators preparing for regional surges. As Combi and Pozzi's detail in their recent IMIA Yearbook Survey, the application of artificial intelligence (AI) in healthcare has become a popular area of research [ 17 ], and many now assert the potential for AI systems to contribute to containment of the COVID-19 pandemic [ 122 ]. Debnath et al. wrote a perspective in Bioelectronic Medicine outlining the points in a typical patient care pathway where machine learning and AI systems can potentially augment clinical decisions [ 123 ]. These include diagnosis and screening, initial and ongoing risk stratification, prognosis and prioritization of care, and allocation of resources. Obeid et al. in JAMIA describe an application of natural language processing and AI to improve their virtual care screening algorithms [ 111 ]. During their telehealth consultation and screening visits, data is directly entered by symptomatic patients via both a template-based form and free text entry. The resulting data was analyzed for differences in word frequencies between patients who ultimately tested positive and negative for COVID-19. Interestingly, words mentioned in higher frequency by patients who tested positive included those such as “smell”, “taste”, “sense”, and “loss”. This analysis was embedded in a convolutional neural network (CNN) AI model for predicting COVID-19 positivity based on patients’ reported symptoms. The performance of the model was relatively low, with an area under the curve (AUC) of 0.729, limiting its usefulness and generalizability. A more frequent application of AI was in the diagnosis of COVID-19 based on radiology studies. Zhang et al. in Cell detail the development of one such system based on chest computed tomography (CT) images [ 124 ]. The authors used a large database with images from patients with novel coronavirus pneumonia (n=3,777), other common forms of non-COVID-19 pneumonia, and normal controls. Their AI system consisted of two models, a lung-lesion segmentation model and a diagnosis prediction model. They used clinical and radiological features to create a prognostic model to predict progression to critical illness. The AI system was tested and validated in an initial retrospective cohort study, three prospective pilot studies within separate Chinese provinces, and a final open source and international prospective pilot study. The results were promising, demonstrating an overall accuracy of 92.49% and an AUC of 0.9813 (95% CI: 0.9691 – 0.9902), establishing an ability of AI systems to assist in rapid diagnosis and prognosis of COVID-19. Li et al. in Radiology published similar results in utilizing a deep-learning AI model to differentiate novel coronavirus pneumonia from other forms of community-acquired pneumonia based on CT images [ 125 ]. Murphy et al. published a study in Radiology of a deep learning-based AI system that analyses frontal chest radiographs rather than CT images [ 126 ]. The system was compared with the performance of six independent thoracic radiologists and in most cases was found to perform at the same level or better. Achieving an AUC of 0.81, the authors show the potential to utilize machine learning algorithms to support radiologists in resource-limited settings who may not have access to CT imaging. Both Zhang et al. and Murphy et al. made their models publicly available online. Similarly, Hurt et al. from UCSD published a deep learning localization of pneumonia for COVID-19 assessment in the Journal of Thoracic Imaging [ 127 ]. This work was rapidly implemented as a cloud-based tool that processed every frontal chest radiograph and chest CT across the UCSD health system. Carlile et al. subsequently found that 86% of emergency department physicians agreed that the intervention was easy to use in their workflow, and 20% of respondents reported that the algorithm impacted clinical decision making [ 128 ]. In a different approach, Goodman-Meza et al. in PLOS One described the creation of a machine-learning algorithm that uses blood laboratory values to predict the presence or absence of a COVID-19 infection [ 129 ]. Pattern recognition of characteristic changes in the complete blood cell count and elevations in inflammatory markers such as C-reactive protein and lactate dehydrogenase were included from 182 COVID-19 positive patients. The model achieved a sensitivity of 0.93 (95% CI 0.85–0.98) and specificity of 0.64 (95% CI 0.58–0.69) with an AUC of 0.91 (95% CI 0.87–0.96). Their results show that ancillary laboratory values and machine learning algorithms may be used as screening tools in settings where PCR testing is limited without the use of chest CT. Mei et al. in Nature Medicine integrated chest CT findings, clinical symptoms, exposure history, and laboratory values in comparing three diagnostic AI algorithms [ 116 ]. The first was based on chest CT images and two levels of CNNs (abnormal slice selection followed by diagnosis). The second used demographic and clinical data within a machine learning model. Finally, features generated by the first two models were integrated by multilayer perceptron to produce the final output of the algorithm. Among a test set of 279 patients, the AI system had comparable sensitivity with a senior thoracic radiologist. However, one key finding was that of 25 patients with a normal chest CT at presentation, 17 (68%) were classified at COVID-19 positive by the AI model, whereas 0 (0%) were classified as positive by the senior thoracic radiologist. The major application of such a system could be as a useful triage tool for positively identified patients, particularly those with otherwise negative imaging, while awaiting definitive PCR results. Liu et al. created a dynamic risk assessment decision support system that was used by Chinese general practitioners in Zhejiang Province during the outbreak [ 112 ]. They constructed a dynamic risk stratification model based on a multiclass logistic regression algorithm to classify patients as low, medium, or high risk of having COVID-19 infection. Patient demographics, clinical symptoms, contact history with or without blood results, and CT imaging were used in the model to produce high sensitivity and specificity. The model was used to assist general practitioners in assessing the appropriate risk, triage, management, and follow-up for suspected patients. Liang et al. in Nature Communications applied a deep-learning neural network-based survival model to predict the risk of critical illness for patients diagnosed with COVID-19 [ 130 ]. Impressively, the model, which uses ten clinical variables, was developed from data obtained from 575 medical centers across China. Following internal and external validation, they created an online tool embedding the model within a simple nomogram which predicted the probability of critical illness within 5, 10, and 30 days. This model offers a user-friendly way to use AI to triage patients at the time of hospital admission. A common limitation to all of the aforementioned studies was the small volume of COVID-19 positive patients that were used as data points. In an effort to support research and development of AI technology, Jacob et al. developed a National COVID-19 Chest Imaging Database in the United Kingdom (UK) [ 131 ]. Briefly described in the European Respiratory Journal, the authors highlight the long-standing challenge of data-sharing and acquisition of large-scaled volumes of imaging data needed to produce the most successful computer algorithms. In response, the British Society of Thoracic Imaging research network in partnership with the National Health Service in the UK created a repository of chest radiographs, CT scans, and magnetic resonance imaging images. The goal is to join with other international entities to make data available to research and commercial groups to accelerate technological innovations designed to improve COVID-19 patient care. Despite the potential, there are several limitations to the widespread use of AI to combat COVID-19. Firstly, we found no evidence of improved outcomes over standard provider-based patient care. Second, use in resource-limited areas is challenging. Third, there are potential ethical concerns using AI to make patient care decisions [ 132 , 133 ]. An early systematic review of COVID-19-related prediction models published in the British Medical Journal by Wynants et al. reported underwhelming results [ 134 ]. Upon review of 107 studies of 145 models (91 diagnostic, 50 prognostic), the authors did not recommend any prediction models for use in current practice. They offer a detailed analysis, but important limitations included a high risk of bias, non-representative selection of control patients, and vague reporting of study population or intended use of the models. Chen and See also performed a review published in the JMIR and found several similar limitations [ 135 ]. In July of 2020, Bakker et al. from the Netherlands published a systematic review of the health and economic impact of big data analytics for clinical decision making (not specific to COVID-19) in the JAMIA [ 136 ]. They reviewed 12,311 papers for eligibility and found only seven papers that reported cost-savings and improved outcomes from implemented “big data analytics” models. They discussed validation and deployment costs as an important barrier to implementing models into clinical practice and a lack of consistent definitions of big data. Bakker et al. concluded that the high expectations of big data and AI systems have not yet been realized. However, we remain optimistic for the potential of AI in diagnosis, public health, clinical decision-making, and therapeutics. Improved data sharing to expand and diversify data sources, increased focus on translational research in diagnostic and predictive analytics, and the expanding role of clinical informaticists due to the pandemic can increase the usability of such systems in the future.

9 Data Sharing and Interoperability

The prevalence of the EHR creates the potential for timely health information exchange (HIE) that can enhance both clinical care of shared patients and enable the aggregation of big data. For example, the effectiveness of the AI systems described in the previous section is reliant on the size and quality of the datasets upon which they are built. Plasek et al. in JAMIA offer a perspective on the importance of cross-border data sharing during the COVID-19 pandemic and the use of diverse and heterogenous datasets in order to overcome inherent limitations of localized data sources [ 137 ]. Their article highlights that critical public health decision making, clinical research, and pharmaceutical development all rely on global data, coordination, and solidarity. Zeng et al. provide a review of classification systems, taxonomies, ontologies, and subject headings used during the COVID-19 pandemic [ 138 ]. Proper use of these tools are essential to support HIE and information management. Unified classification systems such as the ICD are critical in providing a common language for reporting and monitoring disease-related outcomes. Done correctly, aggregating big data across populations, across geographic boundaries, and over time can enable a better understanding of the novel coronavirus, intelligent predictions, and effective tracking and surveillance of populations. For more than a decade, common data models (CDMs) have been developed in healthcare with the intent of standardizing and facilitating HIE. One example of a large, commonly used CDM is the Observational Medical Outcome Partnership (OMOP), a public-private partnership founded by the US Foundation for the National Institute and now maintained by the Observational Health Data Sciences and Informatics consortium [ 139 , 140 ]. OMOP enables large-scale comparative effectiveness and safety evaluation across millions of patients [ 141 ]. And in fact, several observational studies of patients with COVID-19 have been performed based on OMOP or similar CDMs [ 142 143 144 145 146 ]. However, despite substantial efforts to achieve cohesive HIE, highly effective data sharing remained elusive when the COVID-19 pandemic started. As a result, several new large data consortia initiatives emerged specifically in response to COVID-19 [ 147 ], including the National COVID-19 Cohort Collaborative (N3C, Haendel et al. [ 148 ]), the Consortium for Clinical Characterization of COVID-19 by EHR (4CE, Brat et al. [ 149 ]), and the Secure Collective Research (SCOR, Raisaro et al. [ 147 ]). None are universally adoptable though each offers potential for collaborative learning and eventual clinical translation. In addition, there are examples of smaller initiatives and tools developed to enhance interoperability of health systems. Dong et al. created a normalization system for mapping heterogenous locally developed COVID-19 testing nomenclature to the international standard Logical Observation Identifiers and Codes (LOINC) codes [ 150 ], termed COVID-19 TestNorm [ 151 ]. With an accuracy of 97.4%, the authors report that widespread use of the tool can facilitate data aggregation and research. To aid local, national and international HIE across all stakeholders, the US CDC developed the CDC COVID-19 Information Management Repository [ 152 ]. Garcia et al. in JAMIA provided a thorough introduction to the in-depth repository [ 153 ]. The website supported by the CDC coalesces information from national organizations such as the Council of State and Territorial Epidemiologists and the Office of the National Coordinator for Health Information as well as international organizations such as the WHO, LOINC, ICD-10, and Systematized Nomenclature of Medicine (SNOMED). The repository categorizes COVID-19 data interoperability information into the following sections, among others: General resources: regulatory guidance documents, data sources, foundational semantic interoperability standards; Representing healthcare data for emergency medical services; Patient's clinical encounter: standard codes for diagnoses, clinical procedures, billing, and death certificates; COVID-19 public health reporting; Laboratory data exchange and laboratory surveillance: codes for COVID-19 specimen submission, test orders, test results; and geospatial data sets and reference sources.

However, the global infrastructure for HIE is still lacking and gaps in interoperability and data sharing have long been challenging. Cosgriff et al. emphatically state in The Lancet Digital Health, “never before has the failure to build robust data sharing systems for large-scale near real-time analysis in health care been more obvious” [ 154 ]. There is no single publicly available, multi-national registry containing patient-level COVID-19 data that can be vigorously studied and used to develop highly predictive models or answer potential clinical questions. Similarly, a recent study by Bruthans in IJMI found that the cross-border interoperability of electronic prescribing systems in the European union and USA is limited, directly affecting the efficiency of healthcare systems [ 155 ]. O’Reilly-Shah et al. in Anesthesia & Analgesia comment on this shortcoming and describe several forces that undermine efforts to improve interoperability [ 156 ]. Counterincentives of EHR vendors, lack of standardized specifications, and privacy and ethical concerns are among many reasons for persistent information silos. Furthermore, data collection and processing must be done responsibly and with respect for privacy and confidentiality [ 157 ]. Ienca and Vayena in Nature Medicine articulate how the failure to do so can violate individual privacy rights, override informed consent, and fuel distrust of the geopolitical and health systems [ 157 ]. This is no easy task as well-intentioned regulations can sometimes result in more harm than good. Lenert and McSwain discuss how federal regulations in the USA created to protect patients’ rights have limited the flow of information across systems, negatively affecting patient care during the pandemic [ 158 ]. Operationally, inadequate interoperability significantly impacted the ability of providers and hospitals to effectively report confirmed cases of COVID-19 to public health agencies. An analysis of national health security capacities published in The Lancet by Kandel et al. found that only 76% of countries globally had a robust detection capacity [ 159 ]. Holgrem et al. in JAMIA discussed barriers to effective and timely reporting during the COVID-19 pandemic [ 160 ]. They surveyed American Hospital Association data fielded from chief executive officers of US hospitals to identify barriers to electronic submission of health information to public health agencies, including public health agencies’ inability to electronically receive data (reported by 41.2% of hospitals), interface-related costs (31.9%), difficulty extracting EHR data (14.7%), varied vocabulary standards (14.2%), inability of hospitals to send data electronically (8.3%), and not knowing where to send data (3.3%). The finding that 40% of public health departments required manual reporting highlights that investment in technology and interoperability infrastructure is necessary for highly functioning CIS. Stenner et al. contributed a viewpoint in ACI making a distinct yet important point that more information is not always beneficial [ 161 ]. They describe how overloading the information receiver with unfiltered electronic health information can present more data than can be reasonably processed. The result may be a paradoxical decrease in efficiency and the ability to find clinically relevant data points. Foraker et al. discuss the well-known challenges to effective data-sharing [ 162 ]. The authors offer a framework to address inadequate infrastructure, duplication of data requests, and insufficient coordination by responsible entities: (1) Identify and fill technology gaps relevant to data sharing efforts in order to reduce the time to implementation; (2) Engage in the collaborative design of data-sharing requirements and transmission mechanisms to reduce redundancies and establish economies of scale; (3) Facilitate cross-domain discussion involving legal and research compliance to identify pathways for data-sharing efforts to be appropriately and effectively managed from a regulatory perspective; and (4) Establish or participate in multi-institutional convening or coordinating activities. Sim et al. in Science called for the US National Institutes of Health (NIH) to take the lead on achieving “findable, accessible, interoperable, and reusable” scientific data sharing [ 163 ]. As the largest global funder of biomedical research, the authors argue for NIH policy to include mandatory data sharing beginning with any registered clinical trial. Given the variability in international healthcare delivery systems and governance of health IT infrastructure, development and implementation costs, language barriers, and other misaligned incentives, it stands to reason that truly effective HIE will be a challenge for the next decade and beyond. Though one single international system seems unlikely, the trend in collaborative efforts focused on standardized data collection and nomenclature remain encouraging.

10 Epidemiologic Reporting

A core foundational tenet in the effective management of any disease is the ability to have timely and accurate data around underlying structural factors, processes and outcomes. Similarly, management of a pandemic requires large-scale data across both time and space. Many academic organizations, private institutions, and governmental agencies have developed geographical information systems (GISs) -- largely in the form of online dashboards -- to enable real-time epidemic monitoring and subsequent clinical responses for COVID-19. The first and most notable global COVID-19 dashboard came from Lauren Gardner and her team at the Center for Systems Science and Engineering at John's Hopkins University (USA) [ 164 ]. They developed an interactive web-based dashboard to visually display and track confirmed cases, deaths, and recoveries from COVID-19 at the national, provincial, or city level depending on the country of interest [ 164 ]. They adopted a semi-automated living data stream strategy aggregating data from multiple authoritative sources including the WHO, regional public health departments and respective CDCs, as well as social media, news feeds, and direct communications. First publicly shared on January 22 nd , 2020, the real-time COVID-19 dashboard was able to report newly infected cases ahead of the WHO. It quickly became a widely cited, highly reliable source of data for academics, public health authorities, and the general public at large. Several other well-known COVID-19 trackers have been developed and made available including the WHO Coronavirus (COVID-19) Dashboard, the CDC COVID-19 Data Tracker, and HealthMap [ 165 166 167 ]. Kamel Boulos and Geraghty in the International Journal of Health Geographics wrote an early editorial outlining how these and other GISs can support the fight against infectious disease outbreaks through data transparency, communication, and by creating a shared understanding of the pandemic [ 168 ]. There are limits to the level of granularity of data able to be displayed within global GISs, as well as limits around regional and other needed micro-analyses. As such, there was a need for additional sources of information to assist local municipalities. In one example from the USA, Wissel et al. created an open access interactive dashboard that aggregated data on a county-level to display temporal changes in testing capacity, cases, and deaths [ 169 ]. Similarly, CovidCounties, a COVID-19 County Tracker from the Butte Lab and OptumLabs (USA), was developed and published online [ 170 ]. In many instances, local public health departments and health systems created their own dashboards based on automatic, semi-automatic, or manually reported case data [ 15 , 19 , 171 ]. This locally derived data was frequently integrated with local EHR instances to give administrators and front-line providers real-time information to guide clinical decisions. In another intriguing application of visual reporting, Thorlund et al. developed a COVID-19 clinical trials registry utilizing AI to supplement identification of qualified trials [ 172 ]. They then mapped trials based on location, patient, and intervention characteristics to create a visual dashboard of global COVID-19 research efforts. Dixit et al. in JAMIA published the development of eVisit Operations Dashboards for executives, telehealth operational leaders, and management teams at a large health system in the USA [ 173 ]. They share several lessons regarding rapid implementation of visual reporting displays: use a lightweight and iterative user-centered approach; basic (accurate) information is better than no visualization; clearly indicate development status and timeliness; develop prioritization criteria for visualization and feature development; stabilize the data environment as much as possible; and metric design, user literacy, and user control should be based on audience.

11 Digital Contact Tracing and Exposure Notification Systems

Contact tracing, a key component of minimizing disease transmission and essential tactic of pandemic management consists of identification, notification, and subsequent surveillance of individuals who have been exposed [ 174 , 175 ]. Conventional measures of contact tracing, largely based around manual interviews, are laborious and time-consuming and have been challenging to do effectively during the COVID-19 pandemic [ 176 ]. Furthermore, manual tracing is prone to recall error from interviewed patients. Overwhelmed with new cases, many organizations explored technology-based tools to create automatic or partially automatic digital contact tracing [ 177 ]. As described above, Taiwan was one country with a robust mechanism of digital contact tracing that putatively helped to reduce transmission [ 14 , 98 ]. In an early publication in Science, Ferretti et al. utilized mathematical modelling to demonstrate that algorithmic, immediate notifications to exposed individuals with a contact-tracing mobile phone application could contain the spread of COVID-19, especially if used in concert with physical distancing [ 177 ]. The beneficence of electronic contact allows for quick identification of close contacts of positive cases followed by rapid testing and quarantine of suspect or positive cases to contain the spread of the virus [ 178 ]. Several countries, including Singapore, South Korea, and Australia quickly developed electronic contact tracing applications and processes [ 179 , 180 ]. Using Bluetooth technology or Global Positioning Systems, mobile phones can securely track an individual's proximity to other mobile devices that are using the same technology. Should one of those individuals become infected with COVID-19, it would be recorded in either a centralized or decentralized database and an automatic notification would be sent to the exposed individual with applicable instructions to self-quarantine. Online reports indicate some success of these applications. For example, Singapore reported that as of November 3, 2020, their contact tracing application had helped identify 25,000 close contacts of COVID-19 cases, of which 160 tested positive [ 181 ]. Dozens of nations have now published these smart phone-based digital contact tracing applications [ 182 ]. However, we found no reports of successful widespread implementation of such a strategy. Huang et al. in JMIR Mhealth Uhealth compared the effectiveness of the Bluetooth-based TraceTogether contact tracing app with a wearable tag real-time locating system (RTLS) in the National Centre for Infectious Diseases, the national referral center for COVID-19 screening in Singapore [ 183 ]. They found that RTLS had a higher sensitivity of 95.3% vs 6.5% for detecting patient contacts. The authors discussed that while wearable RTLS is more effective in a healthcare setting, deployment and implementation would be substantially more challenging in a community setting. With increased penetration of the app and token in the country, officials hope to see updated figures in the near future. Two systematic reviews were conducted assessing the effectiveness of digital contact tracing. Braithewaite et al. identified 4,036 studies and included 15 studies in their final analysis. They found no epidemiological studies that offered empirical evidence comparing automated with manual contact-tracing systems [ 184 ]. Anglemeyer et al. performed a Cochrane review and concluded that the effectiveness of digital solution in contact tracing is largely unproven [ 185 ]. There are several important reasons why this strategy has not widely succeeded in practice. First, effective transmission prevention during a pandemic would require that most of the population has access to smart phones. At the beginning of 2020, 46.4% of the world population had no access to the internet [ 186 ]. This perhaps surprising figure highlights that digital contact tracing and many of the technologies described in this survey serve mainly to benefit a relatively privileged segment of society. Furthermore, to effectively impact COVID-19 transmission, various models suggest end-user adoption thresholds between 56% to 95%, accurate self-reporting, and adherence to home isolation notifications of 90 to 100% are necessary [ 184 ]. End-user compliance however, is largely voluntary and thus presents major barriers to effectiveness. Finally, digital contact tracing poses privacy concerns related to accessing personal information [ 101 ]. Most app developers have taken considerable measures to encrypt data, delete after its useful time, or otherwise protect confidentiality [ 180 ]. However, even perceived potential to expose personal information substantially limits adoption, highlighting that trust between the government and the population needs to be present for electronic contact tracing to be successful. However, both Braithewate et al. and Anglemeyer et al. reported the potential for partially or fully automated digital contact tracing to provide more complete contact identification and reduce the amount of time required to complete contact tracing [ 184 , 185 ]. Digital contact tracing has also been applied within hospitals and among healthcare workers [ 187 ]. By integrating data from five different hospital-based information systems, Venkataraman et al. rapidly and comprehensively tracked COVID-19 patients’ activity within their system to identify both healthcare workers and patients at risk [ 188 ]. Staff were then able to focus time on interviews and risk assessment rather than contact identification. Following implementation of the algorithm, the authors observed a greater than 60% reduction in time needed to complete contact tracing. For most countries, reliance on digital contact tracing is perhaps unrealistic. However, incentive-based digital strategies can serve to augment rigorous manual contact tracing and increased access to digital technology among general populations could offer a more reliable strategy for outbreak management in the future.

12 Outcomes from Clinical Information Systems on Non-COVID-19 Patient Care

Providers and the healthcare delivery system in which they practice must ultimately strive and be held accountable for continually optimizing patient outcomes. This is reflected by improving disease specific outcomes, process outcomes, and minimizing medical errors and adverse events. It is too early to fully understand the role that CIS played in the morbidity and mortality of COVID-19 patients or the transmission of COVID-19 across populations. However, there is mounting evidence from the past decade examining how CIS are frequently used to improve patient outcomes in other areas of medicine. The opioid crisis is a recent example where frontline care providers and public health policy aligned with CIS to make meaningful improvements. Before COVID-19 emerged, the opioid crisis was the most prominent global epidemic with over 58 million reported users and over 350,000 deaths related to opioids worldwide [ 189 , 190 ]. CIS have been leveraged to support decision making around analgesia with opioid and non-opioid prescribing recommendations as well as to facilitate adherence to state-level requirements for prescription drug monitoring in all settings [ 191 ]. Weiner et al. in the Joint Commission Journal on Quality and Patient Safety report how a collaborative approach utilizing health IT reduced the flow of opioids into communities while increasing prescriptions for medication assisted treatment, key process measures [ 191 ]. Chronic opioid addiction is complex and observed reductions in opioid overdoses have been less consistent highlighting the need for continued advancement to translate process improvements to measures like mortality. In the inpatient setting, CIS have been leveraged to successfully influence provider behavior with the aim of improving clinical outcomes [ 192 ]. Emergency medicine and critical care medicine are two areas that have embraced the use of data and CIS to support management of persistently challenging diseases. However, the literature shows mixed results, a perfect example being sepsis. EHR-based tools have been credited with improvement in the prediction of sepsis and the appropriate and timely delivery of proven interventions such as intravenous fluids and antibiotics, yet other studies have shown limited clinical impact on sepsis treatment performance measures and overall mortality [ 193 194 195 ]. Similarly, implementation of non-EHR based care bundles to manage sepsis have shown no impact on mortality, highlighting that adherence to protocolized care strategies still has limitations among difficult-to-treat diseases [ 196 ]. There is ample evidence that clinical pathways and clinical guidelines are more likely to result in practice changes – usually determined by process measure compliance – if accompanied by effective integration into an EHR. For example, in the surgical areas, most hospital-based implementations of evidence-based clinical pathways have leveraged order sets embedded in the EHR to incorporate processes related to enhanced recovery and CDS alerts related to preventable harms [ 197 ]. Importantly, EHR modification alone may be effective at improving process measure compliance. However, this may not be sufficient for moving the needle on clinical outcomes like surgical site infection, venous thromboembolism and length of hospital stay [ 198 ]. Rather, EHR-based tools must be used in combination with thoughtful programmatic implementation and operational change management to meaningfully improve patient outcomes as evidenced by experience in colorectal, gynecologic and urologic surgery [ 199 ]. When considering vexing clinical problems, it is important to consider the role CIS can play in optimizing disease management, but this must be done in a deliberate manner that weighs both the technical and adaptive components of clinical care and provider behavior [ 200 ]. Convincingly, clinical informaticists, together with engaged leadership and empowered frontline providers, have helped to advance quality and patient safety along the continuum of healthcare before and during the COVID-19 pandemic.

13 Discussion

To effectively control the COVID-19 pandemic, the following global strategic objectives are advised by the WHO [ 201 ]:

Case prevention across all sectors of government and society through hand hygiene, respiratory etiquette, and individual level physical distancing;

Rapid and accurate case identification and isolation;

Effective tracing, quarantining, and support of exposed individuals;

Context appropriate infection prevention and control through population level physical distancing measures, and appropriate and proportionate travel restrictions;

Reduce mortality by providing appropriate clinical care, protecting frontline workers, and ensuring continuity of essential health services;

Development and distribution of safe and effective vaccines and therapeutics.

In the current survey, we present examples from the literature demonstrating how CIS have supported these management strategies during the COVID-19 pandemic as the world eagerly awaits effective vaccines and therapeutics. CIS enabled or enhanced the implementation of health safety protocols; the ability to screen, test, and diagnose at scale; data collection, analysis and reporting; population cohorting and contact tracing; the rapid advancement of research; and the provision of safe and effective essential healthcare through telehealth services. Informatics has also been used in the search for effective therapeutics and vaccinations through predictive frameworks relying on immuno-informatics computational modeling [ 202 203 204 ]. In addition, informatics infrastructures such as immunization registries will be essential to the timely and equitable distribution of an effective vaccine [ 205 , 206 ]. Thus, CIS and health informatics have been successfully utilized and will continue to be deployed across the spectrum to facilitate pandemic management [ 207 ].

Despite our optimism in the current capabilities of CIS to support the management of COVID-19, everyday healthcare, and yet unknown public health crises, there remain significant gaps in the field. Challenges persist in translating promising research into clinical practice in areas such as AI and predictive analytics [ 208 , 209 ], achieving end-user adoption of tools such as digital contact tracing applications [ 185 ], addressing ethical and privacy concerns, and realizing impacts on clinical outcomes such as improved mortality following successes in process measures. Perhaps the most significant challenges to address are a lack of generalizability, interoperability and data sharing, and coordination among healthcare organizations, public health agencies, and other healthcare entities.

Unlike a novel disease-altering medication or a superior anti-hypertensive that can be dispensed to pharmacies and patients around the world, proven EHR-based interventions are rarely shared across health systems or internationally [ 210 ]. Herein, we discuss several use cases of CIS-based tools that can substantially improve processes and patient care when facing a pandemic. However, the generalizability of most is quite limited as end-user adoption requires system-wide CIS development, accommodation for local workflows, and programmatic implementation. Informaticists from different healthcare organizations are persistently re-inventing the wheel, developing systems and configuring EHRs repeatedly to address what are most often shared challenges. Similarly, effective HIE and big data sharing are challenging due to varied data definitions and collection, lack of interoperability between EHRs, and uncoordinated efforts among stakeholders [ 11 ]. Even large health systems cannot fully achieve the potential of CIS alone. Madhavan et al. in JAMIA offer a thorough perspective on the issue of uncoordinated responses outlining the need to engage stakeholders from the informatics, public health, epidemiologic, and administrative communities in public, private, and non-profit sectors to strive for standardization and collaboration [ 211 ]. Among many others, these authors submit a call to action to unify, simplify, standardize, and share.

Since the onset of the pandemic, research articles have been published via open access, GIS dashboards have been shared publicly, algorithms and codes have been posted online for others to use, and multiple large data consortiums have re-emerged in an effort to collaborate globally in the fight against COVID-19 [ 124 , 131 , 147 , 177 ]. It remains to be seen whether the widely prevalent intentions to equitably share and standardize information technology will prevail. Time will tell if lessons learned from COVID-19 allow a more coordinated and potentially more effective informatics response to the next pandemic. Certainly, substantially more informatics infrastructure has been developed in the past several months when compared to the decades following previous infectious disease outbreaks. However, we suspect coordination and generalizability will be significant challenges for the informatics community for the foreseeable future.

In the face of these challenges, CIS and clinical informaticists made prominent and influential contributions in the global response to the COVID-19. Through the timely and effective distribution of reliable information, CIS can enhance patient care during a public health crisis.