Visual Analytics for Decision-Making During Pandemics.

We introduce a trans-disciplinary collaboration between researchers, healthcare practitioners, and community health partners in the Southwestern U.S. to enable improved management, response, and recovery to our current pandemic and for future health emergencies. Our Center work enables effective and efficient decision-making through interactive, human-guided analytical environments. We discuss our PanViz 2.0 system, a visual analytics application for supporting pandemic preparedness through a tightly coupled epidemiological model and interactive interface. We discuss our framework, current work, and plans to extend the system with exploration of what-if scenarios, interactive machine learning for model parameter inference, and analysis of mitigation strategies to facilitate decision-making during public health crises.

Making timely, effective, science-based decisions to mitigate the impact of a pandemic is a very difficult and highly complex task requiring a decision-maker to evaluate multiple disparate data sources. Decisions such as when to reopen require collecting and integrating accurate information on an array of the characteristics, including: disease spread dynamics, prevalence of infections when initially detected, capacity and supplies available at all hospitals in the state, the effectiveness of each potential mitigation strategy, the delay in adopting mitigation behaviors, and assessing public compliance with public health measures, and other public behavior.

Our approach to addressing these problems is to combine available data from State Departments of Public Health, population demographics, fine-scale population behavior and mobility data and predictions, multiple advanced epidemiological models, epidemiologists, and continuously refined predictions and data-driven model parameters, into a decision-making and situational awareness dashboard to enable evaluation of current situation and strategies for implementing nonpharmaceutical interventions (NPIs).

The goal is to create an interactive decision-support and dashboard system for interactive public health situational awareness, planning, and response.

Given that the data landscape surrounding COVID-19 is evolving so quickly, we emphasize that work will continue for quite some time. We present the initial results of a trans-disciplinary collaboration between researchers, healthcare practitioners, and community health partners in the Southwestern U.S. to help enable improved management, response, and recovery options for our current pandemic and future health emergencies through the development of an integrated data dashboard. This dashboard and associated technical advances enable a decision maker to:

model and visualize how NPIs impact the spread of COVID-19;

monitor the spread of COVID-19 related news on various social media platforms;

design effective risk communication strategies to ensure compliance with NPIs.

These advances are central to the new Center for Integrated Public Health Monitoring, Analysis and Decision-making (CIPH-MAD), a collaboration of researchers and decision-makers dedicated to predicting and mitigating public health emergencies in the South-western US. This work is applicable to future pandemic and public health emergencies by providing a framework to integrate and synthesize multiple disparate data sources.

BACKGROUND

Early detection and action are key to mitigating the effects, as demonstrated with the successful response to Ebola. Effective detection, mitigation, and response rely on accurate information, analytics, and predictions of the effect of interdiction/mitigation strategies. Over the past nine months, there has been great progress in gathering data on the COVID-19 pandemic. New data sources, including social mobility data and public health electronic surveillance data, are becoming available and are being used effectively in a few locations. These new sources greatly increase situational awareness and provide rapid feedback regarding the effectiveness of various public health actions and policies. However, the available data are often conflicting, biased due to sampling, and incomplete. Given the size, scope, and complexity of pandemic data, it can be difficult for a decision-maker to gauge the effectiveness of different mitigation strategies without effective computational support.

Fortunately, Visual Analytics (VA) tools and techniques allow a decision-maker to address certain problems whose inherent size, complexity, and need for closely coupled human and machine analysis may make them otherwise intractable. The advantage of using a VA system in a disease modeling and mitigation context is that a decision-maker can compare and contrast the effect different intervention techniques—including social distancing, mask usage, or closing schools—will have on the spread of a given disease throughout a region. Furthermore, a decision-maker can use a VA system to explain to other stakeholders and the public what the near- and long-term effects of a particular decision will be. VA systems can communicate complicated and nuanced findings from statistical models to a larger audience.

Previous VA work has demonstrated the effectiveness of interactive decision support tools at identifying intervention policies. One tool, PanViz, was initially developed to provide public health officials from the Indiana State Department of Health with a suite of visual analytic tools for analyzing pandemic influenza spread, while enabling these officials to analyze various decision points (e.g., school closure, strategic national stockpile release) and their impact on disease spread. The tool allowed for demographic filtering by age ranges, and interactive manipulation of model parameters to allow users to create various levels of pandemic severity in order to assess various situations. Using this tool, health officials could analyze resources and decision outcomes in order to prepare more effective measures for potential pandemics. This work was expanded upon in subsequent iterations to model the spread of Rift Valley Fever and Dengue Fever and predict successful public health containment procedures. PanViz is not the only VA tool developed for tracking and predicting the spread of pandemics; however, it is one of the only ones that focuses on multiple levels of preparedness and mitigation.

A limitation of PanViz is that it was not initially designed to update with incoming surveillance data. Moreover, it does not accommodate behavioral or social science data. Understanding behavioral data is beneficial to an analyst because it permits a deeper understanding of how the public views, interprets, and responds to information about the pandemic, enabling a decision-maker to tailor public announcements about the need to close schools or engage in social distancing while gauging how the public perceives the burden. To address this gap, members of our team are currently engaged in a continuously updated behavioral monitoring survey, focused on COVID-19 related beliefs and behaviors in the U.S. (NSF RAPID Grant 2026763), as well as analysis of Twitter data related to COVID-19. We are also collaborating with researchers at Purdue University, University of West Florida, and Arizona State University to explore social mobility data for more effective measures of the impact of NPIs (NSF RAPID grant 2027524). These data will be leveraged to permit analysis of how changing perceptions and behaviors among the public affect later infection rates across regions of the US. The results will be incorporated into PanViz 2.0, allowing for a blending of more traditional Susceptible, Infected and Recovered (SIR) infection spread models, data-driven models, and agent-based models with up-to-date perception, attitudinal, and effectiveness models.

PANVIZ 2.0 OVERVIEW

The decision making in the context of combating COVID-19 or similar future pandemics requires a workflow to transform raw data into actionable information including statistics, visualization, and interaction from a variety of sources. The workflow employed by PanViz2.0 was inspired in part by our previous research developing and evaluating novel VA applications. Below, we introduce the reader to the PanViz 2.0 interface and computational architecture while providing a road map for planned improvements.

PanViz 2.0 Design

Our current decision-support framework system builds upon our earlier PanViz work in public health syndromic surveillance, pandemic preparedness, and decision support for other person-spread and mosquito-spread conditions. PanViz was used extensively during the 2008–2012 pandemic preparedness activities in the United States by numerous counties and states. The PanViz 2.0 visual analytic framework prototype is a re-engineered design based on our experience in developing and deploying visual analytic decision support systems over the past decade. The prototype, shown in Figure 1, is built upon a mathematical epidemic model to calculate population dynamics and infection rate data and enables decision-makers to interactively choose mitigation strategies and see the impact of their decisions. PanVis 2.0 improves upon PanViz in several respects, including the system architecture, ability to incorporate behavioral, social movement and dynamics, and observed data for improving accuracy and predictions in the system, and adds interactive decision-making features. In concert, these changes facilitate data- and human-guided, science-based AI-driven analysis of public health data for improved preparedness.

Figure 1.

Overview of new PanViz 2.0: (left) control panel for fine-tune of model parameters and configuration of visual representations, (middle) spatial distribution of a selected case category, (right) temporal visualization of different case categories.

PanViz 2.0 System Architecture and Interface

PanViz 2.0 has been converted from a desktop system to an intuitive web-based application to address the portability and scalability problems, as multiple users can now access the system from any web browser. It consists of a backend Flask server and frontend web user interface. The server dynamically executes the model simulation for a given set of parameters (as controlled and updated by the user). The web interface (see Figure 1) uses the React JavaScript library for efficient frontend rendering and interactivity. Users can modify the model features, parameters, and parameters to be visualized (first panel on left-hand side of Figure 1), interactively visualize the model simulation over time in the geographic space (center panel of Figure 1), and simultaneously visualize time series data of county-, state-, and national-level infection, death, and hospitalization numbers (panels on right-hand side of Figure 1).

PanViz 2.0 also supports county-level disease parameterization. Users can configure model parameters for each county to appropriately account for locally dependent transmission dynamics, such as the demographic impact, spread rate (as controlled by population density), mortality rate, implemented decision measures, and hospital capacity. We plan to incorporate sophisticated machine learning and data mining techniques to learn highly accurate county-level parameters from collected data for improved decision-making.

Base Epidemiological Model

The mathematical model underpinning PanViz (Malone et al.) calculates disease dynamics per county using a system of nonlinear difference equations derived from traditional epidemic models with homogeneous population mixing derived from previous influenza and pandemic data and integrates an airport transportation spread model.

Disease dynamics are evaluated by combining user-supplied values for county demographics and population density, mortality and recovery rate of the disease, hospitalization rate, and baseline and modified disease prevalence. The baseline prevalence is approximated using the gross attack rate, which is the percentage of the entire U.S. population that will have the disease if no interventions are enacted. The total number of infections in a county is calculated by age group and is the product of county population, age group specific disease modifiers, and the presence of decision measures. Simulated individuals can be either healthy, infected, recovered, or deceased and will transition between these states at a certain rate. The likelihood an individual from any age category will become infected is influenced by the population density of the county they reside in and the baseline and modified prevalence of the disease. For more information on the formulation of the model, please see the paper by Maciejewski et al. and Malone et al.

The model assumes a disease originates from a user-defined location. The county to county spread rate is dependent on the distance from the origin to the county centroid, the county population density, and demographic composition. The inclusion of the airport transportation dynamics enables the transmission within a day of the disease to all connected airport hubs once the disease reaches an airport.

Time-Based Interdiction for Interactive Decision-Making

Decision measures are critical for impeding virus spread, although such measures may vary considerably over time and space. Comparative analysis of such decision measures is equally crucial for assessing how effective and suitable they may be for different situations. In particular, it is important to answer policy questions such as “What decision measures should be implemented?” PanViz 2.0 will allow users to answer such questions by visually comparing the effect that different decision measures have on virus spread, as well as infection, mortality, and hospitalization rates (see Figures 2 and 3). When the exact parameters of such interdiction strategies are difficult to estimate a priori or from previous pandemic data due to transmission novelty, users will be able to explore different estimates, such as in Figure 4, and update them as new data are collected.

Figure 2.

Mockup of the next generation of the PanViz 2.0 UI. Users will be able to visualize the impacts of different decision measures and what-if scenarios on infection, death, and hospitalization counts, compare epidemiological model data with observed data, and estimate future predictions from observed data with interactive machine learning. These views will be tightly linked such that users can input different interdiction strategies and parameter combinations and visualize the resulting model data in other views.

Figure 3.

Users can compare the cumulative effects (here in terms of lost and saved lives) of various decision measures against the baseline simulation to assess their effectiveness. Example result shown is from previous work with Rift Valley Fever.

Figure 4.

User-configurable parameter window. Users can set the appropriate parameters for each county.

Public officials may be interested in determining when and how long citizens should be required to wear masks, practice social distancing, and avoid public gatherings to effectively curtail the spread of COVID-19. Officials can investigate their concerns with PanViz 2.0 by comparing the virus spread and number of infections when these intervention measures are implemented early on in the pandemic as opposed to later. Officials can also assess these measures’ effectiveness under environmental assumptions such as the probability of an individual complying with mask wearing or the continued occurrence of large social gatherings. Using such results, officials can appropriately determine the best course of action for intervention policies. The system will also enable the starting and stopping of different measures over time during each wave of virus spread.

Incorporating Epidemiological Expertise

We have been working with state and local public health officials to integrate their feedback, domain expertise, and perspectives into the design and implementation of PanViz 2.0. Since portions of the team have worked with public health professionals in the past, we recognize the need to closely collaborate with and include domain expert opinions to ensure the PanViz 2.0 interface and application components support effective and actionable decision-making.

Planned Work: PanViz 2.0 for COVID-19

Due to the novelty of COVID-19, traditional epidemiological models may fail to accurately simulate the virus spread. Current epidemiology studies suggest that the virus spreads from person to person just as any other virus spreads, but with potentially different parameters for when symptoms start, time when contagious, etc. This problem is exacerbated for lower population regions and counties with limited case histories or areas with unique characteristics that small towns with large university student populations. At the same time, these areas are also the least prepared for an onslaught of COVID-19 cases.

Regional hospitals serving such areas need to evaluate how best to utilize their limited budgets and resources, to meet the upcoming demand; however, access to good information to support such decisions is poor. Therefore, PanViz 2.0 will ingest collected data for data-driven model comparisons against baseline simulations. In particular, users will be able to visualize the differences between baseline simulations and observed COVID-19 data to explore and infer model parameters and adjust the model's settings for future predictions (see Figure 2). PanViz 2.0 will also incorporate interactive machine learning for user-steerable parameter learning to improve model predictions. Multiple model simulations with different parameters will be used to train a neural network to predict the corresponding parameters after human-review for appropriateness. The trained model can then be used to predict parameters for real-life data that can be further tuned and adapted through interactive user analysis. Our previous work in syndromic surveillance to more accurately analyze surveillance data, reducing anomaly false positives while modeling and predicting incident occurrence in the upcoming 14 days, will be incorporated as well to harness trustable data-driven predictions for interdiction planning.

While past data can be useful for tuning the model, there is still a degree of uncertainty regarding future trends. Therefore, PanViz 2.0 will also support sophisticated exploration of what-if scenarios by allowing users to select different parameter combinations and visually compare them (shown in Figure 2). For example, the user may be interested in determining how the virus spread changes under different spread rates, hospital capacities, and demographic impacts. Finally, we are in the implementation and refining phase of both the visual design and technical implementation of PanViz 2.0.

PANVIZ 2.0 AND SOCIAL MEDIA DATA

Since early 2020, part of our team at the University of Oklahoma National Institute for Risk and Resilience (NIRR) has pursued several projects focused on the COVID-19 pandemic. In January, they implemented a broad collection of social media posts from Twitter's API using a basket of search terms. That collection includes approximately 300 million posts, amounting to half a TB of data. These data are used to identify the evolving array of COVID-19 communication networks using the Louvain community detection algorithm on the largest connected component of the retweet network drawn from the most prolific accounts (minimum three tweets per day) to prune the network. Weekly samples are drawn from the top tweets (according to PageRank scores) within each cluster and categorized by human coders to track the content of social narratives.

As shown in Figure 5, the constellation of network clusters has been quite stable, with seven or eight groupings consisting of relatively consistent leading members. The resulting weekly network snapshots reveal a polarized network structure that reflects current divisions in American politics. Interestingly, the political right displays a remarkable level of stability while the left has exhibited more dynamic cluster with the larger moderate left gradually incorporating a smaller progressive group over time. A common characteristic of these network communities is the presence of dense clusters of users around popular politicians and media figures with relatively short communication pathways that enable the rapid transmission of information.

Figure 5.

Depicts the networks of Twitter users posting about COVID-19 over the period of February 27th through April 16th, 2020. Clusters were identified using the Louvain community detection algorithm.

Of particular interest are patterns of misinformation about the nature, transmission, effects, and protective actions associated with COVID-19 within each of the more stable network clusters. Preliminary analysis of the network structure and content of the most prominent accounts and tweets has found a number of coherent misinformation narratives, including conspiracy theories, cures, and other statements about COVID-19 that are based on verifiably false claims, that have spread throughout the social media landscape. These narratives include claims that COVID-19 is the product of a shadowy conspiracy of powerful individuals, the virus originated as a bioweapon, COVID-19 is no worse than seasonal flu, and hydroxychloroquine is an effective cure. Among the false narratives identified, a disproportionate number of these misleading claims regularly have appeared within the conservative right community on social media, but it remains to be seen if this flow of misinformation will shift.

Starting in mid-March, a rolling nationwide survey, with weekly representative subsamples, was implemented to track the patterns of awareness of and belief about the COVID-19 misinformation identified in the Twitter social media collection. The survey permits assessment of the ways in which social media misinformation (and efforts to counter that misinformation) affect evolving public concern about and response to the pandemic. A key interest is in understanding changes over time in public trust for experts and individual willingness to engage in protective actions.

Also in March, Governor Kevin Stitt of Oklahoma asked the OU NIRR, as part of a team of modelers, to provide regular updates on the projected spread of COVID-19 related hospitalizations and ICU demand within Oklahoma. Early in the pandemic, models were showing wide discrepancies in these estimates, with underlying uncertainties clouding urgent policy decisions. The NIRR proposed and implemented a modeling ensemble approach, utilizing a range of nationally recognized models to provide the Governor, the State Epidemiologist, and cabinet with the range of regularly updated projections for the state. This effort extended through May 2020, when the pandemic (appeared to have) peaked in Oklahoma. These ensemble models are being integrated into PanViz 2.0.

PANVIZ 2.0 AND MOBILITY DATA

One question implied by the current work is: To answer this question, in addition to our work in surveying, social media data collection, and smart app user-provided data, we and our collaborators repurposed existing cyberinfrastructures to analyze the risk of future epidemics in crowded locations by using real-time webcam videos and data provided by location-based services (LBS) (NSF RAPID grant 2027524). The proposed solution incorporates pedestrian dynamics to assess if individuals are complying with social distancing.

How does a decision-maker assess if individuals are complying with public health measures such as social distancing?

We believe this research will yield actionable data that can be incorporated into PanViz 2.0 to asses compliance with NPIs, since LBS data can be used to identify crowded locations with a spatial resolution in the tens of meters, and video data can assess how individuals congregate within and move through large public spaces.

From a decision-making perspective, there are several benefits of this data. The first is the ability to identify potential transmission hotspots on a very-fine grain level, allowing a decision-maker to assess if a particular store or nursing home is at risk of becoming a hotspot. Second, the collection and analysis of LBS and video data can be used in contract tracing applications. Looking toward the future, this information can be used to redesign public spaces for pandemic safety.

These data sources, along with social media data and user-provided app data, have been integrated with the PanViz 2.0 architecture to provide a decision-maker with a series of scaling intervention options. A decision-maker can now suggest a series of behavioral “nudges” be issued to individuals in high-risk areas to remind them to comply with public health practices or simply alter them of the risk. On a regional level, the decision-maker could assess public perception of and compliance with regional level public health measures. Finally, at a national level, a decision-maker can surmise how the public at large views public health measures and asses the types of locations at risk of becoming a hotspot.

PANVIZ 2.0 AND CO-ADVISOR

Gathering detailed information at the individual level and providing individualized communications to help ordinary people respond are also goals of our Center. Part of our team, based in OU's School of Computer Science, has prototyped the Co-Advisor application for this purpose. Co-Advisor is a smart-device app designed to ingest data on the activities of users and others, as well as individual health data, in order to provide users with current and predicted risk assessments based on their current and planned behaviors. The app is designed to communicate risk assessments in a clear, simple, and timely manner. Co-Advisor ingests data in three categories: user activities, user health data, and activities of others. The types of data collected about individual activities include mobility, records of personal behaviors such as mask wearing, and social behaviors such as work environment. An individual user can supply records of their symptoms and/or the system can automatically detect them. Finally, Co-Advisor ingests data about the behaviors of others that users have come in contact with, such as whether a visited location was crowded. These data are collected by sensors including Bluetooth and GPS tracking, and user inputs. One of the primary functions of Co-Advisor is to convert the collected data into actionable information. An example of this can be viewed as a user story. If a user wants to engage in contract tracing, the user enables location sharing (while preserving privacy and security) to help assess where they had been during the past 20 days. This information allows Co-Advisor to determine if they had come in contact with others who were likely to have COVID-19. Assuming the user had been in substantial contact with an infected individual, Co-Advisor informs them of that risk. Effective risk communication is clear, simple, timely, and incorporates awareness of the cultural context of the user. Co-Advisor uses interactive communication strategies in lay language with supporting evidence to make biomedical prevention messages credible in affected communities. This risk is communicated via alerts triggered by public or private data; nudges, which encourage individuals to change their behavior and can be used as just-in-time adaptive interventions; and detailed recommendations based on user behavior.

PLAN FOR A CENTER CIPH-MAD

As mentioned in the introduction, we plan to form a Southwestern U.S. collaboration to enable more effective surveillance, planning, mitigation, and response to public health emergencies. To state, we have initiated a Center for Integrated Public Health Monitoring, Analysis and Decision-making (CIPH-MAD). This Center will pursue opportunities for generating and harnessing new data sources to improve planning, detection, response, communication, and management for pandemics like COVID-19 (see Figure 6). Currently, there is no deployed system that integrates these data and capabilities into a unified decision-making system for hospital and government decision-makers and includes mitigation strategy planning. While large, population dense areas see more infectious disease cases, they also have greater resources. Oklahoma's budget per capita is only 52.6% of New York's. Accurate disease projections are potentially “mission critical” for Oklahoma and other Southwestern states given this relative shortfall of resources. Moreover, Oklahoma and many other states lack real-time electronic syndromic surveillance to provide the base data needed for accurate situational surveillance, virus spread status, and measurement of mitigation actions.

Figure 6.

Configuration of CIPH-MAD, which facilitates public health emergency planning and response through integrated data sources (monitoring), advanced modeling (analysis), and data visualization and risk alert communications (decision-making).

Included in the Center will be development and deployment of a “Co-Advisor” app that would build on the functionality the Google/Apple social-distancing/contact-tracing capabilities to provide user advice and information while providing anonymized input to CIPH-MAD's integrated public health planning system, built upon PanViz 2.0. These data, as well as the social media and survey data described above, will be utilized to expand the capability and functionality of existing pandemic models (like already developed models in PanViz). Furthermore, we will create the PanViz 2.0 decision support framework that is sensitive to the community-specific characteristics and/or specialized subpopulations. We will accomplish this in part through machine learning to create datasets amenable to statistical analyses for forecasting, incorporating features such as population density, income dispersion, age distribution, mobility data, and unique factors such as Native American population and presence of higher education facilities. Small area estimation techniques including missing data imputation for survey data and Bayesian estimation will enhance the informational value of datasets and improve forecasting accuracy. The overall focus is on capturing the synergies across the computing, public health and social sciences to build a data-driven, integrated healthcare modeling system to both act as a sentry for emerging pandemics and as a tool for managing them.

In the coming months, we will expand CIPH-MAD to include researchers from Arizona State University and the University of Texas, Austin. This will expand the reach of the center to cover the South Western portion of the United States.

CONCLUSION

Making timely, effective, science-based decisions to mitigate the impact of a pandemic is a very difficult and highly complex task. As recent news events have shown, the pressures facing public health officials are immense and life-altering for thousands of individuals. Even a decision as seemingly small as when to announce a particular policy can save the lives of tens of thousands of individuals.

However, making these complicated decisions without computational support can be very difficult. In this article, we presented the initial results of a trans-disciplinary collaboration between researchers, healthcare practitioners, and community health partners in the Southwestern U.S. to help enable improved management, response, and recovery to our current pandemic and for future health emergencies through the development of an integrated data dashboard. Our aim is to provide decision-makers with a tool to help them synthesize and visualize a wide variety of data types—ranging from hospital capacity to Facebook posts—to help them evaluate the outcomes of various decisions to further their goal of making life-saving choices. Given that the data landscape surrounding COVID-19 are evolving so fast and the little understood nature of the disease, we emphasize that work will continue for quite some time. Our work to date does facilitate an understanding of the consequences of various interventions. Furthermore, our work is imminently applicable to future pandemic and public health emergencies by providing a framework to integrate and synthesize multiple disparate data sources.