Chatbot use cases in the Covid-19 public health response.

OBJECTIVE: To identify chatbot use cases deployed for public health response activities during the Covid-19 pandemic.
MATERIAL AND METHODS: We searched PubMed/MEDLINE, Web of Knowledge, and Google Scholar in October 2020 and performed a follow-up search in July 2021. We screened articles based on their abstracts and keywords in their text, reviewed potentially relevant articles, and screened their references to (a) assess whether the article met inclusion criteria and (b) identify additional articles. Chatbots, their use cases, and chatbot design characteristics were extracted from the articles and information from other sources and by accessing those chatbots that were publicly accessible.
RESULTS: Our search returned 3334 articles, 61 articles met our inclusion criteria, and 61 chatbots deployed in 30 countries were identified. We categorized chatbots based on their public health response use case(s) and design. Six categories of public health response use cases emerged comprising 15 distinct use cases: risk assessment, information dissemination, surveillance, post-Covid eligibility screening, distributed coordination, and vaccine scheduler. Design-wise, chatbots were relatively simple, implemented using decision-tree structures and predetermined response options, and focused on a narrow set of simple tasks, presumably due to need for quick deployment.
CONCLUSION: Chatbots' scalability, wide accessibility, ease of use, and fast information dissemination provide complementary functionality that augments public health workers in public health response activities, addressing capacity constraints, social distancing requirements, and misinformation. Additional use cases, more sophisticated chatbot designs, and opportunities for synergies in chatbot development should be explored.

INTRODUCTION

Covid-19 imposed significant demands on healthcare systems, making effective use of available resources and capacity expansion vital. The social distancing requirements to slow virus transmission imposed further restrictions on using traditional in-person services and required solutions that accommodated social distancing. Furthermore, the fast-shifting landscape of information about the virus that included reversal of policies, and the rapid diffusion of disinformation and misinformation created anxiety and confusion. This resulted in a surge in the number of calls to helplines.

One response to these issues involved the deployment of chatbots as a scalable, easy to use, quick to deploy, social-distanced solution. Chatbots are algorithms that interact with users using natural language, either text or voice,, with their distinguishing feature being natural language conversational interactions. Given the use of chatbots in a variety of settings prior to Covid-19, the existing infrastructure and abundance of prebuilt modules resulted in their rapid development and deployment to address Covid-19 needs. In addition, off-the-shelf Covid-19 chatbot solutions quickly became available. These offer a wide range of functionality that can be easily customized for each organization, including risk assessment of patients and employees, information dissemination by answering common questions, surveillance, and integration with existing infrastructure.

We systematically searched the literature to identify chatbots deployed in the Covid-19 public health response. We gathered information on these to (a) derive a comprehensive set of chatbot public health response use cases and (b) identify their design characteristics. Our study adds to an emerging literature on the use of chatbots in healthcare in general (see Car et al. and Montenegro et al. for reviews), and in the Covid-19 response in particular (see Almalki and Azeez for a review).

METHODOLOGY

We conducted a scoping review of the scientific literature to identify studies (journal articles, conference proceedings, and preprints) reporting or proposing the use of chatbots in the Covid-19 public health response. We limited our search to articles in English published before October 10, 2020 and conducted a follow-up search on July 29, 2021. Our search terms included all combinations of terms for “chatbots” (e.g., conversational agents) and Covid-19. Figure 1 presents our search terms and summarizes the article selection process which is based on the Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA) guidelines.

Figure 1.

Flow chart illustrating process of article selection and review.

We searched 3 sources (PubMed/MEDLINE, Web of Knowledge, Google Scholar) and engaged in a 2-stage screening process to identify relevant articles. First, we reviewed the title and abstract of articles matching our search terms to identify papers that met the minimum inclusion criteria. Second, we searched the papers’ relevant citations using the same process.

We used the following inclusion criteria: (a) the paper focused on the implementation, use, or potential use of a chatbot and (b) was used in the Covid-19 public health response. We excluded articles where (a) chatbots were tangential to the paper, such as papers discussing use of AI or telemedicine during the pandemic, (b) only included a general discussion of chatbot benefits without referring to a specific chatbot use case, or (c) chatbots were used for other purposes during the pandemic (e.g., domestic abuse victims support,). Our search yielded 57 academic articles and 50 unique implemented chatbots. We excluded proposed chatbots from our sample but discuss additional use cases identified from these articles in our discussion section. We then used Google search to gather additional information on each of the 50 chatbots. An additional 17 chatbots were incidentally identified in this search (e.g., a news article mentioning the chatbot we had identified mentioned additional chatbots). We included these as our objective was to identify a comprehensive set of chatbot public health response use cases.

Our data collection was supplemented by accessing these chatbots to gather more information about their design and use. We gained access to 47 out of the 67 chatbots. For chatbots not conversing in English, we used Google Translate to understand the interaction. We could not access chatbots that required organizational credentials, customer or patient accounts, local phone numbers (except for the USA), or national identification numbers for access. Instead, we relied on the literature and news articles for information on these. Therefore, our analysis of design characteristics has an overrepresentation of publicly accessible chatbots. This does not influence our use cases since chatbot objectives were described in the articles. We excluded 9 cases from our sample since our analysis revealed that they were not chatbots. We performed another search on July 29, 2021 for any additional use cases. We identified 3 new chatbots that focused on vaccination, bringing our final sample to 61 chatbots and resulting in 1 additional use-case category and 1 new use case.

RESULTS

The 61 chatbots reflect a global sample of chatbots deployed in more than 30 countries. These include 33 chatbots that conversed in 45 languages other than (or in addition to) English. Tables 1 and 2 in Appendix 1 provide background information on each chatbot, its use cases, and design features. References to case numbers below refer to the corresponding chatbots in Appendix 1.

Table 1.

Chatbot use cases and definitions

Use-case category and associated use cases		Use-case description	Benefits
Risk assessment		Triage users based on their Covid-19 symptoms and exposure risk and recommend a course of action.	Social distancing, capacity expansion, efficient capacity utilization, prevent virus transmission
Surveillance	Disease surveillance	Gather data about disease occurrence and prevalence to assist in planning public health response policies and interventions	Assess state of virus transmission, predict clusters and outbreaks, inform policy and interventions, enable quick response
	Public reporting	Enable public to report Covid-19 public health concerns such as rumors, misinformation, and violations	Real-time monitoring, identification of public health concerns, inform policy and interventions, prompt investigation of violations
	Covid-19 symptoms surveillance	Collect data on Covid-19 symptoms experienced by those who have or had the virus	Geographical and temporal comparison of symptoms
	Proactive monitoring	Proactive monitoring of at-risk populations (e.g., the elderly) by initiating interactions to check whether users are experiencing symptoms or have been exposed to the virus	Enable social distancing, real-time monitoring, early detection of virus, enable prompt response, prevent virus transmission
Information dissemination	Virus and vaccine education	Provide information about Covid-19 symptoms, virus transmission, and Covid-19 vaccines. Many chatbots added vaccine education; others solely provided vaccine education	Capacity expansion, efficient capacity utilization, reliable information, educate public, reduce vaccine hesitancy
	Misinformation/disinformation debunking	Factcheck prevalent rumors and debunk misinformation and disinformation by providing reliable information. Fight spread of misinformation and disinformation regarding Covid-19 and, more recently, vaccines	Provide reliable information, educate public, factcheck rumors, address misinformation and disinformation, reduce vaccine hesitancy
	Proactive misinformation/disinformation debunking	Monitor for misinformation and disinformation and take counter measures by alerting users	Provide reliable information, educate public, address misinformation and disinformation in real time, prevent its further spread, enable prompt response
	Nonpharmaceutical interventions (NPI) promotion	Provide information about NPI interventions such as frequent hand washing, mask wearing, social distancing	Prevent transmission of virus, educate public
	Virus transmission data reporting	Provide information about the state of virus transmission in a geographical location (e.g., worldwide, country, state, or local)	Educate public, reduce load on helplines
	Available public resources awareness	Inform users about resources available to the public such as reliable official information sources	Educate public, capacity expansion, reduce load on helplines
	Encouragement of activities (other than NPIs) to fight the pandemic	Provide information and encourage users to volunteer for activities to help fight the pandemic such as encouraging recovered patients to donate plasma	Educate public on needs and how they can help, recruit volunteers, capacity and reach expansion
Post-Covid-19 eligibility screening		Screen recovered Covid-19 patients for eligible volunteers for an activity such as plasma donation	Capacity expansion, efficient capacity utilization
Distributed coordination		Help with coordination and communication by providing a 2-way communication channel between frontline healthcare workers and health organizations	Real-time data gathering, enable provision of essential support to frontline workers, improve supply chain efficiency, inform policy, enable quick response, compensate for weak technology infrastructure in less developed countries
Vaccine scheduler		Assist in finding available vaccine appointments and making the appointment, automating the process	Capacity expansion, efficient capacity utilization, ease of finding and making appointments, speed-up vaccination process

Table 2.

Information dissemination chatbot use-case combinations

Number of information dissemination chatbot cases Displaying this combinationa	Virus and vaccine education	Misinformation and disinformation debunking	Proactive misinformation and disinformation debunking	Nonpharmaceutical interventions promotion	Virus transmission data reporting	Available public resources awareness	Encouragement of activities (other than NPIs) to fight the pandemic
4	X
3				X		X
3	X			X	X	X
3	X	X		X	X	X
2				X
2					X
2						X
2						X	X
2	X	X		X
2	X	X		X	X
2	X	X		X	X		X
2	X					X
1			X
1		X		X		X
1		X		X	X	X
1	X			X
1	X			X		X
1	X			X	X
1	X	X			X	X
1		X
No. of combinations presentb	10	7	0	11	7	9	2

Column 1 shows the number of chatbots for each combination of information dissemination use cases.

Last row shows number of different combinations in which the use case is present (excluding single purpose).

We categorized these chatbots based on (a) their use case which reflects the public health response activity they supported and (b) their design characteristics. We used qualitative methods to allow our use cases and use-case categories to emerge from our data. Specifically, both authors engaged in open coding (see Miles and Huberman) where we identified the public health response activities that the chatbots supported. We iterated between our data and our codes to identify use cases. We then engaged in axial coding to aggregate use cases into use-case categories. Finally, we independently categorized the chatbots based on their use case(s) and design features. We resolved differences through discussion. We were unable to assess some chatbots on some attributes because of variations in available information. Further, as a chatbot could belong to multiple categories (e.g., delivered multiple use cases), our numbers do not always add up to 61.

Chatbot use cases

We identified 6 use-case categories that captured how chatbots were deployed in the Covid-19 public health response: risk assessment (34 cases), disease surveillance (12 cases), information dissemination (42 cases), post-Covid eligibility screening (2 cases), distributed coordination (1 case), and vaccine scheduler (1 case). These 6 categories consist of 15 distinct chatbot public health response use cases described in Table 1.

Risk-assessment chatbots

Risk-assessment chatbots (N = 34) triage users based on their Covid-19 symptoms and exposure risk and make behavioral and clinical (e.g., go to emergency room) recommendations. They enable social distancing by reducing footfall in healthcare facilities, which results in capacity expansion and better capacity utilization of the healthcare system. Such chatbots have been developed by countries to help reduce the burden on their healthcare systems and by healthcare providers and institutions for their patients, customers, students, or employees.,

Risk-assessment chatbots ask a series of questions to triage users. While many chatbots leverage risk-assessment criteria from official sources (WHO, CDC, or other government health agency), the questions asked vary significantly across chatbots, and as does the order in which they are asked. Some ask general questions about exposure and symptoms (e.g., Case 7), whereas others also check for preexisting conditions to assess high-risk users (e.g., Case 1). Based on the assessed risk, the chatbot makes behavioral recommendations (e.g., self-monitor, quarantine, etc.). In cases of Covid-19 exposure combined with symptoms, recommendations across chatbots vary. Chatbots from healthcare facilities provide links to establish a video call or make an appointment or to initiate a telemedicine session (e.g., Case 4). Other chatbots ask users to call an emergency number or their physician and provide links to official resources (e.g., Case 5). Two chatbots direct users to another chatbot for a more detailed screening (Cases 8 and 29). Although not claiming to diagnose, a few chatbots also try to eliminate differential diagnoses by asking more detailed questions (e.g., Case 41).

The chatbots also differ in how they present their assessment. Most present the user’s exposure risk and next steps. A smaller group (3 cases) provides a report and explains the reasons behind their recommendation (Cases 15, 22, and 36).

Data collected by some chatbots are used for disease surveillance (7 cases), which is a separate use case. They do so by periodically checking with users for symptoms or Covid-19 exposure. The joint use of risk assessment and disease surveillance use cases is a prevalent combination for chatbots by organizations and schools which monitor disease among their employees or students (3 of 5 cases).,,

Surveillance chatbots

We identified 4 use cases for surveillance activities. The first use case, disease surveillance (7 cases), involves gathering disease prevalence data which enables early assessment of virus transmission and prediction of clusters and outbreaks. Early detection allows officials to formulate a prompt and appropriate response, including the nature and timing of nonpharmaceutical interventions such as lockdowns and travel restrictions. Some chatbots also collect concerns about the pandemic from the public to inform policy, which illustrates a second surveillance use case—public reporting (3 cases). For example, some allow users to report rumors and misinformation (e.g., Case 57) which enables officials to quickly respond to curb misinformation. Another (Case 19) also allows users to report guideline violations, which enables prompt investigation and violations reduction.

The third use case, Covid-19 symptoms surveillance (2 cases), collects data on Covid-19 symptoms experienced by those who have or had the virus. This enables the mapping of the multitude of Covid-19 symptoms, their frequency, and the sequence in which they manifest. Cross-location comparison of symptoms and sequences allows assessment of how the virus affects different communities.,

The final use case, proactive monitoring (3 cases), involves proactively monitoring at-risk populations, such as the elderly, by checking whether users are experiencing symptoms or have been exposed to the virus. Unlike disease surveillance chatbots where the user initiates the interaction, these chatbots initiate contact with the users and ask questions about symptoms.

Information dissemination chatbots

At the onset of the pandemic little was known about Covid-19 and information and guidelines were in constant flux. The public had many questions and concerns regarding the virus which overwhelmed health providers and helplines. As a response, chatbots were deployed to provide current and reliable information and to educate the public about the coronavirus, its symptoms, transmission, and more recently, vaccines; to fight misinformation and disinformation (passively or proactively); to promote nonpharmaceutical interventions to slow transmission (e.g., social distancing, wearing masks); to inform the public about the state of the virus at local and national levels; and to provide information regarding available resources including vaccinations. We were able to assess the type of information provided for 37 of the 42 information dissemination chatbots (see Table 2 in Appendix 1). Based on the information they provided, we identified 7 use cases for information dissemination (see Figure 2).

Figure 2.

Information dissemination chatbot use cases.

Many chatbots provided multiple types of information (26 cases). Education and information about nonpharmaceutical interventions was the most common combination with 15 chatbots providing both, but as Table 2 shows, chatbots exhibited a wide range of configurations of information dissemination use cases.

Post-Covid eligibility screening chatbots

A different screening use-case category involves screening recovered Covid-19 patients for eligible volunteers for an activity such as plasma donation (Cases 2 and 9). These chatbots first ask if the user has been diagnosed with Covid-19 and check if they have been symptom-free long enough to be eligible for the activity.

Distributed coordination

An interesting use case—mHero,—involves facilitating coordination between distributed frontline healthcare workers and health organizations or the Ministry of Health in areas of poor technology infrastructure (1 case). The chatbot can gather real-time data from frontline workers to enable provision of essential support, answer their questions, and provide them with real-time information. Originally developed in response to the Ebola outbreak to reach frontline workers with basic text and audio messages, it can now also be implemented in WhatsApp and Facebook messenger. Its impact on fighting the pandemic in countries with less developed technology and healthcare infrastructures makes this an important use case, and unlike our other use cases that are public-facing, this illustrates how chatbots can be used to support frontline healthcare worker needs.

Vaccine scheduler

Chatbots were also used for scheduling vaccine appointments (1 case). The chatbot searches for appointment availability across various locations and automates the appointment scheduling process. This enables more efficient utilization of available vaccines, reduces wait times in vaccine centers, and allows users to easily find available appointments.

Target audience for use cases

Chatbots were designed either for the general population (35 cases) or for a specific population (17 cases). The general population audience could be as broad as the world (e.g., the WHO chatbot) or a country (e.g., the CDC chatbot in the United States). Many state or regional governments also developed their own chatbots; for instance, Spain has 9 different chatbots for different regions. Specific population audiences were more diverse. Some focused on at-risk groups. For instance, in Brazil, a chatbot designed to support LGBT teenagers added Covid-19 information. Others were developed by healthcare institutions and a diverse range of organizations for their customers, patients, students, and employees. Many of these screen employees (and students) to assess whether they are safe to report to work, and some provide a badge to “cleared” employees so they can report to work. While the majority of the specific population chatbots are developed for organizational use, an example of a countrywide-specific audience chatbot is the mHero platform deployed in several African countries, which connects frontline healthcare workers to the ministry of health.  Table 4 shows how the chatbot target audience varies across use cases.

Table 4.

Chatbot use cases and their design features

Chatbot use case/Design features		# of Casesa	Audience		Multi -Purpose		Bilingual	Platformb					Anonymous	Interface design			Genderc
			General	Specific	Multi	Single		Web	Social Media	Standalone app	Text	Call		Text based	Voice-based	Both	Female	Male
Risk assessment		34	16	10	22	12	9	26	5	5	2	3	18	23	4	2	7	1
Surveillance	Total	12	5	5	11	1	4	5	3	4	3	2	1	5	2	1	2	0
	Disease surveillance	7	2	3	7	0	3	4	1	2	2	2	1	3	1	1	1	0
	Public reporting	3	3	0	3	0	1	0	2	1	1	0	0	1	0	0	0	0
	Covid-19 symptoms surveillance	2	0	1	1	1	1	2	0	2	0	0	1	2	0	0	1	0
	Proactive monitoring	3	1	2	3	0	1	0	1	0	0	1	0	0	2	0	1	0
Information dissemination	Total	42	26	10	21	21	14	19	20	6	4	2	13	27	2	4	6	4
	Virus and vaccine education	22	15	4	11	11	7	11	12	2	2	0	6	15	1	2	4	1
	Misinformation and disinformation debunking	13	13	0	6	7	7	5	10	0	1	0	4	11	0	0	1	2
	Proactive misinformation and disinformation debunking	1	0	1	0	1	0	0	1	0	0	0	0	1	0	0	1	0
	Nonpharmaceutical interventions (NPI) promotion	22	16	4	12	10	9	12	11	2	3	0	7	17	0	0	1	2
	Virus transmission data reporting	15	13	1	6	9	6	5	10	2	2	0	3	11	0	0	0	2
	Available public resources awareness	19	11	5	8	11	6	8	9	2	2	0	7	11	0	2	1	1
	Encouragement of activities (other than NPIs) to fight the pandemic	4	3	1	2	2	2	4	1	0	0	0	3	3	0	1	0	0
Post-Covid-19 eligibility screening		2	1	1	2	0	2	1	0	1	0	0	0	1	0	0	0	0
Distributed coordination		1	0	1	0	1	0	0	0	0	1	1	0	0	0	0	0	0
Vaccine scheduler		1	1	0	0	1	1	0	1	0	0	0	0	1	0	0	0	0
Total		61	35	17	25	36	20	34	22	8	6	4	23	42	4	4	9	5

Given that we were unable to assess all chatbots on all categories, and that some categories are not mutually exclusive, the numbers do not always add up to 61.

Several chatbots were deployed on multiple platforms.

There are also one transgender chatbot, one where gender is randomly assigned, and one where the user can choose the gender.

Chatbot design

As shown in Figure 3, the chatbots in our sample varied in their design along a number of attributes.

Figure 3.

Chatbot design characteristics.

Multipurpose versus single purpose

Thirty-six chatbots delivered use cases in a single use-case category (which we term single purpose), and 25 delivered use cases across multiple categories (which we term multipurpose). The most common single-purpose chatbots were for information dissemination (21 cases) and risk assessment (12 cases). The most common categories to be combined were risk assessment (22 cases) and information dissemination (21 cases), with the most common multipurpose chatbot combination being these 2 categories (18 co-occurrences). Five chatbots delivered use cases in 3 or more categories (see Table 3). Appendix 2 shows the chatbot use-case combinations for the 15 use cases we identified.

Table 3.

Use-case category combinations

Number of chatbot casesa	Risk assessment	Surveillance	Information dissemination	Post-Covid eligibility screening	Distributed coordination	Vaccine scheduler
21			X
12	X
1						X
1					X
1		X
13	X		X
4	X	X
4	X	X	X
2		X	X
1			X	X
1	X	X	X	X
No. of combinations presentb	4	4	5	2	0	0

Note: Shaded rows are single use-case category chatbots.

First column shows number of chatbots for each multipurpose chatbot use-case category combination.

Last row shows number of different combinations in which the use-case category is present (excluding single purpose).

Chatbot platform

Chatbots were deployed on a variety of platforms, the most common being web-based (34 cases) and social media (22 cases). Less common were SMS (6 cases), phone call (4 cases), and standalone or healthcare apps (8 cases).

Thirty chatbots were embedded within a specific organization’s platform (e.g., Case 1, Clara on the CDC’s website). Embedding a chatbot within a high-traffic platform can enhance its visibility and discoverability and reduce the effort required to engage with it.

Anonymity

Chatbots varied in the degree of privacy afforded the user. While some do not require user identifying information (23 cases), one asks the user to voluntarily disclose identifying information (e.g., email) to enable the chatbot to follow-up (Case 34), while the rest require identifying information for access (29 cases) (e.g., telephone number, national identification number, social media account, institutional credentials).

Anthropomorphism

Most chatbots (70%) lacked anthropomorphism. The most common anthropomorphic feature was gender with 9 chatbots being female, 5 male, and 1 transgender. In addition, 1 chatbot had its gender randomly assigned for each interaction (Case 22) and 1 gave the user the option to choose (Case 28).

Interface design

Most chatbots were text-based (42 cases), 4 were voice-based, and 4 had both text and voice options. Further, most chatbots (35 cases) interacted via predetermined choice and response options (i.e., use system-directed initiative), 4 used free-form questions and responses (i.e., use user-directive initiative), and 9 used a hybrid dialog style. Finally, interactions with chatbots were primarily designed to be user-initiated, with only 3 chatbots initiating conversations (Cases 29, 34, and 51).

Follow-up and recurring conversation

Only 3 chatbots were designed to initiate follow-up (Japan’s Prefecture Line chatbots (e.g., COOPERA) and CareCall), or recurring conversation (Alexa—My day for seniors skill) (Cases 34, 51, and 29). Another, at the University of California San Francisco (Case 12), sends an email or text to employees to remind them to use the chatbot to get an entry pass (e.g., prior to their shift) that clears them to report to work.

Table 4 summarizes the chatbot design features for each chatbot use-case category and chatbot use case.

DISCUSSION

Chatbots are being used as a complement to healthcare and public health workers during the pandemic to augment the public health response. Their scalability, wide accessibility, fast information dissemination, and substitution for in-person contact provide the functionality required to address the capacity expansion, social distancing requirements, and quick accurate information transmission needs of the public health response. The chatbots’ ability to automate simple, repetitive tasks and to directly communicate with users enables quick response to multiple inquiries simultaneously, directs users to resources, and guide their actions. This frees up healthcare and public health workers to deal with more critical and complicated tasks and addresses capacity bottlenecks and constraints.

Our findings extend those of a prior study on Covid-19 chatbots that, based on a review of 9 studies, identified 5 chatbot application domains: disseminating health information and knowledge, self-triage and personal risk assessment, monitoring exposure and notifications, tracking Covid-19 symptoms and health aspects, and combating misinformation and fake news. Leveraging a larger sample of 61 chatbots, we add to these findings by (a) identifying additional use-case categories (post-Covid-19 eligibility screening, distributed coordination, and vaccine scheduler); (b) identifying specific use cases based on the variability of chatbot functionality in each use-case category; (c) identifying use-case combinations; and (d) examining the chatbots’ design features.

The majority of our chatbots focused on risk assessment or information dissemination and about half involved multiple use cases, mostly combining risk assessment and information dissemination together or with other use cases. While use cases were combined in many distinct combinations, which of these are most effective is an open question.

A few observations from our results suggest future opportunities. Most risk assessment and disease surveillance chatbots did not follow-up on symptomatic users. Privacy concerns and regulations may have precluded this since following up requires that chatbots capture identifying information. Further, while surveillance could provide insights into the state of the pandemic, fewer chatbots were deployed for surveillance, and all but one combined surveillance with other use cases, suggesting that surveillance functionality was often a by-product.

Moreover, though many chatbots leveraged risk-assessment criteria from official sources (e.g., CDC), there was variability in criteria across chatbots. A comparison of symptom-checker tools indicated great variability in effectiveness in terms of their sensitivity and specificity, with some outperforming the CDC symptom-checker. Therefore, while utilizing official sources is a prudent practice, especially for off-the-shelf solutions and for non-healthcare organizations, more work is required to understand best practices.

In a rapidly changing information environment (e.g., frequently updated guidelines, guideline reversal), information dissemination chatbots face the critical challenge of keeping their knowledge base up-to-date. Automating knowledge extraction of the latest information from reliable sources to automatically create question and answer pairs and update the chatbot’s knowledge base is a viable solution to this challenge.,

With one exception, chatbots were designed for use by the public. The mHero use case for distributed coordination between frontline healthcare workers and healthcare organizations or the Ministry of Health highlights the potential for chatbots to support and gather timely information from frontline healthcare personnel. The development of multiple such use cases, including surveillance and logistics, would be especially beneficial as a frugal solution to bridge the digital divide in areas of poor infrastructure.

The provision of behavior support is another promising area for chatbot use cases. While chatbots were commonly used to provide information, there is a gap between knowledge and action that chatbots could help bridge through repetition, monitoring and interventions, and by providing real-time suggestions to users., For instance, they could automatically suggest a route to avoid walking through crowded areas. Such use cases are more sophisticated and would require the use of sensor or geolocation data.

More broadly, while the Covid-19 pandemic crisis had agencies and organizations scrambling to create their own chatbot solutions to address their needs, it may also be important to step back and understand where and how synergies and efficiencies of effort can be achieved. Differences in language even within the same country, differences in local information and guidelines, and differences in popularity of different social media platforms across countries may limit the scope of such efforts. Similarly, local development efforts may be necessary for chatbots that deliver combinations of use cases, some more universal (e.g., risk assessment) but others requiring local solutions (e.g., providing an entry pass after risk assessment (Case 12)). Nonetheless, economies of effort can occur (as we have observed) through off-the shelf solutions from vendors that organizations can customize to their needs. The 15 use cases that we have identified provide a basis for identifying functionality and customization options for different organizations or constituents. A set of guidelines and best practices to chatbot development vendors and to organizations by agencies such as the CDC can aid in coordinating efforts and in preparedness for the next pandemic.

Design-wise, ease of development, ease of access, and platform penetration have likely contributed to the prevalence of web-based and social media chatbots. For example, the built-in modules for chatbot design on social media platforms allow for fast and easy development. Coupled with the popularity of social media, this fueled deployment on social media apps such as Line, WhatsApp, Facebook Messenger, WeChat, and Telegram. Furthermore, accessibility via both smartphones and personal computers makes such chatbots widely available. Though a minority, we highlight the importance of SMS-based and phone-call-based chatbots to bridge the digital divide and reach people who lack access to smartphones or reliable internet connections or lack the skills to use technology.

Further, because of the vital need for a quick response to mitigate the surge in the healthcare system, most chatbots were developed very quickly. As such, their design is relatively simple using decision-tree structures, system-directed initiatives, and focused on a narrow set of simple tasks. More sophisticated designs based on machine learning and sensor data are fruitful directions for new or enhanced public health response use cases. Tighter integration with organizational processes may also be beneficial in some cases., Importantly, a digital divide persists among vulnerable populations worldwide. For example, tech savvy individuals can use chatbots to quickly find available vaccine appointments broadening the vaccine gap among such populations. While the chatbot conversational interface and voice modalities are well suited to bridge the digital-skill divide, increased attention to bridging the digital divide with respect to platform accessibility can help broaden use, reduce access disparities, and help ameliorate pandemic health disparities.

CONCLUSION

We identified 6 broad use-case categories and 15 use cases where chatbots were deployed in the Covid-19 public health response. Chatbots are scalable, enable social distancing, augment the capacity of healthcare and public health workers, broadly disseminate information, and gather real-time information from a broad audience to inform public health interventions. They can easily be deployed on different platforms and have easy-to-use conversational interfaces that enable broad reach and access to different demographics. Chatbots are most commonly used for information dissemination and risk assessment, which are critical to public health response. Scant attention has been devoted to chatbot deployment to support or gather data from frontline healthcare workers which is suggestive of future work. Finally, the vast majority of chatbots do not follow-up on users beyond an initial interaction or users need to initiate contact. This highlights a potential tension between privacy and functionality, and balancing these could benefit use cases where follow-up or proactive contact may be useful. Our study did not examine factors related to chatbot effectiveness across different use cases or use-case combinations, and given alternative digital health tools (e.g., form-based symptom-checkers for risk assessment), whether and under what conditions chatbots offer a comparative advantage. These are fruitful directions for research.

FUNDING

None.

AUTHOR CONTRIBUTIONS

The authors contributed equally to this work.

SUPPLEMENTARY MATERIAL

Supplementary material is available at Journal of the American Medical Informatics Association online.

CONFLICT OF INTEREST STATEMENT

None declared.

DATA AVAILABILITY

The data underlying this article are available in the article and in its online supplementary material.

Click here for additional data file.