Perceived Utilities of COVID-19 Related Chatbots in Saudi Arabia: a Cross-sectional Study.

INTRODUCTION: Health chatbots are increasingly being utilized in healthcare to combat COVID-19. However, few studies have explored the perception and willingness of end-users toward COVID-19-related chatbots. Furthermore, no studies have been conducted in Saudi Arabia. AIM: This paper explored 166 end-users' perceived utilities of health chatbots in Saudi Arabia, and how their characteristics affect their perceptions.
METHODS: We conducted a quantitative descriptive study by implementing an online survey. The survey asked 20 questions on participants' demographics and their perception of health chatbots' usefulness.
RESULTS: We found that users were more willing to use health chatbots to seek general information about COVID-19 (82.5%) over seeking information regarding COVID-19 medical treatments (72.3%). Furthermore, participants with undergraduate degrees tend to use them to learn how to prevent COVID-19's spread (P = 0.015), to self-track COVID-19 symptoms (P = 0.028), and to seek information about medication (P = 0.035) in comparison to those who had postgraduate degrees. Participants who frequently searched for health information on the internet were more likely to look for nearby medical services using health chatbots (P = 0.023). Lastly, participants who provided any sort of healthcare services information were more likely to self-assess COVID-19 symptoms by using health chatbots (P = 0.036).
CONCLUSION: Participant awareness and use of health chatbots were low; however, most had positive perceptions of these emerging technologies and displayed willingness to use them. Further research is needed to capture the real-world usability of these novel technologies by employing more rigid methodological designs (e.g, field trials).

INTRODUCTION

During the novel SARS-CoV-2 coronavirus (COVID-19) pandemic, health chatbots have been increasingly used to combat the virus’s spread (1). Health chatbots offer a form of instantaneous conversations between a human person and an artificial intelligence (AI) program, or through voice and/or text-based commands using natural human languages (2). Health chatbots can be launched in diverse platforms such as messaging mobile-based applications (apps), social media networks (e. g, Twitter), emails, and websites (3, 4). Examples of health chatbots related to COVID-19 are the Saudi Ministry of Health’s chatbot (5), the World Health Organisation’s (WHO) chatbot (6), the UK government’s chatbot (7), the VA coronavirus chatbot from US Department of Veterans Affairs (8), and India’s government chatbot (9).

Our recent scoping review (10) provided insights on how health chatbots have been used in multiple domains. This review finds that their utilization spans paediatric care, geriatric care, public health and surveillance, large scale monitoring systems, and artificial intelligence to support healthcare personnel (e. g, automate tasks like filling forms and scheduling appointments).

In addition, this scoping review (10) sheds light on the multiple applications of health chatbots for the end-users. Firstly, chatbots can be used to combat misinformation and fake news by providing a broad level of information from reliable sources to answer users’ inquiries in a timely manner. Secondly, chatbots have been utilized to disseminate health information and knowledge that caters to the needs of both current patients as well as general information seekers alike about the novel coronavirus, symptoms, medical treatments, and precautions to reduce the chance of being infected or spreading COVID-19. Thirdly, chatbots can facilitate self-triages and personal risk assessments by asking the users to complete a questionnaire about their symptoms. If the severity of the user’s symptoms reaches a pre-designated threshold, the chatbot instantly alerts them to contact their health specialist (11).

Fourthly, chatbots can track COVID-19 symptoms (e. g, fever, headache, shortness of breath) and various health aspects (e. g, physical activities, heart rate, diet) on a daily basis with ongoing symptom measurements. The users’ collected data allows the bots to categorise them into groups based on the severity of their symptoms, and–using machine learning–predict the cases who are at higher risk. Customised health advice and recommendations about self-care are then provided to the user based on the recorded symptoms. The final purpose of health chatbots concerns contact tracing and notifications of potential exposure. When the users are within close proximity of someone who has contracted the virus, they will be automatically alerted by the chat bots.

On the other hand, many challenges are reported in the literature concerning the use health chatbots. For example, some health chatbots become virtually inactive due to the lack of interest from the community (12). Moreover, perceptions about chatbots’ integrity, benevolence, and ability to provide accurate information varies based on the end-users’ demographic characteristics – often in a negative way (13). For instance, Caucasian participants perceived the chatbot to have lower ability and benevolence than Asian participants. In addition, some end-users raised concerns about their privacy as chatbots collect personal data including their names, phone numbers, and locations, and this may hinder the effective deployment of health chatbots (11).

AIM

Few studies to date have explored the perception and willingness towards COVID-19 related chatbots; however, no studies investigating this topic have been conducted within the Saudi Arabia community (10). Therefore, this study was conducted to elicit information from the end-users in Saudi Arabia regarding their perception and willingness to use health chatbots during COVID-19, and the how their individual characteristics may affect their perceptions of chatbots’ usefulness.

METHODS

Data collection design and procedures

After securing the approval of Jazan University’s Human Research Ethics Committee to conduct a quantitative descriptive study, we designed and implemented an online survey. The first page of the survey included the plain language statement and the participant consent form. The participants received information about the aims of the study, eligibility criteria, information about the kind of questions that participants would be asked during the survey, before proceeding to the first section, the candidate received the author’s contact information and was required to indicate that she or he agreed to participate. Participants were also informed that they could withdraw from the survey without repercussion before submitting the form. Completing the survey took approximately 10 minutes. A copy of the questionnaire can be found in Appendix 1.

The online survey was developed after conducting the aforementioned scoping review (10). The questionnaire was built in Microsoft Form and comprised two sections and 20 questions. Section one included 11 questions that asked for demographic information (i.e, gender, age, nationality, their highest level of education, and current occupation) and general information about participants (i.e, provided healthcare-related services, had medically diagnosed conditions such as high blood pressure and diabetes, perceived information technology (IT) skills, searched health information on the internet, awareness about health chatbot, and past health chatbot use). This section was comprised of multiple choice, closed-ended questions. Section two had 9 questions that asked participants to rate their perceptions of using health chatbots to perform various actions (see Appendix 1), and these questions were designed to emphasize the purposes of health chatbots identified in the scoping review. A five-point Likert scale (ranging from “strongly unlikely” to “strongly likely”) was used in this section.

Participants recruitment

Participants were recruited via social media and messaging apps such as WhatsApp, Twitter, Telegram, Instagram, Snapchat, and Facebook, which have the most popular user engagement in Saudi Arabia (14). Recruitment was conducted over the period of June 17th to July 15th, 2020. People were eligible to participate if they were over 18 years old, lived in Saudi Arabia, and had a smartphone that supported 3G network or above. Other characteristics such as gender, nationality, language, and occupation were not restricted.

Data analysis procedures

Raw data from eligible participants were downloaded via Microsoft Form and imported into IBM SPSS Statistics software (version 1.0.0.1032). The dichotomous variables (e.g, gender: Female/Male, past health chatbot use: Yes/No) were not coded; however, the rest of the demographic information was re-coded into binary variables to avoid potential issues in statistical model fitting (more information can be found in Appendix 1). For example, the options of the variable ‘the highest level of education’ (i.e, secondary school, diploma, Bachelor’s degree, Master’s degree, Ph.D. or equivalent) were re-coded and organised into two groups ‘undergraduate degrees’ if the participant had a bachelor degree or less and ‘postgraduate degrees’ if the participant had a Master’s degree or above. Data was analysed through descriptive (i.e, frequency and percentage for categorical data) and inferential statistics (i.e, Mann-Whitney U-Test for not normally distributed data). Values were considered statistically significant at (P <0.05).

RESULTS

Characteristics of the participants

A total of 166 out of 173 individuals who participated provided sufficient information for analysis. The highest proportion of respondents were male (n= 87, 52.5%). More than two thirds (n=113, 68%) were aged 30 years and above and were employed. The majority were Saudi (90%) and healthy (i.e, no diagnosed medical conditions, 82%). About 57.2% had at least a university undergraduate degree. 42.2% provided healthcare-related services in settings such as healthcare facilities, quarantine or isolation centres or check-up points; or home-based care for a family member, relative or friend. Nearly 64.5% of participants reported having high to a very high level of information technology (IT) skills. Almost 70% searched frequently for health information. Of 166 participants, 40% were aware of health chatbots and 24% had used chatbots before. Table 1 summarises the characteristics of participants.

Table 1.

Characteristics of Participants

Variable	Value	N (166)	%
Gender	Female	79	47.6
	Male	87	52.4
Age	Under 30	53	31.9
	30 and above	113	68.1
Nationality	Saudi	149	89.8
	Non-Saudi	17	10.2
Highest level of education	Undergraduate degree	95	57.2
	Postgraduate degree	71	42.8
Current occupation	Unemployed	52	31.3
	Employed	114	68.7
Provided healthcare related services	No	96	57.8
	Yes	70	42.2
Med. diagnosed cond.	No	136	81.9
	Yes	30	18.1
Perceived IT skills	Low or medium	59	35.5
	High or very high	107	64.5
Searched health info	Less frequent	50	30.1
	Frequent	116	69.9
Health chatbot awareness	No	99	59.6
	Yes	67	40.4
Past health chatbot use	No	126	75.9
	Yes	40	24.1

Table 2 presents the perceived utilities and usefulness of health chatbots in Saudi Arabia. The majority of participants (82.5%) reported that they would use health chatbot to seek general information about COVID-19 (e.g, news, the number of active cases), and to learn how to prevent the spread of COVID-19 (e.g, how to wash hands, wear face masks). Nearly 82% endorsed that they would use health chatbots to seek general health information about COVID-19, and 81% would use them to learn about COVID-19 symptoms and other health aspects. Additionally, 79% of participants indicated that they would use these chatbots to look for nearby medical services (e.g, testing centres for COVID-19). The least common purposes to use chatbots were to self-assess COVID-19 symptoms (76%), to self-track my COVID-19 symptoms (73%), to find out if someone who has contracted COVID-19 is in close proximity (73%), and to seek information about COVID-19 medical treatments (72%).

Table 2.

Perceived Utilities of Health Chatbots in Saudi Arabia

Purposes from the scoping review	Questions from the online survey	N (166)	Likely %
Combating misinformation and fake news	To seek general information about COVID-19 (e.g, news, the number of active cases)	137	82.5
Disseminating health information & knowledge	To seek general health information about COVID-19	136	81.9
	To seek information about COVID-19 symptoms	134	80.7
	To seek information about COVID-19 medical treatments	120	72.3
	To learn how to prevent the spread of COVID-19 (e.g, how to wash hands, wear face masks)	137	82.5
Self-triage and personal risk assessment	To self-assess COVID-19 symptoms	126	75.9
	To look for nearby medical services (e.g, testing centres for COVID-19)	131	78.9
Tracking COVID-19 symptoms	To self-track my COVID-19 symptoms	121	72.9
Contacts tracing and exposure notifications	To find out if someone infected with COVID-19 is near me	121	72.9

Participants’ characteristics and their perceived utility of health chatbots

The statistical comparisons performed with Mann-Whitney U Tests revealed no statistically significant differences in all perceptions of health chatbots’ utilities between groups of the following variables: gender, age, nationality, health status (medically diagnosed conditions), perceived IT skills, awareness of health chatbots, and past use of health chatbots (see Table 3 for further information). For example, one of the conducted Mann-Whitney tests showed no significant differences between males and females in their tendency to seek general health information on COVID-19. The following results present statistically significant differences in scoring tendencies at the 5% level:

Table 3.

Summary of the Mann-Whitney U Test. “*” Means that the difference between the category of mentioned variable is statistically significant at the level of 0.05 by Mann-Whitney U Test.

		1	2	3	4	5	6	7	8	9
Variable	Values	Mean Rank	Mean Rank	Mean Rank	Mean Rank	Mean Rank	Mean Rank	Mean Rank	Mean Rank	Mean Rank
Gender	Female	82.7	82.7	82.4	81.3	79.5	81.1	84.3	84.4	79.7
	Male	84.2	84.2	84.5	85.5	87.1	85.7	82.8	82.7	87.0
Age	Under 30	89.3	92.4	87.6	85.5	86.9	85.8	86.3	88.8	86.9
	30 and above	80.8	79.3	81.6	82.5	81.9	82.4	82.2	81.0	81.9
Nationality	Non-Saudi	72.9	66.1	83.1	78.7	70.0	78.4	87.4	78.3	89.3
	Saudi	84.7	85.5	83.5	84.1	85.0	84.1	83.1	84.1	82.8
Highest level of education	Undergraduate degree	86.8	88.2	89.9*	90.7*	86.7	87.7	83.7	90.2*	85.8
	Postgraduate degree	79.0	77.2	74.9*	73.9*	79.2	77.9	83.2	74.5*	80.4
Current occupation	Unemployed	80.0	77.8	76.1	80.2	74.8	76.0	73.0*	77.9	75.0
	Employed	85.1	86.1	86.9	85.0	87.5	86.9	88.3*	86.0	87.4
Provided healthcare related services	No	81.7	80.9	81.4	86.8	78.4	77.2*	83.3	78.8	79.8
	Yes	86.0	87.0	86.4	78.9	90.5	92.1*	83.8	90.0	88.5
Health status	No medically diagnosed conditions	83.2	83.3	82.8	83.4	83.9	82.9	85.6	83.2	83.1
	Health issues medically diagnosed	84.7	84.2	86.5	84.1	81.6	86.4	74.1	84.8	85.2
Perceived IT skills	Low or medium	81.7	81.9	85.9	81.3	81.2	91.2	84.8	84.9	83.3
	High or very high	84.5	84.4	82.2	84.7	84.8	79.3	82.8	82.7	83.6
Searched health information	Less frequent	75.5	77.8	74.1	77.8	71.5*	78.3	82.5	79.2	75.9
	Frequent	87.0	86.0	87.6	86.0	88.7*	85.7	83.9	85.4	86.8
Health chatbot awareness	No	80.4	79.8	85.4	83.2	82.6	85.1	84.4	86.2	82.6
	Yes	88.0	89.0	80.6	83.9	84.8	81.2	82.2	79.5	84.9
Past health chatbot use	No	79.6	81.5	83.0	83.2	82.3	82.5	81.0	82.4	83.5
	Yes	95.7	89.7	85.2	84.3	87.3	86.6	91.4	86.9	83.6

We found a statistically significant difference in comparing the participants who had undergraduate degrees (n=95) with those with postgraduate degrees (n=71). The analysis indicated that the participants with undergraduate degrees tend to use health chatbots to learn how to prevent the spread of COVID-19 (P = 0.015), to self-track COVID-19 symptoms (P = 0.028), and to seek information about medication (P = 0.035) more frequently than those with postgraduate degrees.

The Mann-Whitney U Test also revealed a statistically significant difference (P = 0.042) how groups with different employment status seek information about COVID-19 symptoms. Both groups were found to be willing to use chatbots; however, employed participants (n=114) scored higher in using chatbots to seek information about their symptoms than unemployed ones (n=52).

There was a statistically significant difference found by comparing the participants who had provided any sort of healthcare-related services (n=70) with those who had not (n=96). This difference indicates that participants who provided any healthcare-related services are more likely to self-assess for COVID-19 symptoms by using health chatbots (P = 0.036).

Lastly, there was a statistically significant difference between the participants who frequently searched for health information on the internet (n=116) and those who sought out health information less frequently (n=50). The participants who frequently searched for health information on the internet scored higher in having tendencies to look for nearby medical services by using health chatbots (P = 0.023) than the others.

Note: the numbers from 1 to 9 in the first row indicate the perceived utilities of health chatbots reference as illustrated in Table 2.

Discussion

To our knowledge, this is the first study exploring Saudi Arabia’s acceptability of health chatbots related to COVID-19 from the perspective of the end-users. We concluded that most of the participants had limited experience with these emergent technologies (i.e, had not used chatbots in other settings before); however, they were willing to use health-specific chatbots for various activities, including seeking information about healthcare services, how to prevent the spread of COVID-19, and how to regularly self-assess symptoms. In addition, our findings indicated that the participants were more willing to use health chatbot to seek general information about COVID-19 over seeking information about COVID-19 medical treatments.

Furthermore, the majority of the participants were not aware of health chatbots; despite this, we found that these previously-unaware participations had positive perceptions of these novel health technologies’ functionalities. The statistical comparison revealed perception of health chatbots’ functionality and usefulness varied based on the people characteristics; in particular, the person’s education level, employment status, frequency of seeking health information on the internet, and if they had been provided with any sort of previous healthcare-related services. Those findings are consistent with previous research studies exploring peoples’ acceptance of novel health technologies (15-18).

As this study revealed lack a of familiarity and experience of health chatbots, we encourage future research, treatment teams and public health officials to raise peoples’ awareness about these emergent technologies and their diverse applications in healthcare (19). Health mass media campaigns, social role models’ endorsements, or adaption of health chatbots by national healthcare providers may help in this regard by influencing their popularity and feelings of legitimacy (15).

This study has its own strength and limitations. Our sample was comprised of a diverse selection of males and females with different levels of education, age, health status (healthy individuals and people with chronic conditions), employment status cohort, and low and high levels of familiarity with technology. Thus, our sample may be representative of the population that might be asked to use chatbots related to COVID-19. However, the recruitment technique through social media may introduce a selection bias (i.e, internet users) which may compromise the generalisability of our findings. Therefore, our future work includes utilizing a more rigid methodological design (i.e, field trials) to capture the real-world usability and overall perception of health chatbots to combat COVID-19, and how using these emergent technologies influence public opinions in Saudi Arabia.

CONCLUSION

This paper presents a cross-sectional study of the use of health chatbots to combat COVID-19. Participants’ awareness and experience were found to be low, but most participants had positive perceptions and willingness to learn more about and use these emergent health technologies. Our future work will explore options to increase the implications of the positive influence chatbots may have in Saudi Arabian healthcare.

	Original Variable Original Values	Coded new variable Coded new values
1.	Gender Male Female	Gender Male Female
2.	Age 18–19 20–29 30–39 40–49 50–59 60 and above	Age Under 30 30 and above
3.	Nationality Saudi Emirati Kuwaiti Omani Bahraini Y emeni Egyptian Sudanese Others	Nationality Saudi Non-Saudi
4.	Highest level of Education Secondary school Diploma Bachelor degree Master degree PhD or equivalent	Highest level of Education Undergraduate degrees or less Postgraduate degrees or above
5.	Your occupation in the last 6 months). Government or Private health sector employee General/higher education employee Army/security employee Student Unemployed Others	Current occupation Unemployed [students+ Unemployed] Employed [all the other options]
6.	Within the last six months, have you provided one of the following healthcare services (not limited to confirmed Covid-19)? Direct medical care services such as in healthcare facility, in Quarantine or Isolation centers, checkup points. Indirect health care services such as administrative work, organizing awareness campaigns, conducting scientific studies or statistics. Home-based healthcare for a family member, relative, or friend. I did not provide any healthcare services to anybody.	Provided healthcare related services Yes, direct or indirect No
7.	I have: Health issues medically diagnosed e.g. high blood pressure, diabetes, etc. No medically diagnosed conditions	Health status Health issues medically diagnosed e.g. high blood pressure, diabetes, etc. No medically diagnosed conditions
8.	In your opinion, your Information Technology skills are: Low (e.g. use internet browser like Google, Email) Medium (e.g. use a word processor program, social media) High (e.g. use peer-to-peer file sharing for exchanging movies, music etc.) Very high (e.g. create a web page, write a computer program using a specialized programming language)	Perceived IT skills Low or medium High or very high
9.	Within the last 6 months how many times you searched for health information. Many times a day (high) A few times a day (medium) Once a day (low) Less than once a day (very low)	Searched for health information Frequent [high + medium] Less frequent [low + very low]
10.	Do you know there is a chatbot that provides health information and tips? Yes No	Health chatbot awareness Yes No
11.	Have you ever used a chatbot to seek health information or tips? Yes No	Past health chatbot use Yes No
12.	To seek general information about COVID-19 (e.g., news, the number of active cases)	Strongly likely, likely, not sure, unlikely, strongly unlikely
13.	To seek general health information about COVID-19	Strongly likely, likely, not sure, unlikely, strongly unlikely
14.	To seek information about COVID-19 symptoms	Strongly likely, likely, not sure, unlikely, strongly unlikely
15.	To seek information about COVID-19 medical	Strongly likely, likely, not sure, unlikely, strongly unlikely
16.	To learn how to prevent the spread of COVID-19 (e.g., how to wash hands, wear face masks)	Strongly likely, likely, not sure, unlikely, strongly unlikely
17.	To self-assess COVID-19 symptoms	Strongly likely, likely, not sure, unlikely, strongly unlikely
18.	To look for nearby medical services (e.g., testing centres for COVID-19)	Strongly likely, likely, not sure, unlikely, strongly unlikely
19.	To self-track my COVID-19 symptoms	Strongly likely, likely, not sure, unlikely, strongly unlikely
20.	To find out if someone infected with COVID-19 is near me	Strongly likely, likely, not sure, unlikely, strongly unlikely