Demand for Mobile Health in Developing Countries During COVID-19: Vietnamese's Perspectives from Different Age Groups and Health Conditions.

BACKGROUND: Vietnam's economy and intellectual standards have witnessed significant development, improving conditions for residents to acquire novel mHealth applications. Additionally, the outbreak of the COVID-19 pandemic has influenced Vietnamese awareness of healthcare; however, previous studies have only been clinician-centered rather than customer-centered.
METHODS: This study addresses this literature gap by interviewing 50 Vietnamese participants grouped by age, namely Generation X, Generation Y, and Generation Z, and health conditions, namely whether participants or family members have chronic illness. The study utilized semi-structured and in-depth interviews to collect the data and used thematic analysis to analyze the data under the unified theory of acceptance and use of technology framework.
RESULTS: Most participants were willing to adopt this technology and demanded a convenient and user-friendly one-stop-shop solution, endorsements from credible and authoritative sources, and professional customer services. However, each group also had distinctive demands and behaviors.
CONCLUSION: This study contributes theoretically by providing context-rich demand for Vietnamese customers across three generations and healthcare conditions during the COVID-19 pandemic and comparing their behavior with pre-COVID literature. While this research provides helpful information for potential app developers, this study also suggests that mHealth developers and policymakers should pay more attention to the differences in the demand of age groups and health conditions.

Introduction

In recent years, intellectual standards have increased considerably with the significant development of the Vietnamese economy. According to data from The World Bank, the annual percentage of Gross domestic product (GDP) growth in Vietnam has never been below 4% from 2009 to 2019. Moreover, Vietnam’s GDP per capita is rising continuously and strikingly each year.1 As citizens’ incomes and intellectual standards have improved, people are focusing more on their well-being. Furthermore, with around 61.37 millions smartphone users by May 2021 and approximately 68.7% of the country’s residents having access to the Internet in 2019,1,2 numerous impediments to Vietnamese’ intention to improve their quality of life have been generally removed. From the researchers’ speculation, there might be an increase in the number of Vietnamese citizens who are ready to acquire new technologies to make life safer and more convenient.

During the COVID-19 crisis, the country first applied effective measures with strict procedures to tackle this novel virus;3 however, since the arrival of the delta variant of SARS-CoV-2, several cities in Vietnam have had to enter rigorous social distancing periods. Therefore, it is challenging for residents to go to doctors and receive direct medical care at hospitals as people are afraid of spreading the disease or being infected by the virus.4 In addition, Vietnam’s healthcare system has been suffering from COVID-19 and has become overloaded. Numerous doctors, nurses, and healthcare staff are required to concentrate on the treatment of COVID-19 patients, which also means that other minor diseases might be placed under the priority list.5

Understanding the context of Vietnam, mHealth might provide a potential solution to COVID-19 and promote healthcare services. According to the Global Observatory for eHealth (GOe) by the World Health Organization (WHO), mHealth or mobile health is defined as “medical and public health practice supported by mobile devices, such as mobile phones, patient monitoring devices, personal digital assistants (PDAs), and other wireless devices”.6 mHealth applications have been widely used in developed countries, enhance the effectiveness of health care services.7 Specifically, mHealth can be cost-saving, time-saving, and able to provide better reach, increased interaction between patients and doctors by making it easier and faster to exchange information regarding diseases and health conditions.8 By operating in distanced communication, mHealth applications can also help prevent the contamination of communicable disease within society.9

Recognizing the benefits and suitability of mHealth in Vietnam, we believe that this application should receive more attention than digital health. First, mHealth services operate on portable devices, making them more convenient for busy residents who often have to go out. Second, it is faster to turn on phones, tablets, or smartwatches than to turn on desktops or televisions; therefore, mHealth is more useful in emergencies. Finally, the prices of mobile phones, tablets, and smartwatches are, in general, considerably lower than those of desktop computers, or other immobile digital devices. Therefore, mHealth applications are becoming more convenient for residents. These reasons motivated us to concentrate on the development of mHealth by encompassing eHealth services.

In addition, research on the barriers to and readiness for mobile apps’ involvement in healthcare interventions has mostly been conducted in developing countries in Africa, South Asia, and Latin America.10,11 Moreover, previous studies have used quantitative methods to test and confirm the effectiveness of eHealth interventions. However, before service providers and governments plan to establish mHealth interventions, it is essential to understand whether people accept and adopt these tools. Thus, there should be more qualitative research exploring the factors influencing acceptance of and readiness to adopt eHealth interventions. In addition, research on people’s willingness to use mHealth applications has tended to focus on the general population. Meanwhile, the differences in age groups and health status might also play a significant role in affecting the acceptance and readiness of eHealth applications.

As per the literature gap above, the purpose of this study is to investigate the factors impacting the readiness, acceptance, and adoption of mHealth applications in Vietnam, a developing country. In this study, the researchers first explored Vietnamese people’s level of understanding of mHealth. After assessing the level of necessity and interest of people, the researchers identified the most desired features of promising mHealth applications and elements that could affect the Vietnamese decision to use this type of technology. Moreover, researchers aimed to determine the moderating effects of the COVID-19 pandemic.

This study aimed to provide information regarding people’s acceptance of mHealth applications to both the Vietnamese government and technological service providers and stakeholders. When they acknowledge consumer demand, they can improve their current application or create a new, effective one.12 As the theoretical framework of the research, the Unified Theory of Acceptance and Use of Technology (UTAUT) is used to create the interview guide, to understand and analyze the factors that influence Vietnamese acceptance and adoption of mHealth applications in a developing country context.

The Roles of mHealth in Improving the Healthcare System in the Developing Countries

Since the advent of mobile phones, mobile access and the use of mobile phones have become ubiquitous. The significant growth of mobile phones is the penetration of mHealth applications worldwide.13 People have utilized mHealth technologies as stand-alone intervention tools to help with self-monitoring, patient education, and healthy lifestyle practices, such as better chronic disease self-management. Scholars have conducted various studies to examine the benefits of mHealth applications and have identified gaps in the literature for future improvement. For instance, in light of the rapid virtualization of dentistry, mHealth messages and apps have been considered a viable clinical tool for the enhancement of oral health and the prevention of oral problems.14 Furthermore, researchers have regarded telemedicine and mHealth as relatively novel intervention modes for controlling systolic blood pressure among stroke survivors, especially those with hypertensive stroke.15 Telemedicine is also considered to be equally effective for face-to-face care in hepatitis C virus treatment. In general, mHealth solutions have proven to be effective in improving healthcare services.16

As mHealth applications can provide cost-effective mitigations for prevalent diseases, they can solve financial constraints, which are one of the key obstacles for delivering healthcare services for populations in developing countries.17 Apart from pre existing problems of childhood infectious, perinatal and nutritional disorders, due to the transformation of life habits and urbanization, the populations of developing countries also face chronic non-communicable illnesses such as adult cardiovascular disease, diabetes, and depression.17 However, some of these chronic health conditions can be improved by mHealth technologies. For example, mHealth interventions have been found to improve glycemic control (HbA1c) in patients with diabetes compared to standard care or other non-mHealth approaches.18 While medical non-adherence is a key barrier for improving the outcome of cardiovascular diseases, mHealth tools can improve medication adherence in patients,19 and according to previous studies,20 mHealth applications with a lay support person may yield sustained improvements in antidepressant adherence and depression symptom remission. Therefore, considering the promising effects of mHealth apps, the field of digital health should be researched in developing countries.

However, developing countries face many barriers to adopting mHealth services. A systematic review of developing countries identified up to 73 individual barriers with the top three barriers categories namely infrastructure, equipment lacking and technology gap.11 A systematic review of mobile health adoption for sexual and reproductive health among youth addressed numerous barriers to provider prejudice, stigmatization, discrimination, fear of refusal, lack of privacy, confidentiality, decreased technological literacy, inferior network coverage, and lower linguistic competency.21 Although the aforementioned research provided valuable insights about mHealth in developing countries, there is no specific research concentrating on the barriers in Vietnam to ensure a more localized and effective app development. Moreover, developing countries are currently facing a new challenge: the COVID-19 pandemic. Hence, we aimed to find a solution for mHealth to promote the population’s life conditions and fight the pandemic.

The Roles of mHealth in Mitigating the COVID-19 Pandemic in the Developing Countries

The COVID-19 pandemic has had devastating social, economic and political impacts, making it challenging for governments to act promptly to control the pandemic.22 Many countries have implemented containment and mitigation measures to prevent the spread of diseases. mHealth technologies have proven to have considerable positive effects in preventing the spread of the COVID-19 pandemic. On a social level, when combined with diagnostic and immune status testing, mHealth technology could be a valuable tool to help mitigate, if not prevent, the next surge of COVID-19 cases.23 According to a previous study, digital health can contribute to reducing the spread and improving the effectiveness of pandemic control.22 On an individual level, health apps support users in managing their risk of exposure and symptom progression and help patients to track their symptom development, take medication and maintain a healthy lifestyle.24

Acknowledging the benefits of mHealth in the current health crisis, multiple studies have investigated the factors affecting its adoption in this pandemic context. Using UTAUT2, one study confirmed that multiple variables such as performance expectancy, effort expectancy, social influence, facilitating condition, and hedonic motivation were significant determinants of mHealth usage intention.25 Moreover, another study also identified a strong direct positive effect of both situational constraint and health consciousness on the behavioral intention to use mHealth technologies.26 Second, there have been several papers showing empirical evidence to prove a rising demand for mHealth apps due to the pandemic situation. In the United States, patients with chronic health conditions such as cardiometabolic diseases, respiratory conditions, immune-related conditions, mental health, and obesity are more likely to use COVID-19 mHealth applications.27 Researchers in China have also identified that parents’ willingness to adopt mHealth for their children during the pandemic was relatively high.28 Considering the benefits of mHealth applications investigated in developed countries during the current situation, the authors are motivated to discover mHealth adoption among developing countries.

The researchers were motivated by multiple gaps in the literature regarding mHealth adoption in Vietnam, particularly during the pandemic. First, different countries will require different conditions such as the healthcare environment, status of progress in IT, and economic status which can influence the possibility of introducing telemedicine.29 However, previous research has only shown that Vietnam has a high possibility of introducing telemedicine services based on facilitating conditions. Therefore, researchers wish to contribute by investigating Vietnamese people’s demand, particularly during the COVID-19 pandemic. Second, although patient demand in developing countries during COVID-19 has been widely investigated in Central East Asia and South Asia, there are only two studies on digital health in this pandemic situation in Vietnam but from policy-makers and doctors’ perspectives.22,30 However, the literature has under-researched the desiring features from the users perspective in an mHealth application, specifically to mitigate the current COVID-19 pandemic situation in Vietnam. These gaps motivated the authors to choose Vietnam, a country unfamiliar with digital or mobile health, as the new research context.

mHealth Research and Development in Vietnam Before and During the COVID-19 Pandemic

In Vietnam, mHealth has remained a novel topic both before and during the COVID-19 pandemic. Many product orientation papers published from 2016 to date have examined the practicality, usefulness, and feasibility of some mHealth applications in various fields such as HIV, maternal and infant health, sexual and reproductive health and user satisfaction and acceptance.31–35 Furthermore, several market-orientation investigations have been conducted. Using a quantitative approach, studies have found that there is a low percentage of mHealth application users; however, the younger generations were mostly satisfied with mHealth innovations that they had experienced and were ready, and felt receptive to more useful mHealth applications in the near future.36,37

In response to the COVID-19 pandemic, Vietnamese companies have seen skyrocketing needs and interest in the early stages of the situation.38 However, the majority of mHealth applications are not sufficiently ubiquitous and are unable to reach a large proportion of Vietnamese people. Previous studies have identified that the largest challenges to mHealth sustainability and development in Vietnam are underdeveloped technological infrastructure, lack of attention, and funding from the government.35,39 However, the context of mHealth in Vietnam has changed considerably nowadays mostly due to COVID-19. The government has devoted large amounts of money to developing mHealth applications such as Bluezone, NCOVI and health information systems and has attracted millions of installations.40 However, these mHealth COVID-19 applications only hold attention for a significantly short amount of time and have not diversely solved the demands of the residents.

In conclusion, we believe that there are significant gaps in the field of mHealth in Vietnam. First, more qualitative studies should be conducted to acquire Vietnamese demand from different dimensions.22 Second, recent scholars have mostly focused on the younger generation, while the differences in technology receptiveness and understanding or health awareness between age groups and separated health condition groups are also vital problems that more papers need to investigate carefully.25 Additionally, during the COVID-19 pandemic, there has been a lack of research investigating the demands and viewpoints of vulnerable and limited access to technology citizens such as the elderly or people with chronic diseases, who are sometimes the group of residents requiring the most health care services. For instance, two researchers who concluded that digital health and emerging telehealth technology hold the potential to support Vietnam against the COVID-19 pandemic admitted that their papers came from the perspective of doctors, not residents or patients.22,30 Third, to build a diversity of mHealth interventions, it is essential to study further mHealth fields, as well as the limited fields that developers believe to be important. Finally, there has been a demand to concentrate on mHealth applications as most of the current interventions studied are based on SMS or phone calling foundations.11 Acknowledging several gaps, the researchers decided to conduct this investigation to support other authors in solving the unknown.

The Unified Theory of Acceptance and Use of Technology (UTAUT)

In the literature, various theories can explain user adoption behavior, such as Mental Accounting Theory (MAT), Theory of Planned Behavior (TPB), and the Technology Acceptance Model (TAM).41 Hence there is a need to decide on a theoretical framework suitable for the study goals. Recent literature has employed different versions of The unified theory of acceptance and use of technology (UTAUT) to explore the adoption of mHealth in developing contexts, such as Bangladesh and Iran.42,43 Under a quantitative approach, these scholars have also found that this theory has high explanatory power regarding the acceptance of mobile apps and wearable technology.44 Therefore, this study adopts the UTAUT framework to investigate mHealth in the Vietnamese market.

According to the literature,45 UTAUT is a modified version of TPB and TAM that examines user perceptions using four variables: performance expectancy (PE), effort expectancy (EE), social influence (SI) and facilitating conditions (FC). According to this study, while PE depicts how an individual expects that employing technology will improve his or her performance, EE is the degree to which a person expects that using a given technology will not require a significant amount of effort. SI studies how users’ families and communities affect the adoption rate of new technological innovations, while FC explores how accessibility to technical support and infrastructure can facilitate the use of technology. The data collected from these variables can guide policymakers and healthcare enterprises in tailoring their products and services to fit market demand and readiness.

Finally, while the original UTAUT has four moderators, including gender, age, experience, and voluntariness, the researchers only compared the results between different age groups and the health conditions of the participants and their families to ensure the specificity and concentration of the study scope. Regarding age groups, the existing literature has found that there are different behaviors and motivations to adopt new technology, such as Generation X utilizing mobile phones for information search and Generation Z utilizing mobile phones for entertainment purposes.46,47 Moreover, it is important to understand the gap between technical knowledge and readiness among Vietnamese citizens to tailor better products and services for each group of participants.48 Empirical evidence also suggests that the demand for curative medicine and preventive medicine varies between participants with different health conditions.49 One explanation is that each disease requires treatment. It is common sense that a high perception of health severity can motivate potential users to adopt mHealth technologies. However, one recent study found that health perspectives on perceived susceptibility and perceived severity do not impact the intention to use wearable healthcare technology.50 To investigate more deeply the conflicts in the research findings, the health condition of participants and their families also plays a key role in the analysis of mHealth products in Vietnam. (See the Study design and data collection for more information.)

Methods and Methodology

Study Design and Data Collection

Previous literature has shown the benefits of in-depth interviews and thematic analysis techniques to explore users’ intentions and decisions to adopt a new digital product across the healthcare industry in diverse contexts.51,52 In-depth interviews allow researchers to obtain detailed and valuable information about participants’ attitudes, expectations, and concerns. Thematic analysis provides a guideline for organizing collected data and generating practical insights.53 Hence, this study selected this approach to analyze the motivations and barriers that influence users’ intentions and decisions to adopt mHealth products. Snowball sampling was used to recruit 50 participants, including the results of the pilot study (Table 1). The criterion for selection was that the participants were Vietnamese citizens with a date of birth from 1965 to 2005. The researchers categorized the participants into 6 sub-groups based on the age and health status of the participants or their family members. Regarding age groups, the study follows the Pew Research Center’s categories: Generation X (1965–1980), Generation Y (1981–1996), and Generation Z (1997–2012).54 While Generation Z has an age range from 1997 to 2012, the researchers did not acquire data from participants who were born after 2005 to comply with Vietnam’s Children Law 2021 on research with minorities. Theoretically, this categorization permits researchers to explore and compare the moderating effects of health condition and age on the UTAUT model. Practically, software developers can use this comparative insight to create personalized products that are more serviceable and fulfil the unique needs of each targeted group. Moreover, the researchers followed the COREQ checklist to ensure comprehensiveness and transparency in the interviews and reporting processes (Additional File 1).55

Table 1

Key Sociodemographic Characteristics

Sociodemographic Characteristics		N (%)
Gender	Male	25 (50)
	Female	25 (50)
Age	Generation X	14 (28)
	Generation Y	13 (26)
	Generation Z	23 (46)
Educational attainment	No formal education	0 (0)
	Less than a high school diploma	14 (28)
	High school diploma	6 (12)
	Bachelor’s degree	24 (48)
	Graduate degree	6 (12)
Working industries (multiple answers)	Healthcare-related industry	4 (8)
	Technology-related industry	8 (16)
	Others	38 (76)
Health Conditions	No chronic condition	22 (44)
	Have chronic conditions or family has chronic conditions	28 (56)
The frequency of internet usage	Daily	49 (98)
	1–6 times a week	0 (0)
	Less than once a week	1 (2)
	Never	0 (0)
The exposure level to mHealth	Experts in fields (mHealth developers, mHealth researchers)	0 (0)
	Recognize mHealth applications and engage with mHealth daily	6 (12)
	Engage with mHealth daily, but do not recognize the mHealth applications	18 (36)
	Never heard of the mHealth concept	26 (52)

Two authors (P.D, H.N) conducted the interviews, conducted the qualitative research training and experience. The two interviewers were one male and one female and were pursuing a Biomed and Medical degree. In the pilot interview, the researchers employed convenience sampling to test and reconstruct the semistructured interview guide (); and in the main study, the researchers did not have formal or informal relationships with most participants. Data was collected from July 2021 to September 2021. All interviews had written records, and some had audio recordings, depending on the participants’ permission. The duration of these interviews ranged from 14 to 37 minutes. The researchers determined the sample size based on saturating data; hence, the data collection ended with 50 participants (N=50) because there were no significant and novel data from the additional interviewees.56

Before the official meeting, the researchers sent a Qualtrics survey form to ask for participants’ written consent and socio-demographic characteristics. In this step, the researchers screened out participants who were unsuitable for the scope of this study, such as non-Vietnamese citizens and those older than 65. If a participant passed the screen round, the interviewers scheduled an interview via mobile phone or Zoom. The in-depth interviews began with an introduction of the research topics and goals to the participants. The interviewers then checked the participants’ understanding of the mHealth applications and clarified the important concepts. The interviewers facilitated the questions following the interview guide, which was built based on UTAUT theory. These questions aimed to explore and define Performance Expectation (PE), Effort Expectation (EF), Social Influence (SI), and Facilitating Conditions (FC) in the context of mHealth applications in Vietnam. The interview ended with participants’ conversations and thoughts about the research topics. This final step allows researchers to gain additional information beyond the scope of traditional UTAUT theory.

Data Analysis

After data collection, researchers performed data analysis and interpretation. While all data coders have studied qualitative research, two authors, KT and TN, have previous qualitative publications on healthcare and technology adoption. Moreover, all the coders were fluent in both English and Vietnamese. As with the collected data in Vietnamese, the researchers employed a forward and back translation approach when transcribed and translated into English to preserve the clarity and consistency of the information before and after translation. The researchers combined the transcribed data verbatim and notes using Microsoft Word.

The research followed the guidelines of the thematic analysis developed in the literature.57 In which, the researchers coded interviewees’ responses manually because the study sample was relatively small and this method ensured nuance and accuracy in finding the distinctions between demands and challenges faced by each group (age and health condition). The researchers familiarized themselves with the transcripts by repeatedly listening to and reading the interview audio or notes. The researchers then constructed a coding tree with precise descriptions of major themes and sub-themes to categorize and classify meaningful patterns and trends into relevant UTAUT constructs. For instance, the initial code “leaner medical/ booking procedure” was placed in the effort expectancy. When new themes or additional information emerges, researchers will work together for sorting, regrouping, or adding new nodes to the coding tree (Table 2). This refinement process has enabled researchers to develop insightful data-driven theories in a concise and punchy manner. The research further sorted these themes and subthemes into a matrix table in Excel to compare the data between age groups and healthcare conditions. After finalizing the data analysis, the researchers used the member checking method to enhance the credibility and validation of the data interpretation and ensure that the participants’ thoughts and demands were well understood.58 Finally, the researchers select 1–2 representative quotations for sub-themes as evidence for the analysis following the procedure suggested by Klappe.51 Figure 1 shows the relationship between the variables after the data analysis.

Table 2

The Themes and Its Axial Coding

Themes	Sub-Themes	Axial Coding
Performance Expectancy	One-stop-shop solution	Provide various functions to promote healthy lifestylesPartner with healthcare providers for ease of access to the offline healthcare system
	High-quality information and measurements	Measure and manage health indexEnhance medical self-diagnosis experience
	Personalized recommendations	Make recommendations based on health status with privacy protectionWarn possible health risks by studying the daily habits of users
	Updating content	Automatically update users’ personal health recordGive information related to contemporary events
Effort Expectancy	User-Friendly App Design	Prevent intrusive advertisements from covering up contentsUse more illustrations and less words
	Optimal overall user experience	Not have overly complex login and registrationHave navigation toolbar easy to use
	Language accessibility	Write contents in Vietnamese, English or Ethic minorities languages of VietnamUse simplified terminologies
Social Influence	Word-of-mouth recommendations	Recommended by users’ friends and relatives
	Endorsement from credible and authoritative sources	Endorsed by doctors, healthcare experts, healthcare organizations, large official newspapers, and governments
	Perceived quality from digital platforms	Advertised on social media and television
Facilitating Condition	Careful and friendly customer service and instruction	Provide direct guidance via hotline or chatsHave websites or social media pages for users to be updated with app features
	Technical requirement	Takes up small storage space on devicesBe able to work with several different operating systemsConsume little battery power
	Fee and payment method	Have affordable costs or covered by health insurancesAllow diverse payment methods

Figure 1

The relationships between the variables in accordance with the UTAUT model.

Results

Social-Demographic of the Interview Participants

The data were saturated after the number of participants reached 42, and the interviewers conducted eight more interviews to confirm saturation. The interviewers observed saturation of the data for all three age groups. Moreover, although the researchers firstly categorized “participants with a chronic condition” and “participants whose family member has a chronic condition” into two different categories, they decided to merge these groups due to their similarities in response under UTAUT theory. Hence, 22 participants had no chronic condition, and 28 participants (or their immediate family members) had a chronic condition. The participants’ chronic conditions included osteoporosis, gastritis, hepatitis, hypertension, cancer, diabetes, osteoarthritis, and cardiovascular disease.

After summarizing the socio-demographic information of the participants in Table 1, the researchers identified some potential biases in the responses, which might not sufficiently represent the general Vietnamese population. Nearly all participants (N=49) used the internet daily. Compared with the average internet user rate in Vietnam of 70% of the population, the study sample may be biased toward higher technological readiness. Moreover, nearly half of the participants did not know the concept of mHealth or its potential applications. Although the interviewers had provided simple explanation for these participants, they only provided comments on technology implementation overall rather than mHealth specifically. Nevertheless, the authors still incorporate these suggestions into the results and discussion because their lack of knowledge illustrates a market gap that policymakers and entrepreneurs must address. On the other hand, four healthcare professionals and eight technology experts provided in-depth strategies on technological adoption and features of the mHealth app. Therefore, this study clearly states whether the quote comes from the perspectives of both experts and customers.

Awareness and Necessity of mHealth in Vietnam

In terms of mHealth awareness, this concept was unheard in nearly half of the interviews. Many participants engaged in mHealth applications daily; however, they did not recognize or be aware of the term.

Regarding the necessity of mHealth applications from the interviewees’ perspective, most respondents agreed that they were essential in Vietnam. However, each age group and health condition had different levels of necessity and reasons for their decision. Generations X and Y mentioned that their hectic lifestyle prevented them from taking care of their health. Therefore, mHealth interventions are required to support such patients. On the other hand, Generation Z expressed that because current mHealth applications have not fulfilled their demands, they certainly need alternatives. Furthermore, people with family members with chronic diseases have also reported that Vietnamese healthcare systems are overloaded, requiring more help from mHealth. By contrast, a small number of participants said that mHealth applications are unnecessary in Vietnam as the country’s technology has not been sufficiently developed to make residents rely on them. Therefore, people still prefer face-to-face interactions with medical staff. In addition, some interviewees mentioned that, in general, many Vietnamese citizens do not have sufficient technical skills or knowledge to join mHealth networks. Hence, before deciding to implement any mHealth interventions, the country must first educate its residents, which is a burden on Vietnam’s current government.

Performance Expectancy

A one-stop-shop solution was a sub-theme mentioned by most interviewees when asked about their performance expectations for mHealth applications. In this context, a one-stop-shop solution means that the application offers multiple services or products to users. This provides convenience and efficiency to users because they feel demotivated when downloading apps to serve various purposes in daily life. Some highly mentioned features include counting steps, calculating sleep time, and regular checkup reminders to promote healthy lifestyles. In particular, respondents demanded that the providers of mHealth applications partner with various healthcare providers as they desire more straightforward and faster access to the on-site healthcare system. For example, people often have to wait in long queues to receive health checkup results; therefore, they wish to obtain electronic results. In addition, to satisfy different expectations, mHealth applications can be integrated with online booking appointments, contact with nearest hospitals or clinics during emergencies, and a comparison of hospital service quality. With the growing number of people using online services, attendants believe that mHealth applications will gain attention when offering online healthcare consulting, virtual pharmacies and delivery services. During the COVID-19 outbreak in Vietnam, these functions are more pertinent as they reduce waiting time.

High-quality information and measurements also play critical roles in the adoption of mHealth applications. All delivered information must be accurate and detailed and must come from reliable sources such as experts, practitioners, and hospitals.

I strongly believe one of the main reasons why many people do not use mHealth applications frequently is because the technology nowadays does not provide accurate measurements enough only through mobile phones.

(A, technological engineer, Generation X, respondent or family members have chronic health problems)

According to most participants or their family members who have chronic diseases, they would strongly prefer mHealth applications capable of measuring the health index and comparing users’ index to a healthy state. In addition, many generation X interviewees welcomed articles about herbs beneficial for disease prevention and health enhancement. This is partly because the elderly in Vietnam traditionally favor medicines from wild herbs for mild illnesses. Vietnamese residents tend to self-diagnose their health conditions before seeking medical treatment from doctors. Therefore, respondents also demanded an enhanced medical self-diagnosis experience. They hoped to receive a list of possible medical conditions and corresponding suggestions for self-treatment after entering their symptoms and answering several simple questions.

I want when I put information about my health, the application can list out fully what health problem I may have. When the application feels that my symptoms are serious, it can recommend that I go to see a doctor.

(B, Generation X, respondent or family members with chronic health problems)

One of the most prominent features influencing people’s intention to adopt mHealth applications is personalized recommendations with data protection. According to most interviewees, mHealth applications should provide recommendations based on the health status and medical treatment.

(I want) the apps to be able to give me instructions to cure my own health problems by myself or caring at home, especially in the current situation (the COVID-19 pandemic).

(C, medical student, Generation Z, respondent or family members do not have chronic health problems)

Moreover, each time healthcare is sought, apps ask patients for information related to biographical data and health history to make an accurate diagnosis, creating a complicated and time-consuming procedure. If mHealth applications create and store individualized health checkup results then patients and hospitals can save time and effort. Additionally, as suggested by the respondents, personal recommendations also come from studying the daily habits of the users.

I want the applications to remind me to do some short exercise after a long period of working time … Or maybe tracking my daily meals to advise better food since I am too busy to even think of what healthy food I should eat. It might seem quite robotic; however, in these modern days, I believe we need such care.

(D, Generation Y, respondent or family members with no chronic health problems)

However, many people emphasize data protection when mHealth applications store personal information. These respondents were afraid that ill-intentioned people possibly try to breach their health-related information such as allergies to serve their purposes. People are also worried about identity fraud, such as credit card numbers, or personal privacy, such as the health status of family members that patients try to hide.

Another aspect repeated by many interviewees was the updating of the content. This function mainly revolves around (1) updating the user information and (2) providing information related to the latest health events. Regarding updating user information, interviewees hoped to have their health records automatically updated after each treatment so that they and their doctors could conveniently track their situation. It also includes a quick update of user information changes such as bank accounts, phone numbers and home addresses.

For instance, once the users finish their health checks at hospitals, the applications must immediately update and store information related to the health checks for users to easily keep track of.

(E, student, generation Z, respondent or family members have chronic health problems)

In terms of updating the latest health events, respondents demanded official and accurate news from official sources related to current health issues or breakthroughs in healthcare. Recently, almost everyone has been cared for by the COVID-19 pandemic. Our interviewees also hoped to be informed about vaccination plans, the effectiveness of novel vaccines, or guidance to take care of themselves.

Effort Expectancy

According to most participants, a user-friendly app design is a compulsory feature of mHealth applications. Interviewees from the three age groups and two health condition groups agreed on the critical role of the simple and easy-to-navigate toolbar. Users can find the targeted sections without too many sundry steps, such as hitting too many buttons. Voice search and highlighting warnings or essential information were also two of the most common suggestions for simplifying the interface and improving user experience. Furthermore, many people in the chronic health problems group would like to use applications depicting more illustrations such as symbols, images, or short videos rather than words. While only generation Z suggested that application interfaces should be attractive with neutral colors and eye-catching graphics, only generation X wanted large letters because their vision was aging. Apart from these demands, the youngest generation paid the most attention to irritating advertisements that covered content.

Mobile health service creators should concentrate on providing an optimal overall user experience. Most participants preferred applications with simple sign-in or sign-up mechanisms, such as fingerprints and face recognition.

If my family found an application which was too complicated to create accounts, we would straightforwardly uninstall the application without even bothering to learn more.

(F, music teacher, generation Y, respondent or family members have chronic health problems)

Surprisingly, generation X, generation Y, and those with chronic health problems respondents wanted regular notifications on mHealth apps. However, a significant number of generation Z interviewees said that they would love to download apps with fewer notifications because pop-up lines seem annoying and distracting. Several participants from the chronic health problems group and generation Y suggested applying advanced technologies such as Artificial Intelligence (AI) to input user data. With AI, elderly or busy people do not have to upload their information manually. Additionally, developers may want to create multiple methods for users to input data such as taking photos, scanning, talking, drawing, and typing. Moreover, the participants hoped that AI could analyze their data and provide valuable recommendations.

Language accessibility was also a concern among the interviewees, especially generation X and Y. Applications were advised to be written in Vietnamese, English, and ethnic minority languages of Vietnam if developers aim to nationwide their services.

Most mobile health apps nowadays are developed in English by foreigners so the users have a language barrier and could not take all of the advantage from the applications.

(A, technological engineer, Generation X, respondent or family members have chronic health problems)

Moreover, it was estimated that nearly half of the respondents desired applications that did not use terminologies while providing concise and understandable definitions of medical conditions.

Social Influence

Endorsements from credible and authoritative sources were the most commonly named social influences on participants’ intentions to use an mHealth application. Recommendations from doctors, healthcare experts were the most reliable sources of information mentioned for each group of respondents. Moreover, encouragement from the authority to use an application was frequently shared as many respondents named Bluezone - an application widely introduced by the Vietnamese government. Respondents with family members with chronic health conditions tended to be influenced by applications created by large medical organizations or companies. However, respondents whose family members did not have chronic health conditions saw newspapers as credible sources for the introduced application.

Word-of-mouth recommendations were the second-most frequently mentioned sub-theme across every group of respondents. The introduction of friends was also widely claimed by respondents to be an effective source of introduction. They believe that their friends’ recommendations will be effective and reliable since they have already adopted them. Similarly, suggestions from family members and relatives were strongly trusted as respondents believed:

If my relatives recommend an application, definitely it is useful because they truly care for me … I will install it when I have time.

(G, Generation X, respondent or family members with chronic health problems)

Perceived quality on a digital platform is the impression of excellence that a customer experiences regarding a product through advertisements on a digital platform. Compared to generation X and generation Y, generation Z tended to use a product when it was widely and constantly advertised on social media, especially Facebook, which is the largest social media in Vietnam.59 In addition, respondents from both health groups were more willing to install an application if it was introduced on Facebook pages and groups about health. As one of the most common forms of entertainment in Vietnam, television advertisements and programs are commonly mentioned as an effective source of introduction, influencing respondents’ decisions to use mHealth applications.

Facilitating Conditions

Careful and friendly customer service plays an important role in people’s intentions to use mHealth applications. The respondents hoped to have clear and flexible instructions to familiarize themselves with the application. Moreover, an mHealth application should provide direct guidance via hotlines or chats to save time and ease of effort, especially for users who are unaccustomed to mobile phones.

I want to have someone to tell me to touch which button because I am afraid that if I touch randomly, my application will crash.

(H, Generation X, who or their family members have chronic health problems)

In addition, the respondents valued an application provider who had quick replies to questions or feedback from users and showed a willingness to improve. Furthermore, some generation Y and Z participants required the application to have a support fan page on Facebook or a website providing instant help or important news.

Technical requirements were also a major concern for the respondents when installing and using an application. Generation Z respondents demanded that mHealth applications take up small storage spaces on their devices because they needed to install numerous other applications. This group of respondents also required applications to operate smoothly without crashes. The respondents also mentioned the application’s compatibility with several different operating systems and technological products, especially generation X participants. Additionally, the respondents also paid attention to whether the applications were battery consuming as they frequently used their devices for other purposes.

Fee and payment method was a factor influencing respondents’ intention to use an application. This is similar to Price Value as a separate variable in the Unified Theory of Acceptance and Use of Technology Extension Model 2.60 However, the authors categorized this sub-theme as a factor in the Facilitating Condition. The application price was an essential element affecting many respondents as they wanted to have a free application. Generation Y respondents frequently mentioned fees as a significant concern and required low or reasonable prices for an mHealth application compared with other generations. This concern poses a significant barrier for investors and stakeholders as respondents do not usually pay for an application but prefer an application with no advertisements. Multiple respondents, especially generation Z, required mHealth applications to provide diverse payment methods because many do not possess internet banking services.

Since currently I do not have my own bank accounts, paying and transacting (for the mHealth services) is pretty problematic for me … I think pay by cash (will be easier).

(I, high school student, Generation Z, respondent or family members have chronic health problems)

The Impacts of COVID-19 on the Behavior of Vietnamese Users Toward mHealth

As mentioned in the literature review, this research was conducted during the COVID-19 pandemic; therefore, residents’ demands and attention toward mHealth applications have significant features related to this disease. First, nearly all the respondents mentioned COVID-19 once or twice during the interviews. Some participants shared that they believed mHealth applications would be beneficial in this situation and wished to take advantage of them as soon as possible. Respondents mentioned a desire for app functions related to COVID-19, such as getting official COVID-19 news, managing health declaration and vaccination plans, and providing guidance for self-care. Moreover, because respondents can diversely provide their demands for COVID-19 related apps, they pay significant attention to the problem. In general, the COVID-19 pandemic has made people care more about their health status and induce mHealth applications among people.

The Roles of Age on the Behavior of Vietnamese Users Toward mHealth

Overall, age significantly contributes to people’s intention to use an mHealth application. First, people of different age groups play different roles. Generation Y included people who raised a family, had young children, had old parents, and were primary breadwinners. Therefore, many researchers have focused on application fees and family health tracking services. Furthermore, since generation Y is the primary reproductive age in Vietnamese society, only this age group demanded sex-related applications that protect them from sexually transmitted diseases or birth control. In addition, generation Z is the age of many students and young adults. The researchers found that most people who suggested entertaining content within applications or being influenced by social media were generation Z. Second, we recognized that appearance awareness was also the main difference between generation Z and generation X. The younger generation appeared to focus more on applications that help them improve their skin and weight or make them look more attractive, healthy, and bright. However, the older generation mainly focuses on forming healthy lifestyle applications that improve their health conditions or diseases.

Third, physical health condition differences between age groups were believed to influence the Vietnamese decision to use mHealth applications. Generation X is the group of citizens who might have many health problems due to their older age; therefore, they face the highest risk of being severely affected by COVID-19. Coincidently, this was also the group that concentrated on COVID-19-related applications. Although generation Y also cared about forming healthy lifestyles, they concentrated on preventing possible diseases since their health conditions were still healthy. Finally, while technological proficiency among the three age groups was significantly different, the researchers also observed that only generation Y and Z desired applications with advanced technologies such as AI. Only generation X demanded multiple guidance methods and several needs for instructions. Therefore, technological proficiency is a factor that should be considered.

The Roles of Health Conditions on the Behavior of Vietnamese Users Toward mHealth

This research discovered that health conditions make a big difference between residents’ choices and judgments of mHealth applications. People who themselves or their family members possess chronic health problems appeared to care more about mHealth applications. Furthermore, only this group of interviewees provided suggestions about providing services for particular diseases such as diabetes, blood-pressure diseases, and osteoarthritis. A generation Z student whose father had high blood pressure disease and a generation Y busy piano teacher whose mother had diabetes mentioned the demand for mHealth. They would be very appreciative and feel secure if they could use mHealth applications to control chronic health problems of their parents and notice them if anything happens. Specifically, the chronic health problems group was the only health condition group that proposed more accurate measurement services and online pharmacies with detailed information. This might be because they have to engage with medical activities frequently. Finally, people with chronic diseases also require SOS services with single-touch mechanisms for mHealth applications and express the demands of applying AI. In contrast, people with normal health conditions only provide or require general and basic demands or technologies. It is also important to note that some interviewees in the healthy group admitted that they did not have enough health awareness since they were still fine after all.

Suggestions from Participate to Promote and Retain Customers for mHealth Applications

In our research, the researchers also asked respondents’ suggestions to promote mHealth applications. Apart from the aforementioned factors in Social Influence that can prompt the user’s intention to install a new mHealth application, there are many forms of introduction that mHealth service providers can attract users. Several respondents included advertisements from Key Opinion Leader (KOL) and artistic celebrities.

Usually with different groups of people, they will follow different (Youtube) channels. For me, I usually watch videos by Youtubers with medical content. If it (the promotion) comes from people who have experienced (using) the application, people will try to use it.

(C, medical student, Generation Z, respondent or family members do not have chronic health problems)

In addition, some respondents from generations X and Y recommended that an mHealth application be advertised by celebrities relating to health or medicine; otherwise, people will be skeptical about the quality. In addition, mHealth service providers should have campaigned for young people to go house-to-house to reach and introduce to the elderly. Moreover, mHealth services can also advertise on billboards on the roads as Vietnamese people usually ride motorbikes slowly on the streets. Further, mHealth providers can also place devices on roads for people to give mHeath services a try. In case the users need to pay for the application fee, developers should offer a free trial with access to most of the features to fully test the application. Additionally, Generation Y respondents also suggested introducing mHealth applications in large health events. Since most of them are students for generation Z, they mentioned school as a reliable source of recommendation to them.

Due to the culture of non-paid applications in Vietnam, the respondents did not prefer to pay for mHealth applications directly. However, most respondents were willing to have application fees covered by social health insurance, even though the insurance price may rise. In contrast, a few generation Y respondents rejected this idea and wanted to pay for a monthly subscription.

I want to pay subscription monthly because for example in current pandemic situation, I am not going to work so I do not pay (for the social health insurance), so I can not use the applications although this is a critical health crisis.

(J, Generation Y, respondent or family members do not have chronic health problem)

The researchers also received several suggestions to increase users’ engagement with mHealth applications. Most of the respondents emphasised personal experience when adhering to the application rather than only social influence. Many interviewees also placed the suitability of the mHealth application to fit users’ demands as an essential factor determining the engagement of the users.

If the application fits with my demands I will definitely use it and I believe other people may do so.

(H, Generation Y, respondent or family members have chronic health problems)

Due to the fact that people might get bored or lazy quickly after a particular amount of time using the applications, some respondents proposed that applications should give users scores for frequent access. Moreover, taking Duolingo as an example, mHealth applications should also have cute, sulked and trendy reminders or notifications to attract the users to open the application.

Discussion

The results are summarized and visualized on Figure 1. Regarding people’s awareness about mHealth, most participants are not aware of this application, although many of them use some mHealth applications daily. However, the majority of respondents agreed that they need the support of mHealth applications for a more convenient life. In terms of Performance Expectancy, interviewees demanded a one-stop-shop solution that offers multiple services or products to users at the same time. They hope for a precise measurement of the health index and detailed information for a better self-diagnosis experience. Moreover, the application should make personalized recommendations and possible health risk warnings but still ensure privacy protection. With Effort Expectancy, potential users prefer user-friendly app design such as more illustrations, fewer words, and use easy-to-understand language, especially for generation Y and generation X. With Social Influence, the three important factors are word-of-mouth recommendations, endorsement from credible and authoritative sources, and perceived quality from digital platforms. With Facilitating Condition, this study identifies the importance of direct guidance via hotline or chats and social media on mHealth adoption. On the technical side, the customers prefer that the apps do not consume battery or large space on the phone memory. On the financial side, they want diverse payment options and the fee is covered by insurance.

In 2018, a previous study only researched the availability and potential future of the mHealth industry in Vietnam based on customer demands.35 In 2021, studies researched the role of mHealth in mitigating the COVID-19 impacts only from medical practitioners and policy perspectives.22,30 While these studies have concluded the benefits of mHealth, their authors acknowledged the need to research Vietnamese users’ acceptance and readiness as a future research direction. Since the study had a small sample size, the study findings cannot be generalized to the Vietnamese population. However, this study’s findings shed light on the empirical gap proposed by the international literature.

This study re-confirms friendly visualization and easy-to-navigate interfaces as important dimensions of effort expectancy. Regarding social influence, credible and authoritative sources and word of mouth from close networks also contribute to the adoption of mHealth services. These findings are indifferent to pre-COVID-19 empirical studies in Vietnam and other developing countries.36,61 Hence, practitioners may invest in the user interface and experience to improve service delivery and adoption of mHealth in Vietnam and concentrate on social branding to attract potential customers. The advertisement should concentrate on the ease of user-app interaction and seek partnerships with well-known medical influencers in social media or recognition and certification by government departments.62 As these demands and behaviors are not affected by the COVID-19 situation, the authors suspect that this strategy might be transferable to the post-COVID-19 situation. Since the current study only gathers customers’ opinions, the authors encourage future research to conduct a cost-benefit evaluation or structural equation modeling (SEM) to confirm and compare the effectiveness of our suggestions on customers’ behavioral intentions and customer loyalty.

This study acknowledges new demands for mHealth services compared to previous studies. First, the author acknowledges the importance of language accessibility to ethnic groups. While the scoping study only recognized language barriers as a minor theme in their analysis, the author argues that this sub-theme must not be overlooked.35 According to the Development Policy Center of The Australian National University and United Nations Vietnam, minority and migrant groups are facing a disproportionate impact caused by the COVID-19 pandemic.63,64 As these citizens usually lack access to healthcare facilities and even the internet, developers should remove the language barriers to adopting mHealth. The authors encourage mHealth developers, the government, and policymakers to provide minority languages in their app interfaces. This consideration also follows the Vietnamese government motto: “To leave no one behind.”

Second, the scoping study mentioned that none of the participants in the 2018 study mentioned the need for privacy and confidentiality in health services.35 However, in 2021, two studies mentioned that Vietnamese users value patient privacy, but this demand is difficult to achieve with the current technology.22,30 Our study confirms the demand for privacy in the context of mHealth. Moreover, our study participants also emphasized the convenience of app usage, including the login and registration processes. The app can demotivate users if login and registration require a lengthy process, such as two-layer passwords or compulsory reading of data rights and data collections. Because users are not keen on layers of security or read data collection and usage policies, the demand for convenience and privacy creates a paradox in app development. Two potential solutions were proposed. During the open conversation, some participants suggested fingerprint or iris authentication, which has become a primary authentication process in smartphones. This process can shorten login time and ensure user authentication. Second, app developers should create short and easy-to-comprehend consent forms to collect unclassified health data from the users. Future research should investigate the design of shortened and simplified consent forms and the technical effectiveness of these methods.

During this COVID-19 pandemic, most participants look for an app that can provide personalized treatments or healthcare advice as access to physical healthcare is limited. The authors also recognized another paradox regarding user demands. While users are reluctant to allow the mHealth application to process their healthcare data and are not active in updating their medical information, they demand personalized and up-to-date information from the internet. Because the authors of this study are not Artificial Intelligence or Computer Science experts, this study encourages future researchers to find technical solutions or business models that can comply with the dual standards of users. Meanwhile, quantitative research can be conducted to investigate the extent to which Vietnamese users value personalized advice concerning their privacy. This data-driven research may guide app developers, policymakers, and entrepreneurs to tailor mHealth apps and develop effective mHealth launch strategies.

By separating the collected data into age groups and health conditions, this study adds new information to the current literature. First, the study recognizes that Gen Y are the most potential user because they are responsible for caring for their children and elders in the family. They are also financially and technologically ready to adopt the services. This finding supports the previous research in developing country context.65 Second, the study also sees that Gen Y and Gen X pay great attention to the perceived severity of their health; hence, they are more likely to adopt mHealth service. These groups also demand more convenience and ease-to-use elements of the app. However, the younger group demands hedonic values, such as gamify mHealth services, to attract their adoption. This finding also supports the previous meta-analysis in China.66 Since the previous studies are researching a quantitative approach, they cannot fully explore and customize the mHealth service according to the demands of users. Our study provides a context-rich comparison and analysis demands on each group. Moreover, since the behavior of Vietnamese users is similar to that of China and Bangladesh, Vietnamese practitioners may seek advice and follow successful mHealth practices from these countries. Still, future research in Vietnam should employ quantitative tests to confirm the behavior of each age group.

This study also responds to the recommendation for further research about the apps that connect doctors and patients from patients’ perspectives.30 Our study shows that while mHealth services for treatment are demanded more by Gen Y, Generation X, and people with chronic health conditions, all participants are interested in adopting mHealth solutions. Unlike during pre-COVID-19 time, where study recognized the mHealth apps were designed to target vulnerable and hard-to-reach populations,35 future mHealth apps must account for the behaviors and demands of all populations as the COVID-19 pandemic impacts upon them. Moreover, during 2018, they also recognize that each mHealth app typically serves a different purpose, such as raising awareness and monitoring health.30 This development is suitable for the pre-COVID-19 situations because the demand for healthcare services and healthcare information is lesser than the current time. In contrast, most participants in our studies demand a more “one-stop-shop solution” with a strong interest in health indicator measurements, personalized medical advice, and up-to-date content. Future researchers and app developers should pay attention to this new demand change in Vietnam.

Conclusions

This study employs qualitative methods to investigate the acceptability and readiness of Vietnamese residents towards mHealth application under the UTAUT framework. All participants show interest in adopting mHealth applications and demand a comprehensive health solution to protect them from the COVID-19. The study also compared the behaviors between different age groups. The most important remark is that while most Gen X and Gen Y participants demand mHealth applications to prevent the severity of their health, Gen Z demands more vivid, sociable, and gamify products.

Concerning the theoretical contribution, this research highlights a context-rich demand from Vietnamese customers across three generations and different healthcare conditions. It provides a first step for future researchers to formulate a conceptual model that targets the needs and demands of each different group. This study also addresses the gap identified by the recent literature on mHealth. First, it compares and contrasts how users’ behaviors in adopting mHealth applications differ before and during the COVID-19 pandemic. Second, the study addresses the acceptability of mHealth in Vietnam, answering the limitations of previous studies. This finding can be combined with previous studies to aid anti-pandemic policies in Vietnam, and develop mHealth solutions. This study also recognizes and proposes two paradoxical demands of Vietnamese users and the barriers to adopting mHealth services, providing directions for future marketing strategies and tech innovations.

On the practical contribution, this study suggests that mHealth developers and policymakers pay more attention to the minority ethnic group and the differences in demand of age groups and health conditions. The study observed the strong roles of family recommendations and government endorsed on the users’ adoption of mHealth service. Hence, the mHealth developers should collaborate more closely with the Vietnamese government, medical experts, and key opinion leaders on healthcare to build trust and encourage the adoption of mHealth products. Moreover, Gen Y participants have responsibility for their family health; therefore, the mHealth providers and government can target them first to increase the adoption of mHealth solutions in Vietnam.

This study also has limitations that future research should address. Firstly, although this research has reached saturation at all sub-group levels, the sample size of 50 is insufficient to represent the demands, acceptability, and acceptance of all Vietnamese populations. For instance, due to the COVID-19 pandemic, all of the interviews are conducted online or via telephone and have financial stability; hence, the study cannot study the demands and needs of Vietnamese residents who do not have access to the Internet and mobile devices. Subsequent studies should concentrate on a larger and more diverse population. Since the research is conducted via qualitative methods, the authors cannot compare the cost-benefit effectiveness of each suggestion or rank between different demands of participants. Finally, this study views Vietnamese residents’ demand, acceptability, and readiness from the UTAUT framework; hence, it oversees many variables that might influence the users’ adoption intention, such as trust or financial contribution. Future research can account for other models to develop a more comprehensive view of the topic.

Supplementary Material

The authors have provided the question guide () to increase the transparency in data collection and data analysis.