Understanding Users' Continuance Usage Behavior Towards Digital Health Information System Driven by the Digital Revolution Under COVID-19 Context: An Extended UTAUT Model.

Background: Improving the health status of users through the use of digital health information systems has drawn the attention of practitioners and academics under the tide of the digital revolution, specifically in the time of global pandemic of COVID-19, and as a result, online medical consultation has developed rapidly. Purpose: Empirical studies, however, are lacking in terms of gaining insight into use digital health information system driven by the digital revolution under COVID-19 and identifying the factors for retaining users and encouraging their continuing use. To solve this problem, this study seeks to explore the factors that influence users' intention to use digital health information system.
Methods: This study extended the Unified Theory of Acceptance and Use of Technology (UTAUT) model by introducing components of perceived risks into the model. Structural equation modeling was adopted to evaluate the research model based on an empirical survey of 241 users in China.
Results: As indicated by the results, users' continuance usage behavior to digital health information system is shaped by intention to use and facilitating conditions, with effort expectancy, social influence, perceived ease of use and perceived enjoyment exerting indirect positive effects on continuance usage behavior via intention to use. In contrast, perceived risk and perceived cost have indirect negative impact on continuance usage behavior.
Conclusion: The findings of this study can not only help the practitioners better understand the users' continuance usage behavior towards digital health information system driven by digital revolution in the time of COVID-19 pandemic and further tap into the potential market but also make up the short of traditional technology acceptance model explanatory by the extended UTAUT model.

Introduction

Mobile penetration is skyrocketing with the digital revolution, but mobile commerce is where some truly eye-popping figures can be found.1 For example, digital health information system is gaining traction in the digital revolution. Compared with the outpatient medical treatment, online medical consultation by digital health information system is not limited by time or space. As a result, users can consult health issues by using digital health APP at anytime and anywhere at will. In addition, with the outbreak of COVID-19, this enables rapid growth in online medical consultation under digital revolution. Online healthcare is ever more popular in China, which is especially significant for mobile online healthcare. In addition, in the mobile Internet environment, online healthcare’s audience is benefiting primarily the university students. Under the influence of COVID-19 pandemic, they have been confined to the campus for a long time and faced many psychological and physical problems.

Many college students frequently use their mobile phones to access online medical website or APPs. They can consult online health websites or APPs for health issues anytime, anywhere via their mobile devices. They can even use their cell phones for mobile online consultation in digital health information systems, especially the COVID-19 system, while taking a bus or on the way to class. For some college students, mobile online consultation under COVID-19 has become part of their daily life when they have health problems. Digital health information system is changing their lifestyle.

Digital health information system, as a new form of health-care mode, has attracted the attention of practitioners and academia as well, especially in the time of COVID-19. Due to the particularity of mobile Internet, there are some discrepancies between digital health information system and traditional offline healthcare. Therefore, better understanding of users’ intention to use digital health information systems driven by the digital revolution under COVID-19 context, and consequently, in-depth analysis of users’ continuance usage behavior, has emerged as a heated topic in the field of social behavior and psychology. However, there is a lack of empirical studies on this issue, especially ones made under COVID-19 context. Moreover, previous research on digital health information systems have mainly focused on the willingness to accept, with little emphasis on sustained use behavior, and research theories have been limited to the technology acceptance model. Therefore, there is a need to understand continuance usage behavior towards digital health information system driven by the digital revolution under COVID-19 context integrated the technology acceptance model and behavior theory.

Consequently, this study extended the Unified Theory of Acceptance and Use of Technology (UTAUT) model by introducing components of perceived risk into the model, and structural equation modeling was employed to assess the research model. This helps the online medical website or APP suppliers to better know the users’ behavior and propose measures to improve their service level and efficiency.

The remainder of this study is organized as follows. Literature Review is the literature review; Hypothesis and Research Model proposes hypotheses and gives the research model; Result Analysis and Discussion gives an analysis of the results; Conclusions draws conclusions.

Literature Review

The Unified Theory of Acceptance and Use of Technology

The Unified Theory of Acceptance and Use of Technology model was developed from the technology acceptance model.2 The technology acceptance model suggests that perceived usefulness, perceived ease of use, social influences and cognitive instrumental processes have significant influence on information technology adoption.3 However, the lack of consideration of human factor has led many scholars to further study and extend the TAM.4–8 The Unified Theory of Acceptance and Use of Technology (UTAUT) model has risen to the occasion.9 The Unified Theory of Acceptance and Use of Technology (UTAUT) model integrated both human and social factors and included four core determinants (effort expectancy, performance expectancy, facilitating conditions and social influence) and four control variables (age, experience, voluntariness of use and gender). In light of the above advantages, the UTAUT model is widely adopted in many cases.10–12 Therefore, this study will develop an extension of the UTAUT model to analyze users’ continuance usage behavior towards digital health information system driven by the digital revolution under COVID-19.

Using TAM and UTAUT Model to Study Digital Health Information System

Digital health information systems are effective tools for health consultations.13 Mobile online medical has become a new health service model driven by digital revolution in the time of COVID-19 pandemic. When exploring the users’ continuance usage behavior towards digital health information system driven by digital revolution under COVID-19, consideration of user perceptions requires the use of TAM and UTAUT. Yuan et al investigated the predictors of user intention to use health and fitness applications using the extended UTAUT2 model.14 Agrebi et al analyzed the acceptance of purchasing using smartphones by using an extended TAM.15 Gross et al explored the acceptance of online service technologies by using a modified TAM.16 Hassan et al used modified UTAUT to study students’ online behavior.17 Celik et al investigated the influence of anxiety on the customer adoption of online service based on the UTAUT.18

TAM is one of the most frequently used theoretical frameworks. Beldad and S. M. Hegner proposed the influence of the social influence, health valuation, and inclusion of trust in using a fitness App on customers’ attitudes and behavioral intentions towards usage with the expanding the TAM.19 Gu et al studied the main mechanisms of how the features and users’ experiences by using health services under mobile Internet environment influenced patient satisfaction and continuous use behaviors of the system.20 Gu et al analyzed the factors influencing sustained engagement with Social HMS by combining the big-five theory and the theory of planned behavior with the trust theory.21 Akroush et al used an integrated model to examine factors affecting attitudes and intentions toward mobile APPs.22 Huang G. and Ren Y. built an extended TAM to analyze the intention towards using of fitness mobile apps.23 Schomakers et al applied the UTAUT2 model which trust and privacy concerns are added to assess health APPs.24 Boluwaji et al adopted the UTAUT model to study the factors influencing users to accept COVID-19 digital tackling technologies.25

Prior studies have analyzed the user’s willingness of accepting digital health information system or online medical by using TAM and UTAUT and revealed that several factors, such as self-efficacy, effort expectancy and performance expectancy, influence users’ behavior and intention. These findings can be regarded as contributions to understanding the user’s willingness of accepting digital health information system or online medical. However, few studies involved users’ behaviors and intentions towards digital health information system and online healthcare driven by digital revolution in the time of COVID-19 pandemic. The digital health-care information system driven by the digital revolution under COVID-19 possesses new characteristics. Therefore, the factors affecting users’ behavior and intention are different compared to traditional offline healthcare. In addition, perceived risk also affects users’ behavior and should be considered as a factor. Therefore, there is a need to study the continuance user behavior towards digital health information systems driven by the digital revolution under COVID-19 context in the context of TAM or UT.

Hypothesis and Research Model

Hypothesis

From Venkatesh’s perspective, performance expectancy has a positive bearing on the intention to use information technology.9 Performance expectancy is similar to short-term usefulness and is an essential predictor of an individual’s intention to use a new technology.6 Also, performance expectancy influencing users’ intention to use mobile Internet has been certified.26 This positive relationship was also proved in consumer studies.27 Digital health information system is the application of a new technology. Users will be willing to continue using digital health information systems under COVID-19 context if they find it helpful to them. Consequently, from the above analysis, the following hypotheses can be made.

H1: Performance expectancy (PE) has a positive impact on users’ behavior intention to use digital health information system driven by the digital revolution under COVID-19 context.

The effort expectancy refers to users’ comfort and acceptance in using a new technology.28 Effort expectancy directly influences users’ continuance intention to use a new technology.29 Nysveen and Pedersen’s study also proved this point.11 Consequently, from the above analysis, the following hypotheses are made.

H2: Effort expectancy (EE) has a positive impact on users’ behavior intention to use digital health information system driven by the digital revolution in the time of COVID-19.

Social influence (SI) is an individual perception that important others believe the person should adopt the new technology.9 Social influences can positively affect the intention to use a new technology.30 Moreover, Venkatesh’s research suggested that social influence contributes positively to users’ intention to use online service.26 If a user strongly recommends others to use a digital health information system, the recommended person may give it a go. Consequently, from the above analysis, the following hypotheses are made.

H3: Social influence (SI) has a positive impact on users’ behavior intention to use digital health information system driven by the digital revolution under COVID-19 context.

Facilitating conditions is the ease with which a new technology is used by the user.31 Venkatesh has demonstrated that facilitating conditions have a positive impact on both usage of mobile service and users’ intention to use them.26 Lai’s study also supported this positive relationship.32 Therefore, from the above analysis, the following hypotheses are proposed.

H4: Facilitating conditions (FC) has a positive impact on continuance usage behavior towards digital health information system driven by the digital revolution under COVID-19.

Privacy concern is very important factor influencing whether people use mobile services in the present information age, especially driven by digital revolution in the time of COVID-19. Many scholars have studied privacy issue and its influence on new technology.33 Some previous studies also indicated that privacy concerns negatively influence users’ perception towards online service.34,35 Accordingly, drawing from the above analysis, the following hypotheses are made:

H5: Privacy concern (PC) has a negative impact on users’ behavior intention to use digital health information system driven by the digital revolution under COVID-19 context.

The concept of perceived risk was proposed by Cox and Rich which refers to the individual’s feeling of subjective certainty of success or failure (uncertainty) and the amount at stake (consequences).36 A study by Tao Zhou backed up the idea that privacy risk can negatively affect intention to use online services.37 Digital health information system driven by the digital revolution under COVID-19 is online medical in nature. The users may be disappointment with online service when it does not meet their expectations.38,39 Therefore, from the above analysis, we propose the following hypothesis.

H6: Perceived risk (PR) negatively impact on users’ behavior intention to use digital health information system driven by the digital revolution under COVID-19.

Perceived usability is influenced by users’ confirmation level in the B2C electronic commerce.40 It has been shown that perceived usability significantly influences users’ behavior intention to use a new technology.6,9,29,41 Digital health information system belongs to B2C M-commerce services; therefore, under COVID-19 context, perceived usability relates positively to users’ use of digital health information systems driven by the digital revolution. Motivated by the above analysis, the following hypotheses are proposed.

H7: Perceived usability (PU) has a positive impact on users’ behavior intention to use digital health information system driven by the digital revolution under COVID-19 context.

Perceived ease of use is how easy it is for users easy to find the service, then make a decision, and access the service.42 Perceived ease of use is an important factor affecting the application of a new technology.2,6 When users feel that digital health information system is easy to use, they may adopt it. There is no doubt that perceived ease of use will influence users’ intention and behavior to digital health information system, especially in the time of COVID-19. Consequently, from the above analysis, the following hypotheses are made:

H8: Perceived ease of use (PEU) has a positive effect on users’ behavior intention to use digital health information system driven by the digital revolution under COVID-19.

Perceived cost refers to the cost that one person perceives to adopt a new technology or use a new service.41 Perceived cost resources is an important factor influencing users’ intention to use an information system.2,6,43 As a matter of fact, when users feel that using digital health information system wastes money and time or they need to expend too much money on the mobile device and online consultation, they may be giving up for digital health information system. Therefore, the above analysis leads us to the following hypothesis.

H9: Perceived cost (PCO) has a negative impact on users’ behavior intention to use digital health information system driven by the digital revolution under COVID-19.

Perceived enjoyment is perceived to be the enjoyment that users feel when using a new technology.44 Some scholars proved that perceived enjoyment is a significant factor on behavioral intention to use a new information technology.45 Users’ behavior in using online service in the mobile internet environment may be significantly influenced by perceived entertainment.46 If users feel enjoyable when they carry on digital health information system, they will take this new form of medical treatment and recommend it to others. Consequently, from the above analysis, the following hypotheses are made:

H10: Perceived enjoyment (PEN) is found to positively influence users’ behavior intention towards usage of digital health information system driven by the digital revolution under COVID-19 context.

Many studies have reviewed the relationship between intention to use and sustained use behavior.47 The intention to use has an important impact on continuance usage behavior was proposed by some scholars.29,40,43 When users have the intention to use digital health information system driven by digital revolution under COVID-19, it will promote their continuance usage behavior towards digital health information system driven by digital revolution under COVID-19. As a result, the above analysis leads us to the following hypothesis.

H11: Intention to use has a positive influence on continuance usage behavior towards digital health information system driven by the digital revolution under COVID-19.

In summary, the research model proposed in this study is shown in Figure 1.

Figure 1

An extension of the UTAUT model.

Research Model

Why Use Structural Equation Model

How to understand users’ continuance usage behavior towards digital health information system is inferential problem. There are many factors affecting users’ choice of mobile online healthcare, especially driven by digital revolution in the time of COVID-19. However, structural equation model is well suited for data analysis for inferential purposes. Structural equation modeling is capable of either assessing or correcting for measurement error compared with other multivariate procedures. In addition, both unobserved and observed variables can be measured by structural equation modeling.48 Finally, structural equation modeling can be highly effective for studying research problems involve non-experimental research. Therefore, in this paper, structural equation modeling is used to explore the users’ continuance usage behavior towards digital health information system driven by digital revolution.

Data Collecting

This study designed questionnaire to survey the users’ usage behavior towards digital health information system under COVID-19 context and verify the hypotheses. Both face-to-face survey and online survey were conducted in this study. The questionnaire consisted of 51 items and used a 7-point scale to measure the variables, among which, 1 represents “strongly disagree” and 7 represents “strongly agree.” The majority of the questionnaires were distributed to the user via online methods. Face-to-face interviews were also adopted to ensure that the sample was random and comprehensive.

Sample Size

The sample contained 241 young people, of whom Chinese university students made up the largest proportion. The collection period was from 2021 to 2022 and the effective sample was 212 (156 online, 85 offline, and 29 invalid). The total efficiency of the questionnaire was 87.97%.

Validity and Reliability Testing

This study applied Cronbach’s alpha to verify reliability of the sample data. The results of this model indicated total Cronbach α coefficients were 0.945 and the reliability was secured. Factor analysis was used to verify the validity of the sample data. The results of this model indicated the value of KMO was 0.823. This means the data is valid and can be used for subsequent analysis.

Data Modeling Based on Structural Equation Model

Structural equation modeling was used to fit the data. The structural equation modeling of this study showed in the Figure 2, and will be analyzed by IBM.SPSS.Amos.v22.

Figure 2

The structural equation modeling of this study.

Here, e4 to e60 stand for Error Variable. PE1 to PE3 are observed variable of Performance expectancy (PE). EE1 to EE3 are observed variable of Effort expectancy (EE). SI1 to SI3 are observed variable of Social influence (SI). PC1 to PC4 were observed variable of Privacy concern (PC). PR1 to PR5 were observed variable of Perceived risk (PR). PU1 to PU4 were observed variable of Perceived usability (PU). PEU1 to PEU4 are observed variable of Perceived ease of use (PEU). PCO1 to PCO3 were observed variable of Perceived cost (PCO). PEN1 to PEN4 were observed variable of Perceived enjoyment (PEN). FC1 to FC3 were observed variable of Facilitating conditions (FC). W1 to W4 were observed variable of Intention to use. CB1 to CB5 were observed variable of Continuance Usage Behavior.

Result Analysis and Discussion

Descriptive Statistics

Descriptive statistics analysis of the sample is given in Table 1.

Table 1

Descriptive Statistics of Sample (N = 212)

Classification	Statistics	Count	Percent
Gender	Male	96	45.28%
	Female	116	54.72%
	Total	212	100%
Age	≤18	5	2.36%
	>18 and ≤20	38	17.92%
	>20 and ≤23	116	54.72%
	>23	53	25%
	Total	212	100%
Daily Using mobile Internet time	≤1hour	0	0
	>1hour and ≤2hour	6	2.83%
	>2hour and ≤3hour	37	17.45%
	Beyond 3hour	169	79.72%
	Total	212	100%

As can be seen from the table, female users account for 54.72% and the proportion male users is 45.28%. Respondents’ age was concentrated in 20 to 23 years old, which accounts for more than half of the respondents. Most users spend more than 3 hours a day using cell phone to access the Internet, making them ideal candidates for research. Based on the above statistics, this study can be considered that the sample has a certain representative, and can be used to study continuance usage behavior towards digital health information system driven by the digital revolution under COVID-19.

Model Fit

Model fit is very important to a model. The better the fit of the model, the higher rationality of the model. In this study, the following index including to evaluate the model, as shown in Table 2.

Table 2

Index of Model Fitting

Index	The Full Name of Index
CMIN/DF	Minimum discrepancy/degrees of freedom
P	Probability level
GFI	Goodness of fit index
AGFI	Adjusted goodness of fit index
RMSEA	Root mean square error of approximation

According to the calculations, the evaluation result of the initial model (CMIN/DF=5.438, P≤.001; GFI = 0.73; AGFI = 0.67; RMSEA = 0.09) indicate that the model does not fit well. Therefore, the model needs to be modified. According to the modification indices, some covariances are repeated in this paper as free parameters. The optimized results indicate the model fits the data reasonably well and the model is acceptable (CMIN/DF = 2.438, P≤.001; AGFI = 0.92; GFI = 0.95; RMSEA = 0.046).

Model Testing Results

The results of estimates are in Table 3.

Table 3

Estimated Results

Hypotheses	Estimate	P value	Significance
H1	0.59	0.046	Yes
H2	0.10	0.028	Yes
H3	0.13	***	Yes
H4	0.23	***	Yes
H5	0.02	0.701	No
H6	−0.17	0.043	Yes
H7	0.02	0.733	No
H8	0.12	0.048	Yes
H9	−0.19	***	Yes
H10	0.22	***	Yes
H11	0.41	***	Yes

Note: Asterisks (***) means P value <0.001.

As can be seen from Table 3, the P-value of H5 and H7 are more than 0.05, which means that Privacy concern (PC) and Perceived usability (PU) are not significant with intention to use digital health information system. Some of P value which are *** stand for the P-value is less than 0.001.

Discussion

Summary

According to the above analysis, all hypotheses were tested in this study, and nine hypotheses were supported. Some interesting results are discussed below and the corresponding recommended measures are proposed.

Performance expectancy: the major drivers of continued digital health information system driven by the digital revolution under COVID-19 usage behavior across the model

As can be seen from Table 3, among all factors, performance expectation has the greatest influence on users’ intention to use digital health information system driven by the digital revolution under COVID-19 context. Although the previous researches have shown that social influence positively affects users’ intention to use a new technology, few studies have demonstrated that performance expectancy is the major driver of continued usage behavior. The possible reason for this point is that users may think the extent to which digital health information system will benefit them. The users want to be healthy, so there are reasons to expect performance expectancy will have a great influence on their intention to use digital health information system driven by digital revolution. This means the users’ behavior towards digital health information system is affected greatly by their expectations. They will continue to use digital health information systems if they believe that they will benefit their health, especially in the time of COVID-19.

Powerful and Sustained Influence of Facilitating Conditions and Perceived Enjoyment in the Model

As can be seen from Table 3, facilitating conditions exert an indispensable part in the model which was verified. This is somewhat different from previous research. This may be attributable to the nature of digital technology. Along with the development of digital technology, new technologies, such as digital medicine and teleconsultation, have been introduced into the medical field. Users need to learn these new digital medicine technologies so that they can enjoy the convenience of digital revolution. Therefore, facilitating conditions for digital health information system become more and more essential for their continuance usage behavior. Meanwhile, perceived enjoyment is another factor which has been confirmed in this study. Therefore, the quality of digital medicine and the convenience of mobile terminal will directly affect the users’ behavior towards digital health information system driven by digital revolution under COVID-19.

Perceived Cost and Perceived Risk as Negative Factors

According to Table 3, this study also verified that perceived risk and perceived cost are negative factors of digital health information systems. This is because users are concerned about their private information being exposed when using digital health information system or conducting online consultations. Meanwhile, users are also very sensitive to the cost of online consultations. Once they feel that online health-care services are higher, they may drop using the service.

Conclusions

The objective of the current study is to explore and analyze factors influencing users’ continuance usage behavior towards digital health information system driven by digital revolution under COVID-19 context and further explains why some users will give up using digital health information system via mobile phone, which are not well explained in prior research, especially since digital health information system were adopted following the outbreak of COVID-19. In order to better explain this problem, this paper built a UTAUT model which integrated the theory of perceived risk and tested the model through a survey of 241 users in China. The main conclusions and specific strategies of this paper are presented below.

(1) The factors of users’ intention to use digital health information system under COVID-19 context can be understood from seven dimensionalities, which have been touched upon in the model. Therefore, service providers of digital health information systems should deploy and maintain multiple service to reach the users according to the seven dimensionalities. For example, they should improve the online medical APP and make it more friendly and more convenient to users so as to retain and attract more users.

(2) This paper found the performance expectancy, effort expectancy, social influence, perceived enjoyment and perceived ease of use has a positive impact on users’ intention to use digital health information system driven by digital revolution under COVID-19 context. Among all these factors, performance expectancy is most prominent. This means that the higher the users’ perceived performance expectancy, the stronger their intention to use digital health information system.

For this point, medical suppliers should provide better online medical services to fit the users’ performance expectancy. On the other hand, the digital service operators should further tune their networks to allow more people habitually use digital health information system. In addition, perceived risk and perceived cost as negative factors have been validated. Aimed at this, the digital health information system supplier should enhance the privacy and security and ensure that users’ personal information is not disclosed.

(3) Users’ continuance usage behavior towards digital health information system depends on two factors: intention to use and facilitating conditions. They both positively influence users’ continuance usage behavior towards digital health information system driven by digital revolution in the time of COVID-19. Pointed at this, the digital terminal manufacturer should further improve the digital terminal’s maneuverability. In addition, the digital health information system supplier should provide more convenient services to meet users’ need and further stimulate their willingness to use them.

The contribution of this paper is its successful use of the extension UTAUT model to study users’ continuance usage behavior towards the digital health information system, which is different from the prior studies. Meanwhile, this study extends its generalizability to digital health information systems by validating the direct relationship between “Intention to Use” and “Continuance Usage Behavior” in the field of digital health information systems driven by the digital revolution under COVID-19 context. In short, the findings of this paper can not only help the online medical provider to pursue strategies that increase digital health information system usage, but can also provide key insights for the social behavior research in the field of digital transformation.

Future studies should also consider the changes of digital technology, as it is updated at a very fast rate. With the revolution of information technology, some new digital devices and digital technology will emerge, which will exert influence on people’s behaviors in using digital health information system. Besides, there are some shortages in the sample. The study concentrates on the youth, especially university students. Different demographics may result in substantially different results. Moreover, among young people, some may not use digital health information system regularly, or even never used it before. Hence, the results may be partly based on potential rather than actual digital health information system in the digital revolution under COVID-19 context, and this may lead to measurement error somehow. All the above issues would need to be further studied in future research efforts.