Indonesian hospital telemedicine acceptance model: the influence of user behavior and technological dimensions.

Since the COVID-19 pandemic hit, several hospitals in Indonesia have started to develop teleconsultation services to expand the reach of health services and prevent the spread of the COVID-19 virus. Therefore, this study was conducted with the aim of analyzing the influence of user and technological dimensions on the acceptance of the use of hospital teleconsultation applications by users. The research was conducted using quantitative methods with data from questionnaires distributed online. The respondent criteria for this study were Indonesians aged 17 years and over who knew about or had used a hospital's teleconsultation application, and this resulted in 534 respondents. Data processing was carried out using the covariance-based structural equation modeling method with the AMOS 24 application. The user behavior and technology dimensions are influence intention to use telemedicine applications. Furthermore, the results showed that contamination avoidance, safety, reliability, professionalism, perceived ease of use, perceived usefulness, and information quality have a significant positive influence on the behavioral intention to use hospital telemedicine applications. Meanwhile, facilitating conditions and social influence were not shown to have a significant positive effect on the behavioral intention to use such applications. Finally, behavioral intention to use significantly and positively influences actual use. This research can help hospitals and the government in providing guidance in terms of developing telemedicine applications.

Introduction

Currently, many countries are struggling to cope with the COVID-19 pandemic, and many people are afraid to go to health facilities. One solution is to assist patients in performing non-emergency treatments using telemedicine. Indonesia has 1,392,945 cases of COVID-19 as of April 2021 (Coronavirus Update, 2021). Based on Soenarso (2020), 71.8% of respondents said they had not visited health facilities, such as a clinic or hospital, since the start of the COVID-19 pandemic because these places had the potential to spread the coronavirus. Similar results were found in Yuswohady et al. (2021), where 80% of 1,121 respondents said they were afraid to go to a hospital for fear of being exposed to the COVID-19 virus.

According to the World Health Organization, telemedicine is defined as the provision of remote health services by professionals using information and communication technology (WHO, 2010). Telemedicine services consist of teleradiology, teleectrocardiography, teleultrasonography, clinical teleconsultation, and other services (WHO, 2010). During the COVID-19 pandemic in Indonesia, the switch from face-to-face to online consultations led to a 600% surge in the use of telemedicine applications, including both those developed by electronic health startups and hospitals (CNN Indonesia, 2020). Until April 6, 2020, the number of people accessing such telemedicine applications for information about the coronavirus reached 15 million (Ashar, 2020).

Even though technology startups' telemedicine applications have proven to be popular, the public still feels that hospitals must provide these services (Yuswohady et al., 2021). The public puts more trust in telemedicine services provided by hospitals compared to those provided by technology companies because, based on the Indonesian Minister of Health Regulation No. 20 of 2019, institutions that are officially categorized as telemedicine providers are hospitals owned by the central government and local governments and private health facilities. Furthermore, based on Yuswohady et al. (2021), 71% of respondents considered hospitals to have better health service ecosystems, both in terms of their facilities and the expertise of their health workers. In addition, the community recognizes that hospital services are safer and more reliable because their quality has been tested (Katadata, 2021). Hospitals in Indonesia that have developed such a service include Cipto Mangunkusumo Hospital with the SiapDok application, Siloam Hospital Group with the AIDO application, and YARSI Hospital with the MAUDOK application.

Even though 20% of hospitals in Indonesia already have clinical teleconsultation services, they are still far from meeting expectations because they have not been able to compete with the popularity of hospital clinical consultation services abroad (Katadata, 2021). Although the popularity of hospital telemedicine and teleconsultation services is still lacking compared to telemedicine services developed by technology companies because of their limited features, hospitals that provide telemedicine services outperform technology companies in terms of their credibility, licensing, and health facilities. In addition, unlike platforms that are not allowed to manage patient-owned medical records (Harjono, 2020), hospitals as health service providers have the right to manage such information. The quality of health services can also be controlled more easily and to a better standard because all health workers providing telemedicine and teleconsultation services are under a hospital's auspices, and any medical records in the hospital's teleconsultation application will be integrated with the medical record database of the hospital.

Zhou et al. (2019) analyzed the factors that influence the behavioral intention to use telehealth. The results show that the relationship between telehealth systems and physical medical services is seen as mutually reinforcing and that they are not observed to be alternatives or substitutions for one another. Zhou et al. (2019) focused more on telehealth acceptance factors in terms of socio-economic and technical aspects, as well as on the relationship between behavioral intentions regarding telehealth systems and satisfaction with medical services, but did not analyze the actual use factors of telehealth. Moreover, Baudier et al. (2021) analyzed the perceptions of patients in Europe and Asia regarding teleconsultations during the COVID-19 pandemic. The results showed that performance expectancy and habit factors directly influence intention to use, while social influence and effort expectancy factors are proven to have no effect on intention to use (Baudier et al., 2021). This study introduces a new scale for contamination avoidance factors, which are based on concerns about exposure to virus contamination during the COVID-19 pandemic (Baudier et al., 2021). Kamal et al. (2020) analyzed the reception of telemedicine services in rural areas of Pakistan. The results show that the variables of perceived usefulness, perceived ease of use, social influence, facilitating conditions, and trust were the main factors affecting the reception of telemedicine services (Kamal et al., 2020). However, Kamal et al. (2020) did not review psychological parameters, such as the habits or cultural factors of patients, which could explain the adoption of telemedicine services in developing countries.

As of yet, no study has analyzed the user acceptance of telemedicine applications in the context of applications developed by hospitals, specifically in Indonesia. In fact, Yuswohady et al. (2021) stated that consumers hope that hospitals will provide telemedicine services, even though many health startups are starting to experience a boom. As many as 71% of the 441 respondents stated that they have more confidence in hospital teleconsultation applications compared to teleconsultation applications owned by technology startups, such as Halodoc, Klikdokter, and Alodokter (Yuswohady et al., 2021). This study combines the technology acceptance model (TAM) with several other factors such as user behavioral and technological dimensions that can affect Indonesian society's acceptance of hospital teleconsultation applications and relates these to the actual use factor to analyze whether there is a significant influence of the intention to use factor on the actual use of hospital teleconsultation applications. The TAM has been used in various health fields to assess users' propensity to accept technology (Yu et al., 2009).

Therefore, this study aims to analyze the influence of the user dimension and the technology dimension on user acceptance of hospital telemedicine applications. This study can be used as a reference for hospitals when analyzing the advantages and disadvantages of their teleconsultation services compared to other similar services. The results of this study are also expected to be of benefit to the government in its role as a regulator. The government is expected to improve the existing regulations regarding telemedicine and implement new measures to strengthen the legal basis for telemedicine in Indonesia.

Literature review and hypotheses

Telemedicine

Telemedicine or telehealth uses information communication and technology (ICT) for the exchange of information during the diagnosis and treatment of diseases and injuries and research and evaluation, and for the continuing education of health professionals (WHO, 2010). According to Aziz and Abochar (2015), in terms of its implementation, telemedicine is divided into two types: store-and-forward telemedicine and real-time telemedicine. Store-and-forward telemedicine or telemedicine services are provided asynchronously so that the data exchange process can be carried out even though the sender and receiver are not present at the same time (Aziz and Abochar, 2015). One example is an X-ray of a patient that is sent to a health professional via email (Aziz and Abochar, 2015). Real-time telemedicine or synchronous telemedicine services are defined by Aziz and Abochar (2015) as telemedicine services that require health workers and patients to interact at the same time interactively, such as online health consultation services conducted via video.

In Indonesia, hospitals wishing to provide telemedicine services must meet the standards and conditions as stated in the Indonesian Minister of Health Regulation No. 20 of 2019 concerning the implementation of telemedicine by health facilities. Since the COVID-19 pandemic hit, hospitals have started providing telemedicine services to expand the reach of health services for patients throughout Indonesia. However, telemedicine service platforms have also been developed by technology companies since 2016, including the Klikdokter, Alodokter, and Halodoc applications. Unlike hospitals, telemedicine application development technology companies are not registered as telemedicine service providers according to the law, and they only act as a platform to bring together health service providers from various hospitals with patients throughout Indonesia. Doctors registered with the Indonesian Doctors Association and affiliated with any hospital can register as health providers on these platforms. The aim of technology companies in developing telemedicine services is none other than to facilitate the search for health services for patients, through which patients can consult online with doctors, buy medicines from pharmacies that have become partners of the technology company, and access services regarding health issues, such as registering for the administration of vaccines or immunizations. The health service platform can carry out health consultations or counseling and because the platform provider does not have a doctor-patient relationship with the user, the platform is not allowed to perform medical services and manage patient-owned medical records (Harjono, 2020). In addition, platform provider technology companies are not responsible for the negligence of health service providers (Halodoc, 2021) and, rather, only for data security in their role as electronic platform providers (Harjono, 2020). This is different from the responsibilities that a teleconsultation service provider has, and a hospital is responsible for everything from data security and the medical services provided by doctors affiliated to the hospital to patients' medical records.

Research hypotheses

The model for this study is to modify the TAM and Kamal et al.’s research (2020), which analyzes users' acceptance of telemedicine, particularly teleconsultations, in developing countries. Because the research was conducted after the COVID-19 pandemic spread to Indonesia, we modified several existing variables by mapping several factors from previous studies that had similar meanings. We modified the TAM variables only using perceived ease of use, perceived usefulness, behavioral intention, and actual use variables. Then, because not many previous studies have reviewed the relationship between the behavioral intention to use teleconsultation applications and actual use, we added the actual use factor from Venkatesh et al. (2003). Based on this explanation, the research model compiled consists of 11 exogenous variables, 2 endogenous variables, and 48 indicators. The existing variables are grouped into two, namely the user dimension and the technology dimension. The user dimension includes everything that is directly related to the user, both from the health worker as a service provider and the user. Meanwhile, the technology dimension includes software and hardware used to provide health services. The total number of hypotheses contained in this model is 10. The proposed model can be seen in Figure 1.

Figure 1

Proposed model.

Facilitating conditions are defined as the extent to which a person believes that an organizational and technical infrastructure exists to support the use of a system (Venkatesh et al., 2003). Facilitating conditions relate to the availability of sufficient resources and support for individuals in terms of using technology (Neslin and Shankar, 2009). In addition, limited resources can prevent individuals from accepting technology (Kamaghe et al., 2020). This construct is a significant predictor of the intention to use an information and communication technology, where if individuals believe in the existence of a support system to help them, their intention to use the mobile health application will be greater (Yi et al., 2006; Bhattacherjee and Hikmet, 2008). In the Indonesian context, a study conducted by Ardiansyah and Rusfian (2020) shows that facilitating conditions are one of the factors that affect the user acceptance of telemedicine. Knowledge concerning how to use telemedicine technology will reduce doubts and increase user habitability (Ardiansyah and Rusfian, 2020). Departing from this, we intend to test whether there is a significant effect between facilitating conditions and a user's intention to adopt a specific hospital's teleconsultation application. Phichitchaisopa and Naenna (2013) stated that one of the factors that has a significant impact on the adoption of mobile health applications is facilitating conditions, and another study conducted by Wang et al. (2021) supports this. The research results of Wang et al. (2021) stated that facilitating conditions had a significant positive impact on patients' behavioral intention to use online hospital services. Thus, we propose the following hypothesis:

According to Venkatesh et al. (2003), social influence (SI) is the extent to which a person believes that other people, especially their acquaintances and friends, believe that the person in question must use a certain system. A study conducted by Kijsanayotin et al. (2009) predicts patient behavior in terms of the acceptance of health information and communication technology, and the results of the study suggest that peer and colleague opinions have a strong influence on user behavior. The acceptance or rejection of a technology by patients also depends on the extent to which household nurses, their children, or their grandchildren urge them to use it (Or and Karsh, 2009). Positive social support for using technology will increase user acceptance and the intention to use a technology (Venkatesh et al., 2003). In addition, in developing countries, people often live together with extended families and depend on one another in various social and economic contexts (Kamal et al., 2020). Moreover, the percentage of elderly individuals living in large families has reached 63% (Katadata, 2018). The extended families in question consist of three generations, namely the elderly who live with their children/children-in-law and grandchildren, or with their in-laws, in one household (Katadata, 2018). The use of telemedicine services in an environment like this will not only be observed by family members, neighbors, and acquaintances of the person in question, but the opinions that these people then have can also encourage or prevent users from taking advantage of telemedicine services (Kamal et al., 2020). The hypothesis that SI has an impact on behavioral intention to use is then put forward based on what has been described above.

Since the emergence of new diseases in modern society, such as AIDS and Ebola, many researchers have tried to understand the social impact these diseases have on individuals (Kamal et al., 2020). Humans have developed a behavioral immune system that prompts them to change their behavior in response to signs of contamination (Curtis et al., 2011). Contamination problems can affect user experience (Klaus and Manthiou, 2020) as well as user perceptions and behaviors, such as looking for clean, familiar, or new products and services (Griskevicius and Kenrick, 2013). Thus, contamination avoidance can be defined as the extent to which consumers tend to avoid contact with other people or places for fear of being contaminated (Kamal et al., 2020). During the COVID-19 pandemic, one place considered to have a high level of contamination is medical facilities (Hazée and Van Vaerenbergh, 2020). This was observed in Indonesia, where a hospital cluster contributed the highest number of COVID-19 cases compared to other clusters (CNN Indonesia, 2020). Patients who have the need to consult with medical professionals during the pandemic may feel that they are in danger (Moroni et al., 2020) and start looking for other alternative ways to obtain medical services to avoid the risk of contamination in a hospital environment; one such approach is through teleconsultation. Therefore, we propose a hypothesis that states that contamination avoidance will affect behavioral intention to use.

The safety factor is a basic requirement of medical care for patients. There are many factors that can affect a patient's sense of security, including experienced medical and nursing teams, and hospital security measures (Koch, 2006; Gore et al., 2009; Hsieh et al., 2015). In the context of this research, the safety factor refers to how safe the facilities, human resources, and medical services of a hospital are. The Patient Safety Culture Survey (HSOPS), which is a survey designed to assess the perceptions of doctors and staff regarding safety culture in hospitals (HSOPS, 2013), provides useful tips for investigating patient safety perceptions, both in terms of hardware and software (Gore et al., 2009; Hsieh et al., 2015). Safety in terms of hardware refers to medical equipment and hospital medicines, while safety in terms of software includes medical services provided by doctors and nurses (Zhou et al., 2019). Emergency assistance is also an important factor that affects patients' perceptions of safety (Zhou et al., 2019). According to Yuswohady et al. (2021), the COVID-19 pandemic has made consumers re-prioritize basic needs, namely clothing, food, shelter, safety in terms of health, and security. Cleanliness, health, and safety, and environment concerns are the main priorities of consumers during a pandemic (Yuswohady et al., 2021), and this shows that the safety value of health facilities and services provided by hospitals with teleconsultation applications is a factor that can influence patients' desire to use the applications in question. Therefore, we formulate a hypothesis that predicts that safety influences behavioral intention to use.

Parasuraman et al. (1985) define reliability as the ability to perform the promised services reliably and accurately. In the health context, reliability is defined as the level of trust that patients have in telemedicine technology, health care professionals, and service quality (Saigi-Rubio et al., 2016). Telemedicine services mostly rely on the use of the internet, telecommunication systems, and cellular devices, so reliability from a technical point of view is important. Reliability affects the level of use because users of telemedicine will lose confidence in the system if they cannot use it easily (Schultz, 2018). Apart from reliability from a technical point of view, one factor that influences patient confidence is the success of the health services provided (Lin, 2017), based on the accuracy of the results of online diagnoses. The accuracy of such results is important in the choice of health services due to the existence of a lifetime threat in the event of a wrong treatment step (Alam et al., 2018). The higher the reliability of the application and the diagnostic results of the hospital's teleconsultation application, the higher users' acceptance of the application. Therefore, we hypothesize that reliability has a significant impact on behavioral intention to use.

According to George et al. (2009), one factor that affects patients' trust in teleconsultation services is the identity of the doctors and their qualifications. The professionalism of health workers refers to their core knowledge, basic skills, interpersonal skills, and lifelong learning, as well as their ability to apply their knowledge when providing health services (Epstein, 2002). Based on Yuswohady et al. (2021), 71% of the 441 respondents admitted that one of the reasons they believed more in a hospital's teleconsultation services was from an experiential point of view of health workers affiliated with the hospital. The results of Yuswohady et al. (2021) study indicate that the professionalism of hospital teleconsultation service providers is also an important factor that needs to be studied because, in the context of health services, the intention to use teleconsultation services will increase if doctors have proven their identity and professionalism.

Perceived ease of use is defined as the extent to which the use of a system is free from additional efforts (Adams et al., 1992). This factor, initially identified by Davis (1986), has been shown to have a significant effect on intention to use by many studies (Lin, 2017). In addition, efforts are needed to help patients gain access to telemedicine services and to understand and learn how to use this new technology (Lin, 2017). In Indonesia, several previous studies have proven that there is a significant relationship between perceived ease of use and the intention to use e-health (Manda and Salim, 2021). The less effort users have to put into using e-health, which includes telemedicine and teleconsultation services, the more positive their behavioral intention to use the system will be. Therefore, we formulate a hypothesis that states that perceived ease of use affects behavioral intention to use.

Davis (1993) defines perceived usefulness as his/her perception that the use of a new technology will improve their performance. In the context of health and telemedicine, Kamal et al. (2020) redefined the perceived usefulness factor because the usefulness of the system for patients has a different meaning and does not match the actual definition of perceived usefulness. Patients feel that adopting telemedicine services will only be useful if telemedicine services provide faster health services, lower medical examination costs, better documentation, and reduced waiting times (Kamal et al., 2020). In the context of e-health in Indonesia, this statement is supported by Manda and Salim (2021) regarding the user acceptance of teleconsultations, where perceived usefulness has been shown to have a significant effect on behavioral intention to use. If a hospital's online health application can fulfill user expectations, users' intention to use will automatically increase. Thus, we propose the following hypothesis:

Information quality is a multi-dimensional concept, with its most widely adopted definition referring to fitness for use (Neely and Cook, 2011). Information that is deemed appropriate for one user may not be sufficient for another user, and this thus depends on the context of use, where the subjectivity component will influence perceived usefulness (Hausvik, 2017). Information quality includes concepts such as completeness, ease of understanding, and the relevance of the information (Lau et al., 2007). According to Zhou et al. (2019), information quality is an important factor in determining the competitiveness of a telehealth system. The accessibility of medical records and advice from health professionals have a significant positive impact on patients' acceptance of the telehealth system (Rho et al., 2014). The quality of information can influence the extent to which the system is used, the intention to use the system, and user satisfaction with the system (Lau et al., 2007). Currently, apart from diagnoses from health professionals, health services provided through telemedicine are recorded in the medical records of the relevant hospital (Mediatama, 2020). Therefore, we propose another hypothesis:

Behavioral intention is defined as an individual's goal or plan to carry out a certain behavior (Hill et al., 1977). In the context of this study, behavioral intention to use is an individual's goal or plan to use a hospital's teleconsultation services. According to the TAM, the actual use of a system depends on the intention to use it (Davis et al., 1992). Research conducted by Hoque et al. (2016) showed that there is a relationship between patients' interest in using it and the actual use of an e-health application. Another study conducted by Hu et al. (1999) succeeded in proving the relationship between the intention to use and the actual use of telemedicine technology, but this study was conducted from the perspective of health workers, not users. In addition, the actual use variable is too rarely measured in research, which leads to a lack of opportunities to test some important relationships in the TAM, such as that between intention and actual use (Verkasalo et al., 2010; Holden and Karsh, 2010). This motivates the writers to examine the relationship between behavioral intention to use and actual use from the user's perspective by formulating the following hypothesis.

Research methods

Data collection and analysis

This study uses a quantitative approach using an online questionnaire. Questionnaire that has been prepared must pass a readability test before it can be distributed and filled out by respondents. Such a readability test was carried out to ensure that all items in the questionnaire would be understood by the respondents in terms of writing rules, the relevance of the questionnaire to the research, its suitability within the research context, and the respondents' understanding of all the items in the questionnaire.

The criteria regarding the respondents are people who know about or have previously used a hospital's teleconsultation application. The readability test was carried out for one week from February 17, 2021, to February 24, 2021 (8 days). The readability test for this study was conducted with 10 respondents, who responded by providing criticism of or suggestions to improve the questionnaire. The feedback from the readability test results included the use of standard words, adding instructions for filling out questionnaires, the consistency of the terms used, adding answer choices, typos, writing questions, and input for indicator statements to make them easier for the respondent to understand. Moreover, Tsang et al. (2017) stated that it is advisable to test the questionnaire items if there is confusion about any items, and whether respondents have suggestions for possible improvements of the items. Since we have already got the suggestions from the readability test to improve our questionnaire; thus, we were not do the pilot test.

After the questionnaire was compiled and refined through the readability test, it was distributed online to begin the data collection stage. The survey links created were distributed through social media, such as Instagram, Twitter, and Facebook, as well as through instant messaging applications, such as WhatsApp and LINE. The online questionnaire distribution method was chosen to expand the data collection area and in consideration of Indonesia being affected by the COVID-19 pandemic during the data collection process. The target respondents for this study were Indonesians aged 17 years and over who knew about or had used hospital telemedicine applications. All respondents who participated in filling out the online questionnaire have provided informed consent to be involved in this study. The data collection was carried out from February 22, 2021, to March 28, 2021 (7 days).

Analysis methods

The data collected were processed mathematically using the IBM SPSS AMOS 24 software and the covariance-based structural equation modeling (CB-SEM) method. The data analysis process consisted of six stages, namely the specification of the research model, the identification of the research model, the estimation of the research model, the feasibility test of the research model, the modification of the research model, and the testing of the research hypotheses. The feasibility test of the research model is divided into two stages, namely the measurement model test and the structural model test. The measurement model test is used to measure the validity and reliability of the construct size on an outer model, while the structural model test includes a goodness-of-fit (GoF) test.

Research instruments

The questionnaire was prepared in Indonesian and consisted of demographic questions as well as questions that were arranged based on indicators that represented each variable in the research model. These questions were answered using a 5-point Likert scale, where 1 represented strong disagreement, and 5 represented strong agreement with the statements in the questionnaire. The demographic questions contained in the questionnaire included age, gender, most recent education, domicile, monthly income, and details regarding the use of hospitals' teleconsultation applications. We compiled the research instrument referring to Kamal et al. (2020), Baudier et al. (2021), Zhou et al. (2019), Lin (2017), and Binyamin and Zafar (2021) which have contextual compatibility with the study conducted. Appendix A describes the questionnaire instrument used in this study.

Respondent demographics

The survey was filled out by 664 respondents, however 113 respondents had incomplete data, and the remaining valid data obtained was 551. Table 1 provides a demographic summary of the respondents.

Table 1

Respondents' demographics.

Demographics		Number of Respondents	Percentage
Gender	Men	206	38.6%
	Women	328	61.4%
Age	17–25 years old	462	86.5%
	26–35 years old	36	6.7%
	36–45 years old	18	3.3%
	45–55 years old	12	2.2%
	>55 years old	6	1.1%
Education	High School	95	17.8%
	Diploma	19	3.6%
	Bachelor	409	76.6%
	Master	10	1.9%
	Others	1	0.2%
Domicile	Greater Jakarta	410	76.8%
	Outside Greater Jakarta on Java Island	63	11.8%
	Sumatera Island	21	3.9%
	Sulawesi Island	20	3.7%
	Kalimantan Island	8	1.5%
	Bali, NTT, NTB Island	10	1.9%
	Maluku and Papua Island	2	0.4%
Monthly income	< IDR 2,500,000	374	70%
	IDR 2,500,001–IDR 5,000,000	91	17%
	IDR 5,000,001–IDR 7,500,000	36	6.7%
	> IDR 7,500,000	33	6.2%
Frequency of visits to a hospital during the past year	Never	138	25.8%
	1–3 times	330	41.8%
	4–6 times	44	8.2%
	More than 6 times	22	4.1%
Average time spent in hospital during the past year	Under 30 min	122	22.8%
	30 minutes–1 hour	200	37.5%
	1 hour–2 hours	138	25.8%
	More than 2 h	74	13.9%
Types of hospitals visited by respondents in the past year	Government hospital	345	87.1%
	Private hospital	51	12.9%
The use of hospital teleconsultation applications by respondents	Yes	286	53.5%
	No	248	46.4%
Reason for using a hospital teleconsultation application	Shorter waiting time	323	35.4%
	To avoid exposure to the COVID-19 virus	287	31.5%
	Cheaper price	207	22.7%
	Advice from others in the surrounding environment	66	7.2%
	Advice from medical personnel	29	3.2%
Constraints faced when using a hospital telemedicine application	Application is difficult to use	119	19.7%
	Internet connection is not stable	199	33%
	Difficulty communicating with doctors	285	47.3%
Expectations regarding the development of hospitals' teleconsultation applications in the future	The addition of payment methods for services	221	21.3%
	The addition of a redemption feature for the drugs prescribed in teleconsultation sessions	251	24.2%
	The addition of review and comment features for each doctor	266	25.6%
	The addition of a feature to view one's teleconsultation history and medical records	300	28.9%

Results

Measurement models

The measurement model test included a convergent validity test, reliability test, and discriminant validity test. A convergent validity test is used to determine the level of correlation between indicators in a construct with factor loadings values ˃ 0.70 and average variance extracted (AVE) values ≥0.50 (Hair et al., 2011). After modifying the indicators, they all have a factor loadings value of ˃ 0.70. Hair et al. (2017) and Beckett et al. (2017) stated that a reliability test should be carried out by reviewing the Cronbach's alpha (CA) and composite reliability (CR) values. The CA and CR values must be more than 0.70 to meet the reliability test requirements (Hair et al., 2017). Table 2 shows that all the CA and CR values for each construct in the model are above 0.70 and that the AVE is above 0.5.

Table 2

AVE, CA, and CR values.

Variable	AVE	CA	CR
FC	0.979	0.745	0.989
SI	0.784	0.815	0.888
RA	0.810	0.812	0.887
PRO	0.847	0.836	0.903
CA	0.746	0.878	0.898
SF	0.569	0.839	0.841
PEU	0.650	0.846	0.848
PU	0.685	0.805	0.813
IQ	0.552	0.828	0.831
BI	0.621	0.830	0.831
AU	0.676	0.858	0.862

A discriminant validity test is carried out by looking at the value of cross-loadings by comparing the outer loading value of an indicator in its own construct with the value of other outer loadings (Hair et al., 2017). Table 3 demonstrates how the model passes the discriminant validity test.

Table 3

Cross-loading values.

	PRO	IQ	RA	PU	PEU	CA	SI	FC	SF	BI	AU
PRO3	0.753	0.537	0.512	0.436	0.514	0.4	0.345	0.156	0.607	0.432	0.3
PRO2	0.825	0.588	0.561	0.477	0.563	0.438	0.378	0.171	0.665	0.473	0.329
PRO1	0.817	0.582	0.556	0.473	0.558	0.434	0.374	0.17	0.659	0.469	0.326
IQ3	0.499	0.7	0.446	0.564	0.591	0.384	0.288	0.141	0.462	0.531	0.369
IQ2	0.535	0.751	0.478	0.605	0.634	0.412	0.309	0.152	0.496	0.57	0.396
IQ1	0.544	0.763	0.486	0.615	0.644	0.419	0.315	0.154	0.504	0.58	0.403
RA3	0.539	0.756	0.482	0.609	0.638	0.415	0.312	0.153	0.5	0.574	0.399
RA2	0.531	0.497	0.78	0.534	0.54	0.362	0.48	0.176	0.595	0.556	0.386
RA1	0.532	0.498	0.782	0.535	0.541	0.362	0.48	0.176	0.596	0.557	0.387
PU3	0.501	0.469	0.736	0.504	0.509	0.341	0.453	0.166	0.562	0.525	0.365
PU2	0.498	0.693	0.589	0.861	0.706	0.4	0.402	0.121	0.519	0.663	0.461
PEU3	0.532	0.658	0.539	0.64	0.78	0.464	0.362	0.112	0.482	0.609	0.423
PEU2	0.57	0.705	0.577	0.684	0.835	0.497	0.387	0.12	0.516	0.652	0.453
PEU1	0.548	0.677	0.555	0.658	0.803	0.477	0.372	0.116	0.496	0.627	0.436
CA4	0.479	0.495	0.418	0.419	0.536	0.901	0.21	0.12	0.512	0.408	0.284
CA3	0.476	0.491	0.415	0.416	0.532	0.895	0.209	0.119	0.508	0.405	0.281
CA1	0.42	0.434	0.367	0.368	0.47	0.791	0.184	0.105	0.449	0.358	0.249
SI3	0.329	0.296	0.441	0.335	0.333	0.167	0.717	0.15	0.396	0.456	0.317
SI2	0.37	0.333	0.496	0.377	0.374	0.188	0.807	0.169	0.446	0.513	0.356
SI1	0.368	0.331	0.494	0.376	0.373	0.187	0.804	0.168	0.444	0.511	0.355
FC3	0.206	0.2	0.223	0.139	0.143	0.131	0.207	0.989	0.246	0.218	0.152
FC2	0.205	0.201	0.224	0.140	0.145	0.132	0.207	0.99	0.246	0.219	0.153
SF4	0.626	0.513	0.592	0.468	0.48	0.44	0.429	0.193	0.776	0.509	0.354
SF3	0.579	0.475	0.548	0.434	0.444	0.408	0.397	0.179	0.718	0.472	0.328
SF2	0.642	0.526	0.607	0.48	0.492	0.452	0.44	0.198	0.796	0.523	0.363
SF1	0.585	0.479	0.553	0.438	0.449	0.412	0.401	0.18	0.725	0.476	0.331
BI1	0.46	0.609	0.571	0.618	0.626	0.363	0.509	0.177	0.527	0.802	0.557
BI2	0.441	0.584	0.547	0.592	0.6	0.348	0.488	0.17	0.505	0.768	0.534
BI3	0.455	0.602	0.565	0.611	0.619	0.359	0.504	0.175	0.521	0.793	0.551
AU1	0.342	0.452	0.424	0.459	0.465	0.269	0.378	0.131	0.391	0.595	0.857
AU2	0.316	0.418	0.392	0.424	0.43	0.249	0.35	0.121	0.361	0.55	0.792
AU3	0.326	0.431	0.404	0.438	0.443	0.257	0.361	0.125	0.373	0.568	0.817

Structural model

A structural model test is carried out by reviewing the GoF parameter in a model, which states how good a fit the model is, and this is followed by a review of the value (Hair et al., 2014). GoF is defined as a measure of the correspondence between the expected model and the actual covariance matrix (Hair et al., 2013). GoF includes the CMIN/DF (relative chi-square), RMSEA (the root means square error approximation), NFI (normed-fit index), CFI (comparative fit index), GFI (goodness-of-fit index), TLI (Tucker-Lewis index), and RMR (root mean square residual). After making modifications, the GoF results obtained from the final model can be seen in Table 4.

Table 4

GoF values.

Goodness-of-Fit Index	Cut-off Value	Value	Description
CMIN/DF	<2.0	1.977	Good fit
RMSEA	0.03 ≤ x ≤ 0.08	0.041	Good fit
NFI	≥0.9	0.937	Good fit
CFI	≥0.9	0.968	Good fit
GFI	≥0.9	0.929	Good fit
TLI	≥0.9	0.956	Good fit
RMR	≤0.05	0.044	Good fit

Because the model met the GoF criteria, we continued with the structural model test using the coefficient of determination (). Table 5 shows the values for the BI and AU variables. The value for the BI variable is 0.754, which is classified as a strong effect size. This shows that 75.4% of the change in the BI variable is strongly explained by exogenous variables, namely FC, SI, RA, CA, PU, PRO, PEU, SF, and IQ. Meanwhile, the value for the AU variable is 0.375, which is classified as a low effect size. The variables that may drive the conversion from intended use to actual use include technical issues, such as the compatibility of the application with the smartphone being used, a user-friendly design or user interface, and user experience of having used similar applications before (Lim et al., 2011).

Table 5

Values.

Parameter	R^2	Effect Size
BI	0.754	Strong
AU	0.375	Weak

Hypothesis testing

After the feasibility test of the research model was completed, the next step was to test the research hypotheses. Hypothesis testing is carried out by looking at the p value using a significance level of 5% or 0.05 (Hair et al., 2017). Table 6 and Figure 2 explain the results of the hypothesis testing.

Table 6

Hypothesis testing.

Hypothesis				Estimate	P	Description
1	BI	<– – –	IQ	0.147	0.044	Accepted
2	BI	<– – –	RA	0.155	0.004	Accepted
3	BI	<– – –	SF	0.107	0.002	Accepted
4	BI	<– – –	FC	0.047	0.123	Rejected
5	BI	<– – –	SI	0.102	0.054	Rejected
6	BI	<– – –	CA	-0.113	0.01	Accepted
7	BI	<– – –	PEU	0.636	0.003	Accepted
8	BI	<– – –	PU	0.097	0.039	Accepted
9	BI	<– – –	PRO	-0.162	0.021	Accepted
10	AU	<– – –	BI	0.591	0.002	Accepted

Figure 2

Final model.

Discussion

The provision of the necessary knowledge, competence, and skills to use telemedicine services is needed (Edirippulige and Armfield, 2016). Based on the results of testing H1, FC proved to have no significant effect on the behavioral intention to use a hospital's teleconsultation application. The acceptance of this hypothesis contradicts the research results of Kamal et al. (2020), where facilitating conditions were found to be one of the supporting factors for receiving telemedicine technology in developing countries. However, these results are consistent with the results of the study by Venkatesh et al. (2003), who did not find a relationship between facilitating conditions and behavioral intention. From the results of the questionnaire, it is clear that members of Generation Z believe that it is necessary to own and use a smartphone to do many things, such as blogging, online shopping, and job hunting. In addition, the members of Generation Z are also self-educators who learn about everything themselves through the internet (Rettig, 2020). Therefore, it can be concluded that most of the respondents felt that they already had adequate knowledge and facilities to access and use a hospital's teleconsultation services and did not need help from others. This meant that facilitating conditions were not a factor that significantly influenced their interest in using a hospital's teleconsultation application.

H2 is rejected in that social influence does not have a significant positive effect on behavioral intention to use. This is not in line with the results of the studies by Kamal et al. (2020) and Baudier et al. (2021), but is in agreement with research results from Lestari and Rofianto (2020). Lestari and Rofianto (2020) conducted an analysis of user value and the adoption of mobile health services in Indonesia during the COVID-19 pandemic, where one of the variables studied was the external cues to action variable from the health belief model. External cues to action are defined as stimuli that can trigger the decision-making process to accept a recommended health action (LaMorte, 2019). These stimuli can be in the form of advice from other people, illnesses suffered by people around them, or articles from the internet (LaMorte, 2019). Lestari and Rofianto (2020) show that external cues to action factors do not have a positive effect on the adoption of mobile health technology by patients. In addition, the answers to the questions on the questionnaire distributed to respondents indicated that only 66 people used a hospital's teleconsultation application because it was recommended by people in the surrounding environment, and only 29 people used it because it was recommended by the medical personnel of their chosen hospital.

The results of the testing of H3 show that contamination avoidance has a significant positive impact on behavioral intention to use. This is in line with the results of research from Baudier et al. (2021), where contamination avoidance was proven to have a significant effect. During the COVID-19 pandemic, one place that has a high level of contamination is medical facilities (Hazée and Van Vaerenbergh, 2020). Patients doubt that hospitals are ready to face the COVID-19 pandemic, both in terms of their facilities and medical personnel, and this perception looks set to persist even after people have been vaccinated (Yuswohady et al., 2021). Of the 441 respondents, 80.3% stated that they will still be afraid of going to the hospital because of the risk of contracting the COVID-19 virus (Yuswohady et al., 2021). Indonesian people see telemedicine as a health service solution because they are worried about the potential spread of the COVID-19 virus (Yuswohady et al., 2021).

Based on the results of H4, safety has a significant influence on behavioral intention to use. This is consistent with the results of the study by Zhou et al. (2019). According to Yuswohady et al. (2021), the COVID-19 pandemic has made consumers prioritize their basic needs, namely clothing, food, shelter, and safety in terms of health and security, and cleanliness, health, safety, and environment (CHSE) concerns are consumers' top priorities during the pandemic (Yuswohady et al., 2021). Yuswohady et al. (2021) also stated that as many as 71% of 1,121 respondents considered that health services from hospitals as telemedicine providers had been tested and were considered capable of providing safety value. This shows that the safety factor plays an important role in influencing a user's interest in using a hospital's teleconsultation application.

The results of the testing of H5 indicate that reliability has a significant effect on the behavioral intention to use a hospital's teleconsultation application. This is in line with Lin (2017), who states that the higher the reliability of a telemedicine service, the higher users' desire to use it and the more satisfaction they experience after using the service. The results of research conducted by Handayani et al. (2018) on the success factors in terms of implementing a mobile health application in Indonesia prove that the reliability of mobile health with regard to completing a service is one of the success factors in developing a mobile health application. If a health application is reliable when providing services to users, this will increase user confidence in the application, which will then affect their interest in adopting the application in question. Another study conducted by Tantarto et al. (2020) reveals the reliability and responsiveness of telemedicine applications in Indonesia and shows that there are still many patients who think that telemedicine is not reliable because patients cannot see the doctor's face. They feel that because doctors cannot touch their patients, they cannot provide reliable consultations (Tantarto et al., 2020). According to Silalahi et al. (2018), the reliability of the service is the most important factor affecting user preferences for using teleconsulting applications, and the services provided must be able to help convince users that the application can understand their health care needs. If a telemedicine application is less reliable, this will affect patient satisfaction after using the application and a patient's desire or interest in terms of adopting the application (Silalahi et al., 2018).

One factor that affects patients' trust in teleconsultation services is the identity of the doctors and their qualifications (George et al., 2009). The professionalism of health workers refers to their core knowledge, basic skills, interpersonal skills, and lifelong learning, as well as their ability to apply their knowledge when providing health services (Epstein, 2002). The H6 test results show that professionalism has a significant impact on the behavioral intention to use a hospital's teleconsultation application. This is in line with the study by Zhou et al. (2019), which states that professionalism affects patient satisfaction with a telehealth service, which, in turn, affects patients' acceptance of the service. In Indonesia, Yuswohady et al. (2021) showed that 71% of the 441 respondents admitted that one reason they trusted a hospital's teleconsultation service was an expert doctor who was affiliated with the hospital. Furthermore, the professionalism of health workers who provide advice and medical care online must meet the requirements of the professional code of ethics, and the limitations of diagnosis recommendations made online must be explained (Garcia, 2021). Hospitals as health service providers are expected to instill certainty through their professionalism in every interaction and meeting with patients to increase patient confidence in these services (Akter, 2012). High trust in the professionalism of hospital teleconsultation service providers will affect patients' acceptance of the use of e-health services (Akter, 2012), which in the context of this study are hospitals' teleconsultation applications.

In the telemedicine service process, efforts are needed to gain access to telemedicine services, to understand them, and to learn how to use this new technology (Lin, 2017). The H7 test results show that perceived ease of use has a significant impact on behavioral intention to use, which is in line with the research results of Lin (2017), Kamal et al. (2020), and Baudier et al. (2021). In Indonesia, several previous studies have also proven the relationship between the perceived ease of use of and the behavioral intention to use telemedicine applications. Hartono and Laurence (2019) and Indriyanti and Wibowo (2020) show that perceived ease of use has a significant effect on user interest in technology startup telemedicine applications. If the use of the application tends to be complicated for a patient, the patient will feel reluctant to use it (Indriyanti and Wibowo, 2020). Therefore, the features and functions of the application must be made attractive and not messy to increase a user's desire to use it (Christian, 2019).

In addition, patients feel that adopting telemedicine services will only be useful if such services will provide faster health services, lower medical examination costs, better documentation, and reduced waiting times (Kamal et al., 2020). The results of the testing of H8 indicate that perceived usefulness has a significant effect on the behavioral intention to use a hospital's teleconsultation application. This is in line with Kamal et al. (2020) and Baudier et al. (2021). This statement is supported by Lestari and Rofianto (2020) regarding the adoption of mobile health in Indonesia during the COVID-19 pandemic. The results of this study state that the value of usefulness has a significant effect on the adoption of mobile health. During the pandemic, Indonesians have sought alternative health care with the aim of managing and improving their own health (Lestari and Rofianto, 2020).

The results of the testing of H9 indicate that information quality has a significant effect on behavioral intention to use. This is consistent with Zhou et al.’s (2019) research, which states that information quality is an important factor in determining the competitiveness of a telehealth system and affects user interest. The accessibility of medical records and advice from health professionals have a significant positive impact on the patient acceptance of the telehealth system (Rho et al., 2014). In addition, e-health that can produce timely, accurate, and relevant information will have an impact on user interest in using a system (Nurhayati et al., 2019). Sesilia (2020) also showed that one of the factors that users consider when using telehealth services is the quality of information submitted by doctors. Moreover, Handayani et al. (2018) found that information quality is the most important determining factor. Patients feel that the completeness and relevance of the information is the most important thing and that the information in the application should match their medical records (Handayani et al., 2018).

The H10 test results indicate that behavioral intention to use is proven to have a significant impact on the actual use of a hospital's teleconsultation application. This is in line with the results of Hoque et al. (2016), which state that patient intention to use has a significant positive effect on the actual use of e-health in developing countries. Hu et al. (1999) revealed that there is a relationship between the intention to use and the actual use of telemedicine technology. In Indonesia, research conducted by Manda and Salim (2021) also showed that actual use is significantly influenced by attitudes towards the use of the Halodoc teleconsultation application. Behavioral intention to use is a predictor of the actual use of a system (Bhattacherjee and Hikmet, 2008), where the correlation between the two variables is high (Turner et al., 2010). The higher the support for the use of a hospital's telemedicine services by its users, the bigger the increase in the actual use of its services (Kissi et al., 2019).

Implications

The results obtained from this study corroborate the research results of Kamal et al. (2020) regarding the acceptance of telemedicine services by patients in Pakistan. The perceived usefulness, social influence, and perceived ease of use variables are proven to be facilitators that have a significant impact on behavioral intention to use. However, this study proves that facilitating conditions and social influence do not have a significant effect in the context of hospitals' teleconsultation applications in Indonesia. The results of this study also confirm those of Zhou et al. (2019) regarding the influence of the professionalism of health workers on patients' acceptance of teleconsultation services and strengthen the results of research by Lin (2017), which state that reliability is a factor affecting interest in adopting the use of telemedicine from a patient's perspective. In addition, this study adopts the contamination avoidance variable from Baudier et al. (2021) regarding the perceptions of patients in various countries in the world concerning teleconsultation services during the COVID-19 pandemic. The results of this study indicate that users in Indonesia see hospital teleconsultation services as an alternative that can reduce their chances of being exposed to the COVID-19 virus on their way to the hospital, in the waiting room with other patients, when handling contaminated items, and when encountering health workers in the doctor's office.

This research is expected to be useful for hospitals looking to develop their teleconsultation applications by providing information on the factors that can influence users' interest in adopting such teleconsultation applications. The results of this study indicate that there are several aspects that must be considered by hospitals in their role as the owners of the teleconsultation applications. Using this knowledge, hospitals are expected to be able to improve and develop the applications concerned so that they can provide better health services to the people of Indonesia. Based on the results of the study, some of the application features that the public expects from hospitals' teleconsultation applications include the addition of payment methods for teleconsultation services. Information quality in terms of a hospital's teleconsultation application is one of the important factors influencing users' interest in adopting it. To maximize the quality of the information in its application, a hospital can add information regarding the complete profiles of its doctors, its locations and hours of operation, and the prices of its teleconsultation sessions, as well as developing features for viewing health articles consultation histories, and user medical records, and for providing and viewing reviews.

Conclusion

The results obtained in this study revealed how facilitating conditions and social influence do not significantly influence behavioral intention to use. This shows that the availability of facilities that enable the use of a hospital's teleconsultation application and influences in the surrounding environment, including friends, family, and medical personnel, do not significantly affect a user's intention to adopt a hospital's teleconsultation application. However, contamination avoidance, safety, reliability, professionalism, perceived ease of use, perceived usefulness, and information quality are found to have a significant effect on behavioral intention to use. This shows that users chose teleconsultation applications as an alternative to health services during the COVID-19 pandemic to avoid the risk of being exposed to the virus. Users also feel that their interest in using a hospital's teleconsultation application will increase if the teleconsultation provider prioritizes safety and has professional medical personnel. These results also indicate that the intention to use a hospital's teleconsultation application will increase if the application is easy to use, useful, and has good quality information. In addition, behavioral intention to use has a significant effect on actual use, which indicates that the interest in using a hospital's teleconsultation application has a significant effect on the actual use of the application. Most of the respondents have age between 17 to 25 years old and live in the Greater Jakarta area which limit the result of this study. Hence it is recommended for future studies to use a research sample that is more distributed in terms of age groups and domicile. Future studies can also include psychological factors which are not included in this study to further learn about users' perspectives regarding teleconsultation applications and also analyze and compare users' acceptance of hospital teleconsultation services provided through other media besides mobile applications, such as teleconsultation services conducted via video conferencing or WhatsApp.

Declarations

Author contribution statement

Steffi Alexandra: Conceived and designed the experiments; Performed the experiments; Contributed reagents, materials, analysis tools or data; Wrote the paper.

Putu Wuri Handayani: Conceived and designed the experiments; Analyzed and interpreted the data; Contributed reagents, materials, analysis tools or data; Wrote the paper.

Fatimah Azzahro: Conceived and designed the experiments; Contributed reagents, materials, analysis tools or data.

Funding statement

This work was supported by /Badan Riset dan Inovasi Nasional Indonesia for the Grant Hibah Penelitian Dasar Unggulan Perguruan Tinggi (Number NKB-075/UN2.RST/HKP.05.00/2021).

Data availability statement

The authors do not have permission to share data.

Declaration of interests statement

The authors declare no conflict of interest.

Additional information

No additional information is available for this paper.

Code	Statement	References
FC1	I have Information Technology knowledge to use the teleconsultation app developed by the hospital	Kamal et al. (2020)
FC2	I can get help from other people if there are difficulties in using the teleconsultation application developed by the hospital
FC3	I have the necessary facilities to use the teleconsultation application developed by the hospital (example: compatible smartphone, adequate internet network)
SI1	The environment around me (i.e., family, friends) encourages me to use the teleconsultation application developed by the hospital	Baudier et al. (2021)
SI2	The person who significantly influenced my behavior advised me to use the teleconsultation app developed by the hospital
SI3	Health workers such as doctors and nurses advised me to use the teleconsultation application developed by the hospital
CA1	I felt teleconsultation could prevent me from being exposed to viruses and bacteria while I was on my way to the hospital	Kamal et al. (2020)
CA2	I feel teleconsultation can prevent me from being exposed to viruses and bacteria while I am in the waiting room with other patients
CA3	I feel that teleconsultation can prevent me from being exposed to viruses and bacteria when I come into contact with health workers
CA4	I feel teleconsulting can prevent me from being exposed to viruses and bacteria when I touch contaminated items (example: doorknobs)
SF1	The hospital facility that owns the teleconsultation application and the medicines have met the applicable standards and regulations	Zhou et al. (2019)
SF2	The doctor care affiliated with the hospital owner of the teleconsultation application is systematic and safe
SF3	The hospital that owns the teleconsultation application has been proven to be able to handle emergency cases well.
SF4	The procedures used by the hospital prioritize patient safety
RA1	The online diagnosis provided via the teleconsultation application developed by the hospital can be trusted	Lin, 2017; Zhou et al. (2019)
RA2	Without face-to-face examination, I am sure the doctor can find out about my health condition
RA4	The information provided by the teleconsulting application developed by the hospital is reliable
RA5	The teleconsultation application developed by the hospital will not experience any technical problems
PRO1	Doctors affiliated with the hospital that owns the teleconsultation application have sufficient theoretical knowledge and can apply this knowledge when providing medical services.	Zhou et al. (2019)
PRO2	Doctors affiliated with the teleconsultation app owner's hospital have good professional skills
PRO3	The medical services provided by the hospital which owns the teleconsultation application are efficient
PEU1	I believe that using the teleconsultation app developed by the hospital is easy for me	Kamal et al. (2020)
PEU2	I believe that I can easily interact with doctors in using the teleconsultation application developed by the hospital
PEU3	I believe that I understand how to interact with a teleconsulting application
PU1	I believe that the quality of my health care will improve thanks to the teleconsultation application developed by the hospital	Kamal et al. (2020)
PU2	I believe that it will be easier for me to access health services because I use the teleconsultation application developed by the hospital
PU3	I believe that using the teleconsultation app developed by the hospital will be useful in my daily routine
IQ1	The teleconsultation application developed by the hospital can provide accurate and useful information about the hospital and doctor information	Zhou et al. (2019)
IQ2	The teleconsultation application developed by the hospital can provide accurate and useful information about the prevention of common diseases
IQ3	Teleconsultation can provide accurate and useful information about first aid measures
IQ4	nt medical record data in the teleconsultation application is integrated with existing medical record data in the hospital
BI1	Assuming I have been given the opportunity to access teleconsultation, I intend to use the hospital's teleconsultation services	Kamal et al. (2020)
BI2	Whenever I need remote medical care from professionals, I will happily use the hospital's teleconsultation service
BI3	I intend to inform my relatives and friends about the hospital's teleconsultation service
AU1	I use teleconsultation a lot	Binyamin and Zafar (2021)
AU2	I intend to use teleconsultation for as long as I need it
AU3	I already use teleconsultation regularly