Health referral system user acceptance model in Indonesia.

OBJECTIVES: This study aims to identify a user acceptance model for the health referral system in Indonesia. The following factors classified into dimensions of organization, technology, process, and individual, were examined: patient centricity, regulation, data security, integration, responsiveness, effectiveness, efficiency, personal beliefs, and social influence.
METHODS: Quantitative data processing methods were used, including the online distribution of questionnaires to a total of 283 valid respondents who had previously used health referrals. Data processing was performed according to the ordinal logistic regression method using IBM SPSS Statistics 24.0 software.
RESULTS: The user acceptance model fit with a significance of 0.084, while only regulation, data security, integration, responsiveness, effectiveness, efficiency, personal beliefs, and social influence significantly influenced the patients' acceptance of health referrals.
CONCLUSIONS: This study may build awareness in the community regarding the health referral system along with the ideal factors that encourage patients to utilize health referrals. In addition, the provision of health services by health facilities and regulators may take these factors into account so they may provide fair and equitable services for all the people of Indonesia; for example, providers and regulators can improve the utilization of information technology and guidebooks on the health referral system to facilitate communication and standardization among health facilities.

Introduction

Health is necessary for every human being and it serves as an indicator of a country's development. Countries that provide sufficient health facilities and services can increase the productivity of their citizens. The distribution of healthcare in Indonesia is mandated in the fifth principle of Pancasila – Indonesian state principles, which calls for “social justice for all Indonesian people”; however, until now, health referral services have not been able to reach all levels of society. Furthermore, when healthcare is needed for a suddenly-occurring condition, such as an unexpected illness or accident, a significant financial burden may result because healthcare is still considered a luxury in Indonesia. Indonesia's current population of approximately 261.1 million individuals [1] renders the need for health services increasingly important due to various limitations of basic needs, social facilities (health facilities, schools, public areas), employment opportunities, and clean water [2]. In the health field, there is a considerable gap between the size of the population and the number of health facilities and personnel, making the need for integrated systems—namely referral systems—an urgent concern.

Considering this issue, the Government of the Republic of Indonesia enacted the Law on National Social Security System (Law no. 40 of 2004) on October 19, 2004. This law was issued based on the amended Indonesian 1945 Constitution, which mandates the development of a national social security system. Furthermore, in 2011, the government enacted the Law on the Social Security Agency for Health (BPJS-K) (Law no. 24 of 2011), which mandated the establishment of BPJS-K no later than January 1, 2014.

BPJS-K has a tiered referral system, which is a health service arrangement that regulates the delegation of duties and responsibilities of health services on both a vertical and horizontal, reciprocal basis. This referral system must be implemented by health insurance participants, social health insurance companies, and health facility providers. This tiered referral system occurs on a hierarchical basis from primary (being the nearest health facilities to the community), secondary, and tertiary health facilities. Referral to second-level health facilities can only be administered by a first-level health facility (FKTP). FKTP includes primary healthcare centers (Puskesmas or Pusat Kesehatan Masyarakat), physician private practices, dentists, primary or equivalent clinics, and class D hospitals (small hospital) or their equivalents. Likewise, referrals to third-level health facilities can only be administered by second-level health facilities. Higher-level health facilities can refer patients to lower-level health facilities according to their competence and authority [3].

In practice, there are many Indonesian people who do not know about BPJS-K's tiered referral system [4]. According to Ibrahim, there are four factors that contribute to health referral problems in Indonesia [5]. First, very few health facilities display a list of services provided, making it difficult to determine the pattern of referral and the clinical pathway toward it. Second, there is a catchment or coverage area for health referrals. Currently, the referral area still follows the boundary of an administrative area between regions; although, in some areas, higher-level health facilities are closer in proximity to other administrative areas. For example, District Temajuk, Sambas District, and West Kalimantan Province are directly adjacent to Malaysia; nearby health facilities in Malaysia can be reached within two hours, but health facilities in other administrative areas of Indonesia can be reached within six hours by boat. The other factor is a lack of strong commitment from referral actors or stakeholders, including district hospitals, military hospitals, private hospitals, clinics, health service providers, the House District of Representatives (DPRD), the Ministry of Health (MOH), and the Ministry of Finance, all of which act as the regulators. Due to those conditions, some people in Indonesia also resist participating in health referrals even in chronic and dire conditions. Amoah and Phillips's study found that the knowledge and willingness of participants in Ghana to adhere to the referral policy were limited due to their fear of attending advanced health facilities, and they often insist upon receiving treatment in their current health facility and lack general understanding of the referral process [24]. Patient refusal as well as health worker refusal occurred because the latter did not want to change the current process by not referring patients [25]. Thus, this study focuses on understanding patients' acceptance of the health referral system.

A health referral system can be implemented by three major health service entities, one of which is a health insurance participant. Health insurance participants have the authority to apply for and receive vertically- or horizontally-tiered BPJS-K referrals. Several current studies related to health referrals served as the authors' reference points in developing this research. Most studies focused on subjects in developing countries, such as Kenya [7] and Swaziland [8], and examined the types of diseases that are often problematic in some Canadian clinics, such as knee jerking [9]. In developing countries, health referrals are critical because there is no sufficient distribution of health services, and referrals must be made for patients who should receive follow-up care. However, there are still some obstacles; in Kenya, people are more aware of referrals, but still rarely receive them [7]. In Swaziland, more than 30% of health workers still do not understand the flow of referrals [8], while in Canada, additional coordination is needed to further reduce referral time [9]. Another referral-related study examined non-communicable disease referrals using examples from diabetes and explaining the key factors for an effective referral system [12].

This study is expected to enrich the research on health and information technology (IT) and provide input factors that influence the acceptance of the health referral system in Indonesia. This research also conducted a study among the Indonesian community on awareness of health referrals in order to provide Indonesians information about receiving referrals. This kind of research is rare in Indonesia, and thus it was meant to have a positive impact on the development of its health systems as a developing country with the fourth largest population in the world. Based on these problems, the research question is: how can patients' acceptance of the health referral system be modelled?

The paper is divided into seven sections, where Sections 2 and 3 discuss the literature review and hypothesis, respectively. Section 4 discusses the research methodology used, and Section 5 describes the results of this study. Discussion and research implications are described in Sections 6, respectively. Finally, Section 7 describes the conclusions of this study.

Theory

Health referral system

The health referral system is defined in the Minister of Health Regulation on Individual Health Service Referral System (IHSRS) (Permenkes No. 001 of 2012). Article 3 of this regulation defines the referral system as the organization of health services that regulates the delegation of duties and responsibilities of health services in reverse—either horizontal or vertical. This regulation also governs the procedures for conducting individual health referrals. A horizontal referral arrangement is a referral between same-level health facilities. This occurs when an FKTP cannot afford a patient's service due to limited human and facility resources. A vertical referral is a referral to a higher-level facility for more complex and specialist services. In addition to specialist services, vertical referrals can only be administered for 155 diseases, and each FKTP requires a referral quota. Health insurance participants, social health insurance companies, and all health facilities must participate in the referral system.

The referral system originated at the FKTP. Health referrals are organized according to a patient's medical needs, and they can deliver patients to second- or third-level health facilities. Service at second- and third-level facilities can only be performed if there exists a referral from the previous-level health facility. Other referral cases, such as referrals from midwives or health nurses, can only be administered by a doctor or dentist at an FKTP. Fig. 1 illustrates the health referral process in Indonesia.

Fig. 1

The health referral process in Indonesia [23].

Health referral application implementation in Indonesia

Currently, the most widely used referral applications in Indonesia are the Integrated Referral Information System (SISRUTE), P-care (primary care), and the Online Referral Scheduling System (SPRO). P-care is a web-based application used within first-rate health facilities. The P-care application was developed in 2014 by BPJS-K and has been implemented in almost all primary healthcare centers in collaboration with BPJS-K. This application has two main functions: registration and patient service. Online referrals can also be used with this application.

SPRO is a scheduling application among health facilities in the Daerah Khusus Ibukota (DKI) Jakarta provincial area, the capital city of Indonesia. According to the data on the SPRO information portal (http://spro-dki.net), this application has facilitated more than 24,000 patient referrals who are participants enrolled in BPJS-K. The application creates a value-added referral service by booking an integrated referral schedule to reduce patient waiting times.

SISRUTE is the application of duty assignment and health service responsibility in both horizontal and vertical directions. This system was first developed by the Government General Hospital (RSUP) of Dr. Wahidin Sudirohusodo and is only used in several hospitals in South Sulawesi Province. The main reasons for the development of this system were patient rejection and the typically slow responses from the emergency department. This application is also suitable for use along the BPJS-K-level referral flow. SISRUTE has been fully supported by the Ministry of Health and is currently implemented in other provinces in Indonesia.

Hypothesis

The conceptual model used and designed in this study was derived from research projects that were conducted by Ajwang, Senitan et al., Handayani et al., and Macintyre et al. [7, 12, 27, 28]. The selection of dimensions in this study refers to Handayani et al. [27], who mentioned three characteristics of the acceptance of health information systems (human, technology, and organization). Furthermore, the process dimension derived from the study conducted by Senitan et al. [12] includes effectiveness and efficiency factors. Then, a definition of the variables from each dimension was executed; subsequently, each factor was obtained from related studies.

The most widely used factor in a health referral system is the evidence-based practice factor introduced in Senitan et al.'s study [12]. Ten factors in that study include timeliness, safety, effectiveness, efficiency, patient centricity, equity, structuring of referral documents, dissemination of referral system guidance information, use of computer systems, and inclusion criteria of referral patients. Factors in the structuring of referral documents, dissemination of referral system guidance information, use of computer systems, and inclusion criteria of referral patients are generalized into one factor known as “regulation”. Patient centricity and regulation factors are incorporated into the organizational dimension, timely factors are translated into responsive factors, and security and responsive factors fall into the technological dimension. Macintyre et al. added research regarding the integration of existing data between health facilities so that the integration factor also entered into the technological dimension [28]. Furthermore, the process's dimensions cover the factors of effectiveness and efficiency that had previously been generalized. In the individual dimension, these factors are divided into two based on Ajwang's research, which mentions the existence of internal and external influences [7]. The internal influence is stated as a personal belief that is used in the health belief model (HBM). HBM was chosen because it has been used extensively in the health field to conduct research and predict public health behavior for both the short and long term [6]. HBM is also often used to predict preventive health behaviors, behavioral responses for the treatment of patients with acute and chronic diseases, and the referral process. Thus, the “personal beliefs” factor was included in this model as an individual internal factor that could influence individuals' conducting of the referral process. Subsequently, the external factors are used as social influence factors based on Holden and Kash's research [13].

Finally, the proposed conceptual model includes organizational, technological, procedural, and individual dimensions (Fig. 2). Organizational dimensions relate to organizational issues in the implementation of the prevailing system, such as management support, planning, implementation, field conditions, and user roles [11]. According to the 2013 National Reference Guideline of the Ministry of Health, the implementation of health referrals requires clear, organizational arrangements so that referrals may run effectively and efficiently. These health referrals should be able to benefit the referred patient as well as the health facilities. Referrals should be patient-centric and focused on meeting the needs of patients and their families and should provide them with current and actual information [12]. Senitan et al. explained how processes to quickly meet the needs of patients and staff as well as health procedures to provide services have been standardized [12]. Senitan et al. also mentioned the need for guidance in the referral guidelines and the need for written evidence of referrals in the form of referral letters that contain inclusion criteria [12]. Both measures have been undertaken to ensure that the health facility that receives a referral knows the correct action that should be given to the patient. The requirement for guidelines, letters, and inclusion criteria is regulated by the Ministry of Health. Therefore, this study proposes the following hypotheses (H):

Fig. 2

Proposed health referral system conceptual model.

Patient centricity (PST) influences the acceptance of health referral systems.

Regulations (REG) influence the acceptance of health referral systems.

The technology dimension includes safety, responsiveness, and integration. According to DeLone and McLean, the technological dimension can be divided into the quality of information and the quality of the system [11]. The quality of information is defined as the value of information generated by the system; it should be precise, accurate, relevant, and should follow a standard format. System quality is defined as the value of the software, as determined by interface consistency parameters, ease of use, system response capabilities, documentation, and lack of bugs [11].

Referring to the workforce security standards of the United State Department of Health and Human Services (HHS), as established by the Health Insurance Portability and Accountability Act (HIPAA), access to medical and administrative data in the system is limited to authorized health personnel; protection of patient data also covers other security aspects, such as damage, loss, and data leakage. The next dimension is responsiveness; according to Senitan et al., responsiveness includes the commitment of health workers to respond to patient referrals, take action when needed, and avoid procrastinating [12].

In the 2013 National Reference Guideline of the MOH, responsiveness is one of the factors in the monitoring and evaluation of the referral system. Responsiveness is important for ensuring the best service possible and continuing to make improvements to the health referral service. The integration dimension is a major component in health referrals because it connects health facilities together. The Minister of Health Regulation on IHSRS requires communication between health facilities on the availability of health facilities and infrastructure as well as the competence and availability of health personnel; thus, existing information systems at health facilities must be able to connect with one another. Furthermore, according to Fyie et al., every health facility must have electronic data on physicians and health services at other health facilities in order to facilitate integration [9]. Therefore, this study proposes the following hypotheses:

Data security (AMN) influences the acceptance of health referral systems.

The responsiveness (RSP) of health workers influences the acceptance of health referral systems.

System integration (ITG) among health facilities influences the acceptance of health referral systems.

The process dimension includes effectiveness and efficiency. According to the 2013 National Reference Guideline of MOH, a process is any referral sent directly or through an IT device within the national reference system in accordance with the health facilities' level. This process is regulated by the government and is agreed upon by all health facilities. According to Senitan et al., effective referrals are appropriate references for applicable scientific knowledge, and procedures and are executed with the utmost care to provide maximum benefits [12]. Health officials should make a diagnosis before administering a referral. In addition, effective health referrals should ensure that referrals are linked with documentation (i.e., medical resume) to ensure a follow-up will occur [7]. Effective referrals also provide limited services to patients to avoid giving conflicting services to a single patient [12]. In addition, the provision of effective referral services can provide access to specialist health personnel [7]. Therefore, this study proposes the following hypotheses:

Effective health referral services (EFK) influence the acceptance of health referral systems.

Efficient health referral services (EFS) influence the acceptance of health referral systems.

The individual dimension includes personal belief and social influence. Personal belief includes user experience, conformity with existing processes, security expectations, and user capabilities [11]. Social influence factors can be internal or external; internal factors are personal beliefs, and this study uses a health belief model as a framework to measure an individual's view of a susceptibility outbreak as well as seriousness, benefits, barriers, and self-confidence [6]. These beliefs can influence a person to receive health referrals. According to Ajwang, a person may receive a referral when his/her disease is considered serious and an epidemic may spread; such a referral might be for the curing of a disease [7]. Furthermore, family considerations that include awareness of referrals as well as financial and transportation capabilities can influence whether an individual is able to use health referrals [7].

External factors influences patient decision making caused by external environment factors, such as family, friends, relatives, and experts. Opinions from relatives and health officials can influence a patient's behavior regarding medical measures [13]. Additionally, Balatsoukas et al. examined nine studies in which social media interventions led to positive changes in user health behavior [14]. Therefore, the present study proposes the following hypotheses:

Personal beliefs (KPP) influence the acceptance of the health referral system.

Social influence (PSO) influences the acceptance of health referral systems.

Table 1 summarizes the list of dimensions, variables, and indicators used in this research.

Table 1

List of dimensions, variables, and indicators.

Dimension	Factor (Variable)	Factor Description	Indicator Code	Indicator
Organization	Patient Centricity (PST)	Health officials prioritize health services for the needs and referral preferences of patients and their families	PST1	Health officials prioritize health services for the needs and referral preferences of patients and their families
			PST2	Health officials educate patients on health referral information
			PST3	Health officials are always available when required by the patients
			PST4	The health worker monitors and ensures the patient arrives at the health facility on health referral purpose, based on his medical needs
	Regulation (REG)	Written rules governing the procedures and processes of health services	REG1	Health officials should utilize Information Technology in the implementation of referrals
			REG2	Health facilities use inclusion criteria (e.g., for handling patient A with ear disease under age 50) for each patient reference to avoid unwanted cases
			REG3	There is clear regulation and guidance on the referral process
			REG4	Regulations are issued by official institutions (i.e., health regulator)
			REG5	The referral process has formal structural documentation
Technology	Data Security (AMN)	Ensure information security in the patient referral process so that patient information is not leaked or damaged	AMN1	The migration of data is centralized in the system so that the referred health facility can know the patient's data automatically
			AMN2	Health data can only be accessed by authorized health personnel
			AMN3	Health data is non-transferable without the consent of the parties concerned
	Responsiveness (RSP)	The transition process on referrals can be done in a fast and responsive manner, according to the needs of the patient	RSP1	The health offical is committed to act in responsiveness for patient referral to the next level
			RSP2	Referral can be done quickly without going through a long processes or bureaucracy
			RSP3	Patient referrals are not delayed when needed
	Integration (ITG)	Connect health systems between health facility units for referrals	ITG1	Submitting patient data to the next health facility
			ITG2	The next health facility could access the doctor data
			ITG3	Have a doctor schedule data at the next health facility level
Process	Effective (EFK)	Referrals are based on scientific knowledge and implemented according to applicable process standards	EFK1	Diagnosis has been made by health professionals before referral
			EFK2	Health referrals can be ensured sustain through entire of health processes
			EFK3	Carry out referral activities to comply with applicable regulations
	Efficiency (EFS)	Provide adequated services to fulfill the needs of patients, thus avoiding multiple services	EFS1	The health officials only provide services to the referred patient
			EFS2	Health referrals make it easy to improve access to health specialists
			EFS3	Health services provided to patients are not the same (redundant) treatment between health facilities
Individual	Personal Beliefs (KPP)	One's personal thoughts that believe in an impact of a susceptibility, seriousness, benefits, barriers, and self-confidence are part of the health belief model. This has an impact on the individual's desire to follow the referral system at the health facility.	KPP1	Incidence of the disease is considered serious and the epidemic is spreading (severity level)
			KPP2	Referral is considered to provide prevention of disease/epidemic
			KPP3	Health services and resources at referral health facilities are considered to be better than previous health facilities (benefit)
			KPP4	A family situation that includes awareness of the referral facility
			KPP5	Official referral can strengthen the relationship between the patient's preferred health facility and the primary health facility
			KPP6	Has financial constraints in meeting referrals
			KPP7	Has transportation constraints in meeting referrals
	Social influence (PSO)	The influence of a patient's decision in approving referrals made by the health facility caused by the surrounding environment (e.g., family, friends, and relatives)	PSO1	Opinion of relatives about certain medical actions on referrals
			PSO2	The influence of health workers advising on the referral process
			PSO3	The effect of social media reviews on referral health facilities

Methods

Research methods

This study has obtained approval from the Head of Public Communications of BPJS-K under number 3184/VIII.2/0317, dated March 14, 2017; thus, this study is in accordance with all regulations, and the authors confirm that informed consent was obtained. The present study uses a quantitative approach, which specifically involves the employing of a questionnaire. This research was executed systematically and gradually. The stages of the study consisted of problem formulation, literature study, research instruments formulation, legibility testing, data collection, data analysis, and conclusion identification (Fig. 3).

Fig. 3

Research stages.

Prior to dissemination, the authors tested the legibility of the questionnaire with nine respondents who had previously used health system referrals to ensure the questionnaire was understandable and unambiguous, and that it conveyed proper sentence structure. Furthermore, those respondents were chosen due to their understanding of the concepts in the questionnaire development as well as their understanding of Indonesian grammar. Following the legibility test, the authors revised the questionnaire based on the comments and suggestions provided by the respondents. Because referral patients may be enrolled in some health facilities, the questionnaire was distributed online through social media (i.e., Facebook and Twitter) and instant messaging platforms, such as LINE and WhatsApp, to reach more respondents within different levels of society. Moreover, we used those forms of media because we are able to disseminate the questionnaire links quickly and easily, as referral patients are spread throughout the provinces in Indonesia. Target respondents were individuals who had participated in at least one health referral process in Indonesia (purposive sampling).

Data analysis was performed using the ordinal logistic regression method, which showed the relationship between predictor variables, to statistically confirm the research model. Ordinal logistic regression was considered an appropriate method for processing this research data to determine whether or not the proposed model was appropriate. Furthermore, this method may demonstrate the association between the response or dependent variable (referral within last six months) and the predictor or independent variable (all variables used in the conceptual model), along with how changes in the predictor variables affected both the response variable and the estimation of the response variable based on the predictor variable [15]. In addition, ordinal regression is an extension of the binomial logistic regression, which determines the value of an ordered logit dependent variable with its predictor variable [16]. All existing datasets should satisfy four assumptions that are to be tested using ordinal regression [17]. Four assumptions that should be met include: (1) there is one dependent variable that has an ordinal data type; (2) there is one or more than one independent variable in continuous, ordinal, or categorical form; (3) there is an independent predictor variable (no multicollinearity); and (4) there are proportional data odds, where the odds ratios of each logit are assumed to be the same (full likelihood ratio test) [17]. After the data is deemed eligible using the assumption test, the ordinal regression test is performed on the dataset to determine the interrelationship between the variables and then achieve the desired output. Data was processed using IBM SPSS Statistic version 24.0.

Instruments

The questionnaire consisted of two parts: demographic data and research questions. In the data section, there were questions related to name, gender, age range, occupation, geographical location, monthly income range, highest level of education, BPJS-K membership, the presence or absence of a referral in the last six months, and the insurance used at the time of referral. According to Table 1, the instrument's questions were based on the model that became the authors' hypotheses. Questions employed a 5-level Likert scale, with level 1 indicating “strongly disagree” and level 5 indicating “strongly agree”. Appendix 1 describes the complete questionnaire.

Results

Respondent demographics

There were 527 respondents who filled out the questionnaire, of which only 290 respondents had received health referrals; thus, we only proceeded with 290 questionnaires for the next steps. Those data are greater than the minimum sample size of 30, as described by Roscoe [29]; thus, those data may be processed in the research. Then, data cleansing was performed on 290 questionnaires to check for duplicates and deleted or missing data, resulting in a total of 283 respondents. A summary of the overall demographics of the study respondents is illustrated in Table 2.

Table 2

Respondent demographics.

Demographics		Number of Respondents (Percentage)
Gender	Men	196 (69.26%)
	Women	87 (30.74%)
Age	<20 years	48 (16.96%)
	21–30 years	189 (66.78%)
	31–40 years	13 (4.59%)
	>40 years	33 (11.66%)
Occupation	Student	145 (51.24%)
	Civil Servant	13 (4.59%)
	Private Employee	70 (24.73%)
	Entrepreneur	9 (3.18%)
	Housewife	14 (4.95%)
	Other	32 (11.31%)
Education Level	Primary, Secondary, and High School	98 (34.63%)
	Diplom	18 (6.36%)
	Bachelor	143 (50.53%)
	Master	20 (7.07%)
	Doctoral	1 (0.35%)
	Other	3 (1.06%)
Revenue per month	<Rp500.000	62 (21.91%)
	Rp500.001 - Rp1.000.000	55 (19.43%)
	Rp1.000.001 - Rp5.000.000	106 (37.46%)
	Rp5.000.001 - Rp10.000.000	40 (14.13%)
	>Rp10.000.000	20 (7.07%)
BPJS-K Participation	Yes	245 (86.57%)
	No	38 (13.43%)
Geographical location	Jabodetabek	158 (55.83%)
	Non- Jabodetabek in Java Island	95 (33.57%)
	Outside of Java Island	30 (10.60%)
Referral in last 6 months	Never	92 (32.51%)
	1-2 times	158 (55.83%)
	3-4 times	21 (7.42%)
	5-6 times	5 (1.77%)
	More than 6 times	7 (2.47%)
Insurance preferences at referrals	BPJS-K	192 (67.84%)
	Private insurance	58 (20.49%)
	Other	33 (11.66%)

Data analysis using ordinal logistic regression

In this study, there were both dependent and independent variables. There were nine acceptance factors for the referral of the independent variable: patient centricity, regulation, data security, responsiveness, integration, effectiveness, efficiency, personal beliefs, and social influence. The dependent variable in this study was the acceptance of the referral system obtained from the frequency of patients' use of the health referral system (patient arrivals). The frequency of use of the health referral system was weighted on an ordinal scale from 0 to 3. The scoring was recorded as such: “0” indicates arrival occurred zero times (no arrival); “1” indicates arrival occurred between one and two times; “2” indicates arrival occurred between three and four times; and “3” indicates arrival occurred more than five times. All frequency of use was calculated within only six months from the most recent reference.

In an ordinal regression method, one answer was used as the basis with which the other answers could be compared. For example, the answer with a significant “influence” value, when compared to the base, was believed to affect the reference. Ordinal regression was used when there were more than two response variables that were not scalar, but the coding process was conducted so that they were ordinal. Data that may be processed in an ordinal regression test constituted continuous data because regression is a quantitative test, and therefore ordinal data, such as a 1–5 Likert scale, and nominal data must be changed into indicator variables or dummy variables to make them quantitative with one attribute that was used as a base [26]. The regression model on ordinal variable responses is a continuation of logistic regression for dichotomous data.

Ordinal regression is processed the same as is binomial logistic regression, as it compares two categories of two dependent variables. However, in ordinal regression variables, categories are combined and combined in one category according to the chosen approach in the form of dummy variables. The approach used in this study occurred by combining both the agreed and disagreed values on the Likert scale. The merger produced two new variables, namely the dummy of the agreeing variable and the dummy variable that did not agree, and the neutral variable became the basis. This merger was facilitated to make it easier to find out whether the direction of significance of each independent variable was positive or negative. In addition, ordinal regression equation notation (Eq. (1)) was adapted from the following work [16]:

Furthermore, this section discusses the data processing step using ordinal logistic regression along with the assumption test that was performed on the dataset. The ordinal logistic regression assumption test is a multicollinearity test; the full likelihood ratio test was performed prior to the regression test [17] and was followed by reliability testing on each variable. Hypothesis testing was performed on the ordinal logistic regression test to determine the acceptance of the overall model and the significance of each variable.

The multicollinearity test

A multicollinearity test was performed to test independence between variables. Multicollinearity occurs when two or more independent variables have a high correlation between them, which makes it difficult to determine the contribution of each variable to the dependent variable [17]. To test multicollinearity, we used a correlation coefficient between the Variance Inflation Factors (VIF) free variable and the tolerance value. These variables were considered to have passed the multicollinearity test if all tolerance values were greater than 0.1 or all VIF values were below 10; the results here indicate that all variables passed these tests. Therefore, there was no correlation between the present variables; thus, the multicollinearity test was passed.

The full likelihood ratio test

The full likelihood ratio test was performed to determine the proportional odds of the dataset. Proportional odds are a basic assumption of ordinal logistic regression. This assumption states that each independent variable has an identical effect on each cumulative split of the dependent ordinal variable [17]. According to Lund, the full likelihood ratio test is necessary because odds ratios for each cumulative logit are assumed to be equal so that the odds ratios for each dichotomous variable are also the same [17]. This test was executed by comparing the two models that can be seen in the -2 log likelihood between the null hypothesis—which is the proportional odds model—and the general model, without assuming proportional odds. This test can be rejected if the comparison between the two models is statistically significant (p < 0.05). The full likelihood ratio test results may be observed in Table 3. In the test results, the obtained value -2 log likelihood on the null hypothesis model was 486.891, and the value on the general model was 398.766. The chi-square value on the general model was 486,891, which is the interval between -2 log likelihood null hypothesis and the general model, with a degree of freedom (df) of 132. The resulting significance value was 0.999, thus indicating the full likelihood ratio test was passed.

Table 3

Full likelihood ratio test results.

Model	-2 Log Likelihood	Chi-square	Df	Sig.
Null Hypothesis	486.891
General	398.766	88.125	132	0.999

Validity and reliability test

In order to analyze the correlation between independent variables, we calculated the discriminant validity. Table 4 demonstrates that there exists no correlation between indicators from different independent variables. Furthermore, the most common instrument used to measure reliability is Cronbach's Alpha (CA); specifically, it is used to measure internal consistency and determine the number of questions in the same dimension [17]. Usually, CA is used with research instruments that use the Likert scale to determine whether or not the measurements are correct. Questionnaires may be used when processed data has passed the reliability test. According to Lund, CA values range from 0 to 1 with an acceptance value above 0.6 [30]; the greater the CA estimates, the more consistent the measuring tool [17]. Results were acquired from 66 indicators of research. The resulting CA value was 0.753; since this is greater than 0.76, the instrument of this study is considered consistent and can be viably used for research purposes. Table 5 describes the CA score for each variable, and all variables possess a CA value >0.6.

Table 4

Discriminant validity results.

	ACC	AMN	EFK	EFS	INT	KPP	PSO	PST	REG	RSP
ACC	1.000
AMN	-0.160	0.573
EFK	0.053	-0.021	0.462
EFS	0.072	0.310	-0.059	0.770
INT	-0.038	0.444	0.078	0.400	0.837
KPP	0.140	-0.035	-0.063	-0.069	-0.050	0.470
PSO	0.030	0.098	-0.047	0.150	0.135	0.187	0.784
PST	0.127	0.058	-0.032	0.361	0.088	-0.160	0.012	0.523
REG	-0.146	0.400	-0.018	0.252	0.243	-0.030	0.053	0.183	0.570
RSP	-0.051	0.142	0.021	0.399	0.318	-0.132	0.021	0.257	0.157	0.721

Table 5

Cronbach's Alpha (CA) values.

Variables	Cronbach's Alpha (CA)
AMN	0.708
EFK	0.737
EFS	0.679
INT	0.794
KPP	0.641
PSO	0.706
PST	0.810
REG	0.773
RSP	0.819

Ordinal logistic regression test

After testing assumptions on the dataset, the ordinal logistic regression test may be performed. This test uses ordinal regression with dummy variables on the agreed Likert scale and does not agree with the reference parameter of the respondents who answered neutrally. The results obtained from this test constitute the model-fitting information shown in Table 6, which is a likelihood test that determines whether or not the relationship between the dependent variable and the overall independent variable is statistically significant [16]. The relationship is accepted on the model when the value of Sig. below 0.1 possesses a significance level of 10%. In addition to determining the significance of the overall model, the influence of each indicator on the model may determine which indicator most significantly affects the model. Each indicator was tested with two dummy variables: agree (S) and disagree (TS), with reference to the neutral variable.

Table 6

Model-fitting information.

Model	-2 Log Likelihood	Chi-square	Df	Sig.
Intercept Only	569.224
Final	486.891	83.060	66	0.084∗∗∗

Significant at coefficient of 10%.

The results of the model-fitting information in Table 4 produced an “intercept only” value of 569.224, final value of 486,891, and chi-square value of 83,060. The resulting df value was 66 and the resulting significance value was 0.084, which shows that the relationship between the dependent variable and the independent variable was significant because the value was below 0.1 at the 10% level.

Organizational dimension

There are two variables in the organization dimension: patient centricity (PST) and regulation (REG). PST has four indicators, and REG has five indicators. Each indicator was checked to determine whether or not it had a significant influence on the overall model. PST had a significant influence on predicting the acceptance of health referrals. Table 7 illustrates a magnitude of contributions given by each indicator, while in PST, each indicator did not contribute a significant influence. The value is based on the Sig. column, which should have a value below 0.1 to be considered statistically significant. For this variable, the most closely related indicator is the educational trend of the patient, with an estimated value 0.530 times greater than the neutral answer and a significance value of 0.147; the health service indicator was less than 0.639 times that of the neutral answer, with a significance value of 0.251.

Table 7

Influence of indicators in overall model on patient centricity variables (PST).

Indicator	Estimate	Std. Error	Wald.	df	Sig.	95% CI
						Lower Bound	Upper Bound
PST1: Health Service (S)	-0.187	0.465	0.161	1	0.688	-1.098	725
PST1:Health Service (TS)	-0.639	0.557	1.316	1	0.251	-1.731	0.453
PST2: Education (S)	0.530	0.365	2.107	1	0.147	-0.186	1.245
PST2: Education (TS)	0.609	0.525	1.345	1	0.246	-0.420	1.637
PST3: Health Worker Availability (S)	-0.026	0.410	0.004	1	0.950	-0.829	0.778
PST3: Health Worker Availability (TS)	0.485	0.498	0.947	1	0.330	-0.429	1.461
PST4: Monitor (S)	0.485	0.373	0.183	1	0.669	-0.571	0.890
PST4: Monitor (TS)	-0.313	0.415	0.569	1	0.451	-1.127	0.501

Overall, the regulation variables had a significant influence on predicting the acceptance of health referrals. In Table 8, it can be observed that the contribution given by each indicator in the regulation variables are the most significant among other indicators, with an estimated value that is 0.789 times smaller than the neutral answer. Guidelines and regulations were very positively influential on the acceptance of referrals; thus, sufficient guidelines and regulations greatly affect the acceptance of health referrals.

Table 8

Influence of indicators in overall model on regulation variables (REG).

Indicator	Estimate	Std. Error	Wald.	df	Sig.	95% CI
						Lower Bound	Upper Bound
REG1: Utilization of IT (S)	0.545	0.453	1.448	1	0.229	-0.343	1.434
REG1: Utilization of IT (TS)	0.593	0.479	1.535	1	0.215	-0.345	1.531
REG2: Inclusion Criteria (S)	0.129	0.458	0.079	1	0.779	-0.770	1.027
REG2: Inclusion Criteria (TS)	-0.925	0.585	2.503	1	0.114	-2.072	0.221
REG3: Clear Regulation (S)	-0.789	0.426	3.426	1	0.064∗∗∗	-1.625	0.046
REG3: Clear Regulation (TS)	-0.627	0.482	1.689	1	0.194	-1.572	0.318
REG4: Regulator (S)	0.722	0.402	3.225	1	0.073∗∗∗	-0.066	-1.510
REG4: Regulator (TS)	1.090	0.678	2.585	1	0.108	-0.239	2.420
REG5: Structural Documentation (S)	0.549	0.386	2.020	1	0.155	-0.208	1.305
REG5: Structural Documentation (TS)	-0.548	0.664	0.681	1	0.409	-1.850	0.753

Significant at coefficient of 10%.

Technological dimension

There are three variables in the technological dimension: data security (AMN), responsiveness (RSP), and integration (ITG). AMN, RSP, and ITG each have three indicators. Each indicator was checked for whether or not it had a significant effect on the overall model. Overall security variables had a significant influence on predicting the acceptance of health referrals. Table 9 shows the contribution given by each indicator; for the security variable, the access authority indicator was the most significant, with an estimated value of tending to disagree 1.965 times greater than those who felt neutral about this issue. The access to authority was most influential among all the indicators for this variable. All existing indicators had an effect on health referrals; therefore, sufficient system security will affect the acceptance of referrals.

Table 9

Influence of indicators in overall model on data security variables (AMN).

Indicator	Estimate	Std. Error	Wald.	df	Sig.	95% CI
						Lower Bound	Upper Bound
AMN1: System Centralization (S)	0.064	0.400	0.026	1	0.872	-0.720	-0.849
AMN1: System Centralization (TS)	-0.613	0.420	2.124	1	0.145	-1.437	0.211
AMN2 Access Authority (S)	-0.265	0.392	0.458	1	0.499	-1.034	0.504
AMN2 Access Authority (TS)	-1.965	0.909	4.676	1	0.031∗∗	-3.746	-0.184

Significant at coefficient 5%.

Overall responsiveness variables significantly influenced the predicting of health referral acceptance. Table 10 demonstrates the contribution given by each indicator; for the responsiveness variable, the immediate indicator without delaying service is the most significant indicator, with an estimated value 1.145 times greater than the neutral answer and a significance value of 0.015. The responsiveness of the referral system affected the acceptance of the referral system; therefore, the more responsive the existing referral system, the greater the impact on the acceptance of a referral.

Table 10

Influence of indicators in overall model on responsiveness variables (RSP).

Indicator	Estimate	Std. Error	Wald.	Df	Sig.	95% CI
						Lower Bound	Upper Bound
RSP1: Commitment (S)	-0.217	0.425	0.260	1	0.610	-1.051	-0.617
RSP1: Commitment (TS)	0.010	0.676	0.000	1	0.988	-1.315	1.335
RSP2: Bureaucracy (S)	-0.034	0.450	0.006	1	0.939	-0.916	0.847
RSP2: Bureaucracy (TS)	-0.659	0.496	1.768	1	0.184	-1.631	0.312
RSP3: Not Delayed (S)	1.145	0.469	5.956	1	0.015∗∗	0.225	2.064
RSP3: Not Delayed (TS)	0.979	0.484	4.080	1	0.043∗∗	0.029	1.928

Significant at coefficient 5%.

Overall integration variables significantly influenced the prediction of health referral acceptance. Table 11 shows the contribution given by each indicator; the patient data indicator is the most significant, with an estimated value 1.066 times smaller than the neutral answer in the positive direction.

Table 11

Influence of indicators in overall model on integration variables (ITG).

Indicator	Estimate	Std. Error	Wald.	df	Sig.	95% CI
						Lower Bound	Upper Bound
ITG1: Patient Data (S)	-1.066	0.390	7.470	1	0.006∗	-1.830	-0.301
ITG1: Patient Data (TS)	-0.770	0.473	2.652	1	0.103	1.697	0.157
ITG2: Doctor Data (S)	0.316	0.472	0.450	1	0.502	-0.608	1.241
ITG2: Doctor Data (TS)	0.729	0.493	2.188	1	0.139	-0.237	1.694
ITG3: Doctor Schedule Data (S)	-0.408	0.477	0.731	1	0.393	-1.342	0.527
ITG3: Doctor Schedule Data (TS)	0.240	0.487	0.243	1	0.622	-0.714	1.195

Significant at coefficient 1%.

Process dimension

There are two variables in the process dimension: effectiveness (EFK) and efficiency (EFS), each of which possess three indicators. Each indicator was checked for whether or not it had a significant effect on the overall model. Overall effectiveness variables significantly influenced the predicting of health referral acceptance. Table 12 shows the contribution given by each indicator; the continuous referral indicator is the most significant, with an estimated value 1.407 times less frequent than those who felt neutral about this issue and a significance value of 0.002.

Table 12

Influence of indicators in overall model on effectiveness variables (EFK).

Indicator	Estimate	Std. Error	Wald.	df	Sig.	95% CI
						Lower Bound	Upper Bound
EFK1: Diagnosis (S)	0.586	0.400	2.146	1	0.143	-0.198	1.371
EFK1: Diagnosis (TS)	0.904	0.527	2.942	1	0.086∗∗∗	-0.129	1.936
EFK2: Sustainable Health Process (S)	-1.407	0.447	9.905	1	0.002∗	-2.283	-0.531
EFK2: Sustainable Health Process (TS)	-0.209	0.618	0.115	1	0.735	-1.421	1.002
EFK3: Comply with Regulation (S)	-0.093	0.399	0.054	1	0.817	-0.875	0.690
EFK3: Comply with Regulation (TS)	-0.676	0.753	0.805	1	0.370	-2.152	0.800

Significant at coefficient 1%.

Significant at coefficient 10%.

Overall, the regulation variables had a significant influence on predicting the acceptance of health referrals. Table 13 illustrates the contribution given by each indicator; the service focus indicator is the most significant, with an estimated value of 1.085 and a significance value of 0.016. The focus of services is very influential on the acceptance of referrals, demonstrating that good service is very influential.

Table 13

Influence of indicators in overall model on efficiency variables (EFS).

Indicator	Estimate	Std. Error	Wald.	df	Sig.	95% CI
						Lower Bound	Upper Bound
EFS1: Focus on Referral Service (S)	1.085	0.450	5.806	1	0.016∗∗	0.202	1.968
EFS1: Focus on Referral Service (TS)	1.223	0.801	2.334	1	0.127	0.346	2.792
EFS2: Access (S)	0.022	0.454	0.002	1	0.961	-0.867	0.911
EFS2: Access (TS)	0.313	0.637	0.241	1	0.623	-0.936	1.561
EFS3: Not Redundant Service (S)	-0.678	0.353	3.684	1	0.055∗∗∗	-1.370	0.014
EFS3: Not Redundant Service (TS)	-0.238	0.436	0.299	1	0.585	-1.903	0.616

Significant at coefficient 5%.

Significant at coefficient 10%.

Individual dimension

There are two variables in the individual dimension: personal beliefs (KPP), which has seven indicators, and social influence (PSO), which as three indicators. Each indicator was checked for whether or not it had a significant effect on the overall model. Overall, the KPP variable significantly influenced the predicting of health referral acceptance. Table 14 shows the amount of contribution given by each indicator; the benefit indicator is the most significant, with an estimate value 0.923 times greater than the neutral answer and a significance value of 0.021. In addition, susceptibility indicators significantly influenced the acceptance of referrals, with a value 0.881 times less than the neutral answer.

Table 14

Influence of indicators in overall model on personal beliefs variables (KPP).

Indicator	Estimate	Std. Error	Wald.	df	Sig.	95% CI
						Lower Bound	Upper Bound
KPP1: Severity (S)	0.615	0.501	1.506	1	0.220	-0.367	1.597
KPP1: Severity (TS)	1.007	0.808	1.553	1	0.213	-0.576	2.590
KPP2: Susceptibility (S)	-0.881	0.499	3.121	1	0.077∗∗∗	-1.859	0.096
KPP2: Susceptibility (TS)	-0.494	0.768	0.413	1	0.520	-1.999	1.012
KPP3: Benefit (S)	0.923	0.399	5.342	1	0.021∗∗	0.140	1.706
KPP3: Benefit (TS)	0.721	0.793	0.827	1	0.363	-0.834	2.276
KPP4: Awareness (S)	-0.311	0.445	0.488	1	0.485	-1.183	0.561
KPP4: Awareness (TS)	-0.132	0.821	0.026	1	0.872	-1.741	1.476
KPP5: Relationship (S)	-0.618	0.350	3.127	1	0.077∗∗∗	-1.304	0.067
KPP5: Relationship (TS)	-0.374	0.618	0.367	1	0.545	-1.585	0.837
KPP6: Financial Barrier (S)	0.070	0.467	0.022	1	0.881	-0.845	0.985
KPP6: Financial Barrier (TS)	-0.376	0.435	0.748	1	0.387	-1.228	0.476
KPP7: Transportation Barrier (S)	-0.265	0.454	0.341	1	0.559	-1.156	0.625
KPP7: Transportation Barrier (TS)	-0.006	0.452	0.000	1	0.990	0.893	0.881

Significant at coefficient 5%.

Significant at coefficient 10%.

The overall social influence variable had a significant influence on predicting the acceptance of health referrals. Table 15 shows the contribution given by each indicator; the social media review indicator for the most disagreeable had an estimated value of 1.022, which is greater than the neutral answer, and a significance value of 0.0016.

Table 15

Influence of indicators in overall model on social influence variables (PSO).

Indicator	Estimate	Std. Error	Wald.	df	Sig.	95% CI
						Lower Bound	Upper Bound
PSO1: Relatives (S)	0.238	0.423	0.316	1	0.574	-0.591	1.066
PSO1: Relatives (TS)	0.177	0.422	0.176	1	0.675	-0.651	1.004
PSO2: Health Workers (S)	-0.013	0.410	0.001	1	0.974	-0.817	0.790
PSO2: Health Workers (TS)	-0.789	0.503	2.463	1	0.117	-1.774	0.196
PSO3: Social Media Review (S)	0.228	0.421	0.295	1	0.587	-0.596	1.053
PSO3: Social Media Review (TS)	1.022	0.424	5.801	1	0.016∗∗	0.190	1.853

Significant at coefficient 5%.

Hypothesis testing result

The following sections explain the data analysis by linking the results to the research hypotheses. Based on the model-fitting information, the regression model has fulfilled the significance of 0.084 at the 10% level. Regarding H1, service indicators had a 0.187-times smaller chance for respondents who sensed that health service indicators possessed more frequency in receiving referrals, whereas those who did not feel this way had a 0.639-times smaller chance to conduct health referrals than those respondents who answered neutrally. In addition to indicators of information and education provided by health personnel, educated patients were 0.530 times more likely to receive health referrals, while patients who were not educated were 0.609 times less likely to receive referrals. This leads to the indication that a patient's education regarding the existence of health referrals was neutral and could be represented by other indicators. Meanwhile, those who disagreed with referrals tended to be 0.485 times more likely to be referred than those who were neutral. Regarding monitoring indicators, patients who felt monitored by health personnel received referrals 0.485 times more frequently than those who were neutral on this topic, whereas patients who did not feel monitored by health personnel received health referrals 0.313 times less frequently. However, the variable relationship between patient centricity and referral admissions did not significantly affect the model because no indicators demonstrated statistical significance. These results are not in accordance with Senitan et al., who stated that one of the contributing factors in the referral system is, in fact, patient centricity [12].

Next, regarding H2, regulation variables showed that patients who felt that IT had been implemented in health facilities tended to receive referrals 0.545 times more frequently than those who answered neutrally. However, patients who perceived a lack of IT utilization were 0.593 times more likely to receive referrals than those who answered neutrally; therefore, IT utilization was neutral and could be represented by other indicators. Patients who received inclusion criteria tended to be 0.129 times more likely to receive referrals than those who answered neutrally and received referrals 0.925 times more frequently than those who did not know the criteria for referral services. Patients who felt they were presented with clear rules and regulations tended to be 0.789 times less likely to receive referrals; this result was statistically significant, with a 0.064 significance value at the 10% coefficient. Meanwhile, patients who did not feel there were clear rules and regulations tended to be 0.627 times less likely to receive health referrals, and thus these variables tended to be neutral and could be represented by other variables. Patients who did not feel there was a clear regulator were 1.090 times more likely to receive referrals, and thus the regulator became neutral and may be represented by other indicators. On the other hand, patients who experienced structured documentation for referrals were 0.549 times more likely to use a referral, and those who did not feel there was structured documentation were referred 0.548 times less frequently than those who felt neutral about this issue; since this variable was determined to be statistically significant, regulation significantly influenced the model. These results are in accordance with Senitan et al., who mentioned the need for clear and structured regulations that are implemented by regulators in all health networks in Indonesia [12].

Regarding H3, patients who felt there was a centralization of data transfer in the health system tended to receive referrals 0.064 times more frequently than those who felt neutral about this issue. Patients who experienced the absence of centralized systems tended to be 0.613 times less likely to receive referrals than those who felt neutral about this issue. Patients who felt that health data could only be accessed by authorized personnel were 0.265 times less likely to receive referrals, whereas patients who believed health data could be accessed by unauthorized parties were 1.965 times less likely to receive referrals than those who felt neutrally, which is statistically significant. These results are in accordance with the U.S. Public Health Service's HIPAA regulation and Senitan et al.‘s research, both of which require the transfer of health data centrally using IT systems [12]. The AMN3 indicator is not included in the regression test because, with this indicator, the overall model fit value becomes insignificant (i.e., greater than 0.1), and this indicator is considered irrelevant to the whole model.

Furthermore, regarding H4, patients who acknowledged the commitment of health personnel tended to use referrals 0.217 times less frequently than those who answered neutrally, whereas those who did not acknowledge the commitment of health workers used health referrals 0.010 times more frequently. Later, patients who had not been receiving health referrals for long time had a 0.034 lower chance of receiving referrals than those who felt neutral about this issue. However, patients who seldom felt the length of the bureaucracy received referrals up to 0.659 times more frequently. Patients who felt that referrals were administered immediately were 1.145 times more likely to use a referral again than those who were neutral, while those who did not feel referrals were administered immediately received referrals 0.979 times more frequently than those who were neutral. These two indicators had a significant value in the 5% coefficient; therefore, the responsiveness of the service significantly influenced the model. These results are consistent with the research conducted by Macintyre et al. and Senitan et al. concerning the responsiveness of health referral services to make patients more willing to follow health referrals [8, 12].

Moreover, regarding H5, patients who acknowledged the occurrence of more patient data delivery to the next health facility tended to follow referrals 1.066 times less frequently than those who felt neutral about this issue, which is statistically significant at 0.006 with a coefficient of 5%; on the other hand, patients who did not acknowledge the occurrence of patient data delivery tended to be referred less frequently and were 0.770 times less likely to receive referrals than those who were neutral. Patients who felt the doctors' data on subsequent health facilities were owned by the first health facility tended to receive referrals 0.316 times more frequently than those who were neutral, whereas patients who did not perceive that physician data was owned by the first health facility used referrals 0.729 times more frequently. Patients who perceived the doctor's schedule as belonging to the second health facility tended to be 0.408 times less likely to receive referrals than patients who did not. Therefore, integration of the health referral service significantly affected the model. These results are consistent with Fyie et al., who mentioned the need for integration of one health system with other systems along with a sufficient doctors' schedule and hours of practice at existing health facilities [9].

Then, regarding H6, patients who felt their diagnosis had been made prior to the referral tended to receive referrals 0.586 times more frequently than those who felt neutral about this issue, whereas those who did not feel such a way received referrals 0.904 times more frequently than those who felt neutral about this issue, with a 0.086 significance value at the 10% coefficient. Those who did not experience sustainability had a tendency to receive referrals 0.209 times less frequently than those who were neutral. Patients who felt that referrals were in compliance with the regulations tended to be 0.093 times less likely to receive referrals than those who were neutral, while those who did not feel the regulations were met were 0.675 times less likely to receive referrals than those who did not receive referrals. Therefore, the reference variable significantly affected the model. These results are in accordance with research conducted by Ajwang and Senitan et al., who stated that the effectiveness of referrals is based on scientific knowledge as well as health personnel ensuring no further extension of these references [7, 12].

Regarding H7, patients who experienced the provision of health services in referred patients tended to use referrals 1.085 times more frequently than those who felt neutral about this issue, with a significance value of 0.016 at the 5% coefficient. Meanwhile, patients who did not experience the provision of optimal health services tended to use referrals 1.223 times less frequently than those who felt neutral about this issue. Patients who felt referrals could be used to access specialists used referrals 0.022 times more frequently than those who felt neutral about this issue, and patients who did not use referrals were 0.313 times less likely to receive referrals than those who felt neutral about this issue. Patients who felt there was no service redundancy tended to use referrals 0.678 times less frequently, with a 0.055 significance value at the 10% coefficient. Patients who felt there was a service redundancy tended to be 0.238 times less likely to receive referrals than those who felt neutral about this issue. Considering the significance of the service focus, efficiency influenced the model significantly. These results are consistent with Ajwang and Senitan et al., who claimed that efficient health service can be a factor that facilitates patients' conduct regarding health referrals [7, 12].

Next, regarding H8, patients who considered the seriousness or severity of their disease tended to be 0.615 times more likely to receive referrals than those who felt neutral about this issue. Those who did not consider the severity of their disease tended to use referrals 1.007 times more frequently than those who felt neutral about this issue, whereas patients who felt susceptibility rates in their disease tended to be 0.881 times less likely to receive referrals, with a significance value of 0.077 at the 10% coefficient. Patients who considered susceptibility levels used referrals 0.881 times less frequently than those who felt neutral about this issue, and those who did not consider susceptibility levels used referrals 0.494 times less frequently than those who felt neutral about this issue. Patients who benefited from health referrals tended to be 0.923 times more likely to use referrals than those who felt neutral about this issue, with a 0.021 significance value at the 5% coefficient, whereas those who did not feel this way used referrals 0.721 times more frequently than those who felt neutral about this issue. Patients who possessed awareness of referrals tended to receive referrals 0.311 times less frequently, and patients who were not aware of referrals were 0.132 times less likely to receive them. Patients who believed that referrals may strengthen relationships among health facilities used referrals 0.618 times less frequently than those who felt neutral about this issue. Patients who did not trust that referrals could strengthen the relationship between health facilities received referrals 0.374 times less frequently than those who were neutral. Patients with financial constraints tended to receive referrals 0.070 times more frequently than those who felt neutral about this issue, and people without financial constraints tended to be 0.376 times less likely to receive referrals. Patients with transport constraints received referrals 0.265 times less frequently than those who felt neutral about this issue, and patients who did not have such constraints received referrals 0.006 times less frequently than those who felt neutral about this issue. By looking at the significance of benefits, it can be asserted that personal confidence variables significantly affected the model. These results are in accordance with research conducted by Ajwang and Macintyre et al., who stated that referrals occur from a person's personal perspective, such as the view of an existing disease and the benefits of health services regarding his or her health [7, 8].

Regarding H9, patients who listened to opinions from relatives tended to be 0.238 times more likely to receive referrals than those who felt neutral about this issue. Additionally, patients who were unaffected by relatives' opinions tended to receive referrals 0.177 times more frequently than those who felt neutral about this issue. Patients affected by the opinions of medical personnel tended to be 0.013 times less likely to receive referrals than those who were neutral, whereas unaffected patients tended to use referrals 0.789 times less frequently. Patients who looked at reviews on social media tended to use referrals 0.228 times more frequently than those who felt neutral about this issue. Patients who were not affected by social media reviews were 1.022 times more likely to receive referrals than those who felt neutral about this issue, with a 0.016 significance value at the 5% coefficient. Given the significance of the inability of the patients, social media influenced the model significantly. These results are in line with Holden and Karsh, who stated that influence from neighbors, medical personnel, and outsiders may influence patients to follow a healthcare process [13]. The overall ordinal regression equation can be written as follows:

The left side of Eq. (2) explains the dependent variable obtained by using the “ln” function. The right side of Eq. (2) conveys the threshold coefficient value of the logit level of the dependent variable. Then, the value is summed with both dummy variables of each indicator with an estimated coefficient. In this equation, each sigma value represents each latent variable of the whole indicator. Overall, this model has a significant value because independent variables affected the dependent variable with a significance value of 0.084 at the 10% coefficient. In the overall test, this model did not run the test on the AMN3 indicator because the indicator may already be represented by AMN2, which is related to data access authorization. Table 16 summarizes the results of regression tests on each hypothesis.

Table 16

Summary of ordinal regression test results on each hypothesis.

No.	Variable	Result
1.	Patient Centricity (PST)	Does not significantly influence the model
2.	Regulation (REG)	Influences the model significantly
3.	Data Security (AMN)	Influences the model significantly
4.	Integration (ITG)	Influences the model significantly
5.	Responsiveness (RSP)	Influences the model significantly
6.	Effectiveness (EFK)	Influences the model significantly
7.	Efficiency (EFS)	Influences the model significantly
8.	Personal Beliefs (KPP)	Influences the model significantly
9.	Social Influence (PSO)	Influences the model significantly

Discussion

This study has described that the health referral system in Indonesia is influenced by dimensions of organization, technology, process, and individuality. The only factor that does not significantly represent the acceptance of referrals in Indonesia is patient centricity (organization dimension). This contrasts with Senitan et al., who stated that referrals should be able to provide patient-centered services, meet their needs, and provide consultations to raise awareness within the patient himself [12]. A study conducted by Notoatmodjo and Nainggolan concerning community outlooks on community health centers found that patients tend to opt for treatment at health centers because they are close in proximity and are generally inexpensive [18]. This claim is in line with Varshney, who revealed that, in developing countries, patients tend to be less aware of health services [19]. Next, people who tend to agree with the regulations receive referrals more frequently than those who do not. The public is sufficiently aware that regulations apply and that an agency regulates the process. In addition, official documentation is standardized on referrals. This is consistent with Senitan et al., who stated that referrals should be clearly defined by the relevant agencies and their rules [12].

In addition, referral communities rarely have a large or significant tendency to disbelieve the security of existing data on health facilities, and thus these indicators may affect other indicators of security variables. This is in line with the HIPAA regulations and research by Senitan et al. regarding data access authorities and the centralization of security data transfer [12]. This was also reinforced by Kuzu et al., who said that, in developing countries, patients desire protection of their health data [20]. Thus, Fyie et al. and Senitan et al. stated that the referral service should be implemented promptly and executed by committed staff, and referrals should be ready to serve patients and should not require a long bureaucratic process [9, 12]. As a developing country, all regions in Indonesia certainly do not possess the same health service standards. Referencing the International Forum on Indonesian Development's (INFID) research on the perception of inequality of public services in Indonesia, 42.3% of Indonesian people have still not acknowledged the existence of appropriate health services. In addition, according to a report given by the World Bank in 2016, there still exists an inequality of health opportunities acquired by the Indonesian people from technical, non-technical, and structural aspects. Furthermore, according to Fyie et al., both the interconnection between health systems and the availability of consultation schedules among health facilities influence the acceptance of health referral services [9]. According to Varshney and Kuzu et al., developing countries still do not possess adequate infrastructure for implementing the integration of their health services, and not everyone has been well-educated or made aware of the importance of integrated services [19, 20].

Moreover, patients who did not receive diagnosis and patients who agreed upon the continued presence of health facilities were significantly less likely to receive referrals. According to Ajwang and Senitan et al., effectiveness referrals are references based on science and applicable standards [7, 12]. The referral is made by diagnosing the patient and implementing documentation to ensure a follow-up occurs on the referral [19, 20]. Lack of health awareness and education are reasons why many people may not be aware of the need to receive a diagnosis and follow up on referrals from previous health facilities. Lack of education also causes people to prefer seeking treatment from alternative providers that may not meet health standards rather than seeking treatment from official health facilities. In addition, research by Ajwang and Senitan et al. indicated that limited healthcare to referred patients as well as limited accessibility to specialists possess a relationship with referral acceptance [7, 12]. According to Sarasati, ill behavior is viewed from the perspectives of both the patient and the healthcare worker [22]. Thus, the effectiveness of promoting appropriate service delivery has a lower influence on the acceptance of referrals [19, 20].

Ajwang asserted that an individual's view of an outbreak or disease may affect the acceptance of a referral [7]. Ajwang also provided a key opinion on the personal confidence indicators using the health belief model framework as perceived benefit, which may be proven statistically [7]. Other health belief model indicators did not show significant results or relatively neutral directions. However, these indicators may be represented by perceived benefit indicators because these indicators represent the first step for patients to prevent or cure diseases in order to affect other health belief model indicators, such as severity, susceptibility, and barriers. The financial constraints described in the study are relatively positive, although insignificant, because people in developing countries who are otherwise unable to attend an official health facility tend to seek treatment from alternative sources, as explained by Varshney [19]. Consequently, communities that are less likely to be affected by social media reviews are significantly more likely to receive referrals. This contrasts with the results achieved by Balatsoukas regarding the role of social media in the intervention of technology acceptance in the health context [14]. These contrasts may occur because developing-country societies use social media to acquire information, but not all members are aware or educated about health; therefore, despite the amount of circulating information, people tend to believe in alternative medicine. Furthermore, communities in Indonesia prefer to compare alternative and general medicinal perspectives.

The results of this study may enrich research related to health referrals, as this field is still growing and requires more relevant literature. Based on pre-existing research, this study combines nine factors that ideally influence patients' acceptance of health referrals: patient centricity, regulation, security, responsiveness, integration, effectiveness, efficiency, personal beliefs, and social influence. Of these, all but “patient centricity” significantly affect patients' acceptance of referrals. These results may be influenced by demographic, geographic, and cultural factors in Indonesian society that differ from other countries. Thus, this study may define the health referral acceptance model in Indonesia, exclusively.

Health facilities and regulators may consider the factors and supporting data analysis to provide a fair and equitable service for all the people of Indonesia. Specifically, they may improve the utilization of IT namely for Electronic Medical Record (EMR) development in order to facilitate the exchange of patient data. Furthermore, health facilities should define guidebooks to facilitate communication and standardization among health facilities, improve the safety of patient data, develop a centralized system for patient data storage, and provide benefits and optimal health services for patients. Understanding these factors can help providers make using referrals easier for patients by, for example, shortening queues for healthcare services, providing quality services, and giving patients substitutes for the alternative medicinal options available in the area.

Conclusions

This study has demonstrated that regulation, data security, integration, responsiveness, effectiveness, efficiency, personal beliefs, and social influence have significant effects on the acceptance of health referrals, while patient centricity does not create any significant impact. These results are due to patients' tendencies to seek treatment at more inexpensive facilities closer to home as well as the lack of general awareness in developing countries regarding what is required for adequate healthcare. In addition, there still exists some inequality in the services received by low-income communities, including unequal access to education, health, and information, which may lead to discrepancies in the acceptance of health-related information. Another factor that causes people to avoid receiving referrals is their tendency to seek treatment of non-medical treatment, such as alternative or traditional medicine outside the official health facilities. This study only discusses acceptance from the patient's perspective and does not interact directly with health facilities or regulators. Further research in this area may improve the design of the integrated referral information system between the regulators, the health facilities, and the patients.

Declarations

Author contribution statement

Putu Wuri Handayani, Ibad Rahadian Saladdin, Ave Adriana Pinem: Performed the experiments; Analyzed and interpreted the data; Contributed reagents, materials, analysis tools or data; Wrote the paper.

Fatimah Azzahro, Achmad Nizar Hidayanto, Dumilah Ayuningtyas: Conceived and designed the experiments.

Funding statement

This work was supported by the Directorate General of Higher Education for the Program Dasar Penelitian Unggulan Perguruan Tinggi (PDUPT), grant number 286/UN2.R3.1/HKP05.00/2018, as well as by the Directorate of Research and Community Engagement at the University of Indonesia.

Competing interest statement

The authors declare no conflict of interest.

Additional information

No additional information is available for this paper.