Impact of doctors' resistance on success of drug utilization review system.

OBJECTIVES: The drug utilization review (DUR) system, which checks any conflict event of medications, contributes to improve patient safety. One of the important barriers in its adoption is doctors' resistance. This study aimed to analyze the impacts of doctors' resistance on the success of the DUR system.
METHODS: This study adopted an augmented the DeLone and McLean Information System (D&M IS) Success Model (2003), which used doctors' resistance as a socio-technological measure. This study framework is the same as that of the D&M IS Success Model in that it is based on qualities, such as system, information, and services. The major difference is that this study excluded the variable 'use' because it was not statistically significant for mandatory systems. A survey of doctors who used computers to enter prescriptions was conducted at a Korean tertiary hospital in February 2012.
RESULTS: This study is very meaningful in that it is the first study to explore the success factors of the DUR system associated with doctors' resistance. Doctors' resistance to the DUR system was not statistically associated with user usefulness, whereas it affected user satisfaction.
CONCLUSIONS: The results indicate that doctors still complain of discomfort in using the DUR system in the outpatient clinical setting, even though they admit that it contributes to patient safety. To mitigate doctors' resistance and raise user satisfaction, more opinions from doctors regarding the DUR system have to be considered and have to be reflected in the system.

I. Introduction

Information technology (IT) in healthcare has become increasingly prevalent since the late 20th century [1,2]. Healthcare information systems (HISs), a typical application of IT in healthcare, were adopted to support Medicare and Medicaid in the United States in the late 1960s. However, their role in improving patient safety was not recognized until a report by the Institute of Medicine, "To err is human: building a safer health system", was published in 1999. According to the report, at least 44,000 people died due to medical errors in hospitals every year. Since then, HIS has been regarded as a tool to improve patient safety [3]. A drug utilization review (DUR) system is a knowledge-based practice of the HIS which gives time-pressed doctors invaluable information when prescribing drugs, and it can prevent any harm to patients from medication errors [4].

Unfortunately, the healthcare sector has been slower in adoption of HISs in comparison with other industries, such as banking and automotive industries [5]. There are a variety of substantial barriers. One of the major obstacles is doctors' resistance [5,6,7,8,9,10]. To successfully implement the HIS in the real healthcare setting, there is a strong need to study how such doctors' resistance affects satisfaction and usefulness after its implementation.

There have been a few formal usability studies on experienced doctors as users. However, there have been few analysis cases on the impact of doctors' resistance with regard to success of the DUR system. In 1992, DeLone and McLean presented the DeLone and McLean (D&M) Information System (IS) Success Model which could be used to assess the success of an IT system based on multi-dimensional constructs rather than by single factor [11]. Thus, the present study aimed to analyze the impacts of doctors' resistance on the success of DUR systems based on an augmented D&M IS Success Model (2003), which used doctors' resistance as a socio-technological measure.

II. Methods

1. Study Site: Samsung Medical Center

Samsung Medical Center (SMC) is a Korean tertiary hospital with a capacity of 2000 beds. The Korean government has guided all hospitals to implement the concurrent DUR (cDUR) system by the end of 2011. To follow this government guideline, the SMC organized a task force (TF) in July 2011. The TF included doctors, pharmacists, nurses, administrators, and IT specialists. Its mission was to design, develop, and implement the cDUR system before 2012. The cDUR system has been tightly integrated with the SMC's computerized physician order entry (CPOE) system in the outpatient clinical setting. Through gathering opinions from several TF meetings, the cDUR system was developed in December 2011 and adopted for outpatients since January 1, 2012 without any trouble. According to the government guidelines, the checking of DUR conflict events is mandatory in hospitals in order for its benefits to be realized [12].

2. Study Setting: DUR System

In Korea, the concept of the DUR was introduced in the 1980s, but it had not been seriously discussed until early 2000. At the end of 2011, the Korean government issued a guideline for the cDUR system to be adopted in all hospitals. If the guideline is not followed, the National Health Insurance Review Agency (NHIRA) may retrospectively limit reimbursement of the hospital's claim. This system automatically checks whether a prescription meets the criteria developed by NHIRA at the time of prescribing. DUR conflict events are based on drug-drug interactions, such as duplicate medication, contraindicated drug interaction, etc., for outpatients. Implementation of the DUR system by Korea hospitals is the first case in the world in which all healthcare providers have used the system nationwide [13].

The first step of the cDUR system is that the doctor enters drug information into the CPOE system (①), as shown in Figure 1. Next, the entered data are exchanged with the national DUR server to check for conflict events (②). The checked results are returned to the doctor's computer display (③). When there are no conflict events, the prescription is stored into the national DUR database, and the prescription process is finished (④). In the case of DUR conflict events, the doctor has to enter the reason why the outpatient needs the drugs (⑤). When the doctor does not have any reason to proceed with the prescription, the prescription has to be canceled or changed (⑥).

Figure 1

Architecture of the concurrent drug utilization review (cDUR) system in Korea.

Real prescription data during one week from January 2 to 8, 2012 was analyzed immediately after adoption of the cDUR system. During the period, the number of outpatients was 40,908. Among them, 21,701 (53.0%) were prescribed drugs by doctors. The number of DUR conflict events was 1,752, which comprised 4.3% of outpatients. Duplicate medication was found in 99.3% of DUR conflict events, and contraindicated drug interaction accounted for 0.7% [11]. The time required for entering a reason for the DUR conflict was about 31.6 seconds in the outpatient clinical setting.

3. Research Model and Hypotheses

According to a literature review, there have been several studies on the D&M IS Success Model of the HIS [14,15,16,17,18]. Studies regarding the success of the DUR system were rare, and there was also a lack of analysis of the impact of doctors' resistance. As seen in the system architecture (Figure 1), the DUR system is very complex, and this leads to an increase in the time required for the prescription process in the clinical setting. This complicated work should be observed to be helpful to improve patient safety. However, doctors may consider it as a laborious task in their already time-pressing outpatient clinical setting, although they recognize its usefulness. Thus, the process may invoke doctors' resistance even if they use it under compulsion. Thus, this study focused on the success of the DUR system based on the augmented D&M IS Success Model, which used doctors' resistance as a socio-technological measure. This framework is the same as that of the D&M IS Success Model in that it is based on qualities, such as system, information, and services [19]. The major difference is that this study excluded the variable 'use' because it was not statistically significant for mandatory systems [20]. Another very specialized construct is newly added doctors' resistance in this study to examine whether this has an impact on user satisfaction or user usefulness in a statistically significant fashion or not. The net benefit was specified as 'user usefulness' to reflect doctors' viewpoint, since doctors are the main users of the DUR system. Pertaining to net benefit, benefits for patients or organizations may also be considered. Any improvement of patient service, satisfaction of patients, cost issues of organizations, etc., can be examples of net benefit. However, this study selected 'user usefulness' in that it is meaningful for the system to be evaluated in stakeholders' interests, as recommended by Seddon et al. [21]. Hence, this study tried to measure whether the DUR system could be useful in improving patient safety or quality for doctors to care patients or not. Review of previous studies led to the conclusion that DUR systems could be considered in relation to the following possible benefits: 'useful for making prescription', 'improving decision-making', and 'giving doctors positive impact'. Considering these aspects, the research model of this study is presented in Figure 2.

Figure 2

Research model and hypotheses. DUR: drug utilization review.

Hypothesis 1 (H1): The overall quality of the DUR system positively impacts user satisfaction but user satisfaction is negatively impacted by doctors' resistance.

H1-1: System quality positively impacts user satisfaction.

H1-2: Information quality positively impacts user satisfaction.

H1-3: Service quality positively impacts user satisfaction.

H1-4: Doctors' resistance negatively impacts user satisfaction.

Hypothesis 2 (H2): User satisfaction positively impacts user usefulness but user usefulness is negatively impacted by doctors' resistance.

H2-1: User satisfaction positively impacts user usefulness.

H2-2: Doctors' resistance negatively impacts user usefulness.

Hypothesis 3 (H3): Both user satisfaction and user usefulness negatively impact doctors' resistance.

H3-1: User satisfaction negatively impacts doctors' resistance.

H3-2: User usefulness negatively impacts doctors' resistance.

4. Variables and Measures

The questionnaire used in this study was composed of six variables based on the hypotheses (Appendix 1). In the system quality variable, the measured items were ease of use, response time, and system reliability. In the information quality variable, the measured items were understandability, accuracy, and timeliness. In the service quality variable, the measured items were sincerity, right time, and high service quality. The measured items in the user satisfaction variable were overall satisfaction, user friendliness, and positive attitude. In the case of doctors' resistance, the measured items included workload increase, time consumption and no tangible benefits. Usefulness, decision-making, and positive benefits were included in the variable measuring user usefulness.

The measures for system quality, information quality, user satisfaction, and user usefulness were mainly based on studies by van der Meijden et al. [22] in 2003, Jen and Chao [14] in 2008, and Petter and Fruhling [16] in 2011. The measures for doctors' resistance were based on challenges or barriers described by Lee et al. [8] in 1996, Sittig et al. [5] in 1999, Murff and Kannry [23] in 2001, Carroll et al. [9] in 2002, Bates and Gawande [7] in 2003, Berger and Kichak [6] in 2004, Aarts et al. [10] in 2007, and Ventura et al. [24] in 2011. A 5-point Likert-type scale was used for all items with 1 being 'strongly disagree', 3 being 'neutral', and 5 being 'strongly agree'.

5. Survey and Data Analysis

A survey of doctors who have used computers to enter prescriptions was conducted over a period of 3 weeks from February 22 to March 13, 2012. The questionnaire was based on the self-reported usage pattern. An informal informed consent was included at the head of the questionnaire. IBM SPSS Statistics ver. 20 (IBM, Armonk, NY, USA) was used for data analysis. Although the variables and measures were already validated in previous researches, validity was checked through factor analysis as a confirmatory method because some of them were newly combined in this study. To check the validity between variables and measures, factor analysis with IBM SPSS Statistics ver. 20 was performed. The analysis of results was based on convergent validity, discriminant validity, and nomological validity [25]. The extraction method of factor analysis was principle component analysis, and the rotation method was Equamax with Kaiser normalization. After analysis, measures with low construct validity, low convergent validity, or low discriminant validity were excluded. With Cronbach's α, the reliability of the measures was assessed. Multiple linear regression analysis was used to examine the hypotheses. In addition, in-depth interviews with doctors, who have rich experience in leading IT projects at the hospital, were conducted to discuss the results of data analysis.

The Institutional Review Board (IRB) review has been obligatory for all research related to human subjects since February 1, 2013 in Korea. According to the IRB office, this study is eligible for a waiver of IRB review because this study did not store any personal information which could be used to identify respondents.

III. Results

1. Validity and Reliability

Of 650 doctors who entered prescriptions in January 2012, 54 answered the questionnaires. Of the returned 54 questionnaires, 20 were invalidated because 7 were returned incomplete, and 13 were insincere. This yielded a total of 34 usable questionnaires for data analysis with a validation rate of 63%. The response rate was 5.3%, that is, 34 valid surveys out of 650 doctors. Respondents of valid questionnaires were 68% male; 71% were between the ages of 30 and 50; 50% were professors, 15% were fellows, and 35% were residents; 47% had working experience of more than 10 years, 12% had 5 to 10 years, and 41% had less than 5 years. The distribution of specialties was 32% in internal medicine, 35% in surgery, 9% in pediatrics, 6% in neurology, 6% in family medicine, and 12% in other areas. Even though the response rate was very low, this distribution was very similar to the distribution of doctors in the hospital. Therefore, this survey was able to cover a representative sample of the hospital.

According to the results of checking the validity between variables and measures, as shown in Table 1, response time and reliability are relevant to system quality; understandability, accuracy, and timeliness are relevant to information quality; sincerity, right time, and high service quality are relevant to service quality; overall satisfaction and user friendliness are relevant to user satisfaction; increase in workload and time consumption are relevant to doctors' resistance; usefulness, decision-making, and positive benefits are relevant to user usefulness.

Table 1

Results of factor analysis

The method for factor extraction is Principal Component Analysis, and the method for rotation is Equamax with Kaiser Normalization. Rotation converged in 14 iterations.

The digits in bold style mean that the measures are relevant to the intended variables based on the literature reviews. However, the measures in italic style, such as A1 ease to use, D3 positive attitude, and E3 no tangible benefits, are excluded because of their irrelevances.

However, ease of use was relevant to user satisfaction rather than system quality in this study, although it was the most frequently addressed measure of system quality. Thus, it was excluded from further analysis. Positive attitude was not relevant to user satisfaction, even though it was recommended as a measure of user satisfaction [22]. It was also excluded from this analysis. From the perspective of nomological validity, no tangible benefits were also excluded since it is strongly related to doctors' resistance rather than to user satisfaction according to the literature review [6].

According to a result of the reliability assessment with 6 variables and 15 measures except ease of use, positive attitude and no tangible benefits, Cronbach's α for the 5 variables indicated good reliability with reliability values higher than 0.7 representing the minimal standard [25]. System quality had a slightly weak reliability of 0.640. The mean, standard deviation, and Cronbach's α for the variables are listed in Table 2. The means of all variables were above 3.0. Pearson correlation coefficients indicated low and high existent correlations among the variables examined for the DUR system. Among the variables, doctors' resistance showed a strong correlation with user satisfaction, whereas it was less correlated with user usefulness.

Table 2

Means and correlation of variables

SD: standard deviation.

*p < 0.05, **p < 0.01.

2. Hypothesis Test

Typically, studies on IS success use structural equation modeling (SEM) for large sample sizes. However, due to the small sample size of the present study, multiple linear regression, rather than SEM, was adopted to examine the relationships among the hypotheses in this research. The results of the regression analysis are presented in Table 3.

Table 3

The results of regression analysis

*p < 0.05, **p < 0.01, ***p < 0.1.

System quality, information quality, service quality, and doctors' resistance (H1) did not behave as hypothesized. For user satisfaction, the regression model was significant (R2 = 0.529, p = 0.000). Of the 4 variables, only service quality was significant with a positive beta (β = 0.611, p = 0.001), whereas doctors' resistance was negatively significant (β = -0.279, p = 0.035). Thus, hypothesis H1 was partially supported, as shown in Table 3.

Similar to hypothesis H1, hypothesis H2 was partially supported, as shown in Table 3. With regard to user usefulness, the regression model was significant (R2 = 0.377, p = 0.001). User satisfaction had a significant positive association (β = 0.429, p = 0.002), while doctors' resistance had no significant association.

Hypothesis H3 was not supported, as shown in Table 3. For doctors' resistance, the regression model was not strongly significant (R2 = 0.222, p = 0.020) and neither user satisfaction nor user usefulness was associated with doctors' resistance, while user satisfaction was suggestively significant (β = -0.370, p = 0.097).

IV. Discussion

This study aimed to analyze the impact of doctors' resistance on the DUR system, which is a system that checks any conflict events of medications when doctors prescribe drugs in the outpatient clinical setting. An augmented D&M IS Success Model with doctors' resistance was employed to evaluate the DUR system. As expected, the D&M Success Model was a useful multi-dimensional tool to evaluate a specialized HIS, such as the DUR system, and these empirical results provide considerable support for the model. As mentioned before, this study is very meaningful in that it is the first study to explore the success factors of the DUR system associated with doctors' resistance.

The time required to enter a reason for the DUR conflict was about 31.6 seconds in the outpatient clinical setting. In Korea, doctors work in time-pressed environments such that only 3 to 5 minutes are allowed for one outpatient examination. Due to the DUR conflict check, an additional half minute was required to examine one outpatient. Thus, doctors were concerned that the DUR system increased their additional workload when prescribing drugs, and it disrupted their sense of professionalism. This attitude could explain why doctors have anxiety regarding the introduction of new IT system and why the healthcare industry is late to adopt new technology in comparison with other industries, as described in previous studies [5,8,9,10,23]. In this case, they express dissatisfaction with the DUR system, even though it is useful for patient safety. This is the reason why the doctors relentlessly opposed the adoption of the DUR system by Korea government.

The results show that: first, service quality is positively associated with user satisfaction; second, doctors' resistance is negatively associated with user satisfaction, whereas it is not associated with user usefulness; third, user satisfaction is positively associated with user usefulness; finally, neither user satisfaction nor user usefulness are statistically significant in association with doctors' resistance (Figure 3). In-depth interviews provide an effective insight into what things have happened and would happen [16]. We conducted in-depth interviews with several doctors to provide relevant explanations on results of hypothesis test in the research finding section. Doctors who participated in the interviews had extensive experience in leading IT projects at the hospital, especially computer-supported care services and theoretical insights from medical sociology. They included 3 doctors from internal medicine, 2 from surgery, and 1 from pediatrics.

Figure 3

The results of hypotheses test. DUR: drug utilization review.

Only service quality among the overall quality of the DUR system, as measured by system quality, information quality, and service quality, was significantly associated with user satisfaction in this study (H1-3). Considering that both system quality and information quality are strong predictors for IS success [16], the current study results are inconsistent with those of most previous studies. In the healthcare domain, users highly value systemic response time, reliability, precision, and timeliness in their work [14]. The in-depth interview results revealed that HIS has to basically ensure those qualities, of course. However, those interviewed expressed that what is more important is how to make rapid response to the problem in the time-critical outpatient setting. It appeared that respondents in this study focused more on aspects of service quality, such as service sincerity, right time service, and high service quality. They also said that they could not overlook the importance of system and information quality, although these did not have an influence on user satisfaction in this study.

We found that doctors' resistance is negatively associated with user satisfaction (H1-4). This may be explained by the fact that the DUR system increased doctors' workloads because they had to enter specific reasons for every DUR conflict event popup. In case of a DUR conflict event, it took doctors an additional 0.5 minutes to enter the reason. Considering that only 3 to 5 minutes are used to examine one outpatient in Korea, this additional time cannot be considered short. Doctors, who participated in the interview, also expressed annoyance with DUR conflict alarms and felt that entering reasons was tiresome. This opinion is very similar to descriptions from other previous studies [5,8,9,23]. According to previous studies, the HIS had increased consultation times due to causes, such as DUR conflict event popup. Doctors expressed that they disrupted their clinical workflow and decreased their professionalism. Sometimes, this led to a boycott of HIS adoption as a form of resistance. Thus, it is quite natural that doctors' resistance is negatively associated with user satisfaction (H1-4). Further research is needed to identify whether the resistance comes simply from inconvenience or other reasons, such as invasion of their professionalism.

One of the most commonly measured attributes in the D&M Model is user satisfaction [17]. We found that user satisfaction positively affected user usefulness (H2-1). The result is consistent with most previous HIS studies in that user satisfaction might be a strong predictor of individual impact, such as user usefulness [14,15,16]. An interesting finding in this study is that doctors' resistance is not associated with user usefulness (H2-2). With regard to the impact on the individual of the D&M IS Success Model, the stakeholders to be evaluated were doctors who use the DUR system and user usefulness which they felt was set as the impact in this particular context based on the research of Seddon et al. [21]. Usefulness was specified with the three measures of usefulness, decision-making, and positive benefits. User usefulness is an objective fact, whereas satisfaction is the user's subjective feeling. Among those interviewed, some stated that doctors did not deny the usefulness of the DUR system, even though they suffered from its inconvenience. They also expressed that doctors might tolerate such inconvenience since the DUR system could contribute to improving the quality of care. This may be explained by the fact that doctors, who have strong professional ethics, put patient safety as their top priority. All those interviewed mentioned that it is necessary to improve convenience from the view point of doctors to lower barriers to HIS adoption in the healthcare industry [9].

However, those interviewed had a general impression that doctors' resistance may influence user usefulness if more valid questionnaires would be secured. Furthermore, they stated that doctors' resistance could affect the variable 'use', which was excluded from this study, if the DUR system was voluntary rather than mandatory according to the government guidelines. Under such conditions, doctors' resistance could negatively influence user usefulness as impacts user satisfaction.

The results for H3 showed that neither user satisfaction nor user usefulness affected doctors' resistance. However, when we closely looked at the association between user satisfaction and doctors' resistance, it was suggestively significant because the p-value was 0.097, smaller than 0.1. Thus, it is difficult to say that user satisfaction was not at all associated with doctors' resistance.

Although this study suggests some interesting findings that may be helpful in understanding the success factors of the DUR system, there are limitations to consider. First, the number of respondents for the analysis was relatively small. However, there is no reason to believe that there is systematic response bias because the distribution of the respondents represents the population of users, although valid questionnaires were received from only 5.3% of the entire population of users. Furthermore, the normality of the data collected was also retained because all Shapiro-Wilk statistics of the 3 hypotheses were higher than 0.05 (H1 = 0.980, H2 = 0.980, H3 = 0.955). In addition, only the DUR system was evaluated in this study. Because this DUR system is a very specific clinical information system, more study cases are needed to gain more sufficient insight on the success of the HIS. Another limitation is that the survey undertaken in this study was conducted in only a single organization. Although doctors of different hospitals use the same DUR system, they may have different views according to their clinical context. To establish a common model of IS success in healthcare, various opinions should be considered from different organizations. Another limitation is that some measures are not relevant to constructs in substance. In case of 'ease of use', it is usually relevant to system quality [22]. In this study, doctors answered it for user satisfaction rather than system quality. 'Positive attitude' was not answered as a measure of user satisfaction and 'no tangible benefits' were not related to doctors' resistance. If they were answered as in a previous study [22], the results of multiple linear regression could be different from the results obtained in this study.

In conclusion, it is evident that the HIS contributes to improvement in the quality of patient care [2]. In particular, the DUR system is a very effective tool in preventing medical mistakes because it has a significant impact on doctors' prescribing habits. However, efforts to adopt such beneficial systems are impeded by the resistance of doctors based on concerns about the negative effects of HIS. In this study based on the D&M IS Success Model, doctors' resistance to using the DUR system was not statistically significantly associated with user usefulness, whereas it affected user satisfaction. This means that doctors still complain of discomfort when using the DUR system in the outpatient clinical setting, even though they admit that it contributes to patient safety. All those interviewed suggest that, to mitigate doctors' resistance and improve user satisfaction, it is important to reflect opinions from doctors with strong professional ethics.