National Rules for Drug-Drug Interactions: Are They Appropriate for Tertiary Hospitals?

The application of appropriate rules for drug-drug interactions (DDIs) could substantially reduce the number of adverse drug events. However, current implementations of such rules in tertiary hospitals are problematic as physicians are receiving too many alerts, causing high override rates and alert fatigue. We investigated the potential impact of Korean national DDI rules in a drug utilization review program in terms of their severity coverage and the clinical efficiency of how physicians respond to them. Using lists of high-priority DDIs developed with the support of the U.S. government, we evaluated 706 contraindicated DDI pairs released in May 2015. We evaluated clinical log data from one tertiary hospital and prescription data from two other tertiary hospitals. The measured parameters were national DDI rule coverage for high-priority DDIs, alert override rate, and number of prescription pairs. The coverage rates of national DDI rules were 80% and 3.0% at the class and drug levels, respectively. The analysis of the system log data showed an overall override rate of 79.6%. Only 0.3% of all of the alerts (n = 66) were high-priority DDI rules. These showed a lower override rate of 51.5%, which was much lower than for the overall DDI rules. We also found 342 and 80 unmatched high-priority DDI pairs which were absent in national rules in inpatient orders from the other two hospitals. The national DDI rules are not complete in terms of their coverage of severe DDIs. They also lack clinical efficiency in tertiary settings, suggesting improved systematic approaches are needed.

INTRODUCTION

A drug–drug interaction (DDI) alert is a type of medication-related clinical decision support function in a computerized provider order entry (CPOE) system. These alerts provide the promise of substantially reducing the number of adverse drug events (ADEs) by introducing automation at the time of ordering and by rapidly supplying usable prescribing algorithms (1). However, implementations of DDI alerts do not always provide the anticipated benefits. They can be very difficult to optimize, and alert fatigue can be a major problem if too many false-positive warnings are delivered (2).

Since December 2010, Korean hospitals have been required to provide DDI alerts to physicians using the prospective drug utilization review (DUR) system run by the Health Insurance Review Agency (HIRA) (345). This DUR system aims to provide quality assurance and ensure the provision of appropriate drug therapies, and it was designed to prevent potential ADEs arising from medication errors and inappropriate drug use (35). The national DDI rule set was initiated and included 162 DDI combinations and up to 706 contraindicated co-prescription pairs as of August 2015.

While DDI alerts can reduce the number of DDI-related ADEs, alert fatigue induced by large numbers of DDI alerts can also lead to physicians ignoring clinically significant DDI alerts (2678). A recent review of empirical analyses of CPOEs with the national DDI rules found that physicians in general hospitals overrode a substantial fraction of automated warnings, with one study finding that physicians continuing with prescription orders in 72.2% of DDI cases (9). One study of DDI alert logs revealed a high override rate of 72.8% at a tertiary hospital, even the authors just considered a new alert regarding a DDI during 18 months of observation period (10). Our previous examination of DDI alert logs revealed a high override rate of 83% at a tertiary hospital (8). According to a report which analyzed HIRA DUR data, the override rates for DDI alerts were 76.6% in 2012 and 78% in 2013 (11). Such high override rates have prompted serious concerns that physicians may be overriding or ignoring clinically important warnings, which potentially has major implications both for safety and physician liability. The alerts may be unnecessarily interrupting physicians, which could threaten the provision of a consistent work process and cause inefficient use of time and other resources (12).

In an attempt to address this problem, the present study investigated the national DDI rules using three approaches. The first approach involved comparing the Korean DDI rules with a list of the most dangerous DDIs. These lists, known as high-priority DDI rules, were identified by a sponsor of the Office of the National Coordinator for Health Information Technology in the USA (1314). It is a set of 15 high-severity, clinically significant DDIs were identified for which it was advised that warnings should be generated in all electronic health record systems. The second approach involved investigating the DDI alert log data of a tertiary hospital to identify the rates at which high-priority DDI rules are alerted and overridden in practice. We also compared the results with those obtained at three hospitals in Belgium, the UK, and the USA. The third approach involved examining medication orders from two other hospitals in order to determine whether unmatched high-priority DDI rules appear in prescriptions. Based on the obtained results, we describe several areas that need further investigation in order to achieve the successful application of DDI alerts.

MATERIALS AND METHODS

We compared the current Korean national DDI rules with the lists of high-priority DDI rules found in previous studies (13). To identify the differences, we use a comparison framework (Fig. 1). A retrospective observational design was used to analyze the alert system logs. Fig. 2 shows the study procedure, materials, measurements, and participating hospitals. We investigated data from three Korean hospitals: data from one hospital were used for analyzing alert log data, and the data from the other two were used for analyzing prescription data. The intercountry analyses compared the performance of national DDI rules in South Korea with those of hospitals in three other countries. In this study we included a part of results of another study involving three of the authors (I.C., JH.L. and D.W.B.) (8).

Fig. 1

Comparison framework of the Korean national drug–drug interaction (DDI) rules and the USA high-priority DDI rules.

Fig. 2

Study design diagram including the participating hospitals, data sources, and measurements (labeled from ① to ③). DDI = drug–drug interaction.

Korean national DDI rules and lists of high-priority DDIs in the USA

The national DDI rules were developed based on the product labeling information provided by the Korea Food and Drug Administration and maintained by the HIRA. The list of high-priority DDIs consists of 15 drug class interactions that meet stringent criteria indicating that those drugs should never be prescribed together. The list comprises 2,563 drug pairs, including 2,162 possible pairs of QT-prolonging agents (involving 47 drugs). These pairs were identified on a consensus basis by expert panelists, who included medication knowledge-base vendors, electronic medical record (EMR) vendors, in-house knowledge-base developers from academic medical centers, and both federal and private agencies involved in the regulation of medication use in the USA (13). The panel assigned severity levels to the interactions based on information in various medication knowledge bases, and assessed the consequences of the interactions, the availability of therapeutic alternatives, monitoring and management options, predisposing factors, and the probability of the interactions occurring based on the strength of evidence available in the literature (13).

Settings and data sources

The Korean sites comprised three tertiary teaching hospitals having computerized physician order entry (CPOE) systems and complete in-house-designed EMR systems since 2003 or 2005. The systems were used throughout the hospitals by all physicians working in both inpatient and outpatient settings. The hospitals instituted DDI checking as a decision-support component of the CPOE system with at least the following four different types of alerts from the national DUR program: drug allergy, duplication, pregnancy contraindication, and drug formulary. These alerts were implemented based on the drug list released by the HIRA.

The systems log in one of the hospitals was used to track the interactions between the DDI alerts and physicians. The log was designed to record a patient who had received medications, a drug combination order that triggered an alert, the physician who ordered the prescriptions, the acceptance status of the alert, and any reason(s) for overriding the alert, which were coded options or entered with free text. We collected the log data for four consecutive months, from September 1 to December 31, 2014. The data contained 18,360 and 3,499 alert records for inpatient and outpatient settings, respectively. The results of a more detailed analysis of the system log data by our team will be presented in a future report.

The other two hospitals were involved in the third approach of this study: assessing the frequencies of drug orders corresponding to unmatched high-priority DDI rules. We retrospectively collected medication orders related to 210 and 220 patients who were discharged in 2013. The patients were stratified equally into medical-surgical units and intensive-care units, which involved the review of 55,244 and 14,925 prescriptions, respectively. Because these two hospitals did not have DDI alert log data, they were not included in system log analysis. The hospital that had alert log data did not participated in prescription data analysis due to its internal policy. To get recent DDI rules, alert log data of 2014 were analyzed. So study periods of the former hospital and the other two hospitals were not the same.

Intercountry comparisons were performed based on the analysis results for the log data from the three hospitals in the UK, the USA, and Belgium (8). Table 1 summarizes the information for the sites. They were all tertiary teaching hospitals and had their own DDI rules with severity levels that were developed using in-house approaches. The UK and USA sites had two and three levels of severity: hard stop, interruptive, and non-interruptive.

Table 1

Characteristics of the hospitals involved in this study

Characteristics	Korea	Belgium	UK	USA
Period of time coverage	4 mon (Sep.–Dec. 2014)	1 yr (Jan.–Dec. 2011)	1 yr (Jan.–Dec. 2011)	3 yr (2009–2011)
Hospital type	Tertiary teaching hospital	Tertiary teaching hospital	Tertiary teaching hospital	Tertiary teaching hospital
Clinical setting	Inpatient and outpatient	Inpatient	Inpatient	Outpatient
DDI rule knowledge base	National rules	Commercial rules	Self-developed rules	Self-developed rules
Priority system for DDI rules	No levels (all rules are interruptive)	6 levels (trigger alerts only for level 1 and 2)	2 levels (hard stop, & interruptive)	3 levels (hard stop, interruptive, & non-interrupted)

DDI = drug–drug interaction.

Measurements

Based on our study design diagram (Fig. 2), the primary outcomes of this study were the national DDI rule coverage rate for high-priority DDI lists and the override rates as obtained from analyses of system logs. The secondary outcome was the number of medication prescriptions that were not included in the national DDI rules but which were corresponded to the high-priority DDI list.

Data analysis

Descriptive statistics were used to summarize the rate of rule coverage, the rate of alert overrides, and the number of medication prescriptions. We compared the rule coverage and alert override rates among hospitals according to the high-priority DDI rules. The comparison results are presented as counts with percentage values and probability values obtained in the χ2 test. We used SAS software (version 9.3, SAS Institute, Cary, NC, USA) for the analyses.

Ethics statement

This study was conducted with the approval of the institutional review board at each hospital. (Asan Medical Center, IRB: 2014-1101) Informed consent was not required because the review of retrospective DDI alert logs and prescriptions did not involve individually identifiable data of any sort.

RESULTS

National DUR rule coverage for high-priority DDI lists of the USA

The national DDI rules included 78 (3.0%) drug pairs of high-priority DDI rules (2,563). Drug class pairs constituted 12 (80%) of the 15 rule categories in the national rules (Table 2). The three categories of febuxostat—azathioprine/mercaptopurine, ramelteon—CYP1A2 inhibitors, and tranylcypromine—procarbazine were not found in the national DDI rules. For QT-prolonging agents and tricyclic antidepressants (TCAs) that suggested drug classes rather than explicit drugs, we considered only 47 drugs classified in the high-risk category among QT-prolonging agents and 8 TCAs. We found only 17 QT-prolonging agent pairs in the list of national DDIs (Appendix 1).

Table 2

Cross-matching the national DDI rules with the list of high-priority DDI rules

Rule No.*	List of high-priority DDI rules			National DDI rules
	Object drug/class	Precipitant drug/class	No. of final pairs	No. of matched rules	No. of unmatched national rules
3	Amphetamine and derivatives	MAO inhibitors	60	6	54
4	Atazanavir	PPI	5	4	1
5	Febuxostat	Azathioprine/mercaptopurine	2	0	2
8	SSRIs	MAO inhibitors	55	4	51
11	Irinotecan	Strong CYP3A4 inhibitors	24	1	23
16	Narcotic analgesics	MAO inhibitors	30	3	27
20	TCAs†	MAO inhibitors	40	5	35
21	QT-prolonging agents‡	QT-prolonging agents‡	2,162	17	2,145
22	Ramelteon	CYP1A2 inhibitors	4	0	4
23	Strong CYP3A4 inhibitors	Protease inhibitors	60	2	58
25	HMG Co-A reductase inhibitors	CYP3A4 inhibitors	38	12	26
27	CYP3A4 inhibitors	Ergot alkaloids and derivatives	60	22	38
28	Tizanidine	CYP1A2 inhibitors	7	1	6
30	Tranylcypromine	Procarbazine	1	0	1
31	Triptans	MAO inhibitors	15	1	14
	No. of class-pairs: 15 (100.0%)			No. of matched class-pairs 12 (80.0%)
	No. of drug-pairs: 2,563 (100.0%)			No. of matched drug-pairs 78 (3.0%)

DDI = drug-drug interaction, PPI = proton-pump inhibitors, SSRI = selective serotonin reuptake inhibitors, MAO = monoamine oxidase, TCAs = tricyclic antidepressants.

*Rule number in (13); †List of TCAs: Mayo Clinic, http://www.mayoclinic.org/diseases-conditions/depression/in-depth/antidepressants/art-20046983?pg=2; ‡QT-prolonging agents: https://www.crediblemeds.org (accessed May 15, 2015).

Override rates of matched DDI rules

The review of 21,859 alerts revealed that only 6 rules out of 78 matched rules which are both in national rules and high-priority rules triggered 66 alerts. This represented only 0.3% (66/21,859) of all of the alerts presented to physicians. The average overall override rate of the 66 alerts was 51.5% (ranging from 14.3% to 100%) (Table 3), and the overall DDI override rate was 79.6% in both the inpatient and outpatient settings. The pairs of QT-prolonging agents constituted the most frequently alerted drug pairs, while fluvoxamine-selegiline (amphetamine and derivatives–MAO [monoamine oxidase] inhibitors) and simvastatin-clarithromycin/itraconazole (HMG Co-A reductase inhibitors–protease inhibitors) were the pairs that were most frequently overridden by physicians.

Table 3

Numbers of alerts and override rates of matched high-priority DDI rules

Rule No.*	Object-drug-precipitant-drug pair	No. of alert	No. of overrides (override rate, %)
8	Fluvoxamine-selegiline	3	3 (100)
16	Tramadol-selegiline	7	1 (14.3)
20	Amitriptyline-selegiline	2	1 (50)
21	Amiodarone-dronedarone	9	6 (66.7)
	Amiodarone-sotalol	30	14 (46.7)
25	Simvastatin-clarithromycin/itraconazole	15	9 (60.0)
	No. of drug class-pairs: 5/12 (41.7%)	66	34 (51.1)
	No. of drug-pairs fired alerts: 6/78 (7.7%)

DDI = drug-drug interaction.

*Rule number in (13).

Intercountry comparison of coverage for high-priority DDI rules

In the intercountry comparison, the UK and USA hospitals included 10 and 11 class-pairs, respectively (Table 4). The rules at the Belgium hospital corresponded for 14 out of 15 class-pairs. Rule coverage at the drug level ranged from 3.0% (78/2,563) in Korea to 23.8% (611/2,563) in Belgium, and was 5.1% (131/2,563) at the UK hospital and 4.4% (113/2,563) at the USA hospital. The override rate of high-priority DDI rules was highest for the UK hospital, at 83.3%, followed by the Belgium hospital (74.9%), the USA hospital (56.6%), and the Korean hospital (51.5%).

Table 4

Comparison of the coverage of high-priority DDI rules, and override rates in hospitals in four countries

Measurement Hospital	Class-level coverage (no. of class-pairs = 15)	Rule coverage (no. of drug-pairs = 2,563)	Override rate (%)
Korea	12	78	51.1
Belgium	14	611	74.9
UK	10	131	83.3
US	11	113	56.6

DDI = drug-drug interaction.

Prescriptions unmatched high-priority DDI pairs

The analysis of medication prescriptions examined whether the unmatched high-priority DDI pairs which were absent in national rules were prescribed and how many there were. At one hospital there were 342 drug pairs among 55,244 prescriptions (0.6%) (Table 5), while another hospital had 80 drug pairs among 14,925 prescriptions (5.4%). Most of them were QT-prolonging agent pairs, and there were no pairs overlapping between the two hospitals. The class/drug pairs of ramelteon—CYP1A2 inhibitors and HMG Co-A reductase inhibitor—CYP3A4 inhibitors were observed 3 and 19 times, respectively, at 1 hospital.

Table 5

Prescriptions containing high-priority DDI pairs of unmatched rules

Rule No.*	High-priority DDI pair (object-precipitant class/drug)	No. of unmatched rules	No. of prescriptions
			Hospital A	Hospital B
21	QT-prolonging agents-QT-prolonging agents†	2,145	320	80
22	Ramelteon-CYP1A2 inhibitors	4	3	0
25	HMG Co-A reductase inhibitors-CYP3A4 inhibitors	23	19	0
	Total		342	80

DDI = drug-drug interaction.

*Rule number in (13); †https://www.crediblemeds.org (accessed May 15, 2015).

DISCUSSION

We found that the national rules coverage rate for high priority DDI rule was 3.0%, which was low given that there were 706 contraindicated rules. These matched high-priority DDI rules that triggered 0.3% of the alerts that the physicians had received, and these showed a lower override rate than the overall override rate. The proportion of high-priority DDI rules among the Korean national rules was lower than for those of the UK and USA. We also found noticeable numbers of unmatched high-priority DDI pairs in prescriptions in two inpatient settings, which may be associated with medication safety risks.

The national rules included 80% of the high-priority DDI rules at the class level, but omitted most of the possible high-priority DDI pairs. These findings imply that the national rules include sufficient drug classes for high-priority DDI rules but are deficient at the drug level, even though the total number of rules exceeded 700. We can think of two reasons for this situation. The first is that unmatched high-priority DDI pairs are not prescribed by physicians in Korea, perhaps due to their good education or training. However, our findings of noticeable numbers of unmatched high-priority DDI pairs in prescriptions for inpatients make this hypothesis unlikely. In addition, the small degree of overlap of the unmatched rules between the two hospitals suggests that investigating different hospitals could reveal different unmatched high-priority DDI pairs. Therefore, this might not due simply to the availability in a drug formulary at each site. The second possibility is that the expert committee of HIRA did not consider the unmatched high-priority DDI rules. If that is the case, the unmatched rules should be considered first for inclusion in the national rules. The level of risks of the moderate-priority DDI rules then needs to be considered. We doubt that DDI rules with a moderate level of risk are appropriate in all circumstances, such as “always contraindicated DDIs,” but such a list would be of limited usefulness because it could not adequately reflect the clinical complexity of many treatment decisions, especially in tertiary hospitals. The risk of DDIs often varies from patient to patient, potentially being influenced by factors such as age, sex, kidney function, liver function, genetic makeup, and diseases, as well as the dose and route of administration of the medication (151617).

Our previous investigation of the appropriateness of DDI alert overrides by physicians based on a review of patient EMRs revealed that 68.2% of the DDI alert overrides were considered appropriate (2). This relative high rate of appropriateness implies that simple checking of prescription drug lists could have limitations, because this would not consider the clinical context and patient-specific information. Adapting rules based on patient-specific information could improve the clinical sensitivity of DDI rules and alerts.

Considering the matched high-priority DDI rules, only 7 rules triggered 66 alerts, which represented only 0.3% (66/21,889) of all alerts, and physicians responded to many of these. The override rate of 51.5% is lower than the overall override rate of 79.6%, but none of these alerts should actually have been overridden. This result implies that physicians rarely prescribe drugs involved in high-priority DDIs together, and when they receive alerts related to high-priority DDI rules they are more willing to accept them. Therefore, including more high-priority DDI rules in the national rules might improve the override rate and relieve alert fatigue, as well as reduce interruptions to the work flow of physicians and time-wasting. In addition, organizations might want to consider “hard stops” so that for the very most severe of these, the drugs cannot be ordered together.

The national DDI rules were developed by a process in which the DDIs to be avoided were determined, and involved experts representing various professional organizations (1). However, it should be remembered that there is considerable debate about the scientific evidence for the risk that many DDIs pose to patients (13). DDI information can be obtained from a various sources, but the data are often inconsistent or unreliable. One study of 458 drug pairs found that proprietary databases rated 7.4% of interactions (34/458) as severe while clinicians assessed 6.6% (30/458) as severe; this looks promising, but unfortunately only 3 of the interactions were considered severe according to both the databases and clinicians (18). Another study found that a panel of experts initially agreed on the clinical significance of particular DDIs approximately 50% of the time (19). The comparison of high-priority DDI rules using log data obtained in different countries revealed that even though the coverage of high-priority DDI rules is lower in Korea than in the other investigated countries, the override rates in Korea was similar with that of USA, and were much lower than those in UK and Belgium. This finding implies that Korean physicians responded more likely to accept the alerts of high-priority rules.

We also noticed that the two categories of QT-prolonging agent pairs and the HMG Co-A reductase inhibitors–CYP3A4 inhibitors showed the greatest differences among countries. The Belgium hospital had the largest number of rules for those pairs, but also had a low alert frequency. In contrast, the USA hospital had smaller numbers of such pairs but the largest numbers of alerts with a very low override rate compared to Belgium. The UK hospital had QT-prolonging agents rule, but these inactivated in their system, and only five rules were present for HMG Co-A reductase inhibitors–CYP3A4 inhibitors, with no cases of alerts. The Korean national rules contained small numbers of those pairs with much smaller alert frequencies, but higher override rates than in the USA. We did not know whether or not the high override rate was due to the Korean national rules having smaller rule sets for these class-pairs. However, based on the frequency analysis of drug pairs that were not matched with high-priority DDI rules in prescriptions, it is likely that appropriate rules were missing.

Our examination of unmatched high-priority DDI pairs in prescriptions showed that significant numbers of drug combinations were prescribed. Most of the combinations were QT-prolonging agent pairs, and they constituted 0.5%–0.6% of the total prescriptions at two of the hospitals. For the categories of rifampin–ritonavir and HMG Co-A reductase inhibitors–protease inhibitors, only small numbers of prescriptions were observed in one hospital. Although these are small proportions, they are potentially associated with high DDI risks and their consequences could be serious or even fatal; they therefore need to be considered for incorporation in the national rules.

In summary, the Korean national rules are shortcoming in coverage of contraindications of high-priority rules, which could be responsible for problems of a high override rate and alert fatigue associated with physician liability. When we assessed the rate of overrides by physicians, about 15% of them (80/529) received more than 20 DDI alerts during the 4-month analysis period and overrode all of them; that is, an override rate of 100%. The reason we used the frequency of 20 DDI alerts during the period, was to exclude the miss classification of physicians with small number of alerts having relatively high override opportunity. The 100% override was both unexpected and undesirable, which demonstrates the need to consider several aspects such as the provision of too many unnecessary interruptions and an inadequate consideration of liability. Our findings also identified several possible ways for minimizing the alert fatigue associated with the national rules in tertiary hospitals, including strategies to increase their sensitivity and relevancy. These suggestions involve both government and local hospitals. First, it would actually be unreasonable to expect one set of rules to fit all hospitals due to the diversity of patients with vastly different types of illnesses and levels of severity. An approach based on the DDI priority level would be easier than one based on patient severity, because of a lack of measurements, tools, and interrater reliability. For example, high-priority DDI rules could be considered for all hospitals with a hard stop or interruptive, while moderate-priority DDI rules would be presented as being optional or informative in tertiary hospitals (4).

Second, DDI warnings should provide concise but comprehensive information that will enable physicians to make more knowledgeable and effective prescribing decisions (4). The current simple lists of DDI combinations that are typically used can be misleading, as they do not address clinical complexity or the highly individualized nature of many treatment choices. Such information could include alternative drugs, ways to adjust patient monitoring, and general background information such as evidence of DDIs, the clinical relevance of the possible adverse DDI consequences, specific risk factors, and the incident rate of adverse reactions.

Third, to improve the rule sensitivity, each hospital could utilize patient data stored in their own EMR systems for determining when DDI warnings will be issued. Both the risk of an adverse drug reaction due to a DDI and the quality of the evidence concerning that risk may depend on multiple patient-specific risk factors, some or even all of which may be recorded in the EMR of a patient. The reliability of DDI alerts will then depend on the accuracy of the data in EMRs, which may be compromised by input errors, omissions, and incomplete entries. One research reported that adding clinical laboratory values, concomitant medication, patient demographics, and administration time to basic DDI checking algorithm results in a 55% reduction of the number of alerts and a 45% reduction in time spent on DDI checking, yielding a return on investment of almost 10 years (20). Also, before being deployed, the rules should be piloted in several representative settings to determine whether they are effective in improving prescribing decisions and are not overly disruptive to physicians. Additional or revised rules could be incorporated in the prescribing process through a similar process. These processes could be undertaken by the HIRA.

This study was subject to several limitations. First, practice log and prescription data obtained from three tertiary teaching hospitals were used, and so the results might not be generalizable to other tertiary general hospitals. However, the national rules are mandatory and they are used nationwide in all primary, secondary, and tertiary hospitals. Moreover, our findings were consistent with those in other literature and relevant reports. We also compared our findings in the present study with findings that we obtained in other countries in previous studies. Second, we did not evaluate the appropriateness of the alerts that were overridden and could not follow up the patient outcomes in the overridden cases; further research is therefore needed in order to optimize the national DDI rules.

The current national DDI rules of HIRA are not complete in terms of their severity coverage. They also have considerable shortcomings in clinical efficiency in tertiary settings, which indicates the need for improved systematic approaches (21). To improve safety and reduce alert fatigue, the national rules should be improved by setting up a process of monitoring and refinement of the list, establishing tests and validation processes before releasing them, empowering physicians by providing them with concise and comprehensive information, and using a sophisticated design that is integrated with the patient data available in EMR systems.