Integrating Medication Alert Data into a Clinical Data Repository to Enable Retrospective Study of Drug Interaction Alerts in Clinical Practice.

Current clinical data repositories primarily extract data from multiple administrative and electronic medical record (EMR) data resources (e.g., hospital and physician billing records) containing specific patient-level data including demographics, medications, laboratory results, diagnoses, and procedure codes. It overlooks the importance of EMR system-level data (e.g., medication alerts that are routinely used by physicians, nurses, and pharmacists for decision support) for the surveillance of EMR decision support tools. These medication alerts are a significant source of information for providers, to minimize avoidable adverse drug events. This study describes the integration of medication alert data into an i2b2-based clinical data repository to support the investigation of clinical events occurring around patients with anticoagulation treatment that triggered drug-drug interaction alerts. The integration of medication alerts allows us to repurpose the clinical and translational research infrastructure to conduct retrospective effectiveness surveillance of clinical decision support tools.

Introduction

Clinical care data are increasingly captured electronically and effort in making these data available for scientific research has received widespread support[1-3]. The NIH has funded the Informatics for Integrating Biology and the Bedside (i2b2) National Center for Biomedical Computing to provide an open-source clinical data informatics framework to facilitate clinical data reuse and integration[4]. Since its first release in 2007, over 240 scholarly articles have been published using data derived from i2b2-based repositories[5].

At the University of Kansas Medical Center (KUMC), we established an i2b2 based Healthcare Enterprise Repository for Ontological Narration (HERON) in 2010 that integrated information “siloed” in disparate information systems with available data types including patient demographics, medication, laboratory results, and diagnoses data. While most clinical integrated data repositories are created and used to support clinical trial recruitment, observational analysis of healthcare utilization and cost, and post-marketing surveillance of medications, we believe that integrating data about the use of electronic medical record (EMR) systems and the decision support provided will enable repurposing of the clinical and translational research infrastructure for conducting effectiveness surveillance of clinical decision support tools analogous to post-marketing medication safety surveillance.

Several reviews have found that computerized clinical decision support tools can improve healthcare providers’ performance[6-8] and substantially reduce medication error rates[9]. However, few studies have observed any significant benefits on patient outcomes[8], perhaps due to small sample size or short length of time for any clinically important effects to be revealed. Decision support tools may include passive and active referential information as well as reminders, alerts, and guidelines, among others. Since medications are commonly used for therapy but can also cause harm such as adverse drug reactions, various medication-related decision support tools have been implemented to improve medication safety and efficiency, which include drug-allergy checking, dose guidance, and drug-drug interaction checking.

In this study, we propose to enrich our i2b2-based HERON repository with medication alert data to facilitate retrospective effectiveness studies of EMR decision support tools such as drug-drug interaction alerts. Thus, cohorts can be defined based on the patient providers’ alert exposure and analyzed to gain understanding of how alert response impacts patient outcomes as well as descriptive statistics on the makeup of the patients whose provider received the interventions (e.g., most common alerts and diagnoses).

The national Patient-Centered Clinical Research Network (PCORnet) work is heightening integration of data resources in a standard manner. KUMC has a long tradition of incorporating rich data into i2b2-based clinical repositories. We previously published our work on incorporating observations from nursing flowsheets[10], the University of Kansas Hospital Tumor Registry data dating back to the 1950s (https://informatics.kumc.edu/work/wiki/TumorRegistry), the University HealthSystem Consortium (UHC) data regarding inpatient hospitalizations[11], REDCap patient registries[12], cardiovascular test results (https://informatics.kumc.edu/work/blog/heron-marmaton-update), and microbiology laboratory results including the organisms and their antimicrobial resistance[13]. We believe that adding EMR usage data will be equally important over time, due to the increasing focus on healthcare outcomes. For this pilot study, we start with a case study on the most classic form of alerting in EMR, drug-drug interaction alerts, specifically focused on common alerts for combination of anticoagulant and other medications that heighten the risk of bleeding.

Methods

Enriching i2b2-based Data Repository with Medication Alerts

KUMC’s affiliated clinical organizations adopted the Epic Systems Corporation (“Epic”) as its EMR in 2007 with initial deployment within the hospital and has expanded over time to encompass the emergency department, outpatient cancer center treatment, and is currently being deployed across all ambulatory clinics. Data from the Epic EMR were obtained by exporting the Clarity database. The Clarity module transforms data from Epic’s operational database (Intersystems Cache®) into a relational form (Oracle® 10g Release 2) for reporting. Clarity stores patient and system configuration information in over 7,000 tables with over 60,000 columns. The extracted data is transformed into an i2b2 compatible star schema, de-identified, and loaded on a separate database server to be accessed by the i2b2 application. These Extract, Transform, and Load (ETL) processes are written in Structured Query Language (SQL) statements and the Python programming language. The resulting data used by investigators and analyzed in this study is deemed non-human subjects research by the KUMC institutional review board but data use requires approval by the HERON Data Request Oversight Committee composed of representatives from KUMC and participating clinical organizations.

As a result, 22,927,205 alert observations for 474,035 patients were loaded from Clarity into our i2b2-based HERON data repository. As shown below (Figure 1), we used hierarchical relationship to organize the alerts in the i2b2 framework with alert status (i.e., actions taken) as modifiers. Specifically, the drug-drug interaction alerts are organized in i2b2 as: Alerts → Med Interaction Checking → Drug-Drug → Alert descriptions in alphabetic order.

Figure 1:

Medication interaction alert data represented in i2b2.

Case Study: Effectiveness of Anticoagulant Drug-Drug Interaction Alerts

As a case study, we will focus on a common form of alerting, i.e., drug-drug interaction alerts involving combination of anticoagulant and other medications that heighten the risk of bleeding. This combination of drug- drug interaction, alert, laboratory test, and bleeding outcome provides a specific timeline in clinical practice that can be followed and evaluated rather than a population-based inference through number of alerts and number of outcomes. We will use the enriched i2b2 clinical data repository to obtain data and analyze the clinical events (e.g., INR lab ordering and bleeding events) occurring around hospitalized patients who had anticoagulant drug-drug interaction alerts fired during their stay.

Warfarin is a commonly prescribed oral anticoagulant with a narrow window for safe and effective use. It requires regular laboratory testing to ensure the warfarin dose is appropriate, and a number of foods and medications can have a significant impact on warfarin effectiveness. The addition of new medications to the regimen of a patient who is also using warfarin can require a proactive adjustment of the warfarin dose to maintain effectiveness, or the patient may need additional monitoring to limit the risk of potentially fatal bleeding events. For example, one interacting medication which may require a warfarin dose adjustment is allopurinol, used to treat gout. Titration of allopurinol may be required in order to relieve the symptoms of gout, but concurrent use of allopurinol with warfarin can increase the risk of bleeding.

A patient who is stable on warfarin may be admitted to the hospital for an acute gout attack which requires allopurinol. The attending physician orders allopurinol, and is presented with a drug-drug interaction alert informing her that allopurinol interacts with warfarin and increases the risk of bleeding. The physician has the option of continuing both medications, or discontinuing one. Because the patient is in the hospital, and the gout attack is acute, the physician may feel the risk of bleeding is relatively small and both medications may be continued. While the patient is in the hospital and medication therapy is changing, a physician presented with the drug-drug interaction alert may accept the risk of bleeding while the patient is taking both medications, but will attempt to mitigate that risk by ordering International Normalized Ratio (INR) tests to ensure the patient’s warfarin dose is in the appropriate range to minimize the risk of bleeding.

Drug-drug interaction alerts can produce a number of actions, either from the system or the providers. The system may be designed to filter alerts to reduce alert fatigue for physicians and nurses and in such cases, a filtered alert may only present to pharmacists, even though it was originally triggered by the physician’s actions. The pharmacist, or if the alert is presented to the physician, will be given a number of action options. They may only view the alert, but close it without choosing an action, override the alert indicating they have seen it but will not act upon the information, cancel the alert, remove one of the interacting medications, or hold a medication for a period of time.

Patient Cohort:

We constructed a cohort of 3,506 adult patients (18 < age < 64) who were admitted to KUMC hospital from Q4, 2008 to Q4, 2015 with a length of stay ≥ 2 days and have at least one anticoagulant drug-drug interaction alert fired during their stay. For the purpose of this pilot case to evaluate integrated data, patients over 65 years of age are excluded to reduce confounders as they are more likely to have comorbid conditions and polypharmacy which can increase the risk of a patient having bleeding event. Then for each hospitalization encounter, we extracted data on INR lab ordering for the study patients to assess if INR tests were conducted as a result of the drug-drug interaction alert and any diagnostic codes suggesting bleeding complication. Bleeding events in this study are identified in HERON through the use of ICD-9 and E-codes for bleeds that can be caused or exacerbated by medications (Table 1).

Table 1:

Bleeding related ICD-9 codes used in analysis

ICD-9	Description	ICD-9	Description
285.1	Acute posthemorrhagic anemia	535.61	Duodenitis, with hemorrhage
286.5	Hemorrhagic disorder due to intrinsic circulating anticoagulants, antibodies, or inhibitors	562.02	Diverticulosis of small intestine with hemorrhage
286.7	Acquired coagulation factor deficiency	562.03	Diverticulitis of small intestine with hemorrhage
286.9	Other and unspecified coagulation defects	562.12	Diverticulosis of colon with hemorrhage
430	Subarachnoid hemorrhage	562.13	Diverticulitis of colon with hemorrhage
431	Intracerebral hemorrhage	568.81	Hemoperitoneum (nontraumatic)
432	Other and unspecified intracranial hemorrhage	569.85	Angiodysplasia of intestine with hemorrhage
456.0	Esophageal varices with bleeding	578.9	Hemorrhage of gastrointestinal tract, unspecified
456.20	Esophageal varices in diseases classified elsewhere, with bleeding	578.0	Hematemesis
459.0	Hemorrhage, unspecified	596.7	Hemorrhage into bladder wall
530.21	Ulcer of esophagus with bleeding	626.2	Excessive or frequent menstruation
531.0	Acute gastric ulcer with hemorrhage	782.7	Spontaneous ecchymoses
531.2	Acute gastric ulcer with hemorrhage and perforation	784.7	Epistaxis
531.4	Chronic or unspecified gastric ulcer with hemorrhage	784.8	Hemorrhage from throat
531.6	Chronic or unspecified gastric ulcer with hemorrhage and perforation	785.5	Shock without mention of trauma
532	Duodenal ulcer	786.3	Hemoptysis
533.2	Acute peptic ulcer of unspecified site with hemorrhage and perforation	920	Contusion of face, scalp, and neck except eye(s)
533.4	Chronic or unspecified peptic ulcer of unspecified site with hemorrhage	921	Contusion of eye and adnexa
533.6	Chronic or unspecified peptic ulcer of unspecified site with hemorrhage and perforation	922	Contusion of trunk
534.0	Acute gastrojejunal ulcer with hemorrhage	923	Contusion of upper limb
534.2	Acute gastrojejunal ulcer with hemorrhage and perforation	924	Contusion of lower limb and of other and unspecified sites
534.4	Chronic or unspecified gastrojejunal ulcer with hemorrhage	964.2	Poisoning by anticoagulants
534.6	Chronic or unspecified gastrojejunal ulcer with hemorrhage and perforation	998.1	Hemorrhage or hematoma complicating a procedure
535.11	Atrophic gastritis, with hemorrhage	578.1	Blood in stool
535.21	Gastric mucosal hypertrophy, with hemorrhage	790.92	Abnormal coagulation profile
535.31	Alcoholic gastritis, with hemorrhage	E858.2	Accidental poisoning by agents primarily affecting blood constituents
535.41	Other specified gastritis, with hemorrhage	E934.2	Anticoagulants causing adverse effects in therapeutic use
535.51	Unspecified gastritis and gastroduodenitis, with hemorrhage

Results

Descriptive Statistics of the Population

Table 2 shows the basic demographics of the study patient cohort (3,506 patients with 6,183 hospital encounters). The study cohort contains more female patients than male patients who satisfied the inclusion criteria described above. Nearly half of the patients in the cohort were over 55 years old.

Table 2:

Patient demographics

Demographic		# of Patients = N (%)
Gender	Male	1506 (43.95%)
	Female	2000 (57.05%)
Race	White	2593 (73.96%)
	Black	585 (16.69%)
	Asian	14 (0.40%)
	American Indian	13 (0.37%)
	Pacific Islander	1 (0.02%)
	Other	300 (8.56%)

From the study population, there are 317 distinct admission diagnoses categorized using the All Patient Refined DRG classification system. Table 3 lists the top 10 admission diagnoses or APR DRGs of the study patients, which were tabulated at the encounter level.

Table 3:

Top 10 All Patient Refined DRG diagnoses assigned for the hospital encounters with anticoagulant drug- drug (DDI) interaction alerts triggered

ALL PATIENT REFINED DRG	UNIQUE ENCOUNTERS WITH ANTICOAGULANT DDI ALERTS FIRED
SEPTICEMIA & DISSEM INFECT	412
CARD ARRHYTHMIA & CONDUCTION	331
HEART FAILURE	244
CARDIAC VALVE PROC W/0 CATH	186
PULMONARY EMBOLISM	167
REHABILITATION	161
PERCUT CARDIOVASC W/O AMI	152
KNEE JOINT REPLACEMENT	151
CARD CATHET EXC ISCHEMIA	132
PERIPHERAL & OTH VASC DIS	122

Descriptive Statistics of the Anticoagulant Interaction Alerts

In this study, there are 37 different drug-drug interaction alerts involving warfarin or other anticoagulants. Table 4 summarizes the top 10 warfarin/anticoagulants related interaction alerts fired during the study cohort’s hospital stay. Multiple alerts can be triggered during a single stay.

Table 4:

Top 10 anticoagulant interaction alerts triggered for the study inpatients

ALERT DESCRIPTION	UNIQUE ALERTS
ANTICOAGULANTS / SALICYLATES	6293
ANTICOAGULANTS / CORTICOSTEROIDS	6091
ANTICOAGULANTS / SELECTED PENICILLINS	2706
ANTICOAGULANTS / THYROID	2559
ANTICOAGULANTS / NSAIDS	1613
ANTICOAGULANTS / CEPHALOSPORINS, INJECTABLE	1290
ANTICOAGULANTS / ALLOPURINOL	818
ANTICOAGULANTS / CEFAZOLIN; CEFOXITIN; CEFTRIAXONE	772
ANTICOAGULANTS / VITAMIN K	658
ANTICOAGULANTS / METRONIDAZOLE; TINIDAZOLE	627

Among the 3,506 study patients, there were 6,183 hospitalization encounters with length of stay ≥ 2 days and a total of 26,270 unique warfarin/anticoagulants interaction alerts generated. For the 26,270 interaction alerts, we analyzed the recorded actions taken for the alerts, namely override, remove, filter, cancel, view, or hold (Table 5). In addition, as shown in Table 5, we examined the number of patients who had INR test result immediately within 1 day of the alert trigger vs. no INR result, and the number of patients who had INR result within 1 day of the alert but had bleeding events within 30 days of their hospital discharge.

Table 5:

Statistics on clinical events following alerts according to different actions taken for the alert

	Overridden	Removed	Filtered	Canceled	Viewed	Held
Alerts fired during hospital stay	10933	190	21830	2282	1609	15
Alert → INR (≤ 1 day)	660	5	1533	157	120	0
Alert → NO INR (≤ 1 day)	10167	185	20110	2111	1485	15
Alert → INR (≤ 1 day) → Bleeding (≤ 30 day)	9	0	38	2	1	0

Furthermore, using the i2b2 query analysis tool, we were able to visualize patients’ clinical events on a timeline as shown in Figure 2 below. For example, the second patient (red box) in Figure 2 is a 56 year old white female, who was admitted to the hospital around September, 2014 for a major small & large bowel procedure. During her stay, the patient had three interaction alerts and one INR test result Various interaction alerts fired around the time of the alerts.

Figure 2:

Study patient timeline of clinical events. Note: the i2b2 query analysis tool shows all occurrences of clinical events included in the query on a timeline; thus, alerts fired in all encounters (not only the ones fired during hospitalization) are also shown here.

Discussion

This study was a pilot to enrich an i2b2-based clinical data repository with medication alert data to enable clinical effectiveness studies of EMR decision support tools such as drug-drug interaction alerts. Through the expansion, cohorts can be defined based on the patient providers’ alert exposure and analyzed to understand provider actions which occur in response to drug-drug interaction alerts.

Drug-drug interactions are common in the inpatient setting. Medication alerts are an option for preventing medication-related adverse events by raising prescriber awareness, but hospitals often struggle with the alerts being ignored or overridden [9, 14, 15]. Studies have found override rates for high-severity drug-drug interactions can be as high as 88%[16, 17]. Previous work on alerts and overrides were generally limited to order analysis or prescriber questionnaires/focus groups, which have limitations in both context and recall[15]. Integrating alert actions with other clinical information within i2b2-based clinical data repositories may provide additional insight into provider decision-making to facilitate observational study and design of more effective medication-related alerts.

Oral anticoagulants are a class of medications which is commonly associated with drug-drug interaction alerts due to the number of interacting medications and the potentially serious adverse events associated with the drug interactions[18]. In our study, we identified approximately four medication alerts for each visit related to anticoagulants. Anticoagulants commonly interact with antibiotics, which were further reflected in the single most common DRG code for septicemia and infection. Additionally, oral anticoagulants are typically used to manage cardiac conditions and blood clots, which were reflected in the most frequent DRG codes associated with the drug- drug interactions, including cardiac arrhythmias, heart failure, and pulmonary embolisms. Furthermore, these conditions are frequently associated with age. One study found the average age of warfarin users was 71[19] and this was demonstrated in the age distribution of the patients identified in our study, where nearly half the patients in the cohort (18-64 years old) were over 55.

Despite the potentially fatal adverse drug events which can occur when medications interact with anticoagulants, only minority of patients (660 out of 3,506 = 18.8%) who had a drug-drug interaction alert presented during their hospital stay had an INR test result within a day of the alert, which may be partially due to the delay in obtaining INR test results from the order time. Among our cohort of patients, most patients (3,454 out of 3,506 = 98.5%) had an INR test result during their stay, regardless of when a drug-drug alert appeared. This may suggest that INR tests are likely part of usual care and as a result, are not ordered in response to drug-drug interactions, but are ordered regularly in an attempt to proactively monitor for potential adverse drug events regardless of the concurrent medications being used. Drug-drug interaction alerts are provided to physicians and other healthcare professionals to aid clinical decision making, but in the case of oral anticoagulants, the necessary monitoring may be occurring routinely during the course of regular care, and as a result, presentation of drug-drug interaction alerts for these medications may not aid clinical decision making, and may contribute to alert fatigue without improving patient safety.

: There are several limitations to this study. First, many of the identified drug-drug interaction alerts are for classes of medications rather than individual agents. In the case of anticoagulants, this type of class-level interaction alert may be presented for medication combinations which do not have a true interaction and do not alter the patient’s risk of a bleeding event. Second, we only looked at adults who were under 65 years of age, and do not capture prescribing actions which may occur for older adult patients that could alter the findings. Patients 65 years of age and older have increased likelihood of being on multiple medications or have multiple conditions that would increase bleeding risk. Therefore, in future work, we plan to consider the older population independently from the younger population in analysis. Third, the data collected do not capture information about which medication was being prescribed to produce the alert. A patient may be starting warfarin at the end of a course of antibiotics, which clinically may not pose a risk to a patient because one medication is ending. Fourth, it is not clear how many alerts were filtered so they were only presented to pharmacists, or how many providers may have viewed a specific alert for a specific patient and what the overall action of that group effort was.

: With the medication alert data integrated into an i2b2-based clinical data repository, we will be able to explore many questions regarding the effectiveness of the EMR decision support tools. For instance, one question may be comparing filtered alerts vs. non-filtered ones, and for those filtered alerts, whether pharmacists provide any valuable insight. We can also analyze how effective the alerts are in reducing bleeding events or changing the INR ordering pattern. Last but not least, one can quantify the incidence of bleeding for the various drug interaction responses.

Conclusion

This study describes a pilot to integrate medication alert data and provider actions that occur in response into an i2b2-based clinical data repository to allow repurpose of the clinical and translational research infrastructure to conduct surveillance of EMR effectiveness. We started with the most classic form of alerting in EMR, drug-drug interaction alerts, specifically focused on common alerts for combination of warfarin or anticoagulant and other medications that heighten the risk of bleeding. We showed that the enriched i2b2 framework can facilitate the investigation of clinical events occurring around patients with anticoagulation treatment that triggered the interaction alerts. It has the potential to be a powerful tool in supporting various retrospective effectiveness studies of EMR and the embedded decision support tools.