Adverse Drug Event Ontology: Gap Analysis for Clinical Surveillance Application.

Adverse drug event identification and management are an important patient safety problem given the potential for event prevention. Previous efforts to provide structured data methods for population level identification of adverse drug events have been established, but important gaps in coverage remain. ADE identification gaps contribute to suboptimal and inefficient event identification. To address the ADE identification problem, a gap assessment was completed with the creation of a proposed comprehensive ontology using a Minimal Clinical Data Set framework incorporating existing identification approaches, clinical literature and a large set of inpatient clinical data. The new ontology was developed and tested using the National Inpatient Sample database with the validation results demonstrating expanded ADE identification capacity. In addition, the newly proposed ontology elements are noted to have significant inpatient mortality, above median inpatient costs and a longer length of stay when compared to existing ADE ontology elements and patients without ADE exposure.

Introduction

Adverse drug events, including injury and death, have been reported to affect up to 1.6 million patients annually, according to an Institute of Medicine report1 posing an important clinical problem. Adverse drug events are a major patient safety concern with the Centers for Medicare and Medicaid Services developing early efforts to consider mandating adverse event monitoring and subsequent development of the Sentinel Initiative to facilitate medication safety efforts2. Exact numbers are difficult to assess, due to lack of uniform reporting, but adverse drug events add 2–4 days to hospital length of stay at a cost of $2500–$5500 per patient3–5.

Although adverse drug events may occur frequently, the ability to identify actual adverse events has been difficult due to the limited availability of effective methods. Voluntary reporting of inpatient adverse drug events are the most well-known systems for event tracking, but are the poorest performing of the systems, identifying only around 1 in 20 events due to insufficient provider participation6. Given the difficulty in identifying clinical events due to the limited availability of well curated and standardized electronic medical record data and the lack of effective reporting by providers, automated surveillance methods are needed for population medication safety surveillance. Given that automated system detection rates can be three to twelve times higher than provider driven reporting there is substantial potential to fill this critical safety surveillance gap7.

Background

The inpatient clinical population has a higher rate of adverse drug events given the higher level of clinical acuity and underlying clinical comorbidities requiring hospitalization8. Adverse drug events are difficult to detect using chart reviews due to resource cost and data quality limitations resulting in challenges with data aggregation for population risk assessment of low prevalence adverse events. For clinical sites with advanced electronic medical record capacity and clinical data review resources, there may be enhanced capabilities to identify local clinical events; however, there may not be a sufficient number of drug exposures to effectively identify adverse drug event patterns. In order to maximize the likelihood of detecting adverse drug events, large clinical databases with capacity for efficient data aggregation can more effectively detect rare adverse drug events and accurately assess the risk of adverse events in populations. Efficient ADE identification can also facilitate medication safety initiatives, medication outcomes assessment and drug-drug interaction risk assessment. Pairing a comprehensive ADE ontology with a large clinical database such as the Healthcare Cost and Utilization Project (HCUP) National Inpatient Sample (NIS) can provide important insights on events rates, ADE trends and event predictors at the facility level given that the data includes a 20% US hospital sample stratified for region, location, teaching status, bed size and ownership http://www.hcup-us.ahrq.gov/nisoverview.jsp. Given the critical need for effective, low- cost and computational driven approaches to ADE surveillance, an up-to-date and comprehensive ADE ontology can help identify ADEs present in structured clinical data both to better understand ADE incidence, facilitate quality improvement initiatives, and manage ADE surveillance.

Methods

Ontology Development

A comprehensive ADE ontology was developed from available clinical literature, ADE public use files and a large clinical database using a Minimum Clinical Data Set (MCDS) Framework9. The available literature and ADE public use files were assessed for both ADE identification capabilities and ADE surveillance gaps with each source being iteratively incorporated into the final comprehensive candidate ontology.

Relevant ICD-9 CM diagnostic and E-codes in the current ICD9- CM clinical code set were identified using text searches for candidate identification followed by confirmation with expert review. Two comprehensive public use ADE code lists were analyzed including the 2011 Healthcare Cost and Utilization Project (HCUP-ADE) (http://www.hcup-us.ahrq.gov/reports/statbriefs/sb158_ADE_Appendix.pdf) and 2002 Utah/Missouri Patient Safety Project (UMPSP) (http://health.utah.gov/psi/icd9.htm) with supplementation from prior studies on structured data for ADE identification 8, 10, 11.

Ontology Validation

Using the MCDS approach, the proposed ontology was applied to relevant clinical data to assess viability and impact. Each proposed code was assessed for its application in the clinical data (code viability) as well as its association with clinical outcome (clinical impact). Candidate codes which were not present in the clinical dataset were removed from the candidate list. The proposed new codes were grouped into two parent categories: Administrative ADE and Other ADE medications. The mortality rates, average length of stay and average total charges were assessed for both parent categories with the results summarized in Table 2.

Table 2:

ADE Therapeutic Group Event Rates and Outcomes:

ADE Group	Events	Mortality		LOS		Total Charges
		Freq	Rate	Mean	Std Dev	Mean	Std Dev
Non-ADE patients	7383706	132872	1.8	4.4	6.5	33963	61404
Antibiotics	30075	620	2.07	8.5	10.7	63728	116910
Clostridium difficile infection	79633	6195	7.79	11.6	14.8	88423	166622
Other anti-infectives	9650	164	1.7	6.5	10.6	44597	97223
Steroids	64804	1678	2.6	7.1	8.2	58664	98423
Insulin and Hypoglycemics	9991	133	1.34	4.4	7.7	30911	49320
Other hormones	3984	33	0.83	4.3	6.3	31956	52339
Antineoplastic drugs	62025	2957	4.78	7.8	9.3	62186	103685
Anti-allergy and antiemetic drugs	4784	35	0.73	4.3	6.0	31063	59302
Other systemic agents	1122	20	1.79	4.7	7.5	35579	59829
Anticoagulants	49908	2582	5.19	6.8	7.2	55395	84846
Other agents affecting blood constituents	6755	299	4.43	6.6	7.9	65607	102534
Opiates/Narcotics	36149	698	1.94	5.5	6.9	46970	72129
NSAIDS	40643	655	1.62	4.5	5.7	38029	61957
Hydantoin	4407	65	1.48	6.0	8.3	41433	91714
Other anticonvulsants	10083	78	0.78	5.3	8.3	34470	67551
Anti-Parkinson drugs	1078	8	0.74	5.5	6.5	29971	37292
Sedatives or hypnotics	21044	347	1.65	5.8	7.3	50263	78422
CNS depressants and anesthetics	10832	139	1.29	4.5	7.0	44592	84622
Antidepressants	12990	91	0.7	4.0	6.2	25504	49983
Antipsychotics	9745	90	0.93	6.3	11.1	33473	67547
Benzodiazepine	25118	341	1.36	4.1	6.2	31150	59103
Other psychotropic drugs	9812	125	1.28	5.5	9.1	33206	53089
Central nervous system drugs	6706	133	1.99	4.0	6.5	34601	61335
Autonomic nervous system drugs	3876	74	1.91	5.4	7.3	39578	72908
Digoxin	6508	344	5.29	6.2	6.9	45973	71764
Anti-adrenergics	6695	62	0.93	4.1	5.2	33284	47013
Other cardiovascular drugs	32323	408	1.27	4.8	6.4	38444	68210
GI system drugs	2268	36	1.59	5.7	6.4	42827	64556
Saluretics	7464	57	0.77	4.2	5.0	32404	44814
Other diuretics	17810	429	2.42	6.0	7.8	46836	88445
Other mineral and uric acid metabolism	3106	96	3.1	7.5	11.6	60064	141985
Smooth muscle and respiratory drugs	6084	34	0.56	4.1	5.8	28574	43714
Skin, eye, mucous membrane drugs	1561	16	1.03	5.9	8.7	40979	70417
Vaccines	465	4	0.86	4.2	4.9	29106	37750
Other specific drugs	237	3	1.27	3.3	4.4	27092	44596
Nonspecific ADE causes	79556	2133	2.68	7.2	9.8	56057	98530
Administrative ADE (new)	2268	114	5.03	8.0	10.0	102852	170667
Other ADE medications (new)	148931	2055	1.38	5.8	8.5	43266	84427

Results

The HCUP-ADE code set containing 467 ADE codes in 36 therapeutic categories provided the best currently maintained public use file for initiating the comprehensive ontology development and was the gold standard for the study evaluation. The final comprehensive ADE ontology contained a total of 531 ADE codes after completing the iterative incorporation of source content. The final ontology also added two additional categories including “Administrative ADE” related to adverse events related to medication administration with the remaining additional codes being grouped in the “Other ADE medications” for validation purposes. The “Administrative ADE” group contained a total of 17 proposed ADE codes and the “Other ADE Medications” included a total of 47 proposed ADE codes for a total of 64 new codes proposed for inclusion in conjunction with the current HCUP-ADE framework containing 467 clinical codes.

Ontology Validation: Overall ADE Rates

The total number of hospitalization discharges reviewed from the 2011 HCUP National Inpatient Sample (NIS) database included 8.024 million hospitalization events from a stratified probability sample of hospitals, with sampling probabilities calculated to select 20% of the universe of U.S. community, non-rehabilitation hospitals contained in each sample stratum. Using the HCUP-ADE public use file alone for gold standard benchmarking, a total of 735,050 ADE events were identified. After incorporating the two new proposed ADE code sets into the expanded comprehensive ontology, a total of 905,001 events were identified providing a 23.1% increase in ADE identification. Since each hospitalization could have more than one ADE, the number of hospitalizations with one or more ADE were identified yielding a total of 639,884 unique ADE associated hospitalizations producing a 7.98% ADE hospitalization rate.

Subject Characteristics and Overall Outcomes

In table one, the characteristics of the patients with and without ADEs are summarized and the groups are significantly different for each measure including a 66.7% relative increase in inpatient mortality, a 2.4 day increase in length of stay and $18,827 increase in average cost among patients experiencing an ADE. The ADE>0 group was older with more chronic diseases and had a higher percentage of Medicare patients and a lower numbers of Medicaid and private pay patients (P < 0.01). The ADE group also had slightly higher income and higher proportion of Caucasians (P < 0.01).

ADE Therapeutic Group Rates and Association with Outcomes

The medication group event rates, mortality, length of stay (LOS) and total charges are noted in table 2 to provide an evaluation of the association of the type of ADE with outcomes. The results from the proposed expanded ontology are included in the “Administrative ADE” (new) and the “Other ADE medications” (new) categories with the remainder of the groups from the HCUP gold standard.

The proposed “Administrative ADE” and “Other ADE medications” groups appear to have clinical and economic importance. The proposed “Administrative ADE” group had the fourth highest mortality, third longest length of stay (LOS), and the highest average hospital cost. The proposed “Other ADE medications” had the 22nd highest mortality, the 16th longest LOS and 16th highest average cost among the 39 therapeutic areas providing data on the relative ADE impact and evidence for ontology inclusion.

Code Viability

Each of the 531 codes were assessed for their presence in the 2011 NIS database to provide evidence of content use and viability with the assumption that missing codes in a large clinical database may be antiquated medications or the ADE code may have been changed to a different code. Among the 64 proposed new ADE codes, 62 codes were identified in the database with at least one clinical instance providing a coverage rate of 96.9%. The new codes which were not identified included the “Failure of sterile precautions during infusion or transfusion” code which was in the “Administrative ADE medications” group and “Psychostimulant poisoning” which was in the “Other ADE medications” category. The gold standard HCUP data set was also evaluated with 425 of the 467 ADE codes being noted in the NIS database yielding a coverage rate of 91.0%. The bulk of the codes which were not found in NIS were in the vaccine group (15 codes), CNS depressants and anesthetics (9 codes) and sedative/hypnotic (8 codes), with the other potentially non-viable ADE codes in other categories. Combining the 467 gold standard HCUP-ADE codes with our proposed 64 code addition yielded a composite coverage of 91.7%.

Discussion

The proposed comprehensive ADE ontology provides higher levels of ADE identification than the HCUP-ADE dataset with a greater than 22% expansion in identified events. Given that the proposed ontology expansion groups had significant inpatient mortality, average cost and length of stay, the inclusion of these coded elements appear to be clinically and economically meaningful. The NIS data validation step provided evidence for the need for an expanded structured ADE ontology given the substantially higher ADE rate. Our identified event rates are substantially higher than comparable studies with HCUP-ADE which yielded an ADE rate of 5.3% for Medicare patients8 and 5.64% for Stausberg10 on an earlier but similarly broad NIS data set from 2006. Though our findings had a higher ADE rate than previous publications on older NIS data,8, 10, 11 this may be partially related to the secular trend of increased ADE events noted by Shamliyan8 where ADE rates have increased by 90% from 2000 to 2008 in Medicare specific patients. Further expansion of the ADE ontology may or may not affect the event trend, but the results warrant addition assessment with longitudinal data to better understand the event rate and trend.

The events in the new “Administrative ADE” group had particularly high clinical impact and substantial economic cost providing important evidence for ADE inclusion. The “Other ADE medications” group did not have as high of inpatient mortality as the “Administrative ADE” group but was near the median in terms of inpatient mortality among the established gold standard HCUP-ADE therapeutic groups and was above the median HCUP-ADE therapeutic group LOS and average cost supporting their inclusion in a comprehensive ontology.

The “Other ADE medications” list included a diverse set of ADE codes with a number of therapeutic areas which may not be best represented as a separate category. Two therapeutic areas which were in the new “Other ADE medications” group were psychiatric and pain medications. They may be incorporated into the existing HCUP-ADE Opiates/Narcotics, Antipsychotics, Antidepressants or Other psychotropic drug groups to better cluster the affected patients and events since their inclusion in the “Other ADE medications” list likely overlaps, at least partially, with those existing HCUP-ADE categories. Several of the other ADE codes in the “Other ADE medications” group did not appear to map to existing HCUP-ADE therapeutic groups such as those for drug dermatitis and drug events among obstetrics and newborns. Further exploration of potentially new HCUP-ADE therapeutic categories may be warranted for the obstetrics and newborn category while many of the other ADE may be managed on an interim basis in the existing HCUP “Other specific drugs” and “Nonspecific ADE causes” ADE categories. The proposed “Administrative ADE” category would likely be a viable new addition to the HCUP-ADE since it is clinically and economically important and reflects clinical events which are by and large preventable using appropriate safety measures and are without a close fitting category in the current HCUP-ADE data set.

With the existing HCUP-ADE data set, a substantial number of ADE codes were not found in NIS validation including codes related to older and rarely used medications such as arsenic anti-infectives and mercurial antidiuretics which have been replaced in clinical use with less toxic alternative medications. Also missing from NIS were a number of anesthesia related events and adverse events associated with vaccinations which is in part due to low levels of immunizations for diseases such as smallpox and yellow fever which are not on the usual care pathway for childhood and adult vaccinations resulting in relatively low numbers of exposed patients and associated ADE. Further exploration of other years of NIS data and other clinical data sources may be needed to assess if thesemissing codes are used in older data, other clinical settings or geographic locations where exposure rates are higher due to differences in disease prevalence and treatment care standards.

The study results have a number of important limitations. The NIS data does not distinguish drug associated harms which occur in the outpatient setting leading to hospitalization from those which occur in the inpatient setting. In addition, the NIS data has a number of other deficiencies, most notably, the lack of a full medication profile which limits the ability to associate ADE to exposure medications. The NIS data sampling is based on a hospital sample which may not accurately represent all inpatient admissions, however, in 2012; the NIS dataset methodology was changed to better reflect event rates at the patient admission level rather than facility level which may better address the true event rates.

In future work, the incorporation of specific medication data at the individual level using electronic medical record, Medicare, Medicaid or commercial claims data would provide additional insight on ADE risk factors and their association with clinical outcomes. Such data would be important to increase the granular understanding of the problem as well as to explore ADE risk prevention strategies. The comprehensive ADE ontology may also be used for retrospective ADE identification for targeted risk assessment and mitigation since it uses administrative data which is operational in most US clinical settings with the capacity to quickly identify problem areas and to benchmark against national average data noted in Table 2. This rapid potential to translate into retrospective surveillance could provide an important tool to use alongside data mining, natural language processing and epidemiologic methods to operationalize drug safety initiatives and research.

Table 1:

ADE Patients and Non-ADE Patients

	ADE=0	ADE>0	p-value
Age (sd)	48.7±28	59.4±21	p<0.0001
Length of Stay(sd)	4.4±6.5	6.8±9.4	p<0.0001
Total Charges (sd)	33963±61404	52790±99245	p<0.0001
# of Chronic Diseases	4±3.5	6±3.3	p<0.0001
Female (%)	58.2	56.4	p<0.0001
Mortality (%)	1.8	3.0	p<0.0001