Interoperability of Medication Classification Systems: Lessons Learned Mapping Established Pharmacologic Classes (EPCs) to SNOMED CT.

Interoperability among medication classification systems is known to be limited. We investigated the mapping of the Established Pharmacologic Classes (EPCs) to SNOMED CT. We compared lexical and instance-based methods to an expert-reviewed reference standard to evaluate contributions of these methods. Of the 543 EPCs, 284 had an equivalent SNOMED CT class, 205 were more specific, and 54 could not be mapped. Precision, recall, and F1 score were 0.416, 0.620, and 0.498 for lexical mapping and 0.616, 0.504, and 0.554 for instance-based mapping. Each automatic method has strengths, weaknesses, and unique contributions in mapping between medication classification systems. In our experience, it was beneficial to consider the mapping provided by both automated methods for identifying potential matches, gaps, inconsistencies, and opportunities for quality improvement between classifications. However, manual review by subject matter experts is still needed to select the most relevant mappings.

Introduction

Medication terminologies and ontologies commonly categorize medications by similar properties such as therapeutic intent (i.e., muscle relaxants or analgesics), chemical structure (i.e., sulfonylureas or tetracyclines), mechanism of action (i.e., proton pump inhibitors or beta-adrenergic blockers), or sometimes combinations of the above (i.e., tricyclic antidepressants or amphetamine anorectics). Sets of drugs that share the same property used as a classification criterion (e.g., therapeutic intent) are generally referred to as medication classes (or simply classes in the context of this work). Interoperability among medication classification systems is known to be limited [4], yet it is important for clinical decision support (CDS), allergy checking, translational research, and organizing medication lists [3]. Our objective is to investigate the mapping of the U.S. Food and Drug Administration (FDA) established pharmacologic class (EPC) concepts to the SNOMED CT Substance hierarchy. More specifically, we provide an evaluation of ontology matching techniques and we describe lessons learned mapping medication classifications.

Established pharmacologic classes (EPCs)

In January 2006, the FDA established requirements for prescribing information for pharmaceutical products [2]. The labeling revisions provided additional information and established the structured product label (SPL) format for prescription medication labeling in order to make it easier for health are professionals to access, read, and use prescription medication information. Part of the labeling revisions requires that the statement “(Drug) is an (EPC) indicated for (indication(s))” appear under the Indications and Usage section [1]. The EPC membership is determined by the FDA to classify the medications into medication classes to which the active ingredient belongs; for example, albuterol is a beta2-adrenergic agonist. However, unlike many other medication classes, the EPCs are not organized into a hierarchy, despite the presence of logical groupings in the EPCs. The absence of an EPC hierarchy makes the use of EPCs difficult for accessing and using prescription medication information, and may limit the use of SPLs in clinical decision support [12]. In contrast, SNOMED CT does have a robust medication class hierarchy and could be used to help organize the EPCs.

Medication class representation in SNOMED CT

SNOMED CT is maintained and distributed by SNOMED International (London, UK). SNOMED CT includes clinical terms used in healthcare, among which are two medication hierarchies, namely the Pharmaceutical/biologic Product and Substance hierarchies. However, for this study, we focused on the Substance hierarchy. Medications in SNOMED CT can belong to multiple medication classes, as can the medication classes themselves. For example, Figure 1 shows the medication classifications in the Substance hierarchy for albuterol are listed as a Respiratory sympathomimetic agent, Selective beta-2 adrenoceptor stimulant, and Ethanolamine.

Figure 1

Matching medication classes between EPCs and SNOMED CT, albuterol example. Note: Asserted membership shown with solid arrows and inferred membership shown with dashed arrows. See text for detailed explanation.

Ontology matching

Medication classes are present in most medication ontologies. Comparing and matching ontologies is not a new concept, and can be performed through various techniques [6]. Due to the time and labor-intensive nature of manual matching between ontologies, automatic techniques, such as lexical and instance-based matching have been developed [8; 9; 11]. Lexical matching compares medication classes based on their names (such as Proton Pump Inhibitor [EPC] matching with Proton pump inhibitor [Substance]) and is probably the most common technique for assisting manual matching. However, some authors have suggested that lexical matching may not be appropriate for comparing medication classes [7; 11]. On the other hand, instance-based matching compares the overlap of medication class members (instances) from one medication class to those of another, which may better represent the intended meaning of the medication class than using the class name. Winnenburg et al. used lexical and instance-based matching techniques as tools for medication class ontology matching [11], and Mortensen et al. used instance-based matching to compare medication classes between ontologies as a method of quality assurance to identify medication classes in need of review and/or updating [8]. However, these prior studies did not evaluate these ontology matching techniques against an expert-reviewed reference standard or provide guidance on which threshold for medication class match significance for instance-based matching. The specific contribution of this study is to evaluate automated mapping techniques against a reference standard with application to the mapping of EPCs to SNOMED CT medication classes. Moreover, we conducted a sensitivity study to determine optimal thresholds for the instance-based techniques suggested by Winnenburg et al.

Methods

Data sources

We used the DailyMed index file (February 2014) to develop the list of EPCs with their corresponding medication unique identifiers (UNIIs), and SNOMED CT (March 2014 release) for the Substance hierarchy. We used RxNorm (March 2014 release) to map medications between the EPCs and SNOMED CT using the RxNorm concept unique identifier (RxCUI) for the medication active ingredient (IN). RxNorm represents medications as ingredients (INs), precise ingredients (PINs), and multiple ingredients (MINs). We excluded MINs from the analysis because they may be represented as individual ingredients in the hierarchies and are inconsistently represented in medication classification systems.[10] For example, Combivent® (MIN RxCUI:214199) does not exist in any medication classes, but the ingredients ipratropium (an anticholinergic agent) and albuterol (a beta2-adrenergic agonist) do belong to medication classes. The PINs typically represent salt forms and esters of INs, so we normalized the medications to IN RxCUIs before analysis. For example, albuterol maps to an IN (RxCUI:435) whereas albuterol sulfate maps to a PIN (RxCUI:142153), yet both would be considered the same active ingredient, albuterol, so we normalized them to the IN (RxCUI:435).

Identifying medications and medication classes

As shown in Figure 1, we first obtained a list of EPCs with their respective medications from DailyMed. In SNOMED CT, medications are mixed in with the medication classes at varying levels in the hierarchy, which can make separating the medications from the medication classes challenging. Therefore, for each medication (such as albuterol), we used RxNorm to identify the IN RxCUI (435 for albuterol), and then used RxNorm to map albuterol to the corresponding SNOMED CT concept identifier (372897005 for Substance), we then walked up the SNOMED CT hierarchies to identify medication classes with albuterol listed as a medication.

For example, albuterol has asserted membership to the medication classes Respiratory sympathomimetic agent, Selective beta-2 adrenoceptor stimulant, and Ethanolamine in the Substance hierarchy. We would then infer albuterol membership in ancestor medication classes through transitive closure. For example, since Selective beta-2 adrenoceptor stimulant is a subclass of Beta-adrenoceptor agonist, we would, therefore, infer that albuterol was also a member of Beta-adrenoceptor agonist, which would continue up the hierarchy to a Sympathomimetic and Autonomic agent. Figure 1 shows the asserted relationships with solid arrows and inferred relationships using dashed arrows. To simplify the analysis, we excluded very broad, top-level classes, such as Drug allergen (Substance), Chemical (Substance), and Drug or medicament (Substance), which would not provide meaningful alignment.

Mapping medication classes from EPC to SNOMED CT

Lexical matching techniques

Lexical matching compares medication classes based on their name. Some lexical matching techniques, such as those used in this project, include exact, normalized, and approximate text matching applied to main terms and synonyms. For example, beta2-adrenergic agonist [EPC] would match (partially) with Selective beta-2 adrenoceptor stimulant though synonymy and normalization despite differences in hyphenation (ie, beta2 vs beta-2) or use of adrenergic agonist vs adrenoceptor stimulant. For lexical matching, we utilized the matching and searching algorithms in Termworks (Apelon Inc., 2013, Hartford, CT). Termworks would normalize the medication class names by stemming (ie. matching based on the root of the word, such as “stimul” from “stimulant, stimulants, stimulating, stimulator”) and then provide the top lexical match based on an internal scoring system, giving priority to exact, normalized, then approximate matches, in decreasing order. Termworks used synonyms from the target terminology, SNOMED CT.

Instance-based matching techniques

Instance-based matching techniques compare classes based on the class members they share, in our case medications. Here, we adopted the framework by Winnenburg et al [11]. In summary, EPC concepts were compared to each class in SNOMED CT in a pairwise manner if the classes shared at least one active ingredient. We calculated medication class similarity using the equivalence score (ES) between two classes as the modified Jaccard coefficient to account for small sample size in medication classes [11]. The ES gives a score from 0 to almost 1, with 0 meaning no overlap and higher scores representing greater similarity (overlap) of medication classes. For example, beta2-adrenergic agonist [EPC] has 7 ingredients, of which, all 7 overlap with the 7 ingredients in Selective beta-2 adrenoceptor stimulant (substance), giving an ES of 0.941. The Winnenburg framework also supports the identification of inclusion relations between classes through an inclusion score. However, in the present investigation, we ignored the inclusion score and focused on equivalence relations between classes. Analysis was completed using STATA 13 (StataCorp. 2013. College Station, TX).

Developing a reference standard

The reference standard was developed to contain mappings from the EPCs to SNOMED CT Substance hierarchy. To summarize, two pharmacist informaticists (SN and JP) served as subject-matter experts (SMEs). First, SMEs elicited the meaning of the EPC name and mapped it to an equivalent class in SNOMED CT; then, if there was no equivalent class, the EPC was mapped to a related class using child_of relationships. If both attempts failed, we concluded that no mapping could be established. During the mapping exercise, the SMEs independently reviewed and rated the top lexical EPC to SNOMED CT pairs from Termworks, and all instance-based EPC to SNOMED CT pairs with a ES ≥ 0.2. An ES ≥ 0.2 was qualitatively chosen to provide a limited number of pairs for manual review that had a reasonable overlap. Priority was given to equivalent relationships, if no equivalent relationship was found, then the child_of relationships were used. If the pair was not equivalent, SMEs sought the most proximal SNOMED CT concept providing a more general, yet true representation of the EPC concept using child_of relationships. The SME ratings agreed 90.1% of the time with an average weighted kappa of 0.736, and discrepancies were resolved by discussion and consensus. Next, the ratings based on the automated methods were combined and the SMEs used clinical knowledge to review, investigate, and correct the mapping results, discrepancies, and unmatched EPCs. The reference standard underwent various quality assurance and verification reviews, including a full review of all mappings. The final mapping set was reviewed, and discrepancies reached consensus by an expanded group of physicians and FDA pharmacists.

Evaluating matching techniques against the reference standard

We used descriptive statistics to assess the contributions of each automated matching technique, along with their precision, recall, and F1 score (harmonic mean of precision and recall) compared to the reference standard. We also evaluated if the final equivalent pairs were predicted using lexical, instance-based matching, or both. To compare the lexical and instance-based matching, we limited the analysis to the top lexical match and top instance-based match using only equivalent pairs with the highest ES for each EPC. We then conducted sensitivity analysis across a range of ES thresholds for instance-based matching to determine which threshold provided the best F1 score for finding equivalent pairs.

Of note, in order to maximize the number of suggested matches (recall), we did not use the inclusion score to filter out those matches with a high ES that could also correspond to child_of relationships.

Since instance-based matching is dependent on shared instances, we removed EPCs that did not have medications matched to SNOMED CT. After statistical analysis, we reviewed valid, erroneous, and missing mappings, between lexical and instance-based matching techniques, compared to the reference standard.

Results

We identified 543 EPCs at the time of this study. There were 66 EPCs that were empty (i.e., no medications listed, such as Acetaminophen [EPC], Adrenergic Decongestant [EPC], and alpha-Adrenergic and beta-Adrenergic Blocker [EPC]), or did not have medications that could be mapped to SNOMED CT using RxNorm (such as radioactive therapeutic agents like Iodine ion I-131 and various allergenic extracts). We identified 2,963 Substance classes that shared at least one medication with the EPCs.

Manually established reference standard

About half (284) of the 543 EPCs had an equivalent class in SNOMED CT, while most of the remaining (205) had a child_of relationship. Fifty-four EPCs could not be mapped to SNOMED CT, such as sodium-glucose cotransporter 2 inhibitor [EPC] (new therapeutic target not in SNOMED CT at the time of analysis), calculi dissolution agent [EPC] (mix of various medications with a common therapeutic intent, but not classified by therapeutic intent in SNOMED CT), and potassium channel opener [EPC] (somewhat vague mechanism of action for a specific medication).

Comparison to the reference standard

Using only the top match for each EPC, lexical matching identified 526 potential matches, whereas instance-based matching identified 282 potential matches.

Optimal threshold for the equivalence score (ES)

Sensitivity analysis across a range of ES thresholds showed that an ES ≥ 0.3 was the optimal threshold, maximizing precision and recall. Table 1 shows the performance of instance-based matching for identifying equivalent pairs with an ES ≥ 0.3.

Table 1

Performance of automatic methods from EPC to SNOMED CT for equivalent pairs (ES≥0.3)

Method	Equivalent pairs	Precision	Recall	F1 score
Reference	284	-	-	-
Lexical	176	0.416	0.620	0.498
Instance-based	143	0.616	0.504	0.554

Contributions of each technique for equivalent mappings

As shown in Table 1, both lexical and instance-based matching had low-performance overall (F1 score was 0.498 and 0.554, respectively). Lexical matching had better recall, but lower precision than instance-based matching, which is not surprising, because the lexical matching produced more potential matches than the instance-based matching did. Out of all 284 equivalent EPC to SNOMED CT pairs, there were 98 identified correctly only by lexical matching, 65 only by instance-based matching (ES≥0.3) identified by both, and 43 identified by manual review only.

Examples and failure analysis

There were cases where instance-based matching was able to identify a correct match, but lexical matching was not, and in some cases, lexical matching recommended a contradictory class as equivalent in attempts to maximize recall with approximate matches. For example, lexical matching suggested Androgen Receptor Inhibitor [EPC] be mapped to Androgen receptor (substance); yet, instance-based matching correctly identified Synthetic antiandrogen (substance).

Lexical mappings typically failed to identify matches due to the use of synonyms and ambiguous meaning of class name or differences in classification type, such as between structural and functional groupings or mixing structural and functional classifications. The lexical matching tended to suggest mappings to terms other than medication classes, (i.e., receptors, antigens, lab test ingredients), such as Acetylcholine Release Inhibitor [EPC] and Acetylcholine (substance). In contrast, instance-based matched did not suggest pairs other than medication classes. Another limitation of lexical matching was that many of the suggested pairs had similar names but different meanings, which caused look-alike/sound-alike type errors for SMEs reviewing hundreds of pairs, such as Mood Stabilizer [EPC] and Mast cell stabilizer (substance).

Additionally, lexical matches were typically correct or incorrect, with very few “close” matches, whereas instance-based matching presented many close matches that pointed the mapper to the correct part of the SNOMED CT tree.

Instance-based matching is intrinsically dependent on the classes sharing instances. Potential errors in the instance-based matching were mainly due to small numbers in class size. For example, RANK Ligand Inhibitor [EPC] with the medication denosumab was mapped by the SMEs as a child_of Bone resorption inhibitor (substance); however, Bone resorption inhibitor does not include denosumab, instead, denosumab is a child of Monoclonal antibody agent (substance).

Instance-based matching also had problems with the multiple levels of granularity of SNOMED CT. For example, multiple medication classes were considered equivalent due to a high ES, such as xanthine oxidase inhibitor [EPC] matching with Anti-gout agent (Substance) with an ES of 0.516, and Xanthine oxidase inhibitor (Substance) with an ES of 0.775. Both substance classes were potential matches, but Anti-gout agent (substance) is the parent of Xanthine oxidase inhibitor (Substance). This also occurred with benzodiazepine [EPC] matching with 6 medication classes from SNOMED CT. To help sort this out in our automated analysis, we only used the highest ES scoring pair for each EPC.

Additionally, there were some classes that had high ES ratings but were considered to be child_of relationships due to how the medication class was named, such as the use of qualifiers like “analog” or “recombinant”. For example, Folate Analog [EPC] was considered a child_of Folic acid (substance). Finally, since instance-based matching is dependent on the medication classes having common medications, it was unable to find matches for EPCs without active ingredients listed in them, such as Adrenergic Decongestant [EPC], or classes where the instances could not be mapped to the SNOMED CT.

Discussion

Lessons learned

Overall, comparing and mapping medication classification systems is a very challenging task. Each classification system has its own way of grouping medications, each using different grouping criteria. Thus it is not possible to simply insert one classification system into another since each classification system has its own “world view”, use case, and area of interest, which has resulted in the proliferation of many terminology standards. However, the automatic matching methods helped facilitate the mapping process, for both equivalence and partial mappings. The lexical and instance-based matching methods were each able to correctly identify about half of the 284 manually created equivalent medication pairs (176 and 143, respectively). However, used in combination, the two approaches were able to find 221 of the 284 pairs, and identify most of the child_of relationships, indicating that these approaches have their own individual strengths and weaknesses. Each method contributed specific results, though they may occasionally contradict, we found that providing both results (using OR logic) provided the best compromise and mapping facilitation. Our results suggest that a combination of lexical and instance-based matching could provide improved automated matching suggestions.

Previous studies have not been able to determine an ES threshold for instance-based matching. From our sensitivity analysis, we concluded that an ES threshold of 0.3 or greater provided a good balance between precision, recall and the number of pairs for SMEs to review. Despite the lexical bias in the development of the reference standard, instance-based matching performed slightly better than lexical matching in identifying equivalent pairs and was also useful later in identifying some child_of relationships. The instance-based scores presented in this paper improve upon those proposed by the framework and provide a reference of expected scores; however, scores may vary depending on use case and need to be validated in different data sets. Additionally, due to the incomplete nature of the EPCs, these values likely represent the lower bound of performance for instance-based mapping.

Application of results

First, we found these automated mapping methods to be useful in supporting the development of a mapping between medication classification systems. We want to point out that the goal of these automated matching methods is not to identify 100% correctly the first time, but to help narrow down the thousands of potential matches to a more manageable list so that someone knowledgeable of both terminologies can review, validate, or correct the mappings. Therefore, we found that manual mapping and SME review are still needed, but that the use of lexical and/or instance-based matching facilitated the process and could point the mapper to the right section in the other ontology (particularly true for the instance-based method). These matching methods also reduced the amount of curation needed for the mapping and can be generalized to virtually any pair of medication classification systems (beyond EPCs and SNOMED CT).

Second, we also found these methods useful for quality assurance and validation. For example, both EPC concepts and SNOMED CT classes switched between grouping active ingredients by chemical structure and by therapeutic intent, which was variable within and between the two classification systems; however, we found that the instance-based method was able to inform the meaning of the class names, such as if the class was grouped using structural or functional properties. The variation of naming and grouping medications was especially apparent with the EPCs, such as the use of antibacterial versus antimicrobial classifications. For example, the EPC concept Macrolide [EPC] only has erythromycin listed in it, Macrolide antibacterial [EPC] has only fidaxomicin listed in it, and Macrolide antimicrobial [EPC] has azithromycin, clarithromycin, and erythromycin listed as instances, yet all these instances are antibacterial agents with a macrolide structure, while other non-antibacterial agents with a macrolide structure (ie. tacrolimus or sirolimus) are not included in the Macrolide [EPC] class. One would expect that the concept Macrolide [EPC] would contain all medications with a macrolide structure, not just erythromycin. These inconsistencies arise because there was previously a lack in standardization in how EPCs were assigned [4]. We also found apparent gaps in SNOMED CT related to agonist/antagonist classifications, such as Dopamine-2 Receptor Antagonist [EPC] and Gamma-Aminobutyric Acid A Receptor Agonist [EPC]. Inconsistencies and unused medication classes were reported to the FDA and SNOMED International to help improve the medications classifications. Since then, SNOMED International has been working on revising their medication classes in the Substance hierarchy. The identification of these types of inconsistencies using lexical and/or instance-based matching may provide opportunities for review, validation, correction, and further classification development [8].

Thirdly, since conducting this project, the results of this work helped to inform the development of a new RxNorm tool from the National Library of Medicine named RxClass (https://mor.nlm.nih.gov/RxClass/) [5]. RxClass provides a graphical web interface and application programming interfaces (APIs) for exploring medication class structures from various sources, as well as their corresponding RxNorm ingredients. RxClass can handle the IN/PIN normalization and find similar classes (with equivalence and inclusion scores) using instance-based matching across the various classifications in RxNorm.

Limitations

This study only mapped EPCs to equivalent SNOMED CT medication classes, and may not meet all of the needs or use cases of others. As such, we did not use inclusion scores, which would have represented the EPC as a child_of or parent_of the SNOMED CT class. Additionally, for our use case, our reference standard was developed based on the medication class intention, not always on extension, and as such was biased in favor of lexical matching, as we did not consider class instances in the final mappings. The mappings between EPCs and SNOMED CT were found to be mainly useful for navigation purposes, and their utility in clinical decision support remains to be studied, especially as the mappings did not account for inferred relationships caused by mappings. EPC concepts that could not be mapped to SNOMED CT could have medication classes added to SNOMED CT using extensions.

Secondly, instance-based matching is dependent on classes sharing instances. There were many empty and incomplete EPC classes, such as Narcotic Antitussive [EPC] not having any instances, dextromethorphan not listed as an antitussive, and erythromycin as the only instance in Macrolide [EPC]. Additionally, this study was limited to active ingredients mapped in RxNorm with RxCUIs and SNOMED CT identifiers. Some active ingredients were excluded because they are outside of the scope for RxNorm, such as allergens, pollens, foods, and herbals. These results likely represent the lower bound of the instance-based mapping performance.

Conclusion

Comparing and mapping between medication classes is challenging task due to the different ways of classifying medications. Using lexical and instance-based matching, with manual review, we were able to map most of the EPCs to medication classes in SNOMED CT. Each method had its own unique strengths, weaknesses, and contributions. The use of instance-based matching in addition to lexical matching can help map or compare medication classes. The evaluation and comparison of ontologies is a complex process, and while these automated matching techniques can help, manual review by subject-matter experts is still needed for mapping between medication classification systems.