A survey of SNOMED CT implementations.

The Systematised Nomenclature of Medicine Clinical Terms (SNOMED CT) has been designated as the recommended clinical reference terminology for use in clinical information systems around the world and is reported to be used in over 50 countries. However, there are still few implementation details. This study examined the implementation of SNOMED CT in terms of design, use and maintenance issues involved in 13 healthcare organisations across eight countries through a series of interviews with 14 individuals. While a great deal of effort has been spent on developing and refining SNOMED CT, there is still much work ahead to bring SNOMED CT into routine clinical use.

Introduction

Countries such as the United States, United Kingdom, Canada, New Zealand and Australia have designated the Systematised Nomenclature of Medicine Clinical Terms (SNOMED CT) as the recommended clinical reference terminology for clinical information systems 1 (CIS) [1], [2], [3], [4], [5]. However, despite the reported use of SNOMED CT in over 50 countries [6], there are still few details on how SNOMED CT is implemented.

The International Health Terminology Standards Development Organisation (IHTSDO) Technical Implementation Guide defines three types of implementation: clinical records, knowledge representation, and aggregation and analysis [7]. Clinical records refer to the handling of patient data and include services such as recording, storing, retrieving and communicating SNOMED CT-enabled data in a CIS. Knowledge representation refers to expressing clinical knowledge such as clinical guidelines and care pathways in SNOMED CT. Aggregation and analysis refers to the retrieval of data from CIS for the purpose of secondary analysis. For the purpose of this study, we define implementation as the design, use and maintenance of SNOMED CT in the context of a CIS. “Design” refers to compiling subsets, developing data entry interfaces, programming search algorithms, selecting a data storage method, incorporating cross maps and developing data retrieval functions. “Use” refers to clinicians interacting with the CIS, receiving training and accepting the system. “Maintenance” addresses the continued updating of SNOMED CT-enabled CIS.

Two recent online questionnaires conducted by the IHTSDO [8] and Elhanan et al. [9] have shed light on SNOMED CT implementation. However, these questionnaires did not probe into the finer implementation details such as data entry, storage, retrieval methods and local maintenance. While there are many publications that describe technical aspects of SNOMED CT, compare the content coverage of SNOMED CT to other term sets and report the use of SNOMED CT in research studies [10], our search for publications describing how SNOMED CT is implemented in clinical settings yielded few results. Implementation publications focused mostly on data capture (e.g., structured data entry templates [11], clinical documentation [12], [13], [14], synoptic checklists [15], questionnaires [16], indexing clinical notes [17]) and to a lesser extent data retrieval through the use of synonyms (e.g., clostridium difficile Infections [18], neuromuscular blockade [19]) and decision support (e.g., nursing interventions for pressure ulcer wounds [23], detecting adverse drug events [20]). While there was a fair amount of detail on how SNOMED CT concepts were used in clinical systems, there were fewer details on how those concepts were stored and what methods were used to facilitate retrieval and decision support.

Implementing SNOMED CT is still relatively new and is a challenging proposition; therefore, the motivation behind our study was to conduct interviews with individuals who have implemented SNOMED CT to explore methodologies used to derive SNOMED CT subsets and extensions, how data is entered, stored and retrieved, the use of post-coordination, cross maps, maintenance processes and policies, and lessons learned.

This paper is organised into four sections. First, we describe the materials and methods used in our study. Second, we present the results of the interviews, which include descriptions of the SNOMED CT implementations, challenges, success factors and benefits of SNOMED CT described by the interviewees. Third, we discuss the issues raised and describe the steps towards a successful implementation. Finally, we end with a conclusion.

Materials and methods

Survey instrument

We used the Delphi method to select the interview questions. First, we compiled questions from previous SNOMED CT use questionnaires [8,9] and derived questions pertaining to the Centre for Health Information Research Development’s (CHIRAD) Solutions Support Model [21]. Second, each co-author individually reviewed the questions, added questions of interest and prioritised the questions by importance. Third, we discussed together the validity of the questions, combined duplicate questions and after another round of individual prioritisation, pared down the number of questions to 36, spread across 10 sections based on consensus. The first three sections dealt with background information, lessons learned and usability while the last seven sections centred on specific areas of design, use and maintenance.

The questions were submitted to the IHTSDO Chief Implementation and Innovation Officer (CIIO) for feedback and the first author conducted pilot tests with the other three co-authors, all of whom have been involved in SNOMED CT implementation in clinical settings.

Recruitment process

Our process to recruit individuals involved in SNOMED CT implementation was via email and was carried out in four stages.

First, we invited individuals who responded to the “IHTSDO Survey to Gather the Use, Benefits and Tools of SNOMED CT” [8], One question in the survey was “Would you be willing to accept a follow up regarding the topics of this survey?” As the survey included a confidentiality clause, we contacted the IHTSDO CIIO, who sent out email invitations to the survey respondents to participate in our study.

Second, we invited individuals who were listed on Canada Health Infoway’s SNOMED CT in use website [22]. The publicly accessible website provides a summary of 18 initiatives and contact information. The statuses of the initiatives listed included two that were “on hold/research initiative”, seven that were “in progress/ongoing/development” and nine that were “implemented”. As the catalogue of initiatives was last updated in June 2011, we contacted five individuals whose initiatives were not listed as “implemented” to check on the statuses but they responded that their initiatives were still in progress.

Third, we invited six individuals who made presentations at the IHTSDO Implementation Special Interest Group webinars [23]. The presentations ranged from theoretical demonstrations of SNOMED CT to pre-development work (e.g., the development of reference sets) to the implementation of pilot projects and production systems.

Lastly, we invited other individuals whom we knew had implemented SNOMED CT in clinical settings but were not part of the first three stages of recruitment. The main inclusion criterion was that the individuals had implemented SNOMED CT in a clinical setting.

Interviews and analysis

Participants agreed to the terms of the participation consent form as part of the human ethics requirement at the University of Victoria (Protocol Number 11–535). Doodle was used to schedule individual interviews and participants were interviewed by phone or through Skype.

All interviews were digitally recorded and transcribed to aid in the inductive content analysis. We developed a secure website, whereby the co-authors could review the transcripts, mark-up the transcripts using a colour coding scheme for the inductive content analysis and to add notes.

Results

Subject characteristics

Of the 50 invitations sent out (IHTSDO survey: 30; Infoway catalogue: 9; IHTSDO webinars: 6; Other individuals: 5), 13 interviews were conducted with 14 individuals (IHTSDO survey: 9; Infoway catalogue: 2; IHTSDO webinars: 2; Other individuals: 3) for a 28% response rate. Six other individuals initially expressed interest in participating but did not respond to follow-up emails. One other organisation also expressed interest but their current use of SNOMED CT was in mapping and not implementation. The interviews were conducted over a 7 week period between February and April 2012. The participants were from eight countries and included physicians, academics, clinical terminologists, software developers and vendors.

Description of implementations

As most interviewees had worked on multiple SNOMED CT implementations, we focused on the most mature implementation or one that the interviewee thought was most beneficial to this study. Refer to Table 1 for a summary of the projects. Of the 13 projects discussed, two were pilot projects, two were under development, and two were currently being implemented while seven were production systems. The production systems had been in place between 3 and 10 years. It should be noted that the two pilot projects and one production system are no longer in use. The first pilot project was discontinued upon completion and funding was not available to proceed to a second phase while the second pilot project was discontinued after the trial period due to disappointing usability results. In the third project, the vendor had left the country and the organisation had to switch to a new vendor.

Table 1

Summary of results of interviews.

No	Type	Use	Clinical domain	Extent of encoding	Develop own subsets?	Uses cross maps?	Data storage design	Retrieval	Post-coordination
#1	Academic	Pilot (1 year)	Palliative care	Consult letters (i.e., problems, metastasis and past medical history)	Palliative care (2 k concepts)	No	Interface Terminology	Structural subsumption for study purpose only	Limited qualification for end-users
#2	Vendor	Production (6 years)	Hospital-wide	Problem list	Hierarchy subsets, 4000 concepts	ICD-10	SNOMED CT Concept Id	For reports, not using hierarchy	Not needed
#3	Vendor	Implementation	Personal health records	Patient summary, problem list, allergies	No, but uses CORE Problem List, VA/KP Problem List	ICD-9, UMLS	Interface Terminology	Semantic search using SNOMED CT relationships	Not used
#4	Healthcare enterprise	Development	Hospital-wide	Problem list, laboratory	5000 subsets (range five to several thousand) 5–6000 concepts	No	Interface Terminology	Enumerated list	For terminology team only
#5	Vendor	Implementation	Hospital-wide	Problem list, allergies	No	No	Interface Terminology	No	Not used
#6	Government	Development	Primary care EMR	Problem list, allergies	Yes, unknown, diagnosis up to 70,000	Plan on using ICD-10	SNOMED CT Concept Id	Enumerated list	Not used
#7	Academic	Production		Hospital-wide reports (e.g., radiology, pathology)	Not this particular project	No	SNOMED CT Concept Id	Class expansion, enumerated list	Not used
#8	Healthcare Enterprise	Production (6 years)	Ambulatory Care, Hospital-wide	Complaints, past medical history, signs and symptoms	1.5 k concepts	Plan on using ICD-10	SNOMED CT Concept Id	Hierarchy	Yes, but no further details
#9	Academic	Pilot (1 year)	Intensive care	Reason for admission	Intensive care unit (∼83 k concepts)	No	SNOMED CT Expression, description id	No	Refinement and qualification for end-users
#10	Vendor	Production	Hospital-wide		8 main subsets	No	SNOMED CT Extension Description Id	Hierarchy, attributes, enumerated list	For terminology team only
#11	Healthcare enterprise	Production (3 years)	Primary care EMR	Problem list, diagnosis, procedures	No	Would like to use ICD-9-CM	SNOMED CT Description Id	Enumerated lists	Limited qualification for end users
#12	Healthcare enterprise	Production (3 years)	Interdisciplinary Practice	Chief complaint, past medical history, social history, family history, physical exam	No	ICD-9-CM	Probably SNOMED CT Description Id	Enumerated lists	Limited qualification for end users
#13	Healthcare enterprise	Production (>10 years)	Hospital-wide	Diagnoses	8 main subsets	ICD-9-CM	Interface Terminology	Subsumption	Not used

The domains included ambulatory care, intensive care, palliative care, primary care and specialist care. The extent of encoding included problems and complaints, signs and symptoms, past medical history, patient summary, allergies, metastasis, reason for admission, procedures, various reports (e.g., radiology, pathology) and partial laboratory results.

The reasons for using SNOMED CT included pilot implementation for proof of concept, replacing an interface terminology, a mandate to migrate from a previous standardised terminology, SNOMED CT being the best available terminology for the use case, complying with government requirements for meaningful use and certification and to facilitate decision support.

We report the results of the implementations according to our definition of implementation: design, use and maintenance. It should be noted that there are some overlaps between design and use. For example, data entry interfaces need to be developed for and used by clinicians. In this case, the emphasis is on the use as opposed to designing the interface.

Design

Subsets

Two types of subsets were identified: data entry and data retrieval (see Section 3.2.2.3). Data entry subsets were used to help constrain the concepts to a specific domain for use in recording patient encounters. They were used in eight projects and derived in four main ways: SNOMED CT hierarchy (n  = 2), local or standardised terminologies (n  = 3), patient records (n  = 3) and expert selection (n  = 2). Refer to Table 2 for a summary of the data entry subsets and extensions. In addition to developing subsets, one project used the Clinical Observations Recording and Encoding (CORE) Problem List subset 2 and the Veterans Health Administration and Kaiser Permanente (VA/KP) Problem List subset. 3

Table 2

Subsets and extensions.

No	Domain	Source	Hierarchies	Concepts	Extensions
#1	Palliative care	IT, encoded historical patient records for problems at referral and diagnosis	Clinical findings	∼2000	Local only
#2	Hospital	SNOMED CT hierarchical subsets	Clinical findings, disorders, procedures and events		None
#4	Problem list and 5000 value sets	IT, historical records and expert feedback	All hierarchies	Each range from 5 to several thousand	Local only
#8	Various	Interviewed own doctors, reviewed paper forms, proposed sibling and children concepts	Clinical findings	∼1500	Local only
#9	Intensive care unit reason for admission	Mapped ∼450 reasons for intensive care unit admission to SNOMED CT and included all subtype concepts	Clinical findings and procedures	∼83,000 plus the defining attributes of those concepts	Local only
#10	Diagnosis, signs and symptoms, drugs and substances, surgical procedures, diagnostic procedures, medical devices, obstetrics and gynaecology	Encoded over 10 million historical patient records with frequency greater than 10	All hierarchies	∼20,000	Yes, using namespace
#12	Haematology and oncology, cardiology, ophthalmology, musculoskeletal, neurology, mental health, other	Clinician selection		∼75,000	Yes, using namespace

SNOMED CT hierarchy. This refers to limiting the concepts to a certain hierarchy or to subtypes of a (set of) concept(s). Examples include limiting the scope to “64572001|Disease (disorder)|” and “71388002|Procedure (procedure)|”.

Local or standardised terminologies. Subsets were derived by mapping a local interface terminology or (inter)national classification to SNOMED CT. For example, an intensive care unit (ICU) classification that contained over 450 terms for reasons for ICU admission was mapped to SNOMED CT. In addition, all subtype concepts of those mapped concepts were also included in the subset. For example, the term “heart attack” was mapped to “22298006|Myocardial infarction (disorder)|”, which contained nearly 60 subtype concepts.

Patient records. Historical electronic patient records recorded using free text were analysed and mapped to SNOMED CT. In one project over 10 million patient records were analysed. Normalisation techniques and encoding algorithms were applied to terms that occurred at least 10 times. The resulting subset contained over 20,000 unique descriptions.

Expert selection. Clinicians listed terms that they wanted and worked with terminology experts, who identified corresponding SNOMED CT concepts and suggested additional terms, such as sibling or child concepts. For example, clinicians listed the commonly used body sites for measuring blood pressure.

Extensions

SNOMED CT extensions are formal additions to the SNOMED CT core using a namespace, which is a unique identifier that is assigned to individuals or organisations. 4 Two organisations created SNOMED CT extensions that were used to replicate components in the SNOMED CT core as an interface terminology and to create new components that did not exist in the core. One organisation submitted extensions to the National Library of Medicine for possible inclusion into the national extension and SNOMED CT core while the other felt it was unnecessary as the extensions they created were highly localised. Both organisations planned to send the parent concept that was part of the SNOMED CT core instead of extensions concepts when sending SNOMED CT data beyond their enterprise.

Five organisations used their interface terminology to create new descriptions that did not exist in SNOMED CT and therefore did not see the need to create formal extensions. The organisations used SNOMED CT extensions or interface terminology as they could not afford to wait for 6 months for a new concept to be released. The example cited was during the outbreak of the severe acute respiratory syndrome (SARS), whereby clinicians needed to code the disorder immediately. In one project, clinicians needed pre-coordinated concepts that included laterality so the vendor worked with the IHTSDO to develop pre-coordinated body structure concepts that included laterality.

Data storage

There were four ways in which SNOMED CT-enabled data was stored: concept ids (n  = 4), description ids (n  = 4), post-coordinated expressions (n  = 1) and interface terminology codes (n  = 5). In three projects the free text descriptions entered or selected were recorded in addition to the codes.

Concept Ids. Concept ids were recorded directly into patient records. As description ids were not recorded, the terms that were displayed back to the clinicians were the preferred terms even though a synonym may have been selected during data entry.

Description Ids. Only description ids were recorded directly into patient records. In this case, the actual description used could be displayed in the record. One project only stored description ids from a formal SNOMED CT extension.

Post-coordinated expressions. The close-to-user form of post-coordinated expressions was stored as a text string directly into patient records

Interface terminology codes. Interface terminology codes, which were mapped to SNOMED CT, were stored in the patient record. Reasons for using interface terminology codes include organisations had been using their own codes for several decades and it would be difficult to change over, and to shield themselves from the updates to SNOMED CT.

Use

Data entry

SNOMED CT data was captured in five ways with projects using multiple data entry methods: drop down lists (n  = 3), browsing the hierarchy (n  = 2), auto-complete (n  = 12), free text with ad hoc coding (n  = 1) and free text with post hoc coding (n  = 1).

Drop down list. Drop down lists were used when the number of items were small, generally less than 20. Examples included positions and body sites for measuring blood pressure.

Browsing the hierarchy. Clinicians were able to browse through a section of the SNOMED CT hierarchy and select concepts. For example, clinicians first selected whether they wanted to record a disorder or procedure, after which the selection was filtered to the hierarchy.

Auto-complete. Clinicians would type the first few letters of a word or words and the system would retrieve potential matches. A wide range of indexed tables and algorithms such as extensive keyword search mechanisms, spell check, the expansion of abbreviations and acronyms, word equivalency and synonym substitution (e.g., “lung” and “pulmonary”) were used. The order of the search results included displaying the terms by relevancy, frequency of term used in historical records or a hotlist. The “hotlist” referred to a set of terms selected by clinicians as the terms they wanted to see first. If clinicians were unable to find the term they wanted, free text was used. Five organisations had processes in place to retrieve free text entries, which were reviewed by terminology analysts and clinicians in order to be coded. Two organisations stated that in the 6 years of using SNOMED CT, there were only a handful of terms that could not be found as clinicians were trained to use different synonyms in order to locate the terms they needed.

Free text with ad-hoc coding. This is similar to the auto-complete but instead of recording a single phrase in a single data element, the auto-complete was used in a narrative. The clinician would type a narrative and suggestions for SNOMED CT concepts would be prompted, which could be selected, where needed.

Free text with post-hoc coding. Narratives were recorded and natural language processing algorithms were used to index the narratives with SNOMED CT concepts.

Post-coordination was used in six projects and can be grouped into limited qualification (n  = 3), qualification and refinement (n  = 1), and full post-coordination (n  = 2).

Limited qualification. Clinicians were allowed to use limited qualification that centred on laterality, severity, episodicity and clinical course. The qualifiers were used as discrete data elements though it is unclear whether post-coordinated expressions were constructed in the background.

Qualification and refinement. Clinicians were allowed to qualify and refine concepts and were shown the defining attributes of concepts once a concept was selected from the auto-complete.

Full Post-coordination. Post-coordination, including the use of Concept Model attributes, was used only by the technical team to map the interface terminology to SNOMED CT as it was felt that post-coordination was too complex for clinicians. It should be noted that an extension concept was created for each post-coordinated expression.

In one CIS, post-coordination was also done in the background for family history. For example, when a clinician typed in “acute myocardial infarction” in the family history section, a look up was performed to determine if there was a pre-coordinated concept for “family history of acute myocardial infarction”. If it existed, that concept was used. If it did not, an extension was created automatically and the new concept was queued for validation by a clinical modeller. This process was transparent to clinicians.

Cross maps

Cross maps are mappings between SNOMED CT to another terminology and were used by four organisations. First, the ICD-10-CM cross map, which was supplied by the National Health Services in the United Kingdom, was used to generate statistics as part of government requirements. Second, two organisations used the ICD-9-CM cross map to generate billing codes. Third, a cross map to APACHE II & APACHE IV codes was used for calculating mortality risks and for benchmarking quality of care.

Two organisations were planning on using cross maps. The first was still in the process of mapping their interface terminology while the second was in the process of implementing SNOMED CT for diagnosis (in addition to the complaints, past medical history and signs and symptoms). One other organisation wanted to use the ICD-9-CM cross map to aid the billing process but the vendor declined to implement this feature.

Data retrieval

The use of retrieval functions on SNOMED CT-encoded data varied across the projects and included retrieving concepts by an enumerated set of concepts (n  = 7), subsumption through the SNOMED CT hierarchy (n  = 5) and defining attributes (n  = 2). Enumerated sets of concepts, or data retrieval subsets, refers to users (either clinicians or technical analysts) selecting concepts individually either by descriptions or concept/description ids. Transitive closure tables were not used when testing for subsumption as built-in database functions such as Oracle’s hierarchical queries were sufficient. Description logic was also not used when testing for equivalency and subsumption. Three projects did not allow for data retrieval, one pilot project only had data retrieval functions for the investigators while clinicians in one project needed the terminology team to run the queries. The reasons for including data retrieval functionality were the need to report statistics for government purposes, to identify patients with chronic diseases, or to conduct research for clinical or educational purposes.

User acceptance

Four interviewees felt that clinicians were generally in favour of using SNOMED CT as long as it did not interfere with their workflow. The use of SNOMED CT helped to demonstrate the importance of using standardised codes and consistent processes. In one project, where historical records were encoded with SNOMED CT to form the basis of their interface terminology, clinicians were surprised at the poor quality of data. In most cases, SNOMED CT had been so seamlessly integrated that users were unaware that they were using SNOMED CT through an interface terminology.

Training

Five interviewees reported that it was difficult to ascertain the amount of training that was needed to use a SNOMED CT-enabled system because training sessions focused on CIS as a whole as opposed to just a specific segment where SNOMED CT was used. In the case where a hospital was changing from paper records to electronic records, the training centred on computer skills and general acquaintance with the CIS interface. With respect to organisations that were already using a CIS, drop down lists and auto-complete functionality were already commonplace; therefore no additional training was required. In terms of post-coordination, the qualifier values were kept as discrete data elements so clinicians did not have to select a Concept Model attribute to link the two concepts together. Only one project chose to display the full set of defining attributes for a concept to allow end-users to refine the concept. Although training was provided, clinicians still found it challenging to use the system and it was concluded that the interface was too complex. In general, organisations that had implemented SNOMED CT via interface terminology did not require any additional training for clinicians.

Maintenance

Three interviewees updated SNOMED CT every 6 months almost immediately, three had an allotted time period to make the switch, one made the switch once a year while two were using the same version for the past 3 years as the vendors had stopped supporting the product or declined to update the version. The other four projects were either pilot projects and did not encounter multiple versions, did not know what version they were using or when it was last updated.

The organisations that had updated SNOMED CT mainly checked to see if any concepts in their subsets or concepts mapped to their interface terminology were inactivated. If they were inactive, terminology analysts searched for alternative concepts and suggested them to clinicians. It is unclear how and if the organisations updated the historical patient records as the interviewees did not have that information. It was mentioned that several countries such as the United Kingdom have some form of clinical safety and legal requirements that the original code should never be deleted or altered in order to preserve them in perpetuity.

In the project that recorded patient records using description ids from an extension, if the concept linked to the description was inactivated, the description id was re-linked to another active concept. The rationale was that the textual description of the description should never change but it was permissible for the description to be re-linked to a more appropriate concept.

Challenges

The challenges indentified by the interviewees can be categorised as SNOMED CT quality challenges; design, use and maintenance challenges; and other challenges.

SNOMED CT quality challenges

The SNOMED CT quality challenges fall into four main categories: content coverage (n  = 5), hierarchical relationships (n  = 4), ambiguity of terms (n  = 3) and syntactic consistency (n  = 1).

Content Coverage

While two interviewees mentioned that they had only encountered a handful of missing terms since implementing SNOMED CT over 5 years ago, another interviewee estimated that SNOMED CT was missing between 1% and 15% of terms needed for any given domain. For example, there was a concept for “302203004|Wife unable to cope (finding)|” but not for “husband unable to cope”. Another area pointed out by three interviewees was the lack of medications and ingredients.

Hierarchical relationships

Interviewees expressed challenges with using the hierarchy because of missing relationships or inconsistent intermediate relationships. For example, the concepts “69973000|Vascular anomaly of eyelid (disorder)|” and “193966008|Eyelid vascular anomalies (disorder)|” both refer to a vascular anomaly of the eyelid structure, with the latter concept referring to a congenital occurrence. The two concepts, however, do not have a subsumption relationship. Traversing the hierarchy and aggregating data using the hierarchy was difficult because of variations in recording the same encounter with different concepts that may reside in different locations in the hierarchy. The complexity of the hierarchy as well as missing concepts led one user to admit that they did not completely trust the hierarchies and that a healthcare organisation cannot solely depend on the hierarchies to develop their decision support features.

Ambiguity of terms

There were synonyms in SNOMED CT that had the same description but referred to different concepts. One organisation admitted to making coding mistakes in the early stages because they did not know how to distinguish between the nuances in SNOMED CT but had improved through training. Incidentally two interviewees cited the same example of “cold”, whereby it could refer to a “common cold” (“82272006|Common cold (disorder)|”) or “cold injury” (“11925005|Effects of reduced temperature (disorder)|”). While only the former includes a synonym of “cold,” this can still cause confusion to clinicians who interpret that to mean the latter. Another organisation dealt with this issue by creating an exclusion subset of ambiguous descriptions.

Syntactic consistency

There were inconsistencies in which punctuation such as hyphens, full stops and commas were used in SNOMED CT. In addition, there were different linguistic styles and a mixture of acronyms in descriptions. For example, there was no single description of “lung cancer”. The fully specified name was “755174012|Malignant tumor of lung (disorder)|”, the preferred term was “482515017|Malignant tumor of lung|” and there was a synonym of “1228498010|CA – Lung cancer|” but there was no synonym for “lung cancer” by itself. This inconsistency was challenging when developing auto-complete and natural language processing algorithms.

Other SNOMED CT quality challenges

Other challenges mentioned include the lack of translation to other languages, the challenge of handling metonymy and relationships using the current flavour of description logic and the lack of cross maps for ICD-10-CM, which is currently under development.

Implementation challenges

There were three main types of SNOMED CT-related implementation challenges mentioned by the interviewees: post-coordination (n  = 7), subsets (n  = 4), and data retrieval (n  = 4).

Post-coordination

First, the interviewees did not have a good strategy on how to design a post-coordination interface that was intuitive and unobtrusive. Second, clinicians were not willing to split their input into separate terms. The example cited was “respiratory failure due to pneumonia”. Separating it into “respiratory failure” and “pneumonia” would lose the context that “respiratory failure” was the result of “pneumonia”. Requiring the clinician to select the Concept Model attribute “42752001|Due to (attribute)|” to bridge the two concepts was deemed cognitively taxing and time-consuming. Third, there were concerns as to whether post-coordinated expressions could be tested for equivalency and subsumption with pre-coordinated concepts and how ICD codes could be retrieved for post-coordinated expressions.

The main challenge was how to craft a subset for domains that were broad (e.g., reason for admission) as concepts could not be easily restricted to a hierarchy or parts of a hierarchy. For example, not all reasons for admission were diagnoses; they could also include events and monitoring after procedures. Interviewees expressed concern for the lack of clear subset development methodologies and felt that the IHTSDO should provide more guidance. Interviewees who have had experience developing subsets suggested starting from a domain as restricted as possible and working towards more complex ones.

Data retrieval using the hierarchy was challenging for three reasons. First, the hierarchy is constantly changing with each release version of SNOMED CT; therefore the answers to clinicians’ queries could change over time. Second, the hierarchies were not always conductive for data aggregation as there were missing hierarchical relationships and intermediate concepts which made it difficult to select appropriate concepts for “roll up”. Therefore clinicians could end up with unexpected results.

Other challenges

Other challenges identified by the interviewees that were not directly related to SNOMED CT content are described here.

Change resistance

There were clinicians who balked at using SNOMED CT as they feared it would interfere with their patients consults. For clinicians who had not heard of SNOMED CT or had only heard negative comments, it was important to introduce them to SNOMED CT, expose them to the benefits of using SNOMED CT, and made aware of the deficiencies of their current coding scheme.

Coding granularity

There was a need to recognise that not everything could be coded and that there was still a place for free text. For example, consult or referral letters had to include narratives and could not just be a list of codes as they needed to be used for legal or insurance purposes. As one interviewee stated, “the whole world is not a template you tick off” (Interviewee #12).

Policies

The lack of policies was a barrier as vendors did not necessarily see the benefits of incorporating a new complex terminology when there was no clear government mandate. The example cited was in a country where SNOMED CT was identified as the most suitable clinical terminology for 24 sub-domains but there was a lack of promotion of SNOMED CT or how it should be used. The provincial EMR adoption certification programs bear little references to how SNOMED CT should be implemented.

Success factors

The success factors described by the interviewees fall into five categories: simplicity, clinician involvement, expertise and collaboration, demonstrate value and training.

Simplicity

The most common success factor was to keep the user interface simple for clinicians by hiding the complexities of SNOMED CT. As one interviewee described it, “our model is simplicity” (Interviewee #2). In one pilot project that did not adequately hide the complexities of SNOMED CT, clinicians were confused with the number of options available. Multiple references were made about how simple Google was to use and how suggestions and relevant results were provided quickly.

Clinician involvement

Engaging clinicians during the development phase to solicit their input and act upon their suggestions and concerns were important to gain their support. Interviewees felt that migrating to SNOMED CT was easier when there were clinicians who understood the value of having a longitudinal electronic health records and had positions of decision making influence.

Expertise and collaboration

Having a terminology team comprised of terminology experts, analysts, clinicians and programmers was necessary to ensure both clinical and technical viewpoints were represented. One the interviewees pointed to the lack of a reference implementation as one of the pitfalls of implementation and that it was important to contact other organisations that have implemented SNOMED CT to learn from their experience.

Demonstrate value

There was a need to demonstrate immediate value to clinicians for using SNOMED CT and the value depended on the maturity level of the implementation. Having a legible patient record was adequate for first time CIS users while experienced CIS users required more functionality such as decision support features.

Training

The amount of training needed varied depending on the stage an organisation was with their CIS implementation. The organisation that changed from a paper-based system to a CIS required more training and was a central success factor compared to other organisations that already has been using a CIS. For organisations that had just made the switch to CIS, the key to training was to start in very small areas and add complexities later on once the foundation was there.

Benefits of using SNOMED CT

None of the initiatives had carried out extensive evaluations to determine the benefits of using SNOMED CT. Reasons included SNOMED CT applications were still being developed or implemented and they had other priorities or had no capacity to carry out evaluations. The interviewees were, however, able to describe some of the benefits they had observed: direct data entry (n  = 4), data reuse (n  = 4), content coverage and subset development (2) and legibility (n  = 1).

Direct data entry

The large number of synonyms in SNOMED CT enabled clinicians to record the exact diagnosis they had in mind in contrast to post-coding, whereby a terminology analyst reviewed the free text entry and selected a concept that he/she thought was appropriate. An interviewee believed that direct data entry through the intuitive interface increased the accuracy and speed in which records were coded.

Data reuse

Organisations that used SNOMED CT in conjunction with ICD cross maps were able to re-use the data that was captured via SNOMED CT to generate ICD codes for billing and statistical reports. One organisation profited from using SNOMED CT by being able to identify patients with certain clinical conditions for referral to clinical trials conducted by pharmaceutical companies and universities. Another way in which SNOMED CT helped to generate revenue was by enabling clinicians to identify patients with chronic diseases such as diabetes mellitus and congestive heart failure in chronic disease management programs that included remuneration for registration.

Content coverage and subset development

Interviewees felt that SNOMED CT provided them with the best content coverage for their use cases compared to other terminologies and that it was still a good starting point for developing subsets despite the issues of missing concepts and relationships.

Legibility

While not directly related to SNOMED CT, the legibility of the patient record was one of the immediate benefits in one project that made the switch from a paper-based system to an electronic system using SNOMED CT. In a project where SNOMED CT was used in an existing CIS, the problem list was standardised.

Discussion

There are few publications that describe the implementation of SNOMED CT in clinical settings in detail, whereas the majority of publications focus on comparing SNOMED CT to other terminologies and to illustrate terminology systems theory [10]. The lack of publications that span the full scope of design, use and maintenance has made it challenging for organisations that are looking for reference SNOMED CT implementations. This study attempted to fill the gap through a series of interviews that looked that the benefits, success factors, challenges and implementation approaches.

Towards a successful SNOMED CT implementation

In this section we attempt to enumerate the steps towards a successful SNOMED CT implementation drawn on lessons from this study as well as other publications.

Understand process of SNOMED CT implementation

A multi-disciplinary team [24], [13] that includes terminology experts, clinicians, technical experts and project managers should be formed to understand the technical details of SNOMED CT as well as the challenges, success factors and benefits of using SNOMED CT such as those described in this study. They should also be made aware of limitations such as data quality issues [25] and the inability to adequately represent negation and disjunction [26]. In addition, SNOMED CT should be used in conjunction with an information model to represent data types such as dates and numeric values [27].

Encode local terms to SNOMED CT

Allowing clinicians to use an interface terminology they are familiar with may make the transition to SNOMED CT smoother [28], [29], [30], [31]. The most common source of local terms may be from currently coded value sets but historical patient records recorded with free text should also be considered as they contain a rich source of terms [32]. The interface terminology layer will help to shield end-users from changes made to SNOMED CT although the technical team should review any change made to SNOMED CT [33], [34].

Create extensions

Where applicable, extensions should be created for concepts that do not exist in SNOMED CT [35]. While informal extensions can be created using an interface terminology, formal extensions that are defined can contribute to the semantics. Extensions should be submitted to national release centres for possible inclusion into national extensions or into the SNOMED CT core.

Compile subsets

Subsets should be compiled to constrain the relevant concepts to use cases and may be compared to publicly available subsets (e.g., VA/KP, CORE and Convergent Medical Terminology (CMT)) for content coverage. Effort should be still spent on developing search algorithms as there is still a need to be able to retrieve terms within a subset.

Design intuitive data entry interfaces

Auto-complete is well-suited data entry method as opposed to long drop down lists or browsing the hierarchy [36]. Depending on the context of the data elements, hotlists and most frequently used terms should be made available. Search algorithms should include partial multi-word searches [37], spelling corrections, word equivalency, and abbreviation and acronym expansion. The use of templates can also help to facilitate the recording of data into the correct data elements [38], [39].

Select a data storage method

Interface terminology codes and SNOMED CT ids are the two main data storage methods and each has its advantages and disadvantages. The use of interface terminology codes in patient records is less invasive method as minimal changes will be made to the database backend and historical patient records can remain unchanged. Any changes made to SNOMED CT will only need to be reflected in the mapping tables. On the other hand, storing SNOMED CT ids directly will enable decision support systems to be supported directly without the need of interface terminology mapping tables and will not require the maintenance of the interface terminology.

Incorporate cross maps

Cross maps from SNOMED CT to classification systems such as ICD-9-CM are needed to facilitate reimbursement processes and the generation of statistical reports without the need to code multiple times [40] and to allow for the continuity of historically records coded with classification systems [41].

Design retrieval functions

Data retrieval functions should be included to harness the rich semantics in SNOMED CT and enable clinicians to run patient case queries and to facilitate decision support. Organisations that do not have the capacity to develop advanced retrieval functionality can consider using third party terminology services or application programming interfaces (APIs) such as the open-source Apelon Distributed Terminology System (DTS), 5 IHTSDO Workbench 6 or National Health Services Snofyre 7 to compute the aggregation of SNOMED CT concepts that can then be used in CIS.

Conduct training sessions

Training sessions should be conducted as not all clinicians have the same technical capabilities [42] and it provides a forum where clinicians can be educated on new features and have the opportunity to have their questions answered. If data entry interfaces are intuitive and search algorithms are effective and efficient, the amount of training needed may be reduced [43].

Develop maintenance policies

Two types of maintenance policies should be in place. First, unencoded terms should be routinely extracted for review to determine if they can be coded. Second, when a new version of SNOMED CT is released, the mappings between SNOMED CT and the interface terminology and classification systems should be reviewed to ensure that only active concepts are used.

Incremental value of SNOMED CT

The promoted benefits of using SNOMED CT [44] were generally not realised to its full potential in the projects we examined. It should be noted that none of the organisations have conducted full-scale evaluations so it is premature to come to a decisive conclusion. Granted two were pilot projects and a few were still early in the implementation stages, but the organisations that had been using SNOMED CT for at least 3 years have not reported significant incremental value.

While considerable benefits have been realised such as being able to query patients with specific clinical conditions, record clinical records at a granular level and automatically generate ICD codes for statistical reports, it can be argued that these benefits may have been realised without the use of SNOMED CT. For example, the organisation that was using an interface terminology that was compiled and refined over several decades was already able to capture data at a granular level and facilitate decision support functionality, thus limiting the incremental value of using SNOMED CT. One of the reasons for using standardised terminologies is to enable interoperability; however, none of the organisations were communicating SNOMED CT data beyond their enterprise as part of routine operations. The benefit of using standardised terminologies can be achieved once (inter)national bodies formalise their guidelines and quality indicates based on SNOMED CT concepts.

We suggest some reasons for the limited benefits. First, large organisations already have robust interface terminology and will see limited incremental value until information is exchanged with external organisations. As there are very few SNOMED CT implementations in clinical care settings, it is unlikely that an organisation will transmit SNOMED CT-encoded data with other organisations in the near future. Second, the organisations that have been developing their own SNOMED CT solutions have thus far focused on data entry as opposed to data retrieval and decision support functionality and therefore are unable to demonstrate improvement in areas such as the quality of care. Third, organisations that rely on off-the-shelf vendor solutions have been unable to convince vendors to add new functionality due to the limited demand for SNOMED CT and therefore use SNOMED CT in a very basic manner.

Outstanding issues

From this study, there remain three outstanding issues that require further study: how to implement post-coordination, retrieval and extensions.

Post-coordination

Implementing post-coordination continues to be a challenge both from a graphical user interface design and clinical terminology point of view (e.g., creating clinically nonsensical concepts, concept duplication and inefficiency of concept composition). In some cases there are some alternatives to post-coordination, such as using an information model. For example, instead of including the subject relationship context in a single post-coordinated expression, the relationship is stored as a separate data element.

Retrieval

While retrieving pre-coordinated concepts and defining attributes are relatively straightforward, retrieval of post-coordinated expressions is relatively new. There are also unresolved issues as to how to retrieve corresponding ICD codes from post-coordinated expressions. A possibility would be to test if the concept used in the mapping subsumes the post-coordinated expression but this method has been untested. Tools such as the IHTSDO Workbench and National Health Services Snofyre have the potential to simplify the retrieval process by computing complex SNOMED CT-related calculations such as the testing for equivalency and subsumption outside of a CIS, but to date there are no published studies on the effectiveness or efficiency of these tools.

Extensions

As only one of the 13 organisations consistently submitted extensions for possible inclusion into a national extension set or the SNOMED CT core, there needs to be a streamlined method of submitting extensions and to monitor the progress. It remains to be seen how the creation of extensions will affect interoperability. Interviewees who stated they created formal extensions were asked this question and the answer given was that the parent concept from the SNOMED CT core would be transmitted instead. The implications of using the parent concept instead of the extension concept are unclear.

Limitations

The main limitation of this study was the small sample size. However, there are very few known SNOMED CT implementations in clinical care settings and the 13 interviews covered an adequate proportion of known implementations.

Conclusion

This study examined the implementation of SNOMED CT in healthcare organisations in terms of design, use and maintenance issues involved. Implementing SNOMED CT is a challenging proposition but there are organisations that have navigated the pitfalls and have successfully implemented SNOMED CT in both small and large sites. While a great deal of effort has been spent on developing and refining SNOMED CT, there is still much work ahead to bring SNOMED CT into routine clinical use. The steps outlined in this paper are based on the lessons of early adopters through experimentation and refinement over time. These are necessary steps toward what we hope are eventual successful implementations.