Standardising the Classification of Harm Associated with Medication Errors: The Harm Associated with Medication Error Classification (HAMEC).

Classifying harm associated with a medication error can be time consuming and labour intensive and limited studies undertake this step. There is no standardised process, and few studies that report harm assessment provide adequate methods to allow for study replication. Studies typically mention that a clinical review panel classified patient harm and provide a reference to a classification tool. Moreover, in many studies it is unclear whether potential or actual harm was classified as studies refer only to 'error severity'. The tools used to categorise the severity of patient harm vary widely across studies and few have been assessed for inter-rater reliability and criterion validity. In this paper, we describe the systematic process we undertook to synthesise the defining elements and strengths, while mitigating the limitations, of existing harm classification tools to derive the Harm Associated with Medication Error Classification (HAMEC). This new tool provides a harm classification for use across clinical and research settings. The provision of an explicit process for its application and guiding category descriptors are designed to reduce the risk of misclassification and produce results that are comparable across studies. As the World Health Organisation embarks on its international safety challenge of reducing medication-related harm by 50%, accompanying methodological advances are required to measure progress.

Key Points

Introduction

The World Health Organisation (WHO) announced the third Global Patient Safety Challenge as ‘medication without harm’ in 2017 [1]. The Challenge presents the ambitious target to “reduce the level of severe, avoidable harm related to medication by 50% over 5 years, globally”. Assessing efforts to meet this challenge requires accurate information on the prevalence and consequence of medication errors (defined as unintended failure in the drug treatment process that leads to, or has the potential to lead to, harm to the patient [2]). Over the last two decades, the prevalence of medication errors has been the focus of a large body of work, with systematic reviews targeting studies of neonates [3-5], children [6, 7], adults [8-11] and the elderly [12]. Where pooled estimates are provided, the rate of medication errors is estimated to occur in 10–20% of medication orders [7, 8, 10, 11]. The disparate medication error rates reported across studies have been attributed to diversity in the definition of errors [13-15], the methods used for their detection and classification [16-18], and the chosen denominator of prevalence [19]. In contrast, research on the consequence of medication errors, namely the extent to which medication errors cause harm (including injury, damage or illness), or have the potential to cause harm, is relatively scant [20-22]. A review of the prevalence of medication-related harm estimates that < 10% of inpatients will experience such harm, though variation across studies in the definition of what constitutes harm is consistently cited [20, 23, 24]. Standardising the definition of harm associated with medication errors [15], and the development and implementation of tools to measure harm are important patient safety priorities [25].

The assessment of harm associated with medication errors typically involves two fundamental components: the identification of potential or actual patient harm (hereby referred to as any harm) related to a medication event; and the classification of the degree or seriousness of that harm. Identifying any harm associated with medication errors can be time consuming and labour intensive, and there is no standardised process [16]. Assessing potential and actual harm are separate processes, each with unique challenges. It has been suggested that both processes ideally involve a panel, usually comprised of pharmacists, nurses and/or medical clinicians [16]. When assessing potential harm, the panel considers whether the error would normally be expected to cause harm to a patient, or if the panel has access to information regarding a patient’s clinical status, this can be used to inform what level of harm would be expected in that patient. This process is undertaken regardless of whether the specific error being assessed reached the patient. In contrast, actual harm can only be classified in situations where the patient experienced the error (e.g. the incorrect dose was administered to the patient). This classification requires specific information about the circumstances of the patient receiving the error and the actual harm. Once a potential or actual harm is identified, the degree or seriousness of harm can be classified. This classification relies on the quality of information recorded in patient records. Unfortunately, poor record quality is consistently cited to be a significant limitation in assessing adverse drug events [26-38]. The difficulty of classifying either potential or actual harm is reflected in reports of low inter-rater reliability among clinicians [39-41], although experience as a clinician, and engaging more than two clinicians has been shown to improve the consistency of these ratings [42].

An important, though infrequently employed step when assessing patient harm associated with a medication event involves considering the likely probability that the identified harms are causally linked to the medication error [16, 24, 43–52]. This causality assessment is typically completed with the use of a confidence tool; most frequently the Naranjo Algorithm [53], or the World Health Organization–Uppsala Monitoring Centre criteria [54]. Here, it is not the level of confidence that is important per se, but rather that a possible association between the medication error and the identified harm is established [55, 56].

Learning from previous studies that have classified the seriousness of medication-related harm is challenging as few studies have documented the exact processes used. Instead, studies typically mention that a clinical review panel classified harm, and provide a reference to a classification tool such as the National Coordinating Council for Medication Error Reporting and Prevention index (the NCC MERP [57]). In addition, widespread inconsistency in the terms used to define medication errors and the harm associated with medication errors make reporting and comparing research findings difficult [15]. Although significant efforts to provide standard definitions of medication errors and related harm have been made, for example the EU regulatory network has recently published a good practice guide [58], no standard tool for the classification of medication-related harm has been widely adopted. This is important as the use of a single tool across settings would contribute to the identification of trends in the kinds of errors that cause patient harm and provide focus to prevention efforts.

A review of the harm classification tools employed prior to 2013 identified 40 different classification tools among 61 studies of prescribing errors [39]. That review sought to identify tools with acceptable inter-rater reliability (κ > 0.7) and criterion validity (involving comparison of reviewer judgments of potential harm against actual harm measured in situations where the actual harm was known). Only two tools met this criteria: the NCC MERP for classifying actual harm [57] and Dean and Barber’s 10-point Likert scale for classifying potential harm [59]. However, these tools have also been the subject of some criticism [39].

The initial reliability testing of the NCC MERP was conducted under controlled conditions involving a panel of trained professionals (mostly pharmacists) classifying the actual harm associated with 27 medication errors identified in incident reports [60]. That study reported ‘substantial’ inter-rater agreement (κ = 0.61); however, the review panel reported being confused by the individual category definitions [60]. When presented with a modified scale (with categories C and D combined, and E, F and H combined—see the Electronic Supplementary Material [ESM]), the panel inter-rater reliability improved (κ = 0.74) [60]. In a study designed to determine if high inter-rater reliability could be obtained when the tool was used in everyday practice, the reliability was found to be fair (κ < 0.4) [40]. That study included a review panel of pharmacists and physicians who used the NCC MERP to classify actual harm associated with 30 medication errors identified by chart review.

Some author groups have discussed the limitations of Dean and Barber’s tool, including concerns that it is too time consuming to apply [39], and requires at least four reviewers to achieve an acceptable generalisability coefficient (> 0.8) [59]. Neither Dean and Barber’s tool nor the NCC MERP have been widely used in studies outside of the UK or US, respectively.

No tool for the classification of medication-related harm has been recommended for use across clinical and research settings. Therefore, there is scope for a new tool to be developed; designed with the goal of standardising the classification of harm severity associated with medication errors. In this article, we summarise the lack of standardisation in the process of classifying potential and/or actual patient harm following a medication error and propose a way forward. We begin by identifying inconsistencies in the nomenclature used across studies that have assessed harm. We then identify the similarities between the tools and the published protocols for their use and highlight limitations of these tools. Finally, we propose a new tool for the classification of potential and actual patient harm that addresses these limitations and describe its development.

Inconsistencies in the Nomenclature Associated with Harm Assessment

Error Severity

The current convention is to cite ‘error severity’ when referring to the extent of the potential or actual impact of medication errors. However, this term does not refer to the error as such, but to the potential or actual patient harm thought to be associated with the error. Thus, the common practice of reporting that the ‘prevalence and severity of medication errors were investigated’ (particularly in study abstracts) often does not make it clear whether potential or actual patient harm was assessed. Making this distinction clear is important given the obvious gravity of an actual patient harm event over potential for harm. Further, as many errors are intercepted before reaching the patient, errors that actually cause harm make up only a small fraction of the errors with the potential to cause harm [20].

Classification of Potential Harm

Different author groups have referred to harm that could be potentially caused by a medication error in various (often interchangeable) and inconsistent ways. These include ‘likely harm’ [61, 62], a ‘potential consequence’ [63, 64], an ‘assumed consequence’ [65], ‘predicted outcome’ [66], ‘potential risk’ [67-69], and/or ‘potential severity’ [70]. Adding to this inconsistency are studies that refer to errors with the potential to harm a patient as either ‘clinically significant’ [61, 71–75], ‘major’ [70, 76–78], ‘serious’ [42] [79, 80], ‘important’ [81], ‘problematic’ [74] and/or ‘high risk’ [82]. Essentially, each of these terms reflects a classification of potential patient harm into two broad levels of ‘severity’. That is, the error is called significant/major (potential for harm) or insignificant/minor (no or little potential for harm).

Potential harm caused by a medication error could be confused with ‘potential adverse drug events’, or ‘near misses’. The latter are defined as medication errors with the potential for injury but in which no injury occurred due to either patient circumstance or an intervention [83, 84]. That is, a random break in the chain of events leading up to a potential adverse event that has prevented injury, damage, illness or harm, but the potential for harm was nonetheless very near [2].

Classification of Actual Harm

Classifying actual harm that was caused by a medication error typically involves identifying the severity of patient harm caused by the error that has been observed or documented to have occurred. Across studies, actual harm has been referred to as either the ‘medication error outcome’ [85, 86], ‘patient outcome’ [87], ‘patient sequelae’ [88], ‘error consequence’ [89], ‘resulting harm’ [90], ‘related harm’ [89], ‘adverse drug event’ [44, 91, 92], ‘patient injury’ [45], and/or ‘error repercussion’ [93]. Notably, the ‘outcome’, ‘sequelae’ or ‘consequence’ across these studies referred to either patient harm [86, 88], the levels of care required to respond to the harm [87, 89], or both [85].

Review of Published Harm Classification Tools

Following systematic reviews of the literature on potential and actual harm [24], and dose errors [22], and an additional targeted non-systematic literature review (detailed in ESM 1), we identified almost fifty different tools for classifying the level of potential or actual harm associated with medication errors. Each of these tools and the associated guidelines for their use are detailed in ESM 2. With the exception of Dean and Barber’s 10-point tool [59], these tools classify harm into at least three levels, typically four or five levels, and up to seven levels. As detailed in ESM 1, the most frequently used labels of harm were, consecutively, ‘minor’, ‘moderate’, ‘serious’, ‘severe’, ‘life threatening’ and ‘death’.

Each level of harm is often accompanied by a list of situations that exemplify the level of harm severity. These situations are either examples of medication errors (such as an overdose), harms (such as a temporary low-grade pain), or the level of care required to respond to a harm (such as additional monitoring). The tools that provided examples of medication errors were typically published prior to 2000. For example, in order to guide researchers in the use of their harm classification tool published in 1987, Folli and colleagues suggest that the potential harm from a “less than four times overdose of a drug with low therapeutic index” be considered ‘significant’, while the potential harm from a “more than 10 times overdose of a chemotherapy agent” be considered ‘lethal’ [94]. In studies providing examples of actual harm, two dimensions of harm are typically considered. That is, harm permanence (temporary versus permanent harm), and the extent of harm (variously described by rating distress or inconvenience, up to loss of function, quality of life or death and/or by signs of disability, toxicity, or disease). For example, Gokhul and colleagues [86] suggest that “rash or diarrhoea” is a ‘significant’ harm while “bleeding or an altered mental state” is ‘serious’. Many harm classification tools provide examples of the level of care required to respond to a harm experienced to exemplify levels of harm severity. These tools reference changes in treatment plans, prolonged length of hospital stay, and intensiveness of treatments (from monitoring to surgery). For example, the NCC-MERP distinguishes between errors that result in harm that required intervention or prolonged hospitalisation, from those that required interventions necessary to sustain life [57].

Limitations of Harm Classification Tools

Each of the identified tools are designed to assess either potential or actual harm. The difference can be discerned from the language used, where potential harm tools refer to ‘likely’ outcomes while tools designed to assess actual harm refer to actual consequences. Despite this difference, many researchers employ tools that were designed to assess actual harm to assess potential harm and vice versa. For example, the frequently cited NCC-MERP tool for classifying actual patient harm has been used to assess both potential harm [61, 62, 95] and actual harm [85, 87, 90, 92, 96–102], and has also been used to ‘categorise errors’ without explicitly identifying whether potential or actual harm was assessed [103-107]. As such, comparisons between study results using the same harm classification tool are muddied by inconsistencies in either the use of the tools, or a lack of information about whether actual or potential harm was assessed.

The utility of providing examples of medication errors, harms or treatments as a means of ensuring consistent tool use may be limited. This is because not all clinicians may agree with the level of harm assigned to an example and instead perceive that the example typifies a less or more ‘severe’ harm. As a result, inconsistent clinician ratings are a frequent occurrence and highlights the need for processes to achieve reviewer consensus [42].

Finally, the use of these tools is limited by the availability of information on patient harm and the corresponding treatments documented in patient records [40]. That is, using a tool that classifies harm based on the treatments administered to a patient requires all such actions to be documented to ensure an accurate classification of harm. Importantly, the occurrence of these actions relies on the detection of the medication error during the admission. Previous studies have shown low levels of detection and high rates of under-reporting of medication incidents by hospital staff [108-110]. Minor to moderate consequences from medication errors are also less likely to be detected and therefore not monitored or documented [110]. Thus, unless the panel assessing harm accounts for whether the medication error was detected, assessment of actual harm will be biased towards non-minor harms. Finally, assessing actual harm requires information on the individual patient and the treatment received to be documented, and the poor quality of this information is frequently described to be a significant limitation [26–38, 111].

Proposed Harm Assessment Tools for Classifying Potential and Actual Harm Severity Related to Medication Errors

Our proposed harm assessment tools were designed using a systematic approach (detailed in ESM 1). As a first step, we selected the nomenclature used to label and define the levels of harm severity that is most common among existing tools, but that did not include examples of error, harm or treatments that could be misinterpreted. Next, we assessed the guidance associated with existing tools and chose examples of harms and corresponding treatments that were the least open to misclassification. Finally, we used the NCC-MERP [57] as the framework for an iterative revision process based on the outcomes of several workshops and harm assessment panels. These panels assessed real examples of medication errors as part of our ongoing randomised controlled trial of the effectiveness of an electronic medication management system introduced to reduce medication errors and medication-related harm in two paediatric hospitals [50].

Following the work of Morimoto and colleagues [16], we recommend that the panel using our proposed tool comprise at least three members from multiple professions (usually pharmacists, nurses and medical clinicians) with majority consensus reached via discussion led by a panel chairperson.

The Harm Associated with Medication Error Classification (HAMEC)

Prior to classifying harm severity with the Harm Associated with Medication Error Classification (HAMEC) tool, the process of confirming that a medication error occurred (sometimes referred to as determining error preventability), and that harm was at least possibly associated with the error (referred to as the causality assessment), is expected to have been completed. When classifying the severity of potential harm associated with a medication error using the HAMEC (see Table 1), the panel is instructed to assume that the error was undetected and reached the patient. The panel does not necessarily need to consider the specific clinical context of the individual patient involved. That is, the panel is asked “regardless of patient circumstance, what level of harm would this medication error be expected to cause to a patient”? If the clinical status of the patient is known to the panel, this information can be used to inform the classification and the panel is asked “what level of harm would this medication error be expected to cause this patient”. In contrast, use of the HAMEC to classify actual harm severity (see Table 2) assumes that a medication error occurred, reached the patient, and harm was identified. When classifying the degree of actual harm, the harm assessment panel must take into consideration the patient’s age, comorbidities and overall condition.

Table 1

The Harm Associated with Medication Errors Classification (HAMEC): potential harm

Level	Reference	Description
0	No harm	No potential for patient harm, nor any change in patient monitoring, level or length of care required
1	Minor	There was potential for minor, non-life threatening, temporary harm that may or may not require efforts to assess for a change in a patient’s condition such as monitoringa. These efforts may or may not have potentially caused minimal increase in length of care (< 1 day)
2	Moderate	There was potential for minor, non-life threatening, temporary harm that would require efforts to assess for a change in a patient’s condition such as monitoringa, and additional low-level change in a patient’s level of careb such as a blood test. Any potential increase in the length of care is likely to be minimal (< 1 day)
3	Serious	There was potential for major, non-life threatening, temporary harm, or minor permanent harmc that would require a high level of careb such as the administration of an antidote. An increase in the length of care of ≥ 1 day is expected
4	Severe	There was potential for life-threatening or mortal harm, or major permanent harmc that would require a high level of careb such as the administration of an antidote or transfer to intensive care. A substantial increase in the length of care of > 1 day is expected

aMonitoring refers to the minimally intrusive observation of the patient’s condition over time. Observations are typically made for urine output, general level of consciousness, or vital signs including heart or breathing rate

bLevel of care refers to the degree of active treatments that are initiated in response to actual or potential change in the patient’s condition

cPermanent harm is such that, as a consequence of the drug event, the patient would require ongoing care, or experience an ongoing disability, beyond the index admission

Table 2

The Harm Associated with Medication Errors Classification (HAMEC): actual harm

Level	Reference	Description
0	No harm	There was no actual patient harm, nor any change in patient monitoring, level or length of care required
1	Minor	There was actual minor, non-life threatening, temporary harm that may or may not have required efforts to assess for a change in a patient’s condition such as monitoringa. These efforts may or may not have actually caused minimal increase in length of care (< 1 day)
2	Moderate	There was actual minor, non-life threatening, temporary harm that did require efforts to assess for a change in a patient’s condition such as monitoringa, and additional low-level change in a patient’s level of careb such as a blood test. Any actual increase in the length of care was minimal (< 1 day)
3	Serious	There was actual major, non-life threatening, temporary harm, or minor permanent harmc that required a high level of careb such as the administration of an antidote. An increase in the length of care of ≥ 1 day occurred
4	Severe	There was actual life-threatening or mortal harm, or major permanent harmc that required a high level of careb such as the administration of an antidote or transfer to intensive care. A substantial increase in the length of care of > 1 day occurred

aMonitoring refers to the minimally intrusive observation of the patient’s condition over time. Observations are typically made for urine output, general level of consciousness, or vital signs including heart or breathing rate

bLevel of care refers to the degree of active treatments that are initiated in response to actual or potential change in the patient’s condition

cPermanent harm is such that, as a consequence of the drug event, the patient required ongoing care, or experiences an ongoing disability, beyond the index admission

Conclusion

To further our understanding of the impact of medication errors on patient safety, we need to go beyond just measuring the frequency of errors and assess the severity of harm associated with those errors. However, to date the process of classifying this harm has been fraught with inconsistencies and lacks standardisation. Indeed, the frequent use of the term ‘error severity’ invites confusion as, without further clarification, it remains unclear if potential or actual patient harm was assessed. We propose that the term ‘error severity’, in the absence of specifying what outcome is being assessed, is misleading and suggest future research instead refer to the classification of potential or actual harm associated with a medication error. Following the synthesis of the defining elements from existing tools for classifying harm, we have developed the HAMEC, a tool that allows for the classification of harm across clinical and research settings. While the HAMEC offers a more structured framework for further discussion, validation is required.

Below is the link to the electronic supplementary material.

Supplementary material 1 (DOCX 37 kb)

Supplementary material 2 (DOCX 61 kb)

There is great diversity in the definitions of, and methods used for classifying, medication-related harm in hospitals.

Comparisons of the severity of medication-related harm across studies and clinical settings are limited by inconsistency in the tools used for the classification of medication-related harm.

We propose the Harm Associated with Medication Error Classification (HAMEC) tool, a new tool derived from the common nomenclature used to label and define levels of medication-related harm.

HAMEC includes clear definitions and does not include examples of error, harm or treatments, which are prone to misinterpretation.