Medication Errors and Patient Safety: Evaluation of Physicians' Responses to Medication-Related Alert Overrides in Clinical Decision Support Systems.

BACKGROUND: Clinical decision support systems (CDSS) can enhance patient safety and reduce medication errors by giving physicians alerts while dispensing medications. Physicians inappropriately override these alerts for various reasons, which can possibly lead to medication errors and impact patient safety.
OBJECTIVE: To assess the appropriateness of overridden major medication-related alerts, to investigate the reasons behind inappropriate overriding, and to evaluate if medication errors occur in inappropriately overridden alerts.
METHODS: A mixed-methods study was conducted.. Quantitative: Retrospective observation to evaluate the appropriateness of major drug-dose related alert overrides. A simple random sample was taken from appropriate and inappropriate overrides and reviewed for medication errors. Qualitative: Semi-Structured Interviews were conducted with ten consultant physicians from various specialties. Interviews were transcribed and coded inductively then analyzed using Thematic Content Analysis.
RESULTS: Out of 1087 alert overrides that were evaluated for appropriateness, 738 were inappropriately overridden (67.89%). In a sample of 283 inappropriate and 92 appropriate overrides; the resulted medication errors were 7 and 0, respectively. Qualitative analysis resulted in three emergent themes; Judgement, Experience & Guidelines, CDSS Issues & Limitations, Physician Behavior & Safety.
CONCLUSION: The majority of alerts were found to be inappropriately overridden. This can be attributed to physician reliance on their clinical knowledge and medication databases, having the pharmacists' checks, and alert fatigue. CDSS alerts can be improved by making them more prominent and suppressing or descaling unnecessary alerts. The drop-down justification list can be enhanced by adding free text options and relating recommended dosing to disease or specialty.

BACKGROUND

When medications are administered in an inappropriate way, they can cause significant harm to the patients seeking medical help (1). Despite the improvements in medical practice, medication errors are common and tend to occur all over the world, with an estimated error rate ranging between 5% to 20% depending on various healthcare facilities (1). Medication errors affect patient safety and the healthcare system in several ways, as it has been linked to an increase in mortality and morbidity rates (1). Furthermore, it has been documented that medication errors also carry an economic burden (2).

Whenever medication errors cause harm to the patient, it is termed as an adverse drug event (ADE) (1,2). Research conducted in the United States indicates that an estimated total of 770,000 cases of patient injury and death have been recorded in hospitals annually due to medication errors and ADE’s (2, 3). ADE’s also have a hospital incidence rate with a range of 2 to 7 incidences per 100 admissions, with an approximate percentage of 28% of these ADE’s being preventable because of their association with medication errors (3). Studies also show that 56% of preventable ADE’s occur during drug ordering (3). Therefore, it is of great importance to improve medical safety by taking preventative measures to minimize medication errors. It has been reported that clinical decision support systems (CDSS) could potentially prevent medication errors (1,4-5). CDSS are valuable tools that can improve patient safety and enhance decision making in clinical settings by assisting physicians when they prescribe medications (4,6). The knowledge-based system automatically checks if the medication order is safe and warns the physician if the prescribed medication is not compliant with guidelines (7). A study shows that these systems help in reducing medication errors, which are noted to occur in 4% to 6% of drug orders, by a percentage of 81% (8-9). The use of CDSS has been reported to subsequently reduce preventable adverse drug events, which are reported to be the single leading cause of preventable iatrogenic injury (8, 10). CDSS reduce errors by providing the physician with an alert if the prescribed medication has the dangerous potential of causing an allergic reaction to the patient. They can also alert physicians when a medication is at risk of interacting negatively with another prescribed drug, or if it is over the maximum recommended dosage (11).

Physicians are sometimes noncompliant to the CDSS warnings. Studies show that physicians override 50% to >90% of CDSS alerts for admitted patients, with the range varying based on the healthcare facility (1,4-5). One study shows that the most important reason for overriding may be due to the overload of alerts caused by a low threshold of alerting. This makes the alerts inconvenient sometimes and, to some extent, bothersome leading physicians to experience alert fatigue (also known as pop-up fatigue) (4-5). Alert fatigue is a term that describes a state of paying less attention to alerts in general (1). Another study shows that clinicians average 49 minutes processing an average of 56 alerts received per day, making CDSS alerts a weighty component of the physicians’ daily workflow (8). With that, too many alerts that are clinically unimportant are time consuming and mentally exhausting, which might lead physicians to override the clinically important ones (4). Other reasons could be a clinical judgment made by the physician based on their scientific knowledge, as the physician might justify the override of an alert due to it being clinically irrelevant, or by following different guidelines (12). The physician might have previous knowledge that a specific medication dosage is tolerable by the patient, so an override is executed to commence the treatment (12). Moreover, the physician could simply override the alert with an intention to monitor the patient (8, 12). With the various presented causes in mind, even though the CDSS is designed to improve patient safety, research shows that only 62% of drug-drug interaction and 88% of drug dose overridden alerts were considered appropriate (1,13). This leads to the realization that there is a percentage of inappropriate overriding and sometimes vital alerts that can be missed, increasing the risk of medication errors negatively impacting patient safety (5, 12, 14). It is reported that ADE’s increase from 1.3% in appropriate overriding of alerts to 5% in inappropriate overriding (13).

The studies on the appropriateness of overridden alerts in the Saudi context are limited. It is therefore important to focus this work on the region, since little attempt has been made in Saudi Arabia to study the incidence of ADEs for inpatients (2, 15, 16). Assessing the validity of these overridden alerts by physicians and finding a way to make medication alerts more practical by improving its sensitivity, and specificity to avoid alert fatigue are important. Investigating the reasons behind the overriding of medication-related high-severity alerts, whether they are related to the physician’s behavior or technical insufficiency from the CDSS systems are of great importance in the medical field in order to improve medication safety and possibly avoiding serious health implications of preventable adverse drug events.

OBJECTIVE

This study aims to assess the appropriateness of overridden major medication-related alerts by physicians in the CDSS. It also investigates if medication errors are related to inappropriate overriding of major alerts. Last, the study evaluates the behavioral reasons behind physicians’ responses to these medication alerts.

MATERIAL AND METHODS

Study site

The study included data from King Abdulaziz Medical City (KAMC) electronic medical records system, known as “BESTCare 2.0A”. This system provides CDSS tools to deliver proven solutions to medication-related issues. Seven alert categories were provided via CDSS; drug–drug interactions, drug-dose screening, drug allergy alerts, drug related disease screening, age contraindication, duplicate drugs and route contraindication. These alerts’ logic were sourced from Medi-Span (Master Drug Database (MDDB), Wolters Kluwer Health, Inc., Conshohocken, PA). The CDSS provides alerts for four levels of severity; Level 1 indicates major severity alerts with an override reason requirement. Level 2 indicates moderate severity alerts as the reason might have been an available option to override the alerts. Level 3 is for minor severity alerts as an override reason was optional as well. Lastly, Level 4 is for informational alerts, which involves informational alerts and function as an indicator only with no requirement to provide a reason for overriding the system. A color-coding system was also applied to the four levels in their respective order, which are red, orange, yellow and grey to facilitate the identification.

The prescribing physician is required to choose a rationale to override an alert from a drop-down menu known as system-coded reasons. Those choices are; “benefit outweighs risk”, “dose checked and confirmed”, “allergy not proven”, “not clinically significant”, “compatibility confirmed”, “dose altered for patient characteristics, patient is being monitored” and “side effect not allergy”.

Methodology

After receiving IRB approval from King Abdullah International Medical Research Center in July 2019, we proceeded to conduct a mixed-methods study. A retrospective chart review was conducted on all inpatient overridden major medication-related drug-dose alerts shown on CDSS in KAMC-Jeddah from January to May 2019, and the prescribing physicians’ chosen response to override the alert. The appropriateness of overridden major drug-dose related alerts provided by the Pharmacy Department, was evaluated using criteria guidelines based on previous studies (5), a guidelines matrix, and a multi-disciplinary group, then categorized to “appropriate” and “inappropriate” overrides.

A random sample was selected from the inappropriate overrides and was evaluated for any preventable medication errors. Patient chart reviews of individuals that fell under the “inappropriate override” category were evaluated to identify if there were preventable medication errors. The period of evaluation started after the inappropriate override and continued for the time that the medication(s) remained active in the patient’s medication orders, which could have persisted to hospital discharge in some cases. The evaluation included data relevant to medication errors such as lab-tests, medication orders and patients’ progress notes documented by providers. Statistical Analysis was performed using SPSS IBM version 25. A p-value of 0.05 was used. Descriptive statistics were applied using frequencies and percentages for categorical data, while inferential statistics were analyzed using Chi-square. In the qualitative part of this study semi-structured interviews were conducted in July of 2019 with ten Consultant Physicians from various specialties who have overridden major medication-related alerts. Participants were recruited using purposive and snowball sampling, guided by theoretical saturation. Interviews were audio recorded with permission; totaling 190 minutes. Recordings were transcribed verbatim, yielding 20,635 transcribed words. Transcripts were reviewed by a second author for accuracy then analyzed using Thematic Content Analysis assisted by NVivo12.

Appropriateness criteria

The criteria for appropriateness of the overridden drug-dose related alerts was developed from validated frameworks that were gathered from previous studies and modified for this study. An overridden alert was deemed appropriate if the reason that was chosen by the physician to override was acceptable according to the criteria and clinical context. For example, if the provider chose “benefit outweigh risk” or “dose altered for patient characteristics, the patient is being monitored” after receiving the “exceeds the maximum single dose” it is considered possibly appropriate, but confirmed as being appropriate only after the patient’s record was put forward for detailed chart review, including notes from physicians and previously tolerated drug doses to determine clinical context, and assessed for whether the required action such as “-, the patient is being monitored” has been carried out. If all are true, then the override is appropriate, if not then it is inappropriate. If the provider chose “Allergy not proven”, “Side effect not allergy”, or “Compatibility confirm” for the same prompt then the override is automatically considered inappropriate without looking for clinical context, as the reasoning irrelevant to the given alert.

RESULTS

random sample of 1087 alerts from a total of 4426 drug-dose related alerts overridden by physicians was obtained and evaluated for appropriateness. Of which, 32.1% (n=349) of these overridden alerts were considered appropriate, as shown in Figure 1. The most frequently given reason for the overrides was “Benefit outweighs risk” (n=702, 64.6%) with 71.9% of them being inappropriate; followed by “Dose checked and confirmed” (n=129, 11.9%) with only 3.1% of them being inappropriate. “Allergy not proven” was the third most common (n=110, 10.1%) and all of them were inappropriate (100%). Figure 2 demonstrates the number overridden alerts by each physician’s level and the percentage of appropriateness. There is no significant relation found between the physicians, regardless of their level, and the likelihood of inappropriate overrides occurring (p=0.1290). Conversely, there appears to be an association between different age groups of patients and the appropriateness of overridden alerts (p=0.0007), where the Infants and Neonates category has the highest inappropriate overrides rate (80.6%), as shown in Figure 3. The findings also show that there is a significant relationship between the type of medication and the appropriateness of the override (p<0.0001). Enoxaparin shows the highest appropriate overrides rate (84.5%) in contrast to Acetaminophen which showed the lowest rate (11.1%). The rest of the most commonly overridden medications and their appropriateness are demonstrated in Table 1. When reviewing the inappropriately overridden alerts for medication errors, 7 were found. On the other hand, no medication errors were found resulting from appropriately overridden alerts. Qualitative analysis resulted in 9 categories of findings emerged and regrouped under 3 broader themes, as shown in Table 2.

Figure 1.

Sampling Technique and Findings

Figure 2.

Appropriateness by Physician Position

Figure 3.

Appropriateness by Age Group

Table 1.

Appropriateness by Medication

Medication	Inappropriate Overrides	Appropriate Overrides
Acetaminophen (n=100)	89 (89%)	11 (11%)
Augmentin (n=49)	40 (81.6%)	9 (18.4%)
Enoxaparin (n=58)	9 (15.5%)	49 (84.5%)
Hydromorphone (n=28)	21 (75%)	7 (25%)
Insulin (n=71)	60 (84.5%)	11 (15.5%)
Morphine Sulfate (n=67)	54 (80.6%)	13 (19.4%)
Oxytocin (n=124)	46 (37.4)	77 (62.6%)
Perindopril (n=58)	50 (86.2)	8 (13.8%)

Table 2.

Qualitative Themes and Sub-categories

THEME	JUDGEMENT, EXPERIENCE & GUIDELINES	REFERENCES	CDSS ISSUES & LIMITATIONS	REFERENCES	PHYSICIAN BEHAVIOR & SAFETY	REFERENCES
REFERENCES	127		113		88
SUB-CATEGORY	LEVEL OF EXPERIENCE	61	SYSTEM & ALERTS	44	PHARMACIST SAFETY NET	22
QUOTE	“I would say maybe inexperienced people with ordering and they’re just overriding. Like a junior, I mean. A senior would probably know or think more about it”.		“Four? But where are the colors? Where do they appear? The colors don’t come up for me. I’ve never paid attention to the colors”.		“We have clinical pharmacists, they’re very helpful. We put our problems to them most of the time or work together and they help and fix it for us, they are very helpful”.
SUB-CATEGORY	DRUG INFORMATION DATABASES	15	OVERRIDE JUSTIFICATION	50	ALERTS & SAFETY	39
QUOTE	“I rely on my knowledge and experience and if I have doubt I check Medscape”.		“I think this is one of the major issues. We should have a free text for clinical decisions”.		“I think it should always be there, at least the juniors can learn what they're doing and get the appropriate alerts. So, I think I still want it to be in the system”.
SUB-CATEGORY	MEDICAL SPECIALTY	39	DOSING ISSUES	19	ALERT FATIGUE	27
QUOTE	“Major cause of this difference is our specialty group, we are taking care of extreme pre-term infants sometimes which are neonates with a birth weight of less than 1.5kg. In the system they put mainly the adult and pediatric doses”.		“The usual dose in the standard of care that we are using in the adult or pediatric; the system thinks it is not right, so that’s why we say ‘benefit outweighs risk’ ”.		“I just override any alert that comes in without seeing it, and I’ll be very frank with this; so sometimes it gives you two to three alerts almost for each and every patient, and some of these alerts are not related to my discipline… find ways to get rid of the alert fatigue!”

DISCUSSION

When examining physician overrides, we found that physicians had a high level of inappropriate overrides as the appropriateness rate was 32.1% for inpatient drug-dose alert overrides in the CDSS. This percentage is low in comparison to other studies, some of them reporting an overall appropriate drug-dose alert override rate as high as 88.8% (12-13). Although many of the CDSS alert override evaluation studies were conducted in specific settings like the ICU, their appropriateness rate is still much higher than what we found (12, 13). This poses a safety concern since other studies suggest that the rate of adverse drug events is four times more likely to occur with inappropriate overrides (12, 13). This indicates the critical need to improve patient safety by improving the alert system. The high rate of inappropriate overrides observed may be due to several factors, including the reliance of physicians on clinical pharmacists to double check their prescriptions; “as per the system here, even if the overridden alert is inappropriate it should go to the pharmacist, that’s a very good way to prevent the inappropriate or dangerous actions from happening”. It could also be due to the quantity and low threshold for these alerts to pop-up, causing the prescribing physician to experience “alert fatigue”; “I just override any alert that comes in without seeing it, and I’ll be very frank with this; so sometimes it gives you two to three alerts almost for each and every patient”, which is a commonly reported issue ( Furthermore, when physicians choose from the 7 reasons to justify their override of a drug-dose alert, we found out that the most common chosen reason was ‘Benefit outweighs risk’ which had a high inappropriate rate (71.9%). When asked about this, physicians commented; “usually I just read what it is and I go ‘benefit outweighs risk’”, “usually I choose benefit outweighs risk, that’s the usual” and “most physicians will use the ‘benefit outweighs risk’”. This could be due to physicians preferring to depend on their clinical judgement when prescribing medications, the fact that the other justification options, such as “Allergy not proven” or “side effect not allergy” are not applicable to most cases, or the lack of a free text option to write a more specific justification. It is also important to note that the sentence “Benefit outweighs risk” has a broad meaning, which might make it reasonable for physicians to choose as a justification to override every type of alert.

Another commonly given override justification was ‘Allergy not proven’ which was observed as not having any relation with the alert that they received (drug-dose). What is worthy to note is that this is the first option in the drop-down list, and might be spontaneously chosen because of ‘alert fatigue’, as one physician explains; “most of them I override them and usually I put the first response in the drop list to justify”.

The interviewees also suggested providing alternative more accurate reasons (relevant to the alert itself) to choose from, or providing a free text space; “we need to have a free text that may improve the way physicians insist to use the same medication despite the system alert”, “the clinical decision is much superior and important that is why the free text for the reasons should be added”, “We should have a free text for clinical decision”.

Many of the CDSS major red alerts are considered inaccurate warnings, as one physician explains; “in paediatrics we often override dosing because of the other references that recommend different doses than whatever is programmed in the system”. Furthermore, he added that “for MS we give a huge amount of steroids if there is an attack of the disease, which is almost 10 times the usual dose. The system gives you an alert immediately, however the system should be smart enough to know that this is an MS patient and this dose is for that particular disease”. In our study we also found that the infants and neonates patients population had the highest rate of inappropriate overriding (80.6%). This could be due to the systems strict limitations in dosing guidelines, which in practice can differ due to specific patient factors, including the patient’s age group and particular diseases which require a higher medication dosage. This is also echoed in a systematic review about medication perscribing (17). The consultants generally did not trust junior physicians/residents to override major alerts, because of their lack of experience and voiced their concerns saying; “do not accept overrides from juniors“, “a senior would probably know or think more about it. It's always the responsibility of the senior in the clinic to make sure that these alerts have not been overridden by the juniors”. While from our sample we found no significant relation between the level of the physician and their inappropriate overrides, and other studies have come to reach similar findings (7, 17).

CONCLUSION

The majority of major medication-related alerts were found to be inappropriately overridden by physicians. This can be attributed to physician reliance on their clinical knowledge and drug medication databases, and having the pharmacist as a safety net. Physicians also suffered from alert fatigue which affected their responses to the alerts. The physicians view the CDSS to be a valuable medication dispensing safety tool that can reduce medication errors. However, the alerts in the CDSS can be improved by making them more prominent and suppressing or descaling unnecessary alerts. The drop down justification list can also be enhanced by adding a free text option, and by relating recommended dosing in alerts to disease or specialty. It is also essential to highlight the wider implications of this work. Inappropriately overriding medication alerts could impact patient safety. Any measures to study the systems that provide the alerts and the behaviors of those using them are important. What is also worthy to note, is that although many of the factors and actions reported here are similar to those described in other studies, there is still not much reported in the local context of Saudi Arabia, and further studies in this area are warranted.