Signal and noise: applying a laboratory trigger tool to identify adverse drug events among primary care patients.

BACKGROUND: The extent of outpatient adverse drug events (ADEs) remains unclear. Trigger tools are used as a screening method to identify care episodes that may be ADEs, but their value in a population with high chronic-illness burden remains unclear.
METHODS: The authors used six abnormal laboratory triggers for detecting ADEs among adults in outpatient care. Eligible patients were included if they were >18 years, sought primary or urgent care between November 2008 and November 2009 and were prescribed at least one medication. The authors then used the clinical / administrative database to identity patients with these triggers. Two physicians conducted in-depth chart review of any medical records with identified triggers.
RESULTS: The authors reviewed 1342 triggers representing 622 unique episodes among 516 patients. The trigger tool identified 91 (15%) ADEs. Of the 91 ADEs included in the analysis, 49 (54%) occurred during medication monitoring, 41 (45%) during patient self-administration, and one could not be determined. 96% of abnormal international normalised ratio triggers were ADEs, followed by 12% of abnormal blood urea nitrogen triggers, 9% of abnormal alanine aminotransferase triggers, 8% of abnormal serum creatinine triggers and 3% of aspartate aminotransferase triggers.
CONCLUSIONS: The findings imply that other tools such as text triggers or more complex automated screening rules, which combine data hierarchically are needed to effectively screen for ADEs in chronically ill adults seen in primary care.

Introduction

An adverse drug event (ADE) is defined by the Institute of Medicine as ‘an injury resulting from medical intervention related to a drug’.1 It is estimated that 2.4 of every 1000 emergency department visits per year are attributed to ADEs, with approximately 0.32% of hospitalised patients incurring fatal ADEs.2–6 Recent research estimates between 14.9 and 50.1 ADEs occur per 1000 person months.7 8 Approximately 20% of ADEs identified are thought to be preventable and a significant proportion are serious, life-threatening or fatal.9 Unfortunately, even though most medical care occurs in the outpatient setting, the incidence of ADEs among ambulatory patients is unknown.

A recent study estimated that approximately 4.5 million ambulatory visits per year in USA are related to ADEs, making this a significant public health concern.10 However, the full burden of outpatient ADEs remains unclear, because of our inability to detect and monitor these events, especially those that occur when patients are at home between visits. ADEs are normally detected by voluntary reporting and tracking of errors. It is estimated that only 10–20% of all errors are reported.11 In-depth chart review captures significantly more ADEs than voluntary reporting, but this is time consuming and expensive.7 9

An ADE trigger tool may make chart review more efficient.12 Triggers allow for targeted chart review by identifying possible ADEs via abnormal laboratory values, text phrases, or automated ‘rules’ combining multiple data elements available in medical records. Charts identified to have a specific trigger can then be reviewed for ADEs. Trigger tools have been studied in their ability to detect ADEs and research has shown that computerised and non-computerised trigger tools are more time-effective than complete chart review and more sensitive than voluntary reporting.13 14

Most research on trigger tools used to identify ADEs has focused on hospitalised patients.15–17 In one of the first studies of a trigger tool in the outpatient setting, Singh et al created a 36-item trigger tool and investigated its efficacy in detecting ADEs among older patients seen in primary care settings. Analysis revealed that six laboratory values and three text triggers had high positive predictive values for identifying ADEs. These nine items accounted for 94.4% of ADEs detected.18 Because this trigger tool seems promising, we applied it in a single, publicly funded primary care clinic with an ethnically diverse, largely chronically ill population. We sought to determine whether using the six laboratory values as triggers would (1) identify ADEs (2) identify stage of medication use and (3) characterise ADEs detected with respect to preventability and severity. The goal of the study was to determine whether this trigger approach could efficiently identify ADEs, in order to facilitate the clinicians' ability to intervene in incipient ADEs prior to patient harm. We wanted to evaluate whether the triggers would identify ADEs in order to determine whether real-time screening of our laboratory data would be an efficient way of intervening in outpatient ADEs.

Methods

Study design and patients

We conducted a retrospective electronic medical review to validate the use of a six-item trigger tool to detect the occurrence of ADEs. A computer-based search was completed to identify patient charts that had at least one of the six laboratory value triggers (international normalised ratio (INR) >5, serum creatinine (SCr) >2.5, blood urea nitrogen (BUN) >60, alanine aminotransferase (ALT) >84, aspartate aminotransferase (AST) >80, thyroid-stimulating hormone (TSH) undetectable while on levothyroxine).18 The laboratory triggers were chosen because it was feasible to extract the visit data associated with particular laboratory values. We omitted the three text triggers in the study by Singh et al,18 because it was not feasible to perform automated text searches of electronic visit notes using the current electronic health record. We searched the electronic medical record, which is used within the city-and-county-wide, publicly funded healthcare system, including all outpatient clinics, urgent care centre, dialysis unit, acute rehabilitation, emergency department and hospital, for the above laboratory values that occurred at any location during the study period from November 2008 to November 2009.

The study evaluated patients of a general internal medicine clinic that operates within an integrated healthcare system of the San Francisco Department of Public Health. Eligible patients were included if they were greater than 18 years old, were prescribed at least one medication, and had at least one laboratory value trigger. As stated above, laboratory values taken at various sites within the healthcare system were identified; however, only patients established in the general internal medicine clinic were included in the study. To determine whether a patient received care from the general internal medicine clinic, patients were included if they were evaluated in this clinic at any time within the study period or within 90 days of the study period. Patients were excluded if their only visit was in the vaccination clinic or in the women's clinic for a routine pap smear. Patients were also excluded if their only visit was with a nurse.

The study protocol was evaluated and approved by the Committee on Human Research at the University of California, San Francisco.

Event detection and classification

Once the above six laboratory values were identified, as a ‘trigger,’ we found that often there were repeat laboratory values on subsequent days as a patient was being monitored by a healthcare provider. We determined that the same abnormal laboratory value identified for the same patient within a 30-day period was classified as one unique ‘episode.’

To determine whether the abnormal laboratory value episode resulted from an ADE, physicians reviewed electronic charts, as in prior studies of ADEs.19 A physician (SB) independently conducted an in-depth chart review for all charts found to have any of the six laboratory value triggers. A second physician (AD) conducted an in-depth chart review for a randomly selected 15% of the triggers. Differences were resolved by discussion. The two physicians reviewed all patient related documentation from 90 days before, 12 months during and 90 days after the study period. The physician-reviewers first determined the care setting in which the trigger occurred. If any laboratory trigger value occurred during the admission (as long as it was not related to an outpatient medication), it was excluded. For example, if the creatinine was normal on admission but increased during the admission, then that was excluded. In addition, if a laboratory value was taken in the emergency department and was related to their admission, then it was excluded. For example, if someone had an elevated serum creatinine on admission but this resulted from acute renal failure due to sepsis, his/her case was excluded. The physicians first determined whether or not an ADE occurred. For each ADE identified, the physician then determined the stage of the medication process where the event occurred, the severity of the effect on the patient and the preventability of the ADE.

For all ADEs identified, the stage of medication use was classified as prescribing, dispensing, monitoring and patient self-administration. The stage was defined as the point of medication use where the ADE occurred, regardless of whether it was preventable.20 We classified the stage as ‘patient self-administration,’ only when there was clear evidence from the medical chart that the patient took the medication incorrectly. The severity of the effect was divided into no harm, minimal harm (no change in symptoms but have abnormal laboratory results), mild harm (mild reversible complications not requiring hospitalisation or symptoms lasting less than one day), moderate harm (reversible complications not requiring hospitalisation or prolonged symptoms lasting more than one day) and severe harm (irreversible complications or requiring hospitalisation, permanent disability or death).21 Finally, the preventability of the ADE was grouped based on established definitions of preventability from prior patient safety studies.22 The preventability of the ADE is classified as follows: (1) ameliorable—severity could have been substantially reduced with different actions or procedures (on the part of the patient, provider or system), (2) preventable—could have been avoided as probably a result of an error or system design flaw or could have been avoided with a patient action that is, reasonable to expect, or (3) nonpreventable.21

Finally, we reviewed the medical centre's adverse reporting system, known as the unusual occurrence reporting system, during the study period and found that none of the ADEs identified were voluntarily reported.

Results

Our study included 583 patients. The mean patient age was 55 years with a SD of 14; 64% were men and 70% were English-speaking (table 1). We identified 2662 laboratory values that met one of the following criteria: INR>5, SCr >2.5, BUN >60, ALT >84, AST >80 or TSH undetectable while on levothyroxine. We excluded 1322 laboratory values that were drawn in the hospital setting and related to that hospital admission. There were 1342 triggers that represented 622 unique episodes among 516 patients (figure 1). Agreement between physician-reviewers on ADE occurrence was 94%.

Table 1

Patient characteristics

	N (SD or %)
Mean age	55 (14)
Male N=576	369 (64)
Language N=524
English	366 (70)
Spanish	96 (18)
Cantonese	29 (6)
Other	33 (6)
Race N=575
White	126 (22)
Black	158 (27)
Asian	102 (18)
Hispanic	174 (30)
Other	15 (3)

Demographic data not available for all patients.

Figure 1

Flow chart of adverse drug events (ADEs).

From the 622 unique episodes, the trigger tool identified 91 (15%) as ADEs. We evaluated each laboratory trigger separately to determine each trigger's yield in identifying ADEs. Table 2 describes the 622 laboratory triggers and 91 ADEs by individual laboratory trigger. For each laboratory trigger, the percentage of triggers that identified an ADE is also listed. This table highlights that each trigger varied in its yield to predict an ADE. For example, approximately 96% of triggers for INR>5 were associated with an ADE compared with 3% of triggers for AST >80 detecting an ADE. Of note, the laboratory trigger of undetectable TSH while on levothyroxine did not occur in our sample.

Table 2

Adverse drug events (ADEs)

	Triggers (n)	Unique episodes (n)	ADEs (n)	Proportion triggers that were ADEs (95% CI)	Stage	Effect on patient
INR >5	85	48	46	0.96 (0.90 to 1)	0 Prescribing 0 Dispensing 0 Self-administration 46 Monitoring	0 None 35 Minimal 8 Mild 3 Moderate 0 Severe 0 NA
BUN >60 mg/dl	618	103	12	0.12 (0.06 to 0.18)	0 Prescribing 0 Dispensing 12 Self-administration 0 Monitoring	0 None 7 Minimal 4 Mild 0 Moderate 1 Severe 0 NA
ALT >84 lU/l	644	205	19	0.09 (0.07 to 0.11)	0 Prescribing 0 Dispensing 16 Self-administration 2 Monitoring 1 NA	0 None 16 Minimal 0 Mild 1 Moderate 1 Severe 1 NA
Creatinine >2.5 mg/dl	123	862	10	0.08 (0.06 to 0.10)	0 Prescribing 0 Dispensing 10 Self Administration 0 Monitoring	0 None 4 Minimal 5 Mild 0 Moderate 1 Severe 0 NA
AST >80 IU/I	453	143	4	0.03 (0.00 to 0.06)	0 Prescribing 0 Dispensing 3 Self-administration 1 Monitoring	0 None 3 Minimal 0 Mild 1 Moderate 0 Severe 0 NA
Total	1342	622	91	0.15 (0.12 to 0.18)	0 Prescribing 0 Dispensing 41 Self-administration 49 Monitoring 1 NA	0 None 65 Minimal 17 Mild 5 Moderate 3 Severe 1 NA

ALT, alanine aminotransferase; AST, aspartate aminotransferase; BUN, blood urea nitrogen; INR, international normalised ratio.

We also evaluated the medication stage and degree of harm when the ADE occurred. Of the 91 ADEs included in our analysis, 49 (54%) occurred during medication monitoring, 41 (45%) during patient self-administration, and one could not be determined. Of the 91 ADEs, 65 (71%) caused minimal harm, 17 (19%) caused mild harm, 5 (5%) caused moderate harm, 3 (3%) caused severe harm and one could not be determined. Examples of severe harm include gastrointestinal bleed requiring transfusion and haemothorax necessitating chest tube placement, with both ADEs leading to hospitalisation.

Finally, of the 91 ADEs, only 3 (3%) were deemed non-preventable. Otherwise, 4 (4%) were preventable and 84 (92%) were ameliorable. An example of a non-preventable ADE that occurred was hydralazine-induced lupus; this was an adverse reaction that could not have been predicted. An example of a preventable ADE was excessive dosing of furosemide leading to an increase in serum creatinine, which could have been avoided.

Discussion

We were able to successfully apply a laboratory-value trigger tool in an ambulatory care clinic and to achieve clinician agreement in ADE ascertainment using the chart review. We found a significant number of ADEs using a trigger tool. While the utility varied among the six laboratory value triggers, INR>5 efficiently identified ADEs.

Our determinations of preventability are consistent with prior literature showing that many ADEs are preventable.7 8 23 In this sample, only 3% of the triggers were felt to be non-preventable. In terms of clinical severity, the majority of ADEs identified by this trigger tool were in the mild-to-moderate-harm range. Consequently, there is an opportunity to intervene and prevent ADEs in the outpatient setting before significant harm occurs. We believe this underscores the importance of including ADE surveillance in outpatient safety initiatives.

Importantly, we found that most ADEs occurred during the self-management and monitoring stages of medication use, rather than being prescribing or dispensing errors. Therefore, interventions to prevent outpatient ADEs will need to be patient-centred and focus on safe medication self-management. Gaps in medication monitoring clearly led to ADEs, but there is scant evidence underlying current monitoring recommendations, and more study of ambulatory medication monitoring is needed.24 Because the number of medications is strongly associated with ADEs,10 25 with population ageing and increasing medication use we expect outpatient ADEs to become more frequent.

There were limitations to our study. We relied upon medical chart review to determine whether an ADE occurred. Similarly, because we relied on medical records, we could not pinpoint the cause of the ADE beyond the medication use stage. As an example, it is often unclear from chart documentation whether the error occurred because of a lack of monitoring or in the course of adhering to recommended medication monitoring. However, the use of a comprehensive electronic medical record mitigated some of the flaws and biases often associated with chart review studies. The patient population is from a single, ethnically diverse, safety net clinic setting and that is both a strength and weakness. It is a strength in that findings from this clinic are likely to have public health importance, but the population may be more ill and less comparable with the usual primary care population in USA.

While the INR >5 successfully identified ADEs, the other laboratory triggers had much lower yield than previously described.18 Several factors could contribute to this difference. First, our population has a significant prevalence of chronic kidney disease, as reflected in the above-threshold values for BUN and creatinine not attributable to ADEs. Similarly, there is a high prevalence of hepatitis C infection in this population that likely reduces the yield for ALT and AST trigger tools. Second, it is possible that the electronic medical record captures different information that the paper charts from the prior study. Third, because our patients are largely uninsured, virtually all of their laboratory tests and subspecialty care occurred within the integrated public healthcare delivery system. This may account for differences compared with the prior study.

Clearly, the performance of a trigger tool depends on its intended use. For our purposes of identifying patients with incipient ADEs in order to intervene clinically, we concluded that, with the exception of INR>5, using abnormal laboratory values is a ‘noisy’ method to identify the signal of ADEs. INR>5 did detect ADEs efficiently, and surveillance at this threshold may reduce the harm from ADEs associated with anticoagulation. For the other laboratory-based triggers, the labour associated with reviewing many charts in order to identify a single ADE was prohibitive for our clinical practice. For purposes of conducting research or for acquiring prevalence data for operational purposes, this method certainly would be more efficient than comprehensive or randomly selected chart review. Moreover, during the study period, there were 19 066 potentially eligible visits, so using the triggers did significantly reduce the pool of eligible records.

We were encouraged that the method of physician chart review resulted in reliable identification of ADEs, and our findings of problems in self-management and monitoring suggest the need for patient-directed interventions to reduce ADEs. Our results also suggest that in order to use trigger tools effectively, outpatient care settings may need to change the value at which a laboratory value triggers (the ‘trigger threshold’), considering other laboratory-based triggers, and, where feasible, consider testing text triggers. Finally, our findings imply that other tools, such as text triggers, or more complex automated screening rules which combine data hierarchically may be needed to efficiently screen for ADEs in adults seen in primary care.