Tailoring adverse drug event surveillance to the paediatric inpatient.

INTRODUCTION: Although paediatric patients have an increased risk for adverse drug events, few detection methodologies target this population. To utilise computerised adverse event surveillance, specialised trigger rules are required to accommodate the unique needs of children. The aim was to develop new, tailored rules sustainable for review and robust enough to support aggregate event rate monitoring.
METHODS: The authors utilised a voluntary staff incident-reporting system, lab values and physician insight to design trigger rules. During Phase 1, problem areas were identified by reviewing 5 years of paediatric voluntary incident reports. Based on these findings, historical lab electrolyte values were analysed to devise critical value thresholds. This evidence informed Phase 2 rule development. For 3 months, surveillance alerts were evaluated for occurrence of adverse drug events.
RESULTS: In Phase 1, replacement preparations and total parenteral nutrition comprised the majority (36.6%) of adverse drug events in 353 paediatric patients. During Phase 2, nine new trigger rules produced 225 alerts in 103 paediatric inpatients. Of these, 14 adverse drug events were found by the paediatric hypoglycaemia rule, but all other electrolyte trigger rules were ineffective. Compared with the adult-focused hypoglycaemia rule, the new, tailored version increased the paediatric event detection rate from 0.43 to 1.51 events per 1000 patient days.
CONCLUSIONS: Relying solely on absolute lab values to detect electrolyte-related adverse drug events did not meet our goals. Use of compound rule logic improved detection of hypoglycaemia. More success may be found in designing real-time rules that leverage lab trends and additional clinical information.

Introduction

Compared with adults, less information is available on the epidemiology of paediatric adverse drug events (ADEs), although they are more likely to occur.1 ADE rates ranging from 6.6 to 15.7 events per 1000 patient days have been reported for paediatric inpatients.2–4 Guiding agencies have suggested adoption of health information technology (HIT) tools to reduce medical errors, but the lack of paediatric-specific information to develop standards remains a barrier.1

Duke University Hospital (DUH) employs two ADE detection systems: a voluntary safety reporting system (SRS) and computerised ADE surveillance (ADE-S).5–7 SRS is a web application where staff report safety incidents, whereas surveillance applies trigger logic to medical records and notifies of potential ADEs.5 6 In paediatrics, ADE-S did not outperform SRS in ADE detection as it does in adults.5 6 8–10 The ADE detection rate by surveillance is over four times that of paediatrics, despite the expected higher incidence of events.5 6 Paediatric HIT products should rely on independent data and not extrapolation from adults.1 This deficiency, as well as the low paediatric ADE detection rate using adult-focused triggers, clearly demonstrates a need for rules tailored to paediatric care. Our goal was to devise paediatric-focused trigger rules that are both sustainable at DUH and productive enough to serve as a longitudinal safety indicator. To meet these objectives, trigger rules must have the following design criteria: (1) objective evidence of harm inferable from the medical record, (2) reviewer time investment justified by alert volume and true positive rate, and (3) potential to generate sufficient ADE data for monthly rate trending. We retooled surveillance using insight from voluntary reporting, lab results and guidance from practising paediatricians.

Methods

Study design

We performed a retrospective, cross-sectional study at DUH, a tertiary care teaching hospital having seven paediatric inpatient units (three intensive-care units (ICUs), two general care, two transitional care). We defined an ADE as an ‘injury resulting from the use of a drug.’11 Phase 1 entailed a three-pronged approach for rule development: review of historical ADEs, analysis of lab data and paediatrician consultation. New surveillance rules were implemented in Phase 2. This study was approved by the Duke University Health System Institutional Review Board.

Phase 1

Medication-related voluntary reports to SRS are routinely investigated by clinical pharmacists and scored for severity (table 1).5 Many hospitals turn to voluntary reporting to find emerging risk areas12–14 despite reporter bias in the data.15 16 We reviewed inpatient paediatric (age <18) ADEs reported to SRS between 12 January 2002 and 12 September 2007, and grouped them by primary medication class. We examined classes with the greatest event volume, not the highest ADE rate, since voluntary reporting is not a quantitative metric.

Table 1

Duke University Hospital 7-point severity scoring system for patient harm

Score	Severity definitions
0	Caught before reached patient, but a system failure occurred
1	Reached patient, but there were no patient effects or consequences to medication use system; single dose or infusion problem >4 h
2	Reached patient, there were no detectable patient effects, but change(s) occurred in medication schedule dosing, duration or monitoring; multiple doses or infusion problem >4 h
3	Transient adverse patient effects occurred that required some corrective therapy, increased length of stay by 1 to 2 days, or resulted in laboratory values, vital signs or medication effects outside desirable parameters. A severity of 3 or greater is considered an adverse drug event if there is evidence that a drug is at fault.
4	Significant adverse patient effects occurred that required aggressive intervention, such as code, intubation, transfer to intensive-care unit, interventional drug therapy or increased length of stay >2 days
5	Permanent adverse patient effects occurred, such as paralysis, brain damage, disability or loss of limb, organ or bodily function
6	Patient death

Given that electrolyte preparations and total parenteral nutrition/lipids (TPN/lipids) were primary causes of ADEs, we gathered paediatric lab results for ionised calcium, chloride, magnesium, potassium, sodium and triglycerides from 1 August 2005 to 31 August 2007 to pinpoint critical values for trigger logic. Total bilirubin was examined as well, given concern regarding neonate monitoring.17 We plotted histograms showing the distribution of lab value results (figure 1). As with development of any alerting functionality, balance between event detection and alert volume is necessary to ensure sustainability. For most labs, the upper and lower 0.5% of distributions defined critical values for rule development. For ionised calcium, the upper 2.5% was selected as the critical value given the shape of the distribution. Although the upper 0.5 percentile for triglycerides was 1376 mg/dl, this threshold would generate few triggers each month, which defies our design requirement for trigger volume. Consequently, we sought guidance, and 500 mg/dl was suggested as a threshold by the clinical administrator of SRS, who is also a practising neonatologist at DUH.

Figure 1

Histograms showing the distribution of paediatric lab test results. The distributions of paediatric lab test values for (a) potassium, (b) ionised calcium, (c) chloride and (d) magnesium were examined over a 2-year time period. Chloride, triglycerides and total bilirubin were examined but are not shown. In each graph, the vertical axis comprises the counts of lab results. The arrows represent the thresholds for the upper or lower 0.5% of all test results (upper 2.5% for calcium, B). These values were considered extreme outliers and guided trigger rule logic. Accepted reference values for each test were dependent on age, but the most common ranges are shown.

An additional rule was developed given a known issue with ADE-S detection of hypoglycaemia. The existing rule was based on the adult protocol to administer dextrose 50% as an antidote to low blood glucose (BG). However, most paediatric protocols do not call for this drug. The paediatric hypoglycaemia-related ADE rate at DUH is only 0.08 events per 1000 patient days compared with 4.47 events per 1000 patient days in adults.5 6 Therefore, we created a paediatric rule to alert for BG<50 mg/dl concomitant with an insulin order. We did not include oral hypoglycaemic agents in this logic, since nearly 90% of DUH hypoglycaemic events are due to insulin.

Phase 2

The DUH ADE-S system has been described previously.5 6 10 18 Each evening ADE-S reviews the electronic records of all DUH inpatients to identify potential ADEs. Clinical pharmacists perform a chart review to confirm ADE occurrence (ie, documented patient harm). Events are scored by three pharmacist reviewers (κ≥0.88) for severity (table 1)5 and causality.19 Paediatric-specific rules were evaluated between 27 November 2007 and 12 February 2008. Fifteen days were excluded due to reviewer unavailability.

Data analysis

Phase 1 lab data histograms were created using JMP 7.0 (SAS Institute, Cary, North Carolina). Phase 2 analysis included all alerts for patients discharged from a paediatric location. Positive predictive values (PPVs) were calculated as the percentage of alerts deemed ADEs subsequent to evaluation. Rates were calculated as ADEs per 1000 patient days. The Fisher exact test was used to seek significant differences in categorical data. CIs were calculated using the binomial approximation. Where appropriate, the ADE yields of the new and old trigger rules were cross-compared.

Results

Phase 1: Review of historical ADE data and critical lab values

SRS documents 385 medication-related ADEs occurring in 353 paediatric patients from 12 January 2002 to 12 September 2007 (table 2). Electrolyte preparations and TPN/lipids account for 24.4% (94/385) and 12.2% (47/385) of events, respectively. Representative cases include an incident where dispensing of dextrose-containing intravenous fluids was delayed, resulting in a BG of 27 mg/dl and administration of an interventional glucose bolus to reverse hypoglycaemia. The American Diabetes Association holds that hypoglycaemia, regardless of symptoms, harms the patient.20 Another patient developed premature atrial contractions from hyperkalaemia due to failure to discontinue potassium-containing intravenous fluids. This event was scored a severity of 3, since transient adverse effects occurred, and corrective therapy was used to lower the potassium level.

Table 2

Inpatient paediatric adverse drug events found by voluntary reporting (n=385)

Event location	No (%)
Critical care unit	257 (66.8)
General care unit	128 (33.2)
Medication category	No (%)
Electrolyte preparations	94 (24.4)
Total parenteral nutrition/lipids	47 (12.2)
Anti-infectives: antibiotics	45 (11.7)
Cardiac drugs	37 (9.6)
Narcotics and benzodiazepines	36 (9.4)
Other*	126 (32.7)

Other includes 20 categories, such as anticoagulants, vaccines, anticonvulsants, and antineoplastics.

From 1 August 2005 to 31 August 2007, over 450 000 lab results for paediatric inpatients were available for ionised calcium, chloride, magnesium, potassium, sodium, total bilirubin and triglycerides. Figure 1 shows histograms for these results as well as the outlier values selected.

Phase 2: Implementation of paediatric-specific ADE-S trigger rules

Development of paediatric-specific trigger rules (table 3) was guided by Phase 1 results. Only two of the original adult-designed rules involve electrolytes: dextrose 50% with a BG <50 mg/dl (A1) and an order for polystyrene, a reversal agent for elevated potassium levels (A2). Between 27 November 2007 and 12 February 2008, DUH had 796 paediatric admissions and 9270 patient days across the general care (41% of patient days), ICU (53% of patient days) and transitional care (6% of patient days) environments. These visits produced 225 trigger alerts (rules P1–P9) for 103 patients (table 3). Fourteen alerts from the hypoglycaemia rule (P9) were deemed ADEs following chart review (PPV=6.2%). The electrolyte-based rules did not detect ADEs, as no objective evidence of harm was documented.

Table 3

Trigger rules employed in paediatric adverse drug event (ADE) computerised surveillance

Rule code	Rule name	Rule type*	Total trigger rule alerts	Total ADEs	Positive predictive values (%) (95% CI)	ADEs per 1000 patient days
						(95% CI)
New paediatric-specific trigger rules (patients <18 years old)
P1	Calcium ionised >1.5 mg/dl	Lab only	44	0	0	0
P2	Chloride <80 mEq/l	Lab only	19	0	0	0
P3	Magnesium >3.5 mEq/l	Lab only	26	0	0	0
P4	Potassium >7 mEq/l	Lab only	32	0	0	0
P5	Sodium <120 mEq/l	Lab only	9	0	0	0
P6	Sodium >157 mEq/l	Lab only	25	0	0	0
P7	Total bilirubin >20 mg/dl	Lab only	10	0	0	0
P8	Triglycerides >500 mg/dl	Lab only	37	0	0	0
P9	Insulin and BG <50 mg/dl	Drug-lab	23	14	60.8 (40.7 to 77.9)	1.51 (0.72 to 2.23)
		Total	225	14	6.2 (3.0 to 11.9)	1.51 (0.72 to 2.23)
Existing adult trigger rules
A1	Dextrose 50% and BG <50 mg/dl	Drug-lab	7	4†	57.1 (25.0 to 84.3)	0.43 (0.01 to 0.44)
A2	Polystyrene use	Drug only	5	0	0	0

Example trigger description: lab only: fire if a paediatric patient has magnesium >3.5 mEq/l in the previous 24 h. Drug-lab: fire if a paediatric patient is on insulin and has a blood glucose (BG) <50 mg/dl in the previous 24 h.

All four events were discovered by the P9 rule as well.

Both the adult- (A1) and paediatric-focused (P9) rules for hypoglycaemia were active to evaluate differential ADE capture. A total of 30 triggers fired on paediatric units from these two rules (table 3). Some alerts were not deemed ADEs due to erroneous BG readings or undocumented insulin administration despite an active order. The PPVs for both P9 and A1 were similar (60.8% and 57.1%), yet a greater number of ADEs were detected with the paediatric rule (1.51 vs 0.43 events per 1000 patient days, p=0.03). All ADEs captured by the adult rule were detected using the paediatric-focused logic.

Discussion

Our goal was to combine clinical evidence and provider insight to develop a sustainable, paediatric-relevant surveillance model that yields trendable data for ADE monitoring. A review of voluntarily reported safety incidents revealed electrolyte and TPN/lipid preparations underlie many ADEs. No rule targeting lab abnormalities detected ADEs in this study, although we improved hypoglycaemia ADE capture.

Our results are consistent with other paediatric trigger development efforts. Sharek and colleagues developed a neonatal intensive care unit (NICU)-focused manual tool for adverse event discovery.21 They included rule logic for rising creatinine (PPV=11%) and an ‘abnormal electrolytes’ rule with a PPV of 8%. Since this latter rule is not retained in the trigger tool online (http://www.chca.com), we infer it was ineffective. A second trigger tool for paediatrics was deployed across 12 children's hospitals,4 and compound rule logic produced better PPVs. Although a single-value hyperglycaemia trigger resulted in a PPV of only 0.60%, combining absolute lab values with age logic to detect hyperkalaemia or nephrotoxicity resulted in PPVs of 3.57% and 3.85%. A recent study modifying a surveillance system for paediatrics had limited success with ADE detection using hypokalaemia and hypomagnesaemia rules.22 The authors also report a PPV of 0.08 for hyperkalaemia combined with age logic (potassium >6.0 mEq/l and age >1 year), whereas our PPV for potassium >7.0 mEq/l was 0.

These studies indicate that the use of compound rule logic improves paediatric ADE discovery. At DUH, the issue with using absolute lab values for ADE-S may lie in how they are drawn. For example, NICU heel sticks are used to obtain small blood samples, yet they frequently haemolyse, resulting in elevated potassium values. In the case of elevated lipid levels, the nurse may have drawn the sample from the line infusing lipids. Such details go undocumented, meaning relying on lab values retrospectively to suggest that ADE occurrence is ineffective for our goals. Options to improve rules using compound logic include reasoning over lab value changes (eg, rising bilirubin), since utilising more than two values may bypass unwanted alerts from a single, aberrant result. Additionally, concomitant active drug orders and lab results (eg, use of electrolyte preparations, potassium supplements, or potassium-sparing diuretics and hyperkalaemia) may be of value. Institutions with text scanning capabilities may be able to combine keywords from the medical record relevant to a lab result (eg, altered mental status and the presence of hyponatremia). Since ADE-S runs in a batch process, we are evaluating logistics for real-time alerting of concerning lab trends. Takata and colleagues reported two-thirds of their ADEs were due to lapses in monitoring medications, including assessment of lab results.4 This series of strategies may reduce false-positive alerts and improve ADE detection. Although lowering lab value thresholds may capture more ADEs, the expected increase in alert volume without improving the PPV would be a barrier to sustainable review—a central goal of this study.

There are several limitations to this work. DUH has a large paediatric ICU population (53% of study patient days), which accounts for 61% of trigger alerts. This may limit the generalisability of our findings to hospitals with a similar composition, yet these are likely the same facilities that would have IT resources to implement a computerised surveillance programme. A second limitation is that rules were evaluated over 2.5 months, which may have under-represented detection of rare ADEs. The authors felt that 2.5 months was adequate to evaluate alert utility given the goals of (1) sustainability and (2) data capture for aggregate ADE rate monitoring. Though not useful for our purposes, other organisations may have more success with these rules if able to sustain a longer study period. However, a recent 6-month study with over 40 000 patient days similarly reported low PPVs for electrolyte-based rules (except hypokalaemia and hypomagnesaemia).22 They also noted the need to modify or eliminate low-performing rules to balance reviewer effort and event discovery despite the possibility of rare ADE detection.22 A final limitation to any surveillance approach is its reliance on objective data in the medical record to confirm ADE occurrence. Near misses cannot be captured if errors did not reach the patient.

Conclusion

Although computerised surveillance effectively detects ADEs, developing paediatric rules based solely on critical lab values did not meet our goals. However, enhancement of a hypoglycaemia trigger rule significantly increased our detection rate. Investigators should be encouraged not only to report their successes in this area but also to describe ineffective approaches. The sustainability of trigger evaluation should be a consideration for any rule design. Development of compound rule logic may be a worthwhile approach.

Paediatric patients are at high risk for adverse drug events in the hospital setting. The application of novel detection strategies, such as computerised surveillance, however, is underexplored in this fragile population. To date, inpatient paediatric trigger tool rules have mainly been extrapolated from logic designed for adults.

We used a three-pronged approach to develop new trigger rules specifically for hospitalised paediatric patients: analysis of a voluntary safety reporting database, evidence from historical lab values and concerns from practising paediatricians. We share our experiences with these new rules and evaluate their efficacy when added to a sustained computerised surveillance safety programme at a large academic medical centre. Our lessons learnt may aid other organisations seeking to create paediatric-targeted rule sets that support their safety event-detection programmes.