Unintended consequences of reducing QT-alert overload in a computerized physician order entry system.

PURPOSE: After complaints of too many low-specificity drug-drug interaction (DDI) alerts on QT prolongation, the rules for QT alerting in the Dutch national drug database were restricted in 2007 to obviously QT-prolonging drugs. The aim of this virtual study was to investigate whether this adjustment would improve the identification of patients at risk of developing Torsades de Pointes (TdP) due to QT-prolonging drug combinations in a computerized physician order entry system (CPOE) and whether these new rules should be implemented.
METHODS: During a half-year study period, inpatients with overridden DDI alerts regarding QT prolongation and with an electrocardiogram recorded before and within 1 month of the alert override were included if they did not have a ventricular pacemaker and did not use the low-risk combination cotrimoxazole and tacrolimus. QT-interval prolongation and the risk of developing TdP were calculated for all patients and related to the number of patients for whom a QT-alert would be generated in the new situation with the restricted database.
RESULTS: Forty-nine patients (13%) met the inclusion criteria. In this study population, knowledge base-adjustment would reduce the number of alerts by 53%. However, the positive predictive value of QT alerts would not change (31% before and 30% after) and only 47% of the patients at risk of developing TdP would be identified in CPOEs using the adjusted knowledge base.
CONCLUSION: The new rules for QT alerting would result in a poorer identification of patients at risk of developing TdP than the old rules. This is caused by the many non-drug-related risk factors for QT prolongation not being incorporated in CPOE alert generation. The partial contribution of all risk factors should be studied and used to create clinical rules for QT alerting with an acceptable positive predictive value.

Introduction

Many computerized physician order entry systems (CPOEs) generate drug safety alerts to remind physicians of potentially unsafe situations. Drug safety alerts are frequently overridden, for example because the alert is not patient-tailored or because the disadvantages of the situation do not outweigh the advantages. A high number of low-specificity alerts may cause physicians to override important alerts along with unimportant ones, thereby decreasing safety [1].

In the Netherlands, all hospital CPOEs make use of the national drug database, which is updated monthly. This ‘G Standard’ contains safety information for all drugs licensed in the Netherlands [2]. The G Standard introduced drug-drug interaction (DDI) alerting on QT prolongation in March 2005. QT prolongation may predispose patients to developing Torsades de Pointes (TdP) and to sudden cardiac death. After many complaints about low-specificity alerts in the CPOEs, several drugs were excluded from QT-alert generation in May 2007 [3, 4] without any outcome measurements.

The aim of this study was to compare the rules for QT alerting to see whether the 2007 rules would identify patients at risk of developing TdP better than the 2005 rules.

The following questions were to be answered:

In what percentage of patients at risk of developing TdP due to a combination of two QT-prolonging drugs is a QT-prolongation DDI alert generated (sensitivity)?

In what percentage of generated QT-prolongation DDI alerts is the patient really at risk of developing clinically significant QT prolongation (positive predictive value of the QT alert)?

Background

Many cardiac and noncardiac drugs can prolong the QT interval on the electrocardiogram (ECG), thereby increasing the risk of serious ventricular arrhythmias (e.g., TdP) and sudden cardiac death. TdP has a low incidence, and the prolongation of the absolute QTc interval beyond 500 ms and/or an increase of more than 60 ms are regarded as leading to an increased risk of TdP [5-7].

Many risk factors may increase the risk of developing TdP such as gender, age, cardiovascular disease, and electrolyte disturbances; elderly females are especially at risk. Many drugs increase this risk to different extents, and higher doses and renal failure may add an additional risk [5,7].

The Dutch national drug database, the G Standard, has included DDI alerts on QT prolongation since March 2005 [8]. At first, drugs from lists D and E from De Ponti [9, 10] generated this alert, as well as all class Ia and III antiarrhythmics [4]. List D contained all drugs clinically associated with TdP, and list E included drugs with clinical evidence for TdP plus an official warning of causing TdP [8-10]. In 2006 a discussion took place about the relevance and urgency of this DDI. Some hospital pharmacists concluded after studying the literature that many combinations were of minor importance with no need for action [8, 11], although hospital pharmacists responsible for the DDI alerts in the G Standard disagreed [12-14].

Since May 2007 Dutch QT alerting has been based on the system of the Arizona Center for Education and Research on Therapeutics [15]. This system earlier consisted of four drug classes with a different risk of causing TdP: class 1 drugs were known to cause TdP, class 2 drugs had a probable risk, and class 4 were unlikely to cause TdP. Class 3 drugs were contraindicated in patients with (congenital) long QT-syndrome [3]. At present, three categories exist: drugs with a risk of causing TdP (formerly class 1), drugs with a possible risk (formerly class 2) and drugs with a conditional risk (including the former class 4 drugs) [15].

In May 2007, the G Standard limited DDI alerting for QT prolongation to combinations of class 1 drugs and terfenadine and adjusted the information content of the alert text (Figs. 1 and 2). Furthermore it introduced contraindication alerting for patients with a prolonged QT interval taking single drugs from classes 1 and 2, sympathicomimetic drugs or terfenadine. The new rules for DDIs resulted in a reduction in the number of drugs generating the QT alert (from 30 to 20) and were based on expert opinions formulated after studying and discussing the available literature. Outcome measurements were not performed [3].

Fig. 1

First part of the old alert text

Fig. 2

First part of the new alert text

Methods

Setting

The 1,237-bed Erasmus University Medical Center, Rotterdam, the Netherlands, uses the CPOE Medicatie/EVS (Leiden, the Netherlands) [16] on all wards except ICUs. This CPOE system for prescribing medication generates intrusive drug safety alerts for DDIs, overdoses, and therapeutic duplications based on information held in the G Standard database. Overridden drug safety alerts are routinely logged for pharmacy review.

Study population

All overridden QT-prolongation DDI alerts generated in Medicatie/EVS version 2.20 between 1 February 2006 and 31 July 2006 in the Erasmus MC-Center location (a general hospital) were used for patient selection. Outpatients, patients with ventricular pacemakers, transplanted patients treated with the low-risk combination of tacrolimus with cotrimoxazole (class 2 and 4), patients who were long-term users of QT-prolonging drugs with unknown start dates or who were no longer using the combination were excluded. The secondary inclusion criterion was patients with ECGs available from before and within 1 month of the QT-alert override.

Measures

For each patient included, the interacting drugs, risk factors for TdP, and digital ECG recordings (12-lead resting ECGs recorded with a Mortara electrocardiograph) were collected. Risk factors for TdP were defined as female gender, age > 65 years, presence of cardiovascular disease (myocardial infarction, heart failure, atrial fibrillation, hypertension, cerebrovascular accident, peripheral vasculopathy), diabetes mellitus (use of glucose-lowering drugs), renal failure (glomerular filtration rate < 50 ml/min), and potassium level < 3.5 mmol/l. Increased risk of TdP was defined as QTc interval > 500 ms or an increase in the QTc interval > 60 ms [6]. Sensitivity was calculated as true positives/(true positives + false negatives). Positive predictive value was calculated as true positives/(true positives + false positives).

Results

In the 6-month study period, DDI alerts on QT prolongation were overridden for 368 patients. Of these, 319 patients were excluded for different reasons (Table 1). The most frequent reasons for exclusion were the use of tacrolimus and low-dose cotrimoxazole in transplant recipients (n = 124, 34%), the unavailability of ECG recordings before and after initiation of the drug combination (n = 119, 32%), and the patient being treated on an outpatient basis (n = 35, 9.5%).

Table 1

Patient selection

Patient category		Number
Patients with overridden drug safety alerts on QT prolongation from 1 February – 31 July 2006		368
Patients excluded		319
Treated on an outpatient basis	35
Using tacrolimus and low-dose cotrimoxazole	124
Combination not used any more	22
Long-term use of combination (start date unknown)	7
Ventricular pacemaker	4
Other reasons	8
< 2 ECGs	119
Patients included		49

Forty-nine patients met the inclusion criteria; Table 2 presents the patient characteristics. The mean number of non-drug-related risk factors was 2.7 (SD 1.1). All patients had at least one non-drug-related risk factor for developing TdP.

Table 2

Characteristics of patients meeting the inclusion criteria (n = 49)

Characteristic	Number (%)
Female gender	20 (41%)
Cardiovascular disease	44 (90%)
Diabetes mellitus	17 (35%)
Renal failurea	19 (42%)
Age > 65 years	29 (59%)
Potassium level < 3.5 mmol/lb	3 (6.7%)

aCalculation based on all patients for whom an estimated glomerular filtration rate was available (n = 45)

bCalculation based on all patients with a measured potassium level (n = 45)

Fifteen patients (31%) were considered at risk for developing TdP; Table 3 shows their patient characteristics. All at-risk patients used two QT-prolonging drugs, ranging from high risk (class 1) to low risk (4). The number of non-drug-related risk factors per patient ranged from 1 to 5.

Table 3

Subjects at risk of developing Torsades de Pointes (n = 15)

Gender	Age	Cardiovasculardisease	Diabetes mellitus	GFR (ml/min)	K+ level (mmol/l)	Risk factors	Drug 1	Drug 2	QTc2 (ms)	ΔQTc (ms)	New alert
Female	75	+	+	13	4.1	5	Haloperidola (1)	Amiodaronea (1)	504	29	+
Female	71	+	−	37	4.2	4	Indapamidea (2)	Promethazinea (NC)	470	64	−
Male	68	+	−	49	4.1	3	Amiodarone (1)	Haloperidola (1)	487	100	+
Male	72	+	−	48	3.9	3	Amiodarone (1)	Ketanserina (NC)	537	83	−
Female	62	+	−	7	4.2	3	Amiodarone (1)	Tacrolimusa (2)	592	201	−
Female	53	+	−	49	4.1	3	Haloperidola (1)	Tacrolimusa (2)	530	62	−
Male	72	+	−	48	3.9	3	Sotalol (1)	Erythromycina (1)	501	32	+
Male	51	+	+	33	4.4	3	Tacrolimusa (2)	Mianserina (NC)	510	122	−
Female	81	+	−	80	3.6	3	Domperidonea (1)	Amitriptylinea (4)	438	75	−
Male	68	+	−	>90	4.4	2	Chlorpromazinea (1)	Cisapridea (1)	490	64	+
Male	61	+	−	26	4.1	2	Haloperidola (1)	Sotalola (1)	478	84	+
Male	64	+	+	77	4.7	2	Sotalol (1)	Amiodaronea (1)	502	73	+
Male	64	+	+			2b	Chlorpromazinea (1)	Ketanserina (NC)	467	91	−
Male	64	+	−	59	4.0	1	Haloperidola (1)	Amiodaronea (1)	560	141	+
Male	45	+	−	>90	4.2	1	Haloperidola (1)	Tacrolimusa (2)	492	84	−

Patients are categorized according to number of non-drug-related risk factors

+ Present, - absent, GFR  glomerular filtration rate in ml/min, QTc2 QTc interval after QTc-alert override, ΔQTc change in QTc interval between ECGs before and after QTc alert

Numbers in parentheses indicate drug class according to www.torsades.org; NC  not classified on www.torsades.org

aQTc-prolonging drug started at time of QTc alert

bNumber of risk factors might have been higher due to unknown values

In the new database since May 2007, many frequently encountered combinations of QT-prolonging drugs no longer generate a DDI alert in the CPOE. The last column of Table 3 shows whether combinations would result in a QT alert in the new situation. For 8 of the 15 patients with increased risk of TdP in our study (53%), no alert would be generated with the new rules because the drugs are “not classified” or do belong to classes 2 or 4. Assuming the CPOE with the old “inclusive” drug database identified all patients at risk of developing TdP, the modified database would result in a sensitivity of 47%. Table 4 shows whether an alert would be generated for patients at risk of developing TdP in the new situation. Twenty-three rather than 49 alerts would be generated (47%). The positive predictive value in the study population was 31% (15/49) in the old situation and would be about the same (30%, 7/23) if the CPOE would make use of the modified database.

Table 4

Numbers of patients at risk of developing Torsades de Pointes for whom a QT-prolongation DDI alert is generated in the new situation (database restricted to obviously QT-prolonging drugs)

	Alert generated (n)	No alert generated (n)
Patients at risk of TdP	7 (true positives)	8 (false negatives)
Patients not at risk of TdP	16 (false positives)	18 (true negatives)

Discussion

The decreased number of drugs generating QT alerts successfully lowers the alert numbers in our study population from 49 to 23. However, it does not address the specificity problem adequately, as the positive predictive value does not change. Furthermore, the QT-rule modification introduces a sensitivity problem as the new system would miss 53% of the patients at increased risk of developing TdP. Reduction of the QT-alert overload by excluding several drugs from QT-alert generation clearly has unintended and undesirable consequences.

One question is whether these results can be extrapolated to the entire inpatient population. Only inpatients with an ECG before and within 1 month of QT-alert overriding were included. Thirty-two percent of the patients with QT-alert overrides were excluded because ECGs were not available to calculate the QT interval, and these could have been low-risk patients. However, the excluded patients had a lower average number of non-drug-related risk factors: 2.0 (SD 1.2). The patients included had a higher average number of 2.7 (SD 1.1), which could have led to an overestimation of the proportion of patients considered to be at risk.

None of the patients in our study had zero non-drug-related risk factors, and it is likely that the risk factors for developing TdP (e.g., cardiovascular disease) led to an overestimation of the positive predictive value. Inclusion of the entire inpatient population would have resulted in an even lower positive predictive value.

Furthermore, patients using the combination tacrolimus and cotrimoxazole were excluded because this very frequently used combination in transplanted patients in the Erasmus MC was perceived not to result in TdP. It can be questioned, however, whether this assumption is correct [17]. If the combination really is a low-risk combination not resulting in TdP, inclusion of these patients would have resulted in a higher positive predictive value.

How can these unintended consequences be understood? QT prolongation is dependent on age, gender, co-morbidity, serum potassium level, renal function, drug class, and drug dose. Although age and gender of the patients are known in our CPOE, these items were not used in QT-alert generation and suppression. QT-alert generation in Medicatie/EVS was and is only dependent on drug class and is not tailored to at-risk patients, so accuracy remains low. Furthermore, the drugs now excluded from QT-alert generation are known to have a probable or unlikely risk of causing TdP when used as single drugs, but the effects of combinations of these drugs in patients with non-drug-related risk factors are unknown.

Error management

How should the problem of these low-specificity alerts be managed? Ideally, QT alerts would only be generated for patients really at risk of developing TdP, and they would be suppressed if the risk is low [18]. However, to calculate the overall risk of developing TdP, the contributions of all risk factors, including drug class and dose, should be known. This information is not known, and therefore effective filtering of QT alerts for at-risk patients is not feasible. Only by prospectively collecting ECGs before and after the initiation of combinations of two or more QT-prolonging drugs will we be able to determine the true risk of developing clinically relevant QT-prolongation. It is only with this knowledge that QT alerts with both high sensitivity and specificity (positive predictive value) can be developed. An acceptable positive predictive value is open to debate. Bates proposed an override rate of less than 40% for strongly action-oriented suggestions [19], but this seems to have been chosen arbitrarily. If however this recommendation were to be followed, the current positive predictive value of DDI alerts on QT prolongation should be doubled.

We recommend that ECGs should be performed before and within 1 week of the QT override. If postponement of this drug therapy were undesirable, a single ECG after the QT override would also give useful information. This recommendation to record an ECG should be presented as a clear message during the order entry process. Both old and new alert texts are rather long and complicated (Figs. 1 and 2), and it is easy to modify the messages. It would be very helpful if ECGs could be ordered from the CPOE, but this type of integration is largely absent in Dutch hospital CPOEs.

The drug lists on www.torsades.org are regularly updated in contrast to the De Ponti list [3, 15]. The lists at first only included drugs that were on the market in the United States. Fortunately, drugs that are not (and no longer) available in the U.S. (terfenadine, domperidone) are included now. For the Dutch situation it should be kept in mind however that ketanserin, mianserin, and promethazine are absent on www.torsades.org [3, 15].

Strengths and weaknesses

This study had several limitations. It focused on the risk of TdP by analyzing QT prolongation. Although this relationship is not clear cut, this is the best way to study the risk of developing TdP, as TdP has a low incidence [5-7]. QT intervals show high diurnal variability, may be subject to reading errors, and are dependent on serum drug level [5–7, 20, 21]. The ECGs in this study were not recorded under standardized conditions, and this might have resulted in less accurate QT intervals. This study did not aim to identify risk patients with a high certainty, but mainly focused on the difference between old and new rules for QT-prolongation alerts in a CPOE. It elucidated a problem requiring a prospective study including ECGs recorded under standardized conditions and taking into account drug serum levels.

Due to QT-interval variability, it can be questioned whether it is correct to use absolute QT intervals >500 ms or QT prolongation >60 ms as the best identification of patients at risk of TdP [20, 21]. We used both measures according to the guidelines of the European Medicines Agency and only used the categories with most marked increases to reduce the effect of QT variability [6].

DDIs may have been generated by adding one QT-prolonging drug to an existing therapy containing another QT-prolonging drug, but may also have been the result of two newly prescribed QT-prolonging drugs. Twenty-five patients (51% of the patients included) already used one QT-prolonging drug, resulting in a smaller increase in QT interval and a higher probability of exceeding the limit of 500 ms. This was another reason to include both QT-interval measures to identify patients at risk of TdP.

A weakness of this study is that the study population may differ from the whole patient population. Selection may have been biased because patients taking the combination tacrolimus-cotrimoxazole (34%) and patients without two ECGs (32%) and with a lower number of non-drug-related risk factors were excluded. It is unlikely however that inclusion of the tacrolimus-cotrimoxazole combination would change our conclusions that the new rules are worse. This combination does not result in alert generation with the adjusted rules. If it were a low-risk combination, inclusion would increase the positive predictive value, but the sensitivity would remain low. If it were a high-risk combination, inclusion would result in a decreased positive predictive value and sensitivity. Both effects are unintended.

The modifications of the G Standard excluded 11 drugs generating QT alerts and added 1 drug, arsenic trioxide. This could have had an effect on the sensitivity and positive predictive value, but this drug was not prescribed in our CPOE in the study period.

A drawback of the CPOE used in this study is that only overridden alerts are logged for pharmacy review. Alerts resulting in order cancellation are not available, and override reasons are not required. Disguised observation in the Erasmus MC revealed an override rate of > 90% for DDIs, including QT-prolongation alerts (unpublished data).

Notwithstanding these limitations, this study clearly showed the unintended effects on patient safety of a proposed measure to reduce alert overload, making use of patient data from normal clinical practice.

Conclusion

Reducing QT-alert overload by excluding drugs without proven risk of causing TdP from alert generation would result in a considerable reduction in alert numbers, would not change the positive predictive value, and would introduce a sensitivity problem. The high number of non-drug-related risk factors that are not included in QT-alert generation could explain these unintended consequences. Further outcome measurements should be performed to elucidate the contribution of the non-drug-related risk factors to the overall risk. Ideally, clinical rules incorporating all risk factors could then be developed to generate QT alerts with an acceptable positive predictive value.

Competing interests/Funding None.