The effectiveness of a new generation of computerized drug alerts in reducing the risk of injury from drug side effects: a cluster randomized trial.

CONTEXT: Computerized drug alerts for psychotropic drugs are expected to reduce fall-related injuries in older adults. However, physicians over-ride most alerts because they believe the benefit of the drugs exceeds the risk.
OBJECTIVE: To determine whether computerized prescribing decision support with patient-specific risk estimates would increase physician response to psychotropic drug alerts and reduce injury risk in older people.
DESIGN: Cluster randomized controlled trial of 81 family physicians and 5628 of their patients aged 65 and older who were prescribed psychotropic medication. INTERVENTION: Intervention physicians received information about patient-specific risk of injury computed at the time of each visit using statistical models of non-modifiable risk factors and psychotropic drug doses. Risk thermometers presented changes in absolute and relative risk with each change in drug treatment. Control physicians received commercial drug alerts. MAIN OUTCOME MEASURES: Injury risk at the end of follow-up based on psychotropic drug doses and non-modifiable risk factors. Electronic health records and provincial insurance administrative data were used to measure outcomes.
RESULTS: Mean patient age was 75.2 years. Baseline risk of injury was 3.94 per 100 patients per year. Intermediate-acting benzodiazepines (56.2%) were the most common psychotropic drug. Intervention physicians reviewed therapy in 83.3% of visits and modified therapy in 24.6%. The intervention reduced the risk of injury by 1.7 injuries per 1000 patients (95% CI 0.2/1000 to 3.2/1000; p=0.02). The effect of the intervention was greater for patients with higher baseline risks of injury (p<0.03).
CONCLUSION: Patient-specific risk estimates provide an effective method of reducing the risk of injury for high-risk older people. TRIAL REGISTRATION NUMBER: clinicaltrials.gov Identifier: NCT00818285.

RCT Entities:   Population Interventions Outcomes

Introduction

Injuries are among the leading causes of morbidity and mortality in older adults.1 2 The majority of injuries are fall-related, and 5–10% are fatal.3–5 A further 9–27% lead to a permanent loss of capacity for independent living.2 6 Psychotropic drugs are a potentially preventable cause of injury.7–13 These drugs are commonly used in older adults, often for indications, such as insomnia and pain, where strong evidence of efficacy is lacking.14–16 Based on systematic reviews, the risk of injury is increased by 39%, 59%, and 50% with the use of benzodiazepines, antidepressants, and antipsychotics, respectively.7 Risks appear to be dose-dependent, particularly for antipsychotics and opioids, where the most rapid increase in use is seen for older adults.7 15 17–19

Effective management of psychotropic medication is challenging. In older adults, 21–33% of prescribed psychotropic medication is relatively contraindicated,20–22 and 29% in doses that exceed those recommended.22 Moreover, 20% of older adults use more than one psychotropic drug concurrently, and 69% have more than one physician prescribing treatment, increasing the risk of undetected cumulative toxicity.23

Computerized prescribing and decision support are expected to address preventable medication errors, as these digital technologies can guide dosing and provide alerts on drug treatment duplication, contraindications, and drug interaction errors, especially when integrated with information on all dispensed medication. However, the majority of drug alerts are over-ridden, particularly for psychotropic medication24–27 in both hospital-based and community-based studies. Even when drug alert systems are customized to present only clinically important interactions, physicians over-ride the majority of alerts, because they are deemed not clinically relevant and/or the benefit is believed to exceed the risk.24–26 28 29 Yet, the patient-specific risk is rarely known, even though it can be estimated by incorporating into drug alert systems predictive models of adverse events developed through pharmacoepidemiological studies.30 The advanced computing power available in today's electronic record systems and the focus on individualized medicine provides an unprecedented opportunity to integrate detailed patient data into complex predictive models for estimating patient risk.

In this study, we tested the hypothesis that the incorporation of patient-specific risk estimates into a computerized prescribing decision-support system in primary care would increase physician response to alerts for psychotropic medication and reduce the risk of psychotropic drug-related injury in older adults, particularly for patients with a higher baseline risk.

Methods

Context

The study was conducted in Quebec, Canada, a province with 8.5 million residents and 16 000 physicians. The provincial insurance agency (RAMQ) provides health insurance for all provincial residents, and pays all physicians and community pharmacies on a fee-for-service basis. Beneficiary, medical billing and pharmacy claims data can be used to create longitudinal health histories for each patient. These data have been validated and are often used for health services and epidemiological research.31–34 In 2003, MOXXI, an experimental community-based clinical information system, was the first to link to these databases and integrate this information into electronic health record systems to support clinical decision-making.35

Design and study population

A cluster randomized controlled trial was conducted to test the hypothesis that prescribing decision support that provided patient-specific risk assessment would reduce the risk of injury in older adults. The trial was conducted in a population of 81 family physicians and 5628 of their older patients from September 2008 to July 2010. This sample size was expected to detect a difference in risk of 5% assuming 80 physician clusters, 50 patients per physician, a cluster correlation of 0.01, and a Type I and II error of 5% and 20%, respectively. Injury risk reduction was assessed at the end of the follow-up period (July 2010).

Potentially eligible physicians were identified from the roster of active primary care physicians in the Quebec provincial regulatory authority and invited to participate in the MOXXI primary care research network to evaluate the use of computerized drug and chronic disease management systems in primary care.14–16 27 36 37 To be enrolled in the network, physicians needed to practice actively in an urban community setting in Montreal or Quebec City, and use the computerized drug management system successfully to write a minimum of 10 patient prescriptions per week.

Patients were eligible if they were age 65 years or older, had an active dispensed prescription for a psychotropic drug, or were prescribed a new psychotropic drug at a visit during the follow-up period. Psychotropic drugs included those with central nervous system side effects that increased the risk of injury: benzodiazepines, antidepressants, antipsychotics, anticonvulsants, antihistamines, and opiates.38

Randomization and blinding

To optimize balance in the intervention and control group, we stratified physicians by the number of patients in their practice who were prescribed psychotropic medication in the past year, with a minimum of two physicians per stratum. Within each stratum, an equivalent number of physicians were randomized by the biostatistician to intervention and control groups using a random number table. Physicians were not blinded to the intervention status, but were blinded to the specific study outcomes that were measured. Unless advised by their physicians, patients were blinded to the intervention status.

Intervention and control group

Both the intervention and control groups used the MOXXI community-based clinical information system (CIS). A link to the provincial insurance agency (RAMQ) was used to pre-populate demographic information for the practice population based on a study physician's billings from the previous year. For patients who consented to participate in the research network, all medical services and prescription drugs provided for the past year were loaded into the MOXXI CIS, and thereafter all new records of medical services and prescriptions were refreshed on a daily basis. These data were used to update the profile of dispensed medications, dates and reasons for emergency department visits and hospitalizations, medical and surgical procedures, and health problems.39 40 A commercial drug alert system (http://www.vigilance.ca) automatically reviewed each new prescription for potential contraindications, including therapy duplication, dosing error, cumulative toxicity, and drug–disease, drug–drug, and drug–allergy interactions. Physicians could set the threshold for the alert system to one of three levels (1, severe alerts only; 2, moderate and severe alerts; 3, all alerts), which restricted alerts generated automatically during the prescribing process. By default, the system was set to level 2. However, all alerts generated for a patient were available for the physician to review in a drug alert summary in the patient's electronic chart. Physicians randomized to the control and intervention groups had access to these standard features of the MOXXI CIS.

Physicians randomized to the intervention group received a patient-specific risk of injury alert when a patient was prescribed a psychotropic medication that increased the risk of injury. The personalized alert used a published predictive model38 to estimate the risk of injury based on the patient's age, sex, injury history, presence of cognitive impairment, gait, and balance problems, and doses of selected psychotropic medication (selective serotonin / nor-epinephrine reuptake inhibitors antidepressants, antipsychotics, low-, intermediate- and high-potency opiates, intermediate- and long-acting benzodiazepines, anticonvulsants, and first-generation antihistamines41 42). We set a relatively low threshold for showing the alert—an increase in risk of 1 per 1000—to enable us to assess the likelihood of changes in drug treatment as a function of the magnitude of patient risk.

Graphics, in the form of risk thermometers, were created to show physicians the patient's risk of injury in the next 12 months related to psychotropic medication as well as non-modifiable characteristics (eg, age, sex) (figure 1A). Physicians would see the risk thermometer, annotated with numeric values generated by the risk calculation described above, when they opened the patient drug profile or when they prescribed a psychotropic drug. Drugs that contributed to the risk calculation were highlighted in the patient's drug profile. If the physician attempted to reduce the risk of injury by stopping or decreasing the dose of a psychotropic medication, the absolute and relative reduction in risk would be shown as an adjustment in the level of the thermometer and a change in the numeric values (figure 1B). If a new psychotropic drug was started or the dose was increased, the absolute and relative increase in the risk would be shown (figure 1C). If no change in medication was instituted (or the risk was increased by a medication change), physicians had to select a reason for the decision from a standardized pick-list (eg, prescribed by another physician). A reference section was available with publications on the risk of injury related to psychotropic drug use and methods of tapering benzodiazepines.

Figure 1

(A) Screenshot of the user interface for the ‘risk of injury alert’ showing current risk and lowest possible risk. (B) Screenshot of the user interface for the risk of injury alert showing a reduction in risk due to modified treatment. (C) Screenshot of the user interface for the risk of injury alert showing risk increase with modified treatment.

Physicians in the intervention group received a 5-minute training program on the risk of injury alert that outlined how to interpret the risk thermometer information, the expected changes in risk with new or discontinued medication, the completion of reasons for not changing therapy if applicable, and the location of reference information. Physicians were advised that stopping or reducing the dose of highlighted medications in the patient's drug profile would reduce the patient's risk of injury.

Physician characteristics

Physician age, sex, and years of practice experience were documented at enrollment. Annual measures of practice size, number of practice settings, workdays, and daily practice volume were calculated for the year before randomization from each physician's RAMQ billing data using methods developed previously.40 43 Utilization of the drug management component of the MOXXI CIS was measured by calculating the number of electronic prescriptions written per 100 patient visits billed in the year before randomization. Skill in using the prescribing system was assessed by timed assessment of prescribing speed for four standardized prescriptions 3 months after the initial training for the MOXXI CIS.

Patient characteristics

Patient age, sex, and residential address were documented at enrollment from the RAMQ insurance records. Residential address was linked to neighborhood-level Statistics Canada census information to obtain estimates of patient household income, classified by low- (<$31 753), middle- ($31 754–$80 000) and high- (>$80 000) income neighborhoods. Relevant comorbidity was measured using diagnostic and procedure codes in RAMQ billing claims and the electronic health record problem list for each patient. Injury history was assessed by calculating the number of temporally distinct episodes of billing claim treatments for fractures and soft-tissue injuries in the past 24 months using algorithms that were previously validated.32 Cognitive impairment and gait and balance problems were defined in accordance with International Classification of Disease Ninth Revision (ICD9) codes included in the predictive model for injury.38 Cognitive impairment included dementia (291), alcohol-related cognitive impairment (292), and Alzheimer's and other cerebral degeneration (331). Gait and balance problems included degenerative diseases of the extrapyramidal system (332 333), seizure disorders (345), vertiginous syndromes (386), syncope (780), and orthostatic hypotension (458). The Charlson comorbidity index,44 45 number of emergency department visits, and number of hospital admissions in the past year were also measured using RAMQ billing data to provide descriptive information about population characteristics.

Outcomes

The primary study outcome was the risk of injury at the end of the follow-up period. To assess whether the effect of the intervention was greater for patients at higher risk, each patient's risk of injury was measured twice, once at the patient's first visit after randomization (baseline risk) and again on the last day of follow-up (July 15, 2010) for the intervention and control groups (outcome risk). Overall injury risk was measured using the predictive model used in the intervention, which reflected the probability of an injury in the next year based on age, sex, comorbidity, and daily dispensed doses of benzodiazepines, opiates, antipsychotics, antidepressants, anticonvulsants, and antihistamines (figure 2). Drug doses were based on active dispensed prescriptions on the first and last day of follow-up. To permit multiple drugs within the same therapeutic category to be combined, doses were standardized, using the same approach as the predictive model, by dividing the daily dose by the WHO-recommended daily dose for adults for each drug.46 For example, a patient prescribed 20 mg diazepam and 10 mg lorazepam daily would be using three standardized doses of benzodiazepine (20 mg diazepam/10 mg (WHO-recommended dose) + 10 mg lorazepam/10 mg (WHO-recommended dose) = three doses).

Figure 2

Formula for estimating the risk of injury based on individual patient characteristics.37 NB Baseline risk was specified at 2.06%, representing the incidence of injury among 65-year-old men without any other risk factors. SSRI, selective serotonin reuptake inhibitors; SSNRI, selective serotonin-norepinephrine reuptake inhibitors.

In a secondary analysis, we assessed the physician's response to the injury risk alert, and changes in the use and dose of psychotropic medications in the intervention and control groups. Alert response was retrieved from the MOXXI audit trail for physicians in the intervention group. The audit trail recorded each action taken in relationship to psychotropic medication and its impact on overall and modifiable risk. Actions were classified into three mutually exclusive categories: (1) over-rode alert, (2) reviewed options but made no change, and (3) modified dose or drug to reduce risk. Reasons for over-riding alerts were tabulated.

To assess change in use and dose of medication, we estimated differences between the intervention and control groups in (1) the number of active psychotropic medications prescribed or dispensed, and (2) the standardized dose of psychotropic medication by therapeutic category between the baseline and follow-up period. Data were retrieved from records of all medications dispensed from the daily updates of the drug profile by the RAMQ.

Analysis

To test the hypothesis that the patients in the intervention group would have a greater reduction in the risk of injury than patients in the control group, we used multivariate linear regression within a generalized estimating equation framework. An exchangeable correlation structure was used to account for clustering of patients within physician. Each patient's calculated risk of injury on the last day of follow-up was the continuous outcome, and experimental status (intervention vs control) was the predictor. The patient's baseline risk score was used to test the hypothesis that the benefits of the intervention would be greater for high-risk patients. The interaction term between baseline risk score and experimental status was included in the model, and the Wald χ2 stxatistic was used to assess statistical significance. We used the same approach to estimate the change in the number of psychotropic medications and dose by therapeutic category between the baseline and follow-up period. For medication dose, we estimated one model per therapeutic category.

Results

Among the 410 physicians eligible, 81 consented to participate, representing 5628 eligible patients. All consenting physicians and their patients were included in the analysis (figure 3).47 Physicians in the intervention and control groups were predominantly male, French speaking, and in practice for 25 or more years (table 1). Physicians in both groups worked, on average, in 2.2 practice settings, saw 16–17 patients per day, and used the MOXXI CIS with equivalent frequency and speed of writing electronic prescriptions.

Figure 3

Consort diagram of physicians and patients eligible and enrolled in the trial.

Table 1

Characteristics of the 81 physicians and the 5628 patients in the intervention and control groups

	Control, N=40		Intervention, N=41
Physician demographics	N	%	N	%
Sex
Male	20	50.0	22	55.0
Female	20	50.0	18	45.0
Language
English	14	35.0	12	30.0
French	26	65.0	28	70.0
Practice experience (years)
<15	5	12.5	4	10.0
15–24	11	27.5	10	25.0
≥25	24	60.0	26	65.0
Practice characteristics	Mean	SD	Mean	SD
Annual practice size	1137.4	555.9	1330.6	711.6
Number of practice settings	2.2	1.3	2.2	1.0
Number of clinic days worked	190.6	43.6	186.6	39.1
Number of patients/ clinic day	16.7	7.6	17.4	6.5
Use of MOXXI CIS	Mean	SD	Mean	SD
Electronic prescription/100 visits	40.5	28.7	39.2	24.4
Speed in prescription 4 standard scripts (min)	3.08	0.88	3.04	1.09
Minimum alert setting
Level 1 (severe only)	31	77.5	24	58.5
Level 2 (moderate to severe)	5	12.5	12	29.3
Level 3 (all alerts)	4	10.0	5	12.2

Patients in the intervention and control groups were similar: on average 75–76 years of age, predominantly female, and residing in middle-income neighborhoods (table 1). Approximately 5% of patients had treatment for at least one injury in the past year, and a similar proportion had cognitive impairment. The prevalence of gait and balance problems was slightly higher in the control group as were the proportion of patients with at least one emergency department visit in the past year.

At the first visit after randomization, the most prevalent psychoactive medication used was an intermediate-acting benzodiazepine, taken by approximately one-third of patients, followed by antidepressants, anticonvulsants, and antipsychotics (table 2). For intermediate-acting benzodiazepines, the mean daily dose prescribed was 10–20% over the recommended adult dose, primarily because 4.6% of patients in the control group and 3.8% in the intervention group were using more than one intermediate-acting benzodiazepine concurrently. This was also true for antidepressants, where 1–1.2% of patients were using more than one antidepressant. The overall risk of injury at baseline was comparable in the two groups—the risk of injury in the next year was 4.03 per 100 in the control group, and 3.85 in the intervention group, of which an equivalent amount (control, 0.51; intervention, 0.49) was related to the use of psychoactive medication. The distribution was skewed, with a median of 3.55, an IQR of 2.88–4.51, and a range 2.09 to 45.30.

Table 2

Psychoactive medication at the first visit after randomization

	Control, N=2741		Intervention, N=2887
Distribution by therapeutic class	N	%	N	%
Benzodiazepines
Intermediate-acting	1516	55	1648	57
Long-acting	109	4	111	4
Antidepressants	712	26	721	25
Anticonvulsants	451	16	404	14
Antipsychotics	258	9	278	10
Opiates
High potency	22	1	5	0
Intermediate potency	96	4	104	4
Low potency	174	6	195	7
Antihistamines	7	0	4	0
Standardized daily dose among users	Mean	SD	Mean	SD
Benzodiazepines
Intermediate-acting	1.20	1.71	1.10	1.52
Long-acting	0.81	0.45	0.74	0.37
Antidepressants	1.04	0.55	1.06	0.58
Anticonvulsants	0.51	0.41	0.54	0.42
Antipsychotics	0.59	2.02	0.79	3.38
Opiates
High potency	0.29	0.85	0.16	0.15
Intermediate potency	0.43	0.38	0.50	0.63
Low potency	0.88	0.77	0.89	0.70
Antihistamines	0.13	0.04	0.79	3.38
Overall use and risk	Mean	SD	Mean	SD
Number of psychoactive medications	1.28	0.61	1.25	0.58
Proportion prescribed by study doctor	0.81	0.38	0.78	0.41
Combined standardized dose	1.18	1.59	1.15	1.69
Overall risk of injury	4.03	1.87	3.85	1.70
Risk related to psychoactive medication	0.51	0.80	0.49	1.14

Among the 2887 injury alerts generated at the first visit for patients in the intervention group, 13.7% were for new psychoactive medications that were being started by the physician during the visit, and the remaining alerts (86.3%) were for existing medications (table 3). Physicians responded to 2404 (83.3%) risk of injury alerts, by reviewing options but deciding not to change the treatment (n=1694, 58.7%), by reducing the dose, or by discontinuing psychoactive medications (n=710, 24.6%). Of interest, physicians were more likely to change the treatment plan for existing medication (25.7%) than for newly started therapy (17.7%). The most common reason for not changing treatment was the belief that the benefit would exceed the risk (n=1840, 84.5%), followed by patient resistance to change in therapy (n=179, 8.2%). While the majority of physicians changed their alert settings from the default setting to view only the most severe alerts (table 1), there were no statistically significant differences in the alert levels selected by the control and intervention groups.

Table 3

Response to the TRIPP alert in the intervention group by type of prescription

	Renewing an existing psychoactive medication		Starting a new psychoactive medication
	N	%	N	%
Number of alerts	2491		396
Changed prescription	640	25.7	70	17.7
Reviewed but no change	1399	56.2	295	74.5
No review + over-rode alert	452	18.1	31	7.8
Reasons for over-riding alert
Benefit greater than risk	1545	62.02	295	74.49
Patient demand/resistance	161	6.46	18	4.55
Need to consult prescribing doctor	53	2.13	3	0.76
Will review next time	47	1.89	3	0.76
Drug information incorrect	45	1.81	7	1.77

On average, patients in the intervention and control groups were followed for 467 and 452 days, respectively. Although there was an overall trend for the number of drugs and doses in all psychotherapeutic classes (except intermediate-acting benzodiazepines) to be reduced in the intervention group, no single class decreased significantly in dose, when modeled alone (table 4). However, there were significant interactions between baseline risk and intervention status for antipsychotics (interaction term: p=0.02) and anticonvulsants (interaction term: p=0.03), with reductions in doses occurring for patients with greater baseline risk. The most prevalent medication affected in these therapeutic groups was risperidone for antipsychotics (37.5% of patients (n=201)) and pregabalin and gabapentin for anticonvulsants (71.1% of patients (n=608)). The most common indications for prescribing antipsychotics was psychosis/bipolar disorder (32.2% of patients) and dementia/delirium (20.4% of patients), and for anticonvulsants it was pain (72.6%).

Table 4

Change in intermediate outcomes: psychotropic drug dose and number of drugs

Intermediate outcomes	Dose at the end of follow-up[y]		Dose change[y] (follow-up− baseline)		Cluster adjusted difference[z, x]		Interaction[x] baseline risk × intervention
Dose by therapeutic class[*]	Control	Intervention	Control	Intervention	Difference	95% CI	p Value
	Mean±SD	Mean±SD	Mean±SD	Mean±SD
Benzodiazepines
Intermediate-acting	0.68±0.6	0.66±0.6	−0.79±1.7	−0.74±1.5	−0.008	−0.05 to 0.03	0.91
Long-acting	0.88±0.5	0.70±0.3	−0.32±0.6	−0.43±0.5	−0.006	−0.01− to −0.00	0.19
Antidepressants	1.04±0.5	1.07±0.6	−0.41±0.8	−0.46±0.8	−0.011	−0.03 to 0.01	0.48
Anticonvulsants	0.51±0.4	0.59±0.4	−0.21±0.5	−0.19±0.4	0.006	−0.00 to 0.01	0.03
Antipsychotics	0.44±0.4	0.54±0.7	−0.33±1.9	−0.49±3.2	0.005	−0.00 to 0.01	0.02
Opiates
Intermediate potency	0.38±0.4	0.56±0.6	−0.24±0.5	−0.26±0.7	0.001	−0.00 to 0.01	0.65
Low potency	0.79±1.2	0.49±0.5	−0.63±1.0	−0.70±0.8	−0.004	−0.01 to 0.00	0.29

*There was an insufficient number of patients on high-potency opiates (n=27) or antihistamines (n=11) to model separately.

†Standardized drug dosage = daily prescribed dosage (mg)/WHO-recommended adult dosage (mg).

‡Each therapy class was modeled separately for drug dose in a model that included baseline dose and intervention group status. To test the hypothesis that the intervention was modified by baseline risk, baseline risk was added to the model as well as a two-way interaction term between intervention group status and baseline risk.

§Change in the number of psychotropic drugs was assessed in a model that included baseline number of psychotropic drugs and intervention group status. To test the hypothesis that the intervention was modified by baseline risk, baseline risk was added to the model as well as a two-way interaction term between intervention group status and baseline risk.

Congruent with these changes in therapy, the risk of injury in the intervention group was reduced by 1.7 injuries per 1000 patients (95% CI 0.2/1000 to 3.2/1000; p=0.02) compared with the control group (figure 4). The effect of the intervention was significantly greater for patients at higher risk of injury at the start of the trial (interaction between the baseline risk of injury and the intervention: p=0.03) (figure 4). For patients in the upper decile of risk of 5.76, the intervention produced a reduction in risk of 5.3 per 1000 (95% CI 3.8 to 6.8).

Figure 4

Risk of injury at the end of follow-up in the intervention and control groups and modification of the effect of the intervention by the magnitude of the baseline risk. The model used to estimate the change in the effect of the intervention by baseline risk of injury was: follow-up risk (y)=intercept (0.3696) + baseline risk (0.8427) + intervention (0.9585) + baseline risk × intervention (−0.2584) (p=0).

Discussion

This study tested the benefits of using a new approach for drug safety alerts that provided patient-specific risk estimates of the likelihood of harm. The provision of patient-specific risk information resulted in a therapy review in over 80% of patients and significant reductions in the risk of injury related to psychoactive medication, particularly for those who were at greatest risk of fall-related injuries.

This study produced a greater response to alerts than previous studies,24–27 even those that used highly selected subsets of clinically relevant alerts.48 49 Not only did physicians review consequences of revising treatment for the vast majority of patients, they also modified drug therapy in one-quarter of instances.

Of interest, the most common reason for not changing therapy was that physicians perceived the benefit of treatment to be greater than the risk. This reason was particularly common when the patient was starting new medication, possibly because a physician who starts a patient on medication will generally have decided that the benefit exceeds the risk before prescribing, even if the precise risk and benefit are not known. In contrast, prescription renewals for prevalent users were more likely to be modified, possibly because the effects of treatment (or lack thereof) can be more precisely evaluated, and thus more precise risk information can be factored into renewal decisions. It may also be because the evidence supporting the benefits of treatment is less compelling to start with, particularly in some therapeutic categories. The most significant reductions in therapy in this study were for antipsychotics and anticonvulsants, therapeutic categories that have been reported to have both high levels of off-label use and weak to non-existent evidence of benefit.50 Indeed, 22% of anticonvulsant prescriptions were for gabapentin, and all of these prescriptions were for off-label use for pain.

A ubiquitous complaint about drug alerts is that there are too many ‘nuisance alerts’, ie, alerts for problems of little clinical significance.28 49 51 This study is the first to identify a data-driven approach to identifying clinically meaningful alerts and quantifying the risk associated with an alert. Typically, ‘expert opinion’ is used to classify alerts as mild, moderate, and severe.48 This method of classification has already been shown to lack reproducible results so that different commercial systems will generate different subsets of alerts within these categories.52 53 Empirically estimating the probability of adverse effects in different subpopulations provides a more meaningful clinical metric that could be used to establish alert thresholds. In this study, physicians were more likely to alter treatment when the overall risk of injury was high, and, conversely, physicians were less likely to respond when the risk was low. To study this effect, the threshold to generate an alert was set at a low level of risk: an increased risk of 1 per 1000 (0.1%). We identified the threshold at which physicians were more likely to respond to an alert at an overall risk of 32 per 1000 (3 per 100). In the future, empirical estimates of the risk of adverse outcomes could be used to establish an alert threshold. Many national agencies are establishing methods of conducting robust pharmacosurveillance. Part of the mandate of post-market surveillance could be to assess the risk associated with both new drugs and drug interactions, thereby providing the data needed to establish and calibrate a new generation of smarter drug alert systems.

An important limitation of drug alerts, including this new generation of drug alert systems, is that information is presented only on the risk of treatment and not on the potential benefits. In the future, efforts should be made to generate empirical information on both the risks and benefits of treatment, ideally deriving this information from both clinical trials and post-market surveillance systems. In addition to providing clinicians and patients with relative risk and benefit information, ideally individualized to a patient's risk profile, alternate treatment options that have lower risk with equivalent benefit for the same treatment indication could be provided. Future research should be directed to methods of generating empirical estimates of comparative risk and benefit that can be incorporated into advanced systems of drug decision support, cognitive decision-making studies that will enable the best methods of presenting this information to physicians and patients to be identified, methods of establishing the threshold for generating drug alerts, and the potential benefits of displaying alternate therapy options by treatment indication.

In summary, this study shows that individual risk estimates displayed graphically and numerically are a more effective method of eliciting response to drug alerts and reducing the risk of medication-related injury in higher risk seniors. Future research should estimate the reduction in injury rate related to this new generation of alerts and determine whether similar effects can be achieved in other therapy classes for other important outcomes.