Personalised performance feedback reduces narcotic prescription errors in a NICU.

OBJECTIVE: Neonates are at high risk for significant morbidity and mortality from medication prescribing errors. Despite general awareness of these risks, mistakes continue to happen. Alerts in computerised physician order entry intended to help prescribers avoid errors have not been effective enough. This improvement project delivered feedback of prescribing errors to prescribers in the neonatal intensive care unit (NICU), and measured the impact on medication error frequency.
METHODS: A front-line multidisciplinary team doing multiple Plan Do Study Act cycles developed a system to communicate prescribing errors directly to providers every 2 weeks in the NICU. The primary outcome measure was number of days between medication prescribing errors with particular focus on antibiotic and narcotic errors.
RESULTS: A T-control chart showed that the number of days between narcotic prescribing errors rose from 3.94 to 22.63 days after the intervention, an 83% improvement. No effect in the number of days between antibiotic prescribing errors during the same period was found.
CONCLUSIONS: An effective system to communicate mistakes can reduce some types of prescribing errors.

Introduction

A 1999 report by the Institute of Medicine estimated as many as 98 000 individuals die annually in hospitals as a result of medical errors.1 In a random sampling of 30 000+ medical records by the Harvard Medical Project, as many as 19% of the 1133 adverse events found were medication related.2 Despite efforts to solve this problem, medication-related adverse events continue to be highly prevalent and particularly harmful to a rapidly growing neonatal population.3 4 Medication prescribing errors, the focus of this report, are a significant contributor to medication-related adverse events. The need to pay attention to rapidly changing weights and drug dosing based on multiple parameters, such as weight and gestational age, as neonatal prescribing requires vigilance both at dose initiation and maintenance to reduce medication errors.5–8

Doing ‘the five rights (right drug to the right patient, at the right dose, route and time)’ of medication management remains a huge challenge especially for prescribers. Aside from being the most frequent type of medication errors, prescribing errors have significant downstream effects. In our institution, it is common for the pharmacy to process approximately 34 orders per hour; since resolving a prescribing error takes an estimated 15 min, each prescribing error can, potentially, delay processing of as many as seven other medications. In addition, reconciling prescribing errors adds error opportunities by causing workflow interruption and distractions for pharmacist and prescribers. By the time errors are intercepted downstream, significant operational cost (time, resources and personnel) has been wasted.

Alerts built into computerised physician order entry (CPOE) systems have made significant progress in preventing prescribing errors. But stifled by unintended negative consequences and high degrees of alert override,9 10 they have not been as effective in preventing prescribers from making errors, as previously hoped.11–13 An alternative strategy worth exploring in the prevention of errors is the use of performance feedback to help providers prescribe more carefully.

Published literature, and our experience, indicate that there is low self-awareness of how many errors prescribers make.14 15 Hence, informing prescribers about specific errors they have made, and how to avoid them in the future, may be an important step to engaging them into a team effort to reduce prescribing errors. The importance of feedback and reminders in efforts to improve compliance with clinical care guidelines has been emphasised by published literature.16 17 To this point, similar approaches for safe practice guidelines show that direct constructive feedback to prescribers about their errors, and ways to avoid them in the future, can reduce errors.7 18–20 Some literature suggests that delivering feedback closer to the moment of action has more impact on the recipient.21 22

The purpose of this paper is to report the development of a prescribing error feedback programme, and its impact on narcotic prescribing errors. We also discuss how an approach to synergise appropriate personnel, processes and tools was important to the programme's successful implementation.

Local setting and problem

This study took place in a major 75-bed neonatal intensive care unit (NICU) regional referral centre. The unit uses an electronic health record system with computerised order entry capabilities. The existing CPOE alerts have limited impact, as demonstrated by high override rates similar to the reported literature. Although an electronic formulary is available on the intranet, it cannot be accessed directly from the CPOE system when it is most needed during order entry. On average, 195 medications are prescribed per day by residents, nurse practitioners, physician assistants and attending physicians in the NICU. Everyday, pharmacists review these orders in addition to approximately 3200 other orders from other hospital units, processing approximately 34 orders per hour. Within this hour, the pharmacists have the arduous task of detecting any prescribing errors and calling prescribers for clarification or correction. These ‘incidents’ are recorded in an electronic database that can be queried for quality control activities.

Review of prescribing errors indicated that most errors were associated with narcotics—an Institute for Safe Medication Practices-defined high-alert medication. Although pharmacists intercept most narcotic errors, those that reach neonates can cause respiratory and neurological depression leading to emergent intubation and mechanical ventilation. Therefore, reducing narcotic errors was an important safety goal of this project.

Materials and methods

Contextual barriers

Lukewarm prescriber acceptance and buy-in to the feedback was a critical obstacle to overcome. One provider questioned ‘Why do we need this summary information every 2 weeks when pharmacists already call us for each error?’ ‘Will this go into my evaluation record?’ In order to maintain confidentiality, all feedback communications about prescribing errors remained within the institution firewall system, and all titles and names were removed. No error information was recorded in anyone's evaluation files. We emphasised feedback as an informative tool that offered a deeper review of error themes across all prescribers, which cannot be done on an event-by-event basis. We considered using screensavers to publicly display our findings in the unit, but believed that although this would best alert bedside nurses of prescribing error vulnerabilities, the psychosocial impact on parents were unknown. Hence, our team opted to limit feedback to individuals until we can better understand how such knowledge can best be communicated publicly.

Intervention development

We employed the model for improvement to create, develop and refine the feedback intervention from September 2009 to February 2010 before implementing it in March 2010.23 Our multidisciplinary team consisted of a dedicated neonatal pharmacist with several years of experience, a neonatal fellow, a research assistant and an attending neonatologist who was also the improvement advisor. The team also frequently engaged groups of residents and frontline clinicians to understand the prescribing process and determine the best feedback strategy and content.

The team identified four primary drivers (‘must do's’) for a successful error feedback programme as (1) maximise error reporting, (2) optimise data analysis, (3) determine best feedback strategy and (4) ensure bidirectional communication between prescriber and feedback team. Since the pharmacy was already reporting prescribing errors with high regularity into a database, we focused on improving the latter three drivers (figure 1).

Figure 1

Three critical project drivers and their related Plan Do Study Act cycles.

Optimise data analysis (PDSA ramp #2)

Identifying the true prescribing errors from the many pharmacy interventions recorded in the pharmacy database was a challenge. The first Plan Do Study Act (PDSA) cycle tested an electronic dashboard tool that filtered the raw error data from the database. We learned (1) the dashboard was too cumbersome for the pharmacist to use and (2) the pharmacist was able to accomplish the same task by modifying her existing workflows. The second PDSA cycle tested the new workflow, and by the third PDSA cycle, the process for identifying individual prescriber's prescribing errors from the database was streamlined, choosing specificity over sensitivity in order to minimise false positive feedbacks. The implementation team (one pharmacist, one neonatal attending, one neonatal fellow, one research assistant) met biweekly to review all prescribing errors and determine how the feedback message should be worded in order to minimise ‘blame’, offer constructive recommendations, and encourage prescriber responses.

Determining feedback delivery strategy (PDSA ramp #3)

We conducted a series of PDSA cycles with provider groups to test iterative versions of the feedback strategy in order to refine its content and presentation. We learned that carefully structured electronic emails that were short, personal, informative and constructive worked best. To maximise read, the message was placed within the body of the email instead of an attachment. The subject line was consistently the same, emphasising medication safety. We avoided reporting summary data, but rather gave details of specific errors and advice on how to avoid them in the future. A disclaimer (created by residents) reinforced the non-punitive intent of the feedback and invited comments from prescribers. In total, two emails detailing specific errors and recommendations were sent out biweekly: (1) ‘General Feedback on error trends and themes’ sent to all fellows, nurse practitioners and physician assistants and (2) the ‘individual feedback’ sent to individual frontline clinicians prescribing the errors. Although residents were initially excluded for logistic reasons, they were included several months later to receive the general feedback.

Ensuring bidirectional communication (PDSA ramp #4)

Each email invited prescribers to voice their concerns about potential system issues that contributed to errors. We investigated these concerns and quickly consulted the process owners of these systems, and together made efforts to improve the system.

Implementation

The implementation team educated frontline fellows, nurse practitioners and physician assistants on the intent and content of the feedback programme using emails and in-person meetings, and encouraged responses regarding system vulnerabilities. Many frontline members were already familiar with the feedback programme, having been involved with its development. The system was implemented in March 2010. Every 2 weeks, the team reviewed prescribing errors, looked for common error themes, and sent out the feedback communications. Table 1 contains examples of prescription errors and the feedback provided.

Table 1

Sample prescription errors

Erroneous prescription	Error category	Resolution
IV morphine rate changed from 0.015 to 0.1 mg/kg/h	Units	Overdose based on formulary. MD changed dose to 0.01 mg/kg/h
IV midazolam rate change from 0.1 to 0.9 mg/kg/h	Units	MD intended rate change to 0.09 mg/kg/h
Rotavirus live vaccine 2 ml IM ×1	Route	MD changed order to PO route
IV hydromorphone 0.06 mg/kg/h	Formulary	Overdose based on formulary. Dose corrected to 0.006 mg/kg/h
IV cefotaxime 250 mg every 12 h	Monitoring	Dosing decreased to 125 mg due to declining renal function
Hepatitis B vaccine IM ×1	Duplicate therapy	Patient previously received vaccine. Order discontinued

Sample errors taken from feedback emails over the course of the program.

h, hour; IM, intramuscular; IV, intravenous; kg, kilogram; MD, doctorate of medicine; mg, milligram; ml, milliliter.

Evaluation and analysis

The primary outcome was number of days between narcotic errors and antibiotic prescribing errors. This outcome was represented over time in a time-to-event chart (t-chart), where the y-axis displayed the number of days between consecutive errors arranged in chronological order. The t-chart was chosen over a u-chart (i.e, error rates) for 2 reasons: (1) we were unable to consistently estimate the number of prescriptions (denominator) every 2 weeks and (2) the t-chart is designed to track rare events such as prescribing errors (approximately 0.5% to 1% of prescriptions in the NICU). T-chart control limits were calculated and Shewhart rules for special-cause signals were applied to identify significant shifts in the mean and trends of days between errors. Interventions were directly annotated onto the charts.

To assess whether emails were read (ie, process metric), our team utilised the ‘Read Reply’ function in the email system. The percentage of responses was measured over time by calculating the number of responses over the total number of prescribers who received the email. The email was re-sent to those who did not respond. As a balancing metric, we also tracked the time required to (1) identify and discuss errors and (2) construct and send the emails.

Results

A prescribing error feedback programme was successfully implemented by using the model for improvement for its development and refinement, and engaging frontline clinicians in the process. The final iteration of the prescriber error feedback programme synergised (1) personnel (ie, a multidisciplinary implementation team), (2) process (ie, for analysing data and delivering feedback) and (3) tools (the pharmacy's electronic prescriber error database and email system). Figure 2 is a swim-lane process diagram illustrating how the synergy worked, delivering reports to prescribers that summarised (1) trends in prescribing errors happening in the NICU and (2) their own errors over the preceding 2 weeks.

Figure 2

Prescriber feedback workflow process (A) prescriber enters order, (B) pharmacist reviews and discovers error, (C) team reviews error database every 2 weeks, (D) feedback messages created and sent to prescriber, (E) prescriber receives, reads and may respond to the feedback, (F) team reviews and uses feedback to improve system.

Over the 17 months of the study, pharmacy-intercepted narcotic prescription errors were reduced by 83% after the programme began, increasing the number of days between errors from 3.94 to 22.63 days (figure 3). This sustained improvement triggered a special-cause signal on the control chart. Most errors were related to dosing. The antibiotic prescribing error rate was unchanged over the study with an average of 2.14 days between errors (figure 4). As a control for the ‘opportunity for error’, we were unable to identify the number of daily orders, but as a surrogate, did track average daily census per month. The census, on average, increased over the length of the study with a minimum of 63.2 patients in April 2010 and 74.9 in December 2010.

Figure 3

Days between pharmacy-intercepted narcotic prescribing errors. Feedback program implemented 3 January 2010 in the setting of other initiatives (hospital safety behaviour training, and verbalise medication units completely during rounds).

Figure 4

Days between pharmacy-intercepted antibiotic prescribing errors Feedback program implemented 3 January 2010 in the setting of other initiatives.

The email read reply rate was 8% initially and reached a steady state of 40%. Reviewing the error database and verifying true errors took the pharmacist, on average, 1 h and 15 min. The team's discussion on errors and appropriate feedback content took, on average, 13 min and 5 s weekly. Sending group and individual emails took, on average, 18 min and 57 s. In total, the average time spent in the feedback of medication errors to prescribers was 1 h 43 min and 16 s every 2 weeks.

This programme allowed opportunities for prescribers to voice their concerns about vulnerabilities in the prescribing process, resulting in changes to both the online formulary and computerised prescribing process. For instance, the option of ordering rotavirus vaccine intramuscularly in the CPOE was removed 1 week after a prescriber receiving feedback remarked that the CPOE system should never have allowed rotavirus vaccine to be ordered intramuscularly. In another example, specific education was done after prescribers expressed difficulty in navigating the electronic formulary.

Implementation problems

On occasion, when team members were unavailable for unforeseen reasons, parts of the process could not be carried out. Therefore, some critical processes in our feedback system require redundancies so the system can continue working when one part fails. Hence, cross-training among members of the team on sending emails, data management and facilitating meetings was conducted.

Discussion

Initiation of a prescriber-directed error feedback programme was associated with an 83% reduction in narcotic prescribing errors, an elimination of approximately 5–6 erroneous narcotic prescriptions per month. Interestingly, the rate of overall prescribing errors and antibiotic prescribing errors were unchanged.

Feedback content is critical

We have shown that feedback may reduce some types of prescribing errors if the information is concise and actionable. The timing, format and value of information to recipients are critical factors to consider when reporting errors.24 25 Adjusting the content and thematic delivery of the messages according to suggestions by prescribing recipients was critical. When prescribers found the message too long and complex, they read them less. Hence, shorter, more focused messaging appears to work best.

Performance feedback and safety culture

Feedback can prevent failures or defects in the patient care system, and it also contributes to a culture of safety within the unit. Past studies have shown that audit and feedback can provide the framework for a proactive safety culture. Jenkins et al showed that a multiprofessional regional quality improvement network of NICUs in Ireland provided the framework for a culture of regular evaluation leading to increased awareness among clinicians.26 Carefully implemented real-time patient safety audits can foster a blame-free ‘culture of patient safety’.27 Individualised feedback helps prescribers learn directly from their mistakes, while group feedback permits the entire unit to learn from each other. Encouraging providers to respond to the feedback completes a full circle of shared responsibility, and generates unit and hospital-wide changes to the formulary and CPOE system. Furthermore, feedback provided a vector to communicate and highlight vulnerabilities in medication safety that may change over time.

Performance feedback works, sometimes

While performance feedback helped reduce narcotic prescribing errors, the same feedback strategy did not reduce antibiotic prescribing errors and overall errors. Examination of workflow differences between narcotic and antibiotic prescribing suggests antibiotic ordering in the neonate requires more complex decision making, as one must take into account the patient's day of life, gestational age, dosing regimen, renal function and drug levels—and most antibiotic errors were related to improper accounting of these variables in dosing. Although the proper drug regimens are outlined in the electronic formulary, they are not easily available at the time of prescribing. In Reason's language of errors, antibiotic prescribing errors are likely ‘mistakes’ or errors where an action goes as intended but is the wrong one. Many narcotic prescribing errors were closely related—likely ‘lapses’ or ‘slips’ where there is an error of execution.28 We hypothesise that performance feedback may reduce errors of execution (ie, 10-fold overdoses) while having less of an impact on a multistep complex prescribing process such as antibiotic prescribing. From a systems standpoint, antibiotic prescribing may require additional system fixes, such as easier access to electronic formulary at the time of prescribing, to drive the desired behaviours.

Limitations

It is difficult to discern effects of other quality improvement work happening in the unit (ie, provider turnover, independent increase in safety behaviours) that may have contributed to decreasing the narcotic prescribing error rate. In the spirit of improvement, it would be counterproductive to halt coexisting improvement efforts. Prior to this study, there was a unit-wide effort to change communication around narcotic prescribing on rounds by emphasising correct units. The change in narcotic prescribing errors, however, did not occur until several weeks after this initiative, making it less likely to be the primary reason for improvement.

Coincident to this study, there was an organisational push, and outside consultants were brought into the hospital to promote overall safety behaviours. To this end, we would have expected an overall reduction in prescribing and antibiotic errors. It is possible that the complexity of prescribing a certain medication (ie, antibiotics vs narcotic) is important in shaping the impact of improvement efforts. Of note, although residents continued to rotate through the unit on a monthly schedule, their class and patient load remained consistent throughout the course of the study. There was also minimal turnover in the other direct care providers and physicians.

Our process metric has limitations. We utilised the ‘read report’ function in the email system, and although there was significant discussion within the unit regarding the emails, our read-response rate remained below our goal. Furthermore, our process metric, although documenting the receipt of an email does not confirm full reading and comprehension of the material. It is difficult to verify whether those who did not ‘read reply’ indeed, did not read the emails, since this method relied on the receiver to ‘actively read reply’.

Our analysis did not stratify by prescriber's role and level of training and, therefore, cannot determine whether any particular type of prescriber made most of the errors. In addition, we would have liked to understand the errors in various NICU populations, level of acuity and census. We are currently building the necessary data infrastructure to enable such analysis.

Conclusion

In conclusion, direct provider feedback on medication prescribing errors does not require significant time investment, it can be performed in a non-punitive manner, and may decrease the incidence of narcotic prescribing errors.