Antimicrobial stewardship: a review of prospective audit and feedback systems and an objective evaluation of outcomes.

Antimicrobial stewardship is an emerging field currently defined by a series of strategies and interventions aimed toward improving appropriate prescription of antibiotics in humans in all healthcare settings. The ultimate goal is the preservation of current and future antibiotics against the threat of antimicrobial resistance, although improving patient safety and reducing healthcare costs are important concurrent aims. Prospective audit and feedback interventions are probably the most widely practiced of all antimicrobial stewardship strategies. Although labor-intensive, they are more easily accepted by physicians compared with formulary restriction and preauthorization strategies and have a higher potential for educational opportunities. Objective evaluation of antimicrobial stewardship is critical for determining the success of such programs. Nonetheless, there is controversy over which outcomes to measure and there is a pressing need for novel study designs that can objectively assess antimicrobial stewardship interventions despite the limitations inherent in the structure of most such programs.

Introduction

The concept of antimicrobial resistance was already known even at the very nascence of modern antibiotics—it was famously sounded off by Alexander Fleming himself during his Nobel lecture in 1945. However, the subsequent decades from 1950s to 1970s saw the development and proliferation of multiple new classes of antibiotics, and this, coupled with the excellent safety profiles of most antibiotics, have resulted in lax antibiotic prescribing standards and significant inappropriate antibiotic usage in many parts of the world. In the past decade, there is accumulating evidence linking levels of antibiotic prescription to resistance,, and a paradigm shift framing antibiotics as precious and potentially finite rather than limitless resources was inevitable. Multiple professional and civic organizations, including the World Health Organization, have put forward position papers and recommendations on preserving the beneficial impact of antibiotics, and antimicrobial stewardship is one of the slew of multifaceted interventions recommended for preserving the effectiveness of current and future antibiotics.- It is important to note that antimicrobial stewardship by itself cannot alleviate the problem of antimicrobial resistance. It is a small but necessary part of a larger whole that includes regulatory policies and interventions to control antibiotic use in livestock, educational measures and interventions to stimulate the research and development of new classes of safe and effective antibiotics.,,

Antimicrobial stewardship is an emerging field that is currently loosely defined. Essentially, strategies and interventions aimed toward improving appropriate prescription of antibiotics in humans in all healthcare settings may be considered part of “antimicrobial stewardship” (Table 1)., Antimicrobial stewardship programs (ASPs) are typically run by multidisciplinary teams comprising a mix of physicians, clinical microbiologists, pharmacists, nurses and/or administrative staff and the interventions implemented may differ radically depending on the healthcare and cultural setting., Nonetheless, the aims of every ASP are similar: besides attempting to reduce antimicrobial resistance rates and preserve current antibiotics, ASPs also aim to improve patient outcomes and safety and reduce the financial costs associated with inappropriate antibiotic prescription., It remains unclear at present which interventions work best for achieving the multifold aims of an ASP, and the methods for assessing outcomes are themselves fraught with issues. It is likely that the effectiveness of each intervention may change depending on the underlying healthcare structure and socio-cultural context, and a “care bundle” of antimicrobial stewardship interventions may be more effective than the sum of individual parts.

Table 1. List of interventions considered as part of antimicrobial stewardship.,,

Intervention*	Description/comment	Healthcare setting
Formulary restriction	Antibiotics may be prescribed only:• For certain approved clinical indications• By certain physicians (i.e., infectious diseases specialists)	Inpatient/outpatient
Drug preauthorization	Permission (from ASP team member or infectious diseases specialist) required for release of certain antibiotics. Often implemented together with formulary restriction.	Inpatient/outpatient
Prospective audit and feedback	Case review by trained ASP team member and feedback of recommendations if reviewed antibiotics are deemed to be inappropriately prescribed. Labor-intensive.	Inpatient
Prescriber education	More effective as a supplementary strategy to other interventions.	Inpatient/ outpatient
Patient education	Usually focus groups or mass media campaigns.	Outpatient
Clinical guidelines	Treatment protocols for various infections – may be institution-specific	Inpatient/ outpatient
Clinical decision support systems	Information technology systems for improving antibiotic prescription. Requires existing electronic records and electronic prescribing system to be effective.	Inpatient/ outpatient
Point of care diagnostic tests	Mostly undergoing research evaluation. Diagnosis of non-bacterial etiologies may help reduce antibiotic prescription.	Inpatient/ outpatient
Microbiology laboratory susceptibility reporting	Selective reporting of susceptibility profiles for positive cultures may dramatically alter prescribing patterns of physicians.	Inpatient/ outpatient
Antimicrobial cycling	Substitution of selected antibiotics over pre-defined periods. Little clear evidence for efficacy.12	Inpatient

Note that we consider parenteral to oral antibiotic conversion and dose/duration optimization to be subsets of interventions related to prospective audit and feedback or clinical guidelines.

In this article, we review prospective audit and feedback interventions—one of the two core ASP strategies recommended by the Infectious Diseases Society of America (IDSA) that has been shown to reduce the inappropriate use of antimicrobials—and also the issues surrounding the implementation and objective evaluation of such programs.

Prospective Audit and Feedback in Antimicrobial Stewardship

The concept of prospective audit and feedback is not new, and the older terminology is “immediate concurrent feedback.” One of the earliest descriptions of its implementation comes from the Mercy Catholic Medical Center in Philadelphia, and dates back more than a quarter of a century. The acceptance rate for recommendations made by the infectious diseases physician then was 62.8%, with estimated cost-savings of US$9,758.60 over an 11-week period for the pilot project.

The prototypical prospective audit and feedback ASP team comprises of a physician [usually an infectious diseases (ID) physician or a clinical microbiologist] and clinical pharmacists. Seto and coworkers showed that a trained nurse could take on the role of the pharmacist, whereas Laible and coworkers successfully employed pharmacy residents and students for the same role. It is important to note that appropriate training should be provided for the ASP personnel to attain expertise in appropriate antibiotic use, although it is unclear at present whether employing fully fledged pharmacists would result in significantly better outcomes in a prospective audit and feedback ASP. The role of specialist ID pharmacists in ASPs is undefined and evolving—it is conceivable that they will be able to take over some if not all of the functions of the physician in a prospective audit and feedback ASP.

In a one-step prospective audit and feedback ASP, an ID physician or clinical pharmacist directly audit targeted antibiotics and provide feedback during clinical rounds. In a two-step review method,,,, the pharmacist or nurse will review the case individually. Thereafter, they will present cases that fulfill pre-set criteria for intervention to the physician for vetting, with recommendations for change or discontinuation of antibiotics conveyed to the primary physicians via written forms or direct verbal communication. The general workflow for a two-step review method employed at our institution is summarized in Figure 1, along with the other major antimicrobial stewardship strategy of formulary restriction and pre-authorization.

Figure 1. General workflow schematic for a two-step prospective audit and feedback strategy as well as formulary restriction and preauthorization strategy for antimicrobial stewardship. Added details for prospective audit and feedback pertain to the workflow at the authors’ institute.

There may well be a trade-off between the higher likelihood of acceptance of recommendations for verbal communication vs. the reduced time taken for written communication. However, maintaining a channel of communication for bidirectional feedback is important for sustaining support for the ASP.

The selection of cases for auditing can be performed via a census based on defined medical or surgical disciplines and/or defined antibiotics. A compilation of consumption in the form of defined daily doses (DDD) or days of therapy (DOT) may be used to determine high prescription areas to maximize the effect of interventions and this should be reviewed and updated over time.

The primary advantage of a prospective audit and feedback strategy is that doctors do not perceive the loss of prescribing autonomy in view of the fact that acceptance of recommendations is voluntary.,, It is therefore more acceptable to doctors and less vulnerable to active opposition. This strategy also provides opportunities for education through the feedback mechanism, and can be customized to the size of the institution depending on the resources available. Individualization of therapy is also facilitated by this strategy, allowing socio-economic issues, drug-disease interactions and unique clinical conditions to be taken into account.

There are more models of operation that review antibiotic prescription only after 48 to 72 h, allowing more clinical information—including bacterial culture results, radiological results and response to initial therapy—to be made available before interventions are considered.,, This will circumvent the potential for delay in initiating culture-directed therapy, which is in line with the overall aims of optimizing clinical outcomes and patient safety. However, in this regard, it may be important to concurrently improve the overall microbiological support and reporting system—in some institutions, microbiological reports are only released or confirmed as finalized after more than 96 h from the time of culture. For parenteral to oral conversion of antibiotics and surgical antibiotic prophylaxis, it would be more appropriate to audit cases on the day of antibiotic prescription.

Although most physicians involved in antimicrobial stewardship do not directly review patients in prospective audit and feedback systems, their presence can be invaluable for the success of the ASP where critically ill or clinically complex patients are involved. A prospective quasi-experimental study for adults in an intensive care unit (ICU) showed that the direct involvement of an infectious diseases physician in thrice-weekly interdisciplinary rounds had a higher impact on the ICU team’s behavior, facilitating interactive education and real time discussion as compared with either intervention by the critical care pharmacist or written communication alone. Direct infectious diseases physician review is also applied in the second stage of the prospective audit and feedback model practiced in Hong Kong where the appropriateness of prescription is difficult to judge based on available information.

The disadvantages of the prospective audit and feedback strategy are evident. It is manpower- and labor-intensive, functioning in essence as a second human oversight and check on antibiotic prescription. As such, depending on the size of the ASP, it is also potentially the most costly antimicrobial stewardship strategy, although existing publications suggest that financial sustainability is not an issue.,, A moderate degree of training and familiarization is necessary for team members of prospective audit and feedback ASPs because the specific nature of antibiotic prescription review and/or recommendation is not a part of routine pharmacy or nursing work, and physicians may be uncomfortable with making recommendations without laying eyes on the patients in question.

In view of the fact that acceptance of recommendations is generally voluntary, prospective audit and feedback is less likely to achieve immediate and significant reductions in antibiotic prescription compared with the other major active antimicrobial stewardship strategy of formulary restriction and preauthorization, particularly if reviews occur only 48 to 72 h post-antibiotic prescription. Some barriers to higher acceptance rates for this strategy include the fact that universally, there are physicians that are not keen to de-escalate antibiotics despite microbiology results suggesting that narrower-spectrum antibiotics can be prescribed, as the patients had responded to the initial empiric antibiotics. Physicians also remain concerned about the reliability of an ASP recommendation especially when the patient had neither been seen nor examined by the ASP team.

Clinical decision support systems have also been adopted by many institutions in order to support ASP efforts. These systems have been shown to decrease adverse events, shorten length of stay, decrease cost and improve empiric, therapeutic and surgical prophylactic use of antimicrobials,- However, implementation of CDSS has various barriers and limitations.- Current systems should only serve as a support for clinicians and not a replacement for human ASP personnel in view of the complexity of patient factors.

Lastly, it should be emphasized that antimicrobial stewardship is only one of the strategies to minimize development of resistance. In order to successfully curb resistance, a multi-pronged approached is required, involving the co-operation of antimicrobial stewardship, infection control and healthcare policy makers.

Outcomes for Antimicrobial Stewardship Interventions

McGowan excellently reviewed the issues surrounding the task of objectively evaluating antimicrobial stewardship interventions recently. Because these interventions—including prospective audit and feedback—are costly and intrusive for the large part, it is critically important to determine if each ASP is cost-effective. However, reduction of evaluation to a single summary outcome statistic (cost-effectiveness) is hardly ever performed because of the complexity of analysis and the difficulty in positing value to a public good such as reduction in prevalence of antimicrobial resistance. Where cost-effectiveness studies have been attempted, they are directed toward single clinical conditions such as bacteremia. Most studies have examined cost reduction of antibiotic therapy, which is a secondary objective of antimicrobial stewardship, and have generally shown anywhere from significant to dramatic cost-savings for the healthcare payers, be they individual patients, private insurance or government.

As the cost of antibiotics is also affected by inflation, expiry of patent and drug shortages, antibiotic consumption may be a better measure of direct cost. Currently, there are two main measures of consumption—defined daily dose (DDD) and days of therapy (DOT). Both methods are standardized to a conventional denominator—1,000 patient days. As each measure has their limitations,,, comparison can only be made if the measure used is constant. Studies have reported that comparison of a common antibiotic in a specific discipline or the consumption of antibiotics in a specific patient care area such as the intensive care unit, can uncover more meaningful relationships.

At the next level, antimicrobial stewardship interventions and programs as a whole can be assessed to determine if their aims are met. However, the choice of metrics is complex and it is unclear at present if there is a “right” set of outcomes to be measured. Crude clinical outcomes that measure patient safety in terms of patient survival and length of stay are the most objective and practical, but they have thus far been infrequently assessed in publications on antimicrobial stewardship.,, ASP has been shown to reduce the average length of hospital stay, 14-d re-infection rate and infection-related re-admissions. However, most of the studies assessing 30-d mortality have shown little or no difference,, and the reasons are clear—a large number of factors affect mortality (and length of hospitalization) and the independent effect of the usual antimicrobial stewardship interventions (i.e., choice of a narrower-spectrum antibiotic or shortened duration of therapy) is negligible. An expert panel recently proposed more specific metrics for clinical outcome such as mortality related to antimicrobial-resistant pathogens and “conservable days of therapy” (defined as avoided unnecessary treatment days based on widely accepted targets and benchmarks) instead of length of hospitalization. However, both are relatively subjective measures and may lack real significance to the patients.

Drug outcomes are very commonly used as surrogate outcome measures for antimicrobial stewardship, and can be further categorized into timeliness, correct antibiotic choice, dose and duration of therapy. Appropriate antibiotic therapy—where the pathogen is susceptible to the antibiotic—significantly affects clinical outcomes,, but any difference in outcomes between prescribing a narrow- or a broad-spectrum antibiotic may be slight when the organism is sensitive to either drug. The duration of therapy is relevant in that it may directly affect length of hospitalization, and ASPs can play a role in preventing over- and under-treatment of infections. Increasing evidence supports shorter courses of antibiotics even in critically ill patients,, while for deep-seated infections like endocarditis or osteomyelitis, an adequate duration of therapy is pivotal. Nonetheless, is important to note that not all drug and surrogate outcomes represent significant clinical outcomes.

The monitoring of adverse drug events as a clinical outcome is important as it also represents patient safety. Nonetheless, perhaps because it is difficult to obtain such information accurately, very few studies have described the impact of antimicrobial stewardship on adverse drug events.,,,

The impact of antimicrobial stewardship on antimicrobial resistance is of key interest, as this is the primary aim of stewardship. Monitoring antimicrobial resistance at different areas of an institution may provide more detailed insights. For example, the trend of resistance in the intensive care may differ markedly from the general wards. Although there are publications where no impact on antimicrobial resistance rates were reported,,,, the general trend has been that implementation of ASPs has led to reduction of resistance,,- but publication bias cannot be excluded.

The issue of what antimicrobial resistance parameter to measure is controversial. Changes in antimicrobial resistance based on hospital-wide antibiograms may not be observed after decreasing antibiotic usage, leading certain experts to recommend against their use for the purposes of evaluating ASP outcomes. There have also been suggestions to measure changes in minimum inhibitory concentration (MIC) or by genotyping of isolate-specific resistance patterns, although such information is costly to obtain and represent technological hurdles for smaller hospitals. Based on a modified Delphi-consensus technique, an expert panel suggested measuring the change of resistance by monitoring the number of patients with specific antibiotic-resistant organisms (divided by the total number of patients admitted to a ward or unit of interest), an approach that we favor in view of its lower measurement cost and because it factors in potential denominator changes. Recently, a drug resistance index (DRI) that can be used to communicate the average effectiveness of a set of antibiotics was proposed. This index is comparable to consumer price indices and stock market values, is relatively easy to measure, and as a concept may help non-infectious diseases and non-medical staff in visualizing and understanding the changes and issues with antimicrobial resistance.

Objective Evaluation of Antimicrobial Stewardship Interventions

The objective evaluation of outcomes poses a particularly difficult problem for studies of antimicrobial stewardship interventions. Multiple factors—not just appropriate antibiotic prescription—determine antimicrobial resistance rates and clinical outcomes. Controlling of the confounding factors is difficult for reasons given below.

• Designing a proper randomized clinical trial for antimicrobial stewardship is very complicated, especially for a prospective audit and feedback strategy where allocation concealment is impossible and the inherent education in the feedback given can lead to better antibiotic prescribing practices over time.,

• “Care bundles” of different antimicrobial stewardship interventions are often implemented concurrently (i.e., education of prescribers along with either formulary restriction or prospective audit and feedback) and as with other care bundles for infection prevention, it becomes virtually impossible to piece out the contribution of each intervention.

• For antimicrobial resistance, the effects of antimicrobial stewardship may take time to develop, but there are inherent difficulties in assessing such outcomes over time.

Thus, the great majority of published antimicrobial stewardship studies, including most publications detailing prospective audit and feedback strategies, have been of quasi-experimental design—typically before- and after-implementation studies where “treatment” allocation and other potential confounding factors are not controlled.,, The lack of strong scientific evidence for most antimicrobial stewardship interventions has led to confusion and disagreement about their effectiveness, and both experts as well as professional bodies continue to reiterate the need for more and better research.,,

Conclusion

The impetus for the implementation of ASPs on an almost global level has arisen in response to the growing threat of antimicrobial resistance amidst the diminishing pipeline of new antibiotics. It is likely that support for antimicrobial stewardship will continue to grow in the coming decade, and among the various interventions, the prospective audit and feedback strategy will probably be the most widely implemented in view of its clear advantages—particularly with regards to lack of opposition from prescribers. Because of the nature of antibiotic prescribing, it may prove more effective to incorporate behavior change strategies in addition to current existing interventions, particularly for the prospective audit and feedback strategy. However, there is an urgent need to standardize outcomes measured as well as develop novel study designs that can objectively assess antimicrobial stewardship interventions despite the limitations inherent in the structure of most ASPs.