Impact of Education and Peer Comparison on Antibiotic Prescribing for Pediatric Respiratory Tract Infections.

INTRODUCTION: Inappropriate prescribing of broad-spectrum antibiotics is a significant modifiable risk factor for the development of antibiotic resistance. The objective was to improve guideline-concordant care for 3 common acute respiratory tract infections (ARTIs) and to reduce broad-spectrum antibiotic prescribing in ambulatory pediatric patients.
METHODS: Quality measures were developed for 3 ARTIs: viral upper respiratory infection (URI), acute bacterial sinusitis (ABS), and acute otitis media (AOM). Among 22 pediatric clinics, a collaborative of 10 was identified for intervention using baseline data for each ARTI, and 3 plan-do-study-act cycles were planned and completed. Outcomes included guideline-concordant antibiotic utilization and broad-spectrum antibiotic prescribing percentage (BSAP%). Comparison in number of diagnoses for the ARTI measures and total antibiotic prescribing over time served as balancing measures.
RESULTS: Collaborative clinics had baseline medians for appropriate or first-line treatment of 70% for URI, 53% for ABS, and 36% for AOM. To reach targets for URI, ABS, and AOM required 6, 14, and 18 months, respectively. At 42 months, performance for all 3 ARTIs remained ≥90%. BSAP% decreased from a baseline of 57% to 34% at 24 months. There was a limited effect from financial incentives but a significant decrease was noted in total antibiotic utilization. Diagnosis shifting may have occurred for URI and ABS while the rates for diagnoses for AOM declined over time.
CONCLUSIONS: Through education and peer comparison feedback, guideline-concordant care for 3 ARTIs in collaborative clinics improved and remained beyond above targets and was accompanied by reductions in BSAP% and total antibiotic prescribing.

INTRODUCTION

Outpatient settings account for ~60% of antibiotic expenditures in the United States,[1] and 30%–50% of antibiotic prescriptions for acute respiratory tract infections (ARTIs) for all ages may be inappropriate or unnecessary.[2-4] For pediatric patients in the ambulatory setting, 20% of visits result in an antibiotic prescription while ~50% of antibiotics prescribed are broad-spectrum.[5]

Exposure to antibiotics may lead to adverse reactions, Clostridioides difficile infections, and antibiotic resistance. Inappropriate antibiotic prescribing (including the use of broad-spectrum antibiotics when narrow-spectrum agents are more appropriate) is a significant modifiable risk factor for the development of antibiotic resistance.[6] Broad-spectrum compared with narrow-spectrum antibiotic prescribing in children with ARTIs is associated with higher rates of adverse events but not associated with better clinical outcomes.[7]

In 2015, the White House released a national action plan to combat antibiotic resistance with a goal of reducing inappropriate antibiotic prescribing in outpatient settings by 50% by the year 2020.[8] A variety of interventions have been evaluated to improve antibiotic prescribing for ARTIs in primary care.[9] In the course of developing an ambulatory antimicrobial stewardship program in 2014,[6] we chose to focus on clinician education, audit, and feedback,[6,10-14] targeting improvement in appropriate antibiotic prescribing and first-line antibiotic therapy for specific ARTIs.

Specific Aims

The aim was to improve guideline-concordant antibiotic prescribing in pediatric patients for 3 ARTIs: viral upper respiratory tract infection or common cold [upper respiratory infection (URI); patients 3 months–18 years of age], acute bacterial sinusitis (ABS; patients 1–18 years of age), and acute otitis media (AOM; patients 6 months–12 years of age), with targets of ≥90% for URI and ≥80% for ABS and AOM within 24 months.

METHODS

Setting/Context

In January 2014, Novant Health medical group included 22 pediatric clinics located in Winston-Salem and Charlotte, NC. These clinics participated continuously and included 112 physicians and 53 advanced practice clinicians (APCs). Novant Health uses Epic (EpicCare, Verona, WI) as an electronic health record (EHR) and, through the analytics and informatics group, builds, validates, and records data sets from which quality and safety measurement data are abstracted.

Design

A pre-post, quasi-experimental, quality improvement initiative was designed using the Model for Improvement.[15] Age ranges were chosen to be consistent with the then current published guidelines for 3 ARTIs.[16-18] Targets were set for URI at 90%, using Healthcare Effectiveness Data Information Set (HEDIS®) 2013 90th percentile benchmarks as a guide[19] and, with no HEDIS® measures for ABS and AOM, at 80%, consistent with targets suggested by Hersh et al.[20]

Because narrow-spectrum agents (amoxicillin or penicillin) for group A streptococcal pharyngitis treatment, as recommended,[21] were used 95%–99% of the time by clinicians in these 22 clinics (H. W. Clegg, unpublished data, September 2018), appropriate pharyngitis care was not evaluated.

Based upon guidelines from the American Academy of Pediatrics[16-18] and the Infectious Diseases Society of America,[22] customized quality measures were developed (see Supplemental Digital Content 1, available at http://links.lww.com/PQ9/A111) and validated by selective, manual chart review of EHRs.

Data collection began in January 2014. In July 2014, a baseline 6-month review was conducted, and pediatric clinics were stratified into 2 groups: those which did not meet the preset target of 80% for 2 of 3 ARTI measures (the collaborative, n = 10) and all other pediatric clinics (comparison group to control for temporal trends, n = 12).

For ABS and AOM, patients were included if an antibiotic was provided at the time of the illness encounter or within 3 subsequent days but excluded if they received an antibiotic within the prior 60 days, or a competing diagnosis was documented for which an antibiotic is usually given.

Interventions

The project team consisted of 2 quality specialists, a pharmacist, and 2 physicians (1 pediatrician and 1 pediatric infectious disease specialist). Key drivers were identified (Fig. 1) and, from these and guidelines[16-18,22] a strategy was identified to compare pediatric clinic scores before and after the intervention began in 2014.

Fig. 1.

Key drivers diagram.

The intervention was aimed at clinicians to enhance appropriate antibiotic prescribing for URI (no antibiotic given when a visit diagnosis of URI was listed with no additional diagnosis listed warranting an antibiotic)[16] and for the use of first-line agents for children and adolescents with ABS (amoxicillin or amoxicillin-clavulanate)[17,22] and AOM (amoxicillin)[18] in pediatric primary care clinics.

The intervention included iterative plan-do-study-act cycles:

(1) Educational materials were provided for all pediatric clinicians beginning in June 2014. Clinicians in all 22 clinics received a sheet with tips for score improvement and condensed clinical guidelines for the 3 ARTI measures, emphasizing appropriate documentation of visit diagnoses. A “take and hold” (or “safety net”) prescription strategy[23,24] was discouraged as this is considered a prescription on day 1 when an order is entered in the EHR. A “wait and see” or observational approach for selected circumstances for ABS or AOM was not discouraged as this is a well-recognized strategy to diminish antibiotic utilization.[17,18] The tip sheet suggested that clinicians address parent/patient expectations and to use templated education in the EHR after-visit summary addressing 5 frequently-asked questions about common colds and antibiotics.[25] The tip sheet was provided monthly to all clinics along with performance feedback.

(2) In September 2014, for the collaborative clinics (n = 10), a 1-hour, in-person, educational visit was held with each clinic’s lead clinician and clinic administrator to discuss the clinical guidelines, measure definitions, baseline performance scores, and improvement methods (academic detailing[26,27]). We asked that they provide this information to their clinic’s prescribing clinicians and to discuss performance at monthly meetings.

Clinical decision support was not utilized for an intervention since our EHR had just been introduced in 2013.

Also, in September 2014, performance feedback began for the 3 ARTI measures. An analyst developed clinic comparison reports for all clinics to view and clinician-specific reports for collaborative clinicians to view only for their clinic (see Supplemental Digital Content 2, available at http://links.lww.com/PQ9/A112 and Supplemental Digital Content 3, available at http://links.lww.com/PQ9/A113). A physician prepared and sent the monthly email messages with appropriate reports for each clinic.

In January 2015 and January 2016 (and ending in December of each year), the medical group’s nonproductivity compensation program introduced financial incentives for clinicians, designed for them to be actively engaged in PDSA work and score improvement for selected measures. This afforded the opportunity to determine if a financial incentive created additional improvement. For 2015, all pediatric clinics adopted the acute URI measure as a nonproductivity compensation metric.

For 2016, all pediatric clinics adopted the ABS measure as a nonproductivity compensation metric tied to implementation of a newly developed clinical care pathway for ABS in children and adolescents 1–18 years of age (very similar to the 2014 intervention material).

Data Collection

Following the 6-month baseline data review for January–June 2014, performance was tracked for the 3 ARTIs for 42 months. Visit-level data included ICD-9 codes (and starting in October 2015, ICD-10 codes, see Supplemental Digital Content 1, available at http://links.lww.com/PQ9/A111) associated with a patient’s illness encounter and listed as a visit diagnosis. Antibiotic prescribing was determined by search of medication orders in the record associated with the encounter or within 30 days prior (if URI diagnosis) or within 60 days prior (if ABS or AOM diagnosis) and 3 days subsequent to the encounter. Illness encounters were defined as evaluation and management visits for new patients (with codes 99201–99205) and for established patients (with codes 99212–99215). Broad-spectrum antibiotic prescribing percentage (BSAP%) was determined monthly by enumerating all antibiotics given, stratifying by narrow and broad spectrum (see footnote, Table 1), and dividing by the total number of antibiotics prescribed for any condition, not limited to the 3 ARTIs. For this calculation, patients were excluded if their record showed an allergy to an antibiotic listed as narrow or broad spectrum and/or one of the listed antibiotics had been given in the prior 60 days.

Table 1.

Broad-Spectrum Antibiotic* Usage, January–June, 2014–2017

Two balancing measures were used. One, the numbers of diagnoses per illness encounter were recorded for the 3 ARTIs to evaluate possible diagnosis or code shifting. Two, the numbers of antibiotics prescribed for any condition divided by the total patient illness encounters for the clinic were recorded, without exclusions for allergy or receipt of antibiotics in the prior 60 days.

Statistical Analysis

The primary outcome was the proportion of illness encounters in collaborative clinics when appropriate treatment for URI and first-line treatment for ABS and AOM was provided. We used monthly run charts to observe aggregate changes for these clinics compared with their baseline medians for the period January–June 2014. To evaluate BSAP% over time, we utilized a Shewhart p-chart with center-line adjustment, upper and lower control limits set at 3 sigma from the mean, and standard criteria to determine special cause variation.[15] The p-chart was constructed using Minitab 18 Statistical Software (2017) (State College, PA: Minitab, Inc., www.minitab.com). The percentages for BSAP and rates for antibiotic utilization, and for diagnoses by month were calculated for January–June of each year evaluated to control for possible seasonal variation. BSAP percentages in January–June 2016 and 2017 and diagnosis rates in 2017 were compared with those in January–June 2014 using the chi-square test (SAS Enterprise Guide 6.1, SAS Institute Inc., Cary, NC), and a P value <0.05 was considered statistically significant.

Ethical Considerations

The protocol was reviewed and approved as a quality improvement initiative by the Institutional Review Board of Novant Health Presbyterian Medical Center, Charlotte, and exempted from further review.

RESULTS

During the 4-year period, January 2014–December 2017, we evaluated 922,436 illness encounters in patients 1 month-18 years of age in 22 pediatric clinics.

Baseline medians for appropriate or first-line treatment for the collaborative were 70% for URI, 53% for ABS, and 36% for AOM. In contrast, the other 12 pediatric clinics had baseline medians of 96% for URI, 90% for ABS, and 71% for AOM.

To reach targets, the collaborative required 6 months (December 2014) for URI, 14 months for ABS (August 2015), and 18 months for AOM (December 2015). These percentages remained at or above target at 24 months, and by the end of the planned 42-month intervention phase (December 2017), performance scores for the collaborative were 94.3% for URI, 96.4% for ABS, and 93.3% for AOM (Fig. 2).

Fig. 2.

Percentage of patient illness encounters with appropriate treatment of children 1–18 years of age for 3 acute respiratory tract infections among collaborative pediatric clinicians, January 2014–December 2017.

Guideline-concordant rates for physicians and APCs were evaluated for the baseline period (January–June 2014) and compared with the same 6-month period in 2017 for the collaborative. In the baseline period, physicians under-performed compared with APCs for URI (69.1% versus 72.0%) and AOM (34.2% versus 42.3%) while physician performance exceeded APCs for ABS (56.1% versus 43.3%). For 2017, performance was nearly identical for physicians and APCs for URI (95.3% versus 95.2%) and ABS (93.8% versus 92.1%), but for AOM, physicians performed somewhat better than APCs (92.3% versus 87.3%).

For the duration of the initiative, performance for all other pediatric clinics, began and remained at ≥90% for URI and ≥80% for ABS, but these clinics required 15 months to reach and remain ≥80% for AOM (October 2015). By the end of the 42-month intervention phase, these clinics had scores of 98.3% for URI, 96.8% for ABS, and 95.2% for AOM.

BSAP percentages for the collaborative were evaluated after 24 months as planned at the outset. We saw a decrease from a baseline of 56.6% to 34.1% (P < 0.001), an absolute decrease of 22.5% (95% CI, 21.3%–23.7%), while comparing the same 6-month time periods, January–June 2014 and January–June 2016. This decrease in BSAP% was not matched by all other pediatric clinics, but these clinics started at a much lower percentage: 28.4% for January–June 2014 and 28.1% for the comparison period in 2016. For the comparable period in 2017, further declines in BSAP percentages were noted for both clinic groups (Table 1).

As shown in the p-chart for BSAP% (Fig. 3), downward shifts from the previously established center line occurred following interventions, suggesting special cause variation.[15] The 3 points for October–December 2014 represent a system in transition as education by email and in-person visits created the shift that occurred in January 2015. The final shift occurred in February 2016, leading to sustained reduction in BSAP% through 2017.

Fig. 3.

Statistical process control chart (p-chart) measuring BSAP% in children 1–18 years of age seen for an illness encounter by collaborative pediatric clinicians, January 2014–December 2017. Patients excluded are those with a recorded allergy to a listed antibiotic or who had received a listed antibiotic within the prior 60 days. Subgroup sizes: minimum, n = 998; maximum, n = 3,188.

There was a very limited effect on performance scores derived from the financial incentives as the score for the collaborative for URI was 93.4% for January 2015 when the URI incentive was introduced and for ABS was 92.5% when the ABS incentive was introduced in January 2016 (Fig. 2).

Antibiotic utilization decreased significantly in the collaborative group in 24 and 36 months and slightly increased for all other clinics (Table 2).

Table 2.

Antibiotic Utilization, January–June, 2014–2017

Among collaborative clinics (and all other clinics as well), diagnoses changed significantly from 2014 to 2016 and from 2014 to 2017, for URI, ABS, and AOM (P < 0.001 for all comparisons, chi-square test, Table 3). In particular, as predicted by the team, the rate for ABS diagnoses increased substantially. However, rather than a decrease in the rate for URI diagnoses, a parallel increase occurred beginning shortly after initiation of the June 2014 intervention.

Table 3.

Comparison of Rates for Acute Respiratory Tract Infection Diagnoses, January–June, 2014–2017

Rates for diagnoses for AOM decreased from 69.1 to 46.4 in collaborative clinics and from 67.0 to 50.8 in all other clinics (Table 3).

DISCUSSION

For the ambulatory pediatric clinics in our community-based system, marked baseline variability in adherence to clinical guideline-defined performance measures was noted for 3 ARTI measures, prompting creation of a collaborative for intervention. Education combined with monthly audit and feedback, led to improved guideline-concordant care for viral URI, ABS, and AOM that was associated temporally with significant reduction in broad-spectrum antibiotic prescribing.

These results are consistent with the findings of Gerber et al[10] who, in an ambulatory setting including 18 pediatric clinics in a cluster randomized trial, evaluated a similar intervention to improve guideline-appropriate care for ABS, streptococcal pharyngitis, and pneumonia. They also showed that withdrawal of monthly audit and feedback for 18 months led to a lack of sustained performance for their 3 measures.[28]

Reduction in BSAP percentages was likely driven by the improvement in the use of first-line agents for ABS and AOM as we strongly encouraged the use of amoxicillin as first-line for these 2 ARTIs, consistent with clinical guidelines published in 2013.[17,18]

We could not demonstrate a significant impact from the financial incentive interventions for URI and ABS because performance in the collaborative was already ≥90% for each of these measures when the financial incentives were introduced. However, for BSAP% reduction, Figure 3 shows the beginning of special cause variation in October 2014 after in-person education and in February 2016, shortly after introduction of the ABS intervention.

Although we did not expect a change in total antibiotic utilization since we did not focus on usage reduction nor on appropriate treatment of skin and soft tissue infection, urinary tract infection, pharyngitis, or pneumonia, there was significant reduction from baseline to 24 months (Table 2).

Diagnosis shifting may be observed in improvement efforts like this one (6). We anticipated that, with the intervention, URI diagnoses would decrease and ABS diagnoses would increase, yet both increased well above baseline and in parallel. We believe this is due to increased attention to appropriate coding for illness encounters which was a part of our educational intervention. However, part of the increase in ABS diagnoses may have been due to the use of the specific diagnosis of ABS when patients actually had viral URI. Others have observed diagnosis shifting and a small reduction in ABS diagnoses after education and clinical decision support to improve prescribing for ABS.[29]

Changing behavior of clinicians to improve ambulatory antibiotic prescribing can be difficult. Suggested approaches include peer comparison feedback,[6,10-14,30-32] clinical decision support,[12,29] such as electronic record alerts to best practice or requiring written justification,[14] pay for performance incentives, and communication improvement training.[31]

Peer performance comparison has been used for some time to encourage best practices by clinicians and can have a very positive impact.[32] Using this approach as our main intervention, we were able to achieve sustained performance improvement and overcome common barriers to adherence for antibiotic prescribing guidelines.[33] Because maintaining performance is difficult without ongoing feedback,[28] we continued feedback beyond the planned 24-month period and have adopted feedback as a standard stewardship method to positively influence behavior.

Peer comparison is low cost but requires a common EHR for clinics, dedicated analysts, and a trusted clinician to deliver performance feedback, requiring approximately 4–6 hours per month for this initiative.

Strengths and Limitations

Strengths of this initiative include first, the large numbers of encounters sampled and long duration of observation with sustained results as monthly feedback continued but financial incentives were withdrawn. Second, this project allowed engagement of pediatric primary care clinicians in quality improvement work, particularly equipping them with an understanding of the Model for Improvement which can be applied to subsequent projects. Finally, pediatric clinics now have an expectation of performance feedback for clinical quality measures, and this did foster healthy competition among clinicians.

These results may be limited by use of a 6-month and not 12 months and older baseline for our run chart data, but we are not aware of any reasons to think that results would differ if we had longer-term baseline data. In fact, not all pediatric clinics were on the electronic medical record in 2013 so data before January 2014 would have been incomplete electronically. Another limitation is the lack of a true comparison group that might have allowed cluster randomization. A third possible limitation is that we have limited data to determine comparability of pediatric clinics beyond patient volume and payer mix (data not shown). Yet, all clinics are general pediatric clinics as part of a larger integrated system subject to the same leadership structure, general policies, revenue cycle, and supporting work. Fourth, these results may lack generalizability as these data were derived from a single healthcare system.

Summary and Conclusions

Our initiative adds to evidence that education, audit, and feedback applied as an antimicrobial stewardship intervention in ambulatory pediatric clinics can improve guideline-concordant care for URI, ABS, and AOM, and this may lead to a reduction in prescribing of broad-spectrum antibiotics when narrow-spectrum antibiotics are more appropriate. These findings are limited to pediatric clinicians but may be applicable to family medicine clinicians who also see a large number of children and adolescents. Remaining challenges include determining whether similar improvements can be made with family medicine clinics, whether clinician-specific feedback is superior to clinic-specific feedback, and demonstrating that reduction in ambulatory broad-spectrum antibiotic prescribing can be associated with a decrease in antibiotic resistance.

ACKNOWLEDGMENTS

We acknowledge the technical assistance of Anaite Moore and Matthew Hestwood and the critical review of the manuscript by Amina Ahmed, MD and David Priest, MD.

DISCLOSURE

The authors have no financial interest to declare in relation to the content of this article.