Improved Antibiotic Prescribing Practices for Respiratory Infections Through Use of Computerized Order Sets and Educational Sessions in Pediatric Clinics.

BACKGROUND: Computerized clinical decision support systems (CDSS) have shown promising effectiveness in improving outpatient antibiotic prescribing.
METHODS: We developed an intervention in the form of EPIC (Verona, WI, USA) order sets comprised of outpatient treatment pathways for 3 pediatric bacterial acute respiratory infections (ARIs) coupled with educational sessions. Four pediatric clinics were randomized into intervention and control arms over pre- and postimplementation study periods. In the intervention clinics, education was provided in between the 2 study periods and EPIC order sets became available at the beginning of the postimplementation period. The primary end point was the percentage of first-line antibiotic prescribing, and the secondary end points included antibiotic duration and antibiotic prescription modification within 14 days.
RESULTS: A total of 2690 antibiotic prescriptions were included. During the pre-implementation phase, there was no difference in first-line antibiotic prescribing (74.9% vs 77.7%; P = .211) or antibiotic duration (9.69 ± 0.96 days vs 9.63 ± 1.07 days; P > .999) between the study arms. Following implementation, the intervention clinics had a higher percentage of first-line antibiotic prescribing (83.1% vs 77.7%; P = .024) and shorter antibiotic duration (9.28 ± 1.56 days vs 9.79 ± 0.75 days; P < .001) compared with the control clinics. The percentage of modified antibiotics was small in all clinics (1.1%-1.6%) and did not differ before and after the intervention (for all statistical comparisons, P ≤ .354).
CONCLUSIONS: A computerized CDSS involving treatment pathways in the form of order sets coupled with educational sessions was associated with a higher percentage of first-line antibiotic prescribing and shorter antibiotic duration for the outpatient treatment of pediatric bacterial ARIs.

Between 2010 and 2011, a national survey estimated that at least 30% of antibiotic prescriptions were unnecessary in both adult and pediatric patients [1]. Similar studies revealed that only 67% of pediatric patients received recommended first-line antibiotics for the outpatient treatment of otitis media (OM), sinusitis, and pharyngitis [2] and that broad-spectrum antibiotics accounted for 50% of pediatric antibiotic prescriptions across all infection types [3]. It is well known that the overuse and misuse of antibiotics can lead to side effects, antibiotic resistance, and increased health care costs.

Establishing a robust inpatient antimicrobial stewardship program has been shown to reduce antimicrobial utilization, improve patient outcomes and lower medical costs [4]. Outpatient stewardship has also been gaining increased attention in recent years. In 2016, the Centers for Disease Control and Prevention released the Core Elements of Outpatient Antibiotic Stewardship, which addresses commitment, action for policy and practice, tracking and reporting, education, and expertise [5]. However, the literature about comprehensive outpatient antimicrobial stewardship programs remains sparse.

Several studies suggest that different tools such as education [4], audit and feedback [4], clinical pathways [6], and clinical decision support systems (CDSS) [7] are effective means of improving outpatient antibiotic prescribing. A CDSS can help health care providers optimize medical decisions by linking patient data with an electronic support tool [8]. Since the adoption of electronic medical records (EMRs) by most hospitals and clinics, computerized CDSS have become a more attractive stewardship tool given the automated nature and efficiency. A meta-analysis in 2015 included 4 cluster randomized trials and 3 randomized controlled trials that evaluated the role of computerized CDSS in outpatient antimicrobial stewardship. Five of these trials showed marginal to moderate but statistically significant improvement in antibiotic prescribing [9]. Although the suggested interventions were heterogeneous and the effect size was different in each trial, this meta-analysis suggests a modest but promising role of computerized CDSS in influencing outpatient antibiotic prescribing practices.

In several studies, providers have reported being more likely to incorporate a computer-based intervention if it is easy to use and similar to existing software [10]. EPIC (Verona, WI, USA) is currently one of the leading EMR platforms in the United States. Due to the increased adoption of EPIC at various health care institutions nationwide, it has been suggested as a potential tool to facilitate and implement the goals of antimicrobial stewardship programs [11]. In addition, acute respiratory infections (ARIs) have been the main focus of outpatient antimicrobial stewardship, most likely because they are the most common indication for prescribing outpatient antibiotics [5]. Therefore, we hypothesize that a computerized CDSS in the form of EPIC order sets developed according to the national treatment guidelines for 3 common outpatient pediatric bacterial ARIs (OM, community-acquired pneumonia [CAP], and streptococcal pharyngitis) and coupled with provider education sessions would lead to a higher percentage of prescribing of first-line antibiotics. The primary end point of this study was measurement of the effect of this combined intervention on the percentage of first-line antibiotics prescribed for the 3 pediatric bacterial ARIs. Secondary end points included antibiotic duration of therapy and the proportion of antibiotic prescriptions modified within 14 days.

METHODS

A quality improvement project evaluating the impact of EPIC order sets and educational sessions on antibiotic prescribing for common pediatric bacterial ARIs was conducted. This study was performed within the Parkland Health & Hospital System (PHHS), which is one of the largest public hospital systems in the United States and the primary teaching hospital for the University of Texas Southwestern (UTSW) Medical Center in Dallas, Texas. Among the 22 community-based clinics affiliated with PHHS, some of which see only adult patients, we chose the 4 outpatient clinics with the largest pediatric patient volumes in order to expand our sample size. All clinics are located in the Dallas area and use EPIC as an EMR tool. The 4 selected clinics were randomized to an intervention arm (Garland Health Center and Oak West Health Center) and a control arm (Hatcher Station Health Center and deHaro-Saldivar Health Center). Randomization was performed blindly and completed before evaluating any of the clinics’ characteristics. The study extended over 2 time periods, October 1, 2018, through March 31, 2019 (pre-implementation), and October 1, 2019, through March 31, 2020 (postimplementation). The fall and winter seasons were selected because ARIs tend to occur most frequently during these times of the year. The same time period of each year was evaluated in order to control for these seasonal effects.

The EPIC order sets created for the study consist of outpatient treatment pathways for OM, CAP, and streptococcal pharyngitis based on the most recent guidelines from the American Academy of Pediatrics [12-14] and the Infectious Diseases Society of America [15, 16] (Figures 1–3). The goal of the order sets is to help guide providers toward choosing first-line antibiotics recommended for the treatment of these ARIs. Per the guidelines, first-line agents are narrow-spectrum β-lactams, namely penicillin VK, penicillin G, and/or amoxicillin. Alternatives to first-line agents are more broad-spectrum antibiotics and are included in the order sets in such a way as to highlight that they should only be chosen if certain criteria are met (Figures 1–3). Examples of these criteria are the presence of a penicillin allergy or treatment failure with amoxicillin. The order sets also prepopulate with the most appropriate antibiotic dosing and duration of therapy for each indication. If a patient has more than 1 ARI, the provider should choose the order set most appropriate to cover all diagnoses. The order sets do not auto-populate based on problem list, note documentation, or billing code in the antibiotic prescribing order section of the EMR, meaning that providers have to specifically search for them when they want to prescribe an antibiotic for an ARI from the order sets. As such, providers can also prescribe antibiotics for the specific listed diagnoses without using the order sets. The educational sessions were performed in person by a physician member of the study team (fellow in Pediatric Infectious Diseases) in the intervention clinics. These sessions were in the form of a PowerPoint (San Francisco, CA, USA) presentation and addressed the importance of outpatient antimicrobial stewardship, study end points, contents of the order sets, and instructions on their appropriate utilization. Providers were allowed to ask questions during or after the educational sessions. The educational sessions were provided to the main providers (total of 8) in the intervention clinics between the 2 study periods (August–September 2019) and before the order sets became available online for use (October 1, 2019). The control clinics did not have access to the EPIC order sets and did not receive any educational sessions, but the clinic medical directors were aware of their participation in this study. Additionally, a reminder email about the order sets was sent to all the main providers in the intervention clinics every 2 weeks throughout the postimplementation period. A second in-person meeting was held with the intervention clinic providers in the middle of the postimplementation period (December 2019–January 2020) to serve as another reminder about using the order sets and to answer any questions providers had about the process. Five unique providers were shared between both study arms; however, they were not considered main providers at any study clinic. They had access to the EPIC order sets only in the intervention clinics but were unable to attend the educational sessions or the reminder meetings. One physician champion was designated in each of the intervention clinics from the beginning of the study and helped to reinforce the utility of the order sets, answer basic questions regarding the order sets, and facilitate communication between the study team and the other providers in each of the corresponding clinics.

Figure 1.

Illustration of the EPIC order set of OM. Abbreviations: BID, twice daily; IM, intramuscular; max, maximum; OM, otitis media; QD, once daily; TID, 3 times daily.

Figure 3.

Illustration of the EPIC order set of streptococcal pharyngitis. Abbreviations: BID, twice daily; IM, intramuscular; max, maximum; QD, once daily; TID, 3 times daily.

Illustration of the EPIC order set of CAP. Abbreviations: BID, twice daily; CAP, community-acquired pneumonia; max, maximum.

Patients between the ages of 3 months and 19 years who had received prescriptions for oral antibiotics and/or treatment in the clinics with intramuscular (IM) antibiotics for OM, CAP, and/or streptococcal pharyngitis were included. The diagnoses were identified using a combination of the prescription diagnosis, visit diagnosis, provider assessment in the progress note, and/or after-visit summary. Any antibiotic order with an unclear diagnosis or diagnosis not exclusive to 1 or more of the 3 particular ARIs was excluded. For each patient, only the first antibiotic given, either oral or IM, for these 3 ARIs (index antibiotics) was included during each study period. Antibiotic duration was calculated after excluding antibiotic prescriptions for which durations are unlikely to be affected by the intervention, such as azithromycin, IM antibiotics, antibiotics for mixed infections, and antibiotics for streptococcal pharyngitis. To further characterize whether the intervention had a greater impact on antibiotic durations in OM or CAP, we analyzed duration for each of these 2 infections separately. The index antibiotic prescription was considered to be modified if the dose, frequency, or duration was changed or if it was followed by another antibiotic course for the same infection within 14 days. Changing antibiotic spectrum or class based on treatment response or antibiotic allergy was suggested by the order sets, but other modifications were not dictated by the order sets and were left to provider clinical judgment. Only the modifications completed by the same clinic were included in order to prevent the potential for bias in differing clinical judgments by another set of providers. In order to lessen the potential risk of the intervention cross-contamination to the control arm, antibiotic prescriptions written by providers common to both the intervention and control clinics were excluded from the control arm during the postimplementation period. Data were retrospectively collected from EPIC and included patient characteristics, information about index antibiotics, and prescriber’s title (physician vs physician assistant or nurse practitioner).

The anticipated increase in first-line antibiotic prescribing in the intervention arm from pre-implementation to postimplementation was 10%. This required at least 255 patients in each period or 510 patients during both study periods in the intervention clinics in order achieve a power of 80% using a 2-independent-samples proportions test. For continuous variables, 2-way analysis of variance was used with 2 between effects, study period and intervention; Bonferroni pairwise post hoc tests were used to compare the pre- and postimplementation periods separately for the control and intervention clinics. For the measure of duration of antibiotic use, 5 patient characteristics were included as potential model covariates: age, weight, sex, ethnicity, and primary payer (Parkland vs other); only those covariates significant at P < .15 were included in the model. The Mantel-Haenszel common odds ratio was used to examine the association between 2 binary variables (study arm and categorical variable) while controlling for the study period (2 layers, pre- vs postimplementation). The Breslow-Day test was used to test the homogeneity of the odds ratios across the layers. Once the Mantel-Haenszel and Breslow-Day tests were evaluated, χ 2 tests for clinical and categorical variables were examined by study period and combined study periods. P < .05 was considered statistically significant. The statistical analysis was completed using SPSS, version 26.0 (IBM Corp., Armonk, NY, USA), and SAS, version 9.4 (SAS Institute Inc., Cary, NC, USA). The Institutional Review Board of the UTSW Medical Center approved all study procedures, required verbal informed consent from providers to attend the educational sessions, and waived informed consent for patients.

RESULTS

A total of 2745 antibiotic prescriptions were written for bacterial ARIs over both study periods. However, 55 prescriptions were excluded from the control clinics in the postimplementation period (17 from Hatcher Station Health Center and 38 from deHaro-Saldivar Health Center), as these prescriptions were written by providers common to both arms. One prescription from each intervention clinic in the postintervention period was also excluded because these prescriptions were written before the availability of the EPIC order sets. The remaining total number of antibiotic prescriptions in the 2 study periods was 2690, with more prescriptions written in the control arm compared with the intervention arm (1688 and 1002, respectively) (Table 1). During the pre-intervention period, 21 physicians and 1 nurse practitioner were in the intervention clinics, and 25 physicians and 1 nurse practitioner were in the control clinics. During the postintervention period, the number of nurse practitioners remained the same but the number of physicians increased to 24 and 27 in the intervention and control clinics, respectively.

Table 1.

Analysis of Categorical Variables

							Clinic * Variable		Clinic * Variable
	Pre-implementation Period		Postimplementation Period		Mantel-Haenszel Common Odds Ratio	Breslow-Day Test	Pre-implementation	Postimplementation	Combined Periods
Variable	Intervention Clinics (n = 569)	Control Clinics (n = 862)	Intervention Clinics (n = 433)	Control Clinics (n = 826)	P	P	P	P	P
Male sex, No. (%)	310 (54.5)	434 (50.3)	223 (51.5)	442 (53.5)	.499	.125	.126	.497	.515
Antibiotic allergy, No. (%)	18 (3.2)	34 (3.9)	17 (3.9)	42 (5.1)	.231	.920	.440	.355	.203
Hispanic ethnicity, No. (%)a	334 (61.6)	740 (87.1)	261 (62.4)	733 (89.4)	<.001	.349	<.001	<.001	<.001
Parkland Financial Assistance, No. (%)a	316 (55.5)	503 (58.4)	242 (55.9)	482 (58.4)	.179	.930	.292	.401	.177
Physician prescriber, No. (%)	496 (87.2)	784 (91)	388 (89.6)	705 (85.4)	.914	.002	.023	.034	.992
First-line antibiotic prescribing, No. (%)	426 (74.9)	670 (77.7)	360 (83.1)	642 (77.7)	.589	.011	.211	.024	.664
Antibiotic modifications, No. (%)	6 (1.1)	10 (1.2)	5 (1.2)	13 (1.6)	.578	.769	.852	.552	.552

aPatients with unknown ethnicity or primary payer were excluded from the statistical analysis.

Patients in the control clinics were older, weighed more, and had a higher proportion of Hispanic ethnicity than those in the intervention clinics (Tables 1 and 2). However, there was no difference in terms of sex, antibiotic allergy, and having Parkland Financial Assistance between both study periods (Tables 1 and 2). The most common bacterial ARI leading to an antibiotic prescription was OM, followed by streptococcal pharyngitis and then CAP (Table 3). Only a minority of patients were found to have mixed ARIs (Table 3). Physician and midlevel provider prescribing were not different between the pre-implementation and postimplementation periods (Table 1).

Table 2.

Analysis of Continuous Variables

								Bonferroni Post Hoc Test
					ANOVA			Intervention Clinics vs Control Clinics
	Pre-implementation period		Postimplementation period		Clinic * Period	Clinic	Period	Pre-implementation	Postimplementation
	Intervention Clinics (n = 569)	Control Clinics (n = 862)	Intervention Clinics (n = 433)	Control Clinics (n = 826)	P	P	P	P	P
Age, mean ± SD, y	4.43 ± 4.03	5.02 ± 4.28	4.32 ± 3.70	4.94 ± 3.70	.849	<.001	.760	.014	.070
Weight, mean ± SD, kg	20.90 ± 17.46	24.38 ± 20.20	20.13 ± 16.96	23.14 ± 19.17	.758	<.001	.188	.004	.049
Overall antibiotic duration, mean ± SD, da,b	9.68 ± 0.96	9.65 ± 1.07	9.26 ± 1.56	9.79 ± 0.75	<.001	<.001	.003	>.999	<.001
Antibiotic duration for OM, mean ± SD, dc	9.65 ± 0.99	9.68 ± 1.05	9.23 ± 1.60	9.79 ± 0.74	<0.001	<.001	.002	>.999	<.001
Antibiotic duration for CAP, mean ± SD, d	9.81 ± 0.69	9.24 ± 1.42	9.55 ± 1.06	9.68 ± 0.93	.032	.183	.578	.050	>.999

Abbreviations: ANOVA, analysis of variance; CAP, community-acquired pneumonia; OM, otitis media.

aOverall antibiotic duration was measured after excluding prescriptions for azithromycin, IM antibiotics, antibiotics for mixed infections, and antibiotics for streptococcal pharyngitis.

bAge was a significant covariate in this model (P < 0.001); the covariate appearing in the model was evaluated at age 3.78.

cAge was a significant covariate in this model (P < 0.001); the covariate appearing in the model was evaluated at age 3.71.

Table 3.

Distribution of the Antibiotic Prescriptions Among the 3 Acute Bacterial Respiratory Infections

	Pre-implementation Period		Postimplementation Period
	Intervention Clinics (n = 569)	Control Clinics (n = 862)	Intervention Clinics (n = 433)	Control Clinics (n = 826)
OM, No. (%)	424 (74.5)	614 (71.2)	320 (73.9)	590 (71.4)
Strep pharyngitis, No. (%)	72 (12.7)	111 (12.9)	75 (17.3)	136 (16.5)
CAP, No. (%)	69 (12.1)	125 (14.5)	35 (8.1)	89 (10.8)
Mixed, No. (%)	4 (0.7)	12 (1.4)	3 (0.7)	11 (1.3)

Abbreviations: CAP, community-acquired pneumonia; OM, otitis media; Strep, streptococcal.

The difference in the percentage of first-line antibiotics prescribed by providers in the intervention clinics was significant (74.9% vs 83.1%; P = .002). When examining the study periods at pre-implementation vs postimplementation for first-line antibiotic prescribing by study arm, the Mantel-Haenszel common odds ratio was not significant (P = .589) but the Breslow-Day test was significant (P = .011) (Table 1). During the pre-implementation period, providers in the intervention clinics compared with the control clinics prescribed the first-line antibiotics at approximately the same rate (74.9% vs 77.7%; P = .211), but postimplementation first-line antibiotic prescribing was significantly higher by 5.4% in the intervention clinics compared with the control clinics (83.1% vs 77.7%; P = .024) (Table 1).

In order to more accurately assess the impact of the intervention on antibiotic duration, we excluded the antibiotic prescriptions that were unlikely to be influenced by the intervention, such as azithromycin, IM antibiotics, antibiotics for mixed infections, and antibiotics for streptococcal pharyngitis. Age was the only significant covariate for duration of therapy for all groups combined and the subgroup OM (both P < .001); age was not a significant covariate for the CAP subgroup (P = .415). The interaction between study periods and study arms for overall antibiotic duration was statistically significant (P < .001) (Table 2). Before implementation, there was no difference in mean duration of therapy between the intervention and control clinics (9.68 ± .96 days vs 9.65 ± 1.07 days; P > .999) (Table 2). However, after implementation, the duration of therapy for prescriptions written by providers in the intervention clinics was 0.53 days less than the control clinics (9.26 ± 1.56 days vs 9.79 ± 0.75 days; P < .001) (Table 2). Similarly, the interaction between study periods and study arms for antibiotic duration in OM and CAP individually was statistically significant (P < .001 and P = .032, respectively) (Table 2). At pre-implementation, the mean antibiotic duration for OM was not different between the intervention and control clinics (9.65 ± 0.99 days vs 9.68 ± 1.05 days; P > .999), but subsequently the prescriptions in the intervention clinics were written for a shorter antibiotic duration by 0.55 days compared with the control clinics at postimplementation (9.23 ± 1.60 days vs 9.79 ± 0.74 days; P < .001) (Table 2). The mean antibiotic duration for CAP was not different between the intervention and control clinics at pre-implementation (P = .050) or postimplementation (P > .999) (Table 2).

The percentage of the modified antibiotic prescriptions was small throughout the study overall (1.1%–1.6%) and not statistically different between the intervention and control clinics at pre-implementation (P = .852) or postimplementation (P = .552) (Table 1).

DISCUSSION

We found that a stewardship intervention comprised of computerized CDSS consisting of treatment pathways in the form of EPIC order sets coupled with educational sessions was associated with a statistically significant increase in the percentage of first-line antibiotic prescribing and shorter antibiotic duration for the treatment of common outpatient pediatric bacterial ARIs. Although these differences in antibiotic prescribing practices between the intervention and control clinics are small in the postimplementation period, such differences were not observed in the pre-implementation period, suggesting that the intervention was successful.

Trials that have evaluated the effect of computer-aided CDSS in improving outpatient prescribing have shown a wide effect range (2.5%–44%) [7, 9]. Two other studies have found that various interventions can decrease the use of broad-spectrum antibiotics by up to 12.5% [4, 6]. In our study, first-line antibiotic prescribing increased by 5.4%, which we believe to be indicative of a veritable change in practice, especially in light of the fact that the baseline percentage of first-line antibiotic prescribing was already relatively high at our institution (74.9%–83.1%). We were also able to achieve a small reduction in antibiotic duration for OM, as was also seen in another study evaluating the impact of a computerized CDSS on antibiotic prescribing for OM [17]. No difference was observed in antibiotic duration for CAP, potentially due to the small number of antibiotic prescriptions written for this infection or because selecting 10 days for antibiotic duration is still within what is recommended by our CAP order set. No difference in antibiotic modifications within 14 days was observed following the intervention, which may have been because of the small percentages of antibiotic modifications throughout the study as well as our strict definition of such modifications.

This study has several strengths, including the fact that pre- and postimplementation periods were included and the intervention arm was paired with a control arm during the postimplementation period. Having 2 study periods during the same time of the year allowed for the control of seasonality effects and antibiotic prescribing factors related to the prescribers and/or clinics. Maintaining a control arm in the postimplementation period helped account for other factors such as changing bacterial susceptibility patterns, prevalence of certain bacteria in a specific season, release of new publications that may impact prescribing behavior, media influences, and changing antibiotic availability. During the postimplementation period, it was noted that 5 physicians were common to both the intervention and control arms. Although these common providers had no access to the EPIC order sets while physically in the control clinics, we excluded their antibiotic prescriptions in the control clinics postimplementation as knowledge and use of the order sets could have affected their antibiotic prescribing practices.

This study also carries various limitations. The total number of intervention and control clinics was small, although we chose the 4 PHHS clinics with the largest pediatric patient volumes. Our EPIC order sets were coupled with education as well as email reminders every 2 weeks and a second in-person meeting held in the middle of the postimplementation period as ways to periodically remind providers to use the order sets. Thus, the impact of each separate component of the intervention remains unknown. Studies have indicated that multifaceted approaches are likely required to improve outpatient antibiotic utilization [18]; however, the order sets were thought to have had the greatest impact on antibiotic prescribing practices, as education and reminders were intended only to supplement this electronic tool. Moreover, the inclusion criteria largely depended on the diagnoses linked with the ordered antibiotics, but as there is no requirement to link an antibiotic order with an International Classification of Diseases code at PHHS clinics, we had to rely on the provider documentation in the progress notes or after-visit summaries to collect accurate diagnoses for a large number of patients, which raises the possibility of selection bias. Lastly, providers in the intervention clinics did not receive audit and feedback regarding their antibiotic prescribing. Although audit and feedback are common practices among inpatient antimicrobial stewardship programs, such a strategy proves more difficult to implement in the outpatient setting because most clinics do not have dedicated stewardship personnel. Nevertheless, some studies have shown that audit and feedback can improve outpatient antibiotic prescription when implemented with other interventions [4, 19]. Overall, with its many strengths and despite its limitations, this study adds to the scarce literature in outpatient antimicrobial stewardship and addresses the utility of a computerized CDSS coupled with education in improving antibiotic prescribing practices.

CONCLUSIONS

In this study, a computerized CDSS consisting of treatment pathways in the form of EPIC order sets and coupled with educational sessions resulted in a small but statistically significant improvement in first-line antibiotic prescribing and antibiotic duration for the treatment of outpatient pediatric bacterial ARIs. More studies are needed in order to assess the utility of multimodal approaches to pediatric outpatient antimicrobial stewardship.