Impact of Clinical Decision Support System Implementation at a Community Hospital With an Existing Tele-Antimicrobial Stewardship Program.

Background: Lack of on-site antimicrobial stewardship expertise is a barrier to establishing successful programs. Tele-antimicrobial stewardship programs (TASPs) utilizing a clinical decision support system (CDSS) can address these challenges.
Methods: This interrupted time series study reports the impact of CDSS implementation (February 2020) within an existing TASP on antimicrobial usage in a community hospital. Segmented regression analysis was used to assess differences in antimicrobial usage from January 2018 through December 2021. Pre- and post-CDSS frequencies of intravenous vs oral antimicrobials, time to optimal therapy (TTOT), pharmacist efficiency (number of documented interventions per month), and percentage of hospitalized patients receiving antimicrobials were compared with descriptive statistics.
Results: Implementation of a CDSS into an existing TASP was associated with an immediate 11% reduction in antimicrobial usage (level change, P < .0001). Antimicrobial usage was already trending down by 0.25% per month (pre-CDSS slope, P < .0001) and continued to trend down at a similar rate after implementation (post-CDSS slope, P = .0129). Frequency of use of select oral agents increased from 38% to 57%. Median TTOT was 1 day faster (2.9 days pre-CDSS vs 1.9 days post-CDSS). On average, pharmacists documented 2.2-fold more interventions per month (198 vs 90) and patients received 1.03 fewer days of antimicrobials per admission post-CDSS. Conclusions: Implementation of a CDSS within an established TASP at a community hospital resulted in decreased antimicrobial usage, higher rates of oral usage, faster TTOT, and improved pharmacist efficiency.

There are several barriers to establishing successful antimicrobial stewardship programs (ASPs) at community hospitals, including a lack of on-site infectious disease (ID) providers in many parts of the United States (US) [1]. Tele-antimicrobial stewardship programs (TASPs) leverage remote ID providers to support local clinicians who lack formal ID training and may overcome this barrier [2, 3]. However, TASPs also face challenges including the time and resources required to review patients via phone or video conference, which can lack scheduling flexibility. Additionally, the local team is usually responsible for identification and prioritization of relevant cases by evaluating antibiotic use reports or static patient lists that are curated in the electronic health record. This may lead to missed opportunities for antimicrobial regimen optimization as there is no prioritization of patients and content is not updated throughout the day. Clinical decision support systems (CDSSs) have the potential to address these issues, and several studies evaluating the implementation of a TASP have utilized a CDSS [4-9]. Other reports on programs using a CDSS have originated in academic, urban institutions with well-established on-site ID expertise and oversight [10, 11]. Of note, the definition of CDSS in published reports varies, ranging from paper-based support tools or applications for handheld personal devices to software tools that may or may not be integrated into an electronic medical record (EMR) [12-15]. Some authors have also pointed out that many studies to date did not consider other factors beyond the technology itself that may impact successful use of CDSS for antimicrobial stewardship [16, 17]. With respect to implementation, identifying the key components of effective ASPs that utilize remote expertise and local non-ID-trained healthcare personnel has been identified as a high-value target for antimicrobial stewardship research [18]. Here we describe the impact of CDSS implementation on antimicrobial usage metrics and workflow efficiency at a single community hospital where an integrated TASP model utilizing non-ID-trained pharmacists was already in place.

METHODS

Description of the Health System, Community Hospital, and ASP

UPMC is a large health system in the US servicing western and central Pennsylvania, western New York, and western Maryland. The community hospital in this study is located about 50 miles north of Pittsburgh, Pennsylvania, has 146 licensed beds, and is in a community with an approximate population of 23 000. One ID physician consultant was on-site until July 2018 when ID consult services were transitioned to remote telemedicine utilizing ID physicians from the University of Pittsburgh Division of Infectious Diseases. In January 2018, the UPMC Centralized Health-System Antimicrobial Stewardship Efforts (CHASE) Program (authors T. M. K. and J. R. B.) was created to assist hospitals with the development and maintenance of ASPs and to ensure compliance with accreditation standards. A detailed description of UPMC CHASE has been published previously [19]. In September 2019, CHASE established a formalized tele-stewardship process, inviting 7 community hospitals (including the hospital in this study) to participate in a daily 1-hour teleconference to review and discuss patient cases in a shared EMR. In brief, this teleconference served as a platform to provide recommendations for interventions and real-time education to reinforce core concepts of antimicrobial stewardship. In February 2020, a pilot with ILÚM Insight CDSS (Infectious Disease Connect, Pittsburgh, Pennsylvania), was initiated at the hospital in this current study.

The evolution of antimicrobial stewardship at this community hospital is described in 4 phases as shown in Table 1. In phase 1, prior to CHASE involvement, 2 non-ID-trained clinical pharmacists (1.5 total full-time equivalent [FTE] for all general pharmacy duties) shared responsibilities for local stewardship efforts, which largely focused on decreasing unnecessary fluoroquinolone use via prospective audit with intervention and feedback to prescribers. For both pharmacists, antimicrobial stewardship was a responsibility in addition to their other pharmacist duties and accounted for approximately 3–4 hours of their combined time per day (0.3–0.5 FTE for stewardship). In phase 2, CHASE activities were focused on building the foundation of the tele-stewardship program, including engagement of local staff and key stakeholders, data tracking and reporting, and targeted education. A local emphasis on fluoroquinolones continued during this phase. In phase 3, the daily teleconference was initiated where local pharmacists were responsible for identifying cases to present during the teleconference but were encouraged to prioritize cases as outlined in Table 1. Implementation of the CDSS occurred in phase 4 (February 2020) after approval from local pharmacy and hospital leadership. After transitioning to the CDSS as our primary method of patient review and communication of the TASP, we discontinued the daily teleconference for this hospital. Central team workload at this hospital was divided between the ID-PharmD and ID-MD to asynchronously provide support via CDSS rather than participating in the teleconference at the same time.

Table 1.

Evolution of Antimicrobial Stewardship Activities Between January 2018 and December 2021

Months	Phase	Summary of Primary Activities
Jan–Jun 2018	Phase 1	Pre-CHASE; PAIF of all fluoroquinolone orders, Monday–Friday, by local pharmacists
Jul 2018–Aug 2019	Phase 2	Education from CHASE to local pharmacists regarding more effective fluoroquinolone PAIF Education from CHASE to local prescribers regarding antimicrobial overuse Centralized antimicrobial usage data collection, analysis, and reporting CHASE assistance with individual interventions by local pharmacists when requested by local pharmacists via email
Sep 2019–Jan 2020	Phase 3	Phase 2 activities + Monday–Friday 1-hour tele-stewardship video conference by CHASE (local pharmacists responsible for identification and presentation of cases for feedback by CHASE)   Top priority   Any patient with Staphylococcus aureus bacteremia without an ID consult Any patient on carbapenems, daptomycin, linezolid, ceftaroline, tigecycline, ceftazidime-avibactam, ceftolozane-tazobactam, or antifungal (excluding fluconazole) without an ID consult   High priority   Patients on antibiotics for >48 hours with negative cultures or no clear diagnosis Patients on broad-spectrum antibiotics when cultures show very susceptible pathogens Patients on piperacillin-tazobactam, cefepime, or ceftazidime for >48 hours with no Pseudomonas aeruginosa in cultures Patients on quinolones, aztreonam, or clindamycin without history of severe β-lactam allergy Patients being treated for UTI yet no mention of UTI symptoms   Implementation of MRSA nasal screen to de-escalate IV vancomycin for pneumonia (Sep 2019)
Feb 2020–Dec 2021	Phase 4	CDSS implementation Tele-stewardship video conference replaced by Monday–Friday review of CDSS alerts plus real-time mobile notification of high-priority alerts to prescribers and TASP Asynchronous communication between CHASE and local pharmacists via CDSS

Abbreviations: CDSS, clinical decision support system; CHASE, Centralized Health-System Antimicrobial Stewardship Efforts; ID, infectious disease; MRSA, methicillin-resistant Staphylococcus aureus; PAIF, prospective audit with intervention and feedback; TASP, tele-antimicrobial stewardship program; UTI, urinary tract infection.

Description of the CDSS

ILÚM Insight integrates a hospital's laboratory results, pharmacy data, and admission/discharge/transfer information into a dynamic rules engine. It provides real-time alerts to ASP members such as bug-drug mismatch, positive cultures, de-escalation opportunities, and other scenarios of interest. The CDSS allows for asynchronous communications between all end users, with a user interface similar to text messaging (Supplementary Figure 1). In our setting, communication was between the local ASP pharmacists and the centralized CHASE team of ID experts, where users were alerted via push notifications on the CDSS mobile app of any new communications or high-priority alerts. The CDSS offers reporting functionality for monitoring and tracking purposes. Supplementary Figure 1 shows representative examples of ILÚM Insight desktop and mobile functionality.

Integrated TASP Workflow After CDSS Implementation

We developed a 2-tiered notification structure whereby select alerts were sent from the CDSS to both CHASE (tier 1 and 2) and the local pharmacists (tier 1) for review and intervention as appropriate. Supplementary Table 1 summarizes the alerts that were active for either the local ASP pharmacists and/or CHASE from February to December 2021. The local pharmacists were trained on how to interpret tier 1 alerts and independently intervened with providers or routed the alerts to CHASE for further review and feedback. CHASE received tier 2 alerts and, upon evaluation, routed them to the local pharmacists with recommendations on how to intervene when appropriate. Alerts were categorized as tier 2 if they required an expert level of review by the central team prior to local action. This generally included alerts with potential for high risk (eg, positive blood cultures) and alerts that may have required interpretation of complex microbiology information (eg, bug-drug mismatch, de-escalation by susceptibility results). To lessen the time burden on the local team, the central team also reviewed other, less complex tier 2 alerts (eg, antibiotic time-out alerts) and routed them to the local pharmacists when deemed actionable. As the local team's stewardship skills and efficiency improved over time, the number of tier 1 alerts sent directly to the local team increased, thereby decreasing the number of tier 2 alerts sent only to CHASE. By the end of the study period, all alerts were sent to both the local and central teams. All intervention results were captured within the CDSS.

Process Measures and Statistical Analysis

Antimicrobial usage metrics and key performance indicators were determined with input from local hospital and pharmacy leadership and the local/central ASP teams prior to implementation of the CDSS. Facility-wide inpatient antimicrobial days of therapy (DOT) were calculated monthly using information from our EMR and data warehouse. Inpatient locations and DOT were defined according to the Centers for Disease Control and Prevention's National Healthcare Safety Network antimicrobial use and resistance protocol [20]. Antimicrobial DOT were normalized per 1000 patient-days (PD) and summarized graphically by stewardship phase (phases 1–4, Table 1). We also analyzed hospital charge data to determine what percentage of inpatients received antimicrobials and the percentage of hospital days any antimicrobial was administered. At UPMC, medications are charged upon administration.

For the primary analysis, we used segmented regression analysis of interrupted time series to estimate the association between monthly antimicrobial use, the CDSS intervention, and time after intervention. Regression dummy variables were used to indicate the different segmented periods (pre-CDSS and post-CDSS). Because antimicrobial usage is not normally distributed and is limited to nonnegative numbers, usage as DOT counts per time were estimated using a Poisson regression model, with the log of patient days used as an offset variable. Poisson regression coefficients were exponentiated to produce incident rate ratios, which can be interpreted similarly as relative risk, in this case percentage change in antimicrobial usage. With segmented regression analysis, significant changes in monthly trends (slopes) and/or level changes (changes in intercepts) suggest an intervention effect. Data was analyzed using SAS (version 9.3, Cary, North Carolina).

Frequencies of conversion from intravenous (IV) to oral (PO) formulations for fluoroquinolones, azithromycin, and metronidazole were compared between the first 6 months after CDSS implementation vs the 6 months immediately preceding implementation vs the same 6 calendar months 1 year prior to implementation. These antibiotics were targeted as part of a preexisting UPMC system automatic IV to PO conversion by pharmacy policy. To evaluate time to optimal therapy (TTOT), we randomly selected a sample of 100 patients for chart review (50 patients in the 4 months immediately prior to and after CDSS implementation, using a random number generator). One of the authors (T. M. K., J. R. B., C. A., or E. K. M., all with formal ID training) reviewed patient charts to determine if patients received optimal therapy. The start date/time of the antibiotic order was used as time zero. For patients who were on optimal therapy from the start, TTOT was recorded as zero. For patients who never achieved optimal therapy, the length of antimicrobial therapy was defined as time until the reviewed antimicrobials were stopped or until the patient was discharged from the hospital. Predefined reasons for exclusion were receipt of topical or preoperative antibiotics only, antimicrobials used for <48 hours, transition to hospice, or discharge before there was an opportunity for intervention.

Finally, local pharmacist efficiency is reported as the number of interventions performed and documented by local ASP pharmacists per month, with no changes in the amount of time dedicated for stewardship activities before and after CDSS. Prior to initiation of the CDSS pilot, the routine process for workload documentation by the local ASP pharmacists was via manual entry into an electronic form and was inconsistently completed due to time constraints. To determine a more accurate baseline intervention rate, the local ASP pharmacists were instructed to consistently record all interventions made during the month immediately prior to CDSS implementation (January 2020). Post-CDSS interventions were captured directly within the software as part of the workflow process and retrieved via electronic reports.

Patient Consent Statement

No patient consent was obtained as this project was performed as part of an ongoing UPMC Quality Improvement initiative (Project 3648).

RESULTS

Implementation of a CDSS into an existing TASP was associated with an immediate 11% reduction in antimicrobial usage upon segmented regression analysis (level change 0.8921, P < .0001) (Table 2, Figure 1). Prior to CDSS implementation, antimicrobial usage was trending downward by 0.25% per month (pre-CDSS slope 0.9975, P < .0001). Antimicrobial usage continued to trend down after CDSS implementation by 0.24% per month (post-CDSS slope 0.9976, P = .0129).

Table 2.

Parameter Estimates, Standard Errors, and P Values From Segmented Regression Model Predicting Monthly Antimicrobial Use Rates

Parameter	Poisson Regression Coefficient[a](95% CI)	P Value
Intercept	1.0015 (.9852–1.0181)	.8589
Pre-CDSS slope	0.9975 (.9963–.9986)	<.0001
Level change	0.8921 (.8691–.9158)	<.0001
Post-CDSS slope	0.9976 (.9958–.9955)	.0129

Abbreviations: CDSS, clinical decision support system; CI, confidence interval.

Model coefficients have been exponentiated.

Figure 1.

Interrupted time series antimicrobial use data fit with a segmented regression model. ILÚM Insight was implemented at month 26. Abbreviations: CDSS, clinical decision support system; DOT, days of therapy; PD, patient-days.

Total antimicrobial usage (DOT/1000 PD) summarized descriptively by stewardship phase is graphically displayed in Figure 2. Phase 1 and 2 activities appeared to minimally impact usage, whereas phases 3 and 4 demonstrated sequential declines in total antimicrobial usage. Among the targeted antibiotics displayed in Figure 2, only fluoroquinolone and anti-pseudomonal antibiotic usage started to decline during phase 2, with further reductions in phase 3 and 4. Usage of IV vancomycin started to decline in phase 3 with further reductions observed in phase 4.

Figure 2.

Inpatient antimicrobial usage (days of therapy per 1000 patient-days) by month and by stewardship phase, including all antimicrobials (A) and select antibiotics targeted by the antimicrobial stewardship program (B). Anti-pseudomonal antibiotics include amikacin, aztreonam, cefepime, ceftazidime, ceftazidime-avibactam, ceftolozane-tazobactam, ciprofloxacin, colistin, gentamicin, levofloxacin, meropenem, meropenem-vaborbactam, piperacillin-tazobactam, polymyxin b, and tobramycin. Abbreviations: DOT, days of therapy; IV, intravenous; PD, patient-days.

The percentage of patients receiving antimicrobials was similar in both time periods: 48% of all hospitalized patients received antimicrobials after implementation vs 52% prior to implementation. Table 3 describes the frequencies of what was determined to be optimal therapy before and after CDSS. Out of 100 patients randomly selected for this review, 77 (38 before and 39 after CDSS) were eligible for inclusion. A higher percentage of patients achieved optimal therapy after CDSS (84% vs 56%) and did so faster (1.9 days vs 2.9 days). Additionally, 53% of patients who achieved optimal therapy after CDSS did so within 2 days of starting antimicrobials whereas no patients in the pre-CDSS period who required a change in antimicrobial therapy achieved optimal therapy within 2 days. In this subset, patients also received antimicrobials for a shorter duration after CDSS (56% of their hospital stay vs 69%). On average, these patients received 1.03 days fewer of antimicrobials per unique admission after CDSS implementation.

Table 3.

Summary of Antimicrobial Exposure Days and Time to Optimal Therapy Before and After Clinical Decision Support System

Variable	Pre-CDSSOct 2019–Jan 2020	Post-CDSSMar–Jun 2020
Frequency and duration of antimicrobial exposure-days
Percentage of patients with antimicrobial exposure[a]	52%	48%
Percentage of hospital days with an antimicrobial exposure[b]	69%	56%
Average length of stay, days, of patients receiving antimicrobials	7.03	6.83
Frequency and time to optimal therapy review
No. of patients	38	39
No. of antimicrobial orders reviewed	57	70
No. (%) of antimicrobials optimal from the start	16 (28%)	16 (22%)
No. (%) of antimicrobials eventually optimized	32 (56%)	59 (84%)
Time to optimal therapy, days, median (range)	2.9 (2.0–5.2)	1.9 (0.1–4.0)

Abbreviation: CDSS, clinical decision support system.

Percentage of admitted patients who received at least 1 antimicrobial dose during hospitalization.

Percentage of hospital days with an antimicrobial administration among patients who received at least 1 dose during their hospital stay.

In the 6 months after implementation of CDSS, the percentage of PO agents relative to IV increased by approximately 20% as compared to the immediately preceding 6 months and the same 6 calendar months 1 year prior. Usage of these agents declined overall (Supplementary Figure 2).

Finally, we saw an increase in the number of documented interventions after CDSS implementation. In the 4 months after CDSS implementation, and with no increase in time dedicated to stewardship by the local pharmacists, the average number of documented interventions per month increased by 220% when compared to the month immediately preceding implementation (Supplementary Table 2).

Although not formally documented, total support hours to this hospital from CHASE after CDSS implementation were similar to baseline support hours pre-CDSS (1 hour ID-PharmD, 1 hour ID-MD), with a transient increase in time spent after implementation to account for training and stabilization of workflow using the CDSS.

DISCUSSION

The ILÚM Insight CDSS has been previously evaluated by other ASPs, mostly in relation to its utility for prior authorization in a large academic medical center [21, 22]. Our study is the first to evaluate it within a TASP and within a smaller community hospital. Implementation of this CDSS into an existing, integrated TASP resulted in a significant decrease in total antimicrobial usage (level change) followed by a continued decline in post-CDSS trend (slope) for 23 months postimplementation. In segmented regression analysis of interrupted time series, a change in level immediately following the change point constitutes an abrupt intervention effect, whereas a change in trend (ie, change in slope of the segment after the intervention as compared with the segment preceding the intervention) represents a gradual intervention effect. In this study, we saw an abrupt intervention effect likely due to the CDSS but similar slopes before and after implementation of the CDSS. The lack of change in slopes before and after the intervention indicates that the significant decline in antimicrobial usage trend in the postimplementation period was not a result of the CDSS alone, but likely due to the continued presence and activities of the TASP. The abrupt intervention effect (level change) and continued gradual decline postintervention (slope) was possibly due to multiple factors, including the CDSS's ability to identify and prioritize a greater number of relevant stewardship opportunities, improved workflow efficiency, and increased involvement from the remote central antimicrobial stewardship team and training of the local non-ID-trained pharmacists.

Pre-CDSS activities in phase 3 consisted primarily of a 1-hour teleconference where the local ASP was responsible for identification and presentation of cases for feedback by CHASE on how to intervene. Although effective (93% intervention acceptance rate, per local pharmacist report), these daily teleconferences were often limited by scheduling conflicts, competing priorities, and the amount of time available to review a complete list of cases. Furthermore, cases were identified from static patient lists using pharmacy or microbiology reports without the availability of a rules engine to trigger specific events. Implementation of the CDSS addressed some of these limitations. Daily communication between the central and local team transitioned from scheduled teleconferences to asynchronous review of alerts and text messaging directly within the software, allowing for more flexibility for the end users. Other CDSS functionality included the ability to group and categorize alerts, which assisted with prioritization. Without this added functionality, we suspect we could not have addressed as many patient reviews per day. After CDSS implementation, the local pharmacists documented more interventions without an increase in dedicated time for stewardship activities and without sacrifice of other nonstewarding activities. Although not formally documented, the amount of time spent on stewardship activities by CHASE transiently increased immediately following CDSS implementation to account for additional training and stabilization of workflow. After a few months, the amount of time spent by CHASE returned to pre-CDSS level and eventually decreased even further by the end of the study period when the local team became more comfortable assessing and acting on alerts independently. Of note, the central team's efficiency also improved by dividing the workload (CDSS alerts) rather than both individuals participating in the teleconference at the same time. Although we worked independently, our communications within the CDSS were visible for all end users and we could provide additional insight as needed. We believe our process of communicating on shared CDSS alerts allowed increased involvement and transfer of knowledge from the centralized ID experts and experiential training of the local pharmacists. Over time, the knowledge gained could be applied to future similar clinical scenarios and contribute to their ability to manage the alerts independently.

The interrupted time series design with segmented regression analysis is a strength of this study. Rather than simply comparing the pooled monthly antimicrobial usage rates before and after the intervention with a 2-group test, we analyzed the data using a generalized linear model that can detect changes in outcome levels and trends (eg, monthly increases or decreases in antimicrobial usage rate). Poisson regression is preferred over linear regression because antimicrobial usage rates are not normally distributed and are limited to nonnegative values. Unlike standard regression models, segmented regression models estimate changes in mean outcome levels (ie, intercepts) and trends (ie, slopes) while allowing for different slopes before and after the intervention.

Our report has limitations. Several of our secondary metrics were collected for short time periods and analyzed descriptively and cannot definitively be attributed to the CDSS intervention. Our primary metric, total antimicrobial usage, is not a measure of appropriateness. We addressed this with a TTOT review, but this was descriptive, subjective, and included a small sample of patients that was uncontrolled and underpowered to detect statistically significant differences. The reviews were performed and adjudicated by ID-trained clinicians., but a dual review for formal agreement was not performed. Another limitation of our study was the inconsistent documentation of stewardship interventions prior to CDSS implementation, allowing us to only include 1 month for baseline comparison when interventions were consistently documented. Additionally, aside from consistency in documentation, automation of the CDSS may have captured more interventions that manual recording would not have. However, we believe these study limitations are reflective of real-world limitations in ASP tracking and workload capture. Although it is difficult to quantify the true increase in number of interventions made, utilization of the CDSS addressed one of the more notable challenges we experienced prior to implementation: the burdensome process of manual intervention tracking. Finally, the CDSS intervention occurred immediately prior to and during the first surge of the coronavirus disease 2019 (COVID-19) pandemic in the US. This may have introduced confounding effects we were unable to control for. Patient volumes and types of patients admitted during this time varied as compared to prior time periods at many US hospitals [23]. In addition, rates of antimicrobial usage were very high in patients hospitalized with COVID-19, especially during the first surge [23]. Despite this, we observed an immediate decrease in antimicrobial usage, which was sustained during the pandemic.

Our report specifically focused on the impact on antimicrobial usage after implementation of a CDSS in a single community hospital integrated within an established TASP. The CDSS alerts we focused on are relevant for most ASPs and other care settings. However, other program models might see different results depending on baseline antimicrobial usage rates, local and central staffing resources, remote antimicrobial stewardship expert involvement, and workflow utilized. Future studies evaluating the long-term impact of the CDSS with our TASP on patient-centric and economic outcomes are needed. Additionally, our next steps are to evaluate the impact of implementing this CDSS across the remaining hospitals within our health system.

CONCLUSIONS

Implementation of a CDSS into an established, integrated TASP at a community hospital allowed for increased involvement and transfer of knowledge from remote ID experts, resulting in decreased total antimicrobial usage, increased rates of IV to PO conversion, improved TTOT, decreased antimicrobial exposure, and improved workflow efficiency. Implementation of the CDSS addressed several challenges in our TASP, while minimizing the need for additional staffing support.

Click here for additional data file.