Effective Antimicrobial StewaRdship StrategIES (ARIES): Cluster Randomized Trial of Computerized Decision Support System and Prospective Review and Feedback.

BACKGROUND: Prospective review and feedback (PRF) of antibiotic prescriptions and compulsory computerized decision support system (CDSS) are 2 strategies of antimicrobial stewardship. There are limited studies investigating their combined effects. We hypothesized that the use of on-demand (voluntary) CDSS would achieve similar patient outcomes compared with automatically triggered (compulsory) CDSS whenever broad-spectrum antibiotics are ordered.
METHODS: A parallel-group, 1:1 block cluster randomized crossover study was conducted in 32 medical and surgical wards from March to August 2017. CDSS use for piperacillin-tazobactam or carbapenem in the intervention clusters was at the demand of the doctor, while in the control clusters CDSS use was compulsory. PRF was continued for both arms. The primary outcome was 30-day mortality.
RESULTS: Six hundred forty-one and 616 patients were randomized to voluntary and compulsory CDSS, respectively. There were no differences in 30-day mortality (hazard ratio [HR], 0.87; 95% CI, 0.67-1.12), re-infection and re-admission rates, antibiotic duration, length of stay, or hospitalization cost. The proportion of patients receiving PRF recommendations was not significantly lower in the voluntary CDSS arm (62 [10%] vs 81 [13%]; P = .05). Appropriate indication of antibiotics was high in both arms (351/448 [78%] vs 330/433 [74%]; P = .18). However, in geriatric medicine patients where antibiotic appropriateness was <50%, prescription via compulsory CDSS resulted in a shorter length of stay and lower hospitalization cost.
CONCLUSIONS: Voluntary broad-spectrum antibiotics with PRF via CDSS did not result in differing clinical outcomes, antibiotic duration, or length of stay. However, in the setting of low antibiotic appropriateness, compulsory CDSS may be beneficial.

Increasing antimicrobial resistance due to inappropriate antimicrobial use is a global concern, and antimicrobial stewardship teams have become an integral part of the response to this issue [1, 2]. Through prospective review of antibiotic prescriptions and feedback (PRF) to doctors, patients have improved clinical response, reduced adverse effects, and reduced mortality [3-5]. However, this strategy is labor-intensive, and skilled health care workers are expensive and scarce resources [6, 7].

Antibiotic computerized decision support systems (CDSS) have been used to facilitate these processes to circumvent the lack of manpower. In observational studies, implementation of a CDSS has been correlated with an overall reduction in broad-spectrum antibiotic use and increased susceptibility of Pseudomonas aeruginosa to imipenem and Enterobacteriaceae to gentamicin and ciprofloxacin [8, 9]. CDSS also improved clinical outcomes in a randomized controlled trial [10]. While PRF and CDSS guidance are designed to be in accordance with institutional guidelines, there are differences in acceptance of the recommendations between the 2 systems [3, 10–13]. In previous studies, PRF recommendations had an acceptance rate of 60%–70%, while CDSS acceptance was only 40% [3, 4, 13, 14]. Currently, there are limited studies comparing the combined effects of these 2 strategies [2].

At Tan Tock Seng Hospital (TTSH) in Singapore, antimicrobial stewardship has focused on PRF by a team of infectious disease doctors and pharmacists. Since 2009, this team has reviewed piperacillin-tazobactam and carbapenem orders according to hospital antibiotic guidelines from day 2 of antibiotic prescription during office hours. In 2011, we implemented a CDSS, which was triggered around the clock and at the point of antibiotic ordering of piperacillin-tazobactam and carbapenem in the electronic medical record. The compulsory CDSS provides guidance on antibiotic use and infection management based on hospital guidelines. Theoretically, compulsory CDSS may improve the timeliness of appropriate antibiotic and clinical outcomes such as mortality, but doctors may find it cumbersome and intrusive, preferring on-demand CDSS use [10]. We hypothesized that together with prospective review and feedback, voluntary, on-demand use of CDSS in ordering antibiotics would achieve similar patient outcomes compared with compulsory, automatically triggered CDSS use at the point of antibiotic ordering.

We aimed to investigate in a real-world cluster randomized controlled trial if, compared with a compulsory CDSS, voluntary use of the CDSS when piperacillin-tazobactam and carbapenem were prescribed would achieve similar clinical outcomes, antibiotic prescribing, and requirement for subsequent PRF in the individual patient.

METHODS

Study Design and Patients

ARIES is a parallel-group, 1:1 block, real-world open labeled cluster randomized crossover study conducted at TTSH, a 1700-bed teaching hospital. Waiver for informed consent was approved by the local institutional review board (DSRB/F/2015/00671). The study was conducted in 32 medical and surgical wards over a 24-week period from March 2017 to August 2017. Intensive care unit (ICU), high-dependency, and step-down care wards were excluded, as we had a specialized infectious disease team for ICU and high-dependency patients. Piperacillin-tazobactam and carbapenems are rarely used in our step-down care wards. Patients in the clusters were enrolled at first prescription of piperacillin-tazobactam or a carbapenem. Each participant was only included once.

Randomization

As modifications to the CDSS could only be done at the ward level, we clustered the patients by wards. This would also reduce contamination of intervention effects within wards, which would be present in individual patient randomization. Twenty-five wards (range) had 30 (30–41) or more patients, while 7 wards had fewer than 30 (12–29) patients. Clusters were stratified into 2 blocks based on size and allocated to an intervention or control group using a random number generator. Crossover of the study arms occurred at week 12 without a washout period.

Procedures

To promote understanding and acceptance of CDSS guidance, an educational campaign was conducted. This began 24 weeks before initiation of the study and continued until completion (October 2016 to August 2017). The campaign was comprised of a monthly package of 3-minute videos with an accompanying short quiz. This was disseminated to all doctors via email, hospital intranet, and Facebook, with weekly reminders and complimentary coffee cards to 2 doctors each month as an incentive for participation. The educational materials were developed following focus groups conducted in previous studies and a 1-day prospective evaluation of compulsory CDSS use in 81 patients [12, 14, 16]. The intervention group was comprised of 15 clusters where piperacillin-tazobactam and carbapenem could be ordered by voluntary use of the CDSS (doctors were allowed to order without using the CDSS too), while the control group had 17 clusters with compulsory use of the CDSS when ordering both types of antibiotics. In both arms, PRF occurred on day 2 of prescription and was available only during office hours. Patients’ electronic medical records at our hospital were reviewed prospectively for 6 months from the first prescription based on the periods specified in the primary and secondary outcomes. The CDSS provided antibiotic recommendations adjusted for renal function, and drug allergies were accounted for by available clinical laboratory data or manually entered data such as the type of infection, severe penicillin allergy, and dialysis status. It also provided alerts and clues to help clinicians decide on diagnosis and management. Compulsory CDSS was triggered at the point of antibiotic ordering, while voluntary CDSS was used on demand at the point of antibiotic ordering. Review of CDSS guidance is necessary once activated for the antibiotic order to be completed, but doctors are free to accept or reject its recommendations. Differences in terms of clinical workflows between CDSS and PRF are described in Supplementary Table 5.

Outcomes

To demonstrate the impact of education on acceptance of CDSS guidance, we monitored acceptance rate of CDSS recommendations for the first 12 weeks of the baseline period (April to October 2016, 24 weeks) and compared it with that of the first 12 weeks of the educational campaign (October 2016 to March 2017, 24 weeks). Data were collected for the first 1280 patients during each period.

The primary outcome was 30-day mortality from the date of the first piperacillin-tazobactam or carbapenem prescription. Secondary outcomes included number and types of recommendations from PRF, clinical response at day 7, 30-day re-infection rate, 30-day re-admission rate, length of stay, diarrhea during hospitalization, 6-month incidence of multidrug-resistant organisms, duration of index piperacillin-tazobactam or carbapenem use (days of therapy), overall hospitalization cost, and appropriateness of antibiotic use according to institutional guidelines. Trained pharmacists assigned appropriateness independent of whether CDSS were used, and they were not privy to the randomization process. Clinical response was defined as resolution of systemic inflammatory response syndrome [17]. Recommendations from the CDSS and PRF were classified into de-escalation (switch to a narrower-spectrum antibiotic), dose optimization, antibiotic spectrum optimization (increase in the spectrum of antibiotic therapy), infectious disease referral, additional investigation, and setting antibiotic duration. Multidrug-resistant organisms were defined as methicillin-resistant Staphylococcus aureus, vancomycin-resistant enterococci, third-generation cephalosporin or carbapenem-resistant Enterobacterales, and multidrug-resistant Acinetobacter baumannii or Pseudomonas aeruginosa [18] and Clostridiodes difficile. Overall, hospitalization cost was determined from the final hospital bill size in Singapore dollars (1 Singapore dollar = 1.3822 US dollars on August 8, 2019).

Statistical Analysis

Sample size calculation accounted for intention-to-treat analysis and the following assumptions: 30% of patients in control arm may be transferred out to the intervention arm; recommendations were accepted 50% of the time; the mortality rate of the intervention group was 15% [4]. Simulation using 32 clusters indicated that a total of 1280 patients (16 clusters and 640 patients in each study arm) would have a power of 80% to detect a 5% difference in mortality rates [19]. Uncertainty was set at a 5% level of significance. Equal cluster sizes were assumed. As the primary outcome was binary, intracluster correlation was not needed to simulate power. Analysis for primary and secondary outcomes was carried out by intention-to-treat and per-protocol analyses. Univariate analyses using the chi-square test for categorical variables and Mann-Whitney test for continuous variables were performed. To report time to death, re-admission, and re-infection, Kaplan-Meier estimators were calculated and plotted across time strata. Log-rank tests were performed to test for equality across interventions. Cox proportional hazards models were used to calculate the risk of various outcomes. Subgroup analysis of common departments and sources of infections was performed for 30-day mortality, re-admission, re-infection, and length of stay to identify possible confounders. All tests were done at a 5% significance level. Sample size calculation was performed in R using the clusterPower package. All other analyses were performed using STATA 13. A data safety monitoring board was convened to review the interim results of the study before crossover at week 12. The study was registered at ClinicalTrials.gov (NCT04011657).

Role of the Funding Source

The study was funded by the National Medical Research Council, Ministry of Health, Singapore (CNIG14MAY005). The funder of the study had no role in the study design, data collection, data analysis, data interpretation, or writing of the report. The corresponding author had full access to all the data in the study and final responsibility for the decision to submit for publication.

RESULTS

During the educational campaign, the number of patients who had at least 1 CDSS recommendation accepted was not significantly different during the baseline and educational phases (746/1213 [62%] vs 796/1240 [64%]; P = .11). There were more recommendations accepted during the educational phase (1300/3611 [36%] vs 1571/3640 [43%]; P < .01). Acceptance of recommendations was significantly improved for antibiotic spectrum optimization (343/894 [38%] vs 423/857 [49%]; P < .01), dose optimization (361/430 [84%] vs 442/482 [92%]; P < .01), and setting antibiotic duration (410/728 [56%] vs 509/725 [70%]; P < .01). Acceptance of recommendations was not significantly different after the campaign for de-escalation (19/347 [5%] vs 27/362 [7%]), infectious disease referral (14/228 [6%] vs 15/222 [7%]), and additional investigations (153/984 [16%] vs 155/992 [16%]).

Intraclass correlation was low and insignificant, and the percentage of total variance accounted for by the wards for 30-day mortality was 0.7% (95% CI, 0.01%–23%), for 30-day re-admission it was 2% (95% CI, 0.2%–16%), and for 30-day re-infection it was 1% (95% CI, 0.2%–12%) (Supplementary Figure 1). Therefore, we proceeded to analyze our data using the survival approach.

During the cluster randomized study from March 28, 2017, to August 28, 2017, a total of 4060 patients were prescribed piperacillin-tazobactam or carbapenems over the course of the study and screened for eligibility. One thousand two hundred fifty-seven patients from 32 clusters were randomized to voluntary (n = 641) and compulsory CDSS (n = 616) (Figure 1). Recruitment was stopped after 24 weeks. The baseline characteristics of the patients in both study arms were similar (Table 1). Most patients (92% [1161/1257]) received antibiotics for empirical therapy, with piperacillin-tazobactam accounting for 86% (1076/1257). Most patients were in the departments of General Medicine (29% [365/1257]), Geriatric Medicine (18% [223/1257]), Respiratory Medicine (9% [118/1257]), and General Surgery (7% [84/1257]). Respiratory, urinary tract, intra-abdominal, and skin and soft tissue infections were common.

Figure 1.

Trial profile. a5 antibiotics not served, 4 excluded due to inclusion for another trial, 1 excluded due to age <21 years. Abbreviations: CDSS, computerized decision support system; ICU/HD, intensive care unit/high-dependency; IQR, interquartile range; PRF, prospective review and feedback; TTSH, Tan Tock Seng Hospital.

Table 1.

Baseline Characteristics

Characteristics	Voluntary CDSS (n = 641)	Compulsory CDSS (n = 616)
Demographics
Age, median (IQR), y	74 (45–93)	76 (48–93)
Male	376 (59)	333 (54)
Surgical discipline	103 (16)	101 (16)
Charlson’s score, median (IQR)	7 (2–13)	7 (2–13)
APACHE II, median (IQR)	15 (6–28)	16 (6–29)
Transferred to ICU	38 (6)	47 (8)
Transferred to step down care	102 (16)	104 (17)
Transferred out of randomized study arm	59 (9)	61 (10)
Index antibiotic
Piperacillin-tazobactam	557 (87)	519 (84)
Carbapenem	84 (13)	97 (16)
Empiric therapy	600 (94)	561 (91)
Targeted therapy	41 (6)	55 (9)
Positive microbiology	138 (22)	129 (21)
Active empiric therapy	122/138 (88)	107/129 (83)
Source of infection
Respiratory	415 (65)	420 (68)
Urinary	109 (17)	121 (20)
Intra-abdominal	38 (6)	34 (6)
Hepatobiliary	29 (5)	23 (4)
Bone and joint	19 (3)	8 (1)
Skin and soft tissue	41 (6)	44 (7)
Vascular catheter	7 (1)	6 (1)
Neutropenic sepsis	16 (3)	11 (2)
Unknown source	66 (10)	55 (9)
Othersa	14 (2)	5 (1)

Data are No. (%), unless otherwise indicated.

Abbreviations: CDSS, computerized decision support system; ICU, intensive care unit; IQR, interquartile range.

aIncluding neurological source, ear, nose, and throat, infective endocarditis, eye, paraspinal abscess.

In the voluntary CDSS arm, 132 (21%) patients had their first course of piperacillin-tazobactam or carbapenems ordered using CDSS, compared with 612 (99%) patients in the compulsory CDSS arm. A similar proportion of patients in both study arms were reviewed subsequently by the PRF team: voluntary CDSS (488 [70%]) vs compulsory CDSS (443 [72%]). In the voluntary CDSS arm, 154 (24%) patients had broad-spectrum antibiotics started without CDSS or PRF review. The number and types of CDSS and PRF recommendations are summarized in Table 2. Overall, there were fewer CDSS recommendations in the voluntary arm compared with the compulsory arm (425 vs 1733), and about half of these were accepted (49% vs 46%). Antibiotic spectrum optimization, additional investigations, and setting antibiotic duration were the most common recommendations. De-escalation was more often recommended in the compulsory arm but had lower acceptance than in the voluntary arm. There were fewer PRF recommendations provided in the voluntary arm compared with the compulsory arm (74 vs 99), and >75% of these were accepted. The most common recommendations were de-escalation, additional investigations, and setting antibiotic duration. Among those with subsequent positive microbiology, patients with active empiric therapy were similar: voluntary CDSS (122/138 [88%]) vs compulsory CDSS (107/129 [83%]; P = .20).

Table 2.

Type of Recommendations and Their Acceptance (in Percentages) Provided for the Use of Broad-Spectrum Antibiotics Guided by Voluntary or Compulsory CDSS and PRF

Characteristics	Voluntary CDSS	Compulsory CDSS
Total CDSS recommendations	425 (49)	1733 (46)
De-escalation	15 (47)	174 (9)
Dose optimization	63 (98)	199 (95)
Antibiotic spectrum optimization	111 (51)	403 (48)
Infectious disease consult referral	24 (0)	89 (5)
Additional investigations	117 (11)	495 (17)
Setting antibiotic duration	95 (74)	373 (83)
Total PRF recommendations	74 (76)	99 (80)
De-escalation	37 (76)	42 (91)
Dose optimization	1 (0)	2 (50)
Antibiotic spectrum optimization	3 (100)	5 (60)
Infectious disease consult referral	3 (33)	3 (33)
Additional investigations	14 (64)	18 (67)
Setting antibiotic duration	16 (94)	29 (83)

Data are for the intention-to-treat population and are presented as No. (%).

Abbreviations: CDSS, computerized decision support system; PRF, prospective review and feedback.

Appropriateness of antibiotic use was similar between study arms in terms of indication, dose, and duration (Table 3). There were significantly fewer patients who received CDSS recommendations in the voluntary CDSS arm compared with the compulsory CDSS arm (132 [21%] vs 612 [99%]; P < .01). There were fewer patients who received PRF recommendations in the voluntary CDSS arm compared with the compulsory arm, but it was not statistically significant (62 [10%] vs 81 [13%]; P = .05). When either CDSS or PRF recommendations were provided, most patients (>90%) had at least 1 recommendation accepted. Per-protocol analysis showed similar trends (Table 4).

Table 3.

Appropriateness of Antibiotic Use, Acceptance of Recommendations, and Outcomes of Patients who Received Broad-Spectrum Antibiotics Guided by Voluntary or Compulsory CDSS and PRF Recommendations

Characteristics	Voluntary CDSS (n = 641)	Compulsory CDSS (n = 616)	P
Reviewed by PRF	448 (70)	443 (72)	.43
Appropriate indication under PRF reviews	351/448 (78)	330/443 (75)	.18
Appropriate dose	625 (98)	599 (97)	.77
Appropriate duration	587 (92)	548 (89)	.12
Recommendations
Received CDSS recommendations	132 (21)	612 (99)	<.01
Accepted CDSS recommendationsa	130 (20)	556 (90)	<.01
Received PRF recommendations	62 (10)	81 (13)	.05
Accepted PRF recommendationsa	51 (8)	71 (12)	.03
Outcomes
30-d mortality	123 (19)	102 (16)	.22
30-d re-infection rate	132 (21)	142 (23)	.29
30-d re-admission rate	92 (14)	87 (14)	.91
Clinical response at day 7	535 (83)	517 (84)	.82
Length of stay, median (IQR), d	15 (5–64)	15 (4–70)	.92
6-mo multidrug-resistant organismsb	152 (24)	171 (27)	.10
Diarrhea this admission	89 (14)	86 (14)	.96
Index antibiotic days of therapy, median (IQR)	4 (3–5)	4 (3–5)	.47
Index antibiotic days of therapy ≤3	295 (46)	297 (48)	.45
Gross hospitalization costs, median (IQR), S$	13 301 (7184–24 079)	13 308 (6743–24 904)	.96

Data are presented as No. (%), unless otherwise indicated.

Abbreviations: CDSS, computerized decision support system; IQR, interquartile range; PRF, prospective review and feedback.

aPatients were considered to have recommendations by CDSS or PRF accepted if at least 1 of the recommendations provided by the respective service was accepted.

bMultidrug-resistant organisms were defined as methicillin-resistant S. aureus, vancomycin-resistant enterococci, third-generation cephalosporin or carbapenem-resistant Enterobacterales, and multidrug-resistant A. baumanii or P. aeruginosa and Clostridiodes difficile diarrhea. Data are for the intention-to-treat population.

Table 4.

Appropriateness of Antibiotic Use, Acceptance of Recommendations, and Outcomes of Patients who Received Broad-Spectrum Antibiotics Guided by Voluntary or Compulsory CDSS and PRF Recommendations

Characteristics	Voluntary CDSS (n = 455)	Compulsory CDSS (n = 416)	P
Reviewed by PRF	324 (71)	293 (70)	.80
Appropriate indication under PRF reviews	259/324 (80)	215/293 (73)	.05
Appropriate dose	443 (97)	402 (97)	.52
Appropriate duration	424 (93)	373 (90)	.06
Recommendations
Received CDSS recommendations	91 (20)	412 (99)	<.01
Received PRF recommendations	41 (9)	50 (12)	.15
Outcomes
30-d mortality	85 (19)	85 (20)	.52
30-d re-infection rate	113 (25)	106 (26)	.83
30-d re-admission rate	85 (19)	79 (19)	.91
Clinical response at day 7	383 (84)	344 (83)	.56
Length of stay, median (IQR), d	12 (4–41)	12 (4–35)	.26
6-mo multidrug-resistant organisms	91 (20)	94 (23)	.35
Diarrhea this admission	59 (13)	61 (15)	.47
Index antibiotic days of therapy, median (IQR)	4 (3–5)	3 (3–5)	.23
Index antibiotic days of therapy ≤3	213 (47)	209 (50)	.31
Gross hospitalization costs, median (IQR), S$	10 520 (5826–18 430)	9671 (5734–17 576)	.43

Data are for the per-protocol population and are presented as No. (%), unless otherwise indicated.

Abbreviations: CDSS, computerized decision support system; IQR, interquartile range; PRF, prospective review and feedback.

There were similar 30-day mortality (hazard ratio [HR], 0.87; 95% CI, 0.67–1.12), 30-day re-infection (HR, 1.15; 95% CI, 0.91–1.46), and 30-day re-admission rates (HR, 0.99; 95% CI, 0.74–1.33) between voluntary and compulsory CDSS (Figure 2, Table 3). There was no difference in clinical response at day 7 between the voluntary and compulsory arms (106 [17%] vs 99 [16%]; P = .22). The median length of hospital stay (interquartile range [IQR]) was similar (15 [5-64] days vs 15 [4-70] days; P = .92). Incidence of diarrhea during admission and 6-month acquisition of multidrug-resistant organisms were not significantly different. The median days of therapy of index antibiotic use (IQR) were similar (4 [3-5] days vs 4 [3-5] days; P = .47). The overall median hospitalization cost (IQR) was not significantly different between the voluntary and compulsory CDSS arms in intention-to-treat (SG$13 302 [S$3221–S$67 110] vs SG$13 307 [S$3064–S$64 666; P = .91) (Table 3).

Figure 2.

Survival analysis of 30-day mortality in patients who received broad-spectrum antibiotics guided by a voluntary or compulsory computer decision support system and prospective review and feedback recommendations. Data are for the intention-to-treat population. Abbreviations: CDSS, computerized decision support system; HR, hazard ratio.

Subgroup analysis of the top 4 common departments, namely General Medicine, Geriatric Medicine, Respiratory Medicine, and General Surgery, was performed. There was no difference in clinical outcomes, length of stay, hospitalization cost, or duration of index antibiotics between the study arms in patients of General Medicine and Respiratory Medicine (Supplementary Table 1). Subgroup analysis of common infections, namely respiratory, urinary tract, intra-abdominal, and skin and soft tissue infections did not identify any differences between the study arms (Supplementary Table 2). Among Geriatric Medicine patients, the median length of stay (IQR) was significantly higher (19 [5-83] days vs 14 [4-43] days; P = .03) in the voluntary CDSS arm, corresponding to a significantly higher median overall hospitalization cost (S$13 945 [S$3706–S$57 133] vs S$10 444 [S$3099–S$31 276]; P = .02). There was no difference in patient characteristics between both arms, except fewer patients in the voluntary CDSS arm received (29 [27%] vs 114 [100%]; P < .01) and accepted (24 [22.0%] vs 84 [74%]; P < .01) CDSS recommendations (Supplementary Table 3). Among General Surgery patients, the median length of stay (IQR) was not significantly higher in the voluntary CDSS arm (20 [7-74] vs 16 [6-40]; P = .075); however, the median overall hospitalization cost (IQR) was significantly higher in the voluntary CDSS arm compared with the compulsory CDSS arm (S$35 303 [S$5249–S$82 634] vs S$20 994 [S$4333–S$61 243]; P < .01). Notably, appropriate indications of antibiotics were lower in both departments between voluntary and compulsory CDSS compared with the overall study (351 [78%] vs 330 [75%]; Geriatric Medicine: 52 [48%] vs 57 [50%]; P = .50; General Surgery: 26 [68%] vs 33 [72%]; P = .34). There were no significant differences in patient characteristics in the study arms, other than more CDSS recommendations and acceptance in the compulsory CDSS arm (Supplementary Tables 3 and 4).

DISCUSSION

Piperacillin-tazobactam and carbapenems prescribed in the setting of voluntary CDSS use had similar clinical outcomes and appropriateness of use when compared with compulsory CDSS. It did not increase the need for PRF recommendations by the AMS team. However, compulsory CDSS was associated with significant reduction in hospital length of stay and hospitalization costs for patients when antibiotic appropriateness was low.

In the setting of high appropriate antibiotic use and PRF, it is likely that compulsory CDSS may not have clinical benefits and may inconvenience doctors by causing delay, distraction, or irritation. A more sophisticated CDSS that uses artificial intelligence or machine learning to diagnose an infection rather than relying on doctors to enter their clinical diagnosis may be better. However, benefits may be magnified in settings with lower appropriateness of antibiotic use and when CDSS is implemented as a new system [9]. Notably, voluntary CDSS with PRF did not lead to differences in appropriateness of antibiotic use, duration of index antibiotic use, or clinical outcomes. A recent meta-analysis concluded that CDSS improved the adequacy of antibiotic coverage (measured as compliance with guidelines) and marginally lowered mortality [21]. In addition, a separate report from our group conducted just after CDSS implementation between 2011 and 2012 reported mortality benefit with receipt of CDSS antibiotic recommendations [11]. Our study was conducted several years after implementation of hospital antibiotic guidelines and PRF in 2009 and compulsory CDSS in March 2010. Doctors in the hospital had substantial experience with these interventions before the start of our study in March 2017. The high coverage of PRF may have addressed any possible difference between voluntary and compulsory CDSS use too. Further studies are needed to study the impact of mixed strategies of antibiotic stewardship in hospitalized patients [20].

Our study provided novel insights on the concurrent use of 2 common antimicrobial stewardship strategies of CDSS and PRF deployed in different ways. Piperacillin-tazobactam and carbapenems were mainly used for empiric therapy, and most CDSS recommendations to optimize the antibiotic spectrum suggest additional investigations and setting antibiotic duration. CDSS rarely provided de-escalation recommendations compared with PRF. PRF occurred on day 2 and subsequent days until the antibiotic was stopped. Additional clinical information or changes in patient condition could have driven these differences. Although <50% of CDSS recommendations were accepted in both the compulsory and voluntary CDSS arms, it was interesting to note that >90% of patients in both arms had at least 1 recommendation accepted. A separate cohort study on our CDSS for piperacillin-tazobactam and carbapenem prescriptions found that almost 50% were ordered after office hours [16]. Dose and antibiotic spectrum optimization were the most frequently accepted CDSS recommendations; it is important that these factors be correct early in the treatment of infection.

There are limited randomized studies on CDSS in AMS with mortality as a primary outcome, and these have focused on other surrogate outcome measures and have not reported on the nonexpert end-user workflow [20-22]. We studied mortality as the primary outcome, and our CDSS is integrated with clinical workflow as it is made available at the time of antibiotic prescription [14]. It provides recommendations for investigations and referrals in addition to antibiotics. We studied the implementation hurdles of our CDSS, patients’ and physicians’ predictors, and the psychosocial determinations of physicians’ acceptance of CDSS recommendations previously [11, 12, 14]. We then designed an educational campaign aimed to optimize the nonexpert end-user usage of our CDSS.

Our study did not evaluate the effect of CDSS on other antibiotics such as fluoroquinolones and third-generation cephalosporins. Between clusters, there may be contamination, possible Hawthorne effect, and bias between the study arms due to doctors’ rotations, doctors managing patients in both study arms concurrently on different wards, and patient transfers. We adopted a crossover design to adjust for these effects [15], as CDSS and PRF were considered standard of care at our hospital and were recommended by AMS guidelines [1]. We were unable to introduce a washout period before crossover or have a standalone CDSS or PRF study arm. Hence, we were not able to fully address effects of CDSS on mortality because of the concurrent use of PRF. COMPASS, a cluster randomized controlled trial focusing only on CDSS use, is ongoing. However, the primary outcome is overall antibiotic use [23]. Further studies are needed to evaluate the effects of CDSS on AMS and mortality.

CONCLUSIONS

Voluntary CDSS for piperacillin-tazobactam and carbapenem prescriptions with PRF did not result in differing clinical outcomes, antibiotic duration or length of stay, or PRF recommendations compared with compulsory CDSS and PRF. However, in geriatric medicine patients, where appropriateness of antibiotics was lower, compulsory CDSS with PRF resulted in lower length of stay and overall hospitalization cost.

Supplementary Data

Supplementary materials are available at Open Forum Infectious Diseases online. Consisting of data provided by the authors to benefit the reader, the posted materials are not copyedited and are the sole responsibility of the authors, so questions or comments should be addressed to the corresponding author.

Click here for additional data file.