Long-term effect of computer-assisted decision support for antibiotic treatment in critically ill patients: a prospective 'before/after' cohort study.

OBJECTIVES: Antibiotic resistance has risen dramatically over the past years. For individual patients, adequate initial antibiotic therapy is essential for clinical outcome. Computer-assisted decision support systems (CDSSs) are advocated to support implementation of rational anti-infective treatment strategies based on guidelines. The aim of this study was to evaluate long-term effects after implementation of a CDSS.
DESIGN: This prospective 'before/after' cohort study was conducted over four observation periods within 5 years. One preinterventional period (pre) was compared with three postinterventional periods: directly after intensive implementation efforts (post1), 2 years (post2) and 3 years (post3) after implementation.
SETTING: Five anaesthesiological-managed intensive care units (ICU) (one cardiosurgical, one neurosurgical, two interdisciplinary and one intermediate care) at a university hospital. PARTICIPANTS: Adult patients with an ICU stay of >48 h were included in the analysis. 1316 patients were included in the analysis for a total of 12,965 ICU days. INTERVENTION: Implementation of a CDSS. OUTCOME MEASURES: The primary end point was percentage of days with guideline adherence during ICU treatment. Secondary end points were antibiotic-free days and all-cause mortality compared for patients with low versus high guideline adherence. MAIN
RESULTS: Adherence to guidelines increased from 61% prior to implementation to 92% in post1, decreased in post2 to 76% and remained significantly higher compared with baseline in post3, with 71% (p=0.178). Additionally, antibiotic-free days increased over study periods. At all time periods, mortality for patients with low guideline adherence was higher with 12.3% versus 8% (p=0.014) and an adjusted OR of 1.56 (95% CI 1.05 to 2.31).
CONCLUSIONS: Implementation of computerised regional adapted guidelines for antibiotic therapy is paralleled with improved adherence. Even without further measures, adherence stayed high for a longer period and was paralleled by reduced antibiotic exposure. Improved guideline adherence was associated with reduced ICU mortality. TRIAL REGISTRATION NUMBER: ISRCTN54598675. Published by the BMJ Publishing Group Limited. For permission to use (where not already granted under a licence) please go to http://group.bmj.com/group/rights-licensing/permissions.

RCT Entities:   Population Interventions Outcomes

No previous study has examined the possible long-term effect of a computerised decision support system for antibiotics on adherence to locally adapted guidelines developed by a multidisciplinary team.

The results suggest that adherence to locally adapted guidelines increased following implementation of a computerised decision support system and remained at a reasonable level.

This was paralleled by an increase of antibiotic-free days and lower mortality was observed in patients with high guideline adherence.

Data from this study are based mainly on surgical patients in one university hospital; consequently, external validity is restricted mainly to the postoperative setting and we do not know whether our findings are reliable for other facilities. The unblinded nature and non-randomised design are further study limitations.

Introduction

The use of antibiotics in medicine, which has led to the emergence and dissemination of resistant pathogens,1 2 has major implications for morbidity and mortality in intensive care units (ICUs).3 Early adequate therapy is crucial for the outcome of critically ill patients.4 5 Moreover, it has implications for healthcare costs.6 Despite this, the rate of inappropriate antibiotic usage remains high.7 Barriers to change management have been identified in different healthcare settings. However, although intensive efforts have demonstrated some potential to improve practice, longer-term follow-up has failed to show sustained effects.8 Reasons for delays in initiating adequate therapy include the lack of availability of information and difficulties with prompt prescribing typically due to inefficient communication between staff members. On the other hand, initiating and performing microbiological diagnostics is also challenging, but there is evidence that a less aggressive treatment strategy might be beneficial for patient outcomes as long as expedient microbiology is available.9 For this reason, experts from all over the world have encouraged the development of stewardship programmes as a major goal for the future.10 11 Although locally adapted guidelines may help to enhance microbiological diagnostics and initial antibiotic therapy, these are not always readily available. Thus, computerised decision support systems (CDSSs) have been developed and studied over the past several years.12 13 In the 1990s, Evans et al13 developed a system for antimicrobial therapy and were able to show a reduction in inpatient costs and length of stay. Buising et al14 were able to show that, as an educational measure, a CDSS was the single most efficient method of improving guideline implementation for the treatment of community-acquired pneumonia compared with education alone.

However, there is currently no evidence regarding the long-term efficacy of a CDSS to improve guideline adherence to the treatment of the most relevant infections in a surgical ICU setting. For this study we hypothesised that implementation of a CDSS for antibiotic therapy, specifically one created by a multidisciplinary front-line team, is able to increase and maintain guideline adherence without further efforts. Presumably, this should lead to decreased antibiotic usage and improved clinical outcome.

Material and methods

This prospective cohort study was conducted as a clinical prospective prentervention/postintervention study over four evaluation time periods within 5 years totalling 12 965 patient days in the ICU. The preinterventional period (‘pre’) took place from January to April 2006, the first postinterventional period (‘post1’) from February to April 2007, post2 from August to October 2009 and post3 from February to March 2010.

During the pre-period, paper-based guidelines for antibiotic therapy were made available to every ward physician. Guidelines incorporated national and international guidelines for diagnostics as well as for therapy and were adjusted to the institutional resistances rates. These paper-based guidelines contained a short definition, diagnostic requirements, the spectrum of causative pathogens and empiric therapy recommendations for every infection.

Following this baseline time period, a CDSS for antibiotic therapy was designed as a tool to provide clinicians with essential guideline-based information concerning the treatment of infections, the use of antibiotic agents and microbiological diagnostics in the ICU. The CDSS could be accessed from every computer in the hospital, including all computers used to document electronic patient files. The CDSS contains algorithms and pathways for most infections occurring in patients in surgical ICUs. For every infection included in the CDSS, there is one main five step algorithm. This algorithm leads the user to the first interface to reassess infection signs to evaluate whether diagnosis of an infection is plausible. Next, the user is guided to a page where the suspected infection focus has to be identified. Then, there is a request for differential diagnoses and focus specifications, followed by a fourth page showing diagnostic procedures that are needed for the specific focus. In the fifth step there is a page where suggestions are made for empiric therapy (compare figure 1A). Additionally, the CDSS includes links to references and background information focusing on the infections covered and also points out the tools required, for example, calculators for renal creatinine clearance. Local resistance patterns of bacterial pathogens were included for every study ward (compare figure 1B–C). Each algorithm was developed by a multidisciplinary team of ICU experts from the fields of microbiology, infectious diseases, surgery, pharmacology and specialists for each focus of the algorithms. The team also included front-line providers of patient care and those with less professional experience (residents and medical students).

Figure 1

Presentation of different user interfaces in the computerised decision support program (available at http://www.dgai-abx.de/en/). (A) Summary of infection characteristics; (B) selection menu in the module resistance patterns and targeted therapy; (C) selection menu in the antibiotic agents module. BAL, bronchoalveolar lavage; CPIS, Clinical Pulmonary Infection Score; MRSA, methicillin-resistant Staphylococcus aureus; VAP, ventilator-associated pneumonia.

Continued.

The CDSS also contains a section for bacterial pathogens and their targeted treatment (compare figure 1B). In patients with microbiological pathogen identification the CDSS can be used to guide choice of targeted antibiotic therapy as well. In summary, the programme was initially developed to support empiric antibiotic therapy but also addressed de-escalation strategies and targeted antibiotic therapy when pathogens were identified in microbiological diagnostics. The programme also incorporates educational aspects with tools and evidence from the current medical literature to support recommendations given. After the programme was finished and released, it was introduced in every ICU of the department, giving every staff member the opportunity to be trained in the applications. The programme included a contact link to communicate with the team responsible for drafting the CDSS. Furthermore, one physician provided telephone support. All queries were immediately discussed in the team and, if deemed relevant, incorporated into the decision pathways. One full-time physician was responsible for changing the programme, for example, if new drugs were available or locally updated resistance was available during the course of the study.

Data were collected in five predominantly surgical ICU with altogether 61 beds at the Charité University Hospital in Berlin, Germany, a tertiary medical care centre with 3200 beds. In these ICUs there is constant rotation with every physician working for at least 2 years. In the beginning there is structured training that includes the usage of the antibiotic CDSS. There is always a well-trained senior together with a younger doctor. Additionally, there is a consultant either in the ward or reachable via phone, which is especially critical during night shifts. This team provides 24/7 care for patients. During the study period, there was no restriction of diagnostic procedures or choice of antibiotic therapy for participating ICUs. Antibiotic therapy was prescribed by the physicians in charge under approval of the senior consultant. Most patients were admitted to the ICU postoperatively from different surgical disciplines (abdominal surgery, cardiac surgery, neurosurgery, trauma surgery and gynaecology); other patients included non-operative patients suffering from acute respiratory distress syndrome. For this study we included every consecutive patient older than 18 years of age with a length of stay in the ICU >48 h.

Measurements and data

Patient data were collected directly into electronic case report files for each treatment day in the ICU. Data on gender, age, medical history including substance abuse, surgical procedures, length of stay and mode of discharge (intensive care, intermediate care, normal care or death), vital signs, laboratory findings, microbiological and radiological diagnostics, and anti-infective therapy were obtained. To ensure data consistency, study data were re-evaluated in the middle and at the end of each collection period. ICU scores (SOFA and SAPS II), which are assessed regularly in the participating ICUs, were also documented. Additionally, infection status and the supposed or known focus of infection were documented daily. Immunosuppression was defined as receiving immunosuppressive medications such as corticosteroids above the Cushing level, immune suppressive agents, monoclonal antibodies or chemotherapy within the past 6 weeks. Microbiological findings were grouped by fungal pathogens, Gram-positive and Gram-negative pathogens and pathogens with enhanced resistance patterns. The latter included intrinsic resistances as well as acquired resistances, which are known to be more difficult to treat.

Calculation of the adherence rate

At the end of each evaluation period a team of infectious disease and critical care specialists assessed the antibiotic therapy in terms of guideline adherence. To justify infection management, diagnostic investigations and antibiotic therapy of suspected or known infection were compared with guideline recommendations.15 An ICU day was defined as discrepant from guidelines if one or more of the following criteria were identified: (A) disregarding drug history (eg, starting an antibiotic therapy for a new infection using the same agent that has recently been given to this patient); (B) insufficient diagnostic measurements (eg, no microbiological samples before start of antibiotic therapy, or no X-ray before starting a treatment of suspected pneumonia); (C) delayed antibiotic therapy (eg, documentation of infection with accidental delay of empirical antibiotic therapy); (D) unsuitable combination therapy (eg, combination of two β-lactam antipseudomonal antibiotics); (E) omission of combination drug (eg, inappropriate coverage in patients with increased methicillin-resistant Staphylococcus aureus risks); (F) antibiotic therapy in patients without signs of infection (eg, prolonging antibiotic prophylaxis or treating SIRS with no signs of infection) or (G) continuing empirical broad spectrum coverage without antibiotic de-escalation in patients with documented pathogen detection causative for infection. If the physician who cared for the patients on the ICUs had documented a justification for non adherence, the day was counted as adherent.

After the initial evaluation, data were audited by experts independent from the study team. The expert team was unblinded to the patients and the data collecting part of the study but blinded to the evaluation of the adherence rating of the study team. The members of the expert team had open access to every patient's data file and were able to randomly scrutinise the data and evaluate the adherence independently from the study team. The decision of the expert team was determining. None of the persons evaluating adherence were involved in decision-making on the ward at any time for any patient.

To calculate adherence rates, each treatment day during the study period was recorded and evaluated as described in detail elsewhere.16 Briefly, the binary variable of daily adherence was transferred into a relative per-patient variable of guideline adherence by dividing the cumulative number of days with guidelines conformity by the patient's total number of ICU days. The resulting variable represents a summary of therapy quality throughout the ICU stay and has previously been found to be related to outcome.16

End points

The primary end point is guideline adherence over 5 years. Since resistances change over time, guidelines could be modified during each evaluation period if new microbiological findings became available.

For the evaluation of all-cause ICU mortality, patients were followed until ICU discharge. The patient sample was dichotomised based on a clinically relevant threshold of 70% adherence rate to assess the association of ICU mortality and quality of guideline adherence. Outcomes in patients with high guideline adherence rates of at least 70% were compared to those with lower adherence rate. The cut-off of 70% was selected because it is the quality indicator for ICU implementation rate of our certified institutional quality management according to DIN EN ISO 9001:2000 standard. This cut-off was also taken because 70% accounts for a sufficient first step in the meaning of improving guideline adherence according to the literature and has been published by our working group as having influence on the outcome of ICU patients.15–18

Statistics

All statistical analyses were performed with PASW V.19 (SPSS Inc, USA). Categorical data are displayed as absolute and relative values. Fisher's exact and χ2 tests were applied for statistical hypothesis testing of categorical variables. Continuous data are displayed as means and consecutive SD or median and quartiles depending on their distribution. The Kruskal-Wallis tests and analyses of variance were performed for continuous data accordingly. The significance level was defined as a two-sided p<0.05 for all tests.

Variables were additionally evaluated using multivariate logistic regression analyses. For this, we first examined those factors that were clinically important in univariate analysis focusing on long-term effect of the intervention (study groups pre vs post3). In the consecutive regression model, all parameters were entered that showed a significant difference in the univariate analyses resulting in seven factors (surgical patient, pre-existing pulmonary disease, chronic liver disease and kidney failure, diabetes mellitus, psychiatric disease and guideline adherence). In a second regression analysis on ICU mortality, we also used this approach and evaluated baseline characteristics in regard to the interesting factor of high versus low guideline adherence; the consecutive regression analysis had to include eight factors (age, immune suppression, infection of bones or joints, soft tissue infection, unknown focus of infection, fungal infection, detection of multidrug resistant pathogens and guideline adherence).

Ethics and data safety

The institutional review board waived the need for informed consent owing to the observational character of the study.

Results

Patient and basic characteristics

A total of 1395 patients were treated during the study period. After exclusion of patients with treatment in ICU of less than 48 h, 1316 patients were included in present analysis. Basic characteristics are shown in table 1. Patients included during the four evaluation periods were comparable; however, differences were seen in the distribution of age, gender and pre-existing conditions. An increase of concomitant pulmonary disease and chronic kidney failure over time was noted. Pneumonia and abdominal or surgical site infections accounted for most ICU infections observed.

Table 1

Basic characteristics

	Pren=328	Post1n=311	Post2n=384	Post3N=293	p ValueOverall
Age in years median (25–75%Q)	66 (51–75)	67 (53–74)	62 (51–72)	64 (51–72,5)	0.031
Females (%)	138 (42.1%)	137 (44.1%)	153 (39.8%)	137 (46.8%)	0.321
Initial SAPS median (25–75%Q)	34.5 (25.3–44)	37 (28–48)	35 (24–48.8)	36 (25.5–49)	0.179
Initial SOFA median (25%–75%Q)	4 (2–7)	5 (3–7)	4 (1–8)	5 (1.5–8)	0.118
Comorbidities n (%)
Immune suprressioni	37 (11.3%)	11 (3.5%)	22 (5.7%)	29 (9.9%)	<0.01
Cardiovascular	148 (45.1%)	143 (46%)	147 (38.3%)	150 (51.2%)	0.009
Pulmonary	35 (10.7%)	40 (12.9%)	79 (20.6%)	66 (22.5%)	<0.01
Chronic liver disease	17 (5.2%)	25 (8.0%)	42 (10.9%)	36 (12.3%)	0.007
Chronic kidney failure	58 (17.7%)	45 (14.5%)	84 (21.9%)	80 (27.3%)	0.001
Metabolic disease	169 (51.5%)	123 (39.5%)	142 (37%)	123 (42%)	0.001
Psychiatric disease	75 (22.9%)	48 (15.4%)	24 (6.2%)	35 (11.9%)	<0.01
Surgical vs medical (%)	278 (84.8%)	252 (81%)	282 (73.4%)	223 (76.1%)	0.001
Infections n (%)*
Pneumonia	117 (35.7%)	92 (29.6%)	131 (34.1%)	94 (32.1%)	0.387
Abdominal infections	44 (13.4%)	22 (7.1%)	33 (8.6%)	24 (8.2%)	0.037
Urinary tract infections	29 (8.8%)	28 (9%)	23 (6%)	20 (6.8%)	0.346
Bone or joint infections	16 (4.9%)	2 (0.6%)	10 (2.6%)	7 (2.4%)	0.009
Endocarditis	7 (2.1%)	8 (2.6%)	10 (2.6%)	5 (1.7%)	0.897
Wound and soft tissue	90 (27.4%)	32 (10.3%)	17 (4.4%)	8 (2.7%)	<0.01
Meningoencephalitis	9 (2.7%)	1 (0.3%)	18 (4.7%)	9 (3.1%)	0.002
Bacteraemia	23 (7%)	26 (8.4%)	27 (7%)	20 (6.8%)	0.879
No focus specified	27 (8.2%)	29 (9.3%)	9 (2.3%)	24 (8.2%)	<0.01
Catheter associated	7 (2.1%)	15 (4.8%)	3 (0.8%)	12 (4.1%)	0.003
Pathogens
Clostridium difficile infections	3 (0.9%)	2 (0.6%)	10 (2.6%)	6 (2%)	0.138
Fungi	61 (18.6%)	50 (16.1%)	46 (12%)	61 (20.8%)	0.012
Multidrug resistant pathogens	42 (12.8%)	27 (8.7%)	32 (8.3%)	34 (11.6%)	0.155

*Owing to occurrence of more than one infection per patient and polymicrobial infections, the detection of more than one pathogen is possible.

iImmunosuppressive medications like corticosteroids above cushing level, immune suppressive agents, monoclonal antibodies or chemotherapy within the last 6 weeks.

Primary endpoint: adherence to locally adapted guidelines

Adherence to guidelines was 61.4±37% in the pre period. Guideline adherence increased significantly to 92±19% in post1 (pre/post1 p<0.001). It then decreased to 76.3±30.7% (post1/post2 p<0.001) and remained constant at 71.1±34.0% in post3 (post2/post3 p=0.178, pre/post 3 p=0.001). Thus, a net increase of 10% guideline adherence was observed during the 5-year observation period. Figure 2 shows the guidelines adherence in connection with the antibiotic free days.

Figure 2

Adherence to guideline and percentage of antibiotic free days over study periods. MSSA, methicillin-susceptible Staphylococcus aureus; MRSA, methicillin-resistant Staphylococcus aureus; S. epidermidis Staphylococcus epidermidis.

Confounder analysis in the relationship of guideline adherence and study periods

The validation of the primary endpoint, guideline adherence, between pre and post3 is shown in table 2. The following variables were included in a logistic regression analysis as independent variables based on baseline characteristics: surgical patient, pre-existing pulmonary disease, chronic liver disease and kidney failure, diabetes mellitus, psychiatric disease and guideline adherence. After including these variables, the positive association between guideline adherence and time period (pre vs post3) remained statistically significant with an adjusted OR of 1.905 (1.359 to 2.669; p<0.001).

Table 2

Logistic regression analysis of the relationship between guideline adherence and study period (response: study period)

	Multivariate logistic analysisOR (95% CI)	p Value
Surgery	0.608 (0.396 to 0.932)	0.023
Chronic pulmonary disease	2.362 (1.462 to 3.816)	<0.001
Chronic liver disease	3.035 (1.564 to 5.890)	0.001
Chronic kidney failure	1.634 (1.068 to 2.500)	0.024
Metabolic disease	0.643 (0.454 to 0.909)	0.012
Psychiatric disease	0.382 (0.237 to 0.616)	<0.001
Guideline adherence	1.905 (1.359 to 2.669)	<0.001

Secondary end points

Measures of clinical performance and differences in antibiotic therapy are shown in table 3. Notably, antibiotic-free days (AFD) increased over time starting from 30% of all ICU days (pre) to 32% (post1), 46% (post2) and finally achieving 42% (post3), with a corresponding p<0.01 over all time periods.

Table 3

Secondary end points

	Pre	Post1	Post2	Post3
Characteristics	N=328 (24.9%)	N=311 (23.6%)	N=384 (29.2%)	N=293 22.3(%)	p Value
Length of ICU stay (days)	9.2±10.7	9.1±9.6	9.9±12.1	11.3±12.2	<0.01
Mortality (%)	34 (10.4%)	34 (10.9%)	32 (8.3%)	26 (8.9%)	0.624
Antibiotic-free days (%)	30±27	32±28	46±39	44±38	<0.01

Mortality and confounder analysis in the relationship of mortality and guideline adherence

Mortality during each evaluation period was lower for patients with guideline adherence above 70%. Of 471 patients with an adherence less than 70%, 58 died (12.3%) compared with 68 of 845 with an adherence greater than 70% (8%; p=0.014). Based on this model, guideline adherence remained statistically significant associated with ICU mortality with an adjusted OR of 1.556 (95% CI 1.05 to 2.31; table 4).

Table 4

Logistic regression analysis of the relationship between guideline adherence and ICU-mortality (response: ICU-mortality)

	Multivariate logistic analysisOR (95% CI)	p Value
Guideline low vs high adherence	1.556 (1.047 to 2.314)	0.029
Age	1.028 (1.014 to 1.042)	<0.001
Immune suppression	1.279 (0.652 to 2.509)	0.474
Bone or joint infection	1.168 (0.416 to 3.275)	0.768
Unknown focus	1.252 (0.659 to 2.376)	0.492
Soft tissue infection	1.023 (0.572 to 1.829)	0.939
Fungus	2.862 (1.864 to 4.394)	<0.001
Multidrug resistant pathogens	1.335 (0.769 to 2.318)	0.305

Hosmer and Lemeshow test. χ2 14.82; p=0.63.

Discussion

Adherence to locally adapted guidelines increased following implementation of a CDSS. Meaningful improvement in adherence was observed at 5 years without any further measures or interventions. Antibiotic-free days increased accordingly and lower mortality was found in patients with guidelines adherence above 70%.

Although some of the sample characteristics varied throughout the evaluation periods during this study, these variations were not unexpected, based on previous studies.19 For instance, the prevalence of pre-existing conditions changed over the 5-year course of this study with increased rates of concomitant pulmonary diseases and chronic kidney failure over time. This is consistent with other reports that the number of chronically ill patients increases over time.20 Distribution of infections showed significant variation between the four evaluation periods. Since infections are dependent on different factors, this may be due to seasonal changes or a random spread. After adjusting for potential confounders in a multivariable regression model including relevant factors, guideline adherence was higher in the postintervention periods independent of other pre-existing conditions.

Guideline adherence was highest in the period directly after implementation of the CDSS. Analysis for confounders with univariate logistic regression illustrated that although all relevant variables influenced the primary end point, adherence to guidelines remained significant on its own. The model showed that the likelihood of having higher guideline adherence in the period post3 was still high and not due to a different health status. Guideline adherence decreased initially after the implementation period but remained stable above a critical value of 70%, which is a well-accepted value for guideline adherence.16–18 The level of guideline adherence observed in this study is in line with other studies of antibiotic prescribing employing a similar persuasive methodology.21 Interestingly, after an initial peak in guideline adherence, the long-term adherence in our study remained high compared with the preinterventional period. By contrast, previous studies of efforts to improve guideline adherence have shown that the effect of such interventions is often short-lived. For instance, Ferrer et al8 were able to show a significant increase in guideline adherence after an extensive educational programme; however, without any further intervention the adherence rate 1 year later had fallen nearly to the baseline level. The sustained improvement in guideline adherence observed in our study may be due in part to the process of developing the CDSS, which actively involved ward physicians as a target user group. Similarly, previous studies of quality indicators have shown that implementation works best if front-line staff are involved in the development process.22 One favourable point in our preparation of the locally adapted guidelines is that it helps to overcome a ‘prescribing etiquette’ and the social interferences with reasonable prescriptions described very recently by Charani and colleagues.23 24 By including a clinical hierarchy into the preparation and giving the tool to junior doctors, overall compliance was enhanced; however, we did not address reasons for the higher compliance in specific populations. This suggestion is supported by the meta-analysis by Davey et al,7 who contend that after the initial benefit of restrictive measures, the effects of persuasive interventions, such as ours, are sufficient to keep compliance high. Another advantage is that the computer-based design is well suited to the demands of evidence-based care by keeping doctors abreast of current findings and relevant prescribing information, which are otherwise difficult to address in the clinical workflow.25

Guidelines adherence is a process parameter and does not provide a sufficient measure of the quality of care on its own. Dangers associated with the overuse of antibiotics are well recognised,12 and one objective of antimicrobial stewardship programmes is to reduce unnecessary use of antibiotics.7 In our data set, the number of AFD increased over time. Thus, our data suggest that an individual patient receives shorter courses of antibiotics as guideline adherence increases. This is in line with previous findings that a CDSS can decrease antibiotic usage.26 The use of antibiotics in the post1 period did not fall as much as anticipated in light of the high percentage of adherence. It appears that more time was needed to change the minds of doctors in the ward, as evidenced by the continued increase in ADF over the 5-year observation period. This hypothesis is supported by Samore et al27 in a study of a CDSS on the ambulant prescribing of antibiotics. They found a significant reduction in overall antibiotic use and improved appropriateness of antibiotic selection was observed after CDSS implementation. This group also described that although the rate of prescription did not change much in the first year following the introduction of a CDSS, adequacy did improved. In the second year, they found a decrease in the prescription rate, which translated into a decrease in unnecessary use.27 Unfortunately, this study lacked long-term follow-up; thus, no conclusions may be drawn about the sustained effects of those measures. Evans et al13 were able to show that after the introduction of a CDSS, patients received fewer antibiotics and had a shorter duration of antibiotic therapy. However, this study also only had a 2-year observation period.

Mortality persisted at the same level throughout our study despite the higher rates of pre-existing conditions (eg, greater comorbidity) in the later evaluation periods. To assess impact of guideline adherence on outcome, a 70% cut-off was chosen as first target for adherence to guideline.16 18 We would like to consider this threshold only as a first barrier in change management as it obviously does not reflect the adherence rate that should be achieved in the long run. Nevertheless, mortality for patients with an adherence above 70% was lower than for patients with lower adherence rates. This association persisted after adjusting for baseline characteristics, thus validating the role of guidelines adherence in reducing mortality. Similarly, previous studies showed that inadequate initial antibiotic therapy leads to excessive mortality.5 This is mainly caused by emerging resistances as a result of inadequate and unnecessary use of antibiotics, and is exacerbated by the lack of development of new antibiotic substances.28

Limitations

We believe that a randomised controlled trial would not be a feasible design, since following guidelines is crucial for the outcome and it would be unethical to restrain adequate therapy from control group patients. In concordance to the conclusions of Sintchenko et al,29 a preinterventional and postinterventional design is the best available for this research question. On the other hand, this design is limited in inferring on causalities, as other factors might have influenced the results. In this context, we aimed to carefully assess for potential confounders but we could only control for a defined set of observed variables. For this study, external validity is restricted mainly to the postoperative setting. Also, our recommendations are locally adapted and results of this single centre study in a tertiary university hospital might not be transferable to other settings with its specific decision-making structure. Another potential limitation is related to the unblinded nature of this study. To control at least partially for this effect, a team consisting of a senior consultant and two assistants evaluated the data for adherence. To further control for bias, an expert team randomly scrutinised the data for the results of the study team that was blinded, and had the final decision. Although there is some evidence in the literature that a cut-off for adherence of 70% might be a first pragmatic barrier in change management17 18 associated with improved patient outcome,16 30 in the long run adherence rates of >95% should be targeted to optimise quality of treatment.

Conclusion

To the best of our knowledge, this is the first study to show improved long-term guideline adherence following the implementation of a CDSS. Adherence rate at 5 years remained above a level of 70%. Consequently, we are able to show that CDSS seems to have a lasting benefit, although physicians were only trained in the beginning of the study. Improvement in guideline adherence was paralleled by reductions in individual antibiotic therapy. Furthermore, overall ICU mortality was associated with guideline adherence further supporting the role of CDSS in improving the quality of care.

Success of this implementation is probably due to the multidisciplinary front-line team that developed the CDSS, which included all disciplines involved in the decision-making process in the ICU. One key advantage of this programme is that, with help of a special function, it can easily be customised to meet the needs of any intensive care facility and that it is available to any ward with internet access. As one of the key components to improving prescription patterns, CDSS may help to develop sufficient stewardship programmes in the future.