Hospital-wide ELectronic medical record evaluated computerised decision support system to improve outcomes of Patients with staphylococcal bloodstream infection (HELP): study protocol for a multicentre stepped-wedge cluster randomised trial.

INTRODUCTION: Staphylococci are the most commonly identified pathogens in bloodstream infections. Identification of Staphylococcus aureus in blood culture (SAB) requires a prompt and adequate clinical management. The detection of coagulase-negative staphylococci (CoNS), however, corresponds to contamination in about 75% of the cases. Nevertheless, antibiotic therapy is often initiated, which contributes to the risk of drug-related side effects. We developed a computerised clinical decision support system (HELP-CDSS) that assists physicians with an appropriate management of patients with Staphylococcus bacteraemia. The CDSS is evaluated using data of the Data Integration Cent ers (DIC) established at each clinic. DICs transform heterogeneous primary clinical data into an interoperable format, and the HELP-CDSS displays information according to current best evidence in bacteraemia treatment. The overall aim of the HELP-CDSS is a safe but more efficient allocation of infectious diseases specialists and an improved adherence to established guidelines in the treatment of SAB. METHODS AND ANALYSIS: The study is conducted at five German university hospitals and is designed as a stepped-wedge cluster randomised trial. Over the duration of 18 months, 135 wards will change from a control period to the intervention period in a randomised stepwise sequence. The coprimary outcomes are hospital mortality for all patients to establish safety, the 90-day disease reoccurrence-free survival for patients with SAB and the cumulative vancomycin use for patients with CoNS bacteraemia. We will use a closed, hierarchical testing procedure and generalised linear mixed modelling to test for non-inferiority of the CDSS regarding hospital mortality and 90-day disease reoccurrence-free survival and for superiority of the HELP-CDSS regarding cumulative vancomycin use. ETHICS AND DISSEMINATION: The study is approved by the ethics committee of Jena University Hospital and will start at each centre after local approval. Results will be published in a peer-reviewed journal and presented at scientific conferences. TRIAL REGISTRATION NUMBER: DRKS00014320. © Author(s) (or their employer(s)) 2020. Re-use permitted under CC BY-NC. No commercial re-use. See rights and permissions. Published by BMJ.

RCT Entities:   Population Interventions Outcomes

First randomised study to evaluate the application of a computerised clinical decision support system (CDSS) for the treatment of Staphylococcus blood stream infections.

Illustration of the usefulness of interoperability standards for otherwise heterogeneous medical routine data, enabling its use to improve patient care.

More efficient allocation of infectious disease consultations, as there is a shortage infectious disease specialists in German hospitals.

Participating wards already have a high standard for infectious diseases treatment and may not be representative of average German clinics.

CDSS is restricted to Staphylococcus blood stream infections.

Introduction

Staphylococci are the most commonly identified pathogens in both hospital-acquired and community-onset bloodstream infections.1 Coagulase-negative staphylococci (CoNS), followed by Staphylococcus aureus, are most commonly detected. S. aureus bloodstream (SAB) infection is a serious medical condition. If treated insufficiently, mortality rates can reach up to 40% and recurrences are frequent. Outcomes of SAB can be improved by strict adherence to treatment guidelines, including choice of appropriate antibiotic agent.2–6 In CoNS bloodstream infection, the clinical significance (except for S. lugdunensis, as this pathogen is associated with the same pathogenicity as S. aureus), however, is less clear. CoNS are part of the normal skin flora and identification in blood cultures corresponds to contamination in up to 80% of the cases.7 Nevertheless, in clinical practice, antibiotic therapy is often initiated, which in turn fosters the development of antibiotic resistance, increases the risk of drug-related side effects and the cost of therapy. Souvenir et al,8 for example, reported the use of antibiotics to treat 41% of patients with false-positive blood cultures, with vancomycin used for 83% of the treated pseudobacteraemic patients.

The use of algorithm-based therapy can improve the treatment of patients with staphylococcal bacteraemia as recently shown by Holland et al.9 Here, we examine the effects of a computerised clinical decision support system (CDSS), abbreviated HELP-CDSS, that supports attending physicians of patients with staphylococcal bacteraemia with regard to the implementation of best practice recommendations. This includes recommendations regarding follow-up blood cultures (FUBCs), early source control, early intravenous cloxacillin or cefazolin for methicillin-susceptible isolates and appropriate duration of therapy in patients with S. aureus or S. lugdunensis bloodstream infection. In patients with CoNS bloodstream infection, recommendations will particularly encourage adherence to FUBCs to differentiate whether the CoNS bacteraemia reflects true infection or contamination only.

The HELP-CDSS is evaluated using data that are part of clinical routine documentation. It aims at a more efficient allocation of infectious diseases specialists (IDS) without worsening patient-related outcomes such as mortality and relapse rates and also aims at improving adherence to best practice recommendations in treatment of blood stream infections. The HELP-CDSS operates on the structure of so-called ‘Data Integration Centers’ (DIC), which will be established at each study site by means of the ‘Smart Medical Information Technology for Healthcare’ (SMITH) consortium,10 which is one of four consortia funded by the ‘Medical Informatics Initiative’ of the German Federal Ministry of Education and Research. Two of the main goals of this initiative are to establish interoperability standards for medical data and to enable the integration of heterogeneous clinical routine data to make it available for clinical and scientific use in order to improve patient care.

Methods and analysis

HELP-computerised clinical decision support system

A simplified schematic representation of the HELP-CDSS is presented in figure 1. The HELP-CDSS assigns cases to one of two arms: one for CoNS bloodstream infection and one for S. aureus and S. lugdunensis bloodstream infection. In the CoNS arm, it is first checked whether there are two independent blood cultures available (drawn at different time points or different loci). If there is only one blood culture, the CDSS advises to take a FUBC before starting the antibiotic therapy to rule out a contamination. If there are two positive blood cultures, and they are positive for the same CoNS (ie, species and antimicrobial susceptibility testing), the CDSS recommends checking for clinical significance of the result, including identification of a possible source of infection. It also recommends starting an antibiotic therapy if there are signs for a CoNS infection. If the FUBC is negative (ie, no detection of CoNS or a different CoNS species), the CDSS informs the physician about the high probability that the first blood culture, which is positive for CoNs, represents a contamination and recommends reconsidering the necessity of an antibiotic treatment (if already initiated) or discourages starting a therapy, respectively.

Figure 1

Schematic representation of the HELP-CDSS. CDSS, computerised clinical decision support system.

In the case of S. aureus or S. lugdunensis bloodstream infection, the CDSS gives the following recommendations:

FUBCs should be drawn at intervals of 2–4 days until negative conversion.

Early source identification and source control.

Early use of beta-lactams for methicillin-susceptible infections (within 24 hours of culture).

Treatment duration of at least 14 days for uncomplicated and at least 28 days for complicated SAB.

Perform transoesophageal echocardiogram in complicated SAB (intravenous drug abuse, persistent bacteraemia, embolisation, unknown source of infection, community-acquired SAB, presence of prosthetic valves or cardiovascular implantable devices).

In case of indications for a complicated course (eg, endocarditis, positive FUBCs or prolonged fever), a request for an infectious diseases consultation should be initiated.

Necessary patient-related information on which the CDSS is evaluated will be extracted from different primary data sources of the clinical routine (patient data management system, laboratory information system and so on).11 These data will be represented in an interoperable syntactic format (HL7 FHIR)12 and semantically annotated with standard terminologies (LOINC, SNOMED CT and so on).13 Annotated HL7 FHIR resources will be stored in a ‘Health Data Storage’ of the respective DIC, enabling analytical methods to query all data based on internationally consented code systems or value sets.

Results of the modelling of relevant information, semantic annotation and the application of HL7 FHIR have been made publicly available through tools such as ART-DECOR14 or Simplifier.net, corresponding to processes carried out by standards developing organisations to develop interoperability standards. They can be accessed at https://art-decor.org/art-decor/decor-datasets--help-.

Study design

The study is designed as a stepped-wedge cluster randomised trial (SW-CRT)15 with a preceding pilot phase and will be conducted at five German university hospitals, namely the hospitals of Jena (JUH), Leipzig, Aachen, Halle and Essen with a total of 135 participating wards. At the beginning of the study, which is scheduled for September 2019, the piloting phase starts. In this phase, the data extraction and technical implementation will be tested. When all technical aspects are functioning, the actual SW-CRT phase begins. According to the SW-CRT design, all wards start in the control phase (standard of care, SOC) and enter the treatment phase (application of the HELP-CDSS) in a stepwise fashion. The time of cross-over from control to treatment phase is assigned to the wards by stratified randomisation. A schematic overview of the stepped-wedge design is provided in figure 2. There will be nine randomisation steps with 2 months in between each step, and 15 clusters will enter the treatment phase per step. Including a 2-month control phase when no ward receives the algorithm, this results in a study duration of 20 months plus about 3 months of follow-up. In the participating wards, all adult patients with a blood culture positive for S. aureus/S. lugdunensis or CoNS are included in the study. If the ward is in the control phase when the patient is included, they get the SOC treatment. All participating clinics employ IDS, which can be asked for a consultation by the treating physician no matter which phase they are in. Thus, the SOC differs from the treatment phase only in the fact that the infectious diseases council is refined by the HELP-CDSS. It is important to note this level of standard care is not representative for most German hospitals given that only few hospitals employ IDS. If the CDSS gives IDS more time, this time could in future be used for telephone consultation for hospitals without IDS. This approach is currently being tested in the SUPPORT study,16 and results will be published in the near future.

Figure 2

Stepped-wedge design of the HELP trial. CDSS, computerised clinical decision support system.

If the ward is in the treatment phase, the HELP-CDSS starts working as described above. Neither the data collected for the CDSS nor for the study outcomes go beyond clinical routine data. Only for patients with SAB, a 90-day follow-up by phone is scheduled, which will be announced by letter 15–20 days before the call. The 90 days will be counted from time of inclusion in the study, that is, the first blood culture positive for SAB. Patient recruiting in each clinic will stop 2 months after the HELP-CDSS has been rolled out to all wards.

Study population

Included patients comprise all adults with a blood culture positive for Staphylococcus on the participating wards. Maternity wards, psychiatry and paediatric wards are excluded from the study. While no CoNS/SAB patients are excluded a priori, the treating physician is free to adhere to or disregard any recommendation proposed by the CDSS.

Study objectives and outcomes

On one hand, this study aims at a better allocation of infectious diseases counsels without worsening patient-related outcomes such as hospital mortality. This is an important objective, as there is a shortage of IDS in German hospitals. Thus, a more efficient allocation of their time can, for example, allow more telephonic consultation services for clinics without IDS. Thus, we should demonstrate that the CDSS will do no harm as recently underlined by Wiens et al.17 Consequently, we will first test a non-inferiority hypothesis regarding mortality and relapse rates. On the other hand, the HELP-CDSS should improve adherence to already established recommendations for identification and treatment of blood stream infections. This involves, for example, an escalation to IDS services for SAB patients or a reduction of unnecessary administration of antibiotics when a contamination of the blood culture is likely (CoNS patients).

The coprimary outcomes are the hospital mortality rate (for all patients) as well as the combined 90-day mortality/relapse rate for SAB patients and the cumulative vancomycin use for CoNS patients. A relapse is defined as S. aureus isolated from a sterile site after more than 7 days of apparent clinical improvement. Evaluation of clinical improvement will be made by detailed clinical assessment performed by at least two IDS physicians not involved into the trial. Assessment will include review of medical records (inflammatory parameters, physical examination, performance of adequate focus control and microbiology test results (eg, positive blood cultures after previous negative follow-up cultures)). Due to heterogeneity of the disease and clinical presentation, no specific set of criteria are feasible to define ‘clinical improvement’. The secondary outcomes include acute renal dysfunction, defined as 1.5–2 fold increase in creatinine where we will also look at the time interval between drug usage and the development of renal dysfunction. Further secondary outcomes are the cumulative use of seven additional antibiotics, the number of blood cultures per clinic and year, the costs due to IDS consultations and several technical assessments of the CDSS. A list of the endpoints and documented indicators can be found in table 1.

Table 1

Coprimary/secondary endpoints and key performance indicators of HELP

Coprimary endpoint	Secondary endpoints	Health economic, technical and process key performance indicators
Hospital mortality. Relapse/mortality within 90 days. Cumulative vancomycin use.	Acute renal dysfunction (measured by creatinine) Cumulative use of: linezolid, daptomycin, teicoplanin, fosfomycin, rifampicin, flucloxacillin and cefazolin. Number of blood cultures. Number of administered transoesophageal echocardiographies.	Costs due to infectious diseases specialist consultation. Number of ID consultations per ward (total and per patient day). Adherence to HELP-CDSS recommendations. Number of HELP-CDSS queries. Satisfaction with HELP-CDSS (by survey and interviews after the study).

CDSS, computerised clinical decision support system.

Proposed sample size

Apart from the expected effect and its variance, the sample size of a SW-CRT study depends mainly on the number of clusters (wards) included at each randomisation step and only secondary on the number of patients per cluster. With an intended study duration of 20 months and nine randomisation steps, we calculated the required number of wards using the procedure proposed by Hussey and Hughes18 and tested it with simulation studies. For the baseline mortality of SAB patients, we used the numbers reported by Mejer et al 19 for an orientation, assuming a 90-day mortality of 0.3 in the control group with an expected reduction to 0.15 in the intervention group; for non-inferiority testing, we also checked smaller margins. For the expected number of patients per ward, we examined prevalence rates of Staphylococcus infections of the participating clinics that ranged between 2.0% and 3.7% in the year 2017 and estimated an average of 20 patients per ward for the entire study period. Under these assumptions, we calculated that 15 wards per randomisation step are sufficient to attain a power of 80%. Multiplied by nine randomisation steps and 20 patients per ward, that results in a sample size of approximately 2700 patients. Even for a lower baseline mortality of 20% and a reduction to 15%, 15 wards/randomisation step with 20 patients each are unambiguously sufficient to attain a power over 80%. Even under these margins, the power falls below 80% only when wards/step go below 7 and patients/ward go below 10.

Randomisation and follow-up

Participating wards will be randomised to the treatment condition in a stepwise fashion. The randomisation will be stratified by clinic and by normal versus intensive care units (including intermediate care units). The randomisation list will be generated by the JUH prior to the trial initiation and will be locked there. Every patient will be treated according to the status of the ward at the moment of their inclusion in the study. Patients who are transferred to other wards or hospitals after their inclusion will be analysed according to the intention-to-treat principle. Patients with SAB will be followed up 90 days after their inclusion in the study. The follow-up will be performed by telephone by a trained study nurse of the respective clinic and will be preceded by an announcement letter 15–20 days before the call.

Data analysis

We will use generalised linear mixed modelling to analyse the three coprimary endpoints. Treatment (HELPS-CDSS vs SOC) and ‘time since study-begin’ will be included as fixed effects; the ward (cluster) will be included as a random effect. We will test three hierarchical hypotheses,20 where denotes the respective endpoint under the HELP-CDSS and denotes the endpoint under SOC:

(A) The HELP-CDSS is non-inferior to the SOC regarding the hospital mortality rate (with a non-inferiority margin Δ of 5%):

(B) For patients with SAB, the HELP-CDSS is non-inferior to the SOC regarding the 90-day relapse and mortality rate (with a non-inferiority margin Δ of 5%):

(C) For patients with a CoNS infection, the HELP-CDSS (H) is superior to the standard of care (S) regarding the cumulative use of vancomycin:

The non-inferiority hypotheses will be tested both in the intention-to-treat (ITT) and the per-protocol population; the superiority hypothesis will be tested in the ITT population. The hypotheses will be tested in a confirmatory fashion meaning that they will be processed in the order specified here and that each hypothesis will only be tested if the null hypothesis of the preceding test was rejected at the respective significance level. We will conduct one-sided (A and B) and two-sided (C) significance tests with a significance level of 5%. The treatment effect will in addition be judged using the point estimate of the effect and its 95% CI. Furthermore, we will conduct exploratory sensitivity analyses controlling for additional correlation within clinics and interaction effects of clinic and treatment as well as controlling for temporal trends by including time since study begin in the analysis model. Analyses of the secondary endpoints will also be exploratory and will follow the modelling approach of the primary outcomes. The health economic, technical and process key performance indicators will all be reported descriptively.

Clinical data collection and data protection

Apart from the follow-up for SAB patients, which will be conducted by a trained in-house study nurse, no data beyond clinical routine will be assessed. Most of the data will be collected automatically via electronic health records and other electronic documentation systems. In cases where data are not electronically available, trained study nurses will collect the data via case report forms. The primary data sources store the information using identifying patient data (IDAT) and source-specific patient identifiers (PIDs). These identifiers are used in the Data Integration Engine of the DIC, which transforms and annotates the heterogeneous medical data (see section ‘HELP-computerised clinical decision support system’) and stores it in the ‘Health Data Storage’, where it can be requested for analysis. The steps are performed in the secure internal network of each hospital. The data will leave the clinic only in aggregated or completely deidentified form. For aggregation, every site will get an analysis script to run on their data that produces an aggregated output that will be collected at JUH for the final analysis. Neither the aggregated output nor the deidentified data contain any IDAT or PID.

The study will be monitored by a data safety monitoring board consisting of three independent experts: Professor Dr Uwe Groß (Institute for Medical Microbiology, University Hospital Göttingen), Professor Dr Matthias Schmid (Institute for Medical Biometry, Informatics and Epidemiology, University Hospital Bonn) and Professor Dr Bernd Salzberger (Institute for Hospital Hygiene and Infectiology, University Hospital Regensburg).

Patient and public involvement

As HELP will take place in routine treatment of patients and does not propose new forms of treatment but better adherence to already established guidelines, patients were not included in the planning of this study. However, physicians using the HELP-CDSS contribute to its development, and we perform surveys of their satisfaction with the HELP-CDSS as part of the evaluation. Finally, patient and public involvement is given by the SMITH Congress (www.digital-health-2019.de) and within the activities of the overarching initiative (eg, workshop in German, ‘Gesundheitsdaten für die medizinische Forschung: Wie können Patienten partizipieren?’ https://www.medizininformatik-initiative.de/de/mii-workshop-gesundheitsdaten-fuer-die-medizinische-forschung-wie-koennen-patienten-partizipieren).

Ethics and dissemination

Informed consent

No informed consent is required to collect the routine documentation on which the CDSS operates. The same is true for the follow-up of SAB patients as it will be implemented as a quality assurance measure in each clinic. However, patients will be informed by letter about the telephone call beforehand and will be informed about their right to refuse answering.

Study registration and ethics review

The study will be conducted according to the current version of the Declaration of Helsinki and has been approved by the ethics committee of the JUH as well as the respective data protection commissioner and will start at each centre after local approval. Any amendments to the protocol will immediately be communicated to each centre and their respective ethics committees.

Access to data and dissemination

The results of the HELP study will be presented at scientific and medical conferences and will be published in peer-reviewed journals. The reports will follow the 2018 extension of the Consolidated Standards of Reporting Trials 2010 statement19 regarding SW-CRTs, and we will also consider the suggestions of Hemming et al.15 The publication of the study results will not depend on the nature of the results. The study relies on the infrastructure of the DICs and serves as a use case for their functionality. One of the overarching goals of DICs is to foster data sharing within medicine while meeting the data protection and security laws and requirements.