Early intervention of patients at risk for acute respiratory failure and prolonged mechanical ventilation with a checklist aimed at the prevention of organ failure: protocol for a pragmatic stepped-wedged cluster trial of PROOFCheck.

INTRODUCTION: Acute respiratory failure (ARF) often presents and progresses outside of the intensive care unit. However, recognition and treatment of acute critical illness is often delayed with inconsistent adherence to evidence-based care known to decrease the duration of mechanical ventilation (MV) and complications of critical illness. The goal of this trial is to determine whether the implementation of an electronic medical record-based early alert for progressive respiratory failure coupled with a checklist to promote early compliance to best practice in respiratory failure can improve the outcomes of patients at risk for prolonged respiratory failure and death. METHODS AND ANALYSIS: A pragmatic stepped-wedged cluster clinical trial involving 6 hospitals is planned. The study will include adult hospitalised patients identified as high risk for MV >48 hours or death because they were mechanically ventilated outside of the operating room or they were identified as high risk for ARF on the Accurate Prediction of PROlonged VEntilation (APPROVE) score. Patients with advanced directives limiting intubation will be excluded. The intervention will consist of (1) automated identification and notification of clinician of high-risk patients by APPROVE or by invasive MV and (2) checklist of evidence-based practices in ARF (Prevention of Organ Failure Checklist-PROOFCheck). APPROVE and PROOFCheck will be developed in the pretrial period. Primary outcome is hospital mortality. Secondary outcomes include length of stay, ventilator and organ failure-free days and 6-month and 12-month mortality. Predefined subgroup analysis of patients with limitation of aggressive care after study entry is planned. Generalised estimating equations will be used to compare patients in the intervention phase with the control phase, adjusting for clustering within hospitals and time. ETHICS AND DISSEMINATION: The study was approved by the institutional review boards. Results will be published in peer-reviewed journals and presented at international meetings. TRIAL REGISTRATION NUMBER: NCT02488174. Published by the BMJ Publishing Group Limited. For permission to use (where not already granted under a licence) please go to http://www.bmj.com/company/products-services/rights-and-licensing/

The Prevention of Organ Failure Checklist (PROOFCheck) was designed to be a hospital-based pragmatic trial with a broad, heterogeneous patient population and interventions set in the real-world/usual care setting of an acute hospital using the hospital electronic medical records to deliver the intervention and to collect the data.

The stepped-wedge cluster trial is a robust design that is well suited for studying the implementation of care delivery intervention and quality measures similar to PROOFCheck. Outcomes were chosen to be clinically relevant and simple as they are already collected as part of usual clinical care.

With electronic notifications, there is always a concern for alarm fatigue. To limit the number of alerts, alerts to the physician will be sent only once per day and will stop after the patient is no longer considered at high risk by Accurate Prediction of PROlonged VEntilation (APPROVE). As PROOFCheck will be triggered whenever a patient is intubated, and nearly all intubated patients are cared for in intensive care units, it is unlikely that many patients with prolonged mechanical ventilation (MV) will be missed by the study. Nevertheless, we will monitor for all MV patients that might have been excluded from the study.

The effectiveness of any multifaceted intervention such as PROOFCheck will depend on compliance to the recommendations which will be delivered in the usual care setting without strict enforcement of compliance to the protocol. Therefore, clinicians retain the ability to ignore the alert that their patient may be critically ill and may deviate from checklist recommendations. To measure this, we will collect data on compliance to PROOFCheck to determine which item may require re-education or re-enforcement and which elements may best correlate with improved outcomes.

Introduction

Acute respiratory failure (ARF) is the most common acute organ dysfunction in US hospitals, with incidence of 430 episodes/100 000 population with most (70%) requiring mechanical ventilation (MV).1 ARF with MV is associated with very high mortality at 28% and healthcare costs at US$27 billion, representing 12% of all hospital costs.2 Multiple organ failure (MOF) is common among patients with ARF, with 21% and 18% of patients developing renal and cardiovascular dysfunction, respectively. Most patients who die of ARF do not die of refractory hypoxaemia but of MOF. Even patients who survive ARF with MV suffer from significantly higher long-term mortality and cognitive and functional impairments. Among Medicare beneficiaries, 6-month mortality was three times higher among patients with ARF requiring MV than other intensive care unit (ICU) patients without MV (30% vs 9.6%; adjusted hazard ratio (AHR) 2.26% (95% CI 1.90% to 2.69%)), and among the survivors, mechanically ventilated patients were more likely to be discharged to skilled nursing facilities: 62% of MV patients went to skilled nursing facilities versus 26.4% of non-ventilated hospital controls.2 This is not surprising given the mounting data on the high prevalence of ICU-acquired weakness, found in 25–58% of patients,3 4 and cognitive impairment, found in 76% of survivors with acute respiratory distress syndrome (ARDS).5

Most research on ARF has focused on patients in the ICU. In reality, ARF and other organ failures usually develop and progress well before ICU admission, in the emergency department (ED) or general hospital floor. Between 2001 and 2009, the total annual hours of critical care delivered in the ED increased to 217%.6 In a recent study of non-ICU patients, 14% of hospitalised non-ICU patients screened positive for severe sepsis, and 30% of them had respiratory organ dysfunction while still on the general medical floor.7

Identification of hospitalised patients at risk for ARF

While critical illness and acute organ failure frequently develop outside of the ICU, the acuity and severity of the patient's condition is often not recognised. In one study, 62% of general medical patients transferred to the ICU had serious abnormalities 8–48 hours before ICU admission, which were either not recognised or not acted on.8 Delays in calls to critical care rapid response teams (RRTs) have been found to be independently associated with a 47% higher mortality rate than when acute deterioration is recognised and responded to in a timely fashion.9 However, many RRT systems in US hospitals are triggered by clinician concern or by one or more extreme physiological derangements, often with poor sensitivity and specificity and low rates of recognition of clinical deterioration.10 11 But, automated early warning scores and triggers based on aggregate data in hospital electronic medical records (EMRs) show much better sensitivity and specificity.11–15 For example, the electronic Cardiac Arrest Risk Triage score, using data abstracted from the hospital EMR, showed an area under the curve of 0.77% (95% CI 0.76% to 0.77%) for identifying patients who develop the outcome of cardiac arrest, ICU transfer or death a median of 37 hours before the event.16

Early interventions with best-care practices in ARF

Identification of patients at risk for ARF wherever they may be in the hospital is especially important as early interventions, before end organ damage is established, have been shown to improve survival and mitigate the severity of organ failure. Early recognition and initiation of treatment in septic shock is a fundamental tenant in emergency medicine and critical care medicine.17 Early recognition and resuscitation decreased ARF requiring MV in the 7–24 hours after presentation to the ED.18 Similarly, low tidal volume ventilation in patients with ARDS resulted in less MOF within 72 hours of ARDS, lower overall mortality and more ventilator-free days compared to conventional, larger tidal volume ventilation.19 20 For every 1 cc/kg per predicted body weight increase in tidal volume, there was a 23% increase in mortality in ARDS indicating the importance of early implementation of low tidal volume ventilation in ARDS.21 In addition, a number of hospital-acquired injuries such as aspiration and medical and surgical adverse events (of which nearly 70% are preventable) are associated with increased odds of subsequent development of ARDS.22

Clinically accepted best practices already exist to prevent the hospital-acquired injuries that could increase the risk of developing ARF requiring MV or worsen mortality, but compliance is variable. In a study of one tertiary academic ICU, median rate of compliance with best practices for eligible patients was 56.5%, with the sickest patients being least likely to be prescribed best practices.23 For example, transfusion-related lung injury and other complications of transfusion can be reduced by avoiding unnecessary blood transfusion. The TRICC trial established restrictive transfusion strategy at haemoglobin <7 g/dL as the standard of care in critically ill patients,24 but over 20% of patients who were transfused had pretransfusion haemoglobin of 7–10 g/dL.25 Likewise, while tidal volumes have fallen, 7.5% of patients with ARDS are still ventilated at >10 cc/kg ideal body weight (IBW).26 The Lung Injury Prevention Study found that 62% and 17% of non-ARDS patients are initially ventilated at ≥8 and ≥10 cc/kg IBW, respectively.27 Even greater variability in compliance to these practices exists in the non-ICU areas of the hospital like the ED, where such guidelines have not been adopted consistently. In one study in the ED, low tidal volume ventilation, considered the standard of care in ARDS, was used in only 18% of the patients with ARDS in the ED.28 At many institutions, there are already existing efforts to improve compliance with these best practices, but the efforts are often not tied together and are often not implemented in a timely fashion as patients are either not recognised as critically ill or the clinicians working with these patients are not aware of the existing best practices.

Recent studies have suggested that better compliance with these best-care practices and avoidance of these hospital-acquired injuries may decrease the development of ARDS and improve outcomes. In one study, implementation of practices aimed at reducing lung injury and complications of critical illness (eg, lower tidal volumes for patients with ARDS, restrictive transfusion strategies, recognition and resuscitation in septic shock, and so on) coincided with a >50% decrease in the rates of hospital-acquired ARDS and a significant reduction in ICU and hospital mortality and length of stay.29 In another study of weaning in brain-injured patients, even a minor improvement in compliance with their bundle protocol of protective ventilation, standardisation of antibiotics for pneumonia and protocolised weaning and extubation resulted in significant increased likelihood for extubation, 2-day reduction in the duration of MV and a 7-day increase in ICU-free days.30

Objectives

The Prevention of Organ Failure Checklist (PROOFCheck) was designed to determine whether the implementation of an EMR-based early alert for progressive respiratory failure coupled with a checklist to promote increased compliance to best practice in respiratory failure can improve the mortality and morbidity in hospitalised patients at risk for prolonged respiratory failure and death.

Methods and Materials

Study population

There will be six participating hospitals from three institutions including the Montefiore Medical Center in Bronx, New York (Weiler Hospital, Moses Division and Wakefield Hospital), Mayo Clinic Rochester in Rochester, Minnesota (Saint Mary's Campus and Methodist Campus) and Mayo Clinic Florida in Jacksonville, Florida. Adult patients aged ≥18 in the hospital who are intubated and placed on MV or who are identified as being at high risk for developing severe ARF requiring prolonged MV as indicated by The Accurate Prediction of PROlonged VEntilation (APPROVE), will be eligible for the trial. Patients who are chronically ventilated or who have do not intubate (DNI) orders on hospital admission will be excluded.

On the basis of the data from 2013, areas of the hospitals that have the most patients with acute deterioration or ARF were identified. These areas served as the target areas for the trials. From these high-frequency areas, hospital units were included into the trial based on availability of the technology and clinical staff to respond to the alerts and PROOFCheck. Local champions from these areas of the hospital were recruited to help with education and implementation.

Study design

The study design is a stepped-wedge clustered randomised controlled trial (RCT), as summarised in table 1.

Table 1

Stepped-wedged cluster clinical trial design where all participating hospital units in an institution are randomised as a cluster to a time in which the intervention will start so that by the end of the study, all sites will have implemented the intervention

Stepped-wedge designs are increasingly being used in a healthcare context to evaluate quality improvement interventions when it is either not possible or inappropriate to use individual randomisation.31 Under this design in this study, the unit of randomisation is the hospital and the cluster of participating patient care areas in that hospital. These hospitals are randomly allocated a time when they are given the intervention. The intervention is then implemented sequentially in these hospitals in their selected turn, so that by the end of the study all hospitals have received the intervention.32 This design is useful for evaluating service delivery interventions, such as PROOFCheck. The stepped-wedge design lessens ethical concerns because every hospital eventually receives the intervention, and the intervention is not removed once it has been implemented.33 This design is preferable for evaluating the population-level impact of an intervention when there is no equipoise and provides a robust way of evaluating intervention effectiveness during routine implementation.

Derivation and validation of APPROVE

To better identify clinically deteriorating patients for this trial, the first year of the project in the pretrial period will be spent in the derivation and validation of a clinical score to identify high-risk patients for the trial. The APPROVE score aimed to use clinically available data abstracted from hospital EMRs to identify patients at increased risk for death or ARF at 48 hours prior to the event. These variables include patient characteristics (such as age, gender, weight and race), vital signs, laboratory values and therapeutic interventions like oxygen therapy, type of oxygen device and use of vasopressors. ARF will be defined as the initiation of MV with a duration of >48 hours. Intubations that lead to MV for <48 hours will not be considered to be an event. To achieve this, we intend to abstract and validate data from adult patients admitted to the Montefiore Medical Center and the Mayo Clinic in 2013. The cohort will be randomly split (50-50) into a derivation and validation cohort. Random forest models examining random time points in the cohort will be used to derive the score. The performance of the score will be further tested in the validation cohort using random time points and as an observational cohort looking for when a patient reaches a particular cut-off in the APPROVE score and whether they develop an event within the next 48 hours after that trigger.

Development of PROOFCheck

To address this gap in best practices, we will compile a checklist of best practices known to be beneficial in patients with ARF and critical illness that can be implemented early to decrease these hospital-acquired injuries that can lead to worsening of respiratory failure and prolonged MV and death in the hospital. The PROOFCheck will be compiled by a multidisciplinary team of clinical experts from critical care, pulmonary, surgery, quality improvement, anaesthesiology and emergency medicine in a modified Delphi process to reach consensus of >70% on items to include in the checklist.

Intervention

Preintervention control phase

The preintervention control condition will be usual care: that is, clinicians' practice as usual without clinician notification of risk and without prompts on care practices as recommended by PROOFCheck. Newly intubated patients and patients at high risk for severe respiratory failure requiring prolonged MV will still be identified by APPROVE, and PROOFCheck will still be generated for data collection and outcome measurements. However, clinicians will be blinded to APPROVE and PROOFCheck. Contextual questions to determine relevant elements in PROOFCheck that may not be reliably retrievable from EMR will be answered by the research nurse in order to preserve the blinding of clinicians during the preintervention phase.

Intervention phase

The intervention for this study will consist of three parts: (1) education of clinicians in ED, general medicine and surgery floors, respiratory therapy and ICU on prevention of severe ARF and MOF in and out of the ICU, and best practice with regard to patients with severe ARF. This is targeted in the 2–4 months prior to the intervention period; (2) clinicians will be notified that a patient in their care has been identified as being at high risk for developing severe ARF requiring prolong MV and (3) notified clinicians will be directed to their EMR to access the customised PROOFCheck that was generated for that patient. As all patients who require MV are at high risk for prolonged MV, PROOFCheck will also be triggered after intubation. To minimise alert fatigue, PROOFCheck will be generated no more than once a day for 2 days when the patient is still considered at high risk for severe ARF and prolonged MV.

The educational plan and its implementation will be decided at each site as it must be tailored to the usual care at that institution and must fit into the workflow of that area of the hospital. The plan will be developed in partnership with the local champions identified at each hospital who will help lead the implementation. However, regardless of the site, the educational plan should include these core principles:

The time course for clinical deterioration prior to ARF and how it is often missed clinically.

The rationale behind APPROVE and the performance of APPROVE.

What it may mean clinically for the patient if they are alerted by APPROVE.

The evidence and rationale behind the recommendations in PROOFCheck.

The variability in current compliance with these best-care measures.

The availability of RRT or consults for assistance, if needed.

The target audience for the education will be the clinicians taking care of patients who may be included in the trial. This will be determined by each site as the clinicians taking care of such a patient who may be best able to respond may differ from hospital to hospital and from unit to unit. As such, each site should target the population to educate after consultation with the local champions, but may include physicians, house staff, physician assistants, nurse practitioners, nurses, respiratory therapists and RRTs.

Data collection

To facilitate the large scale data collection of clinical data for this trial, trial outcomes and patients-level data will be designed to be pragmatic and easily accessible from the EMR or hospital administrative database. To enable data collection for the trial from the hospital EMR and administrative databases, a master dictionary of all data elements for the calculation of APPROVE and the trial will be created. At each participating institution, the variables in the data dictionary will be validated against the institutional EMR to ensure that the variables are correctly mapped to the EMR field that corresponds to the clinical intent of the variable after accounting for documentation practices and workflow at each site. In addition, the outcome variables of MV and death will be validated against the EMR and medical records to ensure accuracy. The mapped variables will be validated against medical records to ensure that the data are clinically relevant to the goals of the project and correctly represents the clinical data that clinicians use to make decisions. However, data related to compliance to the checklist that could not be reliably abstracted from the EMR will be manually collected in a random subset of patients.

Baseline data for the study participants will include demographic data such as age, gender, race, height, weight, desire for resuscitation status, Charlson comorbidity score and residence prior to hospital admission. To calculate APPROVE, physiological data (such as heart rate, blood pressure, respiratory rate and oxygen saturation), laboratory data (complete blood count, electrolytes, renal and liver function tests and blood gases) and supportive care (such as vasopressor use, oxygen requirement, MV start and end date and non-invasive ventilation) will be abstracted from the EMR.

Additionally, data to determine compliance to PROOFCheck will be abstracted from the hospital EMR and will include antibiotic start time, transfusions of blood products, urinary and central line use, tidal volume on MV, oral care, use of intravenous continuous sedative (midazolam, fentanyl, propofol, dexametomidine and lorazepam), deep venous and gastric ulcer prophylaxis. Data that cannot be reliably abstracted from the hospital EMR (such as head of bed elevation, discussion of goals of care and determination of palliative care needs) will be collected manually by research coordinators on a random subset (2%) of patients due to limited resources for a research coordinator for the number of patients expected to be enrolled into the trial. These data will be used to help refine and focus the training and education and re-training during the intervention.

Outcomes

All outcome data will be abstracted from hospital electronic medical and administrative records in the preintervention and postintervention phases for all hospitals. The primary hypothesis is to determine whether PROOFCheck can improve hospital mortality in patients on MV or at high risk for progressing to prolonged MV. Secondary outcomes will include 7-day organ failure as indicated by the Sequential Organ Failure Assessment score, hospital length of stay, ICU length of stay, 28-day ventilator-free days, 6-month and 12-month mortality, and ability to be discharged to home among patients admitted from home. For ascertainment of 6-month and 12-month mortality, a state registry will be used and the analysis will be restricted to patients who reside within the state.

We anticipate that early notification to clinicians that their patients may be critically ill and PROOFCheck prompts for early communication of treatment preferences and goals of care before emergent intubation may reduce unwanted aggressive care in patients with terminal conditions. Thus, a priori, we plan to analyse rates of new orders for limitation of resuscitation or MV, palliative care consults and hospice care before and after intervention. There is also a planned subgroup analysis of patients with and without orders limiting resuscitation or MV. Additionally, we will plan subgroup analysis of patients who entered the trial on the bases of new intubation versus patients who triggered APPROVE to determine whether there may be a difference in effect.

Analytical approach

All analysis will be conducted as intention to treat. The unit of analysis is the patient, where patients receiving PROOFCheck in the intervention phase according to the randomised stepped-wedge schedule will be compared to those in the control phase, adjusting for clustering within hospitals and stepped-wedge design. We will use generalised estimating equations (GEE) with the logit link function to estimate the intervention effect, accounting for repeated measures across time, facility random effects, and patient characteristics and potential predictors of respiratory failure in the analysis. This will also allow us to determine whether there is a secular trend in outcomes during the course of the trial. Generalised linear mixed model (GLMM) or GEE have different advantages and disadvantages when applied to a stepped-wedge design.33 Thus, we will also test the use of GLMM in a sensitivity analysis.

Sample size calculation

The sample size must be inflated by estimating the design effect (DE) to account for the intraclass correlation experienced by the clustering of patients at each site and the stepped-wedge design as in the equation below:34where n is the number of clusters, ρ is the intracluster correlation, b is the number of baseline time periods, t is the number of time periods at each step and n is the cluster size. The ρ is assumed to be 0.05 to maximise the sample size inflation.33

On the basis of preliminary data to indicate a hospital mortality of 14.5% for high-risk patients and 60% for patients who develop the event, a planned APPROVE score that results in two of three high-risk patients developing the event, assumption of an α of 0.05, 90% power, a sample size of 7778 patients will allow for the trial to detect a 113.3% relative change in hospital mortality.

Ethics and human subject protection

The study has been approved by the institutional review board of the Albert Einstein College of Medicine and Mayo Clinic which oversees all human subject research in the various hospitals of Montefiore Medical Center and Mayo Clinic. A waiver of informed consent will be obtained for data collection. A waiver of informed consent can be obtained under the common rule (45 code of federal regulations (CFR) 46.116 and 45 CFR 46.408(a)) if: (1) the research presents no more than minimal risk or harm to patients and involves no procedures for which informed consent is normally required outside of the research context; (2) the waiver of consent or alteration will not adversely affect the rights and welfare of the patient, the only record linking the patient and the research would be the consent document and the principal risk of harm would be the potential harm resulting from a breach of confidentiality; (3) the research cannot be practically carried out without the waiver and (4) the patient will be provided with additional pertinent information, if appropriate, after participation. All recommendations in PROOFCheck are minimal risk as they are considered best practices that are supposed to be used for critically ill patients. All elements in PROOFCheck will be evaluated by an expert multidisciplinary panel to have more benefit than harm when applied earlier in patients at risk for prolonged MV. As randomisation is implemented at the hospital level, obtaining informed consent from all potential individuals in the hospital is not possible. Such a strategy is often used in pragmatic studies examining interventions to promote recommended clinical practices.35 36 Indeed, in clustered RCTs, McRae et al37 describe a framework by which informed consent can be waived in clustered RCTs that include: (1) a clustered RCT where the intervention is at the level of the hospital and informed consent of large numbers of individual patients in the hospital is not possible; (2) the study meets criteria for waiver of informed consent (as above) and (3) participants are approached after cluster randomisation with information on the intervention.

A Data Safety Monitoring Board was established to monitor the data privacy and progress of the study.