Clinical Impact of an Electronic Dashboard and Alert System for Sedation Minimization and Ventilator Liberation: A Before-After Study.

Sedation minimization and ventilator liberation protocols improve outcomes but are challenging to implement. We sought to demonstrate proof-of-concept and impact of an electronic application promoting sedation minimization and ventilator liberation.
DESIGN: Multi-ICU proof-of-concept study and a single ICU before-after study.
SETTING: University hospital ICUs. PATIENTS: Adult patients receiving mechanical ventilation.
INTERVENTIONS: An automated application consisting of 1) a web-based dashboard with real-time data on spontaneous breathing trial readiness, sedation depth, sedative infusions, and nudges to wean sedation and ventilatory support and 2) text-message alerts once patients met criteria for a spontaneous breathing trial and spontaneous awakening trial. Pre-intervention, sedation minimization, and ventilator liberation were reviewed daily during a multidisciplinary huddle. Post-intervention, the dashboard was used during the multidisciplinary huddle, throughout the day by respiratory therapists, and text alerts were sent to bedside providers.
MEASUREMENTS AND MAIN RESULTS: We enrolled 115 subjects in the proof-of-concept study. Spontaneous breathing trial alerts were accurate (98.3%), usually sent while patients were receiving mandatory ventilation (88.5%), and 61.9% of patients received concurrent spontaneous awakening trial alerts. We enrolled 457 subjects in the before-after study, 221 pre-intervention and 236 post-intervention. After implementation, patients were 28% more likely to be extubated (hazard ratio, 1.28; 95% CI, 1.01-1.63; p = 0.042) and 31% more likely to be discharged from the ICU (hazard ratio, 1.31; 95% CI, 1.03-1.67; p = 0.027) at any time point. After implementation, the median duration of mechanical ventilation was 2.20 days (95% CI, 0.09-4.31 d; p = 0.042) shorter and the median ICU length of stay was 2.65 days (95% CI, 0.13-5.16 d; p = 0.040) shorter, compared with the expected durations without the application.
CONCLUSIONS: Implementation of an electronic dashboard and alert system promoting sedation minimization and ventilator liberation was associated with reductions in the duration of mechanical ventilation and ICU length of stay.

Acute respiratory failure requiring mechanical ventilation accounts for a substantial portion of ICU admissions, and 12% of hospital costs, totaling nearly 27 billion dollars annually (1, 2). Daily spontaneous awakening trials (SATs) and spontaneous breathing trials (SBTs), as well as sedation minimization protocols, reduce the duration of mechanical ventilation, tracheostomy rates, ICU and hospital length of stay (LOS), and mortality (3–7). Consequently, sedation minimization and ventilator liberation protocols are key components of the Society of Critical Care Medicine’s Assess, Prevent, and Manage Pain, Both SAT and SBT, Choice of analgesia and sedation, Delirium: Assess, Prevent, and Manage, Early mobility and Exercise, and Family engagement and empowerment (ABCDEF) bundle and ICU Liberation initiatives (8), and the American Thoracic Society/American College of Chest Physicians Clinical Practice Guidelines (9, 10).

Despite their evidence base and inclusion in professional society guidelines, implementation is inconsistent, with some studies revealing only 40–44% compliance performing daily SATs (11, 12). Audits at our institution similarly revealed that SBT assessments were inconsistent across ICUs, knowledge of our institution’s SBT eligibility criteria was incomplete, extubation delays were frequently related to over-sedation despite sedation minimization strategies, and that manual review of these processes limited real-time feedback.

Because automated applications have been successful at detecting opportunities for improving care in sepsis, acute respiratory distress syndrome (ARDS) and acute kidney injury (13–16), we hypothesized that an automated application could improve adherence to sedation minimization and ventilator liberation protocols. We designed a novel software platform, the Awakening and Breathing Coordination (ABC) Application, which continuously screens patients for opportunities to minimize sedation and speed ventilator liberation, and is coupled with a dashboard and text-message alerts to inform bedside providers when an opportunity is identified.

In this study, we report the results of a proof-of-concept study and before-after implementation study assessing the impact of the ABC application.

MATERIALS AND METHODS

The ABC Application

The ABC application is a software platform that continuously screens charted data in the electronic health record (EHR), including nursing flowsheets, respiratory flowsheets, and medication administration records, and runs the data through locally developed algorithms to determine if patients meet criteria to undergo sedation minimization and ventilator liberation (Fig. S1 and Table S1, Supplemental Digital Content 1, http://links.lww.com/CCX/A107). The results are communicated through a web-based dashboard and text-message alerts to bedside providers (Fig. ). The web-based dashboard displays real-time SBT eligibility (based on our institution’s ventilator liberation protocol [Table S1, Supplemental Digital Content 1, http://links.lww.com/CCX/A107]), sedation depth (Richmond Agitation-Sedation Scale [RASS]) (17), and sedative infusions for each patient. For patients not meeting SBT criteria, the dashboard displays the reason an SBT is not recommended and displays nudges to wean oxygen, positive end-expiratory pressure, and vasopressors as appropriate. The dashboard also displays a nudge to wean sedation in patients with a RASS less than or equal to –1 who are receiving sedative infusions (Fig. 1). In addition to the dashboard, text-message alerts are sent to bedside providers when patients newly meet SBT and SAT eligibility (Fig. 1).

Awakening and Breathing Coordination (ABC) application dashboard and text alerts. A, Representative screenshot of the electronic dashboard that provides real-time data on each patient’s spontaneous breathing trial (SBT)-readiness, depth of sedation as defined by the Richmond Agitation-Sedation Scale (RASS), and the patient’s current continuous analgesic and sedative infusion doses. For patients that are not SBT eligible, the dashboard provides the eligibility criteria that are not met as well as nudges to wean oxygen, positive end-expiratory pressure (PEEP), and vasoactive medications according to the prespecified algorithms. The dashboard also provides a nudge to wean sedation in patients with a RASS less than –1 who are receiving continuous analgesic or sedative infusions. B, Examples of the real-time HIPAA-compliant SBT and spontaneous awakening trial text-message alerts that are sent to bedside providers (respiratory therapist and nurse, respectively) as soon as a patient newly meets SBT eligibility, such as the patient who is listed as SBT ready and highlighted green in A. CAM = Confusion Assessment Method, MAP = mean arterial pressure, MICU = medical ICU, MV = minute ventilation, PS = pressure support, TV = tidal volume, UTA = unable to assess.

Proof-of-Concept Study

We performed a proof-of-concept study in the ICUs of three university-affiliated hospitals, enrolling all mechanically ventilated patients who triggered an ABC application text-message alert from August 20, 2016, to September 10, 2016 (Table S2, Supplemental Digital Content 1, http://links.lww.com/CCX/A107). Alerts were deemed accurate if all the prespecified criteria were documented in the EHR at the time of the alert. SBT alerts were deemed valuable if they fired while the patient was receiving mandatory ventilation (i.e., Assist-Control). We evaluated perceived barriers to extubation to ensure the alerts were not solely identifying patients who remained ventilated for valid clinical reasons. SAT alert value was measured by how frequently it fired in the SBT alerted population, reflecting an opportunity for earlier sedative weaning, and by the sedation depth at the time of the SAT alert, as more deeply sedated SBT-eligible patients may fail an SBT due to over-sedation and may receive greater benefit from sedation interruption.

Before-After Study

We performed a before-after study of 457 mechanically ventilated patients admitted to the medical ICU at the Hospital of the University of Pennsylvania. We excluded patients with preexisting tracheostomies and overflow patients from nonstudy ICUs (Fig. S2, Supplemental Digital Content 1, http://links.lww.com/CCX/A107). This study was exempt on the basis of quality improvement by the University of Pennsylvania Institutional Review Board.

Pre-implementation (May 1, 2016, to September 25, 2016) the ABC application ran silently, and sedation minimization and ventilator liberation were performed according to existing protocols. Nurses titrated sedatives to the goal RASS defined by the medical team and respiratory therapists screened patients for SBT eligibility daily. A multidisciplinary huddle occurred each morning during which respiratory therapists reviewed each patient’s SBT eligibility and SBT results, and nurses reviewed each patient’s sedative infusions. The medical team made decisions on sedation minimization and ventilator liberation, which were then implemented independent of the team’s in-depth rounds.

Post-implementation (September 26, 2016, to February 1, 2017) the ABC application promoted compliance with existing protocols and provided continuous screening for sedation and ventilator weaning opportunities. The ABC dashboard was reviewed each morning at the multidisciplinary huddle and by respiratory therapists throughout the day. Respiratory therapists and nurses received the SBT and SAT text-message alerts and were instructed to act on the alert if clinically relevant or discuss it with the medical team. If a patient did not formally meet the application’s SBT criteria but was deemed SBT-ready based on clinical judgment, providers were instructed to use standard protocols.

In our primary analyses, we used Cox proportional hazards regression to test for differences in the primary outcomes of time to extubation, time to ICU discharge, and time to hospital discharge. We adjusted for illness severity (Acute Physiology and Chronic Health Evaluation [APACHE] IV score) (18) and receipt of vasopressors. Patients were censored at the time of death or transfer to a nonstudy ICU in all analyses and censored at the time of tracheostomy in the time to extubation analysis. In complementary analyses, we used interrupted time-series analysis (ITSA) with data aggregated by week to test for a difference in each outcome when comparing the postintervention period with the counterfactual scenario (19, 20). We used a level and slope change model, hypothesizing that implementation of the application would cause an immediate change in each outcome and change the slope of each outcome over time as implementation improved. For the ITSA, we excluded patients transferred to a nonstudy ICU while still receiving mechanical ventilation and excluded outliers that appeared to bias the pre-intervention trend away from the null. We performed sensitivity analyses without removing outliers and separately assuming the preintervention trend did not persist in the postintervention period.

Secondarily, we compared 48-hour reintubation rates to ensure the ABC application did not lead to unsafe extubations and compared the time to extubation among patients who never formally met SBT criteria to ensure the ABC application did not delay extubation based on clinical judgment alone. We explored the mechanisms through which the ABC application might have had an impact by comparing the time from initiation of invasive ventilation to meeting SBT criteria, time to cessation of continuous sedative infusions, and time from SBT alert to extubation.

Analyses were performed in Stata Version 15.1 (StataCorp, College Station, TX), and a two-sided p value of less than 0.05 was considered statistically significant. Detailed methods are provided in the supplementary online content (Supplemental Digital Content 1, http://links.lww.com/CCX/A107).

RESULTS

We evaluated 115 SBT alerts and 70 SAT alerts. All alerts fired accurately when evaluated against the EHR; only two SBT alerts (1.7%) were inaccurate and resulted from erroneous documentation of ongoing mechanical ventilation after extubation. Overall, the SBT alerts appeared valuable; 100 fired (88.5%) when the patient was still receiving mandatory ventilation and 13 fired when the patient was weaned but not yet extubated. Most patients, 60.2%, had no perceived barriers to extubation, suggesting the alert was not solely identifying patients who remained intubated for clinical reasons. Of the 45 patients with a barrier to extubation, 21 (46.7%) were due to over-sedation. The SAT alerts also appeared valuable; 70 of 113 SBT eligible patients (61.9%) received a concurrent SAT alert (indicating ongoing sedative infusion and RASS < 0), demonstrating an opportunity for earlier sedation minimization. Furthermore, 71.4% of SAT alerts were in moderately sedated patients (RASS ≤ –2), with 24.3% in deeply sedated patients (RASS ≤ –4).

Patient Characteristics in the Before-After Study

We enrolled 221 patients in the preintervention period and 236 in the postintervention period (Fig. S2, Supplemental Digital Content 1, http://links.lww.com/CCX/A107); clinical characteristics are shown in Table 1. Baseline severity of illness (APACHE IV) increased throughout the study (Fig. S3, Supplemental Digital Content 1, http://links.lww.com/CCX/A107) and was significantly higher post-intervention (93 [interquartile range (IQR), 74–116] vs 83 [IQR, 63.5–105]; p < 0.001). The proportions of patients who received an SBT alert were similar in both periods and averaged 1.2 alerts/day in the postintervention period.

TABLE 1.

Clinical Characteristics of the Study Population Categorized by Pre-Intervention and Post-Intervention (n = 457)

Duration of Mechanical Ventilation

Extubation rates were similar in the two groups, 61.9% post-intervention and 57.5% pre-intervention (p = 0.82). However, the time to extubation was shorter after the ABC application was implemented. As shown in Figure , patients were 28% more likely to be extubated at any time point after implementation, when adjusted for illness severity and receipt of vasopressors (hazard ratio [HR], 1.28; 95% CI, 1.01–1.63; p = 0.042). To ensure the reduced duration of mechanical ventilation was not due to a preexisting temporal trend, we performed an ITSA. We excluded five patients with prolonged duration of mechanical ventilation during the preintervention period (four > 50 d, one > 30 d) to prevent these outliers from increasing the preintervention duration of mechanical ventilation in a biased fashion. As shown in Figure , the median duration of mechanical ventilation rose slightly during the preintervention period, followed by an immediate step-down and slight flattening of the trend post-intervention. Midway through the postintervention period (week 30), the median duration of mechanical ventilation was 2.20 days (95% CI, 0.09–4.31 d) shorter compared with the expected duration, ranging from 1.9 days shorter immediately after implementation to 2.5 days shorter near the end of the study (Table S3, Supplemental Digital Content 1, http://links.lww.com/CCX/A107). Results were similar in a sensitivity analysis including outliers (Fig. S4 and Table S3, Supplemental Digital Content 1, http://links.lww.com/CCX/A107) and remained significant (1.39 d [95% CI, 0.04–2.74 d] at week 30) when assuming the upward trend in the preintervention period did not persist post-intervention (Fig. S5 and Table S3, Supplemental Digital Content 1, http://links.lww.com/CCX/A107). We assessed for potential bias from variations in the time to death and time to tracheostomy pre- and post-intervention and found no significant difference in time to death (HR, 0.90; 95% CI, 0.64–1.27; p = 0.55) or time to tracheostomy (HR, 0.95; 95% CI, 0.43–2.07; p = 0.90).

Duration of mechanical ventilation in the before-after study. A, Cumulative incidence function describing the proportion of patients successfully extubated in the postintervention (blue line) and preintervention (red line) groups, adjusted for illness severity and vasopressor-dependent shock. Time to extubation was significantly shorter in the postintervention group, adjusted hazard ratio (HR) 1.28 (95% CI, 1.01–1.63; p = 0.043). B, Interrupted time series analysis using a level and slope change model with data aggregated on a weekly basis demonstrating the median duration of mechanical ventilation in the preintervention period (white background) and postintervention period (gray background). The solid line represents the median duration of mechanical ventilation in the preintervention and postintervention groups from the regression model, and the dotted line represents the counterfactual scenario, which is the expected trend in the absence of the intervention given the preexisting trend. The model shows a significant reduction in the median duration of mechanical ventilation in the postintervention group. For example, midway through the postintervention period (week 30), the median duration of mechanical ventilation was 2.20 d shorter (95% CI, 0.09–4.31, p = 0.042) compared with the counterfactual scenario.

ICU LOS

ICU mortality was similar in the two groups, 31.8% post-intervention and 35.3% pre-intervention (p = 0.70). However, ICU LOS was significantly shorter in the postintervention period. As shown in Figure , patients were 31% more likely to be discharged from the ICU alive at any time point after implementation, when adjusted for illness severity and receipt of vasopressors (HR, 1.31; 95% CI, 1.03–1.67; p = 0.027). We found similar results using an ITSA. To ensure outliers did not increase the ICU LOS in the preintervention period in a biased fashion, we excluded two patients in the preintervention period with very long ICU stays (> 115 d and > 70 d). As shown in Figure , the ICU LOS trended slightly upward throughout the preintervention period, followed by an immediate decrease and slight downtrend post-intervention. Midway through the postintervention period (week 30) the median ICU LOS was 2.65 days (95% CI, 0.13–5.16 d) shorter compared with the expected duration, ranging from 2.1 days shorter immediately after implementation to 3.2 days shorter near the end of the study (Table S4, Supplemental Digital Content 1, http://links.lww.com/CCX/A107). Results were similar in a sensitivity analysis including the outliers (Fig. S6 and Table S4, Supplemental Digital Content 1, http://links.lww.com/CCX/A107) and remained significant (1.85 d shorter [95% CI, 0.25–3.46 d] at week 30) when assuming the preexisting upward trend did not persist in the postintervention period (Fig. S7 and Table S4, Supplemental Digital Content 1, http://links.lww.com/CCX/A107). We assessed for potential bias from differences in the time to death, and time to ICU discharge among patients discharged to a long-term ventilator hospital in the preintervention and postintervention periods. There was no difference in time to death between the two groups (HR, 0.89; 95% CI, 0.65–1.23; p = 0.48). Seven patients (three pre-intervention, four post-intervention) underwent a tracheostomy and were discharged to a long-term ventilator hospital. These patients appeared to have a shorter ICU LOS in the postintervention period (median 22.3 vs 60.5 d); however, our results were similar when we excluded these patients in a sensitivity analysis (Fig. S8 and Table S4, Supplemental Digital Content 1, http://links.lww.com/CCX/A107).

ICU length of stay in the before-after study. A, Cumulative incidence function describing the proportion of patients discharged from the ICU alive in the postintervention (blue line) and preintervention (red line) groups, adjusted for severity of illness and vasopressor-dependent shock. Time to ICU discharge was significantly shorter in the postintervention group, adjusted hazard ratio (HR) 1.31 (95% CI, 1.03–1.67; p = 0.027). B, Interrupted time series analysis using a level and slope change model with data aggregated on a weekly basis demonstrating the median ICU length of stay in the preintervention period (white background) and postintervention period (gray background). The solid line represents the median ICU length of stay in the preintervention and postintervention groups from the regression model, and the dotted line represents the counterfactual scenario, which is the expected trend in the absence of the intervention given the preexisting trend. The model shows a significant reduction in the median ICU length of stay in the postintervention group. For example, midway through the postintervention period (week 30), the median ICU length of stay was 2.65 d shorter (95% CI, 0.13–5.16; p = 0.040).

Hospital LOS and Secondary Outcomes

Hospital mortality was similar in the postintervention (39.8%) and preintervention (42.1%) periods (p = 0.48), and hospital LOS was not significantly different post-intervention (Fig. S9, Supplemental Digital Content 1, http://links.lww.com/CCX/A107) (HR, 1.15; 95% CI, 0.90–1.47; p = 0.26). We found no difference in 48-hour reintubation rates (10.6% and 9.5%; p = 0.70). There was also no difference in the rate of successful extubation (35.0% compared with 29.3%; p = 0.79) or time to extubation (HR, 0.90; 95% CI, 0.51–1.61; p = 0.73) among patients who never formally met the ABC application’s SBT criteria.

In terms of the mechanisms through which the ABC application might improve outcomes, the time from meeting SBT criteria to extubation (HR, 1.27; 95% CI, 0.99–1.65; p = 0.065) and the time to cessation of sedative infusions (HR, 1.29; 95% CI, 0.98–1.70; p = 0.070) both appeared shorter after implementation, although neither was statistically significant (Fig.).

Time to cessation of continuous sedatives and time from spontaneous breathing trial (SBT) ready to extubation in the before-after study. A, Cumulative incidence function describing the proportion of patients off continuous sedative infusions in the postintervention (blue line) and preintervention (red line) groups, adjusted for severity of illness and vasopressor-dependent shock. The time to cessation of continuous sedative infusions was shorter in the postintervention period, but the difference was not statistically significant, adjusted hazard ratio (HR) 1.27 (95% CI, 0.99–1.65; p = 0.065). B, Cumulative incidence function describing the proportion of patients who were extubated after being identified as SBT ready by the Awakening and Breathing Coordination application. The time from meeting SBT criteria to extubation was shorter in the postintervention period, but the difference was not statistically significant, adjusted HR 1.29 (95% CI, 0.98–1.70; p = 0.070).

DISCUSSION

In this study, we demonstrate proof-of-concept that an EHR-based dashboard and text-alert system can identify opportunities to improve sedation minimization and ventilator liberation. Further, we have shown that implementation of such a system was associated with reductions in the duration of mechanical ventilation and ICU LOS in an ICU with existing sedation minimization and ventilator liberation protocols.

To date, numerous studies have reported improved outcomes with sedation minimization and ventilator liberation protocols, often in the context of clinical trials (3–6). However, outside of clinical trials, implementation of ABCDEF bundle components is often challenging and incomplete (11, 12). Our results add to a growing number of studies that have demonstrated improved patient outcomes by focusing on improving implementation of the ABCDEF bundle through quality improvement initiatives (21–24).

Our data suggest the ABC application improved outcomes through reductions in the time from being SBT ready to extubation and time to cessation of continuous sedative infusions. However, these analyses did not meet statistical significance and the mechanisms by which the ABC application improved outcomes are not entirely clear. A recent review codified the many barriers to ABCDEF bundle implementation (25), including several barriers we identified at our institution. This includes unclear protocol criteria, ineffective interprofessional care coordination, communication barriers, and strained workload and time. It may be that the ABC application helps providers overcome many of these barriers. The automated screening may decrease the workload for bedside providers. The dashboard displaying real-time sedation depth in relation to ventilator status might facilitate clinical decision-making, promote staff accountability, and improve knowledge of protocols. Finally, the text-message alerts may reduce communication barriers and promote interprofessional team coordination. Additional studies are needed to better understand the potential mechanisms through which the ABC application improved outcomes and assess any barriers to the ABC application’s implementation.

Based on its success in our single ICU before-after study, the ABC application has been employed across our health system and validation of our findings in a larger study is underway. Future studies will also seek to expand the application’s technology to improve other aspects of ABCDEF bundle implementation, such as delirium management and early mobilization, and improve implementation of other evidence-based practices, such as lung-protective ventilation in ARDS.

Our study has several strengths. First, our institution had preexisting sedation minimization and ventilator liberation protocols in place prior to implementation of the ABC application. Second, no other quality improvement projects related to sedation minimization or ventilator liberation were undertaken during this study. Last, the ABC application is portable to other EHRs, straightforward and requires minimal training for bedside users, suggesting it could easily be expanded and implemented at other sites.

Our study has several limitations that are worth noting. The before-after study design could be prone to bias, specifically temporal trends. However, we performed interrupted time-series analyses to evaluate temporal trends and found a slight uptrend in the duration of mechanical ventilation and ICU LOS before our intervention. The reasons for this increase over time are not clear but may be related to the increasing severity of illness throughout the study period, and our findings overall suggest the improvements in the postintervention period were not due to preexisting downward trends. As mentioned, we found that the baseline severity of illness increased throughout the study, and although we adjusted for severity of illness, other unmeasured confounders could have influenced our results given our nonrandomized study design. Our study was performed in a single quaternary-care ICU, and therefore, our findings require validation in a larger study. In addition, our study was designed to assess the short-term impact of the intervention and future studies are needed to evaluate whether the intervention’s impact is sustained. In the proof-of-concept study, we focused on ensuring the alerts were accurate, and we did not formally assess the sensitivity of the alert; therefore, it is possible that additional opportunities for earlier sedation weaning and ventilator liberation were missed. Although we demonstrated reductions in the duration of mechanical ventilation and ICU LOS, our study was not powered to detect differences in mortality or rates of tracheostomy as shown in prior studies (5, 7). In addition, our study was not designed to measure other ventilator-associated events such as delirium or ventilator-associated pneumonia.

CONCLUSIONS

In conclusion, we describe an electronic dashboard and text-alert system designed to promote sedation minimization and ventilator liberation and demonstrate its clinical impact through reductions in the duration of mechanical ventilation and ICU LOS. Future studies should seek to leverage similar technology to improve implementation of other evidence-based practices.

ACKNOWLEDGMENTS

We would like to acknowledge contributions from Penn Medicine’s Department of Data Science, Center for Healthcare Innovation, Information Services, and PENN Elert in the development of the Awakening and Breathing Coordination (ABC) application. We also thank the nurses, pharmacists, respiratory therapists, and physicians in the Medical ICU of the Hospital of the University of Pennsylvania for their invaluable support with implementing the ABC application.