Risk adjusted 30-day mortality and serious adverse event rates among a large, multi-center cohort of emergency department patients with acute heart failure.

Background: Admission rates for emergency department (ED) patients with acute heart failure (AHF) remain elevated. Use of a risk stratification tool could improve disposition decision making by identifying low-risk patients who may be safe for outpatient management.
Methods: We performed a secondary analysis of a retrospective, multi-center cohort of 26,189 ED patients treated for AHF from January 1, 2017 to December 31, 2018. We applied a 30-day risk model we previously developed and grouped patients into 4 categories (low, low/moderate, moderate, and high) of predicted 30-day risk of a serious adverse event (SAE). SAE consisted of death or cardiopulmonary resuscitation (CPR), intra-aorta balloon pump, endotracheal intubation, renal failure requiring dialysis, or acute coronary syndrome. We measured the 30-day mortality and composite SAE rates among patients by risk category according to ED disposition: direct discharge, discharge after observation, and hospital admission.
Results: The observed 30-day mortality and total SAE rates were less than 1% and 2%, respectively, among 25% of patients in the low and low/moderate risk groups. These rates did not vary significantly by ED disposition. An additional 23% of patients were moderate risk and experienced an approximate 2% 30-day mortality rate.
Conclusion: Use of a risk stratification tool could help identify lower risk AHF patients who may be appropriate for ED discharge. These findings will help inform prospective testing to determine how this risk tool can augment ED decision making.

INTRODUCTION

Background

Patients presenting to the emergency department (ED) with acute heart failure (AHF) represent a complex and heterogenous population. Patients with stable, chronic heart failure have significant comorbidities contributing to their baseline risk of adverse events, and this risk is compounded during an AHF presentation. Use of ED risk stratification tools, shared decision‐making, and greater evolution of non‐hospital venues of care are important strategies to safely lower hospital admission rates. All of these approaches contribute to the overarching goal of safe ED discharge. However, understanding the incremental value of a risk prediction instrument to aid ED decision‐making and identifying a cohort of lower risk patients is a crucial first step to provide patient and provider reassurance regarding safe ED discharge.

To improve the alignment of patient risk with intensity of care, our study team recently developed and published a novel risk stratification tool to predict 30‐day severe adverse event (SAE) using a large, retrospective cohort of over 26,000 ED patients treated for AHF across 21 medical centers in Kaiser Permanente Northern California (KPNC) from 2017 to 2018. The SAE included death or cardiopulmonary resuscitation (CPR), intra‐aorta balloon pump, endotracheal intubation, renal failure requiring dialysis, or acute coronary syndrome. Our KPNC AHF risk tool had high predictive accuracy with an area under the curve of 0.85.

Importance

This prior study highlighted an opportunity to improve outcomes and alignment of decision making with inpatient resources. In our study cohort, 24.5% of patients were directly discharged from the ED, 17.5% were discharged after a brief observation period, and 57.6% were admitted to the hospital. The 30‐day SAE rate among discharged, observed, and admitted patients was 5.7%, 11.0%, and 26.2%, while 30‐day mortality rates were 3.9%, 5.9%, and 13.9%, respectively, which are similar to those reported in other cohorts from around the world. , We found a large proportion of low‐risk patients (<3% 30‐day SAE risk) were admitted to the hospital, whereas many high‐risk patients (>10% 30‐day SAE risk) were discharged.

Objective

We had 3 goals with this article that expand on our initial work of risk model derivation and internal validation to help prepare for implementation of risk‐based care pathways for ED patients with AHF in our health system. First, we sought to choose an optimal risk threshold to define “lower risk” to identify a subset of patients that could be expected to improve outcomes among discharged patients without inadvertently leading to increases in hospitalization. Second, we wanted to examine whether hospital admission was associated with 30‐day mortality and 30‐day SAE rates among lower risk patients to better understand whether hospitalization is an effect modifier in this cohort. Last, we wanted to separate out 30‐day mortality (in our prior manuscript, we only assessed composite SAE rates) because of its importance to providers, and in quality reporting and clinical trials. We see this as hypothesis‐generating work that will be used to guide implementation of clinical pathways that incorporate individual patient risk to guide ED disposition decisions.

METHODS

Study design and setting

This is a secondary analysis of a multi‐center, retrospective cohort of KPNC members who had an ED visit for AHF. KPNC is a large integrated health care delivery system managing over 1 million ED visits annually. All emergency, inpatient, and outpatient care (primary care and specialty) are documented in an integrated electronic health record (EHR), including comprehensive capture of laboratory, pharmacy, and imaging data. The study protocol was approved by the KPNC institutional review board. We adhered to the Strengthening of Reporting of Observational Studies in Epidemiology (STROBE) criteria; a completed checklist can be found in the Supplemental Appendix.

This retrospective analysis of over 26,000 emergency department (ED) patients with acute heart failure suggests the use of a risk stratification tool can help identify lower risk patients who may be safely discharged. Among the 25% of the population identified as lower risk, the 30‐day mortality rates were <1% and did not vary by ED disposition.

Study participants and outcomes

The cohort assembly and inclusion and exclusion criteria are described in our original publication. We performed the current analysis on the same study population as our original publication. Briefly, we identified all KPNC adult health plan members who had an ED visit for AHF to 1 of the 21 KPNC EDs between January 1, 2017 and December 31, 2018. The ED annual volumes range from 28,000 to 128,000 with variable specialty services and patient acuity. About half of facilities have observation areas in the ED, all are non‐rural hospitals, and while none are academic centers, most are affiliated with residency training programs. Please see the Supplemental Appendix for a flow diagram of the cohort assembly. We included only the first eligible encounter during the study period for each patient.

We used the same composite outcome measure, 30‐day SAE, as the STRATIFY risk tool, which was selected a priori by study investigators for that study. SAE were identified from inpatient or ED encounters within 30 days of the index ED visit using procedure codes and ICD‐10 codes. Death was identified from a composite death database compromised of health‐system, California state, Social Security Administration, and National Death Index data.

Analysis

In our initial publication, we presented findings from the derivation and validation of our novel risk stratification tool using KPNC's comprehensive EHR and machine learning models. We used the variables and outcome measure described in the STRATIFY tool as a basis for development and testing of novel models to accurately predict 30‐day SAE. We added additional variables to the base STRATIFY model and trialed various, machine learning methods. We included the following patient‐level variables as predictors for model derivation and validation : patient comorbid illnesses and medication use; documentation of reduced versus preserved ejection fraction (preserved ejection fraction was defined as left ventricular ejection fraction greater than 49%); residential neighborhood socioeconomic status (2010 US Census data, census block group level); health care use in outpatient and inpatient settings during the past year; ED arrival mode; differences between both baseline and ED weight and blood pressure; ED laboratory and electrocardiogram values; ED chest radiography findings; and use of ventilatory support in the ED.

For the original model derivation and validation, we collected patient demographic information including age, sex, race, and ethnicity from EHR databases. We ascertained information on coexisting illnesses according to diagnoses or procedures, using ICD‐10 and Current Procedural Terminology codes from the electronic health record problem list from prior inpatient or outpatient visits. Relevant laboratory results and ED‐level characteristics were ascertained from EHR databases. When multiple laboratory tests or vital signs were present for an ED visit, we used the first documented results. Unavailable values from continuous covariates were replaced with the median from recorded measurements, and all variables (both continuous and categoric) were defined with an additional level (missingness indicator variable) to encode unavailable values.

In our prior manuscript, we compared the predictive accuracy of various models to identify risk of a 30‐day SAE. We randomly divided the sample into training (80%) and testing (20%) data. For advanced machine learning models, we used 10‐fold cross validation in the training data for hyperparameter tuning. We reported model performance measures (area under the receiver operator characteristic [AUC] and precision recall curves, sensitivity, specificity, likelihood ratios, negative and positive predictive values, and net reclassification) resulting from applying the prediction model from training data to test data. The machine learning model with the highest predictive accuracy used XGBoost, with an AUC of 0.85.

In this article, we utilize this previously developed model to predict each patient's risk of a 30‐day SAE and then divide the study cohort into 4 risk categories: (1) low (≤3% predicted 30‐day SAE risk), (2) low/moderate (3.1% to ≤5% predicted 30‐day SAE risk), (3) moderate (5.1%–10% predicted 30‐day SAE risk), and (4) high (>10% predicted 30‐day SAE risk). These risk thresholds are the same as those used in the original STRATIFY publication and in our recent KPNC risk model publication, although we collapse the number of risk categories from 6 to 4 in this article. ,

We then examine the association of ED disposition (direct discharge, discharge after brief observation, and hospital admission) and the observed outcomes of 30‐day SAE and 30‐day mortality within each cohort of risk. We used logistic regression models with random effect at the hospital level to account for within‐hospital correlations of outcome with ED disposition as the main variable of interest. We ran separate models for the 2 outcomes of interest, 30‐day SAE and 30‐day mortality, in each risk group. We report 30‐day SAE and 30‐day mortality rates with 95% confidence intervals resulting from the model among discharged, observed, and admitted patients for each risk group.

RESULTS

The mean age in the 26,189 patients was 74 years, 51.7% were women and 60.7% were white. Figure 1 displays the 30‐day SAE and mortality rates among discharged, observed, and admitted patients irrespective of risk group. Figure 2 displays the observed 30‐day mortality in each of the 4 risk groups among discharged, observed, and admitted patients. We found 25% of our study population was in the low‐ or low/moderate‐risk groups. The observed 30‐day mortality rate among these low‐ and low/moderate‐risk patients was less than 1% and did not vary significantly between discharged, observed, and admitted patients. An additional 23% of the study population was in the moderate‐risk group, where the 30‐day mortality was approximately 2%. This mortality rate was comparable among discharged and admitted patients and lower among observed patients. The remaining 51% of our population were in the high‐risk group, and the 30‐day mortality rate of these patients ranged from 13% to 20%, with the highest mortality rate experienced by admitted patients.

FIGURE 1

The 30‐day mortality and total severe adverse events (SAE) by emergency department (ED) disposition: direct ED discharge, discharge after a brief observation period, and hospital admission. Notes: SAE include death or cardiopulmonary resuscitation (CPR), intra‐aorta balloon pump, endotracheal intubation, renal failure requiring dialysis, or acute coronary syndrome. In our study cohort, 24.5% of patients were directly discharged from the ED, 17.5% were discharged after a brief observation period, and 57.6% were admitted to the hospital

FIGURE 2

The 30‐day mortality rates with 95% confidence intervals by risk strata comparing discharged, observed, and admitted patients among 26,182 emergency department (ED) patients with acute heart failure (AHF) in Kaiser Permanente Northern California (KPNC). Notes: data from 2017 to 2018 KPNC cohort of 26,189 ED patients with AHF. Number of patients in each risk strata listed above columns (percent of patients in each risk strata noted in parentheses). There were 2926 (11.2%) low‐risk, 3809 (14.5%) low/moderate‐risk, 6069 (23.2%) moderate‐risk, and 13,385 (51.2%) high‐risk patients

Figure 3 displays the total SAE rates (mortality plus non‐fatal SAE) in each of the 4 risk groups among discharged, observed, and admitted patients. Among the 25% of patients in the low‐ or low/moderate‐ risk groups, the observed 30‐day SAE rate was less than 2% and did not vary by ED disposition. Among the 23% in the moderate‐risk group, the 30‐day SAE rate was 3%–5%, with significantly higher rates among admitted patients. Patients in the high‐risk group experienced total 30‐day SAE rates of 19%–38%, again with the highest rates among admitted patients.

FIGURE 3

The 30‐day total serious adverse event (SAE) rates with 95% confidence intervals by risk strata comparing discharged, observed, and admitted patients among 26,182 emergency department (ED) patients with acute heart failure (AHF) in Kaiser Permanente Northern California (KPNC). Notes: data from 2017 to 2018 KPNC cohort of 26,189 ED patients with AHF. Number of patients in each risk strata listed below columns (percent of patients in each risk strata noted in parentheses). There were 2926 (11.2%) low‐risk, 3809 (14.5%) low/moderate‐risk, 6069 (23.2%) moderate‐risk, and 13,385 (51.2%) high‐risk patients

LIMITATIONS

Although our results suggest use of our risk tool could identify patients safe for ED discharge, several limitations should be considered. KPNC is a unique healthcare system and its features, such as reliable outpatient follow‐up and comprehensive EHR, may not generalize to other practice settings. Further, although objective ED risk stratification is a key component to safe discharge, other factors should be considered when determining need for admission, including response to initial therapies, other co‐morbidities, additional ED diagnoses, patient ability for self‐care, and social support.

The reasons for the observed differences in 30‐day mortality and SAE rates by ED disposition among the higher‐risk patients are likely multi‐factorial and not fully unmeasurable. For example, we observed higher 30‐day mortality rates among discharged patients, compared to observed patients, in both the moderate‐ and high‐risk groups. This may have occurred for various reasons including patient preferences on disposition or differences in ED or observation unit care and transition plans. Similarly, the reasons behind the observed higher mortality and total SAE rates among admitted high‐ and moderate‐risk patients may be because admitted patients are by nature sicker and more complex, even when compared to patients in the same risk category who are observed or discharged.

Although we chose objective measures of risk, such as mortality and SAE, we recognize there are other potential outcomes and adverse events that are worth considering in future work, such as return ED visits and hospitalizations.

DISCUSSION

Our findings of risk stratified outcomes in a multi‐center US population suggest up to 50% of patients may be identified as low to moderate risk, with 30‐day mortality rates around 2% or less in this group. This finding contrasts with the observed 3.9% and 5.9% respective 30‐day mortality rates among discharged and observed patients in our cohort where unstructured ED decision‐making did not incorporate our risk models. We found the observed low mortality and total SAE rates among those with predicted low and low/moderate risk (25% of our population) did not differ significantly by ED disposition. This suggests our risk tool identifies truly low‐risk patients who may be safe for ED discharge and that risk is not confounded by hospitalization among these patients.

Our team will soon study how display of personalized risk estimates in real time is associated with admission decisions and 7‐ and 30‐day outcomes. We will determine how these risk estimates are assimilated with standardized workflows to assess patient response to treatment, ability for self‐care at home, shared decision‐making on optimal venue of care, and close outpatient follow up. To prepare for implementation of the risk tool, we wanted to better understand the impact of hospitalization on short‐term outcomes among different risk classes. Specifically, it is still unknown whether hospital admission may confer a short‐term survival benefit, after adjusting for other confounders, among patients who are low (or high) risk. This information would be important to share with providers and patients in shared decision‐making conversations about an optimal venue of care. Figures 2 and 3 display that in our lowest risk groups, which have been modified from our original publication to include the “low” and “low/moderate” strata, the observed 30‐day mortality rate was <1%. This low mortality rate was similar between discharged, observed, and admitted patients, suggesting hospitalization may not offer mortality and SAE benefits for these low‐risk patients. This finding identifies an important inflection point for facilitating safe ED discharge.

Before implementation, we plan to educate providers on how the tool works and how to incorporate risk estimates and disposition recommendations into practice. We expect ED providers will be reassured that we did not see a survival or SAE benefit in hospital admission for low and low/moderate risk patients and may feel more comfortable discharging these patients. If other factors can be addressed (symptom improvement, identifying the precipitant, and no active high‐risk comorbidities or significant barriers to self‐care at home), these patients can be considered safe for outpatient care with close follow up.

Several ongoing studies aim to answer the question of how a risk tool implemented in the EHR can impact admission decisions and downstream patient outcomes. A recently completed trial (results pending) across 10 hospitals in Canada aimed to study whether use of the EHMRG tool coupled with rapid outpatient follow up achieves better outcomes than conventional decision‐making. The dissemination and implementation of STRATIFY, the only risk stratification tool prospectively derived in a US population, is being studied in large healthcare systems in the United States. Our team is currently working on building the KPNC risk model in our EHR and educating providers on how the model works and how to integrate it into clinical work.

A 2017 international consensus manuscript suggested a 40% ED discharge rate for AHF patients (among facilities able to observe patients) with goals for 30‐day all‐cause mortality of <2% and 30‐day ED revisit or AHF hospitalization of <20%. Our findings of a close to 2% or lower mortality rate in nearly 50% of patients suggest this goal may be attainable, particularly in settings with good access to follow up. Use of a risk stratification tool, with a 2% accepted 30‐day mortality rate, could be expected to increase both the number of lower‐risk patients being discharged (directly from the ED or after a brief observation period) as well as the number of higher‐risk patients being admitted to the hospital. Use of risk models with acceptable risk thresholds could better align the intensity of care with heart failure risk and safely lower United States admission rates. However, risk estimations must be validated, tailored to specific ED settings, and prospectively tested to determine their incremental value in ED disposition decision making.

In a large, multi‐center cohort of ED patients with AHF in an integrated delivery system, use of a risk stratification tool helped identify lower risk patients who may be appropriate for ED discharge. Among the 25% of patients classified as lower risk, the 30‐day mortality was consistently <1% regardless of ED disposition. This tool needs to be prospectively tested to determine if its use leads to safe ED discharge and identifies those at high risk who may benefit from hospital admission.

CONFLICTS OF INTEREST

None.

AUTHOR CONTRIBUTIONS

Dana R. Sax, Dustin G. Mark, Jie Huang, Jamal S. Rana, Sean P. Collins, and Mary E. Reed conceived and designed the study and obtained research funding. Jie Huang, Dana R. Sax, and Mary E. Reed supervised the conduct of the study and data collection. Jie Huang and Mary E. Reed provided statistical advice on study design and analyzed the data. Dana R. Sax drafted the manuscript and Dustin G. Mark, Jamal S. Rana, Jie Huang, Sean P. Collins, and Mary E. Reed contributed substantially to its revision. Dana R. Sax takes responsibility for the paper as a whole.

Supporting Information

Additional supporting information may be found online in the Supporting Information section at the end of the article.

Click here for additional data file.