Validity of the Seattle Heart Failure Model after heart failure hospitalization.

AIMS: Heart failure hospitalization is a sentinel event associated with increased mortality risk. Whether long-term heart failure risk models such as the Seattle Heart Failure Model (SHFM) accurately assess risk in the post-hospital setting is unknown. METHODS AND
RESULTS: The SHFM was applied to a cohort of 2242 consecutive patients (50% women; mean age 73) on discharge after acute heart failure hospitalization and analysed for the primary endpoint of all-cause mortality. Model discrimination and calibration were assessed. Direct patient-level comparison between our study cohort and the original SHFM cohorts was also performed to confirm and quantify the degree and extent of increased mortality risk attributable to post-hospital status. The SHFM demonstrated good overall risk discrimination (area under the receiver operating characteristic curve 0.704) and was well calibrated in patients <65 years old. The SHFM significantly underestimated mortality risk in patients ≥65 years old post-hospitalization. Direct patient-level comparison revealed a stepwise increase in adjusted mortality risk attributable to post-hospital status for each advancing age group ≥65 years old. This heightened mortality risk showed a diminishing trend over 18 months after discharge.
CONCLUSIONS: The SHFM accurately predicts mortality risk in younger patients after acute heart failure hospitalization. However, patients ≥65 years old had increased adjusted mortality risk for up to 18 months after discharge compared with ambulatory heart failure patients, a pattern consistent with the well-described post-hospital syndrome.

Introduction

Despite significant advances in heart failure therapy in the past few decades, heart failure remains a condition with high mortality and hospitalization rates.1, 2, 3 Using risk models to estimate the likelihood of adverse events can identify different risk groups and guide healthcare resource allocation. Risk models also provide important prognostic information to patients and can support shared decision‐making regarding critical issues, such as advanced heart failure therapies and hospice care. In recent years, a number of validated heart failure risk models have been developed to predict mortality and readmission rates, and these are recommended for use during heart failure evaluation by the 2013 American College of Cardiology Foundation/American Heart Association Guideline for the Management of Heart Failure.4

Hospitalization for acute decompensated heart failure represents a particularly vulnerable phase for heart failure patients and continues to be associated with high mortality rates. Recent research has shown that the period of time after hospital discharge carries a transiently increased generalized risk, not limited to the condition of the index hospitalization.5 This finding has been termed the ‘post‐hospital syndrome’ and affects heart failure patients, as well as those with pneumonia, asthma, and chronic obstructive pulmonary disease.6, 7, 8 However, the degree of increased mortality risk in heart failure patients post‐hospital discharge vs. stable community heart failure patients is unclear, and it has not been determined which groups of heart failure patients are most susceptible to the post‐hospital syndrome. It is also not known how mortality risk after hospitalization changes over time and when this risk normalizes.

In this context, it is unclear how traditional heart failure risk models, such as the Seattle Heart Failure Model (SHFM), perform in heart failure patients post‐discharge. In this study, we applied the well‐validated SHFM to a cohort of patients on discharge after acute heart failure hospitalization to assess the model's discrimination and calibration in the post‐hospital setting. We also directly compared our study cohort and the original SHFM cohorts on a patient level to confirm and quantify the degree and extent of increased mortality risk attributable to post‐hospital status while adjusting for baseline disease severity.

Methods

Study group and protocol

A consecutive cohort of 2242 patients discharged with a diagnosis of acute heart failure in the cardiology department of a tertiary‐care teaching hospital (Hospital Clínico Universitario de Valencia) from 2004 to 2014 was included in this study. During the index hospitalization, a comprehensive set of clinical, biochemical, and echocardiographic variables was recorded using pre‐established registry questionnaires. The study conformed to the principles outlined in the Declaration of Helsinki and was approved by the local institutional review board.9 All patients gave informed consent. The study protocol has been previously described.10 Our primary endpoint was all‐cause mortality up to 5 years after hospital discharge. Information on the endpoint was collected from the hospital files or outpatient department. In patients who did not return to the hospital or the outpatient department, the information was obtained by establishing contact with the patient, his or her general physician, or the regional mortality registry.

Statistical analysis

Descriptive analyses of patient's baseline demographic, clinical history, discharge lab values, discharge medications, and discharge devices were performed for the entire cohort and by age group (<65 vs. ≥65). Estimated glomerular filtration rate is calculated using the Modification of Diet in Renal Disease equation. The SHFM score was calculated for each patient based on variable values at discharge, and predicted survival was derived using the original SHFM.11 Model performance was evaluated by assessing model discrimination (ability to separate high‐risk and low‐risk patients) and model calibration (predicted vs. observed survival). Model discrimination was assessed by 1‐year area under the receiver operating characteristic curve (AUROC), same as in the original SHFM study.11 Model calibration was assessed by comparing observed vs. predicted survival at 1 to 5 years. Because of an a priori concern that New York Heart Association (NYHA) functional class may be underestimated at hospital discharge, a sensitivity analysis was performed by artificially increasing the NYHA functional class by I for patients with Classes I and II symptoms (so that they are Classes II and III, respectively) and reassessing model calibration.

Our study cohort was also compared with the original SHFM derivation and validation cohorts on a patient level by adding a binary variable indicating post‐discharge status. The hazard ratios (HRs) of post‐discharge status for different age groups and time periods after discharge were calculated using Cox proportional hazard models. As our study cohort of hospitalized patients likely have more advanced heart failure than ambulatory patients, the HRs in our analyses were adjusted for SHFM score to account for disease severity. The HRs calculated for different time periods were independent and not inclusive of previous time periods. A two‐sided P‐value < 0.05 was considered to be statistically significant for single comparisons, and for multiple comparisons, the Šidák‐corrected P‐values were used so that the familywise type I error rate is <0.05. All analyses were performed using R, version 3.25 (The R Foundation for Statistical Computing. Vienna, Austria).

Results

Patient characteristics

Our cohort of 2242 patients was 50% women and had a mean age of 73. Mean left ventricular ejection fraction (LVEF) was 50% and a majority of patients had NYHA functional class II symptoms on discharge. The aetiology of heart failure was ischaemic cardiomyopathy in 37% of patients; 98% of patients were discharged on loop diuretics, 67% were discharged on either angiotensin receptor blockers or angiotensin‐converting enzyme inhibitors, 65% were discharged on beta‐blockers, and 34% were discharged on spironolactone. The average length of hospitalization was 8 days, with an interquartile range of 6 to 12 days. Baseline patient characteristics are summarized in Table 1.

Table 1

Baseline characteristics of patients discharged after hospitalization for acute heart failure

Characteristic	All ages	Age < 65	Age ≥ 65
	N = 2242	N = 443	N = 1799
Age, years	72.8± 11.1	55.0± 8.1	77.2± 6.4
Gender, male	1127 (50.3%)	306 (69.1%)	821 (45.6%)
Ischaemic aetiology	820 (36.6%)	124 (28.0%)	696 (38.7%)
LVEF %	49.8± 15.4	43.6± 16.0	51.4± 14.9
NYHA functional class:
I	692 (30.9%)	232 (52.4%)	460 (25.6%)
II	1174 (52.4%)	166 (37.5%)	1008 (56.0%)
III	370 (16.5%)	43 (9.7%)	327 (18.2%)
IV	6 (0.3%)	2 (0.5%)	4 (0.2%)
Medications at discharge:
Beta‐blocker	1458 (65.0%)	318 (71.8%)	1140 (63.4%)
ACE‐I/ARB	1509 (67.3%)	332 (75.0%)	1177 (65.4%)
Loop diuretic	2188 (97.6%)	427 (96.4%)	1761 (97.9%)
Spironolactone	761 (33.9%)	235 (53.0%)	526 (29.2%)
Digoxin	474 (21.2%)	107 (24.2%)	367 (20.4%)
Devices at discharge (among EF ≤ 35% patients):	N = 488	N = 167	N = 321
ICD	43 (8.8%)	10 (6.0%)	33 (10.3%)
CRT	23 (4.7%)	2 (1.2%)	21 (6.5%)
Lab values at discharge:
Haemoglobin (g/dL)	12.5± 1.9	13.4± 2.0	12.3± 1.8
Lymphocyte %	18.1± 10.0	20.1 ± 9.9	17.6± 10.0
Uric acid (mg/dL)	8.0± 2.4	8.2 ± 2.3	8.0± 2.4
Total cholesterol (mg/dL)	166± 44	171 ± 48	165± 43
Sodium (mEq/L)	138.6± 4.5	138.4 ± 4.4	138.6± 4.5
Creatinine (mg/dL)	1.26± 0.57	1.13 ± 0.47	1.29± 0.59
eGFR (mL/min/1.73 m2)	58.6± 26.1	71.4± 33.5	55.4± 22.9
Length of stay, days	8 (6–12)	8 (5–12)	8 (6–12)

ACE‐I, angiotensin‐converting enzyme inhibitors; ARB, angiotensin receptor blocker; CRT, cardiac resynchronization therapy; eGFR, estimated glomerular filtration rate; ICD, implantable cardioverter‐defibrillator; LVEF, left ventricular ejection fraction; NYHA, New York Heart Association.

Values are mean± standard deviation, n (%), or median (interquartile range).

Follow‐up

Median length of follow‐up was 2.4 years overall and 3.7 years excluding patients who died before the end of study.

Model performance

In our study cohort of 2242 patients discharged after heart failure hospitalization, the SHFM demonstrated adequate risk discrimination with 1‐year AUROC of 0.704 (Figure ). Among the 1541 patients with LVEF > 40%, 1‐year AUROC is 0.666, and among the 701 patients with LVEF ≤ 40%, 1‐year AUROC is 0.789. This discriminative power is similar to that displayed by the SHFM in its original derivation and validation cohorts, which has a combined AUROC of 0.729 (95% confidence interval [CI] 0.714 to 0.744).10

Figure 1

Kaplan–Meier survival curves of the study cohort by quintiles of the SHFM score (log‐rank P < 0.0001). The SHFM score is a significant predictor of survival in this population (P < 0.0001).

Analysis of model calibration showed that observed survival in our post‐discharge cohort was lower than SHFM‐predicted survival over 5 years of follow‐up (Figure ). In addition, observed survival was lower than predicted survival to a similar degree in patients with LVEF > 40% and ≤40% (Supporting Information, ). Because of an a priori concern that NYHA functional class may be underestimated at hospital discharge, a sensitivity analysis was done by artificially elevating all NYHA Classes I and II patients to Classes II and III, respectively. The sensitivity analysis narrowed but did not eliminate the difference between observed and predicted survival (Supporting Information, ).

Figure 2

Seattle Heart Failure Model‐predicted vs. observed survival rates over 5 years.

Mortality after discharge by age groups and time after discharge

Comparing our post‐discharge cohort with SHFM's original derivation and validation cohorts on a patient level, patient age was found to have a significant interaction with post‐discharge status in predicting mortality (P < 0.0001). Specifically, the adjusted HR of post‐discharge status showed a significant and stepwise increase with increasing age above 65 (Figure ). For patients younger than 65 years old, the adjusted HR of post‐discharge status was non‐significant (HR 0.95, 95% CI 0.79–1.16, P = 0.63). The adjusted HRs for age groups above 65 were statistically significant and demonstrated an upward trend with increasing age (age 65–69: HR 1.34, 95% CI 1.09–1.63, P < 0.005; age 70–74: HR 1.41, 95% CI 1.19–1.66, P < 0.0001; age 75–79: HR 1.63, 95% CI 1.42–1.88, P < 0.0001; age 80 or above: HR 2.36, 95% CI 2.11–2.64, P < 0.0001).

Figure 3

Mortality hazard ratios of post‐discharge status by age groups. Non‐significant for age < 65 and P < 0.005 for all other groups (Šidák‐corrected P‐value cut‐off < 0.01).

In addition, in patients older than 65 years of age, there was a statistically significant interaction in mortality hazard between post‐discharge status and time after discharge. A statistically significant increase in mortality attributable to post‐discharge status was noted within 18 months of discharge and diminished over time (Figure ). Within the first 6 months of discharge, the adjusted HR of post‐discharge status was 2.0 (95% CI 1.65–2.42, P < 0.001), which decreased to 1.73 (95% CI 1.39–2.14, P < 0.001) between 6 and 12 months of discharge and further decreased to 1.63 (95% CI 1.29–2.06, P < 0.001) from 12 to 18 months. After 18 months, post‐discharge status carried no statistically significant mortality hazard using the Šidák‐corrected P‐value cut‐off of <0.01. Among patients younger than 65 years of age, there was no statistically significant increase in mortality hazard due to post‐discharge status for any of the aforementioned time periods using the Šidák‐corrected P‐value cut‐off of <0.01.

Figure 4

Mortality hazard ratios of post‐discharge status at 6‐month time intervals after discharge for patients >65 years of age. P < 0.001 for all periods <18 months, non‐significant afterwards (Šidák‐corrected P‐value cut‐off < 0.01).

Discussion

Significance

Applying the SHFM to patients discharged after acute heart failure hospitalization, we showed that the SHFM had adequate discriminative power similar to that in its original cohorts. In younger patients (<65 years old), the SHFM was well calibrated, but the SHFM significantly underestimated mortality risk post‐discharge in older patients (≥65 years old). Through a patient‐level analysis comparing our post‐hospital cohort and the SHFM's original cohorts adjusted for baseline risk, we confirmed that only patients ≥65 years old had increased mortality attributable to hospitalization, and the mortality increase diminished over 18 months after discharge.

The significantly elevated mortality risk for older patients after hospital discharge observed in our cohort is consistent with the post‐hospital syndrome described in previous studies.5, 6, 7, 8 The stepwise increase in mortality hazard associated with post‐hospital status with each advancing age group ≥65 is alarming. In fact, we found that patients ≥80 years of age had, post‐discharge, more than twice the mortality of non‐hospitalized patients even after adjustment for heart failure severity. This finding could be helpful to clinicians as they counsel older patients and their families on prognosis, goals of care, and preparation for the transition from hospital to home. For the healthcare system, awareness of the post‐hospital syndrome could direct resource allocation, such as closer monitoring and follow‐up for older patients post‐discharge.

The increased mortality for older patients is most pronounced in the first 6 months after discharge. This is consistent with the findings of multiple previous studies of post‐hospital syndrome for heart failure and non‐heart failure diagnoses.5, 7, 8, 12 However, in our study, we found that the heightened mortality risk persists up to 18 months, which is a longer duration than previously described and underscores a potentially more persistent post‐hospital syndrome. In addition, in a real‐world epidemiology study, it was shown that heart failure patients with remote (>1 year) hospitalization had increased mortality rate compared with patients never hospitalized, but the analysis was unadjusted for baseline disease severity.13 We showed that even after adjusting for baseline risk using the SHFM, post‐discharge heart failure patients still had increased mortality risk compared with ambulatory heart failure patients up to 18‐months post‐discharge.

There are several hypothesized explanations for the increased mortality associated with heart failure hospitalization in older patients. First, elderly patients likely have more non‐cardiac co‐morbidities as well as increased overall frailty. It is possible that their co‐morbid conditions were exacerbated in the hospital either due to heart failure itself or its treatments. Second, elderly patients may have a particularly difficult transition from hospital to home. Common post‐discharge tasks such as reconciling medications, wound care, arranging follow‐up care, and rehabilitation may overwhelm elderly patients and contribute to the excess mortality. Finally, risk models in general are inherently more accurate within the populations from which they were derived, and applying these models to other patient populations may lead to significantly biased risk estimates.14, 15

Limitations

There are several limitations to address. First, our patient cohort is from a single academic medical centre in Spain, which may limit the generalizability of our findings. However, the relatively large size of our study cohort, as well as the fact that previously published analyses of this cohort did not discover any validity issue, lessen our concern about the generalizability of our results.9, 16, 17, 18 Second, we only analysed post‐discharge mortality and did not include readmissions after index hospital discharge. It is possible that the longer duration of post‐hospital syndrome we observed is due to repeated hospitalizations, which may have artificially extended the observed effect of post‐hospital syndrome. However, the stepwise decrease in the difference between observed and predicted mortality with time after index hospitalization argues against the possibility that repeated hospitalizations significantly biased our results. Finally, the difference between observed and predicted mortality may also be due to an underestimation of NYHA functional class post‐discharge, as this is a key component in the SHFM model and is difficult to accurately assess at discharge. However, our sensitivity analysis demonstrated that this is unlikely the main cause of the higher‐than‐predicted mortality.

Future research

Our study suggests multiple avenues of future research. For example, the mechanisms of post‐hospital syndrome in heart failure patients are poorly understood. Is post‐hospital syndrome a complication of the index hospitalization or the reflection of a challenging transition from hospital to home? Our finding that elderly patients are particularly affected by the post‐hospital syndrome offers a clue that co‐morbidities may play an important role, which needs to be further explored by future research. In addition, similar to heart failure readmissions, whether closer follow‐up or home visits post‐discharge can mitigate the post‐hospital syndrome is an interesting question and remains to be studied.

Conflict of interest

The SHFM copyright is owned by the University of Washington. Licensing fees are paid to the University of Washington. S.L., P.M., E.N., and J.S. have no relationship to disclose. J.N. received board membership fees and travel expenses from Novartis, Roche Diagnostics, Abbott, Rovi, and Vifor. W.C.L. was a consultant for Medtronic and Impulse Dynamics, a steering committee member of ADMIRE ICD (GE Healthcare) and Respircardia – Remedē FDA Post Approval Registry, a clinical endpoint committee member of CHAMPION Post Approval Study (CardioMems – Abbott), SOLVE‐CRT (EBR Systems, Inc.), and local PI for PARAGON HF (Novartis) clinical trial.

Funding

This work was supported by a grant from Spain's Ministry of Economy and Competitiveness through the Carlos III Health Institute: CIBER‐CV 16/11/00420, Madrid, Spain.

Figure S1. SHFM‐predicted versus observed survival rates over 5 years for patients with EF < = 40% and EF > 40%.

Figure S2. SHFM‐predicted versus observed survival rates over 5 years with artificially inflated NYHA functional class (class I increased to II, class II increased to III).

Click here for additional data file.