Meta-Analysis Global Group in Chronic (MAGGIC) Heart Failure Risk Score: Validation of a Simple Tool for the Prediction of Morbidity and Mortality in Heart Failure With Preserved Ejection Fraction.

Background The Meta-Analysis Global Group in Chronic Heart Failure ( MAGGIC ) mortality risk score, derived from a large sample of patients with heart failure ( HF ) across the spectrum of ejection fraction ( EF ), has not yet been externally validated in a well-characterized HF with preserved EF cohort with adjudicated morbidity outcomes. Methods and Results We evaluated the MAGGIC risk score (composed of 13 clinical variables) in 407 patients with HF with preserved EF enrolled in a prospective registry and used Cox regression to evaluate its association with morbidity/mortality. We used receiver-operating characteristic analysis to compare the predictive ability of the MAGGIC risk score with the more complex Seattle Heart Failure Model, and we determined the value of adding B-type natriuretic peptide to the MAGGIC risk score for risk prediction. During a mean follow-up time of 3.6±1.8 years, 28% died, 32% were hospitalized for HF , and 55% had a cardiovascular hospitalization and/or death. The MAGGIC score, a mean± SD of 18±7, was significantly associated with mortality ( P<0.0001), HF hospitalizations ( P<0.0001), and the combined end point of cardiovascular-related hospitalizations or death (hazard ratio, 1.8 [95% confidence interval, 1.6-2.1], per 1- SD increase in the MAGGIC score; P<0.0001). Receiver-operating characteristic analyses showed that MAGGIC and Seattle Heart Failure Model performed similarly in predicting HF with preserved EF outcomes, but the MAGGIC score demonstrated better calibration for hospitalization outcomes. Further analyses showed that B-type natriuretic peptide was additive to the MAGGIC risk score for predicting outcomes ( P<0.01 by likelihood ratio test). Conclusions The MAGGIC risk score is a simple, yet powerful method of risk stratification for both morbidity and mortality in HF with preserved EF . Clinical Trial Registration URL: http://www.clinicaltrials.gov . Unique identifier: NCT01030991.

Clinical Perspective

What Is New?

In heart failure (HF) with preserved ejection fraction, the Meta‐Analysis Global Group in Chronic Heart Failure risk score is predictive of adverse outcomes, and the addition of B‐type natriuretic peptide to the model improves its prognostic capability.

The Meta‐Analysis Global Group in Chronic Heart Failure risk score is shown herein, for the first time, to be predictive of important HF with preserved ejection fraction morbidity outcomes, including HF hospitalizations and cardiovascular hospitalizations, in addition to all‐cause mortality.

What Are the Clinical Implications?

The Meta‐Analysis Global Group in Chronic Heart Failure risk score is a simple tool, incorporates readily available clinical variables, and may be useful in helping in the clinic to risk stratify patients with HF with preserved ejection fraction.

Using specific ranges of the Meta‐Analysis Global Group in Chronic Heart Failure risk score as inclusion criteria in HF with preserved ejection fraction clinical trials may be especially useful to help enroll patients who are neither too healthy nor too sick, thereby ensuring an adequate event rate while simultaneously excluding patients with advanced disease who are unlikely to benefit from the intervention.

Introduction

To date, several randomized clinical trials enrolling thousands of patients and at tremendous cost have been conducted without successfully identifying a single therapy with a clearly proven survival benefit in heart failure (HF) with preserved ejection fraction (HFpEF). In addition to the challenge of improving mortality rates in HFpEF, perhaps equally alarming is the excess morbidity associated with this disorder.1 In fact, recent analyses have demonstrated that HFpEF hospitalizations are exceeding those attributable to HF with reduced EF (HFrEF).2, 3

HFpEF is increasingly recognized as a syndrome with great heterogeneity4, 5; such heterogeneity likely extends to both the clinical features of those with the disorder and the prognosis of the individual patient carrying the HFpEF diagnosis. Because the HFpEF disease burden and prognosis at the population level is of a similar magnitude to HFrEF, a dilemma further compounded by a lack of proven therapies, the ability to more effectively risk stratify patients with HFpEF is greatly needed. In addition to improving the ability to prognosticate and risk stratify patients with HFpEF, an effective risk model may also have the potential to help inform the design of future clinical trials. Although several risk models for HFrEF mortality exist, including the Seattle Heart Failure Model (SHFM)6 and the Heart Failure Survival Score,7 there is a paucity of validated risk models applicable to HFpEF.8

The Meta‐Analysis Global Group in Chronic Heart Failure (MAGGIC) investigators developed a new HF risk model, derived from a large international database of patients with HF, which included those with both HFrEF and HFpEF.9 The MAGGIC risk model is clinically attractive because it includes routinely collected variables, is simple to use, and was shown to be highly effective at predicting 1‐ and 3‐year mortality in patients with HF. The ability of MAGGIC to predict all‐cause mortality has been investigated in a large HF registry in Sweden, and retrospectively in an electronic health record database.10, 11 Both of these studies showed that MAGGIC was able to accurately predict mortality on the population level. However, these prior studies did not adjudicate the diagnosis of HFpEF and outcomes in these patients. Besides these limitations, further external validation of MAGGIC is especially needed because most patients in the MAGGIC derivation and validation cohorts were from clinical trial populations and not from a contemporary patient cohort (studies included in MAGGIC were conducted between 1980 and 2006).9 Most important, similar to most other HF risk models, the MAGGIC risk model evaluated only mortality and not the ability to predict morbidity, which may be more important than mortality in elderly patients with HFpEF who often have multiple comorbidities. Finally, MAGGIC also does not include B‐type natriuretic peptide (BNP), which is a commonly used risk marker in HF.

We thus conducted the present study to do the following: (1) evaluate the external validity of the MAGGIC risk model in predicting both morbidity and mortality in a prospective cohort of patients with prevalent HFpEF and carefully adjudicated outcomes; and (2) compare the MAGGIC risk model with the established SHFM and evaluate whether MAGGIC is more predictive of outcomes when incorporating BNP into the model.

Methods

The data will not be made available to other researchers for purposes of reproducing the results because of institutional review board restrictions. However, the study materials and the methods used in the analyses will be provided to any researcher for purposes of replicating the study procedures on request from the corresponding author.

Study Population

Between March 2008 and May 2011, 420 consecutive patients were prospectively enrolled from the outpatient clinic of the Northwestern University HFpEF Program as part of a systematic observational study of HFpEF, as described previously.5, 12, 13 Briefly, all patients were recruited as outpatients after being hospitalized for HF. Patients were initially identified by an automated daily query of the inpatient electronic medical record at Northwestern Memorial Hospital using the following search criteria: (1) diagnosis of HF or the words “heart failure” in the hospital notes; (2) BNP >100 pg/mL; or (3) administration of ≥2 doses of intravenous diuretics. After this initial screening step, only those patients with an left ventricular (LV) EF (LVEF >50% and who also met the modified Framingham criteria for HF were offered postdischarge follow‐up in a specialized HFpEF outpatient program. The HF diagnosis was confirmed in the posthospitalization, outpatient HFpEF clinic. Patients with greater than moderate valvular disease, prior cardiac transplantation, history of LVEF <40% (ie, “recovered” EF), or a diagnosis of constrictive pericarditis were excluded. All study participants gave written, informed consent, and the institutional review board at Northwestern University approved the study. For the present analysis, 13 patients unavailable for follow‐up were not included, leaving n=407 for the final analysis (all of whom had complete data for calculation of the MAGGIC risk score).

Clinical Characteristics

We collected the following data in all study participants: demographics, race/ethnicity, New York Heart Association functional class, comorbidities, medications, vital signs, body mass index, and laboratory data, including serum sodium, blood urea nitrogen, creatinine, hemoglobin, and BNP. Estimated glomerular filtration rate was calculated using the Modified Diet in Renal Disease equation.

Definitions of Comorbidities

Hypertension was defined by systolic blood pressure >140 mm Hg or diastolic blood pressure >90 mm Hg, physician‐documented history of hypertension, or current use of antihypertensive medications. Diabetes mellitus was defined by the presence of physician‐documented history of diabetes mellitus or the use of oral hypoglycemic agents or insulin for the treatment of hyperglycemia. Coronary artery disease was defined by the presence of physician‐documented history of coronary artery disease, known coronary stenosis >50%, history of myocardial infarction, percutaneous intervention, coronary artery bypass grafting, or abnormal stress test results consistent with myocardial ischemia. Obesity was defined by a body mass index ≥30 kg/m2. Chronic kidney disease was defined as an estimated glomerular filtration rate <60 mL/min per 1.73 m2.

Echocardiography

All study participants underwent comprehensive 2‐dimensional echocardiography with Doppler and tissue Doppler imaging using commercially available ultrasound systems with harmonic imaging (Philips iE33 or 7500 [Philips Medical Systems, Andover, MA]; or Vivid 7 [GE Healthcare, General Electric Corp, Waukesha, WI]). Cardiac structure and function were quantified as recommended by the American Society of Echocardiography.14, 15, 16

LV end‐diastolic and end‐systolic volumes, and left atrial volume, were measured in the apical 4‐ and 2‐chamber views using the biplane method of discs. LVEF was calculated as follows: (LV end‐diastolic volume−LV end‐systolic volume)/LV end‐diastolic volume. LV mass index was calculated using the linear method, as outlined in the American Society of Echocardiography guidelines.14 LV diastolic function was graded according to published criteria by using mitral inflow characteristics and tissue Doppler e′ velocities.17 Tissue Doppler e′ and s′ velocities were measured at the septal and lateral aspects of the mitral annulus. Sample volume size and placement were optimized for all pulse‐wave Doppler and tissue Doppler measurements. All Doppler and tissue Doppler measurements were averaged over 3 beats (5 beats for patients with atrial fibrillation).

Right heart parameters were measured on echocardiography, according to published guidelines.16 Specifically, we measured right ventricular end‐diastolic area and right ventricular end‐systolic area, and calculated right ventricular fractional area change. Tricuspid annular plane systolic excursion was also calculated. Last, pulmonary artery systolic pressure was measured using the peak tricuspid regurgitation velocity (to estimate peak tricuspid regurgitation gradient) and adding that to the estimated right atrial pressure, which was based on size and collapsibility of the inferior vena cava.16

All echocardiographic measurements were made blinded to all other data by an experienced research sonographer using ProSolv 4.0 echocardiographic analysis software (FujiFilm, Indianapolis, IN) and verified by an experienced investigator with expertise in echocardiography (S.J.S.).

Outcomes

After enrollment, study participants were evaluated in the Northwestern HFpEF Program at least every 6 months or sooner, as clinically indicated. At each visit, intercurrent hospitalizations were documented, reviewed, and categorized as being attributable to cardiovascular or noncardiovascular causes. For cardiovascular hospitalizations, specific causes (eg, HF, acute coronary syndrome, or arrhythmia) were identified. Every 6 months, participants (or their proxy) were contacted to determine vital status, with verification of deaths through query of the Social Security Death Index. Enrollment date was defined as the first visit to the outpatient HFpEF clinic. Date of last follow‐up was defined as date of death or last HFpEF clinic visit. Follow‐up was complete in all patients. Outcomes were adjudicated by 2 clinical cardiologists, blinded to all other collected data and independent of each other. Discrepant adjudication of events was resolved by committee discussion. Outcomes were ascertained from the time of enrollment to December 31, 2013.

Risk Models

We evaluated the utility of applying the MAGGIC risk model to predict clinical outcomes in our cohort with HFpEF. MAGGIC consists of the following 13 predictor variables: age, sex, body mass index, systolic blood pressure, EF, creatinine, current smoker, diabetes mellitus, chronic obstructive pulmonary disease, New York Heart Association class, HF duration >18 months, β‐blocker use, and angiotensin‐converting enzyme inhibitor use. In addition, we evaluated and compared the discrimination ability of the MAGGIC risk model with the SHFM. A detailed list of the variables included in both models, each consisting of a combination of dichotomous and continuous variables, is given in Table 1. To calculate the integer risk score for each model, the value of each variable was multiplied by its β coefficient, which was derived from a proportional hazard model from each model's original derivation cohort.6, 9

Table 1

A Comparison of the Components of the MAGGIC Risk Model and the SHFM

Variable	MAGGIC (13 Variables)	SHFM (20 Variables)
Age	X	X
Male sex	X	X
Diabetes mellitus	X
COPD	X
Current smoker	X
Ischemic cause		X
HF duration <18 mo	X
NYHA class	X	X
β‐Blocker use	X	X
ACE‐inhibitor/ARB use	X	X
Aldosterone blocker use		X
Allopurinol use		X
Statin use		X
Loop diuretic dose		X
Systolic blood pressure	X	X
Body mass index	X	X
Serum creatinine	X
Sodium		X
Hemoglobin		X
Total cholesterol		X
Lymphocytes		X
Uric acid		X
Ejection fraction	X	X
Device therapy		X
QRS duration		X

ACE indicates angiotensin‐converting enzyme; ARB, angiotensin receptor blocker; COPD, chronic obstructive pulmonary disease; device therapy, intracardiac defibrillator and/or cardiac resynchronization therapy; HF, heart failure; MAGGIC, Meta‐Analysis Global Group in Chronic Heart Failure; NYHA, New York Heart Association; SHFM, Seattle Heart Failure Model.

Statistical Analysis

Baseline characteristics for all patients in the study cohort were expressed as counts and percentages for categorical variables and mean±SD for continuous variables that were normally distributed. Right‐skewed data were presented as median and 25th to 75th percentile. Clinical characteristics from both risk models for all patients with and without clinical events were compared by χ2 tests for categorical variables and unpaired t tests for continuous variables. Pearson's correlations between the different risk models were also calculated. We used Cox proportional hazards models to determine the prognostic utility of MAGGIC. Event‐free survival rates and rates of freedom from cardiovascular hospitalization, HF hospitalization, and the combined end point of cardiovascular hospitalization and/or death using the MAGGIC risk model were determined using the Kaplan‐Meier method, and survival curves were compared using the log‐rank test. The proportionality assumption was verified for all models. In sensitivity analyses, we also repeated our analyses using competing risks regression analyses and plotted cumulative incidence function curves as alternatives to Cox regression and the Kaplan‐Meier curves, respectively.18

Next, we calculated C‐statistics for each model to determine the discrimination ability of MAGGIC and SHFM (and BNP) for each of the aforementioned clinical outcomes, and we created calibration plots for MAGGIC and SHFM for each of these outcomes. We also compared model‐predicted 1‐ and 3‐year survival of MAGGIC and SHFM with actual 1‐ and 3‐year survival in our HFpEF cohort. Finally, given the absence of BNP in the MAGGIC model, we examined the correlation between BNP and the MAGGIC score. We stratified patients into 4 groups on the basis of median levels of both BNP and MAGGIC and used Kaplan‐Meier analyses to evaluate the event‐free survival in each group, and we determined the utility of adding BNP to MAGGIC for predicting outcomes using the likelihood ratio test. A 2‐sided P<0.05 was considered significant. All analyses were performed using Stata, v.12.1 (StataCorp, College Station, TX).

Results

Tables 2 and 3 list the baseline demographic, clinical, echocardiographic, and invasive hemodynamic characteristics of the study cohort. A total of 407 patients were enrolled in the study, with a mean age of 65±13 years; 62% were women, and 39% were black. Comorbid conditions were common, most patients had either New York Heart Association class II or III HF, and the study patients had evidence of elevated left‐sided filling pressures (mean pulmonary capillary wedge pressure of 23.2±9.1 mm Hg). During a mean follow‐up time of 3.6±1.8 years, 115 of 407 died (28%), 129 of 407 were hospitalized for HF (32%), 174 of 407 were hospitalized for a cardiovascular‐related reason (43%), and 224 of 406 experienced the combined end point of cardiovascular hospitalization and/or death (55%).

Table 2

Baseline Clinical Characteristics of the Study Cohort

Clinical Characteristic	Value (N=407)
Demographics
Age, y	65±13
Female sex, n (%)	253 (62)
Race, n (%)
White	210 (52)
Black	159 (39)
Other	38 (9)
Comorbidities, n (%)
Coronary artery disease	190 (48)
Diabetes mellitus	133 (33)
Atrial fibrillation	108 (26)
Hypertension	314 (77)
Chronic kidney disease	134 (33)
Obesity	216 (53)
Chronic obstructive pulmonary disease	152 (37)
Obstructive sleep apnea	150 (36)
Smoker	165 (40)
Cerebrovascular accident	30 (8)
Peripheral vascular disease	28 (7)
Medications, n (%)
ACE inhibitor or angiotensin receptor blocker	223 (55)
β Blocker	271 (67)
Aldosterone antagonist	48 (12)
Diuretic therapy	290 (71)
Aspirin	184 (45)
Statin	204 (50)
Warfarin	27 (24)
Laboratory data
Sodium, mEq/L	138±3
Serum creatinine, mg/dL	1.2 (0.9–1.6)
Total cholesterol, mg/dL	153±43
Uric acid, mg/dL	7.9±3.2
Hemoglobin, g/dL	11.9±1.9
Lymphocytes, %	21.5±10.0
B‐type natriuretic peptide, pg/mL	230 (80–525)
NYHA functional class, n (%)
I	48 (12)
II	163 (40)
III	187 (46)
IV	9 (2)

All continuous variables are expressed as mean±SD or median (25th–75th percentile) for right‐skewed data. ACE indicates angiotensin‐converting enzyme; NYHA, New York Heart Association.

Table 3

Baseline Echocardiographic and Invasive Hemodynamic Data

Echocardiography	Value (N=407)
Left heart size and function
LV ejection fraction, %	61±7
LV mass index, g/m2	104.0±37.8
LV end‐diastolic volume index, mL/m2	41.1±12.0
LV end‐systolic volume index, mL/m2	16.5±7.2
Left atrial volume index, mL/m2	34.2±14.3
RV size and function
RV fractional area change, %	43±7
TAPSE, cm	2.0±0.6
RV end‐diastolic area index, cm2/m2	13.9±3.8
RV end‐systolic area index, cm2/m2	8.1±2.8
Mitral regurgitation
Absent	234 (58)
Mild	111 (28)
Moderate	57 (15)
Pulmonary hypertension
Estimated pulmonary artery systolic pressure, mm Hg	44.0±15.5
Diastolic dysfunction grade
0 (Normal)	32 (8)
I (Mild)	42 (11)
II (Moderate)	160 (39)
III (Severe)	137 (34)
Indeterminate	36 (9)
Right heart catheterization (N=225)
Right atrial pressure, mm Hg	13.7±6.5
Pulmonary artery systolic pressure, mm Hg	52.3±17.4
Pulmonary artery mean pressure, mm Hg	34.4±10.9
Pulmonary capillary wedge pressure, mm Hg	23.2±9.1
Cardiac output (thermodilution), L/min	6.1±2.2
Cardiac index, L/min per m2	3.0±1.0
Pulmonary vascular resistance, Wood units	2.0±1.5

All categorical variables expressed as number (percentage). All continuous variables are expressed as mean±SD. LV indicates left ventricular; RV, right ventricular; TAPSE, tricuspid annular plane systolic excursion.

The MAGGIC risk model correlated moderately well with the SHFM (r=0.53; P<0.001). As shown in Table 4, several of the individual predictor variables incorporated within each of the risk models were also associated with an increased risk of adverse outcomes in our cohort on univariate analysis. Figure 1A shows the relative contributions of the individual MAGGIC continuous predictor variables (and the lumped categorical predictor variables) to the MAGGIC integer risk score, with renal dysfunction (serum creatinine) carrying the highest predictive value in our cohort with HFpEF.

Table 4

Variables in the MAGGIC Risk Model and the SHFM: Association With Adverse Outcomes

Variable	No Cardiovascular Hospitalization or Death (N=183)	Cardiovascular Hospitalization or Death (N=224)	P Value
Age, y	61.6±13.1	67.3±12.2	<0.001
Male sex	66 (36)	88 (39)	0.51
Body mass index, g/m2	32.2±8.7	32.9±9.9	0.49
Systolic blood pressure, mm Hg	125±19	125±21	0.75
Ejection fraction, %	61±6	61±7	0.92
Current smoker	74 (41)	91 (41)	0.97
Diabetes mellitus	41 (22)	92 (41)	<0.001
COPD	55 (30)	97 (43)	0.006
HF duration, mo	1.0 (1.0–4.4)	1.0 (1.0–4.9)	0.08
NYHA class			<0.001
I	33 (18)	15 (7)
II	90 (49)	73 (33)
III	59 (32)	128 (57)
IV	1 (1)	8 (4)
Creatinine, mg/dL	1.0 (0.8–1.3)	1.3 (1.0–1.9)	<0.001
ACE inhibitor or ARB	102 (56)	121 (54)	0.73
β Blocker	109 (60)	162 (72)	0.007
Statin	79 (43)	125 (56)	0.011
Allopurinol	6 (3)	12 (5)	0.31
Loop diuretic dose, mg furosemide equivalents	40 (20–80)	40 (20–80)	0.16
Sodium, mEq/L	139±3	138±3	<0.001
Hemoglobin, g/dL	12.3±1.7	11.5±1.9	<0.001
Lymphocyte count, %	24.3±10.3	19.5±9.4	<0.001
Total cholesterol, g/dL	164±43	145±41	<0.001
Uric acid, g/dL	7.2±2.7	8.3±3.4	0.08
Heart rate, bpm	76±15	73±14	0.03
Mean arterial pressure, mm Hg	90±13	88±13	0.09
Ischemic cause	92 (50)	146 (65)	0.002
Scores
MAGGIC risk model	14.9±6.3	20.4±7.3	<0.0001 (P=1.94 × 10‐8)
SHFM	0.59±0.72	1.16±0.90	<0.0001 (P=2.97 × 10‐5)

All categorical variables are expressed as number (percentage). All continuous variables are expressed as mean±SD or median (25th–75th percentile). ACE indicates angiotensin‐converting enzyme; ARB, angiotensin receptor blocker; bpm, beats per minute; COPD, chronic obstructive pulmonary disease; HF, heart failure; MAGGIC, Meta‐Analysis Global Group in Chronic Heart Failure; NYHA, New York Heart Association; SHFM, Seattle Heart Failure Model.

Figure 1

A, Relative contributions of individual predictor variables in the Meta‐Analysis Global Group in Chronic Heart Failure (MAGGIC) to total MAGGIC integer risk score. B, Histogram of MAGGIC integer risk scores in the cohort with heart failure with preserved ejection fraction (HFpEF). A, To put all of the MAGGIC risk score component variables on a single graph, each continuous variable that was a component of the risk score (age, creatinine, systolic blood pressure, New York Heart Association [NYHA] class, and body mass index) was standardized to a mean of 0 and an SD of ±1. All other components of the score (ie, categorical variables) were added together, and this summary variable was also standardized to a mean of 0 and an SD of ±1. Next, locally weighted smoothed scatterplots of each of the MAGGIC risk score predictors were plotted against the overall risk score to show the relative contribution of high and low values of the risk score components to the overall MAGGIC risk score. B, The distribution of MAGGIC integer risk scores for all 407 patients in the cohort with HFpEF shows a nearly bell‐shaped curve with a mean score of 20±6.

Figure 1B displays the distribution of MAGGIC risk scores in our cohort with HFpEF. The mean MAGGIC risk score was 18±7. Table 5 displays the hazard ratios (Cox regression) and subdistribution hazard ratios (competing risks regression) for various adverse outcomes per 1‐SD increase in the MAGGIC risk score. The MAGGIC risk score was associated with HF hospitalization, cardiovascular hospitalization, death, and the combination of these outcomes (P<0.01 for all regression models, even after accounting for competing risks). Figure 2 displays these associations graphically by showing Kaplan‐Meier curves of event‐free survival stratified by tertiles of the MAGGIC risk score for each end point. Figure 3 displays the same associations using the cumulative incidence function for competing risks regression. Predicted 1‐year survival was 90% with MAGGIC and 88% with SHFM, and actual 1‐year survival was 92% in the cohort. Predicted 3‐year survival was 79% with MAGGIC versus actual 3‐year survival of 80% in the cohort.

Table 5

Association of the MAGGIC Risk Score With Adverse Outcomes in HFpEF

Outcome	Cox Regression			Fine‐Gray Competing Risks Regressiona
	HRb	95% CI	P Value	sHRb	95% CI	P Value
HF hospitalization	1.7	1.4–2.0	<0.001	1.4	1.2–1.8	0.004
Cardiovascular hospitalization	1.7	1.4–2.0	<0.001	1.4	1.2–1.7	<0.001
Death	2.2	1.8–2.7	<0.001	2.2	1.8–2.7	<0.001
Cardiovascular hospitalization or death	1.8	1.6–2.1	<0.001	1.4	1.2–1.7	<0.001

CI indicates confidence interval; HF, heart failure; HFpEF, HF with preserved ejection fraction; HR, hazard ratio; MAGGIC, Meta‐Analysis Global Group in Chronic Heart Failure; sHR, subdistribution HR.

Competing risks for the following: (1) HF hospitalization: non‐HF cardiovascular hospitalizations, noncardiovascular hospitalizations, and death; (2) cardiovascular hospitalization: noncardiovascular hospitalizations and death; (3) death: no competing risks; and (4) cardiovascular hospitalization or death: noncardiovascular hospitalizations.

Per 1‐SD increase in the MAGGIC risk score.

Figure 2

Kaplan‐Meier curves of event‐free survival stratified by tertiles of the Meta‐Analysis Global Group in Chronic Heart Failure (MAGGIC) risk score. The MAGGIC risk score was significantly associated with mortality and with each of the clinical morbidity end points, including cardiovascular‐related hospitalization, heart failure (HF) hospitalization, and the combined end point of cardiovascular (CV) hospitalization and mortality (log‐rank P<0.001 for all survival curves). MAGGIC risk scores per tertile: tertile 1, 2 to 13; tertile 2, 14 to 20; and tertile 3, 21 to 38.

Figure 3

Cumulative incidence function curves of event‐free survival stratified by tertiles of the Meta‐Analysis Global Group in Chronic Heart Failure (MAGGIC) risk score. The MAGGIC risk score was significantly associated with mortality and with each of the clinical morbidity end points, including cardiovascular (CV) related hospitalization, heart failure (HF) hospitalization, and the combined end point of cardiovascular hospitalization and mortality (P<0.001 for all cumulative incidence curves). MAGGIC risk scores per tertile: tertile 1, 2 to 13; tertile 2, 14 to 20; and tertile 3, 21 to 38.

In an effort to compare the discrimination ability of the SHFM with MAGGIC, we restricted our analyses to only those patients who had complete data for both models (n=350 patients). Receiver‐operating characteristic analyses, as demonstrated by calculating the C‐statistic for each model, showed that MAGGIC and SHFM (and BNP alone) performed similarly for all clinical outcomes of both morbidity and mortality (Table 6). Next, we analyzed calibration plots for each of the 4 outcomes (death, HF hospitalization, cardiovascular hospitalization, and the combined end point of death and/or cardiovascular hospitalization). As shown in Figures 4 and 5, we found that both MAGGIC and SHFM were generally well calibrated for the mortality outcome. However, MAGGIC was better calibrated for the hospitalization outcomes, whereas SHFM was not (particularly at the lower and higher ends of the SHFM risk score spectrum). Finally, we examined whether adding BNP to the MAGGIC risk score improved risk prediction. Although BNP and the MAGGIC risk score were statistically significantly correlated (P<0.0001), the correlation was low (R 2=0.072), suggesting that these 2 risk markers are orthogonal predictors. Indeed, BNP was additive to the MAGGIC risk score on Cox regression analysis, with the combined end point as the outcome (P=0.002 by likelihood ratio test for the combination of BNP+MAGGIC versus MAGGIC alone). In addition, event‐free survival rates in HFpEF stratified by median MAGGIC scores and median BNP values showed highest survival in those with lower MAGGIC and BNP values, respectively, and lowest survival when the converse was true (log‐rank P<0.0001; Figure 6).

Table 6

Comparison of the Receiver‐Operating Curve Statistics for MAGGIC Versus SHFM for the Outcomes of Death, Cardiovascular Hospitalization, HF Hospitalization, and Combined Cardiovascular Hospitalization and Deatha

Outcome	C‐Statistic (95% CI)b
	MAGGIC	SHFM	BNP
Death	0.74 (0.68–0.80)	0.72 (0.67–0.78)	0.76 (0.70–0.81)
Cardiovascular hospitalization	0.66 (0.60–0.71)	0.64 (0.59–0.70)	0.66 (0.60–0.72)
HF hospitalization	0.64 (0.58–0.69)	0.67 (0.61–0.72)	0.66 (0.60–0.72)
Combined cardiovascular hospitalization and death	0.72 (0.67–0.77)	0.70 (0.65–0.76)	0.74 (0.68–0.79)

BNP indicates B‐type natriuretic peptide; CI, confidence interval; HF, heart failure; MAGGIC, Meta‐Analysis Global Group in Chronic Heart Failure; SHFM, Seattle Heart Failure Model.

Analyses performed in n=350 patients who had available data to calculate scores from both models.

All comparisons between C‐statistics were nonsignificant (P>0.05 for all comparisons).

Figure 4

Calibration plots for the Meta‐Analysis Global Group in Chronic Heart Failure (MAGGIC) risk score (predicted vs actual probabilities). CV indicates cardiovascular; HF, heart failure.

Figure 5

Calibration plots for the Seattle Heart Failure Model (SHFM) risk score (predicted vs actual probabilities). CV indicates cardiovascular; HF, heart failure.

Figure 6

Event‐free survival rates of patients with heart failure with preserved ejection fraction stratified according to median Meta‐Analysis Global Group in Chronic Heart Failure (MAGGIC) score and B‐type natriuretic peptide (BNP) value. BNP is additive to the MAGGIC risk score on the basis of stratification of patients by median MAGGIC and BNP scores. Log‐rank P<0.001 for the entire survival curve. On Cox proportional hazards analysis, each of the 4 strata is statistically significantly different from the others (P<0.05), except for the comparison of the middle 2 groups (P=0.076). CV indicates cardiovascular.

Discussion

This is the first study to evaluate the utility of the MAGGIC risk model to predict morbidity outcomes (cardiovascular and HF hospitalization), and further validates the MAGGIC risk score's ability to predict mortality, in a racially diverse, well‐defined, systematic cohort of patients with HFpEF. We found that the MAGGIC risk score can be used to reliably predict cardiovascular and HF hospitalizations in addition to mortality in patients with HFpEF. More important, unlike prior studies that have examined the MAGGIC risk score in registries and electronic health record studies of patients with HF,10, 11 our study, although smaller in size, included only patients with HFpEF in whom the diagnosis and outcomes were carefully adjudicated. Furthermore, our study contained complete data for calculation of the MAGGIC risk score and, therefore, did not require imputation. Finally, we have shown that the MAGGIC risk model's discrimination ability performs similarly to the SHFM (although MAGGIC was better calibrated for hospitalization outcomes), and the addition of BNP to the MAGGIC model significantly improves its prognostic ability. Taken together, these findings have the potential to favorably affect the future evaluation and management of patients with HFpEF by doing the following: (1) improving the prognostic ability in identifying those patients at highest risk for adverse clinical outcomes; (2) translating such risk stratification into more appropriate intensification of treatments and resource allocation to the highest‐risk patients with HFpEF; and (3) informing future HFpEF clinical trial design.

HFpEF Risk Stratification

At the most basic level, risk modeling may play an important role in facilitating patient and provider understanding of likely outcomes, the prediction of which is suboptimal when based on clinician assessment alone.19 Numerous HF risk models have been developed over the years, yet only 6 have ever been validated outside of their original derivation cohort.20 Most important, most of these HF risk models evaluated patients with HFrEF or in “all comers” with HF and, thus, may have questionable applicability to patients with HFpEF. External validation of risk models is critical to determine generalizability and to confirm level of performance of the model. Although 2 prior studies have validated the MAGGIC risk score in large databases of patients with HF of the full spectrum of EF,10, 11 the present study is the first to externally validate an HF risk model (MAGGIC) specifically in HFpEF, and it does so using a more contemporary EF cutoff of >50%. In addition, we now show the utility of the MAGGIC risk score for the prediction of cardiovascular and HF hospitalization.

There was rationale behind our decision to compare the MAGGIC model with the SHFM in particular. Despite being a model derived from, and frequently applied to patients with HFrEF, the SHFM is widely used, has a website‐based accessible risk calculator, and has been extensively externally validated in several HFrEF cohorts, unlike most other HF risk models.20 We demonstrate that, similar to MAGGIC, the SHFM is useful in predicting both morbidity and mortality in HFpEF and, thus, either one could be considered reasonable to use for risk stratification for mortality in this patient population. However, an important limitation of the SHFM is that it contains predictor variables that may not be routinely available in the clinical setting (eg, percentage lymphocyte count and uric acid levels), and it was not well calibrated to hospitalization outcomes. Given the availability of an online risk calculator, both MAGGIC and SHFM are likely to be practical and easy to use. Thus, taken together, the nearly universally available variables found in the MAGGIC risk model coupled with its easily and readily accessible calculator make it attractive for incorporation into the evaluation and management of patients with HFpEF.

The MAGGIC Potential

Although a well‐validated risk model, such as MAGGIC, is of utility in the clinical setting to assist in prognostication and intensification of resources in the higher‐risk strata, an equally important application may be in the clinical research arena. Specifically, because of the heterogeneous nature of patients with HFpEF,4 the considerable morbidity and mortality associated with HFpEF,21, 22 and the lack of successful, highly effective HFpEF therapeutic clinical trials to date, a risk model such as MAGGIC has the potential to inform future clinical trial design.

Currently, although causes of death are noncardiovascular in many patients with HFpEF23 and quality of life is an important factor in their lives,1 large phase 3 clinical trials in HFpEF continue to focus on hard end points (such as HF hospitalization and cardiovascular death). Therefore, these trials typically will use criteria such as elevated BNP or recent HF hospitalization as inclusion criteria to ensure a high enough event rate to be able to detect differences between active treatment and placebo groups.24 However, up to one third of patients with HFpEF may not have elevated BNP levels,25 and many high‐risk patients with HFpEF may not meet HF hospitalization criteria. Furthermore, there may be some patients with HFpEF who are too high risk to benefit from therapies being tested in clinical trials. For these reasons, the MAGGIC risk score could be used as an additional criterion to ensure adequate (but not too high) event rates when planning for and enrolling in clinical trials. In addition, the baseline MAGGIC risk score could be calculated continuously as patients are enrolled and accrued into phase 3 HFpEF trials, with modification of enrollment criteria if the enrolled patients are deemed to low or high risk on the basis of the MAGGIC risk score in an adaptive trial design framework.26

Strengths and Limitations

Strengths of our study include the application of MAGGIC to a diverse cohort of patients with HFpEF who were prospectively enrolled and systematically studied (with adjudicated HFpEF diagnoses and outcomes); the inclusion of morbidity outcomes, such as HF and cardiovascular hospitalization; and the comparison to the SHFM risk score. Limitations include the lack of complete data to calculate SHFM in all patients; and the single‐center academic medical center nature of our study. However, all components of the MAGGIC risk score were available, and the patients in our cohort with HFpEF are more racially diverse than those in multicenter HFpEF epidemiologic studies and clinical trials; yet, clinical characteristics were otherwise similar between our cohort and these other studies. Therefore, we believe that our findings are applicable to HFpEF in general.

Conclusions

The MAGGIC risk model has good discriminatory power and calibration for the prediction of both morbidity and mortality in patients with HFpEF. Although we demonstrated that the SHFM is also able to predict outcomes in HFpEF, the nearly universal availability of the variables in MAGGIC coupled with its easy‐to‐use calculator makes the MAGGIC risk model particularly practical for everyday clinical use and applications to clinical trials. Future studies that use risk models such as MAGGIC in the design and selection of patients for HFpEF clinical trials may be an important element in improving the track record of HFpEF therapies.

Sources of Funding

This work was supported by an American Heart Association Scientist Development Grant (0835488N) and the National Institutes of Health (R01 HL107577 and R01 HL127028) to Shah.

Disclosures

None.