Predicting 30-Day Hospital Readmissions in Acute Myocardial Infarction: The AMI "READMITS" (Renal Function, Elevated Brain Natriuretic Peptide, Age, Diabetes Mellitus, Nonmale Sex, Intervention with Timely Percutaneous Coronary Intervention, and Low Systolic Blood Pressure) Score.

BACKGROUND: Readmissions after hospitalization for acute myocardial infarction (AMI) are common. However, the few currently available AMI readmission risk prediction models have poor-to-modest predictive ability and are not readily actionable in real time. We sought to develop an actionable and accurate AMI readmission risk prediction model to identify high-risk patients as early as possible during hospitalization. METHODS AND
RESULTS: We used electronic health record data from consecutive AMI hospitalizations from 6 hospitals in north Texas from 2009 to 2010 to derive and validate models predicting all-cause nonelective 30-day readmissions, using stepwise backward selection and 5-fold cross-validation. Of 826 patients hospitalized with AMI, 13% had a 30-day readmission. The first-day AMI model (the AMI "READMITS" score) included 7 predictors: renal function, elevated brain natriuretic peptide, age, diabetes mellitus, nonmale sex, intervention with timely percutaneous coronary intervention, and low systolic blood pressure, had an optimism-corrected C-statistic of 0.73 (95% confidence interval, 0.71-0.74) and was well calibrated. The full-stay AMI model, which included 3 additional predictors (use of intravenous diuretics, anemia on discharge, and discharge to postacute care), had an optimism-corrected C-statistic of 0.75 (95% confidence interval, 0.74-0.76) with minimally improved net reclassification and calibration. Both AMI models outperformed corresponding multicondition readmission models.
CONCLUSIONS: The parsimonious AMI READMITS score enables early prospective identification of high-risk AMI patients for targeted readmissions reduction interventions within the first 24 hours of hospitalization. A full-stay AMI readmission model only modestly outperformed the AMI READMITS score in terms of discrimination, but surprisingly did not meaningfully improve reclassification.

Clinical Perspective

What Is New?

Among current readmission risk prediction models for acute myocardial infarction, the acute myocardial infarction READMITS score (renal function, elevated brain natriuretic peptide, age, diabetes mellitus, nonmale sex, intervention with timely percutaneous coronary intervention, and low systolic blood pressure) is the best at identifying patients at high risk for 30‐day hospital readmission; is easy to implement in clinical settings; and provides actionable data in real time.

What Are the Clinical Implications?

The acute myocardial infarction READMITS score can be used by clinicians at bedside to accurately predict which patients hospitalized with acute myocardial infarction are at high risk for readmissions within the first 24 hours of admission, allowing for targeted readmission reduction interventions.

Hospital readmissions after acute myocardial infarction (AMI) are frequent, costly, and potentially avoidable.1, 2, 3, 4 Nearly 1 in 6 patients with AMI have an unplanned readmission within 30 days of discharge, accounting for over $1 billion of annual healthcare costs.1, 2 Since 2012, hospitals have been subject to financial penalties for excessive 30‐day readmissions among patients hospitalized for AMI under the Hospital Readmissions Reduction Program, implemented by the Centers for Medicare and Medicaid Services (CMS). Although federal readmission penalties have incentivized readmissions reduction intervention strategies (known as transitional care interventions), these interventions are resource intensive, are most effective when implemented well before discharge, and have been only modestly successful when applied indiscriminately to all inpatients.5, 6, 7, 8

Predicting which patients with AMI are at highest risk for readmission would enable both clinicians and hospitals to proactively identify and target patients who are the most likely to benefit from intensive readmission prevention interventions, simultaneously optimizing the allocation of scarce intervention resources and maximizing the potential for a successful and sustainable intervention.9, 10 Head‐to‐head comparisons of multicondition versus disease‐specific readmission risk prediction models suggest that disease‐specific models are superior.11 However, a recent systematic review of AMI‐specific readmission models found that current models have only modest predictive ability and with uncertain generalizability because of methodological limitations.12 Furthermore, few existing AMI‐specific models have the potential to provide actionable data early during a patient's hospital course, which is the optimal time to initiate interventions to maximize effectiveness.5, 8, 12

Thus, the objectives of this study were (1) to create a pragmatic, actionable, and accurate prediction model to identify patients with AMI at high risk for 30‐day readmission as early as possible during hospitalization (ie, on the first day); (2) to assess whether including clinical data from the full hospital stay would meaningfully improve model performance compared with using data only from the first day of hospitalization; and (3) compare our AMI models to other published readmissions models.13, 14, 15

Methods

Study Design, Population, and Data Sources

The data, analytic methods, and study materials will not be made available to other researchers for purposes of reproducing the results or replicating the procedure because of the terms of our data use agreements. We conducted a retrospective observational cohort study using electronic health record (EHR) data routinely collected as part of clinical care from 6 diverse hospitals with percutaneous coronary intervention capabilities located in the Dallas‐Fort Worth Metroplex in north Texas from 2009 to 2010, including safety net, community, teaching, and nonteaching hospitals. All hospitals used the Epic EHR (Epic Systems Corporation, Verona, WI). Details of this cohort have been previously published.11, 14, 15, 16, 17 We used data from 2009 to 2010, before hospital‐based readmission interventions became widespread, to ensure that AMI cohorts across all 6 hospitals were comparable. (Although penalties under the CMS Hospital Readmissions Reduction Program were not administered until 2012, many hospitals across the country including in our region began implementing interventions in 2010 after the Patient Protection and Affordable Care Act was signed into law.18)

We included consecutive hospitalizations among adults ≥18 years old discharged with a principal diagnosis of AMI (International Classification of Diseases Ninth Revision, Clinical Modification [ICD‐9‐CM] codes 410.xx, excluding 410.x2 for subsequent episode of care for AMI), consistent with the definition used by CMS for the Hospital Readmissions Reduction Program.19 For individuals with multiple hospitalizations during the study period, we included only the first hospitalization. We excluded individuals who were transferred to another acute care facility, left against medical advice, who died during hospitalization or within 30 days of discharge, or who did not have any abnormal troponin values during hospitalization.

Outcomes

The primary outcome was all‐cause 30‐day hospital readmission, defined as a nonelective hospitalization within 30 days of discharge to any of 75 acute care hospitals within a 100‐mile radius of Dallas, ascertained from an all‐payer regional hospitalization database.

Candidate Predictors

We included all variables from our previously published multicondition EHR readmission models as candidate predictors, including sociodemographics, prior utilization, Charlson comorbidity index, select laboratory and vital sign abnormalities, length of stay, hospital complications (eg, venous thromboembolism), and disposition status.11, 14, 15 Laboratories and vital signs were categorized and/or dichotomized based on cut points identified in previous studies.11, 14, 15 We also assessed additional variables specific to AMI for inclusion that met the following criteria: (1) available in the EHR of all participating hospitals; (2) routinely collected or available at the time of admission or discharge; and (3) plausible predictors of adverse outcomes based on prior literature and clinical expertise of our multidisciplinary research team. These included select comorbidities such as coronary artery disease, depression, diabetes mellitus, hypertension or chronic kidney disease; AMI‐related severity of illness on admission (ie, any of the following occurring within the first 24 hours of admission: presence of heart strain defined as elevated brain natriuretic peptide [BNP] serum level, presence of shock defined as systolic blood pressure ≤100 mm Hg, ST‐elevation myocardial infarction [ICD‐9 codes 410.x, excluding 410.7 for non–ST‐segment–elevation myocardial infarction], elevated troponin level, and transfer to the critical care or intensive care unit); in‐hospital complications and procedures (ie, use of intravenous diuretics as a proxy for acute decompensated heart failure; undergoing coronary artery bypass grafting after the first 24 hours; receipt of blood transfusion as a proxy for potential bleeding complications, since diagnosis codes for such complications were infrequently documented); and the presence of laboratory and vital sign abnormalities within 24 hours of discharge.

Statistical Analysis

Model derivation

We developed 2 separate AMI‐specific models: 1 incorporating data from only the first 24 hours of hospitalization, termed the “first‐day” AMI model, and a second model incorporating data from the full hospital stay, termed the “full‐stay” AMI model. We classified candidate predictors as available either within 24 hours of admission, or by the time of discharge. For example, sociodemographic factors could be ascertained within the first 24 hours of hospitalization, whereas length of stay would not be available until discharge. Clinical predictors with missing values (ie, comorbidities, laboratory values) were assumed to be either not present (for comorbidities) or normal (for laboratory values). Nonclinical predictors such as sociodemographic characteristics, prior utilization, and disposition status as well as vital signs had very few missing values (<1% for each variable). Data on laboratory values were missing for <2% of subjects, aside from brain natriuretic peptide levels (addressed below). We assessed univariate relationships between readmission and each candidate predictor using a prespecified significance threshold of P≤0.20.

Because of the use of both BNP and N‐terminal pro‐B‐type natriuretic peptide (NT‐proBNP) across hospital sites, we categorized natriuretic peptide levels as follows: low=BNP <50 pg/mL or NT‐proBNP <125 pg/mL; moderate=BNP 51 to 99 pg/mL or NT‐proBNP 125 to 299 pg/mL; high=BNP 100 to 999 pg/mL or NT‐proBNP 300 to 4999 pg/mL, and extremely high BNP ≥1000 pg/mL or NT‐proBNP ≥5000 pg/mL. Values for BNP and/or NT‐proBNP were not present for 39% of individuals; these were imputed as “normal/not elevated.” Similarly, because of the use of both troponin I and several different assays for troponin T across hospitals, we transformed troponin into an ordinal variable, defined as multiples of the upper limit of normal, using each hospital's specified reference values. Because neither of these approaches yielded improvement in model performance, we dichotomized both variables to maximize parsimony, using BNP ≥50 pg/mL or NT‐proBNP ≥125 pg/mL to define “elevated” BNP and >10 times the upper limit of normal to define “elevated” troponin.

Significant univariate candidate variables were entered in respective first‐day and full‐stay AMI‐specific multivariable logistic regression models using stepwise backward selection with a prespecified significance threshold of P≤0.10. In sensitivity analyses, we alternatively derived our models using stepwise forward selection using a significance threshold of P≤0.10, as well as stepwise backward selection minimizing the Bayesian Information Criterion and Akaike Information Criterion separately, and also derived models for the composite outcome of both 30‐day readmissions and mortality in addition to readmissions only. These alternate modeling strategies yielded models with predictors and effect sizes nearly identical to those in our final models (data not shown).

Rationale for approach to “missing” data

Our aim was to create a pragmatic prediction model to identify patients with AMI at high risk for 30‐day readmission using readily available real‐time clinical data from the EHR. Because our models were based on existing data collected as part of clinical care (ie, not research or registry data), the rationale for our approach to treating “missing” data on comorbidities and laboratory data (including BNP and NT‐proBNP) as equivalent to “not present/normal” was based on our understanding of clinical documentation workflows, which are largely governed by the concept of “documentation by exception.” This refers to the phenomenon that documentation of comorbidities and laboratory values in the EHR typically only occurs when there is an exception to the expectation that these are not present. For example, “diabetes mellitus” is commonly documented but “diabetes mellitus not present” is rarely documented in medical records used for clinical care. Thus, lack of explicit documentation of diabetes mellitus is highly likely to indicate that diabetes mellitus is in fact not present. Additionally, lack of explicit documentation (ie, “missing” data) on comorbidities in medical records is not likely to be random. This is in contrast to research and/or registry data, where the presence of comorbidities of interest such as diabetes mellitus is more likely to be consistently ascertained across subjects and clearly documented as either “present” or “not present,” and thus a missing value would not necessarily be considered equivalent to “not present.”

Similarly, with respect to BNP and NT‐proBNP, in clinical practice, physicians typically only order these tests in the presence of signs, symptoms, or history that raise clinical suspicion for myocardial strain or heart failure—ie, this is a laboratory test that they would “order by exception,” to parallel the concept of “documentation by exception.” Conversely, the lack of a BNP value implies that the treating physician did not have a concern for a new clinical abnormality, which is valuable clinical information to take into consideration in an EHR‐based prediction model. Thus, we thought it was reasonable to assume that patients who did not have an NT‐proBNP were deemed to be at lower risk for myocardial strain and/or heart failure by treating physicians.

Because the purpose of our study was to develop a pragmatic model based on available clinical data, and because data on comorbidities and laboratory tests such as BNP and NT‐proBNP were not missing at random as described above, we did not apply multiple imputation to impute missing values for our cohort. Our approach is consistent with the approach used in the development of the CMS AMI model, which is based on the presence of comorbidities coded in administrative claims data, and is also the same approach we have used in our past studies on readmission risk prediction modeling.11, 13, 14, 15

Model validation

We validated both the AMI‐specific models using 5‐fold cross‐validation, randomly dividing the cohort into 5 equally sized subsets.20 For each cycle, 4 subsets were used for training to estimate model coefficients and the fifth was used for validation. This cycle was repeated 5 times such that each of the 5 subsets was used once as the validation set. We then repeated this entire process 50 times and averaged the C‐statistic estimates to derive an optimism‐corrected C‐statistic. We qualitatively assessed calibration by comparing observed to predicted probabilities of readmission by quintiles of predicted risk, and with the Hosmer‐Lemeshow goodness‐of‐fit test.21, 22

Scoring system

We derived a point‐based risk scoring system for our final first‐day AMI model. We assigned points to each variable by dividing each β‐coefficient by the lowest overall β‐coefficient and rounding to the nearest integer. We determined point cutoffs to define quintiles of predicted risk and assessed calibration separately from that of the corresponding logistic regression equation model.

Model comparisons

We compared the first‐day and full‐stay AMI models to each other as well as to the corresponding multicondition EHR models our group has separately developed and the CMS AMI model derived from administrative claims data.13, 14, 15 We compared each existing model's performance using the C‐statistic, integrated discrimination index, and net reclassification index (NRI) using the AMI‐specific models as references.23 The integrated discrimination index is defined as the difference in the mean predicted probability of readmission between patients who were and were not actually readmitted between 2 models, where more positive values suggest improvement in model performance compared with a reference model.24 The NRI is defined as the sum of the net proportions of correctly reclassified people with and without the event of interest compared with a reference model.24, 25 Here, we calculated a category‐based NRI to evaluate the performance of AMI‐specific models in correctly reclassifying individuals with and without readmissions into the highest readmission risk quintiles versus the lowest 4 risk quintiles compared with other models. This prespecified cutoff is relevant for hospitals interested in identifying the highest risk individuals for targeted intervention.10 Finally, we assessed calibration of comparator models in our cohort. We conducted analyses using Stata 12.1 (StataCorp, College Station, TX). The UT Southwestern institutional review board reviewed and approved this study with a waiver of informed consent.

Results

Of 826 index AMI hospitalizations, 13.0% had a 30‐day readmission. Individuals with a readmission had markedly different sociodemographic and clinical characteristics compared with those who were not readmitted (Table 1). Troponin values were similar among patients with and without a readmission. ST‐segment–elevation myocardial infarction was less common among those who were readmitted (21.2% versus 26.2%, P=0.30, Table 1).

Table 1

Characteristics of Individuals Hospitalized with AMI

	No Readmission (N=719)	Readmission (N=107)	P Value
Sociodemographic characteristics
Age (y) ±SD	64.8±12.8	70.5±13.0	<0.001
Female, n (%)	237 (33.0)	51 (47.7)	0.003
Race/ethnicity			0.44
White	496 (69.0)	82 (76.6)
Black	90 (12.5)	11 (10.3)
Hispanic	103 (14.3)	11 (10.3)
Other	30 (4.2)	3 (2.8)
Primary payer, n (%)			0.001
Private	3431 (46.0)	36 (33.6)
Medicare	243 (33.8)	58 (54.2)
Medicaid	29 (4.0)	3 (2.8)
Charity, self‐pay, or other	116 (16.1)	10 (9.4)
Median income per ZIP code <$30 000, n (%)	58 (8.1)	11 (10.3)	0.44
Utilization history
≥1 ED visit in past y, n (%)a	118 (16.4)	26 (24.3)	0.05
≥1 hospitalization in past y, n (%)b	135 (18.8)	33 (30.8)	0.004
Clinical factors from first 24 h
Nonelective admission, n (%)c	690 (96.0)	102 (95.3)	0.76
Clinical comorbidities, n (%)
Charlson comorbidity scored			0.005
0	593 (82.5)	74 (69.2)
1	29 (4.0)	7 (6.5)
2+	97 (13.5)	26 (24.3)
History of coronary artery disease	85 (11.8)	20 (18.7)	0.05
History of depression	18 (2.5)	7 (6.5)	0.02
History of diabetes mellitus	60 (8.3)	23 (21.5)	<0.001
History of hypertension	89 (12.4)	20 (18.7)	0.07
History of chronic kidney disease	45 (6.3)	20 (18.7)	<0.001
Severity of illness measures in first 24 h, n (%)
Systolic blood pressure ≤100 mm Hg	373 (51.9)	72 (67.3)	0.003
PCI within first 24 h	341 (47.3)	31 (29.0)	<0.001
ST‐elevation myocardial infarctione	188 (26.2)	23 (21.5)	0.30
Laboratory results in first 24 h, n (%)
Elevated BNP in first 24 hf	282 (39.2)	57 (53.2)	<0.01
Elevated troponin in first 24 hg	537 (74.7)	83 (77.6)	0.52
Creatinine ≥2 mg/dL	63 (8.8)	34 (31.8)	<0.001
Clinical factors from full hospital stay
Length of hospital stay, median (IQR)	3 (2–5)	5 (3–8)	<0.001
In‐hospital treatments and procedures
Underwent coronary artery bypass grafting	57 (7.9)	14 (13.1)	0.08
Use of intravenous diuretics (any)	229 (31.9)	58 (54.2)	<0.001
AMI‐specific hospital complications, n (%)h
Congestive heart failure (new)	222 (30.9)	52 (48.6)	<0.001
Shock (any)	24 (3.3)	9 (8.4)	0.01
Discharge to post‐acute carei	55 (7.7)	21 (19.6)	<0.001

AMI indicates acute myocardial infarction; BNP, brain natriuretic peptide; ED, emergency department; ICD‐9‐CM, International Classification of Diseases, Ninth Revision, Clinical Modification; IQR, interquartile range; NT‐proBNP, N‐terminus pro‐brain natriuretic peptide; PCI, percutaneous coronary intervention.

Prior ED visit at site of index hospitalization within the past year.

Prior hospitalization at any of 75 acute care hospitals in the north Texas region within the past year.

Nonelective admission defined as hospitalization categorized as medical emergency, urgent, or trauma.

Calculated from diagnoses available within 1 year before index hospitalization.

Defined using ICD‐9‐CM codes 410.xx, excluding 410.7x and 410.2x.

Defined as ≥50 pg/mL for BNP or ≥125 pg/mL for NT‐proBNP.

Included both troponin T and troponin I assays; defined as ≥10 times the upper limit of normal, using site‐specific criteria to define upper limit of normal for each of the included hospitals.

Conditions were considered complications if they were not listed as a principal diagnosis for hospitalization or as a previous diagnosis in the prior year.

Discharges to nursing home, skilled nursing facility, or long‐term acute care hospital.

Performance of a First‐Day AMI Model: the AMI “READMITS” Score

Our final first‐day model, termed the AMI “READMITS” score, included 7 variables: renal function (serum creatinine >2 mg/dL); elevated BNP; age (per decade >18 years); diabetes mellitus history; not male, no intervention with timely percutaneous coronary intervention; and systolic blood pressure <100 mm Hg (Table 2). The AMI READMITS score had good discrimination (C‐statistic 0.75, 95% confidence interval [CI], 0.70–0.80; optimism‐corrected C‐statistic 0.73, 95% CI, 0.71–0.74; Table 3). It also effectively stratified individuals across a broad range of risk (average predicted risk by decile ranged from 2.1% to 41.1%) and was well calibrated, with less than a 2% difference between mean predicted and observed readmission rate by quintiles (Table 4). Approximately one third of patients predicted to be at high risk (AMI READMITS score ≥20) had an observed 30‐day readmission versus only 2% of patients who were predicted to be at low risk (AMI READMITS score ≤13).

Table 2

Final AMI Risk Prediction Models for 30‐Day Readmission

	Adjusted OR (95% CI)a	Pointsb
AMI READMITS score (first‐day model)
Renal function (Cr >2 mg/dL)	3.95 (2.52–6.08)	6
Elevated BNPc	6.36 (1.65–24.47)	8
Age (per decade >18 y)	1.26 (0.98–1.61)	1
Diabetes mellitus history	2.41 (1.37–4.24)	4
Not Male (ie, female sex)	1.53 (0.92–2.57)	2
No Intervention with Timely PCId	1.31 (1.02–1.69)	1
Systolic blood pressure <100 mm Hg	2.18 (1.68–2.82)	3
Full‐stay AMI model
Renal function (Cr >2 mg/dL)	3.32 (2.18–5.07)
Elevated BNPc	6.65 (1.80–24.61)
Age (per decade >18 y)	1.14 (0.87–1.50)
Diabetes mellitus history	2.22 (1.16–4.25)
Not Male (ie, female sex)	1.40 (0.85–2.33)
No Intervention with Timely PCId	1.19 (0.91–1.54)
Systolic blood pressure <100 mm Hg	2.19 (1.77–‐2.72)
Use of any intravenous diuretics	1.58 (1.07–2.31)
Anemia on discharge (hematocrit ≤33%)	2.04 (1.20–3.46)
Discharge to post–acute care facility	1.50 (0.90–2.50)

AMI indicates acute myocardial infarction; BNP, brain natriuretic peptide; CI, confidence interval; Cr, serum creatinine; NT‐proBNP, N‐terminal pro‐brain natriuretic peptide; OR, odds ratio; PCI, percutaneous coronary intervention.

Adjusted for all factors listed in the table for each respective model.

We assigned points to each variable by dividing each β‐coefficient by the lowest overall β‐coefficient and rounding to the nearest integer.

Defined as ≥50 pg/mL for BNP or ≥125 pg/mL for NT‐proBNP.

Defined as intervention within the first 24 h.

Table 3

Model Performance and Comparison of AMI Readmission Models Versus Other Models

Model	C‐Statistic	P Valuea	IDI, % (95% CI)	NRI (95% CI)b	Average Predicted Risk, %
					Lowest Decile	Highest Decile
AMI modelsc
AMI READMITS score	0.75 (0.70 to 0.80)	[Reference]	[Reference]	[Reference]	2.1	41.1
Full‐stay AMI model	0.78 (0.74 to 0.83)	<0.01	0.02 (0.01−0.04)	0.03 (−0.04 to 0.10)	1.6	43.9
CMS AMI model	0.74 (0.69 to .79)	0.57	−0.07 (−0.10 to −0.05)	−0.01 (−0.11 to 0.09)	7.2	24.3
Multicondition modelsc
First‐day multicondition	0.70 (0.65 to 0.75)	0.04	−0.08 (−0.11 to −0.05)	−0.19 (−0.30 to −0.18)	6.6	25.7
Full‐stay multicondition	0.73 (0.68 to 0.78)	0.02	−0.06 (−0.09 to −0.03)	−0.09 (−0.20 to 0.03)	5.7	29.6

AMI indicates acute myocardial infarction; CI, confidence interval; CMS, Centers for Medicare and Medicaid Services; EHR, electronic health record; IDI, integrated discrimination improvement; NRI, Net Reclassification Index; READMITS, renal function, elevated brain natriuretic peptide, age, diabetes mellitus, nonmale sex, intervention with timely percutaneous coronary intervention, and low systolic blood pressure.

P values are shown for each model compared with the reference model.

The categorical NRI compares reclassification between the highest risk quintiles and the lowest 4 risk quintiles.

We compared the first‐day (AMI READMITS) and full‐stay AMI models with each other as well as with the corresponding multicondition EHR models our group has separately developed and the CMS AMI model derived from administrative claims data, as described in the Methods section.

Table 4

Observed vs Predicted Readmission Risk for the AMI READMITS Score

Risk Category (Quintiles)	AMI READMITS Score (Points)	Mean Observed Risk	Mean Predicted Risk
Extremely high	≥20	34%	35%
High	18 to 19	17%	16%
Moderate	16 to 17	9%	11%
Low	14 to 15	8%	7%
Extremely low	≤13	2%	3%

AMI indicates acute myocardial infarction; READMITS, renal function,elevated brain natriuretic peptide, age, diabetes mellitus, nonmale sex, intervention with timely percutaneous coronary intervention, and low systolic blood pressure.

Performance of a Full‐Stay AMI Model

Our final full‐stay AMI model included 10 variables, including all 7 predictors from the first‐day model, and 3 additional predictors available at discharge: use of intravenous diuretic medications at least once during hospitalization, anemia on discharge, and discharge to a post–acute care facility. The full‐stay AMI model also had good discrimination (C‐statistic 0.78, 95% CI, 0.74–0.83; optimism‐corrected C‐statistic 0.75, 95% CI, 0.74–0.76), stratified individuals across a broad range of risk (with average predicted risk by decile ranging from 1.6% to 43.9%; Table 3), and was well calibrated (Figure).

Figure 1

Comparison of model calibration. AMI indicates acute myocardial infarction; CMS, Centers for Medicare and Medicaid Services; READMITS, renal function, elevated brain natriuretic peptide, age, diabetes mellitus, nonmale sex, intervention with timely percutaneous coronary intervention, and low systolic blood pressure.

AMI READMITS Score Versus the Full‐Stay AMI Model

Although the full‐stay AMI model had modestly better discrimination than the first‐day AMI READMITS score (P=0.001 for comparison), it did not meaningfully improve net reclassification (NRI 0.04, 95% CI, −0.03 to 0.11; Table 3). The AMI READMITS score and the full‐stay AMI model were similarly well calibrated, with modest overestimation of predicted risk in the highest and lowest risk quintiles by both models (Figure).

AMI READMITS Score Versus Other Models

The AMI READMITS score outperformed the first‐day multicondition model with better discrimination (C‐statistic 0.75 versus 0.70, P=0.04) and had improved net reclassification (Table 3). The AMI READMITS score was better calibrated than the first‐day multicondition EHR model, which overestimated risk in the lower 3 quintiles and underestimated risk in the top 2 quintile risk groups (Figure).

Compared with the CMS AMI administrative model, the AMI READMITS score had similar discrimination with no meaningful improvement in net reclassification (NRI 0.03, 95% CI, −0.07 to 0.14) (Table 3). However, the AMI READMITS score stratified individuals into a much broader range of average predicted risk (2.1–41.1% versus 7.2–24.3%, Table 3) and was better calibrated (Figure).

Discussion

Using data from 6 diverse hospitals, we developed and validated the AMI READMITS score, a parsimonious risk prediction score that can be used by clinicians and hospital systems to identify patients hospitalized with AMI at high risk for 30‐day readmission within the first 24 hours of admission. The AMI READMITS score, derived from an AMI‐specific model using EHR data from the first day, outperformed most other models—including our own multicondition EHR models—in all aspects of model performance (discrimination, calibration, and reclassification). Surprisingly, incorporating more data from the full hospital stay into the AMI READMITS score only modestly improved discrimination but did not meaningfully improve calibration or net reclassification.

The limited improvement in performance of the full‐stay AMI model compared with the first‐day AMI READMITS score suggests that in‐hospital factors such as clinical stability, trajectory during hospitalization, and disposition status are less important predictors of readmissions among patients hospitalized with AMI than in other conditions such as pneumonia.11 Thus, a key finding of our study is that patients' readmission risk can be accurately predicted with the AMI READMITS score on the first day of hospitalization, enabling targeted early intervention to maximize the potential benefit of readmission reduction interventions.5, 8 This approach can be implemented by clinicians at bedside, or by hospitals and health systems by integrating the AMI READMITS score directly into the EHR, or by extracting EHR data in near real‐time, as our group has previously done for heart failure.10

A second key finding of our study is that clinical severity measures directly related to the AMI (shock, heart strain or failure, renal dysfunction) and timely percutaneous coronary intervention were strong predictors of readmission risk. Unlike our multicondition and pneumonia‐specific readmission models, key nonclinical factors such as social factors (ie, marital status, or residing in a low‐income neighborhood), as well as in‐hospital clinical trajectory and complications (ie, changes in clinical status, vital sign abnormality on discharge) were not predictive of 30‐day readmissions in AMI.11, 14 The implications are that readmission risk in AMI may be more influenced by clinical interventions and comorbidity management than in other conditions.

Although elevated cardiac troponin levels have been previously found to be predictive of adverse cardiac events in various settings and populations,26, 27 we were surprised to find that the magnitude of troponin elevation was not different between patients who were and were not readmitted, nor was this an independent predictor of adverse 30‐day outcomes even in models predicting the composite outcome of 30‐day readmissions and mortality that we derived as part of our sensitivity analyses (data not shown). Previous studies have suggested that elevated BNP—which was included in the AMI READMITS score—rather than elevated troponin may be a better predictor of adverse events in patients with acute coronary syndrome.28 Furthermore, the downstream clinical consequences of AMI may matter more than simply the magnitude of infarction as measured through troponin level, since cardiogenic shock, heart strain, heart failure, and renal dysfunction were strongly predictive of readmission in our models.29

Another key implication of our study is that for AMI, a disease‐specific modeling approach has better predictive ability than using a multicondition approach. Compared with a first‐day multicondition model, the AMI READMITS score correctly reclassified an additional 18% of patients. Thus, hospitals interested in identifying the highest risk patients with AMI for targeted interventions should do so using the disease‐specific AMI READMITS score. Of note, another disease‐specific model, the CMS AMI administrative model, had similar discrimination but poorer calibration than the AMI READMITS score in this cohort. Additionally, the CMS AMI model is not usable in clinical settings for near real‐time risk prediction, since it is based on 31 variables ascertained from claims data not available until well after discharge.13

Our study was notable for several strengths. First, the AMI READMITS score is parsimonious and incorporates clinically relevant predictors available within the first day of hospital admission that can be easily calculated by clinicians. Second, we used routinely collected and available data from a common commercial EHR system, allowing for implementation through automation and integration directly into the EHR. Third, our study population was derived from 6 hospitals diverse in payer status, age, race/ethnicity, and socioeconomic status, increasing the potential generalizability of our findings. Fourth, our models are less likely to be overfit to the idiosyncrasies of our data given that the predictors in our final AMI‐specific models have good clinical face validity, and have been associated with adverse outcomes, particularly mortality, in prior studies of this population.28, 30, 31, 32, 33, 34, 35, 36, 37, 38, 39

Our results should be interpreted within the context of several limitations. First, generalizability to other regions or settings is unknown. Our inclusion of a large sample of diverse patients treated in 6 different hospitals (including safety net, community, teaching and nonteaching institutions) should minimize this concern. Future studies should focus on external validation of this model in other populations and settings. Second, while we used cross‐validation and optimism‐corrected estimates of the C‐statistic to reduce the risk of overfitting the AMI READMITS score to our data, the AMI READMITS score has not yet been externally validated in a separate cohort, which would further strengthen its validity. Third, we were unable to include data on medications (aspirin, β‐blockers, and angiotensin‐converting enzyme inhibitors), AMI care process measures (door‐to‐balloon time), and clinical characteristics of AMI (location, size of infarction), which may also influence readmission risk.

In conclusion, the AMI READMITS score is parsimonious, uses clinically relevant risk factors, outperformed the CMS AMI and our previous EHR multicondition readmission prediction models, and yields actionable data on the first day of hospitalization to enable early prospective identification of high‐risk AMI patients for targeted readmissions reduction interventions. The AMI READMITS score can be easily implemented by clinicians at the bedside and/or by hospitals with integration directly into the EHR for near real‐time use.

Sources of Funding

This work was supported by the Agency for Healthcare Research and Quality‐funded UT Southwestern Center for Patient‐Centered Outcomes Research (R24 HS022418‐01) and the Commonwealth Foundation (#20100323). Dr Nguyen received funding support from the UT Southwestern KL2 Scholars Program (KL2 TR001103) and the National Heart, Lung, and Blood Institute (NHLBI K23 HL13341‐01). Dr Makam received funding support from the National Institute on Aging (NIA K23 AG052603). Dr Halm was supported in part by the National Center for Advancing Translational Sciences at the National Institutes of Health (U54 RFA‐TR‐12‐006). The study sponsors had no role in the design and conduct of the study; collection, management, analysis, or interpretation of the data; or preparation, review, or approval of the article.

Disclosures

None.