Development and Validation of a Novel Risk Score for In-Hospital Major Bleeding in Acute Myocardial Infarction:-The SWEDEHEART Score.

Background Bleeding risk stratification in acute coronary syndrome is of highest clinical interest but current risk scores have limitations. We sought to develop and validate a new in-hospital bleeding risk score for patients with acute myocardial infarction. Methods and Results From the nationwide SWEDEHEART (Swedish Web-System for Enhancement and Development of Evidence-Based Care in Heart Disease Evaluated According to Recommended Therapies) register, 97,597 patients with acute myocardial infarction enrolled from 2009 until 2014 were selected. A full model with 23 predictor variables and 8 interaction terms was fitted using logistic regression. The full model was approximated by a model with 5 predictors and 1 interaction term. Calibration, discrimination, and clinical utility was evaluated and compared with the ACTION (Acute Coronary Treatment and Intervention Outcomes Network) and CRUSADE (Can Rapid Risk Stratification of Unstable Angina Patients Suppress Ad verse Outcomes With Early Implementation of the ACC /AHA Guidelines) scores. Internal and temporal validity was assessed. In-hospital major bleeding, defined as fatal, intracranial, or requiring surgery or blood transfusion, occurred in 1356 patients (1.4%). The 5 predictors in the approximate model that constituted the SWEDEHEART score were hemoglobin, age, sex, creatinine, and C-reactive protein. The ACTION and CRUSADE scores were poorly calibrated in the derivation cohort and therefore were recalibrated. The SWEDEHEART score showed higher discriminative ability than both recalibrated scores, overall ( C-index 0.80 versus 0.73/0.72) and in all predefined subgroups. Decision curve analysis demonstrated consistently positive and higher net benefit for the SWEDEHEART score compared with both recalibrated scores across all clinically relevant decision thresholds. The original ACTION and CRUSADE scores showed negative net benefit. Conclusions The 5-item SWEDEHEART score discriminates in-hospital major bleeding in patients with acute myocardial infarction and has superior model performance compared with the recalibrated ACTION and CRUSADE scores.

Clinical Perspective

What Is New?

From a population of patients with acute myocardial infarction representative of a whole nation, we developed a simple 5‐item (age, sex, hemoglobin, creatinine, and C‐reactive protein) in‐hospital bleeding risk score that showed better model performance overall and in all important subgroups where the current guideline‐recommended CRUSADE (Can Rapid Risk Stratification of Unstable Angina Patients Suppress Adverse Outcomes With Early Implementation of the ACC/AHA Guidelines) score and the ACTION (Acute Coronary Treatment and Intervention Outcomes Network) score underachieved.

Our results also demonstrate that calibration as well as discrimination is of upmost importance when evaluating model performance.

What Are the Clinical Implications?

Bleeding risk assessment by the CRUSADE score is listed as one of the main quality indicators for acute myocardial infarction care by the European Society of Cardiology (ESC). However, the ESC guidelines give use of the CRUSADE score a cautious class IIb recommendation restricted to patients undergoing coronary angiography and the American guidelines do not recommend the use of a specific risk score for bleeding.

Our score seems valuable in the early assessment of patients and furthermore our findings stress the need for validation and, if necessary, recalibration before applying a risk score to a new population.

Advances in the management of acute coronary syndrome (ACS), including the use of invasive strategies, the implementation of dual antiplatelet therapy and more efficient antithrombotic treatment, has been associated with a decline in mortality across a number of countries.1, 2, 3 Yet, such improvements come at the price of an increased risk of bleeding and associated mortality and morbidity.4, 5, 6, 7, 8 Given that harm associated with bleeding may attenuate the net benefit of antithrombotic therapies,9, 10 there is international interest in initiatives to reduce bleeding complications in the context of ACS.

Guidelines from North America and Europe emphasize that bleeding risk as well as ischemic risk should be assessed.11, 12 While the GRACE (Global Registry of Acute Coronary Events) risk score13 is advocated for estimating future ischemic risk11, 12 and has been validated in numerous studies to show good or excellent model performance14 there is a paucity of in‐hospital bleeding risk models that demonstrate good model performance. Of the ACTION (Acute Coronary Treatment and Intervention Outcomes Network),15 ACUITY (Acute Catheterization and Urgent Intervention Triage Strategy),16 and CRUSADE (Can Rapid Risk Stratification of Unstable Angina Patients Suppress Adverse Outcomes With Early Implementation of the ACC/AHA Guidelines)17 bleeding risk scores, the latter score is the most established but is limited by weak discriminative ability especially among patients with ACS managed noninvasively, the elderly, patients prescribed oral anticoagulants (OACs), and patients with renal failure (glomerular filtration rate <60 mL/min).18, 19, 20, 21 In the European Society of Cardiology (ESC) guidelines,12 the CRUSADE score has a class IIb recommendation restricted to patients receiving coronary angiography.

The aim of the present study was to develop and validate a new risk score for in‐hospital major bleeding in a population including the aforementioned vulnerable subgroups. The SWEDEHEART (Swedish Web‐System for Enhancement and Development of Evidence‐Based Care in Heart Disease Evaluated According to Recommended Therapies) registry is an established national register highly representative of the Swedish ACS population, thereby enabling generalizable patient‐specific research into the care and outcomes of patients hospitalized with ACS.

Methods

The data and study materials will not be made available to other researchers for purposes of reproducing the results or replicating the procedure. However, a detailed description of the analytical method and the full equation is provided in Data S1.

The SWEDEHEART registry has been previously described.22 Briefly, it is a national registry of patients hospitalized for suspected ACS or undergoing coronary or valve intervention. All Swedish hospitals (n=72) participate, covering ≈90% of patients with acute myocardial infarction (MI) treated in hospitals in Sweden. The registry is regularly monitored with 95% to 96% agreement with regard to key variables between the registry and electronic health records.

In order to use a contemporary and reasonably large population, we selected all consecutive patients with acute MI, defined as discharge International Classification of Diseases, 10th Revision (ICD‐10) code I21 enrolled in the SWEDEHEART registry from January 1, 2009, until October 10, 2014 (n=109 714). After excluding 11 937 readmissions, multiple imputation was performed on the remaining 97 777 index hospitalizations. Subsequently, 180 (0.2%) patients with missing outcome data were excluded, resulting in a derivation cohort of 97 597 patients.

All Swedish residents are given a unique and permanent personal identification number that allows cross‐matching between several national registries. This made it possible to enrich baseline data by merging SWEDEHEART with the National Patient Registry, which contains information on discharge ICD codes from all admissions to Swedish hospitals since 1987.

All patients are informed about their participation in the registry and that they are allowed to opt out. Written consent is not required according to Swedish law. Patients were anonymized by a unique study identification number before the data analysis. The study protocol was approved by the regional ethics committee in Stockholm. The National Board of Health and Welfare approved the merging of registries.

The outcome was in‐hospital non–coronary artery bypass graft major bleeding defined as fatal, intracranial, or bleeding requiring blood transfusion or surgery (including endoscopic and vascular intervention) according to the SWEDEHEART registry. Other bleeding events were not recorded in the registry. For patients who underwent coronary artery bypass graft surgery during the same hospital stay, bleeding was only registered if it occurred before surgery.

All laboratory data (including hemoglobin, C‐reactive protein [CRP], and creatinine) were recorded on admission to the hospital. Renal function was measured as creatinine and the glomerular filtration rate was estimated using the Chronic Kidney Disease Epidemiology Collaboration equation.23 Heart failure at presentation was defined as any pulmonary rales or pulmonary edema. Previous bleeding was defined as any hospitalization with an ICD‐10 bleeding diagnosis (listed in Table S1) before the date of index acute MI.

Statistical Analysis

A detailed description of the statistical methods including an expanded list of references is presented in Data S1. Before any model fitting, 23 predictor variables (age, sex, body weight, hypertension, diabetes mellitus, previous MI, previous percutaneous coronary artery intervention, previous coronary artery bypass graft surgery, previous stroke, peripheral artery disease, chronic heart failure, previous bleeding, single aspirin therapy, dual antiplatelet therapy, OAC therapy, cardiopulmonary resuscitation before hospital, atrial fibrillation at hospitalization, cardiogenic shock, heart failure at hospitalization, ST‐segment–elevation MI, serum creatinine, CRP, and hemoglobin) were selected based on clinical relevance. In the derivation cohort, the percentage of missing values across the predictors and the bleeding outcome varied between 0 and 12% (Table S2). In total, 25% of the records were incomplete in the derivation cohort. Assuming a missing‐at‐random mechanism, the incomplete variables across the index population were imputed under fully conditional specification using a random forest‐based algorithm,24, 25 resulting in 25 imputed sets. Convergence was reached after 30 iterations. Before any model fitting, all patients with missing outcome data were removed as recommended by Von Hippel.26 Models were then fitted to the imputed data sets and the corresponding estimates were combined into one overall estimate.27

The full model including all predictors and 8 prespecified interactions (age and sex, age and diabetes mellitus, age and serum creatinine, age and hemoglobin, sex and body weight, sex and diabetes mellitus, sex and serum creatinine, and sex and hemoglobin) was fitted to the outcome variable using logistic regression. The continuous predictors (age, body weight, serum creatinine, hemoglobin, and CRP) were truncated at the first and 99th percentiles to limit the influence of extreme values. In addition, skewed predictors (body weight and serum creatinine) were log‐transformed. All continuous predictors were modeled using restricted cubic splines to allow for nonlinear relationships. Since the full model was too complex for routine use, a simplified approximate model was developed using the stepdown method.28 The approximate model was termed the SWEDEHEART score and presented in the form of a nomogram and an equation. Risk classes were created according to <0.5%, 0.5% to 1%, 1% to 2%, 2% to 4%, and >4% predicted risk of in‐hospital major bleeding.

Given that the ACTION and CRUSADE scores were poorly calibrated in the derivation cohort (Figure 1), recalibrated scores were obtained by fitting a logistic regression model using the ACTION and CRUSADE score, respectively, as the only predictor and the binary bleeding indicator as the outcome.29

Figure 1

Calibration plots for the ACTION (Acute Coronary Treatment and Intervention Outcomes Network), recalibrated ACTION, CRUSADE (Can Rapid Risk Stratification of Unstable Angina Patients Suppress Adverse Outcomes With Early Implementation of the ACC/AHA Guidelines), recalibrated CRUSADE, and SWEDEHEART (Swedish Web‐System for Enhancement and Development of Evidence‐Based Care in Heart Disease Evaluated According to Recommended Therapies) scores. Deviations from perfect calibration (Ideal) represent bias in the predicted probabilities. For the ACTION and CRUSADE scores, the figure illustrates the observed proportion of bleeding in the derivation cohort (with 95% CIs) vs the proportion of bleeding in the previously published original derivation cohorts (Apparent). For the recalibrated ACTION and CRUSADE scores and the SWEDEHEART score, the figure illustrates the observed proportion of bleeding in the derivation cohort (with 95% CIs) vs the predicted probability of bleeding (Apparent) and its optimism‐corrected estimate (Optimism‐corrected). The apparent and optimism‐corrected calibration curves are almost indistinguishable, indicating no overfitting.

The C‐index was used to assess discrimination. Calibration was assessed graphically using smooth nonparametric regression and the corresponding CIs were obtained using 2000 bootstrap samples.30, 31 Discrimination and calibration were evaluated in the full derivation cohort and in predefined subgroups. The recalibrated ACTION and CRUSADE scores and the SWEDEHEART score were internally validated using 200 bootstrap samples. Temporal validity of the SWEDEHEART score was assessed using internal–external cross‐validation32 where each admission year was omitted in turn. The optimism (bias caused by overfitting) was estimated and optimism‐corrected performance estimates were obtained.

The clinical utility of the ACTION, CRUSADE, and SWEDEHEART scores was evaluated using decision curve analysis.33 Using a score to classify patients into low‐ and high‐risk bleeders requires a decision threshold, classifying all patients with a predicted bleeding probability above the threshold as high‐risk bleeders. Since the score is not perfect, the resulting decision will either be incorrect (a false‐positive bleed) or correct (a true‐positive bleed). The net benefit of using a score at a given threshold is defined as the difference between the proportion of true positives and the proportion of false positives weighted by the odds of the threshold, where the weight corresponds to the cost‐benefit ratio associated with incorrect and correct decisions. For a given threshold, a score is considered clinically useful if it has a positive net benefit and harmful if the net benefit is negative. For each score, the decision curve illustrates the net benefit corresponding to the decision thresholds. The score with the higher net benefit across the range of clinically relevant decision thresholds is preferred.

Methods and results are presented in line with the transparent reporting of a multivariable prediction model for individual prognosis or diagnosis (TRIPOD) statement.34 All statistical analyses were performed with R (version 3.4.3, The R Foundation for statistical computing) using the add‐on packages mice (version 2.44) and rms (version 5.1–1).

Results

Baseline Characteristics and in‐Hospital Course

Baseline characteristics on admission of patients with and without major in‐hospital bleeding are listed in Table 1. The median time to discharge was 4 days (interquartile range, 3–6 days). Major bleeding occurred in 1356 patients (1.4%), and, of these, 50 (0.1%) were fatal, 114 (0.1%) were cerebral, and 1192 (1.2%) required surgery or red blood cell transfusion. Patients with a bleeding event were older (median 77 years versus 72 years), more often female (46% versus 35%) and had an overall higher preexisting burden of comorbidity and more extensive medication, including single aspirin therapy (36.8% versus 44.4%), dual antiplatelet therapy (4.1% versus 6.3%), or OAC therapy (5.6% versus 8.0%).

Table 1

Baseline Characteristics

	No Major Bleed	Major Bleed
	(n=96 241)	(n=1356)
Demography
Age, y	72 (63–81)	77 (69–83)
Weight, kg	78 (69–90)	74 (64–85)
Women	33 779 (35.1)	621 (45.8)
Medical history
Hypertension	54 923 (57.1)	916 (67.6)
Diabetes mellitus	23 172 (24.1)	411 (30.3)
Previous MI	23 268 (24.2)	398 (29.4)
Previous PCI	13 295 (13.8)	200 (14.7)
Previous CABG	8286 (8.6)	145 (10.7)
Previous PAD	5641 (5.9)	157 (11.6)
Previous stroke	11 498 (11.9)	259 (19.1)
Chronic heart failure	13 647 (14.2)	337 (24.9)
Previous bleeding	5918 (6.1)	170 (12.5)
COPD	6986 (7.3)	142 (10.5)
Previous cancer	3176 (3.3)	91 (6.7)
Medication on admission
β‐Blocker	35 771 (37.2)	633 (46.7)
RAS blockade	59 934 (62.3)	739 (54.5)
Calcium antagonist	18 368 (19.1)	324 (23.9)
Digoxin	2052 (2.1)	48 (3.5)
Statins	28 056 (29.2)	481 (35.5)
Diuretics	23 301 (24.2)	500 (36.9)
Aspirin	35 389 (36.8)	602 (44.4)
DAPT	3933 (4.1)	86 (6.3)
OAC	5417 (5.6)	108 (8.0)
Presentation
CPR before hospital	1708 (1.8)	37 (2.7)
Atrial fibrillation	10 262 (10.7)	213 (15.7)
Heart rate, bpm	79 (66–93)	86 (70–100)
Systolic blood pressure, mm Hg	147 (130–167)	140 (120–160)
Symptoms or signs of HF	11 312 (11.8)	288 (21.2)
Shock	1214 (1.3)	36 (2.7)
ST‐elevation	31 991 (33.2)	457 (33.7)
Laboratory data on admission
Hemoglobin, g/L	139 (127–149)	116 (101–131)
Anemia WHO definition	19 729 (20.5)	818 (60.3)
Creatinine, mmol/L	84 (70–103)	99 (77–137)
GFR by CKD‐EPI, mL/min	74 (55–89)	56 (37–76)
CRP	5 (3–15)	17 (5–66)

Values are expressed as median (interquartile range) or number (percentage). bpm indicates beats per minute; CABG, coronary artery bypass graft; CKD‐EPI, Chronic Kidney Disease Epidemiology Collaboration; COPD, chronic obstructive pulmonary disease; CPR, cardiopulmonary resuscitation; CRP, C‐reactive protein; DAPT, dual antiplatelet therapy; GFR, glomerular filtration rate; HF, heart failure; MI, myocardial infarction; OAC, oral anticoagulant; PAD, peripheral artery disease; PCI, percutaneous coronary intervention; RAS, renin‐angiotensin system; WHO, World Health Organization.

Patients with a major bleeding event less frequently received an invasive coronary strategy (angiography 56.9% versus 77.2% and percutaneous coronary intervention 43.9% versus 60.3%). In addition, having a major bleeding event was associated with a 3‐fold incidence of new‐onset atrial fibrillation, cardiac arrest, and death, and a 4‐ and 5‐fold incidence of reinfarction and in‐hospital shock, respectively (Table 2).

Table 2

In‐Hospital Course

Variable	Major Bleeding		Total (N=97 597)
	No (n=96 241)	Yes (n=1356)
Treatment
Angiography	74 284 (77.2) [0]	772 (56.9) [0]	75 056 (76.9) [0]
PCI	58 002 (60.3) [0]	607 (44.8) [0]	58 609 (60.1) [0]
Fondaparinux	48 749 (50.7) [159]	458 (33.8) [4]	49 207 (50.4) [163]
Heparin	5495 (5.7) [159]	77 (5.7) [4]	5572 (5.7) [163]
LMWH	10 057 (10.4) [159]	307 (22.6) [4]	10 364 (10.6) [163]
Diuretics	18 444 (19.2) [92]	646 (47.6) [2]	19 090 (19.6) [94]
Inotropic drugs	2934 (3.0) [88]	193 (14.2) [4]	3127 (3.2) [92]
Thrombolysis	1599 (1.7) [0]	35 (2.6) [0]	1634 (1.7) [0]
Evaluation (echocardiography)
LVEF <50%	30 220 (31.4) [23 891]	595 (43.9) [351]	30 815 (31.6) [24 332]
Complications
New‐onset atrial fibrillation	3936 (4.1) [1603]	158 (11.7) [6]	4094 (4.2) [1609]
Reinfarction	662 (0.7) [131]	37 (2.7) [1]	699 (0.7) [132]
Shock in hospital	1960 (2.0) [1502]	137 (10.1) [2]	2097 (2.1) [1504]
CPR	3038 (3.2) [15]	138 (10.2) [0]	3176 (3.3) [15]
Death	4595 (4.8) [12]	198 (14.6) [0]	4793 (4.9) [12]

Values are expressed as number (percentage) [missing]. CPR indicates cardiopulmonary resuscitation; LMWH, low‐molecular‐weight heparin; LVEF, left ventricular ejection fraction; PCI, percutaneous coronary intervention.

Model Performance and Validation

The full model including all predictors and interactions (Figure S1) had a C‐index of 0.81 (95% CI, 0.80–0.82). The internal validation indicated only modest overfitting with an optimism‐corrected C‐index and a calibration slope of 0.80 and 1.04, respectively. The approximate model including only the 5 most important predictors (hemoglobin, age, sex, creatinine, and CRP) and 1 interaction term (hemoglobin and sex) represented 92.7% of the full model and was termed the SWEDEHEART score, presented as a nomogram (Figure 2) and equation (Figure S2). The SWEDEHEART score had a C‐index of 0.80 (95% CI, 0.79–0.81) with an optimism‐corrected C‐index and a calibration slope of 0.80 and 0.99 respectively, indicating negligible overfitting. The internal‐external temporal validation indicated temporal stability with an optimism corrected C‐index of 0,80. (C in C‐index in cursive). The ACTION and CRUSADE scores had a C‐index of 0.73 (95% CI, 0.72–0.74) and 0.72 (95% CI, 0.71–0.74), respectively (Table 3), which is in agreement with the results presented in the original publications.15, 17

Figure 2

Nomogram for the SWEDEHEART (Swedish Web‐System for Enhancement and Development of Evidence‐Based Care in Heart Disease Evaluated According to Recommended Therapies) score. To use the nomogram for an individual patient, the points for each predictor are read at the vertical line from the predictor value to the Points line (top). The total points are calculated and a vertical line from this value on the Total Points line to the Bleeding Risk line (bottom) provides the estimated probability of in‐hospital major bleeding risk. Total points <6 or >14 corresponds to a bleeding risk <0.5% or >10%, respectively. Example 1: men with hemoglobin=135, age=85 years, C‐reactive protein (CRP)=10, and creatinine=130. The points corresponding to each predictor are roughly 1.8, 2.2, 0.7, and 2.3. The total points are 7, corresponding to an in‐hospital major bleeding risk of ≈0.7%. Example 2: women with hemoglobin=135, age=85 years, CRP=10, and creatinine=130. The points corresponding to each predictor are roughly 3.3, 2.2, 0.7, and 2.3. The total points are 8.5, corresponding to an in‐hospital major bleeding risk of ≈1.2%.

Table 3

C‐index for the ACTION, CRUSADE, and SWEDEHEART Scores

	No.	Bleeds	C‐Index
			ACTION	CRUSADE	SWEDEHEART
Overall	97 597	1356	0.73 (0.72–0.74)	0.72 (0.71–0.74)	0.80 (0.79–0.81) [0.80]
Sex
Women	34 400	621	0.68 (0.66–0.70)	0.66 (0.64–0.68)	0.74 (0.72–0.76) [0.74]
Men	63 197	735	0.76 (0.74–0.78)	0.76 (0.74–0.77)	0.83 (0.81–0.84) [0.82]
Missing	0
Age, y
<75	56 002	572	0.77 (0.75–0.79)	0.76 (0.74–0.78)	0.81 (0.80–0.83) [0.81]
≥75	41 594	784	0.67 (0.65–0.69)	0.65 (0.64–0.67)	0.76 (0.75–0.78) [0.76]
Missing	1
Anticoagulation
No	91 204	1228	0.73 (0.72–0.75)	0.72 (0.71–0.74)	0.80 (0.79–0.81) [0.80]
Yes	5525	108	0.71 (0.67–0.76)	0.70 (0.65–0.74)	0.77 (0.73–0.82) [0.77]
Missing	868
Angiography
No	22 541	584	0.63 (0.61–0.65)	0.59 (0.57–0.62)	0.76 (0.74–0.79) [0.76]
Yes	75 056	772	0.75 (0.73–0.77)	0.74 (0.73–0.76)	0.79 (0.78–0.81) [0.79]
Missing	0
STEMI
No	64 730	896	0.74 (0.73–0.76)	0.72 (0.71–0.74)	0.82 (0.80–0.83) [0.81]
Yes	31 304	447	0.74 (0.72–0.76)	0.73 (0.71–0.75)	0.77 (0.75–0.79) [0.77]
Missing	1563
GFR
<60	28 236	715	0.66 (0.65–0.68)	0.65 (0.63–0.67)	0.75 (0.73–0.77) [0.75]
≥60	63 999	578	0.73 (0.71–0.75)	0.71 (0.69–0.73)	0.79 (0.77–0.81) [0.78]
Missing	5362

C‐index (95% CI) (if possible C in C‐index in cursive). ACTION indicates Acute Coronary Treatment and Intervention Outcomes Network; CRUSADE, Can Rapid Risk Stratification of Unstable Angina Patients Suppress Adverse Outcomes With Early Implementation of the ACC/AHA Guidelines; GFR, glomerular filtration rate; STEMI, ST‐segment–elevation myocardial infarction; SWEDEHEART, Swedish Web‐System for Enhancement and Development of Evidence‐Based Care in Heart Disease Evaluated According to Recommended Therapies.

The SWEDEHEART score achieved a consistently higher C‐index than the ACTION and CRUSADE scores in the subgroups of women, elderly (>75 years), patients managed noninvasively, patients taking OACs, those with ST‐segment–elevation MI/non–ST‐segment–elevation MI, and those with chronic kidney disease <60 mL/min (Table 3). In the subgroup of patients managed noninvasively, which constituted 25% of the study population, the CRUSADE score was only weakly discriminative, with a C‐index of 0.59 (95% CI, 0.57–0.62).

The ACTION and CRUSADE scores were poorly calibrated in the derivation cohort (Figure 1), and when recalibrated had an optimism‐corrected C‐index of 0.73 and 0.72, respectively, and a calibration slope of 1.0 in both cases, indicating no overfitting. The recalibrated ACTION and CRUSADE scores and the SWEDEHEART score showed comparable calibration (Figure 1) for bleeding probabilities between 1% and 4%, whereas the SWEDEHEART score demonstrated superior calibration for bleeding probabilities <1% and >4%. Calibration plots comparing the recalibrated ACTION and CRUSADE scores and the SWEDEHEART score in all subgroups are shown in Figure S3. The observed bleeding risks within the ACTION, CRUSADE, and SWEDEHEART risk classes are illustrated in Figures 3 and 4. The SWEDEHEART score demonstrated improved risk stratification within the ACTION and CRUSADE risk classes (Figures 3A and 4A), whereas the ACTION and CRUSADE scores did not improve the risk stratification within the SWEDEHEART risk classes (Figures 3B and 4B).

Figure 3

Observed bleeding risk within the ACTION (Acute Coronary Treatment and Intervention Outcomes Network) and SWEDEHEART (Swedish Web‐System for Enhancement and Development of Evidence‐Based Care in Heart Disease Evaluated According to Recommended Therapies) risk classes. Observed bleeding risk and corresponding 95% CIs are illustrated within the ACTION and SWEDEHEART risk classes. In (A) the observed bleeding risk within the ACTION risk classes is shown for each SWEDEHEART risk class, demonstrating high variability of the observed bleeding risk and low agreement between the observed bleeding risk and the ACTION risk classes. In (B) the observed bleeding risk within the SWEDEHEART risk classes is shown for each ACTION risk class, showing low variability of the observed bleeding risk and high agreement between observed bleeding risk and the SWEDEHEART risk classes.

Figure 4

Observed bleeding risk within the CRUSADE (Can Rapid Risk Stratification of Unstable Angina Patients Suppress Adverse Outcomes With Early Implementation of the ACC/AHA Guidelines) and SWEDEHEART (Swedish Web‐System for Enhancement and Development of Evidence‐Based Care in Heart Disease Evaluated According to Recommended Therapies) risk classes. Observed bleeding risk and corresponding 95% CIs are illustrated within the CRUSADE and SWEDEHEART risk classes. In (A) the observed bleeding risk within the CRUSADE risk classes is shown for each SWEDEHEART risk class, demonstrating high variability of the observed bleeding risk and low agreement between the observed bleeding risk and the CRUSADE risk classes. In (B) the observed bleeding risk within the SWEDEHEART risk classes is shown for each CRUSADE risk class, showing low variability of the observed bleeding risk and high agreement between observed bleeding risk and the SWEDEHEART risk classes.

Clinical utility was evaluated using decision curve analysis.33 Across the range of clinically relevant decision thresholds, the SWEDEHEART score had consistent positive and larger net benefit in comparison with the recalibrated ACTION and CRUSADE scores (Figure 5). For example, at a threshold of 2% in‐hospital major bleeding risk, the SWEDEHEART score will identify 2.0 and 2.3 additional in‐hospital major bleeds compared with the recalibrated ACTION and CRUSADE scores, respectively, in a population with 13.9 in‐hospital major bleeds per 1000 individuals, without increasing the number of false‐positives. Because of their poor calibration, the original ACTION and CRUSADE scores showed negative net benefit (Tables S3 and S4). In the same setting as above, the SWEDEHEART score will identify 11.3 additional in‐hospital major bleeds compared with the original ACTION and CRUSADE scores.

Figure 5

Decision curve analysis. For each decision threshold, the net benefit of the recalibrated ACTION (Acute Coronary Treatment and Intervention Outcomes Network) and CRUSADE (Can Rapid Risk Stratification of Unstable Angina Patients Suppress Adverse Outcomes With Early Implementation of the ACC/AHA Guidelines) scores and the SWEDEHEART (Swedish Web‐System for Enhancement and Development of Evidence‐Based Care in Heart Disease Evaluated According to Recommended Therapies) score is shown. In addition, the net benefit assuming that all patients have a bleeding risk higher than the threshold (all risk high) as well as the net benefit assuming that all patients have a bleeding risk lower than the threshold (all risk low) are shown. For a given threshold, the difference in net benefit between two scores is the additional number of bleeds identified (per 1000) without increasing the number of false positive classifications. Across the range of decision thresholds, the SWEDEHEART score was consistently positive and had larger net benefit than the recalibrated ACTION and CRUSADE scores.

If CRP was not available, we made an alternate SWEDEHEART score without CRP. This alternate score represented 88.2% of the full model and had a C‐index of 0.79 (95% CI, 0.77–0.80) and an optimism‐corrected C‐index and calibration slope of 0.78 and 1.0, respectively. A more detailed description including nomogram, calibration curve, and equation for the SWEDEHEART score without CRP is shown in Table S5 and Figures S4 through S6.

Discussion

In this study we developed and validated a new risk score for in‐hospital major bleeding in nearly 100 000 patients with acute MI. Our new score, based on the 5 baseline variables (hemoglobin, age, sex, creatinine, and CRP) showed good discrimination and calibration across different subgroups. The ACTION score and the currently recommended CRUSADE score was, even after recalibration, inferior to the SWEDEHEART score and, as previously described, especially unfavorable in patients not undergoing coronary angiography19

Need for Validation and Recalibration of Scores

The ACTION and CRUSADE scores have already been externally validated in several populations. However, both scores were associated with negative net benefit in the derivation cohort, indicating that their use was harmful. This finding is of additional clinical importance since it stresses the need for validation and, if needed, recalibration before applying a risk score to a new population.

Comparison With Other Scores

What makes our score unique is that it was derived from a cohort including almost all patients hospitalized with acute MI in Sweden over a study period of almost 6 years. Of the entire index population of 97 777 patients, only 180 (0.2%) patients with missing outcome data were excluded. An internal–external cross‐validation was used to assess the temporal validity of the score within the geographical area. The ACTION and CRUSADE registers comprised selected patients on a national level since they only covered a small proportion of US hospitals. There is also lack of data regarding the proportion of patients with ACS who were included in the registries at participating hospitals. CRUSADE only included non–ST‐segment–elevation MI and excluded patients receiving OACs. Both the ACTION and CRUSADE derivation cohorts excluded patients with missing data on certain baseline variables as well as patients who died within 24 and 48 hours, respectively.15, 17 The ACUITY score was derived from 2 randomized controlled trials and thus included a strongly selected population.16

Overall Low Incidence of Bleeding

The incidence of major bleeding in our derivation cohort was low (1.4%) compared with ACTION (10.8%), ACUITY (7.3%), and CRUSADE (9.4%). There are many plausible explanations for this finding. Comparison of bleeding incidence is dependent not only on the definition but also the characteristics and treatment patterns of the population. Our bleeding definition did only include non–coronary artery bypass graft–related bleeds that were fatal, were intracranial, or that led to blood transfusion or intervention during a short observation period. Bleedings with only a large drop in hemoglobin were not counted. In the CRUSADE population, femoral access was dominating and glycoprotein IIb/IIIa blockers were used more frequently. Since then, several bleeding avoidance strategies have been implemented and most of the patients in our derivation cohort were treated through radial access, few received glycoprotein IIb/IIIa blockers, and new drugs (fondaparinux and bivalirudin) with a lower risk of bleeding were used. Finally, but not least important, when using a bleeding definition including transfusion, the bleeding incidence may also reflect the transfusion strategy in the study population. Our bleeding definition differed from the CRUSADE definition, which included hemoglobin drops and retroperitoneal bleeds. The transfused bleeds constituted over 6% in the CRUSADE population compared with only 1.2% in the SWEDEHEART population, which may suggest either more severe bleeds that led to transfusion and/or a more liberal transfusion strategy in the CRUSADE population. Geographical variation in transfusion strategy with a significantly higher incidence in US compared with non‐US countries has been described by Rao et al35 even though the rate of transfusion in patients undergoing percutaneous coronary intervention in the United States has declined over the past years36

There may be underreporting of bleeding events but the register is regularly monitored with 95% to 96% variable agreement.

Anemia as a Marker of Increased Risk

The strong association between anemia and bleeding is to some extent a result of our bleeding definition as well as the fact that preexisting anemia by itself may exacerbate the consequences of bleeding. Bleeding in patients with normal or high hemoglobin levels at baseline will more seldom lead to red blood cell transfusion, whereas bleeding in patients with anemia at baseline will more often be followed by an intervention. Nonfatal bleeding with large drop in hemoglobin without red blood cell transfusion or surgery was not registered as a bleeding event in the registry. Finally, preexisting anemia may reflect prior or ongoing occult bleeding, which will become manifest when treated with antithrombotic treatment in the course of an ACS.

Inflammation as a Marker of Increased Risk

The variable CRP may be questioned because it is nonspecific, but markers of inflammation are certainly of interest for prediction of bleeding events. A small study in the elderly has shown that the addition of CRP and previous bleeding improved the discriminative ability of the CRUSADE score37 In the ACUITY score and the PRECISE‐DAPT38 inflammatory status is reflected by white blood cell count. Recent work from the PLATO (Platelet Inhibition and Patient Outcome) and ARISTOTLE (Apixaban for the Prevention of Stroke in Subjects With Atrial Fibrillation) studies showed that growth differentiation factor 15, a marker of inflammation and cell ageing, was strongly associated with bleeding39, 40, 41 Elevated CRP has also been shown to be associated with both upper and lower GI bleeding42, 43 as well as malignancy.44 As expected, the alternate SWEDEHEART score without CRP explained less (88.2%) of the full model compared with the score with CRP (92.6%), but the alternate score without CRP was still useful, with only slightly lower discriminative ability.

Need for Risk Scores

Assessment of patients with ACS and an indication for treatment in clinical practice are often not as clear‐cut as in clinical trials and clinical guidelines. When deciding on intensity of antithrombotic treatment, physicians often have to weigh the ischemic risk against the bleeding risk. Objective assessment of ischemic risk by the GRACE score has been shown to be superior to physician‐estimated ischemic risk.45 With an accutare bleeding risk score the same should probabably hold true for assessment of bleeding risk. Bleeding risk assessment by the CRUSADE score is listed as one of the main quality indicators for in‐hospital risk assessment in acute MI by the ESC Acute Cardiovascular Care Association Quality of Care Working Group.46 Our new risk model is complex but still easy to use provided a calculator on a computer or smartphone or delivered by decision support. A simple head‐calculation score with dichotomized variables may be even more easy to use, but may result in less precise risk estimates and decisions about treatment. Still, we will need future trials that can more exactly define how our score should influence clinical decision making.

Strengths and Limitations

While there are strengths to our methodological approach including size, completeness, and the state‐of‐art derivation and validation, we acknowledge some study limitations.

Although our results should be considered as highly valid in Sweden (with similar patients with ACS and coronary care as the rest of the Western world), lack of external validation in other geographically defined populations in which the score will be used is still a limitation of the study. But this holds true for all scores. Our bleeding definition was limited by the variables available in our register and therefore we could not use the CRUSADE or any other standardized bleeding definition. Nor did we have information on type of bleeding and, thus, outcome associated with different subtypes of bleeding could not be evaluated.

Using a different bleeding definition and also the temporal aspect with implementation of bleeding avoidance strategies since the CRUSADE era may have limited the performance of the CRUSADE score in our study population. Furthermore, hematocrit level was unavailable in the derivation cohort and replaced by hemoglobin converted to hematocrit for calculation of the CRUSADE score, which may be less exact. We also lacked data regarding dosing of antithrombotic treatment, which may have influenced the explanatory value of the full model.

Conclusions

The SWEDEHEART score, based on the variables hemoglobin, creatinine, age, sex, and CRP, offers better prediction of major in‐hospital bleeding in acute MI than the ACTION score and the currently recommended CRUSADE score.

Sources of Funding

This study was funded by the Swedish Foundation for Strategic Research and the Swedish Heart‐Lung Foundation.

Disclosures

None.

Data S1. Supplemental Statistical Methods (detailed).

Table S1. Previous Bleeding ICD‐10 Diagnoses

Table S2. Missing Observations

Table S3. Net Benefit for ACTION and SWEDEHEART

Table S4. Net Benefit for CRUSADE and SWEDEHEART

Table S5. C‐Index for SWEDEHEART Score without CRP

Figure S1. The full model.

Figure S2. SWEDEHEART (Swedish Web‐System for Enhancement and Development of Evidence‐Based Care in Heart Disease Evaluated According to Recommended Therapies) score equation.

Figure S3. Calibration in subgroups.

Figure S4. Score equation for SWEDEHEART (Swedish Web‐System for Enhancement and Development of Evidence‐Based Care in Heart Disease Evaluated According to Recommended Therapies) without C‐reactive protein (CRP).

Figure S5. Nomogram for SWEDEHEART (Swedish Web‐System for Enhancement and Development of Evidence‐Based Care in Heart Disease Evaluated According to Recommended Therapies) score without C‐reactive protein.

Figure S6. Calibration for SWEDEHEART (Swedish Web‐System for Enhancement and Development of Evidence‐Based Care in Heart Disease Evaluated According to Recommended Therapies) score without C‐reactive protein.

Click here for additional data file.