Refining prediction of major bleeding on antiplatelet treatment after transient ischaemic attack or ischaemic stroke.

INTRODUCTION: Bleeding is the main safety concern of treatment with antiplatelet drugs. We aimed to refine prediction of major bleeding on antiplatelet treatment after a transient ischaemic attack (TIA) or stroke by assessing the added value of new predictors to the existing S2TOP-BLEED score. PATIENTS AND METHODS: We used Cox regression analysis to study the association between candidate predictors and major bleeding among 2072 patients with a transient ischaemic attack or ischaemic stroke included in a population-based study (Oxford Vascular Study - OXVASC). An updated model was proposed and validated in 1094 patients with a myocardial infarction included in OXVASC. Models were compared with c-statistics, calibration plots, and net reclassification improvement.
RESULTS: Independent predictors for major bleeding on top of S2TOP-BLEED variables were peptic ulcer (hazard ratio (HR): 1.72; 1.04-2.86), cancer (HR: 2.40; 1.57-3.68), anaemia (HR: 1.55; 0.99-2.44) and renal failure (HR: 2.20; 1.57-4.28). Addition of those variables improved discrimination from 0.69 (0.64-0.73) to 0.73 (0.69-0.78) in the TIA/stroke cohort (p = 0.01). Performance improved particularly for upper gastro-intestinal bleeds (0.70; 0.64-0.75 to 0.77; 0.72-0.82). Net reclassification improved over the entire range of the score (net reclassification improvement: 0.56; 0.36-0.76). In the validation cohort, discriminatory performance improved from 0.68 (0.62-0.74) to 0.70 (0.64-0.76). DISCUSSION AND
CONCLUSION: Peptic ulcer, cancer, anaemia and renal failure improve predictive performance of the S2TOP-BLEED score for major bleeding after stroke. Future external validation studies will be required to confirm the value of the STOP-BLEED+ score in transient ischaemic attack/stroke patients. © European Stroke Organisation 2020.

Introduction

Treatment with antiplatelet drugs is indicated following a transient ischaemic attack (TIA) or non-cardioembolic ischaemic stroke.[1] Bleeding is the main safety concern of antiplatelet drugs, with an average one-year risk of 1%–1.5%.[2,3] Accurate prediction of bleeding risk for an individual patient would allow physicians to weigh benefits and risks of antiplatelet treatment.

Clinical prediction scores, such as the S2TOP-BLEED score, have been developed to improve risk stratification for major bleeding and are mainly based on patient characteristics.[4] However, performance of available scores is modest with c-statistics generally below 0.70,[4-6] and discrimination might be improved by extending models with other known risk factors for bleeding. The presence of a peptic ulcer is a frequent cause of upper gastro-intestinal (GI) bleeding and increases risk approximately three-fold.[7,8] Also, cancer has been associated with risk of bleeding, which is further enhanced by antithrombotic treatment.[9,10] Furthermore, several laboratory characteristics have been shown to increase risk of major bleeding, including renal failure, liver failure and anaemia.[11-13]

The S2TOP-BLEED score was derived from individual patient data from trials.[4] As patients with the highest risk of bleeding are often excluded from trials studying antiplatelet therapy, the predictive value of factors that served as exclusion criteria could not be assessed in the development of the score. Extending the model with other risk factors for major bleeding might improve predictive performance. We aimed to investigate whether we can refine prediction of major bleeding on antiplatelet treatment after a TIA or ischaemic stroke by extending the previously developed S2TOP-BLEED score (S2TOP-BLEED+) in a population-based cohort of patients without exclusions.

Methods

Study population

The Oxford Vascular Study (OXVASC) is an ongoing population-based cohort study of all acute vascular events in a population of 92,728 individuals registered with 100 general practitioners in 9 general practices in Oxfordshire, UK.[14] Overlapping methods of hot and cold pursuit are used to identify all vascular events in the study population. In the present study, we analysed patients who experienced a TIA, ischaemic stroke, or myocardial infarction (either a first event or recurrent event) between 2002 and 2012, and who were prescribed antiplatelet treatment. These included both patients who were on premorbid antiplatelet drugs as well as patients who started antiplatelet drugs after the index event. The use of antiplatelet drugs was self-reported by patients and cross-referenced with primary care and medical records.

During an initial visit, information on baseline characteristics and risk factors was collected. Patients were followed-up at 30 days, 6 months, 1 year, 5 and 10 years by a study nurse or physician. At each follow-up visit, recurrent ischaemic events and bleeding events that required medical attention were recorded. Bleeding events were also identified by daily searches of all hospital admissions, by review of administrative diagnostic codes from hospital and primary care records, and by searches of blood transfusion records. Only bleeds that required medical attention or were fatal prior to medical attention were included. Bleeds secondary to trauma, surgery or haematological malignancy were excluded. Haemorrhagic transformation of the index event was not included as a major intracranial bleed. However, if the patient presented with a recurrent event and no distinction could be made between a primary intracerebral haemorrhage and a recurrent ischaemic stroke with haemorrhagic transformation, the event was included as a major bleed (n = 6). The severity of bleeds was classified according to the CURE criteria.[15] Major bleeds were bleeds that were substantially disabling with persistent sequelae, intraocular bleeds leading to significant loss of vision or bleeds requiring transfusion of two or more units of blood. Bleeding events that required medical attention but did not fulfil the criteria of major bleeding were recorded as significant non-major bleeds.

Candidate predictors that could potentially improve prediction of bleeding were selected based on the literature.[16] Their inclusion was dependent on availability in OXVASC. Predictors of interest that could be studied were history of peptic ulcer[7,8] (based on face-to-face patient interview plus cross-referencing with primary care and medical records), history of cancer[9,10] (based on face-to-face patient interview plus cross-referencing with primary care and medical records). History of cancer included any type of cancer, i.e. GI, non-GI, haematological malignancies and skin cancer, and included both active cancer and cancer in remission, renal failure (any documented history of chronic renal failure – glomerular filtration rate of <30 ml/minute/1.73 m2 estimated using the modification of diet in renal disease (MDRD) equation), liver failure and anaemia[11-13] (haemoglobin <12 g/dL (7.5 mmol/L) for women and <13 g/dL (8.1 mmol/L) for men, as assessed at baseline).

OXVASC has been approved by the local ethics committee. Patients gave written informed consent or assent was obtained from relatives for patients who were unable to provide consent.

Statistical analysis

Data on body mass index (BMI) were missing for 403 patients (13%) in both cohorts combined, and for these patients, the mean BMI was imputed. Twenty-two patients (0.7%) had missing values on variables other than BMI, and these patients were excluded from the analyses. We performed Cox regression analysis to study the association between candidate predictors and major bleeding, while including the linear predictor of the original S2TOP-BLEED score as variable in the model. This method for model updating is referred to as recalibration plus extension.[17] It has shown to provide well-calibrated results for new patients and is recommended when the data set for updating is relatively small compared with the development cohort[17] (for the current study: 2072 patients in the update cohort vs. 43,112 patients in the derivation cohort), as it limits the number of coefficients that is re-estimated. We constructed a Cox regression model containing the candidate predictors and the original S2TOP-BLEED score as variable and applied least absolute shrinkage and selection operator (lasso) regression with repeated cross validation to select the optimal model.[18,19] Compared with traditional stepwise selection, lasso is less likely to produce over-fitted models, particularly when the number of events per variable is small.[20] The baseline hazard and mean linear predictor were re-estimated after addition of new predictors.

We calculated three-year risk of major bleeding according to both the original and the updated model and classified patients into risk categories of low (<5%), intermediate (5%–10%) and high risk (>10%). We assigned weights to the additional variables by dividing the regression coefficients by the smallest coefficient of the original S2TOP-BLEED model and rounded them to the nearest integer.

Performance of the proposed updated model was assessed with discrimination, calibration and reclassification measures. Discrimination reflects the ability of the model to distinguish between someone with and without a major bleed and was assessed with the c-statistic. The c-statistic was calculated separately for intracranial bleedings, non-major bleedings and upper GI bleeding, the most frequent type of antiplatelet-related bleeding, which may partly be prevented by treatment with gastro-protective drugs. Furthermore, we separately assessed discriminatory ability for the first three months, during which dual antiplatelet therapy may be considered. Calibration addresses the correspondence between the observed and predicted risk of major bleeding and was investigated with calibration plots. We calculated the net reclassification improvement (NRI) to assess change in risk stratification with the updated model.[21] The NRI quantifies the percentage of correct movement across risk categories for patients with and without a major bleed. Correct movement is defined as upward classification for patients with an event, and downward classification for patients without an event. Net reclassification was assessed while taking survival time into account.[22] Last, we assessed the improvement without applying risk categories, by means of the continuous NRI and the integrated discrimination improvement.[21] Confidence intervals were obtained with bootstrapping.

We examined the robustness of the updated model by validating the model in MI patients included in OXVASC. We applied the coefficients of the model as determined in the TIA/stroke cohort and calculated the three-year risk of major bleeding for each patient. All patients were assigned zero points for the modified Rankin Scale variable. Again, discrimination, calibration and reclassification were assessed. All analyses were performed with R version 3.3.2. Results are reported in accordance with the TRIPOD statement.[23]

Results

Between 2002 and 2012, 2072 patients with a TIA or ischaemic stroke on antiplatelet treatment were included in OXVASC (895 TIA and 1177 ischaemic stroke). Baseline characteristics of patients are presented in Supplementary Table I. During 8302 person-years of follow-up, 117 patients experienced a major bleed (31 fatal bleeds (26%) of which 19 were intracranial and 12 GI (Supplementary Table II)). Three-year risk of major bleeding was 5.7% (5.9% after exclusion of patients on a short course of dual antiplatelet therapy). The validation cohort consisted of 1094 patients with an MI included in OXVASC during the same period, of whom 70 had had a major bleed (three-year risk 7.1%; Supplementary Figure I).

Among patients with a TIA or ischaemic stroke, we identified history of cancer, peptic ulcer, anaemia and renal failure as independent predictors for major bleeding on top of the S2TOP-BLEED risk factors (Table 1). Addition of those variables to the original S2TOP-BLEED score led to an increment in c-statistic from 0.69 (0.64–0.73) to 0.73 (0.69–0.78), p = 0.01 (Table 2; for regression equation, see Supplementary Table III). Liver failure was initially selected as predictor in the model, but we decided to exclude it given the very low prevalence (1%) and wide confidence intervals. Three-year predicted risk of major bleeding based on the updated score was in close agreement with the observed risk of major bleeding as estimated with Kaplan–Meier (Supplementary Figure II). The discriminatory performance for upper GI bleeds improved from 0.70 (0.64–0.75) to 0.77 (0.72–0.82) with S2TOP-BLEED+. The c-statistic for intracranial bleeding was 0.67 (0.60–0.74) and for non-major bleeds was 0.53 (0.47–0.59). The STOP-BLEED+ score showed a c-statistic of 0.71 (0.62–0.80) for prediction of major bleeding during the first three months (Supplementary Table IV). Discriminatory performance of the S2TOP-BLEED+ score was better for major bleedings than for recurrent ischaemic events (c-statistic 0.57 (0.54–0.61)). The Brier score of the S2TOP-BLEED+ model was 0.05 (scaled Brier score 15%). Figure 1 shows the distribution of patients with and without events among risk categories based on the original and updated S2TOP-BLEED scores. Overall, 70% of patients remained in the same risk category. With the updated model, 4.1% of patients with a major bleed (event NRI) and 4.7% of patients without a major bleed (non-event NRI) were correctly reclassified, leading to an overall NRI of 0.09 (−0.04 to 0.22) (Table 2 and Figure 1). Over the entire range of predicted risks, the S2TOP-BLEED+ score improved reclassification (continuous NRI 0.56; 0.36–0.76, Table 2), mainly driven by correct downward classification of non-events (38%) and a smaller improvement in classification of events (18%) (Supplementary Table V). When patients were divided in quartiles according to their predicted risk, the observed three-year risk of major bleeding ranged from 0.6% to 14.0%, as compared with 0.8% to 11.6% according to the original model (Figure 2). The ratio between risk of recurrent ischaemic events and major bleeding drops from 19:1 in the lowest quartile to 2:1 in the highest quartile of predicted bleeding risk (Supplementary Table VI).

Table 1.

Hazard ratios of candidate predictors for major bleeding, in the presence of original S2TOP-BLEED variables.

	Multivariable HR (95% CI)a	Beta coefficients estimated with lasso
Cancer	2.40 (1.57–3.68)	0.847
Peptic ulcer	1.72 (1.04–2.86)	0.511
Anaemia	1.55 (0.99–2.44)	0.429
Renal failure	2.20 (1.57–4.28)	0.766
Liver failure	1.62 (0.40–6.61)	–

aAdjusted for original S2TOP-BLEED variables (age, male sex, Asian ethnicity, body mass index, smoking, modified Rankin Scale score ≥3, hypertension, diabetes, prior stroke, type of antiplatelet treatment). HR: hazard ratio; CI: confidence interval.

Table 2.

Performance of S2TOP-BLEED and S2TOP-BLEED+ scores in TIA/stroke and MI cohorts.

	C-statistic (95% CI)	Continuous NRI (95% CI)	Categorical NRI (95% CI)	IDI (95% CI)
TIA/stroke cohort
S2TOP-BLEED	0.69 (0.64–0.73)	Ref	Ref	Ref
S2TOP-BLEED+	0.73 (0.69–0.78)	0.56 (0.36–0.76)	0.09 (−0.04 to 0.22)	0.019 (0.007–0.03)
MI cohort
S2TOP-BLEED	0.68 (0.62–0.74)	Ref	Ref	Ref
S2TOP-BLEED+	0.70 (0.64–0.76)	0.49 (0.21–0.78)	0.14 (−0.03 to 0.31)	0.011 (−0.005 to 0.028)

MI: myocardial infarction; NRI: net reclassification index; CI: confidence interval; TIA: transient ischaemic attack; IDI: integrated discrimination index; Ref: reference.

Figure 1.

Reclassification tables in TIA/stroke cohort. Same risk category: (14.6 + 29.1 + 37.6 + 809.4 + 354.3 + 185.4)/2067 = 70%; Event NRI: ((4.4 + 5.2 + 20.8) − (8.9 + 16.3))/130.4 = 0.041; Non-event NRI: ((265.1 + 106.7) − (54.6 + 43.8 + 182.2))/1936.6 = 0.047; Overall NRI: 0.041 + 0.047 = 0.088. KM: Kaplan–Meier.

Figure 2.

Observed risk of major bleeding across quartiles of predicted risk according to S2TOP-BLEED and S2TOP-BLEED+ scores. Q: quartile.

Based on the regression coefficients, we assigned points to anaemia (3 points), peptic ulcer (4 points), cancer (6 points) and renal failure (6 points). An updated score chart is presented in Table 3, with predicted risks displayed in Figure 3.

Table 3.

S2TOP-BLEED and S2TOP-BLEED+ score.

	S2TOP-BLEED	S2TOP-BLEED+
Risk factor	Points	Points
Age
45–54	2	2
55–64	4	4
65–74	6	6
75–84	9	9
85+	12	12
Male sex	2	2
Asian ethnicity	1	1
Current smoking	1	1
Hypertension	1	1
Diabetes	1	1
Prior stroke	1	1
mRS >2	2	2
BMI
<20	2	2
20–25	1	1
Treatment
A, A + D	1	1
A + C	5	5
Peptic ulcer	NA	4
Cancer	NA	6
Anaemia	NA	3
Renal failure	NA	6

A: aspirin; A + D: aspirin + dipyridamole; A + C: aspirin + clopidogrel; BMI: body mass index; mRS: modified Rankin Scale score; NA: not applicable.

Figure 3.

Predicted three-year risk of major bleeding (%) according to the S2TOP-BLEED+ score.

The S2TOP-BLEED+ score as proposed in the TIA/stroke cohort had a c-statistic of 0.70 (0.64–0.76) in the MI cohort, compared with 0.68 (0.62–0.74) for the original S2TOP-BLEED score (p = 0.39), see Table 2. Calibration was slightly better for the updated model than for the original S2TOP-BLEED score, although both models underestimated risk of major bleeding (Supplementary Figure III).

Discussion

Prediction of major bleeding can be refined by incorporating cancer, peptic ulcer, anaemia and renal failure in the existing S2TOP-BLEED score for major bleeding in patients with a TIA or ischaemic stroke on antiplatelet therapy. Compared with the original score, the S2TOP-BLEED+ score showed higher discriminatory ability and a larger range of predicted probabilities. A slight improvement in discrimination and calibration was also observed when the model was applied to patients with an MI on antiplatelet drugs, supporting the robustness of the model extension.

Although the associations between anaemia, renal failure, peptic ulcer, cancer and major bleeding have been established previously,[8,9,24] the S2TOP-BLEED+ score is the first to incorporate these characteristics in a model for bleeding among stroke patients. Other scores like REACH[5] and intracranial-B2LEED3S[6] could not investigate these factors, due to a lack of measurement in the derivation cohorts, and the likelihood that many patients with these characteristics would be excluded from trial populations and selective registries. However, studies in other areas have shown the importance of these variables in prediction of major bleeding, both in patients with atrial fibrillation[25-27] and in patients with acute coronary syndrome.[12,28] The importance of the new variables is also reflected by their relatively large weights in the S2TOP-BLEED+ score.

The primary goal of updating S2TOP-BLEED was to improve the discriminatory ability of the model, aiming to better separate patients with and without a major bleed. Although the increment in c-statistic is small, previous studies have shown that the c-statistic is not very sensitive to addition of new predictors, and possible improvements are highly dependent on the strength of the baseline model.[29] Relying solely on the c-statistic for the assessment of added value of predictors is therefore not recommended, as it may lead to exclusion of risk factors that do have a relevant impact on risk stratification in clinical practice.[30] Reclassification measures such as the NRI have been proposed as alternatives to assess the incremental value of new predictors and aim to assess whether addition of new predictors actually leads to a change in clinical practice. Calculation of both traditional performance measures (c-statistic, calibration plots) and newer reclassification measures is currently recommended to assess added value of new predictors, as these measures provide complementary information.[29] Ultimately, the best way to assess robustness of a model extension is by external validation in an independent population.[31]

The absolute risk of major bleeding was higher among patients with an MI in OXVASC, particularly in the early phase. This is likely explained by a higher incidence of procedure-related bleedings and more frequent prescription of dual antiplatelet treatment (66% among MI patients vs. 26% in TIA/stroke patients in the present study). As a consequence, there was some underestimation of bleeding risk in the MI cohort by both models, which was more pronounced for the original score. Although validation in patients with an MI provides some insight in the robustness of the model extension, a validation study among patients with a TIA or ischaemic stroke is still required.

Although prediction of bleeding may be slightly improved with the updated score, a trade off should be made between increasing complexity and improved performance. The updated score may be less suitable for use at the bedside, due to the larger number of predictors and complexity of the weights assigned to each factor. However, given the increasing use of electronic patient records, calculations could be integrated in health records and performed automatically. Moreover, the new variables are likely to be easily available in all patients with a TIA or stroke and do not require additional imaging or biomarker assessment. An important advantage of the updated model is that it will likely provide better predictions for patients who do not fit in a clinical trial profile, but for whom decisions need to be made in clinical practice. A further advantage is the improved prediction of upper-GI bleeding, which is substantially preventable by co-prescription of proton-pump inhibitor (PPI) drugs, such that the score might be used to target the use of PPI drugs or other similar interventions in high-risk patients.

Strengths of our study include the population-based nature of the cohort, thereby representing the entire range of patients with TIA or stroke on antiplatelet drugs and the validation of the updated model in a separate cohort. A limitation of our study is the relatively small number of bleeding events for model updating and subsequent validation. However, we tried to account for this by using state-of-the-art statistical methods. Another limitation is the fact that we could not validate the model in patients with a TIA or ischaemic stroke, for which the model is intended to be used.

In conclusion, we propose the S2TOP-BLEED+ score as a refinement to the original S2TOP-BLEED score, aiming to predict bleeding after a TIA or non-cardioembolic ischaemic stroke. Addition of cancer, peptic ulcer, anaemia and renal failure improves discriminatory performance and increases the range of predicted risks. An external validation of the S2TOP-BLEED+ score among TIA/ischaemic stroke patients is required to confirm its value.

Click here for additional data file.

Supplemental material, ESO898064 Supplemental material for Refining prediction of major bleeding on antiplatelet treatment after transient ischaemic attack or ischaemic stroke by Nina A Hilkens, Linxin Li, Peter M Rothwell, Ale Algra and Jacoba P Greving in European Stroke Journal