Impact of a Patient's Baseline Risk on the Relative Benefit and Harm of a Preventive Treatment Strategy: Applying Trial Results in Clinical Decision Making.

Background For translating an overall trial result into an individual patient's expected absolute treatment effect, differences in relative treatment effect between patients need to be taken into account. The aim of this study was to evaluate whether relative treatment effects of medication in 2 large contemporary trials are influenced by multivariable baseline risk of an individual patient. Methods and Results In 9361 patients from SPRINT (Systolic Blood Pressure Intervention Trial), risk of major adverse cardiovascular events was assessed using a newly derived risk model. In 18 133 patients from the RE-LY (Randomized Evaluation of Long-Term Anticoagulant Therapy) trial, risk of stroke or systemic embolism and major bleeding was assessed using the Global Anticoagulant Registry in the Field-Atrial Fibrillation risk model. Heterogeneity of trial treatment effect was assessed using Cox models of trial allocation, model linear predictor, and their interaction. There was no significant interaction between baseline risk and relative treatment effect from intensive blood pressure lowering in SPRINT (P=0.92) or from dabigatran compared with warfarin for stroke or systemic embolism in the RE-LY trial (P=0.71). There was significant interaction between baseline risk and treatment effect from dabigatran versus warfarin in the RE-LY trial (P<0.001) for major bleeding. Quartile-specific hazard ratios for bleeding ranged from 0.40 (95% CI, 0.26-0.61) to 1.04 (95% CI, 0.83-1.03) for dabigatran, 110 mg, and from 0.61 (95% CI, 0.42-0.88) to 1.20 (95% CI, 0.97-1.50) for dabigatran, 150 mg, compared with warfarin. Conclusions Effect modification of relative treatment effect by individual baseline event risk should be assessed systematically in randomized clinical trials using multivariate risk prediction, not only in terms of treatment efficacy but also for important treatment harms, as a prespecified analysis. Registration URL: https://www.clinicaltrials.gov; Unique identifier: NCT01206062.

Global Anticoagulant Registry in the Field–Atrial Fibrillation

Randomized Evaluation of Long‐Term Anticoagulant Therapy

systemic embolism

Systolic Blood Pressure Intervention Trial

Clinical Perspective

What Is New?

For translating an overall trial result into an individual patient’s expected absolute treatment effect, differences in relative treatment effect between patients need to be taken into account, preferably assessed with a multivariable risk score.

Individual patient data from large clinical trials provide a unique opportunity to assess relative treatment effect modification across the full range of baseline risk (risk without the allocated intervention), as determined by multiple characteristics.

What Are the Clinical Implications?

Individual absolute treatment effects should be estimated for making treatment decisions in clinical practice, not only in terms of treatment efficacy but also for important treatment harms.

Every patient is different, and every patient will react differently to medication. However, randomized clinical trials (RCTs) usually report results as a single relative effect size. In current clinical practice, this single relative treatment effect is then used and applied to diverse patient categories and a plethora of individual patients. Implicitly, the assumption is made that this single relative effect measure is true for all study participants, independent of an individual’s characteristics.

The anticipated absolute treatment effect is then dependent on not only the relative treatment effect, but also on baseline risk: for example, a relative risk reduction of 20% for an intervention will result in a 5% absolute risk reduction for a patient with a 25% baseline 10‐year risk for the outcome, and a 2% absolute risk reduction for a patient with a 10% baseline 10‐year risk. However, for these absolute treatment effects to be true, the assumption of a single relative treatment effect independent of baseline risk and different clinical characteristics needs to hold. An individual patient in clinical practice is not the same as the average trial participant. The dilemma in clinical practice is whether a single overall relative treatment effect is also true for the patient a health care professional is seeing. The best treatment on average may not be the best treatment for a given patient. Moreover, a given patient may experience more important treatment harms than another patient.

Simple relative treatment effect modification is regularly assessed in trials using subgroup analyses, a one‐characteristic‐at‐a‐time approach. This article explains why such approaches to subgroup analyses could have limitations, and suggests an alternative method to evaluate relative treatment effect modification. In this article, we evaluate whether relative treatment effects of medication in 2 large contemporary trials are influenced by the baseline risk of an individual patient. This method can be used to assess treatment effect heterogeneity (ie, how the relative treatment effect varies across patients) in terms of both treatment benefit and treatment harm.

Subgroup Analyses: One at a Time?

Subgroup analyses, based on single patient characteristics, are frequently performed to assess differences in the relative treatment effect between groups of patients. However, there are major limitations to subgroup analyses in RCTs. , , , First, stratification and subsequent estimation of relative treatment effects within numerous subgroups results in a high risk of chance findings. , Second, one‐at‐a‐time subgroup analyses introduce a “reference class problem.” For example, if both age and sex are effect modifiers, which relative treatment effect measure is the “correct” one for a young woman? Furthermore, by selecting subgroups on >1 variable at a time (eg, by making subgroups based on both age and sex), a low number of end points in each subgroup would preclude reliable subgroup analyses.

A formal statistical interaction test between a patient characteristic of interest and the treatment allocation in a trial is more accurate than the estimation of subgroup‐specific treatment effects. However, a true interaction may not be detected as few trials are adequately powered for single variable‐treatment interaction analysis. At the same time, if there is no actual interaction effect, the probability of finding a false‐positive treatment interaction is 5% per tested characteristic, of which there are many. Therefore, if effect modification by a single factor can be expected, this should be prespecified in the trial design and taken into account in trial power estimation.

Assessment of Relative Treatment Effects by Baseline Risk

Baseline risk, the risk without the trial intervention, for a clinical outcome can be estimated using a risk model composed of multiple prognostic factors, either based on an existing risk model or derived in the trial itself. Assessment of relative treatment effect modification by individual baseline risk can be used as a method to assess treatment interactions. If no treatment effect modification is expected on the basis of previous data or biological mechanisms, a multivariable approach to relative treatment effect modification has important advantages over one‐at‐a‐time subgroup analyses. First, as this method does not rely on stratification into subgroups, sufficient power may be maintained to assess treatment effect differences in the study population. Second, it is possible that a combination of patient characteristics, rather than a single patient characteristic at a time, influences the treatment effect from an intervention. It is also possible that a patient characteristic not assessed in subgroup analysis contributes to relative treatment heterogeneity. As many patient characteristics are correlated, it may be possible that even if this variable is not included in the risk model, heterogeneity based on baseline risk is found. Third, a single multivariable test for treatment effect modification by baseline risk prevents chance findings attributable to multiple testing. Furthermore, this approach is based on the data from the trial itself, can be published with the main results from the trial, and, if relative treatment effect heterogeneity is apparent, this information facilitates clinical decision making. Finally, relative treatment effect heterogeneity not only pertains to efficacy but also to safety. Therefore, although no clear heterogeneity may be present on treatment efficacy in a trial, there may still be clinical relevant heterogeneity in treatment safety, with important consequences for individualized clinical decision making.

Statistical Analysis

The first step is to estimate baseline risk for individual patients, preferably with an existing risk score. If not all variables are collected in the studies, a new risk score needs to be derived. A time‐to‐event survival model (eg, a Cox proportional hazards model) is subsequently fitted with each participant’s linear predictor of the risk model using their patient characteristics (which determines an individual’s baseline risk) and treatment allocation as predictors. Potential relative treatment effect modification by baseline risk is assessed by adding the interaction term “treatment*linear predictor” to a Cox model. As an example, the following notation is used in R (where LP denotes the model linear predictor, and allocation the trial allocation):

> cph(Surv(follow.up,endpoint)~LP * allocation, data=data, x=T, y=T)

The models with and without interaction term are compared using the likelihood ratio test with a P<0.05 indicating a significant interaction. If a statistically significant effect modification is found, the next question is whether this statistical significant difference in relative treatment effect is clinically relevant.

By stratifying the study population into quartiles and estimating the relative treatment effect within these quartiles (quartile‐specific hazard ratio [HR] or relative risk reduction) and comparing these, the presence of relative treatment effect heterogeneity may be assessed. The stratification of the study population into quartiles is merely a way to study the change in relative treatment effect with differing baseline risk, and quartile‐specific HRs cannot be used in clinical practice.

Methods

Study Populations

Data were used from the SPRINT (Systolic Blood Pressure Intervention Trial) and the RE‐LY (Randomized Evaluation of Long‐Term Anticoagulant Therapy) trial. Because of the sensitive nature of the data collected for this study, requests to access the SPRINT data set from qualified researchers trained in human subject confidentiality protocols may be sent to the National Heart, Lung, and Blood Institute Biologic Specimen and Data Repository Information Coordinating Center at https://biolincc.nhlbi.nih.gov/. Requests to access the RE‐LY trial data set may be sent to www.clinicalstudydatarequest.com.

SPRINT (registration number: NCT01206062) included 9361 patients aged ≥50 years with systolic blood pressure of ≥130 mm Hg and an increased risk for cardiovascular disease. Eligible participants were randomized 1:1 to an intensive target systolic blood pressure of <120 mm Hg, or a standard target systolic blood pressure of <140 mm Hg. The study was ended prematurely on the basis of a reduced risk in the primary composite major adverse cardiovascular event end point at interim analysis. The RE‐LY trial (registration number: NCT00262600) included 18 133 patients with atrial fibrillation, and randomized them 1:1:1 to warfarin, dabigatran, 110 mg twice daily, or dabigatran, 150 mg twice daily. Most participants had an indication for oral anticoagulation therapy based on their Score for Atrial Fibrillation Stroke Risk. The RCT demonstrated that in patients with atrial fibrillation, dabigatran given at a dose of 110 mg twice daily was associated with a similar risk of stroke and systemic embolism (SE) compared with warfarin, while having lower rates of major bleeding. Dabigatran at a dose of 150 mg twice daily was associated with lower rates of stroke and SE and similar rates of major bleeding compared with warfarin. Detailed descriptions of both trials and inclusion and exclusion criteria have been published previously. , , , (Ethical) approval was obtained from the national regulatory authorities and ethical committees of the participating centers, and all participants provided written informed consent.

Individual Treatment Effect Estimation of Intensive Blood Pressure Lowering

Because existing risk scores, like the Framingham risk score, Systematic COronary Risk Evaluation risk chart, or atherosclerotic cardiovascular disease risk score, could not be used for risk estimation because of missing variables and different definitions of the primary outcome, , , a prediction model for the primary composite end point (myocardial infarction, other acute coronary syndromes, stroke, heart failure, or death from cardiovascular causes) was derived in the control arm of SPRINT using Cox proportional hazards analysis. Well‐known predictors were selected from previously published risk scores. , , , , Details on model development and model validation for estimation of the risk of the primary end point are presented in the appendix (Data S1 and Figure S1). First, baseline mortality risk was estimated by entering individual patient characteristics in the model formula. Using the method as described above, heterogeneity of treatment effect was then assessed using the linear predictor from the newly derived risk model.

Individual Treatment Effect Estimation of Dabigatran on Stroke and Major Bleeding in Atrial Fibrillation

Baseline 1‐year risk for the primary end point stroke and SE and for the risk of major bleeding was estimated using the externally validated Global Anticoagulant Registry in the Field–Atrial Fibrillation (GARFIELD‐AF) risk models. Details on the model derivation and end point definitions have been published previously and are summarized in the appendix (Data S2). Model performance was assessed with the c‐statistic (95% CI) for discrimination and with calibration plots of predicted versus observed risk (Figure S2). First, baseline risks of stroke/SE and major bleeding were estimated by filling in individual patient characteristics in the GARFIELD‐AF model formulas. Using the method described above, heterogeneity of treatment effect was then assessed using the linear predictors of the stroke/SE and major bleeding GARFIELD‐AF risk functions.

Results

Baseline Characteristics and Events

Baseline characteristics of both trials are shown in the Table. During a median follow‐up of 3.2 years (interquartile range, 2.7–3.8 years) in SPRINT, 319 end points occurred (6.8%) in those who received standard treatment, compared with 243 end points (5.2%) in patients who received intensive treatment. During a median follow‐up of 2.0 years in the RE‐LY trial (interquartile range, 1.6–2.4 years), 160 primary events (2.7%) and 426 major hemorrhages (7.1%) occurred in participants who received warfarin, compared with 295 primary events (2.4%) and 757 major hemorrhages (6.3%) in patients who received dabigatran in either dose.

Table 1

Baseline Characteristics of SPRINT and RE‐LY Trial Study Populations

Characteristics	SPRINT	RE‐LY trial
	(n=9361)	(n=18 113)
Age, y	68±9	71±9
Male sex	6029 (64)	11 514 (64)
Race or ethnicity
White	5399 (58)	12 616 (70)
Black	2802 (30)	176 (1)
Hispanic	984 (11)	879 (5)
Other *	176 (2)	4442 (24)
Current smoking	1244 (13)	5979 (33)
History of cardiovascular disease	1562 (17)	5248 (29)
Systolic blood pressure, mm Hg	140±16	131±18
Body mass index, kg/m2	30±6	29±6
Total cholesterol, mmol/L	4.9±1.1	4.7±1.1
LDL cholesterol, mmol/L	2.9±0.9	N/A
HDL cholesterol, mmol/L	1.4±0.4	N/A
Estimated glomerular filtration rate, mL/min per 1.73 m2	67±24	73±28
Uses a statin	4083 (44)	8057 (45)
Uses antihypertensives	8479 (91)	14 509 (80)

All data are shown as number (percentage) or mean±SD. HDL indicates high‐density lipoprotein; LDL, low‐density lipoprotein; N/A, not available; RE‐LY, Randomized Evaluation of Long‐Term Anticoagulant Therapy; and SPRINT, Systolic Blood Pressure Intervention Trial.

*Other indicates race or ethnicity self‐reported as non‐White, non‐Black and non‐Hispanic.

Baseline Risk and Treatment Effect Heterogeneity

In SPRINT, estimated absolute 3.2‐year risk for the primary end point with standard treatment (ie, baseline risk) varied widely, from 1% to 50% absolute risk (Figure 1C). There was no significant interaction between baseline risk for the primary end point and the relative treatment effect of intensive treatment in the trial (P for interaction=0.92). The baseline risk quartile‐specific HRs are shown in Figure 2.

Figure 1

Distribution of untreated (ie, baseline) risk of stroke/systemic embolism (SE) (A) and major bleeding (B) in the RE‐LY (Randomized Evaluation of Long‐Term Anticoagulant Therapy) trial and of the primary outcome in SPRINT (Systolic Blood Pressure Intervention Trial) (C).

 

Figure 2

Relative treatment effect in SPRINT (Systolic Blood Pressure Intervention Trial) of intensive versus standard blood pressure control in quartiles of baseline risk for the primary end point.

The blue dotted line denotes the overall trial hazard ratio.

In the RE‐LY trial, estimated baseline absolute 1‐year risk of stroke or SE, estimated with the GARFIELD‐AF risk model, varied from 0.1% to 23.5%, whereas the 1‐year risk of major bleeding varied from 0.3% to 13.9% (Figure 1A and 1B, respectively). There was no significant interaction between the baseline risk for the primary efficacy end point and relative treatment effect of either dabigatran, 110 mg, or dabigatran, 150 mg, compared with warfarin (likelihood ratio test P for interaction=0.71). There was, however, a significant interaction between baseline risk of major bleeding and the relative treatment effect of both dabigatran, 110 and 150 mg, compared with warfarin (likelihood ratio test P for interaction <0.001). The baseline risk quartile‐specific HRs for the primary end point and major bleeding are presented in Figure 3A and 3B, respectively. Quartile‐specific HRs ranged from 0.40 (95% CI, 0.26–0.61) to 1.04 (95% CI, 0.83–1.03) for dabigatran, 110 mg, and from 0.61 (95% CI, 0.42–0.88) to 1.20 (95% CI, 0.97–1.50) for dabigatran, 150 mg, compared with warfarin. Baseline characteristics for the quartiles of estimated baseline risk of major bleeding and the quartile‐specific HRs for the primary end point according to major bleeding quartiles are shown in Table S1 and Figure S3, respectively. Table S2 shows how randomization remains intact within the risk quartiles.

Figure 3

Relative effect in the RE‐LY (Randomized Evaluation of Long‐Term Anticoagulant Therapy) trial of dabigatran versus warfarin on the risk of stroke/systemic embolism (SE) in quartiles of baseline risk of stroke/SE, according to the Global Anticoagulant Registry in the Field–Atrial Fibrillation (GARFIELD‐AF) risk model (A), and major bleeding in quartiles of baseline risk of major bleeding, according to the GARFIELD‐AF risk model (B).

The blue dotted line denotes the overall trial hazard ratio.

Discussion

Conventionally, an individual patient’s expected absolute treatment effect is calculated from his/her baseline risk and the average relative treatment effect as observed in a clinical trial. However, the relative treatment effect may not be uniform across the trial population. The relative treatment effect may be dependent on an individual’s baseline event risk. In the current study, relative treatment effect heterogeneity by an individual’s baseline risk was assessed in 2 large randomized clinical trials. In SPRINT, there was no evidence for effect modification of the treatment effect from intensive versus standard blood pressure control on basis of baseline major adverse cardiovascular event risk estimated using a newly derived risk model. In the RE‐LY trial, no effect modification for the treatment effect of dabigatran versus warfarin by baseline risk based on the GARFIELD‐AF risk model was observed for the risk of the primary end point of stroke and systolic embolism. However, we did establish effect modification by baseline bleeding risk on the treatment effect for the risk of harm from major bleeding, with the lowest baseline bleeding risk quartiles having a clear benefit from dabigatran for both doses, whereas in the highest bleeding risk quartiles there is no lower bleeding risk from dabigatran compared with warfarin.

The consequence of this finding is that the average trial result from SPRINT can be applied in all patients. On the basis of the results of the RE‐LY trial, the conclusion is that a single relative treatment effect can be used in all patients for the effect of treatment on the primary efficacy end point, but the single relative effect on major bleeding may not be used in all patients. In individualized clinical decision making, from the efficacy standpoint, this may entail offering treatment to all patients, as all will benefit in accordance with their individual baseline risk in combination with the overall HR from the trial (absolute risk reduction). From the safety standpoint, however, when stratifying the study population in quartiles based on their baseline major bleeding risk (ie, the risk when treating with warfarin), there is a clinically important difference in the relative treatment effect in these quartiles. For dabigatran at a dose of 110 mg, a benefit of dabigatran over warfarin is observed in the lowest quartile of predicted bleeding risk, but not in the highest. For dabigatran at a dose of 150 mg, a benefit of dabigatran compared with warfarin, as observed in the lowest 2 quartiles of risk, is offset by a numerical, although just not statistically significant, detrimental effect in the quartile with the highest baseline bleeding risk. This may explain the neutral average main effect on major bleeding reported in the trial (HR, 0.93; 95% CI, 0.81–1.07). The example shows a potential pitfall in the assessment of heterogeneity of treatment effect; looking at efficacy of a treatment alone potentially gives incomplete information necessary for clinical decision making. Assessment of heterogeneity of treatment effects may be most useful if a treatment strategy is costly or confers harms, as physicians may consider withholding such treatment from patients with low expected benefit or important expected harm. Anticoagulation, with a risk of major bleeding, represents an example where this approach is clinically useful. For all therapies, potential treatment benefits should be weighed against potential harms from treatment. For example, a trial investigating strict blood pressure lowering in frail individuals may warrant investigation of treatment heterogeneity on possible adverse effects, such as falls and cognitive decline.

When effect modification is present, further analysis of the data and literature is necessary. In the case of the RE‐LY trial, effect modification by both renal function and age has been described in univariate subgroup analyses. , However, as discussed before, univariate subgroup analyses have disadvantages, including a limited power with a risk of false‐positive subgroup finding. Our multivariable approach also identified renal function and age as important factors. A reassuring finding is that interaction analysis between both age and renal function and treatment effect remain statistically significant in an exploratory analysis adjusting for potential confounders (including sex, systolic blood pressure, history of cardiovascular disease, and smoking status; data not shown), making it unlikely that these findings are false positive. The advantage of a multivariable risk‐based approach, as described in the current study, includes that the aforementioned “reference class problem” is avoided. For example, an 80‐year‐old patient has an increased risk for extracranial major bleeding with either doses of dabigatran compared with warfarin, according to univariate subgroup analyses. At the same time, an individual with a glomerular filtration rate ≥80 mL/min has a remarkably decreased risk of major bleeding, whereas there is a similar risk of bleeding between dabigatran and warfarin in patients with an estimated glomerular filtration rate <50 mL/min. Using just univariate subgroup analyses, it may be difficult to decide whether age or renal function is the more important factor influencing treatment response. A multivariable risk‐based approach is therefore more appropriate. This can lead to individual absolute treatment effects in terms of both benefit and harm, which can be weighed in clinical practice to make treatment decisions.

When heterogeneity in treatment effect is found, potentially no single characteristic may be identified that drives treatment effect modification. In that situation, a “one treatment fits all” approach does not apply and simple subgroup analyses do not solve the problem.

A risk model can then be used in clinical practice to determine whether an individual patient qualifies for therapy. For example, patients with atrial fibrillation with a low risk of major bleeding will likely be better off with dabigatran, 150 mg, as they will have a lower risk of major bleeding with dabigatran than with warfarin. At the same time, patients with a higher risk of major bleeding will likely be better off with dabigatran, 110 mg, or even warfarin, as they may have an increased risk of major bleeding with dabigatran, 150 mg, that offsets the reduction in the risk of stroke/SE that is uniform across all patients. The best balance between treatment benefit and treatment disadvantages may be difficult to directly estimate for individual patients based on risk estimation alone. Therefore, decision tools or online calculators should be developed or updated to include personalized treatment effect predictions in cases where risk predictions alone are insufficient for making treatment decisions because of heterogeneity of the treatment effects. These can then be used in daily practice for making informed individualized treatment decisions together with patients.

The advantage of using a preexisting, externally validated risk model, such as GARFIELD‐AF, compared with a newly derived risk model, is that it can be used in clinical practice for reliable, unbiased estimates of baseline risk. Furthermore, if relative treatment heterogeneity is present, an existing risk score is likely easier to implement in clinical practice to calculate individual absolute treatment effects for medical decision making. If possible, when designing a new trial, an existing risk model should be selected, so that the appropriate determinants can be collected at baseline. If no risk models exist for the prediction of the primary end point risk, it may be necessary to include the derivation of a new risk model in the trial design if a sufficiently large number of events for model development is anticipated. To deal with the shortcomings of traditional subgroup analyses, we propose that assessment of relative treatment heterogeneity using a multivariable approach should be a prespecified analysis for RCTs. We have shown, using the example from the RE‐LY trial, that this method is also applicable in trials with >2 treatment arms (eg, several doses).

There are several additional points to further consider when assessing relative treatment effect heterogeneity. First, when a relative treatment effect is independent of baseline risk, and thus, the same relative effect applies to all patients, the variation in absolute treatment effect may still be large if the range of baseline risk is large. On the other hand, if there is relative treatment heterogeneity, with a significant P value for interaction, this does not automatically imply an important treatment heterogeneity. Thus, both the assessment of relative treatment heterogeneity and the estimation of individual absolute treatment effects are of critical importance for the translation of clinical trial results to all patients in clinical practice.

There are limitations to the current method to consider. In the example of SPRINT, an internally derived risk model was used. External validation of this model should be performed before it is applied in clinical practice. However, for the assessment of a risk‐treatment interaction, an internally developed model is appropriate, when the internal model performance is sufficient. Thus, the presented model renders unbiased estimates of treatment effect across the spectrum of baseline risk. Furthermore, a possible limitation of the current method is that in trials with strict participant selection criteria, the heterogeneity in patient characteristics may be too small to detect treatment effect modification by baseline risk, and, in addition, the question is whether a risk model derived in a trial population can be generalized to the general population. A careful evaluation of the representativeness of study results is essential for the application of trial results to individuals in clinical practice.

Conclusions

In conclusion, for translation of an overall trial result into an individual’s expected absolute treatment effect, possible differences in relative treatment effect between patients should be taken into account. Effect modification of the relative treatment effect by individual baseline event risk should be assessed systematically in RCTs using multivariable risk prediction, not only in terms of treatment efficacy but also for important treatment harms, as a prespecified analysis. Relative treatment effects can then be translated more reliably to individual absolute treatment effects that can be weighed for individualized clinical decision making. By using individual patient data in a trial, relative treatment effect modification can be assessed across the full spectrum of risk.

Sources of Funding

None.

Disclosures

None.

Data S1–S2

Tables S1–S2

Figures S1–S3

References 19, 20

Click here for additional data file.