Accounting for competing risks in randomized controlled trials: a review and recommendations for improvement.

In studies with survival or time-to-event outcomes, a competing risk is an event whose occurrence precludes the occurrence of the primary event of interest. Specialized statistical methods must be used to analyze survival data in the presence of competing risks. We conducted a review of randomized controlled trials with survival outcomes that were published in high-impact general medical journals. Of 40 studies that we identified, 31 (77.5%) were potentially susceptible to competing risks. However, in the majority of these studies, the potential presence of competing risks was not accounted for in the statistical analyses that were described. Of the 31 studies potentially susceptible to competing risks, 24 (77.4%) reported the results of a Kaplan-Meier survival analysis, while only five (16.1%) reported using cumulative incidence functions to estimate the incidence of the outcome over time in the presence of competing risks. The former approach will tend to result in an overestimate of the incidence of the outcome over time, while the latter approach will result in unbiased estimation of the incidence of the primary outcome over time. We provide recommendations on the analysis and reporting of randomized controlled trials with survival outcomes in the presence of competing risks.

Introduction

Survival or time‐to‐event outcomes are common in randomized controlled trials (RCTs) 1. These are outcomes that are defined as the time (typically from randomization) until the occurrence of the event of interest. Examples include time from randomization to death due to any cause or time from randomization to death due to cardiovascular causes. An important issue in the analysis of survival data is the presence of competing risks. A competing risk is an event whose occurrence precludes the occurrence of the primary event of interest 2, 3, 4. For example, when the primary outcome is death due to cardiovascular causes, then death due to non‐cardiovascular causes serves as a competing risk, because subjects who die of non‐cardiovascular causes (e.g., death due to cancer) are no longer at risk of death due to a cardiovascular cause. However, when the primary outcome is all‐cause mortality, then competing risks are absent, as there are no events whose occurrence precludes the occurrence of death due to any cause.

A unique aspect of survival data is the presence of censoring. At the end of the trial, the event of interest may not yet have occurred for all subjects, with some subjects still being at risk for the occurrence of the event of interest. Such subjects are said to be censored: follow‐up has terminated and all that is known about the time of the occurrence of the event is that it exceeds the duration of follow‐up. Implicit in the concept of censoring is the understanding that if follow‐up had been prolonged, the event of interest would eventually be observed to occur for all subjects. Conventional statistical methods for the analysis of survival data assume that censoring is non‐informative. Non‐informative censoring means that the occurrence of censoring provides no information on the subject's prognosis 5. A related issue that arises in RCTs is that of loss to follow‐up and study dropout 6, 7. Lost to follow‐up means that the subject is no longer available for ascertaining whether the outcome has occurred despite the trial having not yet been terminated (e.g., the subject no longer attends regularly scheduled clinic appointments; the subject moves away from the location in which the trial is being conducted). Subjects who are lost to follow‐up are often analyzed as though they were censored at the date of last contact 6. However, this analysis requires the strong assumption that being lost to follow‐up provides no information about prognosis. It is important to highlight that a competing risk is different than censoring or lost to follow‐up. Using the example from the previous paragraph, if a subject dies of a non‐cardiovascular cause (the competing risk), then the subject is no longer at risk of death due to a cardiovascular cause. The subject has not been censored in the sense defined previously because extending the duration of follow‐up will not result in the subject being observed to die of a cardiovascular cause (as they have already died of a non‐cardiovascular cause). This distinction is important, as competing risks are often mistakenly conceptualized as censoring events.

In the absence of competing risks, the complement of the Kaplan–Meier function can be used to estimate incidence of an outcome over time, assuming the censoring events are independent of the event of interest. However, in the presence of competing risks, this method results in biased estimates of incidence even under the assumption that the competing events are independent of one another (assuming that competing risks are independent of one another implies that knowledge of the likelihood of the occurrence of one type of event provides no information about the likelihood of the occurrence of the other type of event). Moreover, the assumption of independent competing risks is very strong and untestable, and is unlikely to be warranted in most applications (e.g., subjects at increased risk of cardiovascular death are likely to be at increased risk of death due to non‐cardiovascular causes as age is a common risk factor for both events). In the presence of competing risks, cumulative incidence functions (CIFs) should be used to estimate the probability of the occurrence of the event over time. Furthermore, Gray's test should be used to test the equality of CIFs between treatment groups 8 (Gray's test is the competing risk analogue to the log‐rank test that is used for testing the equality of Kaplan–Meier survival curves between groups).

When considering regression models for survival data, the use of the Cox proportional hazards model is ubiquitous in biomedical and epidemiological applications in the absence of competing risks. However, in the presence of competing risks, investigators must choose between two different hazard‐based regression models: estimating the effect of covariates on the cause‐specific hazard function and estimating the effect of covariates on the subdistribution hazard function (or on the CIF). The former estimates the effect of covariates on the rate of occurrence of the event of interest in subjects who are currently event‐free (and thus at risk of occurrence of an event of any type). The latter estimates the effect of covariates on the probability of the occurrence of the event of interest. We refer the interested reader to a number of introductory tutorials on statistical methods for competing risks 2, 3, 4, 9.

The objective of this review was twofold. First, to conduct a review of reports of RCTs in which the primary outcome was survival in nature. In this review, we determined the proportion of studies that were potentially susceptible to competing risks and examined whether appropriate statistical methods were used to account for the presence of competing risks. Second, we provide recommendations about appropriate statistical methods for analyzing RCTs in which competing risks are present. The paper is structured as follows: In Section 2, we describe measures of effect for use with RCTs with survival outcomes. In Section 3, we describe and report on our review of RCTs in the general medical literature in which the primary outcome was time‐to‐event in nature. In Section 4, we provide recommendations for the analysis and reporting of such trials. Finally, in Section 5, we summarize our findings and place them in the context of the existing literature.

Measures of effects in randomized controlled trials with survival outcomes

In this section, we briefly review measures of effect used in RCTs with survival outcomes in the absence of competing risks. To motivate this review, we begin by reviewing measures of effect for use with binary outcomes.

Measures of effect in randomized controlled trials with binary outcomes

When outcomes are binary, there are four possible measures of effect that can be reported: the risk difference (or absolute risk reduction), the relative risk (or relative risk reduction), the number needed to treat (NNT — the reciprocal of the risk difference), and the odds ratio. The CONSORT statement recommends that for RCTs with dichotomous outcomes, both relative and absolute measures of treatment effect be reported 10. Furthermore, the BMJ requests that in RCTs with binary outcomes the event rate in each randomization group, the relative risk reduction, and the NNT or number needed to harm be presented in the abstract 11. Laupacis et al. have suggested that the NNT is the most important measure of effect to inform clinical decision making 12. The importance of reporting both relative and absolute measures of effect has been highlighted by different authors 13, 14.

Measures of effect in randomized controlled trials with survival outcomes

Kaplan–Meier survival curves are frequently reported in RCTs with time‐to‐event outcomes 1. Altman described a method for estimating the NNT in RCTs using information derived from the estimated Kaplan–Meier survival curves 15. Cox proportional hazards regression models are often used in RCTs with survival outcomes to estimate the relative effect of the intervention on the rate of the occurrence of the event of interest 1, 16. These two approaches are complementary: They allow for summarizing the absolute and relative effect of the intervention on the risk of the occurrence of the outcome. In keeping with the spirit of the CONSORT statement concerning RCTs with binary outcomes, both of these measures of effect should be reported in RCTs with survival outcomes.

Review of the analysis of competing risks in randomized controlled trials with survival outcomes

Search strategy

We conducted a systematic review of reports of RCTs published in the general medical literature to identify the frequency of trials in which there was the potential for the presence of competing risks and whether appropriate statistical methods were employed to account for these competing risks. We restricted our attention to four high‐impact general medical journals: BMJ (formerly the British Medical Journal), the Journal of the American Medical Association, The Lancet, and The New England Journal of Medicine. We restricted our focus to articles published in the last 3 months of 2015. Using a search strategy for identifying RCTs that was described by Royle and Waugh 17, we used the following search strategy on PubMed:

(“Randomized Controlled Trial”[Publication Type]) AND (“2015/10/01”[PDAT]: “2015/12/31”[PDAT]) AND ((“Lancet”[Journal]) OR (“BMJ” [Journal]) OR (“JAMA”[Journal]) OR (“The New England journal of medicine”[Journal]))

Our search identified 122 reports of RCTs published in the last 3 months of 2015 in these four journals. A manual search of the 122 articles was conducted to identify those in which the primary (or co‐primary) outcome was time‐to‐event in nature. This resulted in the inclusion of 42 (34%) of the 122 articles. In two of these studies, the primary outcome was a time‐to‐event outcome that was recurrent in nature. As different statistical methods are used for the analysis of recurrent outcomes, we excluded these two articles, leaving 40 articles for analysis (BMJ: one article; Lancet: 17 articles; Journal of the American Medical Association: five articles; New England Journal of Medicine: 17 articles). In a few included studies, the primary outcome was analyzed both as a binary outcome and a survival or time‐to‐event outcome (e.g., death within 1 year of randomization was treated both as a binary outcome and as a time‐to‐event outcome).

We examined these 40 articles to abstract the following information from each article: (i) was the primary outcome all‐cause mortality or was it a composite outcome that included all‐cause mortality as one of its components; (ii) were Kaplan–Meier survival curves (or their complement) estimated in each of the randomization groups; (iii) were absolute differences in survival probabilities between randomization groups reported; (iv) was a Cox proportional hazards regression model fit to the data; (v) was the NNT reported (vi) were cumulative incidence functions estimated in each of the randomization groups; (vii) was a Fine‐Gray subdistribution hazards model fit to the data.

Results

The results of the literature review are summarized in Table 1. In nine (22.5%) of the 40 RCTs with a time‐to‐event primary outcome, the outcome was either all‐cause mortality or was a composite outcome of which all‐cause mortality was one of the components. Of these 40 studies, 33 (82.5%) reported a Kaplan–Meier analysis, while 36 (90%) reported the results of a Cox proportional hazards regression model. Nine (22.5%) studies reported the absolute difference in event probabilities at a given duration of follow‐up, and only three (7.5%) studies reported a NNT.

Table 1

Results of literature review of the handling of competing risks in randomized controlled trials.

40 studies in which the primary outcome was time‐to‐event in nature
The primary outcome was all‐cause mortality or a composite outcome of which all‐cause mortality was a component	9 (22.5%)
Used Kaplan–Meier survival curves	33 (82.5%)
Used a Cox proportional hazards model	36 (90.0%)
Reported the absolute different in event probabilities at a given duration of follow‐up	9 (22.5%)
Reported an NNT	3 (7.5%)
31 studies that were potentially susceptible to competing risks (all‐cause mortality was not part of the outcome definition)
Used Kaplan–Meier survival curves	24 (77.4%)
Used cumulative incidence functions	5 (16.1%)
Reported the absolute different in event probabilities at a given duration of follow‐up	9 (29.0%)
Reported an NNT	3 (9.7%)
Used a Cox proportional hazards model (cause‐specific hazard model)	28 (90.3%)
Used a Fine‐Gray subdistribution hazard model	3 (9.7%)
9 studies that reported the absolute different in event probabilities at a given duration of follow‐up
Did not appear to account for competing risks when estimating the absolute difference in proportions	6 (66.7%)
3 studies that reported an NNT
Did not appear to account for competing risks when estimating the NNT	2 (66.7%)

NNT, number needed to treat.

We then restricted our focus to those 31 (77.5%) studies in which all‐cause mortality was not part of the outcome definition. These 31 studies are potentially susceptible to competing risks, as the occurrence of death (when death was not part of the outcome definition) or death due to a different cause (when cause‐specific death was part of the outcome definition) would preclude the occurrence of the primary outcome. In these 31 studies that were potentially susceptible to competing risks, 24 (77.4%) reported the results of a Kaplan–Meier survival analysis, while only five (16.1%) reported using CIFs to estimate the incidence of the outcome over time in the presence of competing risks. Nine (29.0%) studies reported the absolute difference in event probabilities at a given duration of follow‐up, and only three (9.7%) reported an NNT. Of the nine studies that reported an absolute difference in event probabilities at a given duration of follow‐up, six (66.7%) did not appear to use statistical methods that accounted for the presence of competing risks. Of the three studies that reported an NNT, two (66.7%) did not appear to account for the presence of competing risks when estimating the NNT. Twenty‐eight (90.3%) studies reported the results of a Cox proportional hazards model, which in the given context was equivalent to a cause‐specific hazard model for the primary outcome. Finally, three (9.7%) studies reported using a Fine‐Gray subdistribution hazard model.

Recommendations for analyzing randomized controlled trials in the presence of competing risks

We provide the following recommendations when analyzing RCTs in which the primary outcome is survival or time‐to‐event in nature and is susceptible to competing risks.

First, when estimating the incidence of the outcome over time, investigators should use CIFs, rather than Kaplan–Meier survival curves. Furthermore, Gray's test should be used to test the equality of the CIFs between randomization groups 8. The use of Kaplan–Meier survival curves in the presence of competing risks results in estimates of incidence that are biased upwards, even under the very strong and untestable assumption that the competing events are independent of one another 2, 3, 4, 18. The use of CIFs allows for unbiased estimation of the probability of the occurrence of the event of interest over time. One of the problems with the Kaplan–Meier estimator is that it estimates the probability of the occurrence of the event of interest in the absence of competing risks, which is generally larger than that in the presence of competing risks. Furthermore, the hypothetical population in which competing risks do not exist may not be the population of clinical interest 19. A consequence of the biased estimation of the probability of the occurrence of the event over time is that absolute risk differences derived from Kaplan–Meier estimates may be biased as well. Consequently, clinically useful measures of effect such as the NNT may be biased. We recommend that authors use a method described by Gouskova et al. for estimating the NNT (and the absolute risk reduction) in the presence of competing risks 20. This approach is easy to implement and only requires estimation of the CIFs.

Second, when estimating the effect of the intervention on the hazard of the occurrence of the event in the presence of competing risks, analysts must choose between two different hazard functions: the cause‐specific hazard function and the subdistribution hazard function 3, 4, 21. The use of a cause‐specific hazard model allows one to estimate the effect of the intervention on the instantaneous rate of occurrence of the event of interest in subjects who are currently event‐free. The use of a subdistribution hazard model allows one to estimate the relative effect of the intervention on the cumulative incidence function. Lau et al. suggest that the former is best suited to questions of etiology, while the latter is best suited to settings in which absolute estimates of the probability of the occurrence of the event are required 3. A cause‐specific hazard model can be fit by using conventional statistical software for fitting the Cox proportional hazards model and treating subjects who experience the competing event as censored at the time that the competing event occurred. While epidemiological studies are clearly focused on etiology, it can be argued that RCTs are focused on the absolute (as opposed to relative) effects of treatment, as evidenced by the emphasis on absolute event rates, absolute risk differences, and NNTs (which quantify absolute and not relative effects). The use of subdistribution hazard models in RCTs allows one to estimate the absolute incidence of the primary outcome for different covariate patterns, as well as the relative effect of treatment on the cumulative incidence function. A more comprehensive and scientifically rigorous approach that may also be considered involves fitting both cause‐specific hazard and subdistribution hazard models as this would permit a more detailed and complete understanding on the effect of the intervention on the risk of the occurrence of the outcome of interest 22.

An example of an etiological question is the effect of smoking on lung cancer, treating death due to other causes as a competing event. The cause‐specific hazard model allows one to estimate the effect of smoking on the instantaneous hazard (which is analogous to the instantaneous rate) of lung cancer in subjects who are currently event‐free (who are currently alive and have not developed lung cancer). A subdistribution hazard model would allow an investigator to estimate the effect of smoking on the probability of developing lung cancer over time. One could use a subdistribution hazard model if one wanted to estimate the probability of developing lung cancer over different durations of follow‐up for different covariate patterns. In particular, researchers developing clinical prediction models in the presence of competing risks should use the subdistribution hazard model 23.

Discussion

In our review of RCTs published in four high‐impact general medical journals, we found that survival outcomes are common, with about one third of RCTs having a primary outcome that was survival in nature. Of these studies, over three quarters (77.5%) were susceptible to competing risks. However, in the majority of studies, the potential presence of competing risks was not accounted for in the statistical analyses that were described. A large majority (77.4%) reported the results of a Kaplan–Meier survival analysis, while only five (16.1%) reported using CIFs to estimate the incidence of the outcome over time in the presence of competing risks.

As noted previously and elsewhere, the use of the Kaplan–Meier method to estimate the incidence of an event over time results in biased estimation in the presence of competing risks even if the competing events are independent of one another. The Kaplan–Meier estimator provides estimates that may be valid under this assumption for event probabilities in the absence of the competing risks, which is of questionable relevance in RCTs. However, if the bias is of the same magnitude and direction in the different randomization groups, then these biases will cancel one another out, resulting in an unbiased estimate of the absolute risk reduction and NNT at specific durations of follow‐up 20. As noted by Gouskova et al., if the intervention affects both the event of interest and the competing event, then the magnitude (and direction) of the bias in the Kaplan–Meier estimate may differ substantially between randomization groups. As the effect of the intervention on the competing event is typically not known, a conservative statistical practice would be to use methods described by Gouskova et al. that are based on the CIF to estimate the absolute risk reduction and the associated NNT 20.

Van Walraven and McAlister evaluated a random selection of 100 studies that reported at least one Kaplan–Meier analysis and that were published in a set of core medical journals 24. They found that 46% of studies reported a Kaplan–Meier analysis that was susceptible to bias due to competing risks. In 37.5% of studies that provided the number of primary events and competing events, they estimated that the bias in the Kaplan–Meier estimator was at least 10%. Similarly, Koller et al. reviewed 50 clinical studies performed in frail individuals susceptible to competing risks and that were published in high‐impact clinical journals and found that competing risks issues were present in 70% of articles 25.

Our study had certain limitations. We classified a study as being susceptible to competing risks if all‐cause mortality was not a component of the outcome. However, it was difficult to determine the extent to which an individual trial was actually affected by competing events. Many articles did not report on the frequency of competing events. In general, trials with longer durations of follow‐up would be more likely to be susceptible to competing risks compared with trials with short durations of follow‐up. Furthermore, studies in older or more acutely ill patients may be more susceptible to competing risks compared with trials conducted in younger or healthier patient populations. However, the methods recommended earlier for the analysis of trials in which competing risks are present can all be used regardless of the magnitude of the incidence of the competing events.

A question that we have not addressed is whether any occurrence of competing events is of importance or how large the proportion of observed events that are competing events must be for a conventional statistical analysis to result in biased estimation. We are reluctant to provide guidelines on how frequent competing events must be in order for the analyst to be required to employ methods that formally account for these competing risks. We suspect that an answer would be complex, and would depend on the absolute incidence of the primary event of interest, the difference in the incidence of the primary event of interest between treatment groups (i.e., the effect of treatment), and the difference in the incidence of the competing event between treatment groups (i.e., did the intervention affect the incidence of the competing event). Furthermore, we suspect that in some scenarios, ignoring the presence of an infrequently occurring competing event could result in a meaningful change in estimated NNT. Such questions should be addressed in subsequent research.

Additional observations from our review of RCTs in four high‐impact general medical journals warrant mention. Of those studies with a primary outcome that was survival in nature, only 22.5% reported the absolute difference in event probabilities at a given duration of follow‐up, and only 7.5% reported a NNT. Thus, only a small minority of studies reported information that is considered important for fully informed clinical decision making.

In summary, we found that a large proportion of RCTs with survival outcomes are potentially susceptible to competing risks. In only a minority of these studies are appropriate statistical methods employed. We provide suggestions for the analysis and reporting of RCTs with survival outcomes in the presence of competing risks.