Translating questions to estimands in randomized clinical trials with intercurrent events.

Intercurrent (post-treatment) events occur frequently in randomized trials, and investigators often express interest in treatment effects that suitably take account of these events. Contrasts that naively condition on intercurrent events do not have a straight-forward causal interpretation, and the practical relevance of other commonly used approaches is debated. In this work, we discuss how to formulate and choose an estimand, beyond the marginal intention-to-treat effect, from the point of view of a decision maker and drug developer. In particular, we argue that careful articulation of a practically useful research question should either reflect decision making at this point in time or future drug development. Indeed, a substantially interesting estimand is simply a formalization of the (plain English) description of a research question. A common feature of estimands that are practically useful is that they correspond to possibly hypothetical but well-defined interventions in identifiable (sub)populations. To illustrate our points, we consider five examples that were recently used to motivate consideration of principal stratum estimands in clinical trials. In all of these examples, we propose alternative causal estimands, such as conditional effects, sequential regime effects, and separable effects, that correspond to explicit research questions of substantial interest.

INTRODUCTION

Defining, interpreting, and identifying causal effects in the presence of an intercurrent (post‐treatment) event is not straightforward, even in a randomized controlled trial (RCT). It is well‐known that randomization of a baseline treatment does not ensure identification of estimands that are (implicitly or explicitly) defined conditional on a post‐treatment variable. Moreover, a naive contrast of outcomes conditional on the post‐treatment variable does not have a clear causal interpretation.

In this article, we discuss key issues concerning the formulation and choice of a causal estimand in settings with intercurrent events. Our first message is that an explicit research question should always precede the choice of estimand: it is the question that motivates the estimand, not the estimand that motivates the question. While this point may seem obvious and has been pointed out several times in the causal inference literature, numerous studies fail to give such a motivation. As a result, the interpretation—and practical relevance—of these analyses is ambiguous. Furthermore, an implication of this message is that the choice of estimand critically depends on the context; an estimand that is relevant in one study may be irrelevant in another.

To illustrate the importance of carefully defining a causal estimand, we revisit five recent examples inspired by the International Council of Harmonization (ICH) E9 (R1) addendum. These recently published guidelines stress the importance of choosing a treatment effect (estimand) in a clinical trial that is well aligned with the clinical question of interest, with specific attention given to the handling of intercurrent events. As will become clear in Sections 3 to 5, the intercurrent events in these examples do not prevent a classical intention‐to‐treat (ITT) effect from being identified. However, decisions makers and drug developers will often be interested in additional questions that translate to different estimands. We argue that the causal estimands of interest in these secondary analyses should be selected on a case‐by‐case basis.

Furthermore, we clarify that the translation of a research question into a formal estimand is separate from the question of its identification, that is, whether and how it can be expressed as a functional of distributions of observables. Establishing plausible conditions for identification can be considered a distinct—but important—task, which should be carefully conducted after choosing the causal estimand. The final (distinct) task is estimation, for which one may choose from more or less parametric or robust approaches. In other words, we find it helpful to distinguish between the following three tasks of data analyses: (i) translation of the research question into a formal causal estimand, (ii) assessing conditions for identification of the causal estimand, and (iii) estimation of the causal estimand from observed data. In this article, we will focus on task (i), but also briefly consider tasks (ii) and (iii).

We have structured our arguments as follows. In Section 2, we briefly introduce the intercurrent event settings in the five clinical examples from Bornkamp et al. In Section 3, we describe estimands that have been suggested for causal inference in clinical settings with intercurrent events. In Section 4, we revisit the clinical examples and map the subject matter questions—as described in plain English by the investigators—to causal estimands. In Section 5, we review conditions that allow us to identify these estimands in an RCT, followed by a brief description on how to estimate them in Section 6. In Section 7, we give a discussion.

CLINICAL EXAMPLES

(Multiple sclerosis) Multiple sclerosis (MS) is a progressive neurological disease. Initially most patients have a phase with relapses followed by recoveries, and eventually the patients transform to a secondary phase with less frequent relapses. There is major interest in developing new treatments that delay or prevent disease progression. For example, the EXPAND study was a randomized clinical trial that assigned the drug siponimod vs placebo to patients in the secondary phase, where the primary estimand was the onset of confirmed disability progression. Siponimod was shown to delay the onset of disability progression compared to placebo, and it was also shown to reduce the frequency of relapses. These primary results raised the question whether the treatment could affect disability progression outside of its effects on relapses. , Here, experiencing relapses is an intercurrent event.

(Treatment effects in early responders) A biomarker is a variable that quantifies a biological state. Treatment effects on biomarkers, such as high sensitivity c‐reactive protein, can serve as early predictors of treatment effects on clinical outcomes. The ASA/EFSPI oncology estimand working group further suggested that “biomarkers or early readouts can be useful to investigate whether an investigational medicine works as intended on a biological level.” Thus, a motivation for studying biomarker responses in RCTs seems to be elucidation of causal mechanisms (on a biological level), which can indicate whether the drug acts as intended through particular causal pathways. Here, reaching a biomarker threshold is an intercurrent event.

(Impact of exposure on overall survival) In drug trials, individuals who are given the same dose of a drug can still have different concentration of the active substance of this drug in the blood (serum). The concentration of the active substance can be important for the treatment effect. An RCT that assigned Trastuzumab for gastric cancer showed that those patients in the quartile with the lowest drug concentration had worse overall survival (OS) compared to the other quartiles. Researchers at the Food and Drug Administration subsequently raised the question “whether the lower OS is due to low drug concentration or to disease burden” (see also References 1 and 8). Here, the drug concentration is considered to be an intercurrent event.

(Antidrug antibodies for targeted oncology trials) Immunotherapies are increasingly used to treat several cancers. Yet, there is concern that some immunotherapies can trigger the production of antidrug antibodies (ADAs), which in turn can reduce the treatment effect of the immunotherapies on, say, overall survival. Indeed, “ADAs may be directed against immunogenic parts of the drug and may affect its efficacy or safety, or they may bind to regions of the protein which do not affect safety or efficacy, with little to no clinical effect.” Here, the production of ADAs is an intercurrent event.

(Prostate cancer prevention) There is major interest in developing drugs that prevent development of diseases, such as cancer. In particular, finasteride was shown to reduce the rates of prostate cancer in an RCT. While there were lower rates of cancer in the finasteride arm, those who developed cancer in the finasteride arm had, on average, more aggressive cancers than those assigned to placebo. This conditional association, however, does not necessarily have a causal interpretation—it could just be that the more aggressive cancers cannot be prevented by finasteride while mild ones can be prevented. Thus, after establishing that finasteride reduces the rate of prostate cancer, Bornkamp et al suggested a secondary analysis to assess whether “the effect of finasteride on the severity of prostate cancer among those men who would be diagnosed with prostate cancer regardless of their treatment assignment.” Here, being diagnosed with prostate cancer is an intercurrent event.

Common characteristics of intercurrent events in Examples 1 to 5

In Examples 1 to 5, the intercurrent event does not render the outcome of interest ill‐defined. That is, we can evaluate the effect of siponimod vs placebo on disability progression in the entire study population in Example 1, whether or not patients have relapses. Similarly, we can evaluate the effects of canakinumab on MACE (Example 2), Trastuzumab on gastric cancer (Example 3), immunotherapies on cancer progression (Example 4), and finasteride on prostate cancer incidence (Example 5), whether or not individuals experience the intercurrent events. Thus, we can study total effect estimands in all of the examples, but nevertheless there may be interest in endpoints that take into account the intercurrent events.

Furthermore, we do not know about any intervention that fixes the intermediate event in Examples 1 to 5. In particular, we do not have any current or future treatment that fixes MS patients to relapse or not relapse.

Thus, Examples 1 to 5 have two important characteristics: (i) we cannot conceive plausible interventions on the intercurrent events, and (ii) the occurrence of the intercurrent event does not render the outcome of interest ill‐defined. Other intercurrent events do not necessarily share these characteristics. For example, Michiels et al studied treatment effects in clinical studies where patients sometimes received rescue medications in response to worsening of a disease. Here, the use of rescue medication was considered to be an intercurrent event, and it is easy to conceive an intervention to give or reject rescue treatment. Furthermore, some outcomes are often considered to be ill‐defined in the presence of an intercurrent event. For example, many researchers consider quality of life to be ill‐defined after death, where death can be considered to be an intercurrent event. Whether or not events render the outcome ill‐defined for certain individuals and are interveneable, has implications for the practical relevance and plausibility of estimands, as we discuss in more detail in Section 3 and Appendix B.

NOTATION AND ESTIMANDS FOR SETTINGS WITH INTERCURRENT EVENTS

Average total effects

The primary estimand in most randomized trials is the (average) total effect of treatment assignment. This is called the treatment policy strategy estimand in the ICH addendum, and it coincides with the ITT effect in case of nonadherence. For simplicity, we will consider average effects, often on the additive scale, in the remainder of this article. Such averages of individual level effects are often the most relevant for decision making at this point in time and have a clear causal interpretation. However, our conceptual points are also valid for other causal contrasts.

To formally define this estimand, consider a study where individuals are randomly assigned to a binary treatment at baseline; while, strictly, it is the assignment that is randomized, we will simply speak of the randomized treatment in the following. Let be the outcome of interest measured at a fixed time , and let be an indicator of an intercurrent event that occurs at a time such that . Thus, for the ease of exposition and to align with recent considerations of intercurrent events, , , we have simplified and to be time‐fixed variables, and we suppose that the event of interest occurs after the intercurrent event time . However, most of our arguments can be extended to settings where the intercurrent event and the outcome are time‐varying, for which theoretical results exist. , , , , , , A consequence of our time fixed set‐up is that the outcome of interest must be defined after the intercurrent event.

A causal DAG that is consistent with our setting is show in Figure 1A, where randomization of ensures there are no common causes of and . Let superscripts denote potential outcomes, such that and denote the potential outcome of interest and the post‐treatment event, respectively, had an individual, possibly contrary to the fact, been assigned , where . The average total effect of the treatment on the outcome is a contrast

FIGURE 1

Causal DAGs that describe a randomized trial where baseline treatment is randomly assigned (A), and a sequential randomized trials where and are randomly assigned, where the assignment of depends on the intercurrent event . The intercurrent event and the outcome of interested may be affected by common causes

The average total effect (1) compares the average outcome in the population had everyone been treated () vs not treated (); it ignores any intercurrent events. In all of the examples of Section 2, the average total effect of treatment assignment is well defined and meaningful. It describes the effect of assigning siponimod vs placebo on disability (Example 1), canakinumab vs placebo on MACE (Example 2), immunotherapies vs placebo on cancer progression (Example 3), Trastuzumab vs placebo on gastric cancer in patients receiving chemotherapy (Example 4), and finasteride vs placebo on prostate cancer incidence (Example 5), without considering any intercurrent events. The intercurrent events in these examples are such that the treatment/control and the outcome remain meaningful with or without the intercurrent event. In other situations this total effect will not necessarily be of interest, as discussed in the ICH addendum, such as “discontinuation of assigned treatment, use of an additional or alternative treatment, drop‐out and terminal events such as death” (SectionA.1) (See also References 19 and 20). Nonadherence, or changes in treatment such as rescue treatment, may imply a substantial modification of intended treatment which is why an ITT analysis is sometimes regarded as unsatisfactory; while competing events or drop‐out may make the outcome impossible to occur or be measured.

Consider now a setting where treatments are sequentially given at multiple times, which motivates the study of sequential treatment effects. , For example, let be an immunotherapy, and suppose that the doctor every week (sequentially) recommends whether a patient should initiate, continue or discontinue the therapy. To fix ideas, suppose that treatment can be given at baseline (time 0) and one subsequent point in time (time 1), and let and be treatment indicators at these times (Figure 1B). The causal effect comparing a sequential treatment strategy that fixes and then vs an alternative strategy that fixes and then is where . Importantly, the sequential trial estimand is often of interest, even in clinical trials that assign a single treatment strategy at baseline. For example, formal (causal) definitions of per protocol estimands, which arguably are useful in a range of practical settings such as pragmatic trials, , often require specification of sequential treatment regimes: the per protocol estimands evaluate the effect of taking treatment, as described in the protocol, at every point in time vs not taking treatment at every point in time. Alternatively, we might consider the effect of treatment received at baseline, if no one (or everyone) received treatment subsequently. The hypothetical strategy estimand referred to in the ICH addendum can thus also be viewed as a sequential trial estimand.

Estimand (2) is a special case of a broader class of so‐called dynamic sequential treatment regimes. These regimes are called dynamic, because the sequential treatment decisions can depend on each patient's previous treatment and other characteristics. In other words, the treatment decision at time can be a function of (time‐varying) covariates up until time , and these covariates can include the intercurrent event status and treatment that have been received previously. For example, a decision of continuing a medical treatment at a time may depend on the clinical history up until time , previous treatments that were received and potential side‐effects.

Let be a regime that fixes the treatment at two points in time: First is set to and subsequently is set to where is a deterministic function of the first treatment and the intercurrent event : that is, a regime where the treatment at time 1 depends on whether the patient received the treatment at time 0 and the status of the intercurrent event before time 1. For example, suppose is defined such that only individuals who received treatment at time 0 and did not experience the intercurrent event will receive treatment at time 1.

We can define a contrast of two regimes ,

The total effect (1) and the sequential trial estimands (2) and (3) do not quantify the mechanisms by which the treatment affects the outcome . In particular, these estimands neither quantify effects conditional on nor mediated through the intercurrent events. On the other hand, these estimands are immediately relevant for designing practically feasible treatment regimes for the existing treatment , which can be given at different time points. Clinicians and patients will be particularly interested in the optimal regime that leads to the most favorable expected clinical outcome, that is,

The average total effects (1) and the sequential trial estimands (2) and (3) compare (counterfactual) outcomes under different treatment assignments in the entire study population. These estimands can easily be modified to average effects conditional on (functions of) observed pretreatment variables , such as and we discuss such conditional effects in more detail in Section 4. Alternatively, we can define conditional causal effects of the second treatment, given the first treatment and other events before time . The causal effect of assigning vs among those who received baseline treatment and had intercurrent event status is The estimand (5) only quantifies effects of the second, but not the first, treatment assignment. Furthermore, this estimand is restricted to the subpopulation that had intercurrent event status s under treatment .

Principal stratum effects

The term principal stratification was coined by Frangakis and Rubin as follows: “Principal stratification with respect to a posttreatment variable is a cross‐classification of subjects defined by the joint potential values of that posttreatment variable under each of the treatments being compared.” This idea was introduced by Robins when he considered counterfactual outcomes in individuals who would survive, regardless of treatment assignment. To fix ideas about principal stratum effects in a clinical trial setting, consider a randomized trial where a binary treatment is assigned at baseline. Let be the outcome of interest measured at a fixed time , and let be an indicator of a post‐treatment event defined at a fixed time , where . The additive principal stratum effect in the stratum defined by is Note that there are joint potential outcomes in the conditioning set of (6); this effect is defined in the subset of individuals characterized by the counterfactual intercurrent events and . In the example of a vaccine trial (suggested in the ICH addendum ), one might consider the treatment effect in those who would be infected regardless of vaccine assignment (say, ). Because this subpopulation has the same intercurrent event regardless of treatment, the principal effect can be interpreted as both a direct effect of the vaccine (outside of infection) and a total effect.

The fact that (6) is defined with respect to joint potential outcomes has raised concern in the causal inference literature. , , , , , , , In general, these joint potential outcomes cannot be observed in the same individual, and therefore the principal stratum effects are defined in an unobserveable subpopulation that may not even exist, although it may sometimes be possible to obtain informative bounds on the size of the principle strata. Furthermore, identifying the (members of) principal strata are plausible in certain special settings, in which the principal stratum estimands correspond to questions of clinical interest. , ,

The ICH addendum uses a broader definition of principal stratum effects, which encompasses estimands that condition on combinations (unions) of principal strata. In particular, the contrast where the conditioning set is the union is, in addition to (6), included in the ICH definition of principal stratum estimands. The conditioning set in (7), that is, the union of principal strata, can be identified from a randomized trial. For example, in a vaccine trial we can look at the subpopulation in the vaccine arm who were infected. Yet, it does not necessarily express a direct effect. In particular, (7) possibly quantifies effects both through and outside of the intercurrent event unless it is assumed that the treatment does not affect .

Causal effects that reflect mechanisms

In the presence of intercurrent events, investigators often raise questions about the mechanisms by which the treatment affects the outcome of interest; that is, whether the treatment affects the outcome of interest through or outside of the intercurrent events. To clarify why such questions are of substantial interest, our experience is that investigators give stories about modified treatments that leverage certain mechanisms of the original treatment. , , , , , , , The motivation seems to be that by understanding the causal mechanisms by which the current treatment affects the outcome of interest, we can motivate new, improved treatments in the future. Here we will define a class of mechanistic estimands called separable effects, , , , , , inspired by the seminal treatment decomposition idea from Robins and Richardson, that are particularly helpful in this setting. To fix ideas about separable effects, we start with an example.

Example (Statins). Statins are one of the most commonly prescribed drug classes in the world. They are successful because they reduce the risk of cardiovascular disease in a broad range of individuals with various clinical histories. The main mechanism by which statins reduces cardiovascular risk is lowering of low‐density lipoprotein (LDL) cholesterol. However, a substantial fraction of patients who take statins experience muscle symptoms, which represent a big hurdle and can lead to treatment discontinuation. More recently, new classes of drugs have been developed to specifically lower LDL levels like statins, but nevertheless differ from statins in the effects through other biological pathways. In particular, protein convertase subtilisin/kexin type 9 (PCSK9) inhibitors selectively reduce LDL levels, but PCSK9 inhibitors do not exert effects through other biological pathways that is affected by statins (and can lead to muscle symptoms). Indeed, PCSK9 inhibitors have been successfully shown to reduce cholesterol levels in patients with muscle related statin intolerance.

Separable effects are effects of modified treatments

The separable effects are designed to target effects of (future) treatments, which selectively exert effects through certain (desirable) causal pathways similarly to the original treatment (eg, the LDL lowering induced by statins), but also selectively avoid other (undesirable) causal pathways (eg, non‐selective pathways that can lead to muscle pain). In Section 4, we consider the potential role separable effects in the examples from Section 2, where new (improved) drugs remain to be discovered.

More abstractly, the separable effects are defined with respect to modified treatment components, and , which are linked to the original treatment in the following way: when and are set to the same value , the effects of giving this combination of modified treatments (), is the same as the effects of giving the original treatment for (see previous works , , , , , and Appendix A for a more formal derivation). This modified treatment assumption holds when the original treatment can be decomposed into two components and . However, this assumption can also hold even if cannot be physically decomposed. When the components and are given individually, they exclusively target certain causal pathways (see Appendix A for details).

In practice, a study of separable effects should be motivated by a scientific story about modified treatments and ; the investigators should clarify why modified treatments are of scientific interest. One reason could be to develop improved treatments in the future. For example, let be statin therapy and be cardiovascular risk. The combination and can be conceived as a modified therapy (such as PCSK9 inhibitors) that, like statins, have a cholesterol lowering component , but lacks effects on the intercurrent event such as muscle symptoms, .

More explicitly, we define the separable effect of the component as which quantifies the causal effect of the component on the risk of the outcome under an intervention that assigns . Similarly, quantifies the causal effect of the component on the event of interest under an intervention that assigns . The separable effect (8) quantifies the direct effect of the treatment on the outcome of interest, but this direct effect is different from the principal stratum estimand because it is defined in the entire study population. Similarly, (9) quantifies the indirect effect of the treatment on the outcome of interest through the intercurrent event. This decomposition illustrates that separable effects quantify mechanisms, and, unlike the sequential trial or principal stratification estimands, the separable indirect effect offers a coherent notion of an indirect effect.

In settings with intercurrent events, researchers often express interest in estimands in subgroups defined by the intercurrent event. This is particularly relevant when the outcome of interest only is well defined conditional on the status of the intercurrent event (eg, when there is truncation by death). Following Stensrud et al, under the assumption that partial isolation (A1) holds, we can also define a conditional separable effect as The conditional separable effect is the average causal effect of the (modified) treatment on when all individuals are assigned the other (modified) treatment in those individuals who have intercurrent event status s under (regardless of their value of ).

Relations between estimands

Like conventional mediation estimands, , such as natural (pure) direct and indirect effects, the (marginal) separable effects quantify the mechanisms by which the treatment affects the outcome of interest. , , , , Unlike the conventional mediation estimands, the separable effects do not require specification of any intervention on the intermediate event. This is important in our context, because it is not clear that such interventions on the intercurrent events exist in any of Examples 1 to 5. Furthermore, a feature of the separable effects is that they are defined with respect to modified treatments, and therefore they can be directly relevant in a drug development setting where the interest is tailoring new treatments to include/exclude certain pathways. However, as we return to in Section 5, conventional mediation estimands and separable effects are often identified by the same functionals of the observed data. The implication of this is that the existing computer software for conventional mediation estimands can often be used to calculate separable effects.

Moreover, the conditional separable effects are related to principal stratum effects. Indeed, a conditional separable effect is restricted to subjects who have a certain value of the intercurrent event under the modified treatment . Assuming that does not exert effects on , (10) is equal to which targets the same subgroup considered in the PS estimand (6) rather than (7). However, it also explicitly quantifies a treatment effect that acts outside of the intercurrent event (a direct effect on the event of interest not mediated by the intercurrent event) in this subgroup. See Stensrud et al (15) for a discussion on the link between conditional separable effects and principal stratum estimands.

TRANSLATING RESEARCH QUESTIONS TO ESTIMANDS

We now revisit the Examples 1 to 3 that were discussed in Section 2 (Examples 4 and 5 are discussed in Appendix B). We map research questions to their corresponding estimands. We find that all of these estimands are interventionist estimands that, at least in principle, could be studied in a (future) randomized trial. ,

Example 1 (Multiple sclerosis (cont.)). In a secondary analysis of the EXPAND trial, Magnusson et al studied a principal stratum estimand like (6), but defined on the risk ratio scale, where indicates confirmed disability progression (at some time ) and the conditioning set indicates having no relapses regardless of treatment assignment at baseline. Later this estimand was also advocated by the ASA/EFSPI oncology estimand working group. Estimand (11) quantifies disability under siponimod vs no treatment in subjects who would not relapse under both siponoind and no treatment.

One motivation for studying principal stratum effects, like (11), of siponimod was “understanding the effect of siponimod on progression occurring independently of relapses.” The relevance of effects in those who would not experience relapse regardless of treatment assignment is nevertheless unclear; as we cannot observe this subset of the population of unknown size, it cannot be a direct target population for a new drug in the future.

On the contrary, treatment effects outside of relapse are of interest if the investigators consider the opportunity of leveraging or avoiding these particular effects in future, refined drugs. For example, based on a biochemical evidence, the drug developers may have reasons to believe that the current drug exerts effects through different (biological) pathways, which have differential effects on relapse and disability progression. Questions about such mechanisms of action motivate the consideration of separable effects, , , , , , which explicitly target the mechanism by which a treatment exerts effects on an outcome. , , , , , , In particular, suppose that siponimod exerts effects on disability through a pathway avoiding relapse , and suppose that we could create a new drug that exclusively targets this pathway, but does not act via the pathways by which siponimod reduces relapse in MS patients. Analogously, we could, in principle, imagine a different treatment that exclusively exerts effects on relapse, similar to siponimod, but does not exert effects on disability outside of the relapse pathway. This motivates separable effects estimands, defined with respect to a hypothetical trial where we assign the modified treatments and , which is defined (marginally) in the full study population. The contrast (12) quantifies the effect of siponimod vs the new drug that exclusively exerts effects on relapse. Thus, if the contrast (12) is equal to zero, then there is no effect of siponimod outside of its effect on relapse. If the contrast is different from zero, then siponimod also exerts effects outside of relapse on disability.

Magnusson et al also state that there was particular interest in the treatment effect “among the subgroup of patients for whom relapses would be absent during the study.” We may therefore consider a conditional separable effects estimand like (10), This conditional separable effect quantifies the effect of siponimod vs the new drug that exclusively exerts effects on relapse, but, unlike (12), it is confined to those who would not relapse on siponimod.

So far we have discussed estimands that quantify effects through certain causal mechanisms, which, for example, can motivate the development of future drugs. However, if the aim is to support labeling decisions of siponimod itself, as is also suggested in Magnusson et al, other estimands seem to be more relevant. In particular, doctors and regulators are often interested in whether the treatment effect varies across subgroups of patients. Such subgroup effects can only be useful for practical decision making if the subgroups are observed before the decision is made, which is not the case for principal stratum effects. On the other hand, simple average treatment effects conditional on a set of measured baseline covariates , is of immediate interest. Whereas the conditional average treatment effect is easy to define (and identify), summarizing subgroup effects across covariates is not trivial. One way to summarize (coarsen) conditional effects is to define an auxiliary variable based on expected outcomes under treatment. In particular, instead of studying effects in principal strata, we could study effects for groups of patients who are likely to be in a principal stratum of interest. Following Joffe et al, define the principal score , which in our example denotes the probability of not having relapses under treatment given baseline covariates . The pair of principal scores is, unlike the principal stratum, a baseline covariate, because it is just a function of . We can now define causal effects among individuals with the same principal scores under treatment , such as which are the effect of siponimod on disability among patients with probability or probability larger than , respectively, of developing recurrences under treatment . Similarly, we could study the joint principal scores, such as which is the effect of siponimod on disability among patients with probability of developing recurrences when assigned to sifonimod and probability when assigned to placebo.

Unlike the principal stratum estimands, these principal score effects are identified at baseline. For example, a decision maker can be interested in giving a different treatment to patients with a high risk of relapses compared to patients with low risk of relapses. However, because the principal scores are just functions of , decision rules that use and/or as input will not be better than decision rules that only use as input.

Example 2 (Treatment effects in early responders (cont.)). To motivate the study of treatment effects in early responders, Bornkamp et al gave a subject‐matter example on the effect of canakinumab (an immunotherapy) vs placebo on major adverse cardiovascular events (MACE), where the authors write that “as the mechanism of action of canakinumab is lowering inflammation, one would suspect that patients who do not achieve the biomarker threshold also have a lower benefit in terms of the time‐to‐event outcome.” The question concerns whether canakinumab only exerts effects on the outcome of interest through its effects on inflammation. Knowledge of this mechanism can motivate the development of new (refined) drugs, similarly to the motivating question in Example 1. To answer this question, a separable effect can be explicitly formulated as follows: suppose that we could give a modified drug in which the part of canakinumab that exerts effects on inflammation is blocked, but otherwise this drug is identical to canakinumab. Would this new drug have a beneficial effect on MACE compared to no treatment ?

If this new treatment has a beneficial effect, then there is a component of canakinumab that acts outside of inflammation. Specifically, this question corresponds to a separable effect (10), defined in the subset of the population who would not have a biomarker response on treatment (ie, those with ). Unlike the joint principal stratum estimand proposed in Bornkamp et al, the separable effect explicitly quantifies a direct effect of treatment.

An alternative motivation for studying treatment effects in early responders is to “support the decision on treatment modifications after treatment start.” This motivation does not concern drug development, but rather sequential decision making, and the substantive question motivates the study of a sequential trial estimand, as described in Section 3.1. To illustrate this point in the simplest possible setting, consider a trial in which treatment decisions can be made at two time points, and , and suppose that the biomarker response is known at time but not at time . Suppose further that patients can be assigned canakinumab () or no treatment () at times . The authors' plain English motivation suggests the counterfactual estimand This estimand quantifies the effect of a strategy that assigns canakinumab sequentially at baseline (time 0) and time 1 vs a strategy that assigns no treatment at baseline and time 1.

Example 3 (Impact of exposure on overall survival (cont.)). There is interest in “whether the lower OS is due to low drug concentration or to disease burden.” , , This causal question alludes to an effect of low serum concentration of the active drug vs no treatment on overall survival; for example, a practitioner may ask whether it is sufficient to have a low serum concentration to experience a treatment effect. Alternatively, the question may allude to the effect of low vs high serum concentration of the active drug on overall survival; for example, a practitioner may ask whether there is a dose‐response effect. Both of these effects are specifically defined with respect to an intervention on the drug concentration itself: suppose, for example, that the variable indicates that an individual has a drug concentration in the lowest quartile on treatment, and let indicate that the drug concentration is equal to 0 (the concentration under no treatment). Then, the question posed by the investigators suggests the simple conditional effect where is a set of covariates that are available before the (hypothetical) decisions is made: here, could be a trivial random variable if we are interested in the marginal effect of , or could be other variables temporary ordered before , in particular it is possible that . However, we emphasize that the exposure of interest in this setting is the drug concentration () in the blood, and not the drug administered (). Indeed, a study conducted by researchers at the FDA aimed to evaluate whether individuals with low serum concentration of the active drug would show survival benefit if they had a higher serum concentration. (p166)

One reason for studying (19), which concerns drug concentration, is to assess whether a higher dose of the drug should preferably be administered, in particular to certain subgroups. In practice, however, these subgroups must be known before the treatment is given, which means that these subgroups cannot be principal strata (which are defined by intercurrent events that are unobserved at the time of treatment initiation). On the other hand, conditional treatment effects, possibly coarsened as principal scores like (14) and (15), could be used to make decisions; for example, it is possible to give a higher dose to those individuals who are likely to have a low serum concentration of the drug under a standard dose.

Indeed, after the FDA study, a randomized trial has been conducted to evaluate the effect of different drug doses in certain groups of patients, where these groups were solely defined by pretreatment variables, like in (19).

IDENTIFICATION ASSUMPTIONS

Our arguments in Sections 3 and 4 concerned the translation of a substantial research question, articulated in plain English, to a causal estimand, articulated in counterfactual notation. The arguments for choosing an estimand did not rely on the assumptions that are required to identify this estimand from the data at hand (eg, from an RCT where only is randomly assigned). However, when an estimand of interest is chosen, it is crucial to understand, and critically assess, the assumptions necessary to learn about the estimand from data (ie, to evaluate whether a sufficient set of identifiability conditions hold). Furthermore, when setting up a randomized trial, these assumptions can be helpful in understanding how the study needs to be designed, or which data need to be collected.

Identifiability conditions for all of the estimands in Sections 3 and 4 are thoroughly described in several previous works. , , , , , Here we informally review some important features of these conditions, which should be justified on scientific grounds. We focus on nonparametric identifiability conditions. Sometimes (strong) parametric assumptions can be invoked to obtain alternative identification conditions; that is, identification conditions that only hold under a particular (parametric) model. For example, mixture models and Bayesian methods have often been used to identify principal stratum effects, but as noted by Bornkamp et al, inference is then highly sensitive to the correctness of the parametric assumptions and likelihood estimators can exhibit pathological behavior. Indeed, the practical value of these alternative parametric conditions is limited unless the investigators have convincing arguments why we should believe in the parametric assumptions (which, to our knowledge, is rarely the case in medicine).

The average total effect requires the weakest assumptions

Only the conventional average effect estimand (1) can be identified without additional assumptions from a (perfectly executed) RCT where the baseline treatment is randomly assigned, as illustrated in the simple causal directed acyclic graph (DAG) in Figure 1A. This result is also valid for average effect that are defined conditional on observed baseline covariates, like the effect (19) and principal score effects like (14) and (15). All the other estimands we consider—including principal stratum effects, separable effects and sequential treatment effects—require additional assumptions in this setting, which we discuss in the following subsections.

Sequential trial estimands

Sequential trial estimands can be identified without additional assumptions from a (perfectly executed) trial where the treatment is randomly assigned sequentially, that is, at multiple points in time. However, randomly assigning treatment at baseline (eg, in a randomized trial with treatment switching/discontinuation) is not sufficient to identify sequential trial estimands. , , In particular, it is necessary to adjust for common causes of the outcome and the sequential treatment at time . Causal graphs allow us to visualize and evaluate the conditional independencies that ensure identification of sequential effects. , They are useful for translating and discussing the key assumptions with scientific collaborators, in order to assess their validity. For example, the causal DAG in Figure 1B describes a setting where the sequential trial estimands (5) and (18) are identified provided that and are measured.

Separable effects require dismissible component conditions

The separable effects are defined with respect to modified treatments that, when combined, operate like the study treatments (see Section 3.3). These modified treatments may correspond to decompositions of the study treatments , , or to treatments that are distinct from the study treatments but share the same mechanistic actions. , , Like the sequential trial estimands, identification of separable effects requires conditional independencies (called dismissible component conditions or conditional exchangeabilities) to be satisfied. , , , Like the sequential trial estimands, these conditions require adjustment for common causes of the outcome and the intercurrent event , which may be time‐varying. , , , The dismissible component conditions would also be violated if certain variables are directly affected by both components and .

For example, if is unmeasured in the DAG in Figure 1A, then the identification conditions for these estimands are violated. Variables like are likely to be present in a range of practical settings. In Example 1, any factor (eg, related to genetic or lifestyle) that affects both relapse and disability would be classified as a . Unlike conventional mediation estimands, the separable effects are single world estimands that can be identified in a future randomized trial. , ,

Principal stratum estimands require alternative assumptions

Unlike the other estimands, identification of principal stratum effects require assumptions are empirically unverifiable, that is, untestable, even in principle. , , , , In particular, identifying the subgroup of individuals that constitutes the principal stratum in (6) is impossible in most practical settings without relying on such unverifiable assumptions. , This limits the practical relevance of these estimands. , , , , For example, a principal ignorability assumption is often invoked to identify principal stratum estimands such as (6) and (7). , Interpreting the principal ignorability assumption is not straight‐forward: it is a cross‐world independence assumption, which requires the investigator to reason about (untestable) independencies between counterfactuals under different treatment assignments. It is necessary (but not sufficient) that the investigator adjusts for common causes of the event of interest and the intercurrent event , even in a study where is randomly assigned, for principal ignorability to hold. These common causes may be time‐varying, and, unlike the other estimands, the literature on principal stratum effects in the presence of time‐varying confounding is scant and relies on assumptions that may be hard to justify. , On the other hand, principal score estimands, which are defined with respect to predicted probabilities of belonging to a stratum, can be identified under weaker conditions (see Section 5.1).

A NOTE ON ESTIMATION AND SOFTWARE IMPLEMENTATION

This article focuses on interpretation and identification, not estimation. However, we would like to emphasize that our consideration of interpretation and identification has consequences for estimation. In particular, we can exploit the relations between the different estimands to clarify when the same estimation algorithms (and computer software) can be used to estimate different types of effects.

For example, computer software for natural direct and indirect effects can be used to estimate separable effects in any setting where they are identified by the same functionals of observed data distributions. , , More generally, each separable effects can be estimated using software for path‐specific effects; there is always a path‐specific effect that is identified by the same functional as a separable effect. See Valente et al for a review of available software for causal mediation analysis.

Finally, many trials will be powered for the ITT analysis, and as such will not be guaranteed to be powered for the alternative estimands discussed in this article. Fang and Jin proposed sample size calculation strategies for the estimands described in the ICH E9 addendum, and illustrated them using simulated data examples. Larger sample sizes are needed for all estimands relative to the ITT analysis; for example, a quarter increase is required for a principal stratification analysis. A major limitation of Fang and Jin's work is that post‐treatment confounding is not considered. Confounding complicates power calculations, and may result in much larger sample sizes being needed to detect effects. In practice, some simulation studies based on plausible data generating mechanisms, possibly inspired by previous trial data, may help to give an impression of whether a trial is well‐powered for the question of interest.

DISCUSSION

We have clarified that certain causal estimands—including sequential treatment effects, separable effects and conditional causal effects—are useful in settings where principal stratum estimands are often recommended. , These estimands correspond to different causal questions. If the investigator is ultimately interested in finding subgroups where treatment works better, conditional causal effects are of interest, but principal stratum effects are not relevant because the principal strata are not observable when the decision is made. We illustrate this in Examples 1 to 3. Furthermore, separable effects can disentangle how the treatment affects the outcome outside or through the intercurrent event, as we illustrate in Example 1 (and in Examples 4 and 5, which are discussed in Appendix B).

All of our suggested estimands share key features: they are defined in observable subsets of the population, they correspond to explicit causal inquiries, and they can at least in principle be empirically falsified in a future experiment. , , , These common features do not arise by coincidence. We believe that any estimand that guides practical decision making is in principle verifiable in a well‐characterized population, because any practical decision can be instantiated in this population.

Causal inference is a subtle exercise, and being precise about the interpretation of estimands—and the practical questions that motivate them—is crucial to avoid logical flaws and erroneous decisions. Thus, we encourage investigators to be explicit about why they target a particular estimand, which itself is an exercise that can sharpen arguments and thought processes. ,

When we have data from a trial where only a baseline treatment is randomly assigned, there are limitations as to how statistics can help us to discover causal mechanisms. Causal (structural) assumptions are required for this endeavor. These assumptions are not innocuous and are not guaranteed to hold. Estimates of separable effects can motivate the development of new treatment and generate new hypotheses of causal mechanisms, but the study of separable effects do not give guarantees that these components exist and can be used in future treatment. On the other hand, principal stratum effects do not guarantee that there exist (a substantial amount) of individuals that actually belongs to the principal stratum, do not allow us to characterize these individuals, and do not quantify the effect of future therapies.

If estimands beyond average total effects are of interest in a randomized clinical trial, which often seems to be the case, our work illustrates an important point: the investigator should strive to include a rich set of pre‐ and post‐treatment variables. This is crucial, because identification of mechanistic estimands and sequential trial estimands would require adjustment of (possibly time‐varying) common causes of the intercurrent event and the event of interest.

Finally, different estimands can, under certain assumptions, be identified by identical functionals of observed data distributions, and therefore they can be estimated in the same way. Even if these estimands are identified by the same function of data, it is important that the investigators are explicit about their causal question and corresponding estimand: the interpretation of the estimand, and identification assumptions, can still differ, , , which matters when these estimands are going to be used for practical purposes.