An adaptive method for assigning clinical trials wait-times for controls.

Wait-list control clinical trials are popular among psychologists and rehabilitation specialists partly because all participants receive the intervention. In 2 arm wait-list control trials, individuals randomized to the treatment group receive immediate treatment whereas individuals randomized to the control group wait a fixed amount of time before intervention is initiated. For interventions that have varying durations, careful consideration must be given to the period that participants in the control group have a delay until treatment begins, as incongruent wait times compared to the intervention durations of the treatment group may introduce confounding into the evaluation of the treatment differences. To alleviate this issue, we propose to adaptively assign wait times to individuals randomized to the control group based on the intervention duration of those in the treatment group. Simulations demonstrate the that our method not only results in similar timing distributions between participants in the treatment and control groups, but also allows participants in the control group to initiate treatment earlier than the traditional design. The latter characteristic may reduce dropout and result in more efficient study enrollment.

RCT Entities:   Population Interventions Outcomes

Adaptive designs in clinical trials have allow for a planned modification in one or more of the study parameters [1,2]. The genesis of the adaptively designed clinical trial is found primarily in the field of medicine, particularly from pharmaceutical and medical device development. Unfortunately, there has not been a focus on developing adaptive clinical trials methodology for studies commonly found in other areas, especially behavioral and rehabilitation interventions. Waitlist control (WLC) designs are commonly used in these areas and variations of these designs contain a fundamental characteristic that can be improved through adaptation. WLC designs are longitudinal randomized clinical trials that offer all participants access to the investigational intervention [3,4].

A diagram of typical WLC designs is shown in Fig. 1, which is similar to that described previously [5]. In Stage I of WLC designs, the WLC mirrors a conventional two-arm clinical trial where the treatment and control groups receive their respective conditions. The comparison of the primary and secondary outcomes between the treatment groups typically occurs after this Stage I period. However, unlike traditional parallel 2-arm clinical trials, participants allocated to the control group have the option of receiving the investigational treatment, whereas participants allocated to the treatment group may or may not have continued follow-up as part of the study. Further follow-up in both groups may occur beyond that described in Fig. 1, however, this follow-up is not relevant to the present investigation.

Fig. 1

Schematic of a 2-arm wait-list control design.

Kazdin (2003) suggests that the Stage I duration the treatment and control groups should be the same [4]. Equivalence strengthens the internal validity of the clinical trial, so that any observed differences in the outcome between the treatment groups can be considered causal rather than due to other factors [6]. In situations where the intervention or follow-up time has a fixed duration, the Stage I for the control group would be the length of the intervention. However, certain treatments may allow flexibility in the intervention duration, thus permitting variability in the length of Stage I. Restrictions on when participants in the control group can initiate treatment often leads to unbalanced Stage I durations. For example, participants in the evaluation of the Resilience and Adjustment Intervention (RAI) required between 1.5 and 26 weeks to complete the intervention [7], but participants in the control group had Stage I durations ranging between 11 and 22 weeks.

Differing Stage I durations could have two important impacts. First, as previously mentioned, the internal validity of the study is threatened if the Stage I duration for participants in the treatment groups are dissimilar [6,8]. The differing Stage I durations could act as a confounding variable with the treatment that may influence conclusions regarding treatment efficacy. Second, restricting the active intervention from the control group for longer periods than necessary prevents participants from receiving a potentially beneficial treatment sooner. This restriction may cause threats to participant's health and well-being.

To alleviate the potential confounding that could arise from differing Stage I durations for each of the study arms, we propose an adaptive WLC (aWLC) design that balances the Stage I duration between the groups and allows patients randomized to the control group to receive the study intervention sooner than traditional WLC design strategies.

Methods

The adaptive strategy for assigning Stage I durations for participants randomized to the control group are developed for WLC trials described in Fig. 1. The distribution of the Stage I durations are defined by the cumulative distribution function (CDF) and are denoted by , where t is the time to complete Stage I and the subscript g has values of (T,C) denoting the treatment and control group, respectively. Most WLC designs offer the investigational treatment to the control group after some fixed time, . This value is often determined from ethical or practical reasons. For instance, if an intervention is planned to last 5 weeks, or the intervention requires 5 sessions and it is expected that each patient would attend one session per week, then a logical choice for is 5 weeks. In these situations, the CDF of Stage I durations for the control group is , where is an indicator function with a value of 1 if the inequality is satisfied and 0 otherwise.

In order to achieve a closer balance in the Stage I durations between the treatment arms, we propose that the Stage I duration for the control group be based on the Stage I durations of the participants in the treatment group. In other words, the Stage I duration should be determined from so that, optimally, . In situations where the intervention is a fixed amount of time and the Stage I duration for the WLC group lasts the same amount of time, this condition is already satisfied. Assuming that is known and total participants were enrolled into the study (), the next participant randomized to the control group is assigned a Stage I duration, , by drawing a single random variate from .

An unconstrained Stage I period for the control group may not be advised since it may conflict with ethical or practical constraints. For instance, there may be a non-zero probability that participants may take up to 12 weeks to complete an intervention, however, withholding treatment for 12 weeks for any participant could be considered unethical as delays in a potentially effective treatment may be detrimental participant's well-being. A ceiling for the Stage I duration can easily be incorporated into the proposed paradigm by aiming to satisfy the condition for all times less than a constant , where is the ceiling for the Stage I duration for the control group. In this situation, the simulated and assigned follow-up periods for participant in the control group can be denoted as and , respectively, and the Stage I duration for participant can be denoted by.

Up to this point, this design assumes known treatment duration CDF distributions when, in practice, this is typically unknown and needs to be estimated. Any validated method to estimate the CDF after participants have been enrolled in the treatment group when the kth participant is enrolled into the study, , can be used within this paradigm. However, care should be chosen to include all available information from the treatment group, particularly if patients in the treatment group have initiated, but not completed, the treatment regimen. Methodology to obtain , suited for time-to-event data is favorable, as participants who have not completed the treatment can be considered censored. Ensuring that the estimation method provides a smoothed estimate of the distribution will ensure variability in the Stage I durations. Kaplan-Meier and related smoothed estimates of the survival function, which are simply the complement of the CDF, can be used to provide obtain , [[9], [10], [11]]. Common parametric distributions suited for survival data, including the Weibull and log-normal distributions, can also be utilized.

If too few participants in the treatment group have completed the treatment regimen, the estimator of at can be undefined or unstable. A lead-in period, where the Stage II initiation times for the control group are determined according to a user-defined CDF can be used. In the simplest case, this function can be defined as so that, when the number of available treatment cases are insufficient to estimate , a Stage I ceiling time is used for all participants in the control group. The use of this ceiling time in the lead-in, coupled with the use of a ceiling time in the adaptive portion of the study, will result in our aWLC design degenerating to the traditional design if all of the Stage I durations in the treatment group are larger than the ceiling time. A schematic of the aWLC with a lead-in is displayed in Fig. 2.

Fig. 2

Schematic of the aWLC for assigning wait-times for participants randomized to the wait-list control group. This schematic uses a lead-in period that participants in the control group a constant Stage I duration.

Simulation

The treatment durations used to evaluate our methodology are estimated from those observed in a WLC trial to increase resilience in patients who have suffered a traumatic brain injury (TBI) (ID: NCT01935583) [7]. Study participants had a documented TBI and were referred by rehabilitation providers. Completion of the intervention required seven 1-h therapy sessions administered by a licensed neuropsychologist. Participants that did not complete the required sessions in 1-year were considered lost to follow-up, however, the current study is restricted to those participants in the intervention group who completed the intervention in no more than 100 days. Participants randomized to the WLC group were assigned a Stage I duration of 35 days (5 weeks), afterwards, they were invited to receive the resilience intervention.

The traditional WLC design was simulated by randomizing participants to either a treatment or control arm with equal probability. Participants in the treatment group were given a treatment duration estimated from the Stage I duration distribution observed in the RAI study using polynomial splines [11,12]. Participants randomized to the WLC group were assigned a Stage I duration of 42 days (6 weeks), which is approximately the median treatment duration observed from the RAI study (Fig. 3). This ceiling cutoff differs from the value used in the actual study to focus on the evaluation of the aWLC design.

Fig. 3

Comparison of the CDF of the treatment group from the RAI study to a theoretical control group assigned a constant Stage I duration.

The aWLC was evaluated using a lead-in period such that adaptive Stage I assignment did not begin until four participants completed the intervention. Participants randomized to the treatment group but did not complete the intervention when a participant was randomized to the control group were considered censored in the distribution estimation. An interpolated Kaplan-Meier estimate was used to estimate the Stage I duration distribution for the treatment participants [10]. The assigned Stage I durations for the WLC group were restricted to be not less than the smallest Stage I duration in the treatment group.

Separate simulations were performed for study accrual rates of 0.0001, 1, 5, and 10 participants per an expected treatment duration of 42 days. Total study sample sizes varied from 52, 128, and 352, which correspond to the sample size required to achieve 80% power to detect an effect size d of 0.8, 0.5, and 0.3 using a two-sided two-sample t-test performed at the 0.05 level. Data from one thousand clinical trials corresponding to each permutation of study parameters were simulated. The Kullback-Leibler distance comparing the distribution of the treatment and control group was calculated for each simulated clinical trial and compared over the simulation parameters. The mean percentage of participants in the WLC group initiating Stage II after 28, 35, and 42 days were reported for each set of simulation parameters. Participants randomized during the lead-in were not included the calculations evaluating the methodology.

Results

Fig. 3 shows empirical and estimated distribution of the 141 participants that completed the RAI intervention. Using the estimated distribution of intervention durations, 17.9% and 39.3% completed the intervention in less than 28 and 35 days (4 and 5 weeks), respectively. Fifty-six percent (55.6%) of the participants required 42 or more days to complete the intervention.

The distribution of the Stage I durations in the control group were closer to the respective treatment group distribution in the aWLC compared to the traditional design, as all of the Kullback-Leibler distances were smaller in the proposed methodology (Fig. 4). While there is a slight deterioration of this as the sample size increases, it is small with respect to the gain achieved from the aWLC method. The distributional comparisons did not vary according to the accrual rates.

Fig. 4

Kullback-Leibler distances (KLD) comparing treatment and control group Stage 1 durations compared over a number of trial parameters. aWLC = adaptive waitlist control; AR = Accrual Rate; SD=Standard Deviation.

The aWLC resulted in an average 57–66% of participants in the control group permitted to begin the investigational therapy earlier (Table 1) compared to the traditional WLC design. This compares favorably to the 56% seen in the Stage I durations for the treatment group, which was what was expected from the estimated Stage I duration obtained from the clinical trial data. Clear differences in the percentage of patients beginning Stage I early were observed as a function of sample size. Smaller trials, and therefore, trials with fewer individuals in the control group adaptively assigned Stage I durations, tended to have a higher percentage of control participants (61–66%) than large studies (57%) beginning the treatment early, which may be due to overfitting of the distribution of the treatment durations in small sample sizes. Between 40-46% and 17–19% of the participants were able to begin the intervention at least 1 or 2 weeks early, respectively. Similar to the aforementioned results, these values were slightly inflated from the expected 40% and 18% estimated from the RAI data. This inflation could be due to the simple nature of the Kaplan-Meier survival curve that was used to estimate the distribution of treatment durations. Similar results were seen when the ceiling wait-time was set at 56 and 28 days. Tables with this information can be found in the online supplementary material.

Table 1

Simulation results assessing adaptive WLC methodology. Estimated percentiles for the percentage of patients completing the intervention in less than 42, 35, and 28 days were 55.6%, 39.3%, and 17.9%, respectively. Values are presented as mean (standard deviation).

N	Accrual Rate	<42 days	<35 days	<28 days
52	0.0001	60.7% (16.8%)	41.8% (16.7%)	16.6% (12.6%)
52	1	61.3% (17.4%)	42.5% (17.4%)	17.3% (13.0%)
52	5	63.2% (18.0%)	43.9% (18.3%)	17.6% (13.9%)
52	10	66.2% (18.4%)	46.2% (19.1%)	19.4% (14.6%)
128	0.0001	59.0% (10.7%)	41.0% (10.7%)	17.5% (8.4%)
128	1	58.4% (10.9%)	40.9% (10.6%)	17.4% (8.0%)
128	5	58.8% (11.2%)	40.8% (11.2%)	17.2% (8.5%)
128	10	60.5% (11.1%)	42.3% (11.5%)	18.3% (8.7%)
352	0.0001	57.1% (6.8%)	40.3% (6.5%)	17.8% (5.1%)
352	1	56.8% (6.5%)	39.8% (6.4%)	17.7% (5.0%)
352	5	57.3% (6.6%)	40.4% (6.4%)	18.1% (5.1%)
352	10	57.4% (6.4%)	40.5% (6.4%)	17.8% (5.0%)

Even though the aWLC results in an inflated percentage of individuals allocated to the WLC group than expected, the actual number of individuals who were assigned early start times compared to the expected number is small. For example, in a trial of 52 total participants, we would expect about 15 participants (e.g. 56% x 26 participants in WLC group) to have Stage I durations less than 42 weeks. From our results, at most 66% of WLC participants started early, which would correspond to 18 participants completing Stage I earlier than would be expected from traditional designs. This difference is unlikely to produce any meaningful confounding effects in the evaluation of treatment efficacy. Table 2 shows information on the raw sample sizes from each of the simulation parameters.

Table 2

Number of WLC participants, allocated following the lead-in period, with wait-times less than 6, 5, and 4 weeks. Values are presented as mean (standard deviation).

N	Accrual Rate	<42 days	<35 days	<28 days	WLC Adaptively Assigned Stage I
52	0.0001	12.8 (4.3)	8.8 (3.9)	3.5 (2.7)	21.0 (3.6)
52	1	12.4 (4.2)	8.6 (3.9)	3.5 (2.7)	20.2 (3.6)
52	5	11.4 (4.0)	8.0 (3.8)	3.2 (2.7)	18.0 (3.6)
52	10	10.3 (4.0)	7.2 (3.6)	3.1 (2.5)	15.5 (3.6)
128	0.0001	34.7 (7.1)	24.1 (6.8)	10.3 (5.1)	58.9 (5.6)
128	1	34.1 (7.2)	23.8 (6.6)	10.1 (4.8)	58.4 (5.7)
128	5	33.1 (7.3)	23.0 (6.7)	9.7 (4.9)	56.3 (5.7)
128	10	32.3 (7.1)	22.6 (6.8)	9.8 (4.8)	53.3 (5.4)
352	0.0001	97.5 (12.8)	68.8 (11.7)	30.5 (9.0)	170.8 (9.3)
352	1	97.1 (12.3)	68.1 (11.6)	30.2 (8.8)	170.8 (9.2)
352	5	96.2 (12.4)	67.9 (11.4)	30.4 (8.8)	168.1 (9.5)
352	10	94.9 (11.9)	66.9 (11.4)	29.4 (8.5)	165.2 (9.4)

There was a slight trend such that, for fixed sample sizes, studies with a higher accrual rate had more participants with shorter treatment durations than expected. This trend was most apparent in studies with smaller sample sizes, where 66% of the adaptively allocated WLC group participants were assigned Stage I durations <42 days when the accrual rate would lead to about 10 participants being concurrently enrolled. Based on the participants in the treatment group from the clinical trial, we would expect this value to be 56%. For moderate to large sample sizes, this difference is smaller, with 61% and 57% of the participants in the WLC group having early treatment initiation. We attribute this to the inclusion of participants who are currently undergoing treatment, or equivalently, are censored in our estimation paradigm, when the Stage I durations are estimated.

Discussion

Waitlist control trials are attractive as they allow every patient the option of receiving the investigational intervention. Notably, these trials are designed to maximize the potential benefits of the participants involved in the research [13]. Simulated clinical trials using the aWLC design demonstrate its potential to balance the Stage I distributions between the treatment and control groups, with particular benefits in larger trials. An additional benefit of this design is the decreased the Stage I duration for participants in the control group, which allows them to have access to the investigational therapy sooner than the traditional WLC design.

Our introduction of the aWLC design relied on a lead-in period to obtain a sufficient number of participants to estimate a Stage I distribution. Other strategies for estimating the treatment durations early in the study, such as assigning the first few patients to the treatment group, incorporating preliminary data, or using CDFs reasonably believed to define the treatment duration can be employed. The sample size for estimating the intervention duration can also be increased by utilizing the Stage II duration for the control group if this data is recorded. Lastly, we recommend that the methodology used to estimate be amenable for use in small sample sizes so that a length of the lead-in is minimized so that the Stage I durations are assigned using the observed data as often as possible.

While WLC trials can be attractive in certain situations, they are not appropriate for all clinical trials. Forcing patients into a waitlisted arm is unethical for medical conditions that require anything more than acute care [4,14]. Additionally, WLC trials are not blinded to the participant, so that the participants are aware of the treatment arm that they are allocated. The lack of blinded treatment groups can potentially impact a number of areas in the evaluating the treatment efficacy [[15], [16], [17], [18]]. Previous research suggests that WLC trials yield a sufficient comparison group for the estimation of treatment effects [[19], [20], [21]], while others argue that biases might be introduced by using the this design [8,22]. Thus, care should be taken to ensure the use of WLC design is appropriate to evaluate the efficacy of the investigational therapy.

The aWLC was evaluated under optimal conditions. Namely, the intervention for the control group (e.g. Stage II) was assumed to begin as assigned. This assumption is often not met in clinical conditions. Participant factors including employment flexibility, socioeconomic status, transportation availability, and even underlying physical, mental, or emotional health may affect the ability a participant to receive planned therapy [[23], [24], [25], [26], [27]]. When the aWLC method is applied in clinical situations, we expect the Stage I durations for the control group will be both more variable and longer than the current simulation study implies. This is due the delays in the Stage I duration of the treatment group being passed to individuals in the control group, along with individual delays in treatment initiation encountered by each member of the control group. However, even with these unaccounted delays, we expect that the aWLC will continue to balance the Stage I durations between the groups and allow participants in the control group to begin the interventional therapy at an earlier time compared to the traditional design.

Another clinical factor that was not incorporated into the current study was the relationship between assigned Stage I durations and both participant enrollment and dropout. Potential participants may not be willing to endure the Stage I duration of the traditional WLC trial and may not consent into the study. However, in the aWLC design, this planned Stage I duration acts as a ceiling, allowing participants to receive the proposed intervention earlier and potentially resulting in higher enrollment rates. Relatedly, participants randomized to the control group could have a shorter window to drop out of the study prior to the end of Stage I when using our methodology. In fact, prior studies indicated that longer wait times were related to higher dropout [28], therefore, the incorporation of our methodology may be able to reduce dropout rates in the control group. However, this remains to be evaluated in a clinical setting.

Care should be taken on how the estimated Stage I distribution is estimated as it will have consequential impacts on the assigned Stage I durations for the WLC group. We chose to use interpolated Kaplan-Meier methods [10] over more computationally sophisticated methods, such as using polynomial splines [11], due the ability of the aforementioned methodology to obtain distribution estimates in small sample sizes. The simple nature of this estimated distribution could be one reason why the percentage of individuals in the WLC group with Stage I durations less than 42 weeks were higher than expected. While using the more complex methodology to obtain the Stage I durations from the treatment group may result in a more representative distribution, sophisticated methodology would also require longer lead-in periods. Additionally, our method will always lag behind changes in the intervention durations observed in the treatment group, particularly early in the studies. Runs of small or large treatment durations in the treatment group, particularly early in the trial, will affect the Stage I duration assigned to the control group. Both of these reasons could result in differences in the Stage I durations between the treatment and control groups.

While this work does have the potential to decrease the Stage I duration for participants randomized to the control group in WLC trials, limitations are present. Participants commonly drop out of the research studies, regardless of the treatment arm to which they are randomized. Patient dropout from the treatment group could have an influence on the distribution of Stage I durations by adding influential outliers that, as the trial progresses, are all censored. In our methodology, this extreme information would be passed to the assignation of Stage I durations for the WLC group, potentially extending the Stage I durations. Further work is required to understand these effects and develop methodology to ameliorate these impacts.

Lastly, the aWLC, and adaptive clinical trial methodology in general, may seem like a panacea for ethical issues in study design. However, their incorporation may lead to other concerns [[29], [30], [31]]. Specifically for this methodology, an ethical concern might be raised about assigning participants in the control group differing Stage I durations. If the Stage I duration can be considered a measure of quality of care, our design could affect the characteristic of justice in clinical trials since some participants will receive the intervention sooner than others [32]. This would be particularly evident if this methodology were extended to incorporate participant characteristics in the allocation of a Stage I duration. Additionally, the principles of justice and the potential for investigator bias may be present when incorporating lead-in periods prior to the use of any adaptive methodology [31]. As with all study planning, we recommend that careful consideration of these issues be discussed thoroughly and openly among individuals with clinical, research, and ethical expertise prior to deciding any aspects of the conduct of a study.