Impact of dose feasibility on the conduct of phase I trials of adoptive cell therapy.

BACKGROUND: /Aims: In early-phase cell therapy trials, each dose level being studied is defined by the number of cells infused into the trial participant. The issue of dose feasibility presents itself when the desired number of cells is not reached in the expansion process. Consequently, dose assignments for some patients may deviate from the planned dose according to the chosen design. Widely used algorithmic designs aren't flexible enough to handle this complication and can lead to the exclusion of safety data from the dose assignment algorithm. This article studies the impact of dose feasibility challenges on the behavior of the 3 + 3 decision rule.
METHODS: We conducted a simulation study across six dose-feasibility and dose-toxicity scenarios. Trials are simulated using the 3 + 3 algorithm. We present a novel algorithm for random feasibility curve generation. We used this algorithm to conduct a large-scale simulation study across 100 random scenarios.
RESULTS: We found that the 3 + 3 has problematic characteristics due to the exclusion of safety data from the algorithm. Ignoring toxicity data can complicate the allocation of subsequent patients in the trial and can bias the final maximum tolerated dose recommendation for the next phase of drug development.
CONCLUSION: Our study demonstrates that excluding safety data from the 3 + 3 algorithm can be detrimental to trial conduct. Furthermore, there are existing methods that are flexible enough to include data that is observed away from the planned dose. We recommend that these methods be used in conducting phase I cell therapy trials.

Introduction

Cancer trials investigating adoptive cell immunotherapy are growing in popularity [[1], [2], [3]]. Early-phase cell therapy trials face novel issues that prohibit straightforward application of traditional dose finding methodologies. This article examines the impact of excluding safety data in conducting Phase I trials of adoptive cell therapy. These trials study dose levels that are defined by cell growth in culture, through a process known as leukapheresis [4], for infusion back into the participant to fight the cancer. The number of cells that are grown in culture are counted prior to infusion into the participant, meaning that we know how many cells were expanded before treatment begins. The cell growth process is participant-specific and thus some participants may have cell counts that are less than the dose that the design assigns to them, preventing them from receiving their intended dose. Throughout the conduct of the trial, each participant is treated at his or her highest feasible dose level if the number of cells counted for the participant is below the cell count that defines his or her recommended dose level. Otherwise, each participant is treated at his or her recommended dose level if the number of cells counted for the participant exceeds the cell count that defines his or her recommended dose level. The primary endpoints that sequentially guide dose assignments are both binary outcomes. The first is dose-limiting toxicity (DLT), defined by protocol-specific adverse events, and the second is feasibility, defined by whether or not enough cells were generated to administer the planned dose. The maximum tolerated dose (MTD) is defined by a target DLT rate, and in this case we are interested in an MTD that achieves a minimum acceptable probability of being feasible to administer to participants.

There are three available methods that account for dose feasibility and incorporate safety data observed at unplanned dose levels into the design. The first method was proposed by Thall, Sung and Choudhry [5] in 2001. This method extends the continual reassessment method (CRM) to handle feasibility considerations in addition to safety. Software to enable its general use is not currently available. The method of Wages and Fadul [6] is based on beta-binomial models and isotonic regression, and it is accompanied by an R shiny web application http://uvatrapps.uvadcos.io/wfdesign/. The third method was recently proposed by Devlin, Iasonos, and O'Quigley [7]. This method modifies the CRM so that it can incorporate patient data observed at fractional dose levels.

Despite the availability of these methods, the 3 + 3 algorithm is often used to guide dose allocation in the early development of adoptive cell immunotherapy. It is not clear how data observed in participants deemed infeasible to receive his or her recommended dose are factored into future dosing decisions. A Phase I trial conducted by Lee et al. [8] states that “Participants whose chimeric antigen receptor T cell product did not meet the dose to which they were assigned did not inform dose escalation but were assessed for toxicity and for all other parts of the study.’’ This quote indicates that safety data from participants treated below their assigned dose are ignored. Disregarding safety information on participants treated away from their assigned dose could lead to biased estimates of the MTD, inflate the overall sample size, and increase the cost of conducting the study. The goal of this article is to quantify the impact of ignoring DLT data when using the 3 + 3 in the presence of dose feasibility considerations. We also propose a new algorithm for randomly generating a family of dose-feasibility curves for evaluating early-phase methods in this setting.

Methods

We conducted a simulation study to explore the characteristics of 3 + 3 in adoptive cell therapy trials with dose feasibility considerations. We assumed that the dose level and the probability of toxicity have a monotonic increasing relationship while dose level and feasibility have a monotonic decreasing relationship. We examined six hypothesized scenarios for DLT and feasibility probabilities over four possible dose levels. Additionally, we randomly generated 100 dose-toxicity curves using the algorithm proposed by Conaway and Petroni [9]. There are no published algorithms for randomly generating dose-feasibility curves, so we propose the following new algorithm.

Choose a value for the feasibility parameter that is between 0 and 1. A larger value of will correspond to a higher probability that a number of the dose levels won't be feasible.

For each curve generate a uniform random variable and denote it .

For dose levels we generate the probability: P (dose is feasible) for . Now let uniform and uniform

For any set

To illustrate how the algorithm functions, we work through an example of how one feasibility curve is randomly generated. Let the number of doses According to step 1 set the feasibility parameter as . Then according to step 2 generate a uniform random variable . Thus, the feasibility probability at dose level 1 from step 3 is uniform. We iterate to form the subsequent : uniform, uniform, and uniform. According to step 4 we set . Thus, the final feasibility scenario is (0.72, 0.49, 0.26, 0).

Under each scenario, we simulated 10,000 trials using the 3 + 3 algorithm. Each trial accrued a maximum number of 24 participants while evaluating a maximum of 30 participants for feasibility. These settings mimic those considered in the simulation of a Phase I trial of activated T cells in combination with radiation therapy and temozolomide in newly diagnosed glioblastoma participants (NCT03344250) studied by Wages and Fadul [6]. Following the conduct of Lee et al. [8] and Lum et al. [10] in executing the 3 + 3, participants were enrolled sequentially in each trial and if their assigned dose was not feasible for them to receive, then they were treated at their highest feasible dose level. The participants that received an unplanned dose did not have their DLT outcome inform subsequent participants’ dose assignments. If a participant was evaluated for feasibility and was not feasible for the lowest dose, they were not treated under the protocol.

We focus on the characteristics of the 3 + 3 design that are directly attributable to excluding data observed at unplanned dose levels. We report the following statistics:

Percentage of all participants treated at an unplanned dose that had a DLT excluded from use in future dose assignments or MTD recommendation.

Percentage of trials where there is at least one excluded DLT outcome.

Percentage of trials in which a dose lower than the recommended MTD had an estimated DLT rate higher than the estimated DLT rate at the recommended MTD.

Percentage of total DLT outcomes that were excluded in executing the design.

Average number of trial participants treated at an unplanned dose that had a DLT outcome excluded from use in future dose assignments and MTD recommendation.

Statistic 1 provides a measure of participant safety. When a DLT occurs at an unplanned dose, it is important to ask whether a trial should continue to escalate. Statistic 2 indicates how prevalent undesirable behavior is occurring across all of the simulated trials in a large number of hypothesized scenarios. Statistic 3 illustrates that a fundamental assumption of dose-finding designs, that of an increasing relationship between dose and toxicity, can be violated if some safety outcomes are not used to inform dosing decisions. The possibility of choosing an MTD when there are lower doses with higher observed DLT rates is considerably problematic. Statistic 4 determines how many of the observed DLT outcomes are not factored into the trial design. Statistic 5 gives a per trial sense of how many participants are not having their outcome, regardless of the result, contribute to the design conduct. We have included mathematical formulas for the five statistics in the supplementary appendix.

Results

In our simulations, we assume that, as in Lee et al. [8], participants that are unable to receive their assigned dose “did not inform dose escalation.” Fig. 1 illustrates a single simulated example trial that motivates our exploration. The trial begins treating participants at dose level 1, which all three participants are feasible to receive. No DLTs are observed in this initial cohort, and the design escalates to dose level 2. In the next cohort, the design accrues participants to dose level 2, but one participant in this cohort must receive dose level 1 as this was his or her highest feasible dose. While no participants had a DLT at any level, the design has effectively excluded the non-DLT result of participant 6 and escalates to dose level 3. In the first cohort treated at dose level 3, 1 of 3 participants have a DLT, so the design accrues 3 more participants to dose level 3. However, the next two participants are not feasible to receive dose level 3, so they must receive their highest feasible dose (dose level 2). Both of these participants have a DLT, but the design cannot incorporate their outcomes in dosing decisions, so it enrolls three more participants at level 3, none of whom has a DLT. At this point in the trial, we have observed a DLT rate of 1 of 6 at dose level 3 and a higher rate of 2 of 5 at dose level 2. Yet since the design cannot use safety data observed at unplanned dose levels, the trial escalates to dose level 4. If the design could have incorporated the two DLTs observed at the unplanned dose level 2, then perhaps the trial would not have escalated to dose level 4. At dose level 4, all three participants have DLTs, so the trial stops and dose level 3 is chosen as the MTD according to the decision rules of the 3 + 3. Note that in this example trial dose level 3 is selected as the MTD despite dose level 2 having a larger observed DLT rate. This illustrates that the 3 + 3 may not select a logical MTD in the presence of dose feasibility considerations.

Fig. 1

Simulated example trial guided by the 3 + 3 when dose feasibility issues arise.

We explored the characteristics of the 3 + 3 algorithm over the six combinations of feasibility and toxicity probability curves given in Fig. 2. These scenarios give us a broad range of possible toxicity and feasibility situations to explore. The results for the six scenarios are given in Table 1. The percentage of DLTs at an unplanned dose translates to participants whose DLT was excluded from the dose assignment scheme. Under Scenario 2 this is especially problematic as it occurred in almost one third of all accrued participants. The percentage of trials where there is at least one DLT ignored is large under Scenario 4, which sees more than two thirds of the trials having this flaw. Scenarios 4 and 6 have a troubling percentage of trials that recommend an MTD when a lower dose has a higher estimated DLT rate than the recommended MTD. This sort of behavior is troublesome because it violates our assumptions about the monotonic relationship between toxicity and dose level. The fourth statistic is a general notion of how much of the toxicity information is being ignored in the dose allocation algorithm. Scenarios 1, 4, and 6 have a large number of excluded DLTs because the probability of feasibility and toxicity change quickly at the lower dose levels. The last statistic gives us an idea of how many participants do not have their DLT data included in the dose escalation decisions. Some of these numbers may seem small but consider that the maximum sample size is 24. Under Scenario 4, the value of 13.45 patients on average is equivalent to ignoring 56% of your available patients in the conduct of the trial. This data should be used in implementing a design that accounts for both safety and feasibility. To show that these statistics are not rare events we have included the denominators for statistics 1 and 4 in Table 2. The denominators for statistics 2, 3 and 5 are excluded from the table because they are simply equal to number of trial simulations. We also examined the characteristics of 3 + 3 over 100 combinations of toxicity and feasibility curves (Supplemental Fig. S1). This larger scale simulation provides further evidence that the 3 + 3 will have difficulty using DLT data in the presence of feasibility concerns over a broad range of scenarios. We conducted an additional simulation study where we considered 3 possible dose levels with a maximum of 18 accrued participants while evaluating a maximum of 24 patients for feasibility. The results (Supplemental Fig. S2 and Table S1) were largely similar to the results when considering 4 dose levels with a larger sample size. The simulation scenarios for the 3-dose level study are given in Supplemental Fig. S3. The primary take-away of our study is that when there are dose levels under consideration that are not feasible for some participants to receive, the 3 + 3 algorithm can have major deficiencies in safely accruing participants to the trial. This conclusion may carry over to other rule-based algorithms that focus only on the current dose to make allocation decisions, without borrowing information across dose levels. We have included boxplots to visualize Table 1 in Supplemental Fig. S4 and complete operating characteristics for MTD recommendation and patient allocation in Supplemental Tables S2 and S3.

Fig. 2

Six hypothetical feasibility and toxicity probability curves for 4 dose levels.

Table 1

Results for 6 scenarios under consideration for 4 dose levels.

Statistics for measuring the impact of ignoring dose feasibility (standard errors in parentheses)	Scenario
	1	2	3	4	5	6
% of patients treated at an unplanned dose that had a DLT excluded from use in future dose assignments or MTD rec.	13.20 (0.12)	28.97 (0.71)	19.49 (0.2)	11.15 (0.08)	21.42 (0.43)	10.41 (0.1)
% of trials where there is at least one excluded DLT outcome.	59.96 (0.53)	10.04 (0.3)	37.11 (0.48)	70.89 (0.47)	15.10 (0.37)	63.57 (0.55)
% of trials where a lower dose than the MTD rec had an estimatedDLT rate exceed the estimated DLT rate at the MTD rec.	8.85 (0.28)	3.45 (0.15)	20.13 (0.48)	30.74 (0.53)	10.39 (0.33)	17.26 (0.38)
% of total DLT outcomes that were excluded in executing the design	47.98 (0.44)	5.05 (0.17)	19.69 (0.26)	64.23 (0.4)	6.89 (0.17)	53.11 (0.42)
Average number of trial participants whose DLT outcome did not inform future dose assignments or MTD rec.	10.11 (0.06)	0.47 (0.01)	3.19 (0.04)	13.45 (0.07)	0.89 (0.01)	10.50 (0.05)

Table 2

Denominators for Statistics 1 and 4 for 6 scenarios at 4 dose levels.

Scenario	Denominator 1	Denominator 4
1	101105	27823
2	4739	27208
3	31885	31550
4	134542	23358
5	8900	27687
6	104976	20578

Conclusion

The inability to incorporate all toxicity information is ample evidence to avoid implementation of 3 + 3 in Phase I adoptive cell therapy trials. Several of our simulations produced trials where >50% of DLTs are ignored based solely on the inflexibility of algorithmic designs. For cell therapy trials, we recommend using a design that can incorporate patient toxicity data at unplanned dose levels. All of the available methods [[5], [6], [7]] use statistical models throughout the study to adaptively assign participants. The use of a model allows all available safety and feasibility data to be sequentially used in the trial conduct. No DLT outcomes are excluded when using these methods, which would result in a value of 0 for all five statistics used in this paper to quantify the impact of DLT exclusion. Conversely, since the 3 + 3 does not incorporate feasibility data into the design, leading to the exclusion of DLT data at unplanned dose levels, we recommend that a model-based design be used in designing Phase I cell therapy trials.

The Center for Biologics Evaluation and Research has issued a guidance document to assist sponsors and investigators in designing early-phase clinical trials for cellular therapy products [11]. This guidance provides current recommendations regarding clinical trials in which the primary objectives are the initial assessments of safety, tolerability, or feasibility of administration of investigational products. The document states “For cell therapy products, these early-phase trials often assess not only safety of specific dose regimens and routes of administration, but also other issues, such as feasibility of administration … Therefore, sponsors might include design elements that could help foster further product development.” It also states “In the case of cell therapy products, sponsors should consider designing early-phase trials to identify and characterize any technical or logistic issues with manufacturing and administering the product. Such issues may need to be addressed before proceeding with further product development.” This Food and Drug Administration guidance is calling for early-phase trial designs that formally account for feasibility.

Funding

National Center for Advancing Translational Sciences of the NIH UL1TR003015 (EMB); NCI R01CA247932 and R03CA238966 (NAW).

Declaration of competing interest

The authors declare that they have no known competing financial interests or personal relationships that could have appeared to influence the work reported in this paper.