Practical considerations for the implementation of adaptive designs for oncology Phase I dose-finding trials.

The traditional 3 + 3 design continues to be commonly used for Phase I dose-finding oncology trials, despite increasing criticisms and development of innovative methods. Unfortunately, it is a challenge to convince principal investigators to use novel designs. The goal of this paper is to persuade researchers to break away from 3 + 3 design and provide potential solutions to better designs and implementation strategy. We reviewed the statistical methods for adaptive Phase I designs. The barriers among all the major components of the implementation team have been emphasized and potential solutions have been discussed. Institutional support to the principal investigators and statistician, as well as to other team members is essential to design and implement adaptive trials in academic medical institutions.

Oncology trials are coupled with a very limited number of available patients, especially for a certain rare disease type. In the last few decades, the statistical community has extensively developed and published many innovative Phase I trial designs that have improved operating characteristics over the widely used traditional standard 3 + 3 design [1-3]. Usually, the critical operating characteristics for oncology Phase I trials are: probability of each dose level being chosen as the maximum tolerated dose (MTD), expected number of patients treated at each dose level, expected probability of dose-limiting toxicity (DLT) at MTD, probability of stopping early and expected total number of patients. However, the 3 + 3 design is still the most commonly used design in oncology Phase I dose-finding trials [2,4-8] and remains widely accepted by physicians/researchers. This is primarily due to the simplicity of the design, implementation and data analysis. The primary objective of a Phase I trial is to determine the safety profile and identify a recommended Phase II dose level (RP2D) including the schedule of an experimental drug or drug combination in the target patient population(s). Major technical components in the design of a Phase I trial typically include dose levels (starting dose level, intermediate dose and end dose), dose escalation method, trial-specific DLT and definition of the MTD or RP2D. The principles of a Phase I trial design are ethical consideration, safety and preliminary efficacy. Specifically, properly designed Phase I oncology trials should: minimize patients’ exposure to subtherapeutic dose levels, minimize patients exposed to serious or life-threatening adverse events and identify the optimum dose with therapeutic effect while ensuring safety.

The classic 3 + 3 design starts at a low dose level – dose level 1. A cohort of three patients are assigned to a dose level, this is then escalated or de-escalated according to the following rules. If 0/3 DLTs, assign three new patients to the next higher dose level; if 1/3 DLTs, assign additional three patients at this dose level. If these additional three patients have no DLTs (1/6 DLTs in all patients), assign three patients to the next higher dose. If ≥2/6 DLTs, stop and de-escalate to the next lower dose. If ≥2/3 DLTs, stop and assign three patients to the next lower dose unless there are already six patients at that level. Under this 3 + 3 design, the MTD is the highest dose level with ≤1/6 DLT. The targeted toxicity probability at MTD is in the range of 17–33%. The 3 + 3 design has some advantages, it is simple to understand, easy to implement and there is no need for sample size calculations. However, this design has several disadvantages, it uses a fixed cohort size throughout the study, MTD is determined within a range with no prespecified toxicity probability (i.e., 17–33%), if it underestimates the MTD patients could be treated at subtherapeutic dose levels in subsequent Phase II studies, it has a lower probability of selecting true MTDs and high variability of MTD estimates [9,10] and it cannot be used for delayed toxicity studies.

Rigorous statistical evidence has shown that there are alternative designs that have great potential benefits; however, these novel designs are still underutilized in the real world and more education and training are needed before physicians and their research teams can widely accept these novel designs. A recent study by van Brummelen et al. [2] reported that only 6.4% (11 of 172 oncology Phase I trials) used model-based approaches from 2012 to 2014, slightly increased from about 1.6% (20 of 1235) in the period 1991–2006 [7] and 3.3% (6 of 181) during 2007–2008 [8].

In this article, we review the novel statistical designs for Phase I oncology trial first. The barriers among all the major components and key players of the clinical trial implementation team are explored. The practical challenges and potential solutions to overcome those barriers are discussed. Finally, we share our experience on how to promote adaptive Phase I oncology trial designs in our institution.

Types of Phase I trial designs

According to the statistical methods and assumptions used, the Phase I adaptive designs can be classified into three categories: algorithm-based designs, model-based designs and model-assisted designs. Algorithm-based designs have all the rules determined ahead of time, whereas a model-based design uses a statistical model to determine the dose escalation, de-escalation or stay on a real-time basis. Model-assisted design is a mixture of the previous two designs. The dose escalation/de-escalation rules have been determined using a statistical model before conducting the trial.

Among the algorithm-based designs, beside the conventional 3 + 3 design, the accelerated titration design was discussed in 1997 by Simon et al. [11]. This design consists of an initial cohort of one patient per dose level and only escalates to the next level if no DLT and no predefined toxicity events are observed (i.e., grade 2 toxicity events of any type). When the first DLT or predefined toxicity event is observed, the cohort size is expanded at current dose level and it reverts to the standard 3 + 3 design. The accelerated titration design can reduce the number of patients treated at the low dose with an aggressive dose escalation schedule, which is an advantage for Phase I trials with large number of dose levels. Rolling six is an algorithm-based extension of the 3 + 3 design [12]; the patients can be continuously enrolled at the same dose level, up to a total of six. The dose level allocated to a new patient is based on the number of patients currently enrolled, the number of patients experiencing DLT and the number of patients whose evaluation is pending at the time of new patient entry. De-escalation occurs when two or more DLTs occur at a dose level, whereas escalation can be performed when 3/3, 4/4, 5/5 or ≥5/6 patients are evaluated without DLT. Once six patients have been enrolled at the current dose level, enrollment will stop until at least five of those six patients have completed the evaluation. The rolling 6 design can reduce the total enrollment time by allowing continuous enrollment of up to six patients when not all of them are evaluated for toxicity.

For the model-based adaptive designs, continual reassessment method (CRM) [13] and its various modifications (Bayesian Model Averaging-CRM, TriCRM, time-to-event CRM [TITE-CRM]) [14-16] use statistical probability to model the dose toxicity curve and determine the MTD [17]. The parameters in the probability model are sequentially updated when patients are enrolled and pass the evaluation period of DLT. A new cohort of patients will be assigned to a dose level whose toxicity estimate is the closest to the target probability. Multiple models and approaches can be used to estimate the dose level. CRM and its modifications allow for different target probabilities of the toxicity rate as opposed to the fixed range assumed by the 3 + 3 design and incorporate previous model-based knowledge of toxicity for each dose level. The TITE-CRM provides estimates of the cumulative probability of late-onset toxicity over several cycles when the toxicity data are still pending for some enrolled patients. Escalation with overdose control is also a Bayesian parametric model-based design that can be used to characterize the dose-toxicity relationship [18] while incorporating ethical constraints of minimizing chance of treating patients at unacceptably high doses. This design keeps the expected proportion of patients overdosed less than a prespecified value based on the clinicians’ input about overdosing. However, all the model-based algorithms have a common disadvantage; they are not easily implemented when compared with the 3 + 3 design. Statisticians need real-time information to update the parameters in the model, which requires the rest of the study team to have close and timely collaboration with statisticians, or include another study team member with more training to be able to analyze the real-time data in order to determine the next steps in the trial. Software applications and online tools have been developed to support the implementation of the model-based designs [9,10].

In the most recent decade, a new type of adaptive Phase I design has been developed using Bayesian methods, but could be implemented like an algorithm-based design with predetermined dose escalation/de-escalation rules. They are called model-assisted designs [19]. This type of design does not require the model updating or any software monitoring during an ongoing trial but has better operation characteristics when compared with the algorithm-based designs. Ivanova et al. proposed a simple interval-based design: Cumulative Cohort Design [20]. The dose escalation/de-escalation rules are based on the calculated probabilities with Markov chain theory applied to group up-and-down designs. The modified toxicity probability interval design (mTPI) proposed in 2010 combines posterior inference of Bayesian-based methods with simple up-and-down rules [21,22]. The dose escalation/de-escalation rules of mTPI are based on the unit probability mass of the prespecified dose interval after obtaining the posterior toxicity probability for each dose. In mTPI-2 [23], the single underdosing interval used with mTPI is divided into multiple underdosing intervals and unit probability masses are calculated for all intervals, which makes it more practical for implementation. Another model-assisted design is the Bayesian optimal interval design (BOIN) [24,25], which assumes Phase I dose finding as a decision-making problem. With this design, the MTD is determined while minimizing the probability of inappropriate dose assignments for patients. The BOIN design uses isotonically transformed values of the observed toxicity rates from the pooled information across all doses to obtain an accurate statistical estimate of the MTD after a trial has ended. It can target any prespecified DLT rate. Cohort sizes can be modified while the trial is in progress. Accelerated titration design is a special case of the BOIN design. The BOIN design also has overdose toxicity controls, and therefore it is comparable to the escalation with overdose control. Keyboard design has been developed in 2017 [26], which is similar to mTPI-2, but more transparent and easier to understand. More recently, BOIN has been extended to handle late-onset toxicity [27] and combination of two drugs with multiple dose levels for each drug [28,29]. Software applications and online tools are also available to support the model-assisted designs.

Reasons why novel designs are not being routinely implemented

While these adaptive trial designs have demonstrated great potential benefits [1,10,17,30,31], these alternative designs have yet to be widely embraced. In most situations, the principal investigators (PIs) or physicians for clinical trials do not feel that there is anything wrong with using the same 3 + 3 design they have been using for so many years. From the physicians’ perspective, choosing a trial design that it is well understood and easy to implement is of utmost importance. For example, CRM or any of its alternatives are considered to be statistically and computationally complex by many physicians. CRM design is perceived as a ‘black box’ in terms of dose allocations, as opposed to the very simple 3 + 3 rule. Moreover, the determination of the prior distribution of the toxicity function is also considered subjective. The performance of the CRM could also be impacted by model mis-specifications [3]. After evaluation of the risks and benefits of the adaptive trial design, physicians opt for the simple and traditional 3 + 3 design. Thus, an increase in education and training is required in order to promote the various scientific benefits provided by more rigorous designs. A recent survey by the National Institute for Health Research Early Phase Statistics Working Group identified the top three barriers against the use of advanced design as ‘lack of training,’ ‘PI preference’ and ‘limited resources for study design before funding.’ They also suggested that ‘ignorance of the benefits of model-based designs and disadvantages of algorithm-based designs is blocking wider implementation of more efficient Phase I trial designs’ and ‘encourage funding bodies and ethics committees to question the use of algorithm-based designs, conduct statistical reviews of all Phase I trial applications and embrace model-based studies’ [9].

The research team involved in the clinical trial

To tackle these barriers, first we need to understand the research team that will be involved. The key personnel to be involved in the implementation of Phase I oncology trials in the academic institutions are the physician and the other research team members, including a PI, normally a physician who leads the trial design and implementation, a Statistician that designs the trial, oversees and conducts data analysis, a Research Nurse who is in charge of recruiting and communicating with patients during the trial, a Research Coordinator that works on the implementation of the trial and a Data Manager and IT personnel that are responsible for maintaining data capture system and data collection. The physician plays the pivotal role in selecting the trial design, especially for the investigator-initiated trials. Besides the interaction between the physicians and the statisticians, the rest of the team members are also important in a trial's implementation and conduct. Team effort is required to successfully implement adaptive clinical trials.

Therefore, academic institutions should develop a standard operation procedure (SOP) to educate and train all team members involved in the clinical trial by clarifying the trial design and implementation process.

Is the statistician ready?

It is well documented that the conventional 3 + 3 design underestimates the MTD and does not have optimal operation characteristics. The methodology of novel adaptive designs has been developed and well published [1,2,18,31]. These new designs are in use, especially in immunotherapy trials. Immunotherapy has demonstrated impressive clinical activity but also revealed different toxicity profiles and mechanisms of action from traditional cytotoxic drugs. For immunotherapy studies, the focus and interest may be the optimal biological dose instead of the MTD, which could require new strategies for dose selection and trial design. Statisticians are aware that conventional designs need to adapt to the changes in cancer treatment. The statistical community developed adaptive designs with significantly better performance than conventional designs, regardless immunotherapy or other drugs.

Many adaptive Phase I designs are available for use as we summarized above. Zhou et al. [3,31] and Ruppert and Shoben [10] have conducted comprehensive comparisons among the rule-based (3 + 3 design), model-based designs and model-assisted designs. The results showed that model-based and model-assisted designs substantially outperform the 3 + 3 design. The model-assisted designs have comparable performances to model-based designs in regard to the accuracy of identifying the MTD, allocating patients to the MTD and the risk of overdosing patients.

Although many adaptive Phase I designs have been developed, only a few are implemented in practice because they often require complicated statistical modeling and computation. In some cases, statisticians might not have enough financial support or time to design or implement the adaptive designs as such designs frequently require more time to run simulations in order to determine the optimal values for design parameters. In addition, statisticians may not feel comfortable using or implementing these unfamiliar types of adaptive designs.

To embrace the advanced adaptive trial designs in an institution, statisticians should be provided with institutional support for continuing education such as workshops, seminars or conferences as well as access to various tools and infrastructure developed by the statistical community who specialize in oncology clinical trials. Guidelines on how to choose from the existent statistical methods to design oncology Phase I trials are also required.

Is the physician ready?

Physicians who serve as PIs of clinical trials should be engaged and motivated to use and implement the advanced adaptive trial designs. However, the knowledge of the adaptive designs is, in general, currently lacking among physicians. The major barriers for them to design and implement the adaptive trials are likely due to the limited knowledge of the mechanism for the adaptive designs, lack of experience/confidence on how to conduct the adaptive trials and uncertainty on how to interpret the results. Interactions between statisticians and physicians to promote adaptive clinical trial designs resonate well in the paper written by the National Institute for Health Research Early Phase Statistics Working Group [9]. While the statisticians provide suggestions, the final decision on the study design is usually made by physicians. The selection of the trail design could be affected by the culture of the physician's cancer institution, physician's training and the sponsor of the clinical trial. For example, large cancer centers, especially the cancer centers that focus on clinical research and development of statistical methodologies for clinical trial designs, such as MD Anderson (TX, USA), would be more likely to use more advanced designs. Physicians who received training or had previous experience in those institutions would be more likely to use adaptive designs as well. Next, physician's knowledge about adaptive design is also important when selecting a trial design. If the physicians have knowledge about adaptive designs, they will be more inclined to use these advanced methods. Therefore, it is beneficial to invite some well-known external statisticians to give seminars and meet with physicians to distribute the knowledge of advanced trial designs. The decision for a trial design also depends on the sponsor of the study, institutes such as the NIH (MD, USA) or the pharmaceutical industry. After the physicians submit letters of intent, the sponsor could suggest or mandate changes to the statistical design of study. In addition, some adaptive designs may be hard to understand or implement. For example, physicians perceive CRM as a ‘black box’ type design, due to low comfortability with not knowing dose escalation/de-escalation plan ahead of time. Physicians also worry that the study statistician may not be available to provide the support when needed owing to delay caused by real time decision-making process for some adaptive designs. On the other hand, physicians do not have time to consider adaptive design options due to the last minute submission.

In order to overcome these barriers, statisticians need to be ready and comfortable to design adaptive trials after receiving the institutional support for continuing education and be able to explain the adaptive design using nonstatistical language. PIs are responsible to provide reasonable financial support from the clinical trial budget. Statisticians are recommended to be involved in the planning and designing of the trials as early as possible. The communication and collaboration of the study team should be maintained throughout the entire course of the trial to achieve rigorous and reproducible results by providing periodic data quality assurance checks, data analyses and interpretation of results. Most importantly, physicians should feel confident with the adaptive trial designs, with a high level of knowledge of the advanced trial designs and a clear understanding of the trial implementation procedures. Specifically, physicians should be provided with two primary resources: sufficient statistical support, including educational resources for new novel methods, and institutional support with necessary infrastructural resources and training for the research team who design and implement the clinical trials.

Communication between the statistician & the physician

While the PI of a clinical trial is the primary personnel who oversees the trial, the statistician is also actively involved in the trial from inception, design, implementation and termination of a trial (Figure 1). The communication between statisticians and physicians is essential throughout the entire course of the study; the PI, normally the primary physician treating patients, needs to be exposed and educated about these new designs before the trial design. The time from the call for proposal or letter of intent to submission is limited in many cases. If the physician had a 3 + 3 design in mind, there will be little time for the study statistician to convince the physician that using an adaptive design will be, most likely, the better option. To address this issue, there are two key components that need to be developed and implemented. First, education: the institution and the biostatisticians should provide training/continuing education to clinician scientists about new trial designs. This training can be delivered in the form of seminar, workshops and journal clubs that present the operational details of the adaptive design and comparisons to the traditional 3 + 3 design. In addition, training material such as booklets and posters should be developed and shared with physicians. These materials can be distributed to physicians and research coordinators in the research meetings, clinical trial group meetings, journal clubs and department faculty meetings. These training and education opportunities for physicians will enhance their knowledge of adaptive trial design and communicate the institutional support to implement adaptive designs. Second, study planning time: there should be enough time for the collaborative team including the physician and statistician to plan, design and prepare the study before submission. Statisticians should be involved in the study as a coinvestigator and work with physicians to develop the trial together. During the design process, sufficient planning time will give statisticians the opportunity to develop and plan the details of the adaptive design with the physician, rather than just adding a statistical considerations section into an ‘almost completed’ protocol. Long-term collaborations between physicians and statisticians are encouraged and will result in a trust relationship, which will allow statisticians to propose more novel designs and physicians will feel comfortable to use these novel designs. While the use of adaptive trial designs could improve the care of patients in clinical trials by improving the chance to be treated at the optimal dose level, the collaboration process needs to be enhanced. Both statisticians and physicians should work together to embrace the changes of the working flow when adaptive designs are used.

Figure 1.

An example of standard operation procedure to illustrate the involvement of biostatistician in the design and implementation of Phase I oncology trial.

Is the rest of the clinical research team ready?

Besides physicians and statisticians, the traditional research team also includes data managers and coordinators, research nurses and IT support. As we previously stated, statisticians work closely with physicians to design the adaptive Phase I trial. However, without the rest of the team, it is impossible to successfully implement an adaptive trial.

During the design of the adaptive trial, the statistician needs to listen and learn from other team members and work together to ensure the details for implementation are well understood by all parties. The research team, including research nurse, data coordinator, regulatory staff and data capture system analyst are involved in the design the electronic case report form with the physician and statistician. The study statistician is required to be involved in all forms development to ensure the data are being collected in a manner consistent with the proposed analysis plan. Moreover, a communication plan for dose escalation/de-escalation decision-making needs to be developed. The statistician should also be included in the implementation/kickoff meeting in order to inform and educate the rest of the team about the details of this novel design and assess the readiness for trial. It is beneficial to have a training meeting before the trial starts to make sure all members understand their roles in the trial and how to reach other parties as needed. After the trial is open, communication among physicians, statisticians and other study team members remains important. For example, statisticians should receive email notifications regarding important trial events, such as presence/determination of DLTs, determination of the next dose level and protocol amendments. More frequent quality assurance checks may be performed depending on the trial-specific needs as determined jointly among the statistician, PI and other study team members. Assessment of stopping rules as well as dose escalation/de-escalation decisions should be made jointly among the statistician, physician and other study team members. As needed, a secondary statistician will be identified and included as key personnel in the protocol. When immediate decision-making (e.g., trials with model-based design) is required, the secondary statistician could be contacted if the primary statistician is unavailable. Each research institution is highly recommended to develop SOPs for clinical trial implementation, monitoring and conducting the trials. Figure 1 provides an example of SOP to show when the statistician will be involved in the trial.

An example

The Ohio State University Comprehensive Cancer Center (OH, USA) is a National Cancer Institute – designated comprehensive cancer center aligned with a nationally ranked academic medical center and a freestanding cancer hospital on the campus of one of the nation's largest public universities. The Biostatistics Shared Resource provides support on study design, data management and statistical analysis for oncology trials. A total of 58 Investigator-initiated Phase I trials have been implemented since 2008. Motivated by the visit and workshop of D Berry in 2013, the Biostatistics Shared Resource team has invested time and effort in the area of Bayesian designs. Lectures and workshops have been developed to promote the use of novel designs. Investigators are now accepting the idea of adaptive Phase I designs and understanding that adaptive designs have better operation characteristics and can achieve RP2D more accurately. However, they still see the model-based designs as ‘black box’ in terms of dose allocations. Only a couple of Phase I trials with accelerated titration design (algorithm-based design) were implemented in the last few years. Meanwhile, our statisticians have worked with the Clinical Trial Office to develop SOP to improve clinical trials implementation, monitoring and data collection. This also provides the opportunity for the study team to collaborate for future implementation of adaptive trials.

More recently, our team has worked on proposing the novel model-assisted BOIN design and, after some deliberations and discussions, the first BOIN phase trial was implemented at the Ohio State University Comprehensive Cancer Center in 2017. During the past 2 years, we have designed more than ten Phase I oncology trials using BOIN design and two of them have been implemented. We are closely working with PIs, research team and Clinical Trial Office to monitor the DLT and guide them to enroll patients in the correct dose levels. In early 2019, we invited Ying Yuan to give the CCC-grand round seminar about BOIN design.

In a recent case, physician A contacted statistician B for a Phase I study design of combination drugs C and D for refractory small-cell lung cancer. Physician A has been collaborating with statistician B for a few years and was aware of the disadvantages of 3 + 3 design, though he was not sure how the adaptive trial works. Statistician B introduced the BOIN design to physician A and explained how BOIN design (dose-escalation rules, MTD determination, etc.) works in their face-to-face meeting. Physician A provided the expected DLT rate at MTD based on the toxicity profiles for drugs C and D, as well as the possible maximum number of enrollment during the study period. Based on the provided information, statistician B ran the simulations for different scenarios. The simulation results, dose escalation/de-escalation rule table, the trial design in the detailed steps were presented to physician A. Physician A found the BOIN design was very easy to understand and decided to use this design after discussing with other research team members.

Conclusion

There is strong evidence of superior operation characteristics when adaptive Phase I designs are used, from both simulations in the published papers that we have reviewed [9,10] as well as currently implemented adaptive trials. In particular, when more transparent and user friendly adaptive designs have been established, Phase I dose-finding oncology trials are more likely to be designed in an adaptive way. However, emphasis must be placed upon the fact that there are still gaps to implement Phase I adaptive designs [9,32-34], especially in the academic medical institutions. To encourage physicians to consider adaptive designs, they should be provided with the opportunity to learn about these designs and build confidence with the rest of research team on the implementation process. All these approaches mentioned above cannot be successful without institutional support.

Future perspective

Our statistical community is extensively developing and publishing innovative Phase I trial designs that improve operating characteristics over the widely used 3 + 3 design. At the same time, with increasing the institutional support, more education and training of adaptive design will be offered to physicians, statisticians and their research teams. This will promote the design and implementation of adaptive trials, eventually resulting in improving patient outcomes. We expect more Phase I trials to be designed with novel statistical methods to accurately identify the MTD in the next few years.

Statistician lacks the knowledge, planning time and financial support to propose adaptive trials.

Physicians are unfamiliar with adaptive methods, have insufficient interaction with statisticians and can lack confidence in how to conduct and interpret results of adaptive designs.

In order to overcome these barriers, years of experience have led to different perspective of these gaps and the proposal of some practical approaches:

Principal investigators or institutions provide sufficient financial support/time effort to statisticians to design or implement the adaptive designs.

Institutions provide support for continuing education and access to various tools and infrastructure developed by the statistical community.

Statisticians receive sufficient training about how to select and decide on the different adaptive designs and communication skills to present to physicians and other team members.

Maintain the communications and collaborations of the whole study team throughout the entire course of the trial.

Develop standard operation procedures for clinical trial implementation, monitoring and conducting the trials.