Reporting of master protocols towards a standardized approach: A systematic review.

BACKGROUND: In September 2018 the FDA provided a draft guidance on master protocols reflecting an increased interest in these designs by industry. Master protocols refer to a single overarching protocol developed to evaluate multiple hypotheses and may be further categorized as basket, umbrella, and platform trials. However, inconsistencies in reporting persist in the literature. We conducted a systematic review to describe master protocol reporting with the goal of facilitating the further development and spread of these innovative trial designs.
METHODS: We searched MEDLINE, EMBASE, and CENTRAL from inception to April 25, 2019 for English articles on master protocols. This was supplemented by hand searches of trial registries and of the bibliographies of published reviews. We used the FDA's definitions of master protocols as references and compared them to self-reported master protocols.
RESULTS: We identified 278 master protocol publications, consisting of 228 protocols and 50 reviews. Sixty-six records provided unique definitions of master protocol types. We observed considerable heterogeneity in definitions of master protocols, and over half (54%) used oncology-specific language. The majority of self-classified master protocols (57%) were consistent with the FDA's definitions of master protocols.
CONCLUSION: The terms 'master protocol', 'basket trial', 'umbrella trial', and 'platform trial' are inconsistently described. Careful treatment of these terms and adherence to the definitions set forth by the FDA will facilitate better understanding of these trial designs and allow them to be used broadly and to their full potential in clinical research. We encourage trial methodologists to use these trial designations when applicable.

Introduction

Over the last decade, the concept of master protocols has developed complementary to the precision oncology movement, an emerging field of research designed to target cancers with treatments according to genetic mutations (i.e. targeted therapies) [1]. Master protocols generally refer to a single overarching design divided into sub-studies to evaluate multiple hypotheses, with the goal of improving overall efficiency in clinical trial research [[1], [2], [3], [4]]. Master protocols emphasize the use of a common infrastructure and standardized procedures in order to promote uniformity and are often categorized as basket trials, umbrella trials and platform trials [[1], [2], [3], [4]]. However, researchers have highlighted a lack of standardization and inconsistency within master protocol nomenclature in the published literature [4,5].

The United States (US) Food and Drug Administration (FDA) is one of the leading agencies encouraging and supporting a wider dissemination of master protocols, as demonstrated by their recently released (September 2018) draft guidance on master protocols [2] and a 2017 editorial published by the Directors of Drug Evaluation and Research (CDER) at the FDA in the New England Journal of Medicine [1]. The definitions presented in these papers reflect those that will likely be used by the American research industry in the coming years (Table 1). These definitions are not restricted to a certain field (e.g. oncology), making them widely accessible to a great variety of researchers.

Table 1

FDA definitions and key attributes of master protocols.

Trial Type	Definitions	Key attributes
Master protocol	“…[A] protocol designed with multiple substudies, which may have different objectives and [involve] coordinated efforts to evaluate one or more investigational drugs in one or more disease subtypes within the overall trial structure.”	• Multiple sub studies under one overarching protocol • Interventional (not diagnostic)
Basket trial	“A master protocol designed to test a single investigational drug or drug combination in different populations defined by disease stage, histology, number of prior therapies, genetic or other biomarkers, or demographic characteristics.”	• Single intervention studied • Multiple disease populations
Umbrella trial	“A master protocol designed to evaluate multiple investigational drugs administered as single drugs or as drug combinations in a single disease population.”	• Multiple interventions studied • Single disease population
Platform trial	“[A trial designed to] study multiple targeted therapies in the context of a single disease in a perpetual manner, with therapies allowed to enter or leave the platform on the basis of a decision algorithm”.	• Multiple interventions studied • Perpetual trial • Capacity to add and drop trial arms

The definitions taken from the FDA Draft Guidance on Efficient Clinical Trial Designs and Woodcock et al. (2017) were used to identify and adapt key attributes of master protocols [1,2].

As the adoption of these innovative trial methods continues to spread, ensuring the standardization of master protocol nomenclature will be critically important in order to facilitate communication, regulatory procedures, and patient safety. Efforts towards standardized reporting, such as the CONSORT statement, have historically led to endorsement by journals and improved publication quality [6]. Several reviews of master protocols have already been published [1,3,4,7,8] but, to our knowledge, none have systematically evaluated the reporting of master protocol definitions and the compliance of existing master protocols to existing definitions. We conducted a comprehensive systematic literature review on both individual studies and published reviews to assess the extent to which definitions of master protocols and reporting of these trial types have differed. The purpose of our review is to encourage standardization in the theoretical foundation of master protocols and to facilitate further development and spread of these innovative trial designs.

Methods

Selection strategy and criteria

Any English-language guidance documents from regulatory agencies, review articles on non-original work (e.g. editorials, descriptive or systematic reviews, commentary, and etc.), and original research articles (e.g. registered or published master protocols outlining methods and/or results) pertaining to master protocols were eligible for inclusion (see Supplementary Table 5). We included original research articles that self-classified as a master protocol or that were classified by another source (e.g. a review or methods paper). This study was conducted in accordance with PRISMA reporting guidelines for systematic literature reviews [9]. Draft guidance published by the FDA [2] was reviewed, alongside recent review and guidance publications in this field [[1], [2], [3], [4],10,11], to inform the eligibility criteria and to develop robust search strategies (Supplementary Tables 1–4). We searched MEDLINE, EMBASE and CENTRAL for master protocol publications published from database inception to April 25, 2019. We supplemented this with hand searches of trial registries (ClinicalTrials.gov and ISRCTN registry) to increase the sensitivity of the literature search. Finally, the reference lists of publications identified through either database searches or trial registries were reviewed.

Study selection and data extraction

Three independent reviewers (EGS, JJHPand MJZ) reviewed all abstracts and proceedings identified in the literature searches. The full-text publications of potentially relevant abstracts were retrieved and assessed for eligibility by EGS and JJHP independently and in duplicate. Two independent reviewers screened the bibliographies of the published reviews of master protocols (EGS and JJHP) and trial registries (JJHP and LD). Discrepancies in study selection were resolved by discussion or by a third investigator (KT or EJM), when necessary. Data extraction was completed by two independent reviewers (EGS and JJHP) into a standardized, piloted form. Cross-checking for consistency was conducted by other independent reviewers (MJZ and LD). Definitions of master protocols and their sub-types were extracted. In addition, we extracted whether and how each trial self-classified as a master protocol. A “self-classifying trial” was considered to be any trial calling itself a master protocol, basket trial, umbrella trial or platform trial in its protocol. We also classified all trials according to the FDA master protocol definitions.

Data synthesis

We used descriptive statistics to summarize literature identified through this review. We focused on key themes of master protocols as summarized in prior reviews, such as the types of adaptive trials designs, number of biomarkers studied, number of diseases studied, and number of interventions studied for each master protocol type [[1], [2], [3], [4], [5],11]. We used the FDA definitions as our reference definitions as they represent the definitions that will be used by industry and may therefore already be considered ‘standardized’ [1,2]. The Woodcock et al. paper also represents the most highly cited recent review of master protocols. We considered protocols to be consistent with the FDA definitions only if they hit all ‘key attributes’ of the FDA definitions as presented in Table 1.

Results

Selection of articles

Our search of the literature returned 5346 abstracts, with an additional 150 records identified through trial registries and hand searches. Of 696 abstracts selected for full text review, 278 publications satisfied all eligibility criteria (Supplementary Fig. 1). There were 50 review papers and 228 original research articles pertaining to 99 trials (Supplementary Tables 6 and 7). Eighty-nine of these trials self-classified as master protocols (Supplementary Table 13), while the remaining trials were identified as such by other sources (e.g. reviews or methods papers).

Definitions of master protocols

Reported definitions of master protocol and their sub-types are presented in Supplementary Tables 8–11. Of the 278 records included in this review, 68 records included at least one unique definition of master protocols or its sub-types, resulting in 106 total definitions. Publications that did not report a unique definition typically cited the other review papers included in this study. The most commonly defined trial design were basket trials (n = 35), while definitions of master protocols were reported the least often (n = 20). Reported individual definitions of master protocols (Supplementary Table 8), basket trials (Supplementary Table 9), umbrella trials (Supplementary Table 10), and platform trials (Supplementary Table 11) are provided in the Appendix.

There were four key characteristics found among the 20 unique definitions of master protocols (Fig. 1) [[12], [13], [14], [15], [16]] [[1], [2], [3], [4], [5],8,[17], [18], [19], [20], [21], [22]]. While all sources consistently highlighted that common infrastructure is essential for master protocols, there were some variation with respect to scope, shared ethics review, and centralized data monitoring. Twelve definitions included oncology-specific terms in their definitions (i.e. the reported definitions pertained to oncology or a specific area in oncology only), limiting master protocols to the field of oncology. Only a few definitions specified shared ethics review structures (n = 2) [13,22] and centralized data monitoring (n = 4) [1,2,16,20] as integral to master protocols.

Fig. 1

Master protocol definition characteristics. A summary of key characteristics of the definitions of ‘master protocol’. In this figure, the 20 definitions of master protocols identified were compared based four key attributes of common infrastructure, scope (i.e. oncology focused), centralized data monitoring, and shared ethics review structure.

Definitions of basket trials, umbrella trials, and platform trials

The characteristics of basket, umbrella, and platform trial definitions are illustrated in Fig. 2. While basket trial definitions (n = 35) [[1], [2], [3], [4], [5],11,13,15,18,[23], [24], [25], [26], [27], [28], [29], [30], [31], [32], [33], [34], [35], [36], [37], [38], [39], [40], [41], [42], [43], [44], [45], [46], [47], [48]] were consistent on some aspects, such as that basket trials study multiple disease types (n = 34), other features such as the number of interventions to be studied were inconsistent. Approximately half of the reviewed publications (n = 18) [1,2,4,5,11,13,15,25,30,31,33,35,38,39,41,42,45,49] specified that basket trials are designed to evaluate a single intervention, while around a third (n = 10) [3,18,23,[26], [27], [28], [29],36,40,43] stated that they can evaluate multiple interventions. The remaining definitions (n = 7) specified both [37,44,47] or neither [32,34,46,48]. The number of biomarkers or subgroups studied in a basket trial also varied, with the majority of definitions describing these studies as including only a single biomarker (n = 25) [1,[3], [4], [5],11,13,15,23,[26], [27], [28], [29],[31], [32], [33], [34], [35],[37], [38], [39], [40], [41], [42], [43],45] and considerably fewer including multiple (n = 5) [2,18,25,30,49]. Three basket trials specified the need for seamless design [3,4,39] and one for common control [41].

Fig. 2

Key characteristics of basket trial, umbrella trial, and platform trial definitions. Theree were 35 definitions of basket trials identified, 26 umbrella trial definitions and 25 for platform trial definitions. These different definitions are summarized based on the number of interventions, diseases, and biomarkers involved are illustrated in the left panel (2a), and features of trial designs included in these definitions are mentioned in the right panel (2b). Basket trials are illustrated in color blue, color red for umbrella trials, and color green for platform trials. . (For interpretation of the references to color in this figure legend, the reader is referred to the Web version of this article.)

Umbrella trials (n = 26) were more consistent than basket trials with regards to content (Fig. 2) [[1], [2], [3], [4], [5],11,13,15,18,19,27,28,[31], [32], [33],35,36,38,[45], [46], [47],[50], [51], [52], [53], [54]]. For example, the majority described umbrella trials as limited to a single disease (n = 21) [[1], [2], [3],11,13,15,18,27,[31], [32], [33],35,36,38,[45], [46], [47],[50], [51], [52], [53]], while a single publication proposed the opposite [19]. The remaining four trials specified both [4] or neither [5,28,54]. Most definitions(n = 20) [1,[3], [4], [5],11,15,18,28,[31], [32], [33],35,38,[45], [46], [47],[50], [51], [52],54] described umbrella trials as considering multiple biomarkers with only three [13,27,53] asserting that umbrella trials are restricted to a single biomarker. Moreover, in terms of adaptive trial design, one publication specified the need for seamless design and the ability to add and drop arms [51], and another specified the need for adaptive randomization [55].

Definitions of platform trials (n = 25) consistently referenced the investigation of multiple interventions [[1], [2], [3], [4], [5],7,19,22,30,34,[56], [57], [58], [59], [60], [61], [62], [63], [64], [65], [66], [67], [68], [69]]. References to the number of diseases or biomarkers studied were not included in the definitions as often as for basket or umbrella trials. Moreover, compared to basket and umbrella trials definitions, the definitions of platform trials were much more likely to reference design features, with nine requiring a common control [3,5,19,58,60,63,64,66] and ten mentioning plans for subsequent stages [30,57,58,[61], [62], [63],[65], [66], [67],70]. The ability to add and drop arms was generally consistent across definitions (n = 16/25) [1,3,4,7,19,22,30,56,[58], [59], [60], [61], [62],64,68,70], though two publications specified the ability to drop but not add [65,66] and three the ability to add but not drop [63,67,69]. In addition, although the majority (n = 14/26) [1,4,5,7,30,[56], [57], [58], [59],[62], [63], [64],66,69] described platform trials as perpetual, a minority (n = 3/26) [57,65,70] asserted that platform trials are of finite duration.

Oncology-specific definitions

Basket trial definitions and umbrella trial definitions were equally likely to be oncology-specific, with about three quarters of both basket trial definitions (n = 27/35) and umbrella trial definitions (n = 20/26) using oncology-specific language and thus restricting the scope of their definitions. No platform trial definitions were oncology-specific.

Concurrence of self-classifications with FDA definitions

Overall, only slightly over half of trials self-classifying as a ‘master protocol’, ‘basket trial’, ‘umbrella trial’ or platform trial’ (n = 48/89) were consistent with the FDA definitions (Table 1). Several inconsistencies between the definitions identified in the literature and with the FDA definitions were observed (Fig. 3).

Fig. 3

Proportion of trials classified consistently with FDA definitions. The proportion of master protocols consistent with the FDA definitions are shown in this figure. We compared all self-classified master protocols with the FDA definitions and their attributes (outlined in Table 1). In total, this review identified 89 self-classified master protocols (34 basket trials, 17 umbrella trials, 25 platform trials). 13 protocols self-identified as master protocols and therefore were included under ‘all master protocols’ only.

Of the 41 ‘inconsistent’ trials, four were ‘unclear’; that is, they did not provide enough information for us to designate them as master protocols [[71], [72], [73], [74]]. The majority of these were records from registries that provided insufficient detail with respect to trial design. Four other protocols contained the words ‘master protocol’ in their titles but were in no way master protocols [[75], [76], [77], [78]]. Ten protocols were diagnostic rather than treatment-based [14,16,[79], [80], [81], [82], [83], [84], [85]].

Three trials were misclassified based on FDA definitions; one was listed as a platform trial instead of as a master protocol [86] and one was listed as a platform trial but had no ability to add or drop arms [87]. Others had adaptive designs such as MAMS design (n = 6/32) [[88], [89], [90], [91], [92], [93]] or seamless design (n = 1/32) [94] but were not master protocols. Four protocols enrolled only one arm (n = 4/32) [[95], [96], [97], [98]], while two self-classified as basket trials but contained only one disease population [99,100]. Finally, five trials were classified as basket trials but contained multiple biomarker or intervention-defined cohorts [25,29,[101], [102], [103]]. We have marked these as ‘multi-basket’ trials in Supplementary Table 13.

Discussion

In this study, we used a systematic approach to describe the definitions of master protocols and their sub-types basket trials, umbrella trials, and platform trials that have been reported in the literature, and also to assess the consistency of how different individual studies have self-classified themselves as master protocols. There are considerable variations in the definitions proposed for master protocols and their sub-types, and also considerable inconsistencies in how different trials self-classified themselves as being a master protocol.

Among the reported definitions of master protocols, there were considerable variabilities in terms of the trial designs between the definitions of basket, umbrella, and platform trials. For instance, the definitions of basket and umbrella trials tended to be more biomarker-focused and exploratory than the definitions of platform trials. Only a few basket and umbrella trials definitions specified the need for a common control and seamless design, where the trial could transition seamlessly from an early exploratory or feasibility phase into later pivotal phase (e.g. confirmatory phase) [104,105]. A larger proportion of platform trial definitions, on the other hand, included the use of a common control and pre-specified abilities to add and drop arms during the trial. There were a small number of platform trial definitions that specifically mentioned the use of responsive adaptive randomization [7] [4,61,66], where allocation ratio can be adjusted to favor the arms over the course of the trial based on interim data [104,105]; however, it should be noted the responsive adaptive randomization is a type of adaptive trial designs that can also be applied to non-master protocols.

Our review also indicates that 52% (n = 48/93) of definitions were specific to oncology. Expanding the definitions of master protocols to be non-disease specific will be important to improve adoption of master protocols to the disease areas outside of oncology. Use of non-oncology-specific definitions, such as those presented by the FDA [2] and Butler et al. [64], allow for broader use of these terms, facilitating their spread to other fields in need of efficient trial design such as infectious diseases [16,59,64] Alzheimer's [106] and autoimmune diseases [107,108].

When we compared the consistency of master protocol classifications, only 56% (n = 41/73) of self-classified master protocols were fully consistent with the master protocol definitions proposed by the FDA. We decided to use the FDA definitions as a common reference since the 2017 NEJM editorial by Woodcock and LaVange was the most commonly cited paper. However, it may be important to broaden and expand upon these definitions, as broadening the flexibility of these definitions may contribute to better adherence. For instance, the FDA defined basket trials as having a single intervention, while Table 2 shows that many other basket trial definitions allow for multiple interventions. This issue might be solved by either permitting basket trials to have multiple interventions or by specifying an additional trial type such as a ‘multi-basket’. We therefore propose the cautious adoption of the FDA definitions as standardized definitions, with an understanding that these definitions should continue to be refined. It may be also worth considering whether platform trials need to be perpetual, or whether it may also be possible to have a finite platform trial with the capacity to add and drop arms up to a certain limit.

There are strengths and limitations to our study. The strength of our review lies in the comprehensiveness of our search, which included a systematic query of medical literature databases and clinical trial registries as well as a hand search of relevant reviews and methods papers. However, it is possible that our review may have been limited by variability of indexing and lack of MeSH terms that currently exist for master protocols. The creation of MeSH terms based on standardized definitions for ‘master protocol’, ‘basket trial’, ‘umbrella trial’ and ‘platform trial’ will be an important step to ensure that researchers can efficiently access and build on existing master protocol research. It is in the best of interests of all of those impacted by clinical trials to contribute to the development and maintenance of consistent and inclusive definitions.

Our review confirms that there is considerable work to be done towards the standardization of these terms, confirming the conclusions of previous reviews [4,5]. The standardization of master protocol nomenclature will become increasingly important as countries such as Canada and the US look towards drug approval using these trial designs [2,109,110]. A common understanding of these terms will be necessary to ensure that regulatory boards and potential trial participants know which questions to ask of trial administrators pertaining to patient safety and trial power. Standardizing master protocol definitions may also strengthen collaboration and facilitate further research attributed to more uniform data collection and trial documentation.

The framework of master protocols promotes clinical research that is highly efficient, making the best use of trial infrastructures, resources, and time. However, the terms, ‘master protocol’, basket trial’, ‘umbrella trial’ and platform’ have been inconsistently used and defined in the literature to date. Adoption of and careful adherence to the FDA definitions and will facilitate better understanding of these important new trial design frameworks and encourage them to be used broadly and to their full potentials. We encourage trial design methodologists to use these designations whenever applicable.

Funding

JP was financially supported by the Mitacs Accelerate Fellowship PhD grant. The other authors received no financial support for the research, authorship, and/or publication of this article.

Conflicts of interest

The authors have no conflicts of interest to report.