Aviv Ladanie1, Benjamin Speich2, Matthias Briel3, Francesco Sclafani4, Heiner C Bucher5, Arnav Agarwal6, John P A Ioannidis7, Tiago V Pereira8, Benjamin Kasenda9, Lars G Hemkens10. 1. Basel Institute for Clinical Epidemiology and Biostatistics, Department of Clinical Research, University Hospital and University of Basel, Basel, Switzerland; Swiss Tropical and Public Health Institute (Swiss TPH), Basel, Switzerland. 2. Basel Institute for Clinical Epidemiology and Biostatistics, Department of Clinical Research, University Hospital and University of Basel, Basel, Switzerland; Centre for Statistics in Medicine, Nuffield Department of Orthopaedics, Rheumatology and Musculoskeletal Sciences, University of Oxford, UK. 3. Basel Institute for Clinical Epidemiology and Biostatistics, Department of Clinical Research, University Hospital and University of Basel, Basel, Switzerland; Department of Health Research Methods, Evidence, and Impact, McMaster University, Hamilton, Canada. 4. Gastrointestinal Unit, Department of Medical Oncology, Institut Jules Bordet, Université Libre de Bruxelles (ULB), Brussels, Belgium. 5. Basel Institute for Clinical Epidemiology and Biostatistics, Department of Clinical Research, University Hospital and University of Basel, Basel, Switzerland. 6. Department of Health Research Methods, Evidence, and Impact, McMaster University, Hamilton, Canada; Department of Medicine, University of Toronto, Toronto, Ontario, Canada. 7. Meta-Research Innovation Center at Stanford (METRICS), Stanford University, Stanford, CA, USA; Department of Medicine, Stanford University School of Medicine, Stanford, CA, USA; Department of Health Research and Policy, Stanford University School of Medicine, Stanford, CA, USA; Department of Biomedical Data Science, Stanford University School of Medicine, Stanford, CA USA; Department of Statistics, Stanford University School of Humanities and Sciences, Stanford, CA, USA. 8. Health Technology Assessment Unit, Institute of Education and Health Sciences, Oswaldo Cruz German Hospital, São Paulo, Brazil. 9. Basel Institute for Clinical Epidemiology and Biostatistics, Department of Clinical Research, University Hospital and University of Basel, Basel, Switzerland; Department of Medical Oncology, University Hospital and University of Basel, Basel, Switzerland; Department of Haematology/Oncology and Palliative Care, Klinikum Stuttgart, Stuttgart, Germany. 10. Basel Institute for Clinical Epidemiology and Biostatistics, Department of Clinical Research, University Hospital and University of Basel, Basel, Switzerland. Electronic address: lars.hemkens@usb.ch.
Abstract
BACKGROUND AND OBJECTIVE: Novel cancer therapies are often approved with evidence from a single pivotal trial alone. There are concerns about the credibility of this evidence. Higher validity may be indicated by five methodological and statistical characteristics of pivotal trial evidence that were described by the U.S. Food and Drug Administration (FDA), which may corroborate the reliance on a single trial alone for approval decisions. STUDY DESIGN: We did a metaepidemiologic evaluation of all single pivotal trials supporting FDA approval of novel drugs and therapeutic biologicals for cancers between 2000 and 2016. For each trial, we determined the presence of these five characteristics, which we operationalized as (1) large and multicenter trial (≥200 patients; more than one center); consistent treatment benefits across (2) multiple patient subgroups (in view of FDA reviewers), (3) multiple endpoints (including overall survival, progression-free survival, response rate, health related quality of life), and (4) multiple treatment comparisons (e.g., multi-arm studies); and (5) "statistically very persuasive" results (P-values <0.00125). RESULTS: Thirty-five of 100 approvals were based on evidence from a single pivotal trial without any further supporting evidence on beneficial effects (20 randomized controlled trials and 15 single-arm trials). The number increased substantially from one approval before 2006 to 23 after 2011. Sixty-six percent (23/35) of the trials were large multicenter trials (median 301 patients and 63 centers). Consistent effects were demonstrated across subgroups in 66% (23/35), across endpoints in 43% (15/35), and across multiple comparisons in 3% (1/35). Very low P-values for the primary endpoint were seen in 34% (12/35). At least one of the corroborating characteristics was present in 94% (33/35) of all approvals, two or more were present in 54% (19/35), and none had all characteristics. CONCLUSIONS: Single pivotal trials typically have some of the corroborating characteristics, but often only one or two. These characteristics need to be better operationalized, defined, and reported and whether single trials with such characteristics provide similar evidence about benefits and harms of novel treatments as multiple trials would do needs to be shown.
BACKGROUND AND OBJECTIVE: Novel cancer therapies are often approved with evidence from a single pivotal trial alone. There are concerns about the credibility of this evidence. Higher validity may be indicated by five methodological and statistical characteristics of pivotal trial evidence that were described by the U.S. Food and Drug Administration (FDA), which may corroborate the reliance on a single trial alone for approval decisions. STUDY DESIGN: We did a metaepidemiologic evaluation of all single pivotal trials supporting FDA approval of novel drugs and therapeutic biologicals for cancers between 2000 and 2016. For each trial, we determined the presence of these five characteristics, which we operationalized as (1) large and multicenter trial (≥200 patients; more than one center); consistent treatment benefits across (2) multiple patient subgroups (in view of FDA reviewers), (3) multiple endpoints (including overall survival, progression-free survival, response rate, health related quality of life), and (4) multiple treatment comparisons (e.g., multi-arm studies); and (5) "statistically very persuasive" results (P-values <0.00125). RESULTS: Thirty-five of 100 approvals were based on evidence from a single pivotal trial without any further supporting evidence on beneficial effects (20 randomized controlled trials and 15 single-arm trials). The number increased substantially from one approval before 2006 to 23 after 2011. Sixty-six percent (23/35) of the trials were large multicenter trials (median 301 patients and 63 centers). Consistent effects were demonstrated across subgroups in 66% (23/35), across endpoints in 43% (15/35), and across multiple comparisons in 3% (1/35). Very low P-values for the primary endpoint were seen in 34% (12/35). At least one of the corroborating characteristics was present in 94% (33/35) of all approvals, two or more were present in 54% (19/35), and none had all characteristics. CONCLUSIONS: Single pivotal trials typically have some of the corroborating characteristics, but often only one or two. These characteristics need to be better operationalized, defined, and reported and whether single trials with such characteristics provide similar evidence about benefits and harms of novel treatments as multiple trials would do needs to be shown.
Authors: Aviv Ladanie; Andreas M Schmitt; Benjamin Speich; Florian Naudet; Arnav Agarwal; Tiago V Pereira; Francesco Sclafani; Amanda K Herbrand; Matthias Briel; Juan Martin-Liberal; Thomas Schmid; Hannah Ewald; John P A Ioannidis; Heiner C Bucher; Benjamin Kasenda; Lars G Hemkens Journal: JAMA Netw Open Date: 2020-11-02
Authors: Xiaolu Cheng; Shuo-Yu Lin; Jin Liu; Shiyong Liu; Jun Zhang; Peng Nie; Bernard F Fuemmeler; Youfa Wang; Hong Xue Journal: Int J Environ Res Public Health Date: 2021-04-09 Impact factor: 3.390
Authors: Lourens T Bloem; Rick A Vreman; Niels W L Peeters; Jarno Hoekman; Menno E van der Elst; Hubert G M Leufkens; Olaf H Klungel; Wim G Goettsch; Aukje K Mantel-Teeuwisse Journal: Clin Transl Sci Date: 2021-05-01 Impact factor: 4.689