BACKGROUND/AIMS: For single arm trials, a treatment is evaluated by comparing an outcome estimate to historically reported outcome estimates. Such a historically controlled trial is often analyzed as if the estimates from previous trials were known without variation and there is no trial-to-trial variation in their estimands. We develop a test of treatment efficacy and sample size calculation for historically controlled trials that considers these sources of variation. METHODS: We fit a Bayesian hierarchical model, providing a sample from the posterior predictive distribution of the outcome estimand of a new trial, which, along with the standard error of the estimate, can be used to calculate the probability that the estimate exceeds a threshold. We then calculate criteria for statistical significance as a function of the standard error of the new trial and calculate sample size as a function of difference to be detected. We apply these methods to clinical trials for amyotrophic lateral sclerosis using data from the placebo groups of 16 trials. RESULTS: We find that when attempting to detect the small to moderate effect sizes usually assumed in amyotrophic lateral sclerosis clinical trials, historically controlled trials would require a greater total number of patients than concurrently controlled trials, and only when an effect size is extraordinarily large is a historically controlled trial a reasonable alternative. We also show that utilizing patient level data for the prognostic covariates can reduce the sample size required for a historically controlled trial. CONCLUSION: This article quantifies when historically controlled trials would not provide any sample size advantage, despite dispensing with a control group.
BACKGROUND/AIMS: For single arm trials, a treatment is evaluated by comparing an outcome estimate to historically reported outcome estimates. Such a historically controlled trial is often analyzed as if the estimates from previous trials were known without variation and there is no trial-to-trial variation in their estimands. We develop a test of treatment efficacy and sample size calculation for historically controlled trials that considers these sources of variation. METHODS: We fit a Bayesian hierarchical model, providing a sample from the posterior predictive distribution of the outcome estimand of a new trial, which, along with the standard error of the estimate, can be used to calculate the probability that the estimate exceeds a threshold. We then calculate criteria for statistical significance as a function of the standard error of the new trial and calculate sample size as a function of difference to be detected. We apply these methods to clinical trials for amyotrophic lateral sclerosis using data from the placebo groups of 16 trials. RESULTS: We find that when attempting to detect the small to moderate effect sizes usually assumed in amyotrophic lateral sclerosis clinical trials, historically controlled trials would require a greater total number of patients than concurrently controlled trials, and only when an effect size is extraordinarily large is a historically controlled trial a reasonable alternative. We also show that utilizing patient level data for the prognostic covariates can reduce the sample size required for a historically controlled trial. CONCLUSION: This article quantifies when historically controlled trials would not provide any sample size advantage, despite dispensing with a control group.
Authors: Merit E Cudkowicz; Jeremy M Shefner; David A Schoenfeld; Hui Zhang; Katrin I Andreasson; Jeffrey D Rothstein; Daniel B Drachman Journal: Ann Neurol Date: 2006-07 Impact factor: 10.422