Douglas D Thompson1, Hester F Lingsma2, William N Whiteley3, Gordon D Murray4, Ewout W Steyerberg2. 1. Edinburgh Hub for Trials Methodology Research, Centre for Population Health Sciences University of Edinburgh, Edinburgh EH89AG, UK. Electronic address: d.thompson-3@sms.ed.ac.uk. 2. Department of Public Health, Centre for Medical Decision Sciences, Erasmus MC, Rotterdam, The Netherlands. 3. Edinburgh Hub for Trials Methodology Research, Centre for Population Health Sciences University of Edinburgh, Edinburgh EH89AG, UK; Centre for Clinical Brain Sciences, Chancellor's Building, Royal Infirmary of Edinburgh, 49 Little France Crescent, Edinburgh, EH16 4SB, UK. 4. Edinburgh Hub for Trials Methodology Research, Centre for Population Health Sciences University of Edinburgh, Edinburgh EH89AG, UK.
Abstract
OBJECTIVES: Covariate adjustment is a standard statistical approach in the analysis of randomized controlled trials. We aimed to explore whether the benefit of covariate adjustment on statistical significance and power differed between small and large trials, where chance imbalance in prognostic factors necessarily differs. STUDY DESIGN AND SETTING: We studied two large trial data sets [Global Use of Strategies to Open Occluded Coronary Arteries (GUSTO-I), N = 30,510 and International Stroke Trial (IST), N = 18,372] repeatedly drawing random samples (500,000 times) of sizes 300 and 5,000 per arm and simulated each primary outcome using the control arms. We empirically determined the treatment effects required to fix power at 80% for all unadjusted analyses and calculated the joint probabilities in the discordant cells when cross-classifying adjusted and unadjusted results from logistic regression models (ie, P < 0.05 vs. P ≥ 0.05). RESULTS: The power gained from an adjusted analysis for small and large samples was between 5% and 6%. Similar proportions of discordance were noted irrespective of the sample size in both the GUSTO-I and the IST data sets. CONCLUSION: The proportions of change in statistical significance from covariate adjustment of strongly prognostic characteristics were the same for small and large trials with similar gains in statistical power. Covariate adjustment is equally recommendable in small and large trials.
OBJECTIVES: Covariate adjustment is a standard statistical approach in the analysis of randomized controlled trials. We aimed to explore whether the benefit of covariate adjustment on statistical significance and power differed between small and large trials, where chance imbalance in prognostic factors necessarily differs. STUDY DESIGN AND SETTING: We studied two large trial data sets [Global Use of Strategies to Open Occluded Coronary Arteries (GUSTO-I), N = 30,510 and International Stroke Trial (IST), N = 18,372] repeatedly drawing random samples (500,000 times) of sizes 300 and 5,000 per arm and simulated each primary outcome using the control arms. We empirically determined the treatment effects required to fix power at 80% for all unadjusted analyses and calculated the joint probabilities in the discordant cells when cross-classifying adjusted and unadjusted results from logistic regression models (ie, P < 0.05 vs. P ≥ 0.05). RESULTS: The power gained from an adjusted analysis for small and large samples was between 5% and 6%. Similar proportions of discordance were noted irrespective of the sample size in both the GUSTO-I and the IST data sets. CONCLUSION: The proportions of change in statistical significance from covariate adjustment of strongly prognostic characteristics were the same for small and large trials with similar gains in statistical power. Covariate adjustment is equally recommendable in small and large trials.
Authors: Elizabeth L Turner; Pablo Perel; Tim Clayton; Phil Edwards; Adrian V Hernández; Ian Roberts; Haleema Shakur; Ewout W Steyerberg Journal: J Clin Epidemiol Date: 2011-12-09 Impact factor: 6.437
Authors: Adrián V Hernández; Ewout W Steyerberg; Isabella Butcher; Nino Mushkudiani; Gillian S Taylor; Gordon D Murray; Anthony Marmarou; Sung C Choi; Juan Lu; J Dik F Habbema; Andrew I R Maas Journal: J Neurotrauma Date: 2006-09 Impact factor: 5.269
Authors: Rong Chu; Stephen D Walter; Gordon Guyatt; P J Devereaux; Michael Walsh; Kristian Thorlund; Lehana Thabane Journal: PLoS One Date: 2012-05-22 Impact factor: 3.240
Authors: Nikki van Leeuwen; Christa Walgaard; Pieter A van Doorn; Bart C Jacobs; Ewout W Steyerberg; Hester F Lingsma Journal: PLoS One Date: 2019-02-20 Impact factor: 3.240
Authors: Francesca M Chappell; Maria Del Carmen Valdés Hernández; Stephen D Makin; Kirsten Shuler; Eleni Sakka; Martin S Dennis; Paul A Armitage; Susana Muñoz Maniega; Joanna M Wardlaw Journal: Trials Date: 2017-02-21 Impact factor: 2.279
Authors: Hanaya Raad; Victoria Cornelius; Susan Chan; Elizabeth Williamson; Suzie Cro Journal: BMC Med Res Methodol Date: 2020-03-23 Impact factor: 4.615