Paul H Lee1. 1. School of Nursing, Hong Kong Polytechnic University, GH527, 11 Yuk Choi Road, Hung Hom, Kowloon, Hong Kong. Electronic address: paul.h.lee@polyu.edu.hk.
Abstract
OBJECTIVES: This study aims to show that under several assumptions, in randomized controlled trials (RCTs), unadjusted, crude analysis will underestimate the Cohen's d effect size of the treatment, and an unbiased estimate of effect size can be obtained only by adjusting for all predictors of the outcome. STUDY DESIGN AND SETTING: Four simulations were performed to examine the effects of adjustment on the estimated effect size of the treatment and power of the analysis. In addition, we analyzed data from the Advanced Cognitive Training for Independent and Vital Elderly (ACTIVE) study (older adults aged 65-94), an RCT with three treatment arms and one control arm. RESULTS: We showed that (1) the number of unadjusted covariates was associated with the effect size of the treatment; (2) the biasedness of effect size estimation was minimized if all covariates were adjusted for; (3) the power of the statistical analysis slightly decreased with the number of adjusted noise variables; and (4) exhaustively searching the covariates and noise variables adjusted for can lead to exaggeration of the true effect size. Analysis of the ACTIVE study data showed that the effect sizes adjusting for covariates of all three treatments were 7.39-24.70% larger than their unadjusted counterparts, whereas the effect size would be elevated by at most 57.92% by exhaustively searching the variables adjusted for. CONCLUSION: All covariates of the outcome in RCTs should be adjusted for, and if the effect of a particular variable on the outcome is unknown, adjustment will do more good than harm.
OBJECTIVES: This study aims to show that under several assumptions, in randomized controlled trials (RCTs), unadjusted, crude analysis will underestimate the Cohen's d effect size of the treatment, and an unbiased estimate of effect size can be obtained only by adjusting for all predictors of the outcome. STUDY DESIGN AND SETTING: Four simulations were performed to examine the effects of adjustment on the estimated effect size of the treatment and power of the analysis. In addition, we analyzed data from the Advanced Cognitive Training for Independent and Vital Elderly (ACTIVE) study (older adults aged 65-94), an RCT with three treatment arms and one control arm. RESULTS: We showed that (1) the number of unadjusted covariates was associated with the effect size of the treatment; (2) the biasedness of effect size estimation was minimized if all covariates were adjusted for; (3) the power of the statistical analysis slightly decreased with the number of adjusted noise variables; and (4) exhaustively searching the covariates and noise variables adjusted for can lead to exaggeration of the true effect size. Analysis of the ACTIVE study data showed that the effect sizes adjusting for covariates of all three treatments were 7.39-24.70% larger than their unadjusted counterparts, whereas the effect size would be elevated by at most 57.92% by exhaustively searching the variables adjusted for. CONCLUSION: All covariates of the outcome in RCTs should be adjusted for, and if the effect of a particular variable on the outcome is unknown, adjustment will do more good than harm.
Authors: Maria-Eleni Roumelioti; Jennifer L Steel; Jonathan Yabes; Kevin E Vowles; Yoram Vodovotz; Scott Beach; Bruce Rollman; Steven D Weisbord; Mark L Unruh; Manisha Jhamb Journal: Contemp Clin Trials Date: 2018-09-09 Impact factor: 2.226
Authors: Andrea Monsour; Emma J Mew; Sagar Patel; Alyssandra Chee-A-Tow; Leena Saeed; Lucia Santos; Darren B Courtney; Priya N Watson; Suneeta Monga; Peter Szatmari; Martin Offringa; Nancy J Butcher Journal: BMC Med Res Methodol Date: 2020-05-25 Impact factor: 4.615
Authors: Tzu Tsun Luk; Derek Yee-Tak Cheung; Helen Ching-Han Chan; Patrick Wai-Yin Fok; Kin Sang Ho; Chu Dik Sze; Tai Hing Lam; Man Ping Wang Journal: Nicotine Tob Res Date: 2022-02-15 Impact factor: 5.825