Jill R Glassman1, Susan C Potter2, Elizabeth R Baumler3, Karin K Coyle2. 1. ETR Associates, Scotts Valley, CA, USA jillg@etr.org. 2. ETR Associates, Scotts Valley, CA, USA. 3. University of Texas Health Science Center at Houston, Houston, TX, USA.
Abstract
INTRODUCTION: Group-randomized trials (GRTs) are one of the most rigorous methods for evaluating the effectiveness of group-based health risk prevention programs. Efficiently designing GRTs with a sample size that is sufficient for meeting the trial's power and precision goals while not wasting resources exceeding them requires estimates of the intraclass correlation coefficient (ICC)-the degree to which outcomes of individuals clustered within groups (e.g., schools) are correlated. ICC estimates vary widely depending on outcome, population, and setting, and small changes in ICCs can have large effects on the sample size needed to estimate intervention effects. This study addresses a gap in the literature by providing estimates of ICCs for adolescent sexual risk-taking outcomes under a range of study conditions. METHOD: Multilevel regression analyses were applied to existing data from four federally funded GRTs of school-based HIV/STI/pregnancy prevention programs to obtain a variety of ICC estimates. RESULTS: ICCs ranged from 0 to 0.15, with adjustment for covariates and repeated measurements reducing the ICC in the majority of cases. Minimum detectable effect sizes with 80% power and 0.05 significance levels ranged from small to medium Cohen's d (0.13 to 0.53) assuming 20 schools of 100 students each. CONCLUSIONS: This study provides the first known set of ICC estimates for investigators to use when planning studies of school-based programs to prevent sexual risk behaviors in youth. The results provide further evidence of the importance of using the appropriate adjusted ICC estimate at the design stage to maximize resources in costly GRTs.
INTRODUCTION: Group-randomized trials (GRTs) are one of the most rigorous methods for evaluating the effectiveness of group-based health risk prevention programs. Efficiently designing GRTs with a sample size that is sufficient for meeting the trial's power and precision goals while not wasting resources exceeding them requires estimates of the intraclass correlation coefficient (ICC)-the degree to which outcomes of individuals clustered within groups (e.g., schools) are correlated. ICC estimates vary widely depending on outcome, population, and setting, and small changes in ICCs can have large effects on the sample size needed to estimate intervention effects. This study addresses a gap in the literature by providing estimates of ICCs for adolescent sexual risk-taking outcomes under a range of study conditions. METHOD: Multilevel regression analyses were applied to existing data from four federally funded GRTs of school-based HIV/STI/pregnancy prevention programs to obtain a variety of ICC estimates. RESULTS: ICCs ranged from 0 to 0.15, with adjustment for covariates and repeated measurements reducing the ICC in the majority of cases. Minimum detectable effect sizes with 80% power and 0.05 significance levels ranged from small to medium Cohen's d (0.13 to 0.53) assuming 20 schools of 100 students each. CONCLUSIONS: This study provides the first known set of ICC estimates for investigators to use when planning studies of school-based programs to prevent sexual risk behaviors in youth. The results provide further evidence of the importance of using the appropriate adjusted ICC estimate at the design stage to maximize resources in costly GRTs.
Authors: Jennifer Pillay; Ainsley Moore; Prinon Rahman; Gabriel Lewin; Donna Reynolds; John Riva; Guyléne Thériault; Brett Thombs; Brenda Wilson; Joan Robinson; Amanda Ramdyal; Geneviéve Cadieux; Robin Featherstone; Anne N Burchell; Jo-Anne Dillon; Ameeta Singh; Tom Wong; Marion Doull; Greg Traversy; Susan Courage; Tara MacGregor; Cydney Johnson; Ben Vandermeer; Lisa Hartling Journal: Syst Rev Date: 2018-12-26