OBJECTIVES: The US Department of Education requires schools to choose substance abuse and violence prevention programs that meet standards of effectiveness. The Substance Abuse and Mental Health Services Agency certifies "model" programs that meet this standard. We compared findings from a large, multisite effectiveness trial of 1 model program to its efficacy trial findings, upon which the certification was based. METHODS:1370 high-risk youths were randomized to experimental or control groups across 9 high schools in 2 large urban school districts. We used intent-to-treat and on-treatment approaches to examine baseline equivalence, attrition, and group differences in outcomes at the end of the program and at a 6-month follow-up. RESULTS: Positive efficacy trial findings were not replicated in the effectiveness trial. All main effects were either null or worse for the experimental than for the control group. CONCLUSIONS: These findings suggest that small efficacy trials conducted by developers provide insufficient evidence of effectiveness. Federal agencies and public health scientists must work together to raise the standards of evidence and ensure that data from new trials are incorporated into ongoing assessments of program effects.
RCT Entities:
OBJECTIVES: The US Department of Education requires schools to choose substance abuse and violence prevention programs that meet standards of effectiveness. The Substance Abuse and Mental Health Services Agency certifies "model" programs that meet this standard. We compared findings from a large, multisite effectiveness trial of 1 model program to its efficacy trial findings, upon which the certification was based. METHODS: 1370 high-risk youths were randomized to experimental or control groups across 9 high schools in 2 large urban school districts. We used intent-to-treat and on-treatment approaches to examine baseline equivalence, attrition, and group differences in outcomes at the end of the program and at a 6-month follow-up. RESULTS: Positive efficacy trial findings were not replicated in the effectiveness trial. All main effects were either null or worse for the experimental than for the control group. CONCLUSIONS: These findings suggest that small efficacy trials conducted by developers provide insufficient evidence of effectiveness. Federal agencies and public health scientists must work together to raise the standards of evidence and ensure that data from new trials are incorporated into ongoing assessments of program effects.
Authors: Kelli A Komro; Cheryl L Perry; Sara Veblen-Mortenson; Kian Farbakhsh; Traci L Toomey; Melissa H Stigler; Rhonda Jones-Webb; Kari C Kugler; Keryn E Pasch; Carolyn L Williams Journal: Addiction Date: 2008-02-04 Impact factor: 6.526
Authors: Denise C Gottfredson; Thomas D Cook; Frances E M Gardner; Deborah Gorman-Smith; George W Howe; Irwin N Sandler; Kathryn M Zafft Journal: Prev Sci Date: 2015-10
Authors: Michael J Mugavero; Margaret May; Heather J Ribaudo; Roy M Gulick; Sharon A Riddler; Richard Haubrich; Sonia Napravnik; Sophie Abgrall; Andrew Phillips; Ross Harris; M John Gill; Frank de Wolf; Robert Hogg; Huldrych F Günthard; Geneviève Chêne; Antonella D'Arminio Monforte; Jodie L Guest; Colette Smith; Javier Murillas; Juan Berenguer; Christoph Wyen; Pere Domingo; Mari M Kitahata; Jonathan A C Sterne; Michael S Saag Journal: J Acquir Immune Defic Syndr Date: 2011-11-01 Impact factor: 3.731