Understanding prevention effectiveness in real-world settings: the National Cross-Site Evaluation of high risk youth programs.

The National Cross-Site Evaluation is a large multisite evaluation (MSE) of 48 substance abuse prevention programs, 5,934 youth participating in programs, and 4,539 comparison youth programs. Data included a self-report questionnaire administered at 4 points in time, detailed dosage data on over 217,000 program contacts, and detailed site visit information. In a pooled analysis, the programs did not demonstrate significant positive effects on a composite outcome measure of tobacco, alcohol, and marijuana use in the previous 30 days. However, disaggregated analyses indicated that 1) sites in which comparison groups had strong opportunity to participate in prevention programs suppressed observed effects; 2) youth who had already started using before they entered programs reduced use significantly more than comparison youth who had started using; and 3) both males and females who participated in programs significantly reduced use relative to comparisons, but in very different patterns. Combining these patterns produced an apparent null effect. Finally, programs that incorporated at least 4 out of 5 effective intervention characteristics identified in the study significantly reduced use for both males and females relative to comparison youth. The lessons produced by this study attest to the value of MSE designs as a source of applicable knowledge about prevention interventions.