The creation of evidence in 'evidence-based' drug prevention: a critique of the Strengthening Families Program Plus Life Skills Training evaluation.

School-based curricula have become a mainstay of drug prevention policy in the United States and are increasing in popularity in other parts of the world. The promotion and dissemination of these interventions has been driven in large part by the creation of lists of programmes which, it is claimed, are grounded in scientific evidence demonstrating their effectiveness. Recently concerns have been raised about the data analysis and presentation practices used in evaluations of a number of programmes that appear on these lists. Here we examine a series of papers from an evaluation of an intervention that combined the Strengthening Families Program 10-14 and Life Skills Training Program, each of which is among the most widely advocated universal drug prevention programmes. The data analysis and presentation practices employed in the evaluation of this combined programme include one-tailed significance testing, alpha levels of 0.10, changes in outcome variables across publications and use of the post-test data as the baseline when assessing change over time. Taken together, these practices severely limit the claims that can be made about the results presented in the evaluation. Specifically, we believe that far from supporting the evaluators' claims concerning the rigour of the findings and their generalisability and public health significance, the results are very fragile, of little practical significance and quite possibly analysis-dependent.