The problem of replicability in program evaluation. The component solution using the example of case management.

Specificity and replicability of a program are crucial in order to rigorously evaluate the program's effectiveness. However, the traditional approaches to evaluation have ignored the problem that most social and mental health programs consist of vaguely specified interventions and cannot meet minimal standards of replicability. Consequently, the results of effectiveness evaluations of poorly defined programs have little general generalizability. Nevertheless, effectiveness information concerning su ch programs is much needed. One solution to this problem is to focus on program components, examining interventions at a level where operational definitions can be employed and replicability is possible. In this paper, this approach is expanded using the example of case management programs. A component-oriented approach ensures a systematic evaluation of program effectiveness for which the first question for evaluators is whether a program consists of interventions that reliably produce specific immediate outcomes. Then, and only then, is it possible to address questions concerning whether a program will reliably produce those outcomes that are the overall objectives of the program.