BACKGROUND: There are pitfalls associated with applying a biomedical model with its emphasis on experimental designs to the evaluation of workplace injury interventions. OBJECTIVES: Evaluation over enough time is essential in occupational safety when interventions are expected to have a latent effect as well as to assess sustained effects. Controlled trials are not well-suited to this task and are not even possible in circumstances where a policy change, such as legislative action, affects a population of workers simultaneously. Social context influences occupational injury interventions, their evaluation and the wider generalization of findings but is lost in the pooling of data for meta-analyses. Some of these issues can be addressed through recognition of the contribution of diverse observational methodologies in intervention evaluation, improvement and maintenance of robust surveillance systems, and inclusion of qualitative methodologies not typically embraced by epidemiologists or medical researchers. METHODS: Through consideration of an evaluation of a legislative effort to prevent falls from height in construction, we demonstrate lack of flexibility in current methods used for evaluating time series analyses in systematic reviews of occupational injury intervention effectiveness. DISCUSSION AND CONCLUSIONS: These include the manner in which downward change in slope is assessed and the call to demonstrate a significant initial downward change in level. We illustrate essential contextual detail regarding this intervention that is lost in the pooling of data from multiple studies into a combined measure of effect. This reduction of occupational injury intervention evaluation to one of pure statistical significance is ill-conceived, irresponsible, and should be stopped.
BACKGROUND: There are pitfalls associated with applying a biomedical model with its emphasis on experimental designs to the evaluation of workplace injury interventions. OBJECTIVES: Evaluation over enough time is essential in occupational safety when interventions are expected to have a latent effect as well as to assess sustained effects. Controlled trials are not well-suited to this task and are not even possible in circumstances where a policy change, such as legislative action, affects a population of workers simultaneously. Social context influences occupational injury interventions, their evaluation and the wider generalization of findings but is lost in the pooling of data for meta-analyses. Some of these issues can be addressed through recognition of the contribution of diverse observational methodologies in intervention evaluation, improvement and maintenance of robust surveillance systems, and inclusion of qualitative methodologies not typically embraced by epidemiologists or medical researchers. METHODS: Through consideration of an evaluation of a legislative effort to prevent falls from height in construction, we demonstrate lack of flexibility in current methods used for evaluating time series analyses in systematic reviews of occupational injury intervention effectiveness. DISCUSSION AND CONCLUSIONS: These include the manner in which downward change in slope is assessed and the call to demonstrate a significant initial downward change in level. We illustrate essential contextual detail regarding this intervention that is lost in the pooling of data from multiple studies into a combined measure of effect. This reduction of occupational injury intervention evaluation to one of pure statistical significance is ill-conceived, irresponsible, and should be stopped.