Yuval Ofek1. 1. Center for Public Management and Policy, University of Haifa, Haifa, Israel yuvalofek@poli.haifa.ac.il.
Abstract
BACKGROUND: Despite rapid advances in research on the evaluation of complex interventions, debate on evaluation methods and approaches still mainly revolves around the conventional and mostly outdated positivist-constructivist dichotomy. The lack of a clear conceptual and theoretical framework from which to choose appropriate evaluation approaches and methods means that approaches are often misused by both researchers and practitioners. RESEARCH DESIGN: Using three case studies, this article shows how different approaches should and should not be used in practice according to levels of nonlinearity. Both the theoretical development and the case studies presented in this article rely heavily on interviews conducted by the author with program management and staff, evaluation managers, heads of evaluation units, and evaluators in several countries across two continents, along with a quantitative survey. RESULTS: This article expands the classic discussion on evaluation approaches, adapting it to current managerial demands, increased complexity, and newly developed methodologies. It suggests an operational tool for categorizing evaluations and then matching evaluation approaches to the circumstances and the evaluation objectives. CONCLUSION: The findings suggest that approaches which are not congruent with levels of nonlinearity may hinder attempts to accurately evaluate results, causing dissatisfaction of evaluation commissioners from the evaluation process and methods applied. In contrast, analyzing the nonlinear and structural elements of complexity separately allows an extended categorization of evaluation approaches to be matched to the nonlinearity of programs to be evaluated.
BACKGROUND: Despite rapid advances in research on the evaluation of complex interventions, debate on evaluation methods and approaches still mainly revolves around the conventional and mostly outdated positivist-constructivist dichotomy. The lack of a clear conceptual and theoretical framework from which to choose appropriate evaluation approaches and methods means that approaches are often misused by both researchers and practitioners. RESEARCH DESIGN: Using three case studies, this article shows how different approaches should and should not be used in practice according to levels of nonlinearity. Both the theoretical development and the case studies presented in this article rely heavily on interviews conducted by the author with program management and staff, evaluation managers, heads of evaluation units, and evaluators in several countries across two continents, along with a quantitative survey. RESULTS: This article expands the classic discussion on evaluation approaches, adapting it to current managerial demands, increased complexity, and newly developed methodologies. It suggests an operational tool for categorizing evaluations and then matching evaluation approaches to the circumstances and the evaluation objectives. CONCLUSION: The findings suggest that approaches which are not congruent with levels of nonlinearity may hinder attempts to accurately evaluate results, causing dissatisfaction of evaluation commissioners from the evaluation process and methods applied. In contrast, analyzing the nonlinear and structural elements of complexity separately allows an extended categorization of evaluation approaches to be matched to the nonlinearity of programs to be evaluated.