Comparing Kirkpatrick's original and new model with CIPP evaluation model.

Dear Editor,

In a young field like educational program evaluation, it is inevitable that conceptual frameworks such as Kirkpatrick model are revised with time and with greater knowledge. The New World Kirkpatrick Model (NWKM) is the new version of Kirkpatrick model which is more welcome to context and process, and hence probably much closer to the context-input-process-product (CIPP) model (1). The aim of this paper is to explore the similarities and differences between three well-known evaluation models including the original and new versions of Kirkpatrick model and CIPP model.

The original version of Kirkpatrick model is an outcome-focused model evaluating the outcomes of an educational program, for instance, in the field of medical education, in four levels of reaction, learning, transfer and impact, respectively (2). The model is rooted in reductionist approach suggesting that the educational program success or lack of success can be explained simply by reducing the program into its elements and examining them (i.e. its outcomes) (3). Yet, Kirkpatrick’s original model fails to provide the evaluators with an insight into the underlying mechanisms that inhibit or facilitate the achievement of program outcomes (4). In response to this shortcoming, the new version of Kirkpatrick model added new elements to recognize the complexities of the educational program context (5).

The most highlighted changes have been occurred at Level 3 to include processes that enable or hinder the application of learned knowledge or skills. The required drivers that reinforce, monitor, encourage, and reward learners to apply what is learned during training, on the job learning that happens outside the formal program and Learners’ motivation and commitment to improve their performance on the job are interfering factors that may influence the given outcomes at level 3. Learners’ confidence and commitment, and learners’ engagement and subject relevance ware added to Level 2 and level 1, respectively, to broaden the scope of evaluation at these two levels (5).

Although the NWKM appears to better embrace the complexity of educational programs, some investigators may declare that it would be similar to CIPP evaluation model. I suppose that there are some fundamental differences between them. The CIPP model stems from the complexity theory that takes into account the educational program as an open system with emergent dynamical interactions among its component parts and the surrounding environment. As a result, CIPP pays explicit and implicit attention to the program context by considering context evaluation as a separate component of four complementary sets of evaluation studies, as well as identifying the contextual factors in other components of the model by employing a variety of qualitative methods (6). On the other hand, the NWKM is limited to measuring some confounding factors such as learner characteristics or organizational factors on program outcome achievement (1).

Kirkpatrick, like many traditional program evaluation models, focuses on proving something (i.e. outcome achievement) about a program. Thus, it is usually conducted at the end of the program. CIPP, on the other hand, acknowledges program improvement, so providing useful information for decision makers during all phases of program development even when the program is still being developed (7). The NWKM has broadened the scope of traditional model by adding some process measures enabling evaluators to interpret the outcome evaluation results, but with the aim of proving an educational program.

Overall, notwithstanding some improvement, NWKM has still some theoretical differences with the CIPP model resulting in varied methodological and practical preferences. However, it is not unexpected to witness more convergence around these evaluation models with greater knowledge and experience in the future.