A critique of the World Health Organisation's evaluation of health system performance.

The World Health Organisation's (WHO) approach to the measurement of health system efficiency is briefly described. Four arguments are then presented. First, equity of finance should not be a criterion for the evaluation of a health system and, more generally, the same objectives and importance weights should not be imposed upon all countries. Secondly, the numerical value of the importance weights do not reflect their true importance in the country rankings. Thirdly, the model for combining the different objectives into a single index of system performance is problematical and alternative models are shown to alter system rankings. The WHO statistical analysis is replicated and used to support the fourth argument which is that, contrary to the author's assertion, their methods cannot separate true inefficiency from random error. The procedure is also subject to omitted variable bias. The econometric model for all countries has very poor predictive power for the subset of OECD countries and it is outperformed by two simpler algorithms. Country rankings based upon the model are correspondingly unreliable. It is concluded that, despite these problems, the study is a landmark in the evolution of system evaluation, but one which requires significant revision. Copyright 2002 John Wiley & Sons, Ltd.