Routine monitoring of performance: what makes health research and development different?

Increasing attention is being directed to measuring and monitoring the use of health-related R&D funding, partly to justify this expenditure and partly to ensure that R&D effort is directed to achieving the paybacks desired by funders. These paybacks include contributing to knowledge, contributing to R&D capacity, political benefits, benefits to the health service and to patients, and more general economic benefits. This paper addresses the issues that must be considered when designing a routine performance management system for health R&D. Conventional methods of routine performance management are often rendered inappropriate in this context by the intangible and unpredictable outcomes of research, which are heterogeneous across projects and programmes and which can be hard to attribute to particular R&D support. Instead, to be effective in this context, a routine system must combine quantitative and qualitative indicators, utilising information from a number of different sources. The system must achieve acceptable levels (defined by the funder) on each of the following criteria: it must measure those dimensions of payback that are valued by the funder; it must be decision-relevant; it must be consistent with truthful compliance; it must minimise perverse incentives; and it must have acceptable net costs. It is vitally important that the system itself generates a positive payback. We illustrate these issues by outlining a system that might be used to monitor the payback from government-funded R&D.