Population health metrics: crucial inputs to the development of evidence for health policy.

Valid, reliable and comparable measures of the health states of individuals and of the health status of populations are critical components of the evidence base for health policy. We need to develop population health measurement strategies that coherently address the relationships between epidemiological measures (such as risk exposures, incidence, and mortality rates) and multi-domain measures of population health status, while ensuring validity and cross-population comparability.Studies reporting on descriptive epidemiology of major diseases, injuries and risk factors, and on the measurement of health at the population level - either for monitoring trends in health levels or inequalities or for measuring broad outcomes of health systems and social interventions - are not well-represented in traditional epidemiology journals, which tend to concentrate on causal studies and on quasi-experimental design. In particular, key methodological issues relating to the clear conceptualisation of, and the validity and comparability of measures of population health are currently not addressed coherently by any discipline, and cross-disciplinary debate is fragmented and often conducted in mutually incomprehensible language or paradigms. Population health measurement potentially bridges a range of currently disjoint fields of inquiry relating to health: biology, demography, epidemiology, health economics, and broader social science disciplines relevant to assessment of health determinants, health state valuations and health inequalities.This new journal will focus on the importance of a population based approach to measurement as a way to characterize the complexity of people's health, the diseases and risks that affect it, its distribution, and its valuation, and will attempt to provide a forum for innovative work and debate that bridge the many fields of inquiry relevant to population health in order to contribute to the development of valid and comparable methods for the measurement of population health and its determinants.

Introduction

Measurement of population health, its causes, and its distribution is fundamental to the development of evidence for health policies [1], and for the evaluation and planning of health systems and intervention programs [2,3]. Health is a complex, multidimensional phenomenon, and efforts to characterize and measure population health have generated a vast array of metrics and indicators, covering mortality, physiological measurements, clinical disease and impairment states, health states characterised in terms of functions or capacities in multiple domains, disability, handicap, quality of life and well-being. Metrics have ranged from simple event counts and rates such as numbers of deaths or incidence rates for specific diseases to more complex measures of multidimensional phenomena such as physical activity (types, durations, intensity), much relying on self-reported data, and quite complex summary measures of health level or inequality, such as health-adjusted life expectancy, or Gini-like indexes summarising health inequality [1,4].

The simplest and most widely used method for producing population health statistics is aggregation of data on individuals in order to generate statistics such as the proportion of the population suffering from a particular health problem or in a particular health state, or the proportion of people in specific groups who die during specified intervals from particular causes. This approach rapidly becomes unwieldy when a number of problems are being monitored and we want to make comparisons over time, across population groups, or before and after some health intervention. But it is not clear how event rates and event counts should relate to health states in individuals and populations, and there is not general agreement as to whether event rates and counts should be seen as health measures or as determinants or causes of population health, defined in terms of health states and mortality risks.

Towards valid and comparable metrics for population health

Valid, reliable and comparable measures of the health states of individuals are critical components of the evidence base for health policy. The discipline of epidemiology has focused on the measurement of generally clearly defined disease states and mortality risks, and has developed a body of techniques to maximise the validity of measurements by addressing issues of bias and confounding in study design and analysis. In particular, epidemiologists have focused considerable attention on the identification of causal risk factors for defined health outcomes (usually risk of death or incidence of clinical disease) and on the unbiased measurement of risk associations (either in terms of relative risks or absolute hazards) [5].

In contrast, clinical trials, economic analyses of cost-effectiveness of interventions, and national health surveys rely heavily on self-reported health measures. During the last thirty years, there have been considerable efforts to develop survey instruments to measure health status conceptualised in terms of multiple domains such as pain, affect, cognition, mobility, self-care, and usual activities [6-8]. However, the interpretation of these self-report measures is complicated by the comparability problems that arise when different persons understand and respond to a given question in different ways.

A number of paradoxical findings have been reported in analyses of population health surveys, suggesting that self-reported health measures may give misleading results if these differences are not taken into account [9,10]. This evidence has been ignored by many who use self-report survey measures of health status to report on population health, health inequalities or intervention outcomes (for example, see [11]). Indeed, comparability is often seen by survey designers as an optional extra, of interest only if one is interested in international comparison, but not needed to inform more localized uses. However, comparability of measurement across population groups, or for the same population group across time, is fundamental to many uses or interpretation of population health data. Measurement is fundamentally comparison. Imagine that you had a thermometer calibrated in an unknown degree scale, and available only at one point in time for one location (any measurement at a later time must use a different thermometer whose scale bears an unknown relationship to the first). The use of such a thermometer provides no usable information about the temperature at the specific location and time point. The use of self-report survey data on population health until recently has been in an analogous state.

Survey developers have emphasized the importance of establishing the validity of instruments and their reliability, but until recently, little attention has been paid to the issue of cross-population comparability. During the 1970s and 1980s much effort was put into development of standard instruments and survey protocols in the belief that this would ensure cross-population comparability [12,13]. During the 1990s, with the increasing use of such instruments in multi-country studies, it has become apparent that these do not solve the problem [10]. The problem of cross-population comparability relates more fundamentally to unmeasured differences in expectations and norms for health, so that the meaning different populations attach to the labels used for response categories in self-reported questions, such as mild, moderate or severe, can vary greatly. Recent developments in survey methodology using measured tests and anchoring vignettes to calibrate self-report health questions hold considerable promise in addressing this problem [14].

If population health measurement is to play its appropriate role in the provision of evidence to support public health policy making, monitoring and evaluation, then there is a crucial need to develop population health measurement strategies that coherently address the relationships between, and integrate, epidemiological measures (including risk exposures, disease and injury incidence, and mortality risks) and multi-domain measures of population health status, while ensuring validity and cross-population comparability.

With the development of measurement strategies that view health as an intrinsic, multi-dimensional attribute of individuals, interest has also grown in using health state valuations to provide a scalar index for the overall levels of health, quality of life, or well-being associated with different multidimensional profiles. Health state valuations are widely used in health economics and in population health measurement to summarize the health status of individuals in terms of a single number on a cardinal scale [15]. This allows health outcomes to be aggregated and compared across individuals in populations, across populations, across time, and across interventions. However, there is no consensus at present about what is actually being valued, with conceptualisations ranging from utility, well-being and overall quality-of-life, through to more narrow conceptualisations of health-related quality of life, or levels of health per se [15].

Monitoring of global efforts to improve population health

Issues of conceptualization and of valid and comparable measurement of population health are of increasing international policy importance. At the global level, a new commitment to track performance of health programs and systems is evidenced by the Millennium Development Goals and in the design of new initiatives such as Global Alliance for Vaccines and Immunizations (GAVI), the Global Fund to Fight AIDS, Tuberculosis and Malaria, and the Global Alliance for Improved Nutrition (GAIN). Seventeen of the 48 Millennium Development Goal indicators are health-related [16], indicating the increasingly dominant role of health in the development agenda. New investment mechanisms such as these are likely to be sustained if the initial investments can be demonstrated to have had a real impact. The importance of population health measurement is demonstrated by the proliferation of meetings and initiatives to strengthen national capacity to monitor critical heath outcomes. The growing momentum behind this new accountability for results might be lost unless a parallel effort is launched to enable access to more reliable metrics for measuring improvements in population health.

The increasingly recognized links between health and development have also focused increased attention on the health of the poor, and its measurement. The recent WHO Commission on Macroeconomics and Health concluded that the bulk of the global disease burden is the result of a relatively small set of conditions, each with an existing set of effective interventions [17]. The main problems are the funding of these interventions and access of poor populations to these interventions. International efforts to monitor the health of the poor requires increased attention to the consistent and comparable categorization of poverty across countries as well as on the comparable measurement of population health in these subgroups, and the development of appropriate measures of health inequality for both absolute and relative inequalities.

Why a new journal devoted to measurement of population health?

Population health measurement potentially bridges a range of currently disjoint fields of inquiry relating to health: biology (for example, of infectious diseases), demography, epidemiology, health economics, and broader social science disciplines relevant to assessment of health determinants, health state valuations and health inequalities.

Many traditional epidemiology journals concentrate on causal studies and on quasi-experimental design. Studies reporting on descriptive epidemiology of major diseases, injuries and risk factors, and on the measurement of health at the population level – either for monitoring trends in health levels or inequalities or for measuring broad outcomes of health systems and social interventions – are not well-represented in traditional journals. To date, there has been a particular bias against research synthesis studies, in which uncertain and incomplete evidence is used to synthesize best estimates of population health measures – ideally with analysis of uncertainty – in order to inform policy, decision makers and the public [18].

In particular, key methodological issues relating to the clear conceptualisation of, and the validity and comparability of measures of population health are currently not addressed coherently by any discipline, and cross-disciplinary debate is fragmented and often conducted in mutually incomprehensible language or paradigms. This new journal focuses on the measurement of the health of populations and will address issues relating to concepts, methods, ethics, applications and results in the areas of health state measurement and valuation, summary measures of level of population health, and inequality in population health, descriptive epidemiology at the population level, burden of disease and injury analysis, disease and risk factor modelling for populations, and assessment of risks to health at population level.

This journal will focus on the importance of a population based approach to measurement as a way to characterize the complexity of people's health, the diseases and risks that affect it, its distribution, and its valuation, and will attempt to provide a forum for innovative work that bridges the many fields of inquiry relevant to population health. In launching this new online journal, we have accepted papers that address a range of these issues. Policy-makers charged with developing health policies to improve population health and contain health system costs must do so at the moment in the face of a remarkable lack of comparable evidence about variations and trends in population health, and causes of these. We hope that this new journal will contribute to the development of valid and comparable methods for the measurement of population health and its determinants and provide a forum for scientific debate about these issues.

Competing interests

None declared. The opinions expressed in this editorial are those of the authors and do not necessarily reflect the views of the World Health Organization.