How reliable are county and regional health rankings?

Surveillance system indicators of morbidity, mortality, or behaviors are used to provide regional information for ever-smaller areas, most recently for ranking counties. These rankings are thought to provide information about the relative standing of regions and provide information about problem areas and the success of programs. We investigate the ability of such rankings to reliably assess health. We assess the reliability of several ranked health indices used at the county level and the consistency of an index's quality across different states. Reliability is assessed using an index of reliability, simulations, and the ability of an index to consistently identify the top 10 % (worst) counties within a state. There is marked variability across measures to provide consistent ranks, across states, and, many times, across states for a particular measure. A few health measures do consistently well, e.g., Teen Birth, Chlamydia, and Years of Potential Life Lost rates. While a few health rankings appear worthy of use for policy and in the identification of local problems, in general, they are not consistent across regions.