Exploring and comparing the characteristics of nonlatent and latent composite scores: implications for pay-for-performance incentive design.

A concise and reliable composite quality score would be helpful in judging the quality of a hospital's services, especially for pay-for-performance (P4P) initiatives. This study compared several nonlatent and latent composite quality scores to evaluate the quality of care using diabetes mellitus (DM) P4P data and discusses their characteristics and implications for P4P policy. The authors describe a cross-sectional study of the DM P4P data collected from the claims data of the Bureau of National Health Insurance (NHI) in Taiwan from January 2007 to December 2007. The DM patient outcome data, such as hemoglobin A1C values, were retrieved from the P4P database sponsored by the Bureau of NHI in Taiwan. The composite scores were derived from the following methods: 1) nonlatent scores methods (e.g., the raw sum score and the all-or-none score methods)and 2) latent scores methods (e.g., item-response theory-based Models I and II and the PRIDIT model). These scores are compared in terms of 2 aspects-agreement of hospital rankings (using Spearman's rank correlation) and reliability (using bootstrap methods). The latent methods were superior to the nonlatent methods because they were more reliable and had specific weighting themes. The correlations among the 3 latent methods were moderately high. The use of the PRIDIT approach, which is moderately difficult compared with item response theory-based model, is recommended if the insurer wants to balance convenience and precision.