OBJECTIVE: We present a Monte Carlo technique to evaluate if observed mortality rates differ from model-predicted rates for situations when the number of deaths is small. DATA SOURCES: We used Medicare hospital claims and model-predicted mortality rates from the Health Care Financing Administration (HCFA) for the 169 acute care hospitals in Georgia. The HCFA data provided model-predicted mortality rates at 30 days postadmission for 17 conditions and procedures of interest. The model-predicted rates calculated by HCFA were adjusted for patient factors, including demographic characteristics, principal diagnosis, and comorbidities. STUDY DESIGN: We test the hypothesis that model-predicted 30-day mortality rates at the 169 hospitals differ significantly from the observed 30-day mortality rates. Our approach uses a test statistic that resembles a chi-square statistic, and Monte Carlo simulations to estimate the distribution of the test statistic under the null hypothesis of no differences between the observed and predicted rates. We illustrate the method using two conceptually similar simulation models. We use results of the simulations to estimate p-values and compare these results with p-values associated with the nominal chi-square distribution. DATA EXTRACTION METHODS: We extracted 30-day observed and predicted mortality rates for Medicare beneficiaries for federal fiscal year 1990 for 17 conditions and procedures of interest. PRINCIPAL FINDINGS: If the number of deaths in some hospitals is small, p-values calculated using the nominal chi-square distribution can be misleading, thus supporting the usefulness of our simulation method. CONCLUSIONS: The Monte Carlo simulation is an appropriate approach to the analysis of hospital mortality or small area analysis for situations in which the number of deaths is small.
OBJECTIVE: We present a Monte Carlo technique to evaluate if observed mortality rates differ from model-predicted rates for situations when the number of deaths is small. DATA SOURCES: We used Medicare hospital claims and model-predicted mortality rates from the Health Care Financing Administration (HCFA) for the 169 acute care hospitals in Georgia. The HCFA data provided model-predicted mortality rates at 30 days postadmission for 17 conditions and procedures of interest. The model-predicted rates calculated by HCFA were adjusted for patient factors, including demographic characteristics, principal diagnosis, and comorbidities. STUDY DESIGN: We test the hypothesis that model-predicted 30-day mortality rates at the 169 hospitals differ significantly from the observed 30-day mortality rates. Our approach uses a test statistic that resembles a chi-square statistic, and Monte Carlo simulations to estimate the distribution of the test statistic under the null hypothesis of no differences between the observed and predicted rates. We illustrate the method using two conceptually similar simulation models. We use results of the simulations to estimate p-values and compare these results with p-values associated with the nominal chi-square distribution. DATA EXTRACTION METHODS: We extracted 30-day observed and predicted mortality rates for Medicare beneficiaries for federal fiscal year 1990 for 17 conditions and procedures of interest. PRINCIPAL FINDINGS: If the number of deaths in some hospitals is small, p-values calculated using the nominal chi-square distribution can be misleading, thus supporting the usefulness of our simulation method. CONCLUSIONS: The Monte Carlo simulation is an appropriate approach to the analysis of hospital mortality or small area analysis for situations in which the number of deaths is small.