Statistical characteristics of area under the receiver operating characteristic curve for a simple prognostic model using traditional and bootstrapped approaches.

Prognostic models are increasingly common in the biomedical literature. These models are frequently evaluated with respect to their ability to discriminate between those with and without an outcome. The area under the receiver-operating curve (AROC) is often used to assess discrimination. In this study, we introduce a bootstrap method, and, using Monte Carlo simulation, we compare three different bootstrap approaches with four commonly used methods in their ability to accurately estimate 95% confidence intervals (CIs) around the AROC for a simple prognostic model. We also evaluated the power of a bootstrap method and the commonly used trapezoid rule to compare different prognostic models. We show that several good methods exist for calculating 95% CIs of AROC, but the maximum likelihood estimation method should not be used with small sample sizes. We further show that for our simple prognostic model a bootstrap z-statistic approach is preferred over the trapezoidal method when comparing the AROCs of two related models.