RATIONALE AND OBJECTIVES: Several methods have been proposed for estimating the standard error (SE) of the area under the curve (AUC) in receiver operating characteristic analysis. The authors examined the validity of three methods--the LABROC procedure, exponential approximation, and the method of DeLong et al (purely nonparametric)--for estimating the SE of the AUC in receiver operating characteristic analysis of quantitative diagnostic data. MATERIALS AND METHODS: The authors conducted a broad numerical investigation to assess how to estimate the SE of AUC in various configurations of binormal and nonbinormal pairs of distributions, in which one or both pair members were mixtures of Gaussian distributions (the samples included 100 in the diseased group and 100 in the nondiseased group). RESULTS: The authors found that exponential approximation of the SE of AUC slightly underestimates the observed SE of a nonparametric estimate of the AUC when the ratio of the standard deviation of distributions for diseased to nondiseased populations was greater than 2. With binormal data the observed SE tended to be smaller with the LABROC procedure (semiparametric) than with the method of DeLong et al, but the LABROC procedure yields more conservative estimates of SE with nonbinormal data. In particular, with bimodal data it often produces a more conservative (ie, larger) estimate of the actual (observed) fluctuation. CONCLUSION: Overall, the LABROC procedure and the method of DeLong et al yielded very close estimates of the SE of AUC, even with data generated from a nonbinormal model. The choice between these two methods can be based on users' preferences and practicality.
RATIONALE AND OBJECTIVES: Several methods have been proposed for estimating the standard error (SE) of the area under the curve (AUC) in receiver operating characteristic analysis. The authors examined the validity of three methods--the LABROC procedure, exponential approximation, and the method of DeLong et al (purely nonparametric)--for estimating the SE of the AUC in receiver operating characteristic analysis of quantitative diagnostic data. MATERIALS AND METHODS: The authors conducted a broad numerical investigation to assess how to estimate the SE of AUC in various configurations of binormal and nonbinormal pairs of distributions, in which one or both pair members were mixtures of Gaussian distributions (the samples included 100 in the diseased group and 100 in the nondiseased group). RESULTS: The authors found that exponential approximation of the SE of AUC slightly underestimates the observed SE of a nonparametric estimate of the AUC when the ratio of the standard deviation of distributions for diseased to nondiseased populations was greater than 2. With binormal data the observed SE tended to be smaller with the LABROC procedure (semiparametric) than with the method of DeLong et al, but the LABROC procedure yields more conservative estimates of SE with nonbinormal data. In particular, with bimodal data it often produces a more conservative (ie, larger) estimate of the actual (observed) fluctuation. CONCLUSION: Overall, the LABROC procedure and the method of DeLong et al yielded very close estimates of the SE of AUC, even with data generated from a nonbinormal model. The choice between these two methods can be based on users' preferences and practicality.
Authors: Carlos A Alvarez; Christopher A Clark; Song Zhang; Ethan A Halm; John J Shannon; Carlos E Girod; Lauren Cooper; Ruben Amarasingham Journal: BMC Med Inform Decis Mak Date: 2013-02-27 Impact factor: 2.796