Hua Ma1, Susan Halabi2, Aiyi Liu3. 1. Merck & Co. Inc., Kenilworth, NJ 07033. 2. Department of Biostatistics and Bioinformatics, Box 2717, Duke University Medical Center, Durham, NC 27710. 3. Biostatistics and Bioinformatics Branch, Division of Intramural Population Health Research, Eunice Kennedy Shriver National Institute of Child Health and Human Development, NIH, Rockville, MD.
Abstract
BACKGROUND: Evaluation of diagnostic assays and predictive performance of biomarkers based on the receiver operating characteristic (ROC) curve and the area under the ROC curve (AUC) are vital in diagnostic and targeted medicine. The partial area under the curve (pAUC) is an alternative metric focusing on a range of practical and clinical relevance of the diagnostic assay. In this article, we adopt and extend the min-max method to the estimation of the pAUC when multiple continuous scaled biomarkers are available and compare the performances of our proposed approach with existing approaches via simulations. METHODS: We conducted extensive simulation studies to investigate the performance of different methods for the combination of biomarkers based on their abilities to produce the largest pAUC estimates. Data were generated from different multivariate distributions with equal and unequal variance-covariance matrices. Different shapes of the ROC curves, false positive fraction ranges, and sample size configurations were considered. We obtained the mean and standard deviation of the pAUC estimates through re-substitution and leave-one-pair-out cross validation. RESULTS: Our results demonstrate that the proposed method provides the largest pAUC estimates under the following three important practical scenarios: (1) multivariate normally distributed data for non-diseased and diseased participants have unequal variance-covariance matrices; or (2) the ROC curves generated from individual biomarker are relative close regardless of the latent normality distributional assumption; or (3) the ROC curves generated from individual biomarker have straight-line shapes. CONCLUSIONS: The proposed method is robust and investigators are encouraged to use this approach in the estimation of the pAUC for many practical scenarios.
BACKGROUND: Evaluation of diagnostic assays and predictive performance of biomarkers based on the receiver operating characteristic (ROC) curve and the area under the ROC curve (AUC) are vital in diagnostic and targeted medicine. The partial area under the curve (pAUC) is an alternative metric focusing on a range of practical and clinical relevance of the diagnostic assay. In this article, we adopt and extend the min-max method to the estimation of the pAUC when multiple continuous scaled biomarkers are available and compare the performances of our proposed approach with existing approaches via simulations. METHODS: We conducted extensive simulation studies to investigate the performance of different methods for the combination of biomarkers based on their abilities to produce the largest pAUC estimates. Data were generated from different multivariate distributions with equal and unequal variance-covariance matrices. Different shapes of the ROC curves, false positive fraction ranges, and sample size configurations were considered. We obtained the mean and standard deviation of the pAUC estimates through re-substitution and leave-one-pair-out cross validation. RESULTS: Our results demonstrate that the proposed method provides the largest pAUC estimates under the following three important practical scenarios: (1) multivariate normally distributed data for non-diseased and diseased participants have unequal variance-covariance matrices; or (2) the ROC curves generated from individual biomarker are relative close regardless of the latent normality distributional assumption; or (3) the ROC curves generated from individual biomarker have straight-line shapes. CONCLUSIONS: The proposed method is robust and investigators are encouraged to use this approach in the estimation of the pAUC for many practical scenarios.
Entities:
Keywords:
ROC curve; area under the curve (AUC); combination of biomarkers; leave-one-pair-out cross validation; min-max method; partial area under the curve (pAUC); re-substitution
Authors: Brian I Rini; Susan Halabi; Jonathan E Rosenberg; Walter M Stadler; Daniel A Vaena; San-San Ou; Laura Archer; James N Atkins; Joel Picus; Piotr Czaykowski; Janice Dutcher; Eric J Small Journal: J Clin Oncol Date: 2008-10-20 Impact factor: 44.544