Douglas Mossman1, Hongying Peng2. 1. Department of Psychiatry and Behavioral Neuroscience, University of Cincinnati College of Medicine, Cincinnati, OH, USA. (DM) 2. Department of Environmental Health, University of Cincinnati College of Medicine, Cincinnati, OH, USA. (HP)
Abstract
BACKGROUND: Receiver operating characteristic (ROC) analysis helps investigators quantify and describe how well a diagnostic system discriminates between 2 mutually exclusive conditions. The conventional binormal (CvB) curve-fitting model usually produces ROCs that are improper in that they do not have the ever-decreasing slope required by signal detection theory. When data sets evaluated under the CvB model have hooks, the resulting ROCs can contain misleading information about the diagnostic performance of the method at low and high false positive rates. OBJECTIVE: To present and evaluate a dual beta (DB) ROC model that assumes diagnostic data arise from 2 β distributions. The DB model's parameter constraints assure that the resulting ROC curve has a positive, monotonically decreasing slope. DESIGN/ METHOD: Computer simulation study comparing results from CvB, DB, and weighted power function (WPF) models. RESULTS: The DB model produces results that are as good as or better than those from the WPF model, and less biased and closer to the true values than curves obtained using the CvB model. CONCLUSIONS: The DB ROC model expresses the relationship between the false positive rate and true positive rate in closed form and allows for quick ROC area calculations using spreadsheet functions. Because it posits simple relationships among the decision axis, operating points, and model parameters, the DB model offers investigators a flexible, easy-to-grasp ROC form that is simpler to implement than other proper ROC models.
BACKGROUND: Receiver operating characteristic (ROC) analysis helps investigators quantify and describe how well a diagnostic system discriminates between 2 mutually exclusive conditions. The conventional binormal (CvB) curve-fitting model usually produces ROCs that are improper in that they do not have the ever-decreasing slope required by signal detection theory. When data sets evaluated under the CvB model have hooks, the resulting ROCs can contain misleading information about the diagnostic performance of the method at low and high false positive rates. OBJECTIVE: To present and evaluate a dual beta (DB) ROC model that assumes diagnostic data arise from 2 β distributions. The DB model's parameter constraints assure that the resulting ROC curve has a positive, monotonically decreasing slope. DESIGN/ METHOD: Computer simulation study comparing results from CvB, DB, and weighted power function (WPF) models. RESULTS: The DB model produces results that are as good as or better than those from the WPF model, and less biased and closer to the true values than curves obtained using the CvB model. CONCLUSIONS: The DB ROC model expresses the relationship between the false positive rate and true positive rate in closed form and allows for quick ROC area calculations using spreadsheet functions. Because it posits simple relationships among the decision axis, operating points, and model parameters, the DB model offers investigators a flexible, easy-to-grasp ROC form that is simpler to implement than other proper ROC models.
Authors: Weijie Chen; Berkman Sahiner; Frank Samuelson; Aria Pezeshk; Nicholas Petrick Journal: Stat Methods Med Res Date: 2016-08-08 Impact factor: 3.021