Lucretiu M Popescu1. 1. Food and Drug Administration Center for Devices and Radiological Health, Silver Spring, MD 20993, USA. lucretiu.popescu@fda.hhs.gov
Abstract
PURPOSE: To develop an efficient nonparametric method for evaluation the detectability of signals at unknown locations in images, as a mean for image quality assessment. METHODS: We use the free-response methodology that allows the image observer to mark and score all locations found as suspicious in an image, summarizing these results in a free-response operating characteristic (FROC) curve. However, unlike the relative (or receiver) operating characteristic (ROC), or the localization ROC (LROC), the FROC curve has an undefined, theoretically infinite, right side limit. Therefore area under the FROC curves cannot be directly used as an overall performance index, as the area under the curve is for ROC or LROC. We circumvent this drawback by using a transformation of the abscissa that leads to a finite integration range. By applying an exponential transformation we derive a nonparametric estimator for such a metric, and we study its properties by deriving analytical expressions for the mean and standard deviation in conditions of scores independence. RESULTS: A comparative study with other related nonparametric estimators for ROC, LROC, and alternative FROC (AFROC) method is presented. CONCLUSIONS: The new nonparametric estimator has sensitivity and scalability properties that make it particularly advantageous for signal detectability evaluation in phantom experiments using model observers.
PURPOSE: To develop an efficient nonparametric method for evaluation the detectability of signals at unknown locations in images, as a mean for image quality assessment. METHODS: We use the free-response methodology that allows the image observer to mark and score all locations found as suspicious in an image, summarizing these results in a free-response operating characteristic (FROC) curve. However, unlike the relative (or receiver) operating characteristic (ROC), or the localization ROC (LROC), the FROC curve has an undefined, theoretically infinite, right side limit. Therefore area under the FROC curves cannot be directly used as an overall performance index, as the area under the curve is for ROC or LROC. We circumvent this drawback by using a transformation of the abscissa that leads to a finite integration range. By applying an exponential transformation we derive a nonparametric estimator for such a metric, and we study its properties by deriving analytical expressions for the mean and standard deviation in conditions of scores independence. RESULTS: A comparative study with other related nonparametric estimators for ROC, LROC, and alternative FROC (AFROC) method is presented. CONCLUSIONS: The new nonparametric estimator has sensitivity and scalability properties that make it particularly advantageous for signal detectability evaluation in phantom experiments using model observers.
Authors: Nicholas Petrick; Berkman Sahiner; Samuel G Armato; Alberto Bert; Loredana Correale; Silvia Delsanto; Matthew T Freedman; David Fryd; David Gur; Lubomir Hadjiiski; Zhimin Huo; Yulei Jiang; Lia Morra; Sophie Paquerault; Vikas Raykar; Frank Samuelson; Ronald M Summers; Georgia Tourassi; Hiroyuki Yoshida; Bin Zheng; Chuan Zhou; Heang-Ping Chan Journal: Med Phys Date: 2013-08 Impact factor: 4.071