Literature DB >> 18258813

Comparing areas under receiver operating characteristic curves: potential impact of the "Last" experimentally measured operating point.

David Gur1, Andriy I Bandos, Howard E Rockette.   

Abstract

A specific issue related to the selection of the analytic tool used when comparing the estimated performance of systems within the receiver operating characteristic (ROC) paradigm is reviewed. This issue is related to the possible effect of the last experimentally ascertained ROC data point in terms of highest true-positive and false-positive fractions. An example of a case is presented where the selection of a specific analysis approach could affect the study conclusion from being not statistically significant for parametric analysis and significant for nonparametric analysis. This is followed by recommendations that should help prevent misinterpretation of results.

Mesh:

Year:  2008        PMID: 18258813      PMCID: PMC2637106          DOI: 10.1148/radiol.2471071321

Source DB:  PubMed          Journal:  Radiology        ISSN: 0033-8419            Impact factor:   11.105


  22 in total

1.  Components-of-variance models and multiple-bootstrap experiments: an alternative method for random-effects, receiver operating characteristic analysis.

Authors:  S V Beiden; R F Wagner; G Campbell
Journal:  Acad Radiol       Date:  2000-05       Impact factor: 3.173

2.  Prevalence effect in a laboratory environment.

Authors:  David Gur; Howard E Rockette; Derek R Armfield; Arye Blachar; Jennifer K Bogan; Giuseppe Brancatelli; Cynthia A Britton; Manuel L Brown; Peter L Davis; James V Ferris; Carl R Fuhrman; Sara K Golla; Sanj Katyal; Joan M Lacomis; Barry M McCook; F Leland Thaete; Thomas E Warfel
Journal:  Radiology       Date:  2003-07       Impact factor: 11.105

3.  Three methods for analysing correlated ROC curves: a comparison in real data sets from multi-reader, multi-case studies with a factorial design.

Authors:  Alicia Y Toledano
Journal:  Stat Med       Date:  2003-09-30       Impact factor: 2.373

4.  Receiver operating characteristic rating analysis. Generalization to the population of readers and patients with the jackknife method.

Authors:  D D Dorfman; K S Berbaum; C E Metz
Journal:  Invest Radiol       Date:  1992-09       Impact factor: 6.016

5.  Multireader, multicase receiver operating characteristic analysis: an empirical comparison of five methods.

Authors:  Nancy A Obuchowski; Sergey V Beiden; Kevin S Berbaum; Stephen L Hillis; Hemant Ishwaran; Hae Hiang Song; Robert F Wagner
Journal:  Acad Radiol       Date:  2004-09       Impact factor: 3.173

6.  Analyzing a portion of the ROC curve.

Authors:  D K McClish
Journal:  Med Decis Making       Date:  1989 Jul-Sep       Impact factor: 2.583

Review 7.  "Binary" and "non-binary" detection tasks: are current performance measures optimal?

Authors:  David Gur; Howard E Rockette; Andriy I Bandos
Journal:  Acad Radiol       Date:  2007-07       Impact factor: 3.173

8.  Some practical issues of experimental design and data analysis in radiological ROC studies.

Authors:  C E Metz
Journal:  Invest Radiol       Date:  1989-03       Impact factor: 6.016

9.  Comparing the areas under two or more correlated receiver operating characteristic curves: a nonparametric approach.

Authors:  E R DeLong; D M DeLong; D L Clarke-Pearson
Journal:  Biometrics       Date:  1988-09       Impact factor: 2.571

10.  Reliable and computationally efficient maximum-likelihood estimation of "proper" binormal ROC curves.

Authors:  Lorenzo L Pesce; Charles E Metz
Journal:  Acad Radiol       Date:  2007-07       Impact factor: 3.173

View more
  8 in total

1.  Evaluating imaging and computer-aided detection and diagnosis devices at the FDA.

Authors:  Brandon D Gallas; Heang-Ping Chan; Carl J D'Orsi; Lori E Dodd; Maryellen L Giger; David Gur; Elizabeth A Krupinski; Charles E Metz; Kyle J Myers; Nancy A Obuchowski; Berkman Sahiner; Alicia Y Toledano; Margarita L Zuley
Journal:  Acad Radiol       Date:  2012-02-03       Impact factor: 3.173

2.  Comparison of semiparametric receiver operating characteristic models on observer data.

Authors:  Frank W Samuelson; Xin He
Journal:  J Med Imaging (Bellingham)       Date:  2014-08-28

3.  Multiple diagnostic task performance in CT examination of the chest.

Authors:  K M Schartz; K S Berbaum; M T Madsen; B H Thompson; B F Mullan; R T Caldwell; B Hammett; A N Ellingson; E A Franken
Journal:  Br J Radiol       Date:  2013-01       Impact factor: 3.039

4.  Multiple diagnostic task performance in CT examination of the chest.

Authors:  K M Schartz; K S Berbaum; M T Madsen; B H Thompson; B F Mullan; R T Caldwell; B Hammett; A N Ellingson; E A Franken
Journal:  Br J Radiol       Date:  2012-09-06       Impact factor: 3.039

Review 5.  Multi-reader multi-case studies using the area under the receiver operator characteristic curve as a measure of diagnostic accuracy: systematic review with a focus on quality of data reporting.

Authors:  Thaworn Dendumrongsup; Andrew A Plumb; Steve Halligan; Thomas R Fanshawe; Douglas G Altman; Susan Mallett
Journal:  PLoS One       Date:  2014-12-26       Impact factor: 3.240

6.  Disadvantages of using the area under the receiver operating characteristic curve to assess imaging tests: a discussion and proposal for an alternative approach.

Authors:  Steve Halligan; Douglas G Altman; Susan Mallett
Journal:  Eur Radiol       Date:  2015-01-20       Impact factor: 5.315

7.  Exploration of analysis methods for diagnostic imaging tests: problems with ROC AUC and confidence scores in CT colonography.

Authors:  Susan Mallett; Steve Halligan; Gary S Collins; Doug G Altman
Journal:  PLoS One       Date:  2014-10-29       Impact factor: 3.240

8.  International evaluation of an AI system for breast cancer screening.

Authors:  Scott Mayer McKinney; Marcin Sieniek; Varun Godbole; Jonathan Godwin; Natasha Antropova; Hutan Ashrafian; Trevor Back; Mary Chesus; Greg S Corrado; Ara Darzi; Mozziyar Etemadi; Florencia Garcia-Vicente; Fiona J Gilbert; Mark Halling-Brown; Demis Hassabis; Sunny Jansen; Alan Karthikesalingam; Christopher J Kelly; Dominic King; Joseph R Ledsam; David Melnick; Hormuz Mostofi; Lily Peng; Joshua Jay Reicher; Bernardino Romera-Paredes; Richard Sidebottom; Mustafa Suleyman; Daniel Tse; Kenneth C Young; Jeffrey De Fauw; Shravya Shetty
Journal:  Nature       Date:  2020-01-01       Impact factor: 49.962

  8 in total

北京卡尤迪生物科技股份有限公司 © 2022-2023.