RATIONALE AND OBJECTIVES: We evaluated by bootstrapping the conclusions obtained by the Dorfman-Berbaum-Metz (DBM) receiver operating characteristic (ROC) method and by the Toledano-Gatsonis (TG) method on a well-known data set. METHODS: We bootstrapped in two ways, resampled cases while holding readers fixed and resampled both cases and readers. RESULTS: When an analysis of variance of pseudovalues implies that reader variance and all random interactions with treatment are essentially zero, then case-resampling bootstrap and the DBM and TG methods should give the same results. Case-resampling bootstrap and the DBM and TG methods did give highly similar results for both individual readers and the averages over all readers. Both the case-resampling bootstrap and the reader-case resampling bootstrap gave smaller standard errors for group than for individual reader means, thereby providing evidence for a trade-off of readers and cases with regard to precision and power in this data set. CONCLUSION: Case-resampling bootstrap provides some justification for the DBM and TG methods.
RATIONALE AND OBJECTIVES: We evaluated by bootstrapping the conclusions obtained by the Dorfman-Berbaum-Metz (DBM) receiver operating characteristic (ROC) method and by the Toledano-Gatsonis (TG) method on a well-known data set. METHODS: We bootstrapped in two ways, resampled cases while holding readers fixed and resampled both cases and readers. RESULTS: When an analysis of variance of pseudovalues implies that reader variance and all random interactions with treatment are essentially zero, then case-resampling bootstrap and the DBM and TG methods should give the same results. Case-resampling bootstrap and the DBM and TG methods did give highly similar results for both individual readers and the averages over all readers. Both the case-resampling bootstrap and the reader-case resampling bootstrap gave smaller standard errors for group than for individual reader means, thereby providing evidence for a trade-off of readers and cases with regard to precision and power in this data set. CONCLUSION: Case-resampling bootstrap provides some justification for the DBM and TG methods.
Authors: Wendy Webber Chapman; Gregory F Cooper; Paul Hanbury; Brian E Chapman; Lee H Harrison; Michael M Wagner Journal: J Am Med Inform Assoc Date: 2003-06-04 Impact factor: 4.497
Authors: Ronald M Summers; Jiamin Liu; Bhavya Rehani; Phillip Stafford; Linda Brown; Adeline Louie; Duncan S Barlow; Donald W Jensen; Brooks Cash; J Richard Choi; Perry J Pickhardt; Nicholas Petrick Journal: Acad Radiol Date: 2010-06-12 Impact factor: 3.173