Stephen L Hillis1, Kevin S Berbaum, Charles E Metz. 1. Center for Research in the Implementation of Innovative Strategies in Practice (CRIISP), Iowa City VA Medical Center (152), Iowa City, IA, USA. steve-hillis@uiowa.edu
Abstract
RATIONALE AND OBJECTIVES: The Dorfman-Berbaum-Metz (DBM) method has been one of the most popular methods for analyzing multireader receiver-operating characteristic (ROC) studies since it was proposed in 1992. Despite its popularity, the original procedure has several drawbacks: it is limited to jackknife accuracy estimates, it is substantially conservative, and it is not based on a satisfactory conceptual or theoretical model. Recently, solutions to these problems have been presented in three papers. Our purpose is to summarize and provide an overview of these recent developments. MATERIALS AND METHODS: We present and discuss the recently proposed solutions for the various drawbacks of the original DBM method. RESULTS: We compare the solutions in a simulation study and find that they result in improved performance for the DBM procedure. We also compare the solutions using two real data studies and find that the modified DBM procedure that incorporates these solutions yields more significant results and clearer interpretations of the variance component parameters than the original DBM procedure. CONCLUSIONS: We recommend using the modified DBM procedure that incorporates the recent developments.
RATIONALE AND OBJECTIVES: The Dorfman-Berbaum-Metz (DBM) method has been one of the most popular methods for analyzing multireader receiver-operating characteristic (ROC) studies since it was proposed in 1992. Despite its popularity, the original procedure has several drawbacks: it is limited to jackknife accuracy estimates, it is substantially conservative, and it is not based on a satisfactory conceptual or theoretical model. Recently, solutions to these problems have been presented in three papers. Our purpose is to summarize and provide an overview of these recent developments. MATERIALS AND METHODS: We present and discuss the recently proposed solutions for the various drawbacks of the original DBM method. RESULTS: We compare the solutions in a simulation study and find that they result in improved performance for the DBM procedure. We also compare the solutions using two real data studies and find that the modified DBM procedure that incorporates these solutions yields more significant results and clearer interpretations of the variance component parameters than the original DBM procedure. CONCLUSIONS: We recommend using the modified DBM procedure that incorporates the recent developments.
Authors: F Collettini; J C Martin; F Diekmann; E Fallenberg; F Engelken; S Ponder; T J Kroencke; B Hamm; A Poellinger Journal: Eur Radiol Date: 2011-09-27 Impact factor: 5.315
Authors: Milica Medved; Xiaobing Fan; Hiroyuki Abe; Gillian M Newstead; Abbie M Wood; Akiko Shimauchi; Kirti Kulkarni; Marko K Ivancevic; Lorenzo L Pesce; Olufunmilayo I Olopade; Gregory S Karczmar Journal: Acad Radiol Date: 2011-10-01 Impact factor: 3.173
Authors: Alexandre Ba; Craig K Abbey; Damien Racine; Anaïs Viry; Francis R Verdun; Sabine Schmidt; François O Bochud Journal: J Med Imaging (Bellingham) Date: 2019-05-20
Authors: David L Leong; Louise Rainford; Tamara Miner Haygood; Gary J Whitman; Philip M Tchou; William R Geiser; Selin Carkaci; Patrick C Brennan Journal: J Digit Imaging Date: 2012-10 Impact factor: 4.056