PURPOSE: To measure observer performance at various levels of prevalence. MATERIALS AND METHODS: A multiobserver multiabnormality receiver operating characteristic (ROC) study to assess the effect of prevalence on observer performance was conducted. Fourteen observers, including eight faculty members, two fellows, and four residents, interpreted 1,632 posteroanterior chest images with five prevalence levels by using a nested study design. Performance comparisons were accomplished by using a multireader multicase approach to assess the effect of prevalence from 28% (69 of 249) to 2% (31 of 1,577) on diagnostic accuracy. The mean times required to review and report a case were analyzed and compared for different levels of prevalence and readers' experience. RESULTS: Area under the ROC curve demonstrated that, with the study experimental conditions, no significant effect could be measured as a function of prevalence (P >.05) for any abnormality, group of cases, or readers. There were no significant differences (P >.05) in the mean times required to review and report cases at different prevalence levels and with different groups of readers. CONCLUSION: The consistency in the results and the size of this study suggest that with laboratory conditions, if a prevalence effect exists, it is quite small in magnitude; hence, it will not likely alter conclusions derived from such studies.
PURPOSE: To measure observer performance at various levels of prevalence. MATERIALS AND METHODS: A multiobserver multiabnormality receiver operating characteristic (ROC) study to assess the effect of prevalence on observer performance was conducted. Fourteen observers, including eight faculty members, two fellows, and four residents, interpreted 1,632 posteroanterior chest images with five prevalence levels by using a nested study design. Performance comparisons were accomplished by using a multireader multicase approach to assess the effect of prevalence from 28% (69 of 249) to 2% (31 of 1,577) on diagnostic accuracy. The mean times required to review and report a case were analyzed and compared for different levels of prevalence and readers' experience. RESULTS: Area under the ROC curve demonstrated that, with the study experimental conditions, no significant effect could be measured as a function of prevalence (P >.05) for any abnormality, group of cases, or readers. There were no significant differences (P >.05) in the mean times required to review and report cases at different prevalence levels and with different groups of readers. CONCLUSION: The consistency in the results and the size of this study suggest that with laboratory conditions, if a prevalence effect exists, it is quite small in magnitude; hence, it will not likely alter conclusions derived from such studies.
Authors: Brandon D Gallas; Heang-Ping Chan; Carl J D'Orsi; Lori E Dodd; Maryellen L Giger; David Gur; Elizabeth A Krupinski; Charles E Metz; Kyle J Myers; Nancy A Obuchowski; Berkman Sahiner; Alicia Y Toledano; Margarita L Zuley Journal: Acad Radiol Date: 2012-02-03 Impact factor: 3.173
Authors: David Gur; Andriy I Bandos; Carl R Fuhrman; Amy H Klym; Jill L King; Howard E Rockette Journal: Acad Radiol Date: 2007-01 Impact factor: 3.173
Authors: Berkman Sahiner; Heang-Ping Chan; Marilyn A Roubidoux; Lubomir M Hadjiiski; Mark A Helvie; Chintana Paramagul; Janet Bailey; Alexis V Nees; Caroline Blane Journal: Radiology Date: 2007-01-23 Impact factor: 11.105
Authors: S J Lewis; C R Mello-Thoms; P C Brennan; W Lee; A Tan; M F McEntee; M Evanoff; M Pietrzyk; W M Reed Journal: Br J Radiol Date: 2014-05-12 Impact factor: 3.039