RATIONALE AND OBJECTIVES: We sought to assess whether or not prevalence levels affected the confidence ratings of readers during the interpretation of cases in a laboratory receiver operating characteristic-type observer performance study. MATERIALS AND METHODS: We reanalyzed a previously conducted observer performance study that included 14 readers and 5 different levels of prevalence. The previous study yielded the observation that in the laboratory we could not detect a "prevalence effect" in terms of differences in areas under the receiver operating characteristic curves. The detection ratings (for presence or absence) of lung nodules, interstitial disease, and pneumothorax for the five prevalence levels were compared, and a test for trend in averaged ratings as a function of abnormality prevalence was performed within a mixed-model setting that accounts for different sources of variability and correlations induced by the study design. RESULTS: The ratings of the cases in terms of confidence that the specific abnormality in question is present tend, on average, to be larger when actual disease prevalence is lower. The rate of the increase of the average confidence ratings with the decreasing prevalence of a specific abnormality is very similar for actually positive and actually negative cases for every considered abnormality. The observed trend in the changes of the average confidence ratings as a function of prevalence levels was statistically significant (p < 0.01). CONCLUSION: Expectations of disease prevalence in the case mix during a laboratory observer performance study may systematically affect the behavior of observers in terms of their actual confidence ratings.
RATIONALE AND OBJECTIVES: We sought to assess whether or not prevalence levels affected the confidence ratings of readers during the interpretation of cases in a laboratory receiver operating characteristic-type observer performance study. MATERIALS AND METHODS: We reanalyzed a previously conducted observer performance study that included 14 readers and 5 different levels of prevalence. The previous study yielded the observation that in the laboratory we could not detect a "prevalence effect" in terms of differences in areas under the receiver operating characteristic curves. The detection ratings (for presence or absence) of lung nodules, interstitial disease, and pneumothorax for the five prevalence levels were compared, and a test for trend in averaged ratings as a function of abnormality prevalence was performed within a mixed-model setting that accounts for different sources of variability and correlations induced by the study design. RESULTS: The ratings of the cases in terms of confidence that the specific abnormality in question is present tend, on average, to be larger when actual disease prevalence is lower. The rate of the increase of the average confidence ratings with the decreasing prevalence of a specific abnormality is very similar for actually positive and actually negative cases for every considered abnormality. The observed trend in the changes of the average confidence ratings as a function of prevalence levels was statistically significant (p < 0.01). CONCLUSION: Expectations of disease prevalence in the case mix during a laboratory observer performance study may systematically affect the behavior of observers in terms of their actual confidence ratings.
Authors: David Gur; Howard E Rockette; Derek R Armfield; Arye Blachar; Jennifer K Bogan; Giuseppe Brancatelli; Cynthia A Britton; Manuel L Brown; Peter L Davis; James V Ferris; Carl R Fuhrman; Sara K Golla; Sanj Katyal; Joan M Lacomis; Barry M McCook; F Leland Thaete; Thomas E Warfel Journal: Radiology Date: 2003-07 Impact factor: 11.105
Authors: Brandon D Gallas; Heang-Ping Chan; Carl J D'Orsi; Lori E Dodd; Maryellen L Giger; David Gur; Elizabeth A Krupinski; Charles E Metz; Kyle J Myers; Nancy A Obuchowski; Berkman Sahiner; Alicia Y Toledano; Margarita L Zuley Journal: Acad Radiol Date: 2012-02-03 Impact factor: 3.173
Authors: D P Chakraborty; T M Haygood; J Ryan; E M Marom; M Evanoff; M F McEntee; P C Brennan Journal: Br J Radiol Date: 2012-05-09 Impact factor: 3.039
Authors: Nicholas Petrick; Berkman Sahiner; Samuel G Armato; Alberto Bert; Loredana Correale; Silvia Delsanto; Matthew T Freedman; David Fryd; David Gur; Lubomir Hadjiiski; Zhimin Huo; Yulei Jiang; Lia Morra; Sophie Paquerault; Vikas Raykar; Frank Samuelson; Ronald M Summers; Georgia Tourassi; Hiroyuki Yoshida; Bin Zheng; Chuan Zhou; Heang-Ping Chan Journal: Med Phys Date: 2013-08 Impact factor: 4.071
Authors: E Arana; F M Kovacs; A Royuela; A Estremera; H Sarasíbar; G Amengual; I Galarraga; C Martínez; A Muriel; V Abraira; J Zamora; C Campillo Journal: AJNR Am J Neuroradiol Date: 2011-04-14 Impact factor: 3.825
Authors: Brandon D Gallas; Weijie Chen; Elodia Cole; Robert Ochs; Nicholas Petrick; Etta D Pisano; Berkman Sahiner; Frank W Samuelson; Kyle J Myers Journal: J Med Imaging (Bellingham) Date: 2019-01-21
Authors: S J Lewis; C R Mello-Thoms; P C Brennan; W Lee; A Tan; M F McEntee; M Evanoff; M Pietrzyk; W M Reed Journal: Br J Radiol Date: 2014-05-12 Impact factor: 3.039