Patrick C Brennan1, Aarthi Ganesan1, Miguel P Eckstein2, Ernest Usang Ekpo3, Kriscia Tapia1, Claudia Mello-Thoms1, Sarah Lewis1, Mordechai Z Juni4. 1. Medical Image Optimisation and Perception Group (MIOPeG), Medical Imaging and Radiation Sciences, Faculty Research Group, Faculty of Health Sciences, the University of Sydney, Cumberland Campus, 75 East St, Lidcombe, NSW 2141, Australia. 2. Department of Psychological & Brain Sciences, University of California, Santa Barbara, CA 93106; Institute for Collaborative Biotechnologies, University of California, Santa Barbara, CA 93106. 3. Medical Image Optimisation and Perception Group (MIOPeG), Medical Imaging and Radiation Sciences, Faculty Research Group, Faculty of Health Sciences, the University of Sydney, Cumberland Campus, 75 East St, Lidcombe, NSW 2141, Australia. Electronic address: ernest.ekpo@sydney.edu.au. 4. Department of Psychological & Brain Sciences, University of California, Santa Barbara, CA 93106.
Abstract
RATIONALE AND OBJECTIVES: To establish the efficacy of pairing readers randomly and evaluate the merits of developing optimal pairing methodologies. MATERIALS AND METHODS: Sensitivity, specificity, and proportion correct were computed for three different case sets that were independently read by 16 radiologists. Performance of radiologists as single readers was compared to expected double reading performance. We theoretically evaluated all possible pairing methodologies. Bootstrap resampling methods were used for statistical analyses. RESULTS: Significant improvements in expected performance for double versus single reading (ie, delta performance) were shown for all performance measures and case-sets (p ≤ .003), with overall delta performance across all theoretically possible pairing schemes (n = 10,395) ranging between .05 and .08. Delta performance for the 20 best pairing schemes was significant (p < .001) and ranged between .07 and .10. Delta performance for 20 random pairing schemes was also significant (p ≤ .003) and ranged between .05 and .08. Delta performance for the 20 worst pairing schemes ranged between .03 and .06, reaching significance in delta proportion correct (p ≤ .021) for all three case-sets and in delta specificity for two case-sets (p ≤ .033) but not for a third case-set (p = .131), and not reaching significance in delta sensitivity for any of the three case-sets (.098 ≥ p ≥ .067). CONCLUSION: Significant benefits accrue from double reading, and while random reader pairing achieves most double reading benefits, a strategic pairing approach may maximize the benefits of double reading.
RATIONALE AND OBJECTIVES: To establish the efficacy of pairing readers randomly and evaluate the merits of developing optimal pairing methodologies. MATERIALS AND METHODS: Sensitivity, specificity, and proportion correct were computed for three different case sets that were independently read by 16 radiologists. Performance of radiologists as single readers was compared to expected double reading performance. We theoretically evaluated all possible pairing methodologies. Bootstrap resampling methods were used for statistical analyses. RESULTS: Significant improvements in expected performance for double versus single reading (ie, delta performance) were shown for all performance measures and case-sets (p ≤ .003), with overall delta performance across all theoretically possible pairing schemes (n = 10,395) ranging between .05 and .08. Delta performance for the 20 best pairing schemes was significant (p < .001) and ranged between .07 and .10. Delta performance for 20 random pairing schemes was also significant (p ≤ .003) and ranged between .05 and .08. Delta performance for the 20 worst pairing schemes ranged between .03 and .06, reaching significance in delta proportion correct (p ≤ .021) for all three case-sets and in delta specificity for two case-sets (p ≤ .033) but not for a third case-set (p = .131), and not reaching significance in delta sensitivity for any of the three case-sets (.098 ≥ p ≥ .067). CONCLUSION: Significant benefits accrue from double reading, and while random reader pairing achieves most double reading benefits, a strategic pairing approach may maximize the benefits of double reading.
Authors: Ralf H J M Kurvers; Stefan M Herzog; Ralph Hertwig; Jens Krause; Patricia A Carney; Andy Bogart; Giuseppe Argenziano; Iris Zalaudek; Max Wolf Journal: Proc Natl Acad Sci U S A Date: 2016-07-18 Impact factor: 11.205
Authors: S Ciatto; D Ambrogetti; R Bonardi; S Catarzi; G Risso; M Rosselli Del Turco; P Mantellini Journal: J Med Screen Date: 2005 Impact factor: 2.136
Authors: Sian Taylor-Phillips; Matthew G Wallis; Helen Parsons; Janet Dunn; Nigel Stallard; Helen Campbell; Sarah Sellars; Ala Szczepura; Simon Gates; Aileen Clarke Journal: Trials Date: 2014-01-10 Impact factor: 2.279