Abdulaziz S Alshabibi1,2, Moayyad E Suleiman1, Salman M Albeshan2, Robert Heard1, Patrick C Brennan1. 1. The Medical Image Optimisation and Perception Group (MIOPeG), Faculty of Medicine and Health, The University of Sydney, Susan Wakil Health Building, Camperdown, Australia. 2. Department of Radiology and Medical Imaging, The College of Applied Medical Sciences of King Saud University, Riyadh, Saudi Arabia.
Abstract
OBJECTIVES: To examine whether radiologists' performances are consistent throughout a reading session and whether any changes in performance over the reading task differ depending on experience of the reader. METHODS: The performance of ten radiologists reading a test set of 60 mammographic cases without breaks was assessed using an ANOVA, 2 × 3 factorial design. Participants were categorized as more (≥2,000 mammogram readings per year) or less (<2,000 readings per year) experienced. Three series of 20 cases were chosen to ensure comparable difficulty and presented in the same sequence to all readers. It usually takes around 30 min for a radiologist to complete each of the 20-case series, resulting in a total of 90 min for the 60 mammographic cases. The sensitivity, specificity, lesion sensitivity, and area under the ROC curve were calculated for each series. We hypothesized that the order in which a series was read (i.e. fixed-series sequence) would have a significant main effect on the participants' performance. We also determined if significant interactions exist between the fixed-series sequence and radiologist experience. RESULTS: Significant linear interactions were found between experience and the fixed sequence of the series for sensitivity (F[1] =5.762, p = .04, partial η2 = .41) and lesion sensitivity. (F[1] =6.993, p = .03, partial η2 = .46). The two groups' mean scores were similar for the first series but progressively diverged. By the end of the third series, significant differences in sensitivity and lesion sensitivity were evident, with the more experienced individuals demonstrating improving and the less experienced declining performance. Neither experience nor series sequence significantly affected the specificity or the area under the ROC curve. CONCLUSIONS: Radiologists' performance may change considerably during a reading session, apparently as a function of experience, with less experienced radiologists declining in sensitivity and lesion sensitivity while more experienced radiologists actually improve. With the increasing demands on radiologists to undertake high-volume reporting, we suggest that junior radiologists be made aware of possible sensitivity and lesion sensitivity deterioration over time so they can schedule breaks during continuous reading sessions that are appropriate to them, rather than try to emulate their more experienced colleagues. ADVANCES IN KNOWLEDGE: Less-experienced radiologists demonstrated a reduction in mammographic diagnostic accuracy in later stages of the reporting sessions. This may suggest that extending the duration of reporting sessions to compensate for increasing workloads may not represent the optimal solution for less-experienced radiologists.
OBJECTIVES: To examine whether radiologists' performances are consistent throughout a reading session and whether any changes in performance over the reading task differ depending on experience of the reader. METHODS: The performance of ten radiologists reading a test set of 60 mammographic cases without breaks was assessed using an ANOVA, 2 × 3 factorial design. Participants were categorized as more (≥2,000 mammogram readings per year) or less (<2,000 readings per year) experienced. Three series of 20 cases were chosen to ensure comparable difficulty and presented in the same sequence to all readers. It usually takes around 30 min for a radiologist to complete each of the 20-case series, resulting in a total of 90 min for the 60 mammographic cases. The sensitivity, specificity, lesion sensitivity, and area under the ROC curve were calculated for each series. We hypothesized that the order in which a series was read (i.e. fixed-series sequence) would have a significant main effect on the participants' performance. We also determined if significant interactions exist between the fixed-series sequence and radiologist experience. RESULTS: Significant linear interactions were found between experience and the fixed sequence of the series for sensitivity (F[1] =5.762, p = .04, partial η2 = .41) and lesion sensitivity. (F[1] =6.993, p = .03, partial η2 = .46). The two groups' mean scores were similar for the first series but progressively diverged. By the end of the third series, significant differences in sensitivity and lesion sensitivity were evident, with the more experienced individuals demonstrating improving and the less experienced declining performance. Neither experience nor series sequence significantly affected the specificity or the area under the ROC curve. CONCLUSIONS: Radiologists' performance may change considerably during a reading session, apparently as a function of experience, with less experienced radiologists declining in sensitivity and lesion sensitivity while more experienced radiologists actually improve. With the increasing demands on radiologists to undertake high-volume reporting, we suggest that junior radiologists be made aware of possible sensitivity and lesion sensitivity deterioration over time so they can schedule breaks during continuous reading sessions that are appropriate to them, rather than try to emulate their more experienced colleagues. ADVANCES IN KNOWLEDGE: Less-experienced radiologists demonstrated a reduction in mammographic diagnostic accuracy in later stages of the reporting sessions. This may suggest that extending the duration of reporting sessions to compensate for increasing workloads may not represent the optimal solution for less-experienced radiologists.
Authors: Sian Taylor-Phillips; Matthew G Wallis; David Jenkinson; Victor Adekanmbi; Helen Parsons; Janet Dunn; Nigel Stallard; Ala Szczepura; Simon Gates; Olive Kearins; Alison Duncan; Sue Hudson; Aileen Clarke Journal: JAMA Date: 2016-05-10 Impact factor: 56.272
Authors: Abdulaziz S Alshabibi; Mo'ayyad E Suleiman; Kriscia A Tapia; Robert Heard; Patrick C Brennan Journal: J Am Coll Radiol Date: 2021-01-19 Impact factor: 5.532
Authors: Elizabeth A Krupinski; Kevin S Berbaum; Robert T Caldwell; Kevin M Schartz; Mark T Madsen; David J Kramer Journal: J Am Coll Radiol Date: 2012-03 Impact factor: 5.532
Authors: Sian Taylor-Phillips; Markus C Elze; Elizabeth A Krupinski; Kathryn Dennick; Alastair G Gale; Aileen Clarke; Claudia Mello-Thoms Journal: J Digit Imaging Date: 2015-02 Impact factor: 4.056