Phuong Dung Yun Trieu1, Kriscia Tapia2, Helen Frazer3, Warwick Lee4, Patrick Brennan2. 1. Medical Image Optimization and Perception (MIOPeG), Faculty of Health Sciences, University of Sydney, Australia, Room M511, 75 East Street, Lidcombe NSW 2141, Australia. Electronic address: phuong.trieu@sydney.edu.au. 2. Medical Image Optimization and Perception (MIOPeG), Faculty of Health Sciences, University of Sydney, Australia, Room M511, 75 East Street, Lidcombe NSW 2141, Australia. 3. New South Wales Cancer Institute, Australia, Eveleigh, Australia. 4. South East Radiology, Australia, Croydon, Australia.
Abstract
BACKGROUND: Breast Screen Reader Assessment Strategy (BREAST) is an innovative training and research program for radiologists in Australia and New Zealand. The aim of this study is to evaluate the efficacy of BREAST test sets in improving readers' performance in detecting cancers on mammograms. MATERIALS AND METHODS: Between 2011 and 2018, 50 radiologists (40 fellows, 10 registrars) completed three BREAST test sets and 17 radiologists completed four test sets. Each test set contained 20 biopsy-proven cancer and 40 normal cases. Immediate image-based feedback was available to readers after they completed each test set which allowed the comparison of their selections with the truth. Case specificity, case sensitivity, lesion sensitivity, the Receiver Operating Characteristic (ROC) Area Under the Curve (AUC) and Jackknife Free-Response Receiver Operating Characteristic (JAFROC) Figure of Merit (FOM) were calculated for each reader. Kruskal-Wallis test was utilized to compare scores of the radiologist and registrars across all test-sets whilst Wilcoxon signed rank test was to compare the scores between pairs of test sets. RESULTS: Significant improvements in lesion sensitivity ranging from 21% to 31% were found in radiologists completing later test sets compared to first test set (p ≤ 0.01). Eighty three percent of radiologists achieved higher performance in lesion sensitivity after they completed the first read. Registrars had significantly better scores in the third test set compared to the first set with mean increases of 79% in lesion sensitivity (p = 0.005) and 37% in JAFROC (p = 0.02). Sixty percent and 100% of registrars increased their scores in lesion sensitivity in the second and third reads compared to the first read while the percentage of registrars with higher scores in JAFROC was 80%. CONCLUSION: Introduction of BREAST into national training programs appears to have an important impact in promoting diagnostic efficacy amongst radiologists and radiology registrars undergoing mammographic readings.
BACKGROUND: Breast Screen Reader Assessment Strategy (BREAST) is an innovative training and research program for radiologists in Australia and New Zealand. The aim of this study is to evaluate the efficacy of BREAST test sets in improving readers' performance in detecting cancers on mammograms. MATERIALS AND METHODS: Between 2011 and 2018, 50 radiologists (40 fellows, 10 registrars) completed three BREAST test sets and 17 radiologists completed four test sets. Each test set contained 20 biopsy-proven cancer and 40 normal cases. Immediate image-based feedback was available to readers after they completed each test set which allowed the comparison of their selections with the truth. Case specificity, case sensitivity, lesion sensitivity, the Receiver Operating Characteristic (ROC) Area Under the Curve (AUC) and Jackknife Free-Response Receiver Operating Characteristic (JAFROC) Figure of Merit (FOM) were calculated for each reader. Kruskal-Wallis test was utilized to compare scores of the radiologist and registrars across all test-sets whilst Wilcoxon signed rank test was to compare the scores between pairs of test sets. RESULTS: Significant improvements in lesion sensitivity ranging from 21% to 31% were found in radiologists completing later test sets compared to first test set (p ≤ 0.01). Eighty three percent of radiologists achieved higher performance in lesion sensitivity after they completed the first read. Registrars had significantly better scores in the third test set compared to the first set with mean increases of 79% in lesion sensitivity (p = 0.005) and 37% in JAFROC (p = 0.02). Sixty percent and 100% of registrars increased their scores in lesion sensitivity in the second and third reads compared to the first read while the percentage of registrars with higher scores in JAFROC was 80%. CONCLUSION: Introduction of BREAST into national training programs appears to have an important impact in promoting diagnostic efficacy amongst radiologists and radiology registrars undergoing mammographic readings.