Stephen Littlefair1, Claudia Mello-Thoms2, Warren Reed2, Marius Pietryzk3, Sarah Lewis2, Mark McEntee2, Patrick Brennan2. 1. Medical Image Optimisation and Perception Group (MIOPeG), Discipline of Medical Radiation Sciences, Faculty of Health Sciences, University of Sydney, Room M213, Cumberland Campus, East Street, Lidcombe, NSW 2141, Australia. Electronic address: s.littlefair@usyd.edu.au. 2. Medical Image Optimisation and Perception Group (MIOPeG), Discipline of Medical Radiation Sciences, Faculty of Health Sciences, University of Sydney, Room M213, Cumberland Campus, East Street, Lidcombe, NSW 2141, Australia; National Imaging Facilities, Brain and Mind Research Institute (BMRI), Faculty of Health Sciences, University of Sydney, Sydney, New South Wales, Australia. 3. Institute of Physics, London, United Kingdom.
Abstract
RATIONALE AND OBJECTIVES: The aim of this study was to measure the effect of prevalence expectation as determined by clinical history on the diagnostic performance of radiologists during pulmonary nodule detection on adult chest radiographs. MATERIALS AND METHODS: A multi-observer, counter-balanced study (having half the readers in each group read a different condition initially) was performed to assess the effect of abnormality expectation on experienced radiologists' performance. A total of 33 board-certified radiologists were divided into three groups and searched for evidence of malignancy on a single set of 47 postero-anterior (PA) chest radiographs, 10 of which contained a single pulmonary nodule. The radiologists were unaware of disease prevalence. Before each viewing of the same dataset, the radiologists were allocated to two of three conditions based on the differing clinical information (previous cancer, no history, visa applicant). Location sensitivity, specificity, and jack-knife free-response receiver operator characteristics figure of merit were used to compare radiologist performance between conditions. RESULTS: A significant reduction in specificity was shown for the cancer compared to that for the visa condition (W = -41 P = 0.02). No other significant findings were demonstrated for this or the other condition comparisons. No significant difference in the performance of radiologists was noted when viewing images under the same conditions. CONCLUSIONS: This study suggested that there is a reduction in specificity with high compared to low prevalence expectation following specific radiological contexts. A reduction in specificity can have important clinical consequences leading to unnecessary interventions. The results and their implications emphasize the caution that should be placed on providing accurate referral criteria.
RATIONALE AND OBJECTIVES: The aim of this study was to measure the effect of prevalence expectation as determined by clinical history on the diagnostic performance of radiologists during pulmonary nodule detection on adult chest radiographs. MATERIALS AND METHODS: A multi-observer, counter-balanced study (having half the readers in each group read a different condition initially) was performed to assess the effect of abnormality expectation on experienced radiologists' performance. A total of 33 board-certified radiologists were divided into three groups and searched for evidence of malignancy on a single set of 47 postero-anterior (PA) chest radiographs, 10 of which contained a single pulmonary nodule. The radiologists were unaware of disease prevalence. Before each viewing of the same dataset, the radiologists were allocated to two of three conditions based on the differing clinical information (previous cancer, no history, visa applicant). Location sensitivity, specificity, and jack-knife free-response receiver operator characteristics figure of merit were used to compare radiologist performance between conditions. RESULTS: A significant reduction in specificity was shown for the cancer compared to that for the visa condition (W = -41 P = 0.02). No other significant findings were demonstrated for this or the other condition comparisons. No significant difference in the performance of radiologists was noted when viewing images under the same conditions. CONCLUSIONS: This study suggested that there is a reduction in specificity with high compared to low prevalence expectation following specific radiological contexts. A reduction in specificity can have important clinical consequences leading to unnecessary interventions. The results and their implications emphasize the caution that should be placed on providing accurate referral criteria.
Authors: Roxanne Wadia; Kathleen Akgun; Cynthia Brandt; Brenda T Fenton; Woody Levin; Andrew H Marple; Vijay Garla; Michal G Rose; Tamar Taddei; Caroline Taylor Journal: JCO Clin Cancer Inform Date: 2018-12