RATIONALE AND OBJECTIVES: The goal of many multiple-observer computer-aided detection (CADe) studies is to estimate the change in observers' diagnostic performance with CADe from their unaided performance. A key issue in these studies is the method for estimating the observers' unaided performance. The crossover design is considered the most valid. The sequential design takes less time and is less expensive but may be biased. We conducted a study to investigate the differences between these two designs. MATERIALS AND METHODS: Data from two large CADe studies using both types of unaided reads were analyzed. The first study involved three radiologists examining the chest x-rays of 200 patients for lung nodules. The second study involved 19 observers interpreting the computed tomography colonography images of 100 patients for polyps. Observers' sensitivity, specificity, and receiver operating characteristic areas were estimated while unaided in both designs and compared to their accuracy with CADe. Bias, inter-observer variability, and correlations between unaided and aided results were assessed. RESULTS: Observers tend to perform better while unaided in the sequential design than while unaided in the crossover design, but the differences are small. The inter-observer variability is larger in the sequential design. The correlations between unaided and aided results are larger in the sequential design. 95% CIs for the change with CADe are narrower with the sequential design. CONCLUSION: The estimated effect of CADe on observer performance is similar regardless of the study design. Use of the sequential design may save investigators time and resources. Copyright (c) 2010 AUR. Published by Elsevier Inc. All rights reserved.
RATIONALE AND OBJECTIVES: The goal of many multiple-observer computer-aided detection (CADe) studies is to estimate the change in observers' diagnostic performance with CADe from their unaided performance. A key issue in these studies is the method for estimating the observers' unaided performance. The crossover design is considered the most valid. The sequential design takes less time and is less expensive but may be biased. We conducted a study to investigate the differences between these two designs. MATERIALS AND METHODS: Data from two large CADe studies using both types of unaided reads were analyzed. The first study involved three radiologists examining the chest x-rays of 200 patients for lung nodules. The second study involved 19 observers interpreting the computed tomography colonography images of 100 patients for polyps. Observers' sensitivity, specificity, and receiver operating characteristic areas were estimated while unaided in both designs and compared to their accuracy with CADe. Bias, inter-observer variability, and correlations between unaided and aided results were assessed. RESULTS: Observers tend to perform better while unaided in the sequential design than while unaided in the crossover design, but the differences are small. The inter-observer variability is larger in the sequential design. The correlations between unaided and aided results are larger in the sequential design. 95% CIs for the change with CADe are narrower with the sequential design. CONCLUSION: The estimated effect of CADe on observer performance is similar regardless of the study design. Use of the sequential design may save investigators time and resources. Copyright (c) 2010 AUR. Published by Elsevier Inc. All rights reserved.
Authors: Brandon D Gallas; Heang-Ping Chan; Carl J D'Orsi; Lori E Dodd; Maryellen L Giger; David Gur; Elizabeth A Krupinski; Charles E Metz; Kyle J Myers; Nancy A Obuchowski; Berkman Sahiner; Alicia Y Toledano; Margarita L Zuley Journal: Acad Radiol Date: 2012-02-03 Impact factor: 3.173
Authors: Stephan H Polanec; Katja Pinker-Domenig; Peter Brader; Dietmar Georg; Shahrokh Shariat; Claudio Spick; Martin Susani; Thomas H Helbich; Pascal A Baltzer Journal: World J Urol Date: 2015-09-25 Impact factor: 4.226
Authors: Feng Li; Roger Engelmann; Lorenzo L Pesce; Kunio Doi; Charles E Metz; Heber Macmahon Journal: Radiology Date: 2011-09-23 Impact factor: 11.105
Authors: Matthew T McKenna; Shijun Wang; Tan B Nguyen; Joseph E Burns; Nicholas Petrick; Ronald M Summers Journal: Med Image Anal Date: 2012-05-03 Impact factor: 8.545
Authors: Thaworn Dendumrongsup; Andrew A Plumb; Steve Halligan; Thomas R Fanshawe; Douglas G Altman; Susan Mallett Journal: PLoS One Date: 2014-12-26 Impact factor: 3.240
Authors: L Barinov; A Jairaj; M Becker; S Seymour; E Lee; A Schram; E Lane; A Goldszal; D Quigley; L Paster Journal: J Digit Imaging Date: 2019-06 Impact factor: 4.056