Comparison of two commercial systems for computer-assisted detection (CAD) as an aid to interpreting screening mammograms.

PURPOSE: To compare the diagnostic accuracy of two commercial CAD systems (CADx and R2) and their impact as an aid to conventional reading of screening mammograms.
MATERIALS AND METHODS: The image set considered consisted of 120 mammograms, 89 confirmed negative and 31 with subsequent interval cancers (11 classified as false negatives (FN), 20 as "minimal signs" (MS)). The set was digitised and processed with CAD, and printouts obtained of the mammograms with indications of the areas warranting review. Six expert radiologists read the mammograms three times, once using conventional reading and twice using CAD reading with CADx and R2, respectively. The two CAD systems were compared in terms of diagnostic accuracy of the marks and the impact of CAD reading compared to conventional reading and to the use of independent second reading simulated by combining pairs of single conventional readings.
RESULTS: R2 highlighted more calcifications (218 vs 132, +65%) and CADx highlighted more masses (208 vs 105, +98%). CADx and R2 marked 15 and 17 out of 31 cancers, respectively (sensitivity 48.3% vs 54.8%, chi squared=6.4, p=0.79), 10 and 6 out of 11 FN (90.9% vs 54.5%, chi squared=2.0, p=0.15), respectively, and 5 and 11 out of 20 MS (25.0% vs 55.0%, chi squared=2.6, p=0.10), respectively. As for specificity, the false positive markings for masses were on average (per case) 1.60 for CADx and 0.75 for R2, those for calcifications were 1.08 for CADx and 1.77 for R2 and the total false positive markings were 2.68 for CADx and 2.52 for R2. CADx and R2 marked 73 and 63 of 89 negative controls (specificity = 0.18 vs 0.29, chi squared=2.52, p=0.11), respectively. All the radiologists showed greater sensitivity with CAD reading compared to conventional reading. On average, sensitivity with conventional reading was 58.6% (109/186), as against 70.9% (132/186) for CADx or R2 (chi squared=5.71, p=0.016). Sensitivity for FN cases was 71.2% (47/66) with conventional reading, 84.8% (56/66) with CADx (chi squared=2.82, p=0.09) and 80.3% (53/66) for R2 (chi squared=1.03, p=0.30) (CADx vs R2, chi squared=0.21, p=0.64). Sensitivity for MS cases was 51.6% (62/120) for conventional reading, 63.3% (76/120) for CADx (chi squared=2.88, p=0.08) and 65.8% (79/120) for R2 (chi squared=4.40, p=0.03) (CADx vs R2, chi squared=0.07, p=0.78). The recall rates were 18.1% (97/534) for conventional reading, 29.7% (159/534) for CADx (chi squared=5.72, p=0.01) and 24.3% (130/534) for R2 (chi squared=10.11, p=10-5) (CADx vs R2, chi squared=3.71, p=0.05). Double reading was significantly more sensitive than conventional reading (chi squared=29.6, p=10-6), CADx (chi squared=5.33, p=0.02) and R2 (chi squared=5.33, p=0.02). The recall rate for double reading was significantly higher than for conventional reading (chi squared=21.5, p=10-6) whereas no significant difference was detected when compared to CADx (chi squared=0.16, p=0.68) or R2 (chi squared=3.4, p=0.06).
CONCLUSIONS: Despite using different algorithms, the two CAD systems exhibit comparable levels of diagnostic accuracy and a similar positive impact on sensitivity when used as an aid to conventional reading. Single reading with either CAD system is as specific but not as sensitive to double independent reading: its use as an alternative to double reading cannot be recommended and should be investigated further by means of controlled prospective studies.