RATIONALE AND OBJECTIVES: The purpose of this study was to validate the performance of a previously developed computer aid for breast mass classification for mammography on a new, independent database of cases not used for algorithm development. MATERIALS AND METHODS: A computer aid (classifier) based on the likelihood ratio (LRb) was previously developed on a database of 670 mass cases. The 670 cases (245 malignant) from one medical institution were described using 16 features from the American College of Radiology Breast Imaging-Reporting and Data System lexicon and patient history findings. A separate database of 151 (43 malignant) validation cases were collected that were previously unseen by the classifier. These new validation cases were evaluated by the classifier without retraining. Performance evaluation methods included Receiver Operating Characteristic (ROC), round-robin, and leave-one-out bootstrap sampling. RESULTS: The performance of the classifier on the training data yielded an average ROC area of 0.90 +/- 0.02 and partial ROC area (0.90AUC) of 0.60 +/- 0.06. The exact nonparametric performance on the validation set of 151 cases yielded a ROC area of 0.88 and 0.90AUC of 0.57. Using a 100% sensitivity cutoff threshold established on the training data (100% negative predictive value), the classifier correctly identified 100% of the malignant masses in the validation test set, while potentially obviating 26% of the biopsies performed on benign masses. CONCLUSION: The LRb classifier performed consistently on new data that was not used for classifier development. The LRb classifier shows promise as a potential aid in reducing the number of biopsies performed on benign masses.
RATIONALE AND OBJECTIVES: The purpose of this study was to validate the performance of a previously developed computer aid for breast mass classification for mammography on a new, independent database of cases not used for algorithm development. MATERIALS AND METHODS: A computer aid (classifier) based on the likelihood ratio (LRb) was previously developed on a database of 670 mass cases. The 670 cases (245 malignant) from one medical institution were described using 16 features from the American College of Radiology Breast Imaging-Reporting and Data System lexicon and patient history findings. A separate database of 151 (43 malignant) validation cases were collected that were previously unseen by the classifier. These new validation cases were evaluated by the classifier without retraining. Performance evaluation methods included Receiver Operating Characteristic (ROC), round-robin, and leave-one-out bootstrap sampling. RESULTS: The performance of the classifier on the training data yielded an average ROC area of 0.90 +/- 0.02 and partial ROC area (0.90AUC) of 0.60 +/- 0.06. The exact nonparametric performance on the validation set of 151 cases yielded a ROC area of 0.88 and 0.90AUC of 0.57. Using a 100% sensitivity cutoff threshold established on the training data (100% negative predictive value), the classifier correctly identified 100% of the malignant masses in the validation test set, while potentially obviating 26% of the biopsies performed on benign masses. CONCLUSION: The LRb classifier performed consistently on new data that was not used for classifier development. The LRb classifier shows promise as a potential aid in reducing the number of biopsies performed on benign masses.
Authors: Elizabeth S Burnside; Jesse Davis; Jagpreet Chhatwal; Oguzhan Alagoz; Mary J Lindstrom; Berta M Geller; Benjamin Littenberg; Katherine A Shaffer; Charles E Kahn; C David Page Journal: Radiology Date: 2009-04-14 Impact factor: 11.105