Research Highlight: Artificial Intelligence for Ruling Out Negative Examinations in Screening Breast MRI.

Take-home points

• Artificial intelligence (AI) application in triaging could optimize worklists after prioritizing examinations according to the complexity or likelihood of cancer diagnosis (i.e., soft triage) or identifying and ruling out negative or cancer-free examinations from the worklist (i.e., hard triage).

• A published AI model ruled out 39.7% of 3796 normal examinations maintaining 100% sensitivity for breast cancer in screening breast MRI for female aged 50–75 years with extremely dense breasts.

• Relative weights of the benefits and harms of dismissing examinations by AI systems should be taken into account.

Related KJR Quick Survey results are provided at the end of the article.

For breast cancer screening, mammography is the current standard modality for average-risk female, decreasing the mortality rate by detecting early-stage cancers. However, screening mammograms have some limitations, such as mammographically missed cancers. The sensitivity of screening mammography is 80%–85%, which declines to 50%–64% for extremely dense breasts [12]. Mammographically dense fibroglandular tissue that appears radiopaque as breast masses can reduce mammographic sensitivity by masking breast cancers. This has led to a growing interest in imaging modalities beyond mammography for supplemental screening, despite the ongoing debate on its benefits.

Breast MRI has excellent sensitivity for breast cancer detection and has been routinely offered as supplemental screening for high-risk populations only [3]. Nonetheless, it has demonstrated an increased detection rate in intermediate-risk and average-risk female [34]. MRI, not limited by mammographic breast density, may be valuable as a supplemental screening tool for average-risk female with dense breasts [5]. The Dense Tissue and Early Breast Neoplasm Screening (DENSE) trial, a randomized controlled trial in the Netherlands, investigated the effect of supplemental MRI on the incidence of interval cancers in female with extremely dense breasts and showed its potential mortality benefit with an additional cancer detection rate of 16.5 per 1000 and a 50% reduction in interval cancer rate [56]. However, considering that extremely dense breasts account for approximately 10% of the screening population, radiologists’ workload associated with the population-wide use of supplemental MRI for female with extremely dense breasts must be overwhelming [7].

Recently, deep-learning-based AI in breast imaging has been rapidly evolving, with promising solutions for diagnosis and decision support in breast cancer. In addition, the shortage of expert breast radiologists has sparked interest in the application of AI to reduce radiologists’ workload, particularly in screening programs with the highest volume of examinations but a low prevalence of cancer (< 1%) [8]. The triaging system, which ranks patients based on urgency initially for emergency room settings, is now used in breast imaging to enhance efficiency while maintaining radiologists’ performance. AI application in triaging could optimize worklists after prioritizing examinations according to the complexity or likelihood of cancer diagnosis (i.e., soft triage) or to identify and rule out negative or cancer-free examinations from the worklist (i.e., hard triage) [191011].

Verburg et al. [12] recently published a study on AI application in automated triaging of 4581 supplemental breast MRI examinations in female with extremely dense breasts. They used a dataset from the first screening round of the DENSE trial for the AI algorithm to test the feasibility of dismissing normal MRI examinations (American College of Radiology Breast Imaging Reporting and Data System [BI-RADS] category 1) without any loss of sensitivity for detecting breast cancer. Their triaging model showed an average area under the receiver operating characteristic curve of 0.83 and ruled out 39.7% of 3796 normal MRI examinations, maintaining 100% sensitivity for breast cancer. A few other studies have reported the triage of screening mammograms using in-house deep-learning models or commercially available AI systems dedicated to cancer detection [13141516]. Optimal thresholds for triaging are supposed to be set regarding the trade-off between reducing the workload (i.e., dismissing normal, negative, or least suspicious examinations) and risking neglect cancers. Depending on the thresholds set in each study, the reduction in reading workload in screening mammography ranged from 17% to 91%. The AI models in these studies or typical commercial software AI tools are set to triage mammograms by choosing “cancer-free” thresholds. Therefore, it is unique that the model by Verburg et al. [12] was trained to triage “lesion-free” breast MRI examinations (BI-RADS category 1), not “cancer-free.” It is inevitable that the model’s performance is different from that of other AI systems for triaging mammograms. In addition, the threshold of the model by Verburg et al. [12] was set at a value corresponding to 100% sensitivity for cancer. Although Lång et al. [16] reported a 19.1% workload reduction by dismissing cancer-free mammographic examinations without missing any cancers, the remaining AI systems for triaging mammography reduced the workload at the expense of missing cancers. Although considered acceptable while maintaining the overall quality of outcomes, that is, with non-inferior performance to radiologists or screening programs, the relative weights of the benefits and harms of dismissing examinations by AI systems should be taken into account [141617]. Moreover, the additional hemodynamic information to morphologic features provided by contrast-enhanced MRI examinations might result in a higher workload reduction (40%) of the model by Verburg et al. [12] than the mammographic AI system by Lång et al. [16] (19%), while the more severe background parenchymal enhancement was, the more normal MRI examination was triaged to radiologic review, which should be considered when applying the screening program for female aged < 50 years. In addition, medicolegal, regulatory, and ethical disputes remain challenging for pre-selecting examinations using a standalone AI system without human reading [916].

In the study by Verburg et al. [12], the model was trained on maximum intensity projection (MIP) images obtained by subtracting the pre-contrast image from the first post-contrast image of the dynamic contrast-enhanced MRI series. To overcome the limitations of full-protocol screening MRI, such as cost, patient tolerance, table time, and reading time, abbreviated breast MRI has been introduced as a screening examination, particularly in average-risk female [418]. Despite variations among institutions, the protocol for abbreviated breast MRI usually includes a pre-contrast image, the first post-contrast image, and their derived images (subtraction and MIP images) [3]. Therefore, the study results are compatible with and may enhance the efficiency of MRI screening programs using the abbreviated protocol. Notably, the study also provided a deep SHapley Additive exPlanations (SHAP) color map that overlaid MIP images with a color range from blue to red, indicating negative and positive SHAP values corresponding to a low or high probability of lesion presence, respectively. Dealing with the “black box” issue of AI, this is likely to increase the transparency and accountability of the AI model by allowing an intuitive and fast explanation of the AI decision [19].

Despite the promising work of the authors on AI triaging, limitations exist. The generalizability of the results is not guaranteed. All female included in the DENSE trial data study were from the Netherlands, and the performance of the AI triaging system would differ according to the demographic characteristics of specific screening populations, such as cancer prevalence. The model should be leveraged in various screening scenarios. As acknowledged by the authors, further application of AI triaging is necessary for not only “lesion-free” but also “cancer-free” breast MRI.