Medical auditing of whole-breast screening ultrasonography.

Since breast ultrasonography (US) has been used as an adjunctive screening modality in women with dense breasts, the need has arisen to evaluate and monitor its possible harm and benefits in comparison with other screening modalities such as mammography. Recently, the fifth edition of the Breast Imaging Reporting and Data System published by the American College of Radiology has suggested auditing methods for screening breast US. However, the method proposed therein is slightly different from how diagnostic performance was calculated in previous studies on screening breast US. In this article, the background and core aspects of medical audits of breast cancer screening will be reviewed to provide an introduction to the medical auditing of screening breast US, with the goal of helping radiologists to understand and identify potential ways to improve outcomes.

Introduction

Mammography is the only screening modality proven to reduce breast cancer deaths among women aged 40 years and older in previous randomized clinical trials [1], despite questions regarding the degree of its contribution in improving breast cancer survival in screening settings [2]. However, the false-negative rates of mammography can reach 50% in dense breasts [3], and the survival benefit of performing mammography is lower in younger women with dense breasts [4], suggesting the necessity of developing adjunctive screening modalities. One of the suggested adjunctive modalities is ultrasonography (US), for which the incremental cancer detection rates have been reported to be 2.7 to 4.2 per 1,000 women screened [5,6]. However, several aspects of breast US need to be verified before it can be used as the primary screening test for breast cancer.

Although improved survival is the main goal of breast cancer screening programs, obtaining follow-up data and evaluating the final outcomes requires a lengthy period of time. Therefore, researchers have identified several surrogate markers that can be assessed within a short-term follow-up and that are also periodically used to monitor the efficacy of screening programs. Medical audits for breast cancer screening programs with mammography have been well established [7,8], along with legislation making such audits mandatory [9]. However, the medical auditing of screening breast US has not been established to date.

In this article, the background and core aspects of medical audits of breast cancer screening are reviewed to provide an introduction to the medical auditing of screening breast US, with the goal of helping radiologists to understand and identify potential ways to improve outcomes.

Background of the Medical Auditing of Screening Mammograms

The main purpose of a medical audit, first described in a white paper of the U.K. National Health Service in 1989, is to improve patient care and outcomes, and the main elements of medical audits are the systematic review of care against solid criteria, the implementation of changes, and monitoring to confirm improvement [10,11]. For mammography, the purpose of a medical audit is to provide feedback to facilities and doctors on their performance relative to established benchmarks and to improve the overall quality of the breast cancer screening program. Several reports have shown the effectiveness of medical auditing. One showed improved sensitivity, with maintained positive predictive values and an increased number of detected cancers, of which most were node-negative, during 2 years of education and feedback on the audit results were provided to radiologists [12]. These results showed improvements in the effectiveness of the screening program after the medical audit, despite a 50% increase in the use of additional mammographic views and sonography. The Breast Imaging Reporting and Data System (BI-RADS) was established by the American College of Radiology (ACR) in response to communication issues and societal concerns about false negatives occurring in the context of a nationwide screening mammography program [13-15]. The ACR organized the overall structure of mammography reports, including assessment categories and the corresponding management strategies, supplied benchmarks for mammographic interpretation, and furthermore, developed the National Mammography Database within the National Radiology Data Registry [15].

Medical audits are also necessary because of one serious harm of mammography that has recently become an issue: anxiety among healthy women due to positive results, of which most will turn out to be false positives, with benign results on biopsy [16,17]. It has been found that two or three breast cancers will be identified among 100 women who receive annual mammograms over the course of 10 years. However, of these 100 women, 62 will eventually have positive results on their mammograms. Furthermore, seven to 10 will undergo biopsies with benign pathology results. Therefore, it must be determined whether the anxiety arising from the 62 false-positive results and 10 benign biopsies is inevitable and/or whether it is justifiable given the two or three detected breast cancers. Searching for ways to lower the excess amount of false-positive results and benign biopsies needs to be a part of the medical audit. The management of screening mammography should be a constant balance between possible harm and benefits; in other words, between missing too many cancers and ordering too many biopsies and recalls. This balance should be attained through medical auditing [9,15,18].

Currently, the following basic elements of a medical audit are recommended: (1) to track all positive mammograms, (2) to correlate the imaging findings with the pathologic results of all biopsies performed, and (3) to review known false-negatives within 12 months of the mammography examination [19]. The majority of approximately 250 radiologists in a survey on medical audits agreed that audit reports were valuable [20]. However, approximately half of the radiologists with less than 10 years of experience and three-fourths of radiologists in their early 30s replied that they would consider ceasing to interpret mammograms if congress mandated more intensive auditing requirements without providing funds to support the regulation. Thus, conducting more extensive medical audits needs both the cooperation of radiologists and the provision of sufficient resources [10,20].

Characteristics and Audit Data of Screening Breast US: Comparison with Benchmarks for Mammography

Cancer Detection Rate

The role of screening breast US in dense breasts has been studied since the early 2000s, and was recently further emphasized by new legislation in the United States requiring patients to be notified of their breast density after a screening mammography [21]. In asymptomatic women, supplementary hand-held US performed by radiologists detected 1.8-5.3 additional cancers per 1,000 women [5,21-24], and the overall cancer detection rate in supplemental screening US was 3.7 per 1,000 women (248 of 66,828) in 10 single- or multi-center studies [25]. Among these asymptomatic women, based on known risk factors, more than three-fifths of women with breast cancers detected only by screening US (89 of 145 women) were found to be at elevated risk in seven studies [5,6,18,21,26-29]. Recently, since the implementation of the Connecticut Public Act, breast US has been performed on average-risk women as well, and the cancer detection rate of breast US for these women (1.6-2.4 per 1,000 examinations) [21,24,30,31] has tended to be lower than the range accepted for screening mammography (≥2.5 per 1,000 examinations) [32] and the previously observed detection rate in high-risk women (3.7-5.3 per 1,000 examinations) [5]. As more women without risk factors for breast cancer undergo screening breast US, the acceptable range of the cancer detection rate needs to be reconsidered for screening breast US [31].

Characteristics of Breast Cancers Detected by Screening US

Supplemental breast US improved the detection of mostly node-negative invasive breast cancer in past studies; approximately 90% of breast cancers detected by screening US were stage 0 or I and node-negative [25], a much higher percentage than the benchmarks for mammography (74.8% and 77.3%, respectively) [32], which suggests that breast US can detect early breast cancer. However, the positive predictive value of the recommendation for tissue diagnosis (PPV2) and the positive biopsy rate (PPV3) showed a large proportion of benign pathologic results (3.3%-11.7%), much lower than the acceptable range for mammography (20%-40%) [5,21,22,24,31]. This suggests that further studies are needed on US interpretation and supplemental sonographic techniques to improve the positive predictive value of screening breast US.

The majority of breast cancers detected by breast US (over 90%) are known to be invasive, which may suggest that breast US can detect early but life-threatening breast cancer [5,33]. However, further studies must be performed to generalize these results, because the data were also based on study populations with a large proportion (70%-100%) of high-risk women, as was the case for the cancer detection rates. Recent studies focusing on women with an average risk for breast cancer showed much lower rates of the detection of invasive cancer (range, 0% to 60%; overall mean, 50% [5 of 10]) (Table 1) [21,22,24,34]. Moreover, these results are also somewhat different from the results of studies on high-risk women or screening mammography. In particular, the audit data for screening breast US among women at an average risk for breast cancer demonstrated a higher abnormal interpretation rate (25.0%-31.1% vs. 20.9% and 11.5%, respectively), a lower positive predictive value for biopsies (3.4%-5.3% vs. 9.0% and 29.2%, respectively), and more common recommendations for short-term follow-up (22.3%-26.6% vs. 11.1%-13.5% and 3.2%, respectively). These disparities indicate that it is necessary to collect various benchmarks for screening breast US according to risk stratification.

Table 1.

Summary of studies on screening breast ultrasonography according to the risk for breast cancer in comparison with results for screening mammography

Variable	Berg et al. [5], 2012		Hooley et al. [21], 2012		Chang et al. [22], 2015	Moon et al. [24], 2015	Weigert and Steenbergen [34], 2015
Modality	MG	US	US		US	US	US
Risk	Increased	Increased	Increased	Average	Average	Average	Not specified
No. of examinations	7,473	7,473	236	614	1,526	2,005	10,282
No. of cancers	59	58	2	1	5	4	24
Yield, per 1000	7.9	7.7	8.5	1.6	3.3	2.0	2.3
Invasive	41/59 (69.5)	53/58 (91.4)	2/2 (100)	0/1 (0)	3/5 (60.0)	2/4 (50.0)	10/21 (47.6)[a)]
Node-negative	18/41 (43.9)	34/53 (64.2)	2/2 (100)	-	3/3 (100)	2/2 (100)	-
Abnormal interpretation (recall) rate	306/2,659 (11.5)[b)]	555/2,659 (20.9)[b)]	234 (25.0)		431 (28.2)	623 (31.1)	1,310 (12.7)
Short-term follow-up	84/2,659 (3.2)[b)]	296/2,659 (11.1)[b)]	About 32 (13.5)	About 155 (25.3)[c)]	340 (22.3)	533 (26.6)	875 (9.0)
Biopsy rate	65/2,659 (2.4)[b)]	233/2,659 (8.8)[b)]	66/935 (7.1)		91/1,526 (6.0)	88/2,005 (4.4)	435/10,282 (4.2)
PPV3	19/65 (29.2)[b)]	21/233 (9.0)[b)]	3/66 (4.5)		5/91 (5.3)	3/88 (3.4)	24/435 (5.5)[d)]

Values are presented as number (%).

MG, mammography; US, ultrasonography; PPV3, positive biopsy rate.

Two carcinomas (invasive/uncertain) and one lymphoma were excluded [34].

Calculated based on a prevalence screen [5].

)Inferred from Fig. 2 of the Hooley et al.’s article [21].

Eight point nine percent if high-risk pathologic lesions are included (39 of 435) [34].

Proportion of probably benign findings

An abundant proportion of BI-RADS 3 (probably benign) findings were found to require additional short-interval follow-up in studies on screening US [5,21,22,24]. Because BI-RADS recommends not using BI-RADS 3 (probably benign) findings in the interpretation of screening mammography, the data for breast US do not correspond to what is expected. Although breast US is used as a screening modality, it includes a diagnostic component as well as a screening component. In fact, no guidelines exist regarding the proportion of BI-RADS 3 findings in diagnostic mammography. Therefore, it is inevitable that a higher proportion of BI-RADS 3 lesions are found through breast US screening than through screening mammography. However, recommending additional examinations prior to the next screening examination results in patient anxiety and demands additional resources. This could be a hurdle in the implementation of screening breast US. Therefore, we need to collect data and establish benchmarks for the proportion of BI-RADS 3 (probably benign) findings in screening breast US.

Most of the audit data on whole-breast screening US reviewed so far have been obtained from previous studies that focused on asymptomatic women who had negative mammograms in dense breasts, in whom screening US was used as a supplemental modality after mammographic information was already obtained, with the exception of the American College of Radiology Imaging Network (ACRIN) 6666 study [5,23]. Therefore, we need to recognize that the currently available data cannot be utilized to evaluate the use of US alone to screen for breast cancer.

Medical Audit of Breast US Suggested by ACR BI-RADS

The definition of a positive screening result in breast US has been continuously debated [35,36]. Based on the principle that its definition should be objective and reproducible, and that comparisons with other modalities should be possible, a positive screening was defined by the ACR as the acquisition of additional recorded images such as additional mammographic views, regardless of the duration of the examination or whether a second opinion was requested [32]. Breast US also includes a diagnostic component as well as a screening component. A single report may be issued for both components because it would be awkward to report the two components of a single US examination separately. However, the ACR suggested separate auditing for the screening and diagnostic components of combined US examinations [32].

According to the protocol of a previous multicenter study, which was proposed as a benchmark study in BI-RADS, the image requirement for a negative examination is at least five images per breast, one for each quadrant, and one for the retroareolar area [37]. If a nonbenign breast lesion is found in the screening setting, the examination is considered to have become diagnostic. Therefore, the screening breast US report should indicate whether only standard images were recorded or whether additional (diagnostic) images were obtained. Exams with additional images should be considered audit-positive for the screening component, regardless of the final assessment of breast US (Table 2). Therefore, a positive screening includes all subcategories of BI-RADS categories 3, 4, and 5, and categories 1 and 2 with more than five images per breast. Negative screenings include only BI-RADS categories 1 and 2 with five images per breast. Additionally, orthogonal images are not allowed for BI-RADS category 2, whereas all BI-RADS categories 1 and 2 findings were considered as negative screenings regardless of the number of images in the previous literature [5,6,18,21,22,24,26,28,30,31,33,34]. Therefore, the medical audit data will change in correspondence with the ACR recommendations.

Table 2.

Reference for interpreting biopsy results (screening and diagnostic) according to the ACR BI-RADS

Interpretation		According to the BI-RADS, fifth edition
		5 Images for each breast			≥6 Images for either breast
		Biopsy-positive	Biopsy-negative	Biopsy-positive		Biopsy-negative
Component: screening (S) or diagnostic (D)		S	S	S	D	S	D
BI-RADS categories 1,2	Image results	Negative		Positive	Negative	Positive	Negative
	Audit data	FN	TN	TP	FN	FP	TN
BI-RADS category 3	Image results	-	-	Positive	Negative	Positive	Negative
	Audit data	-	-	TP	FN	FP	TN
BI-RADS categories 4, 5	Image results	-	-	Positive	Positive	Positive	Positive
	Audit data	-	-	TP	TP	FP	FP

ACR, American College of Radiology; BI-RADS, Breast Imaging Reporting and Data System; FN, false negative; TN, true negative; TP, true positive; FP, false positive.

Potential Ways to Improve the Outcomes of Medical Audits of Breast US

As discussed above, a high short-term follow-up recommendation rate and low positive predictive value are characteristics of screening breast US [38]; however, several studies have reported reducing the number of BI-RADS categories 3 and 4 cases by applying supplemental techniques, such as shear-wave elastography (SWE) and/or Doppler US [39,40], or by applying strict criteria to the imaging findings [41,42]. The addition of SWE and Doppler US to B-mode US increased the specificity by more than 45% points in non-mass lesions detected by screening US without loss in sensitivity (from 10 of 42 to 29 of 42) and helped 65% of patients with BI-RADS category 4a lesions (19 of 29) avoid unnecessary biopsies [39].

By applying the strict criteria of the ACRIN 6666 protocol to imaging findings, 18.3% of BI-RADS category 3 lesions (213 of 1,164) were retrospectively recategorized as category 2 lesions [42]. Over the course of 3 years after education and feedback on audit results were provided in clinical practice, the rate of BI-RADS categories 3 to 4a was halved (from 22.6%-39.4% to 11.1%-16.0%) and the biopsy rates also significantly decreased, from 6.5% to 2.4%, while the cancer detection rate of supplemental screening US was maintained (2.8 per 1,000 examinations) [41].

Conclusion

Medical audits are essential for quality assurance. Therefore, they are not optional for breast cancer screening programs, and surrogate measures for medical audits of screening breast US need to be adjusted and developed. Medical audits will help screening breast US to become widely and effectively used as the primary screening test with the goal of improving the survival of breast cancer patients.