Which supplementary imaging modality should be used for breast ultrasonography? Comparison of the diagnostic performance of elastography and computer-aided diagnosis.

PURPOSE: The aim of this study was to evaluate and compare the diagnostic performance of grayscale ultrasonography (US), US elastography, and US computer-aided diagnosis (US-CAD) in the differential diagnosis of breast masses.
METHODS: A total of 193 breast masses in 175 consecutive women (mean age, 46.4 years) from June to August 2015 were included. US and elastography images were obtained and recorded. A US-CAD system was applied to the grayscale sonograms, which were automatically analyzed and visualized in order to generate a final assessment. The final assessments of breast masses were based on the American College of Radiology Breast Imaging Reporting and Data System (BI-RADS) categories, while elasticity scores were assigned using a 5-point scoring system. The diagnostic performance of grayscale US, elastography, and US-CAD was calculated and compared.
RESULTS: Of the 193 breast masses, 120 (62.2%) were benign and 73 (37.8%) were malignant. Breast masses had significantly higher rates of malignancy in BI-RADS categories 4c and 5, elastography patterns 4 and 5, and when the US-CAD assessment was possibly malignant (all P<0.001). Elastography had higher specificity (40.8%, P=0.042) than grayscale US. US-CAD showed the highest specificity (67.5%), positive predictive value (PPV) (61.4%), accuracy (74.1%), and area under the curve (AUC) (0.762, all P<0.05) among the three diagnostic tools.
CONCLUSION: US-CAD had higher values for specificity, PPV, accuracy, and AUC than grayscale US or elastography. Computer-based analysis based on the morphologic features of US may be very useful in improving the diagnostic performance of breast US.

Introduction

In light of the wide application of breast ultrasonography (US) in daily practice, the American College of Radiology Breast Imaging Reporting and Data System (ACR BI-RADS) for breast US has been universally applied to facilitate communication between radiologists and clinicians and to standardize the management of women with breast abnormalities [1]. Studies have proven the ACR BI-RADS lexicon for US to be an effective system in the differential diagnosis of breast masses and the detection of malignancies [2-4]. However, the US features used in BI-RADS contain an overlap between benign and malignant breast masses, particularly in category 4 lesions, as this category includes a broad spectrum of breast masses with a wide range of risk for malignancy (2%-95%) [5]. At present, no specific US descriptor or any combination of US descriptors has been reported to accurately predict malignancy in breast masses detected on US [6].

With advances in technology, various tools have been developed and applied in clinical practice to improve the diagnostic performance of breast US. For instance, US elastography, which measures and visualizes the intrinsic strain of a target mass, providing additional information for mass characterization, has been applied to breast US [7-9]. Computer-aided diagnosis (CAD) has been applied to breast US interpretation, providing assistance in the morphologic analysis of breast masses according to the US BI-RADS descriptors as well as final assessments [3,10,11]. These additional diagnostic modalities use different characteristics of the target mass in lesion assessment; elastography uses tissue stiffness, whereas US-CAD uses morphologic characteristics. However, to date, no studies have compared the diagnostic performance of these additional imaging modalities.

The purpose of this study was to compare the diagnostic performance of grayscale US, elastography, and US-CAD in the differential diagnosis of breast masses visualized on US.

Materials and Methods

This retrospective study was approved by the Institutional Review Board of (Severance Hospital), and the requirement for informed consent was waived.

Patients

A total of 193 breast lesions in 175 consecutive women who were scheduled for breast US examinations, US-guided biopsy, or surgical excision at our institution from June to August 2015 were included in this study. The mean age of the 175 women was 46.4 years (range, 18 to 81 years). The mean size of the 193 breast masses was 14.9 mm (range, 3 to 52 mm). Of these lesions, 180 (93.3%) were pathologically diagnosed after US-guided core needle biopsy (n=90), vacuum-assisted excision (n=10), or surgical excision (n=80). Thirteen lesions (6.5%) were included based on typically benign US findings; this category included cysts (n=5) and benign masses that had been stable for more than 24 months (n=8).

US Examinations and Biopsies

US examinations were performed using a 3-12-MHz linear transducer (RS80A with Prestige, Samsung Medison, Co. Ltd., Seoul, Korea). Two staff radiologists (J.H.Y and E.-K.K) with 7 and 19 years of experience in breast imaging, respectively, were involved in image acquisition. The clinical information of the patient, including mammographic findings and prior US examinations, was given to the radiologists before the US examination. Bilateral breast examinations were routinely performed, during which the target breast masses were detected. Biopsies were performed of all breast masses classified as BI-RADS category 4 and 5. In addition, 23 masses classified as category 3 were pathologically confirmed on the patient’s request. For image analysis using US-CAD, a single directional movement covering the entire mass with surrounding breast parenchyma was recorded as a video clip. The elastography and US-CAD systems were applied after grayscale US by the same radiologist who performed the US examinations, and the elasticity score [7] and final assessment of the US-CAD analysis were recorded. The final assessments of grayscale US alone, US after incorporating elastography, and US-CAD were also recorded according to the ACR BI-RADS categories [1]. If required, a USguided biopsy was performed by the radiologist who had initially performed breast US after image acquisition.

US Elastography

US elastography examinations were performed using the freehand technique with the transducer placed perpendicular to the skin very softly. The US unit showed the images in a split-screen mode with grayscale images on the left and the corresponding elastography images on the right. For image acquisition, the breast mass was centered in the elastography box, in which the superior margin was set to cover the subcutaneous fat, the inferior margin to cover the pectoralis muscle, and the lateral margins to include a minimum of 5 mm of normal parenchyma neighboring the breast mass [12]. Real-time elastography images were acquired as a 256-color mapping that showed the amount of strain, using a scale ranging from red (largest strain or softest area) to green to blue (no strain or hardest area) [13].

The elasticity score of each breast mass was prospectively evaluated during examination according to the 5-point scale proposed by Itoh et al. [7]. A score of 1 meant even strain within the entire mass, a score of 2 meant even strain throughout the mass with some strain-free areas (a mosaic pattern of green and blue), a score of 3 meant strain only in the periphery of the mass but not in the center, a score of 4 meant no strain within the entire mass, and a score of 5 meant no strain within the entire lesion and the normal parenchyma surrounding it.

Application of US-CAD

In order to analyze the US-CAD data for each breast mass, one radiologist (J.H.Y) retrospectively reviewed the video clips recorded for each breast mass. Representative images were chosen from the clips, and a region of interest (ROI) was automatically drawn by the S-detect CAD system (RS80A with Prestige, Samsung Medison, Co. Ltd.). If the automatically generated ROI was considered inaccurate by the radiologist, it was adjusted manually. US characteristics according to the BI-RADS lexicon were automatically analyzed and visualized by the program, as well as final assessments (Fig. 1). The final assessments from US-CAD fell into the two categories of probably benign and possibly malignant.

Fig. 1.

The procedure for determining the ROI for US-CAD analysis.

After the ROI was drawn along the border of the mass, its ultrasonographic features were analyzed automatically by US-CAD and a final assessment was produced. ROI, region of interest; US, ultrasonography; CAD, computer-aided diagnosis.

Data and Statistical Analysis

Histopathologic results from US-guided core needle biopsies, vacuum-assisted excisions, or surgery were regarded as the standard reference. Patients with high-risk lesions on biopsy, including atypical ductal hyperplasia, atypical lobular hyperplasia, lobular carcinoma in situ, intraductal papilloma, a mucocele-like lesion, or a radial scar, were recommended to undergo surgical excision, based on which the final pathologic diagnosis was made. These pathologic results were regarded as benign for the purposes of statistical analysis. For the comparison of continuous variables, the independent two-sample t test was used. The chi-square test or the Fisher exact test was used in the comparison of categorical variables.

The final assessments based on the US BI-RADS categories were divided into two groups for statistical analysis: negative, made up of category 2 and 3 lesions, and positive, made up of category 4a to 5 lesions. Elasticity scores above 3 were considered positive in accordance with the previous study of Itoh et al. [7]. The final assessment by US-CAD was also in a dichotomized form: negative (probably benign) or positive (possibly malignant). Parameters reflecting the diagnostic performance of grayscale US, the elasticity score, and US-CAD, including sensitivity, specificity, positive predictive value (PPV), negative predictive value (NPV), and accuracy, were calculated and compared using the generalized estimating equations method. The area under the receiver operating characteristics curve (AUC) was acquired and compared using the Delong method.

Statistical analyses were performed using SAS ver. 9.2 (SAS Inc., Cary, NC, USA). All tests were two-sided, and P-values of <0.05 were considered to indicate statistical significance.

Results

Of the 193 breast masses, 120 (62.2%) were benign and 73 (37.8%) were malignant. The histopathologic results of the 193 breast masses are summarized in Table 1. Malignant masses were significantly larger than benign masses (19.7±11.5 mm vs. 11.8±7.0 mm, P<0.001). Women diagnosed with malignant masses were significantly older than women with benign masses (51.7±11.6 years vs. 42.2±12.7 years, P <0.001).

Table 1.

Histopathology results of the 193 breast masses

	Histopathology result	No. (%)
Benign (n=120)	Fibroadenoma or fibroadenomatoid hyperplasia	63 (52.5)
	Intraductal papilloma	8 (6.7)
	Sclerosing adenosis	7 (5.8)
	Radial scar	5 (4.2)
	Fibroepithelial tumor	2 (1.7)
	Apocrine metaplasia	6 (5)
	Fibrocystic changes	3 (2.5)
	Fat necrosis	3 (2.5)
	Stromal fibrosis	3 (2.5)
	Duct ectasia	3 (2.5)
	Granuloma	3 (2.5)
	Mucocele-like lesion	1 (0.8)
	Typically benign[a)]	13 (10.8)
Malignant (n=73)	Invasive ductal carcinoma	56 (76.7)
	Ductal carcinoma in situ	7 (9.6)
	Tubular carcinoma	4 (5.5)
	Mucinous carcinoma	3 (4.1)
	Invasive lobular carcinoma	2 (2.7)
	Medullary carcinoma	1 (1.4)

The category of typically benign includes ultrasonographic findings such as cysts (n=5) or benign masses that have been stable for more than 24 months (n=8).

Distribution of Breast Masses according to US, Elastography, and US-CAD

Table 2 summarizes the categories of the 193 breast masses according to grayscale US BI-RADS assessment, elastography patterns, and US-CAD. Breast masses had significantly higher rates of malignancy in BI-RADS categories 4c and 5, elastography patterns 4 and 5, and when the US-CAD assessment was possibly malignant (all P<0.001).

Table 2.

Categorical assessments of the 193 breast masses according to US, elastography, and US-CAD

	Benign (n=120)	Malignant (n=73)
US
Category
2	10 (8.3)	0
3	26 (21.7)	0
4a	74 (61.7)	15 (20.5)
4b	7 (5.8)	4 (5.5)
4c	3 (2.5)	17 (23.3)
5	0	37 (50.7)
Elastography
Pattern
1	9 (7.5)	0
2	40 (33.3)	14 (19.2)
3	42 (35.0)	17 (23.3)
4	22 (18.3)	25 (34.2)
5	7 (5.8)	17 (23.3)
US-CAD
Assessment
Possibly benign	81 (67.5)	11 (15.1)
Possibly malignant	39 (32.5)	62 (84.9)

Values are presented as number (%).

US, ultrasonography; CAD, computer-aided diagnosis.

Table 3 the summarizes the 193 breast masses according to pathology and imaging features, including US, elastography, and US-CAD. Among the 120 benign breast masses, negative elasticity scores were less common than positive elasticity scores (49 [40.8%] vs. 71 [59.2%], P=0.205). Negative US-CAD assessments were more common than positive US-CAD assessments in benign breast masses, although this trend did not reach statistical significance (81 [67.5%] vs. 39 [32.5%], P=0.564). Among the 84 benign masses assessed as category 4a or higher on grayscale US, more negative US-CAD assessments were found than negative elastography scores, although this trend was not statistically significant (65.5% [55 of 84] vs. 34.5% [29 of 84], P=0.864).

Table 3.

Categorical assessments of the 193 breast masses according to pathology findings and imaging features

US	Pathology
	Benign (n=120)			Malignant (n=73)			Benign (n=120)			Malignant (n=73)
	E (-)	E (+)	P-value	E (-)	E (+)	P-value	US-CAD (-)	US-CAD (+)	P-value	US-CAD (-)	US-CAD (+)	P-value
2	7 (14.3)	3 (4.2)	0.205	0	0	0.592	8 (9.9)	2 (5.1)	0.564	0	0	0.013
3	13 (26.5)	13 (18.3)		0	0		19 (23.5)	7 (17.9)		0	0
4a	26 (53.1)	48 (67.6)		4 (28.6)	11 (18.6)		48 (59.3)	26 (66.7)		4 (36.4)	11 (17.7)
4b	2 (4.1)	5 (7.0)		0	4 (6.8)		5 (6.2)	2 (5.1)		2 (18.2)	2 (3.2)
4c	1 (2.0)	2 (2.8)		4 (28.6)	13 (22.0)		1 (1.2)	2 (5.1)		4 (36.4)	13 (21.0)
5	0	0		6 (42.9)	31 (52.5)		0	0		1 (9.1)	36 (58.1)
Total	49	71		14	59		81	39		11	62

Values are presented as number (%).

US, ultrasonography; E, elastography; E (-), elastography patterns 1-3; E (+), elastography patterns 4-5; CAD, computer-aided design; US-CAD (-), possibly benign; US-CAD (+), possibly malignant.

Among the 73 malignant breast masses, positive elasticity scores were more common than negative scores (59 [80.8%] vs. 14 [19.2%], P=0.592). Positive US-CAD assessments were significantly more common in malignant breast masses than negative US-CAD assessments (62 [84.9%] vs. 11 [15.1%], P=0.013).

Diagnostic Performance of US, Elastography, and US-CAD

The diagnostic performance of grayscale US, elastography, and US-CAD is summarized in Table 4. For grayscale US, the sensitivity was 100.0%, the specificity was 30.0%, the PPV was 46.5%, the NPV was 100.0%, and the accuracy was 56.5%. Elastography had significantly lower sensitivity (80.8%) and NPV (77.8%, all P<0.001), and higher specificity (40.8%, P=0.042) than grayscale US. US-CAD had significantly higher specificity (67.5%), PPV (61.4%), and accuracy (74.1%) than grayscale US, with lower sensitivity (84.9%) and NPV (88.0%, all P<0.001). In comparison with elastography, US-CAD showed better diagnostic performance in regard to all parameters (all P<0.001). The AUC was highest for US-CAD (0.762), and this value was significantly higher than that of grayscale US (0.650, P=0.002) and elastography (0.608, P<0.001) (Fig. 2).

Table 4.

Diagnostic performance of grayscale US, elastography, and US-CAD

	US	E	P-value[a)]	US-CAD	P-value[a)]	P-value[b)]
Sensitivity	73/73 (100.0)	59/73 (80.8)	<0.001	62/73 (84.9)	<0.001	0.531
Specificity	36/120 (30.0)	49/120 (40.8)	0.042	81/120 (67.5)	<0.001	<0.001
PPV	73/157 (46.5)	59/130 (45.4)	0.651	62/101 (61.4)	<0.001	<0.001
NPV	36/36 (100)	49/63 (77.8)	<0.001	81/92 (88.0)	<0.001	0.071
Accuracy	109/193 (56.5)	108/193 (56.0)	>0.999	143/193 (74.1)	<0.001	<0.001
AUC (95% CI)	0.650 (0.609-0.691)	0.608 (0.545-0.672)	0.247	0.762 (0.703-0.821)	0.002	<0.001

Values are presented as number (%) unless otherwise indicated.

US, ultrasonography; CAD, computer-aided diagnosis; E, elastography; PPV, positive predictive value; NPV, negative predictive value; AUC, area under the receiver operating characteristics curve; CI, confidence interval.

Value compared to grayscale US.

Value compared to elastography.

Fig. 2.

ROC curve for US, elastography, and US-CAD.

The solid line, dashed line, and dotted line indicate the area under the ROC curve for US (0.650), elastography (0.608), and US-CAD (0.762), respectively. ROC, receiver operating characteristic; US, ultrasonography; CAD, computer-aided diagnosis.

Discussion

Various imaging methods have emerged that provide additional information in differentiating breast masses seen on grayscale US, such as elastography and US-CAD. These adjunctive imaging tools analyze different characteristics of breast masses. Whereas elastography measures the intrinsic stiffness of the targeted tissue, US-CAD analyzes the morphologic characteristics of a targeted breast mass. This is the first study to compare the diagnostic performance of these breast US imaging tools that are based on different principles, since most earlier studies have focused on the individual or combined performance of elastography or US-CAD in comparison to US [14-19].

Malignant breast masses had significantly higher rates of elastography patterns 4 and 5, as well as US-CAD assessments of possibly malignant (all P<0.001) (Table 2). Among the 120 benign breast masses, while 36 masses (30.0%) were classified as BI-RADS category 2 or 3, 49 masses (40.8%) had negative elasticity scores and 81 masses (67.5%) were assessed as probably benign on US-CAD. In addition, among the 74 benign breast masses (61.7%) classified as BI-RADS category 4a, 26 (35.1%) showed negative results on elastography and 48 (64.9%) were negative on US-CAD (Table 3). Based on our results, by applying elastography and US-CAD to breast US, we may be able to confidently improve the specificity of the diagnosis of breast masses, consistent with earlier studies showing that elastography was able to reduce the number of benign biopsies of category 4a lesions [14,20,21].

Elastography showed a higher specificity (40.8%, P=0.042) than grayscale US, although it had significantly lower sensitivity (80.8%) and NPV (77.8%, all P<0.001). These results are in agreement with those of earlier studies [22-24]. Some studies have evaluated the performance of US-CAD using US BI-RADS features when applied to biopsy-proven breast masses, showing a high AUC of at least 0.7 [18,25]. Another recent study reported that US-CAD had higher specificity than breast US in the differential diagnosis of breast masses [26]. US-CAD showed higher specificity (67.5%), PPV (61.4%), and accuracy (74.1%) than grayscale US or elastography (all P<0.001), consistent with prior reports. US-CAD analyzes the morphologic features of breast masses seen on US, and has been shown to exhibit better diagnostic performance than grayscale US performed by radiologists or US elastography of the breast. That is, the computer-based analysis of US features displayed superior performance in comparison to real-time US or the analysis of the intrinsic characteristics of breast masses. Analyzing the US features alone may have been the cause for the higher performance of US-CAD, since the radiologists who performed real-time US were not blinded to the patients’ clinical conditions or mammographic findings. Suspicious US features identified using the ACR BI-RADS criteria have very high PPVs [1,5]; therefore, analyzing breast masses by strictly considering only their US features may improve diagnostic performance. Moreover, as reported in other studies using elastography [22,24,27], information regarding the intrinsic stiffness of a mass has limitations in providing an accurate diagnosis when used in isolation, and such information should be used in conjunction with the morphologic features seen on US.

This study has some limitations. First, all evaluations were done by two radiologists, and variability between the operators was not considered. Operator dependency of elastography as well as grayscale US has been reported for image acquisition and interpretation [12,28], and this may have affected our results. Second, although a heterogeneous group of breast masses with various pathologic diagnoses was included in this study, we classified them according to the diagnostic assessment of benign or malignant, without comparing the intrinsic features of each pathology. Different levels of strain have been noted among different pathologic entities [29,30], and this may have affected our results. Third, a single radiologist chose representative images and confirmed the ROIs in the US-CAD analysis, and it is possible that another radiologist would have obtained different results. However, the ROIs were manually adjusted only when the automatic ROIs did not contain the entire mass contour in some heterogeneous masses.

In conclusion, US-CAD had higher specificity, PPV, and accuracy than grayscale US or elastography. Our results suggest that computer-based analysis based on the morphologic features found in US imaging may be very useful in improving the diagnostic performance of breast US.