Diagnostic accuracy of S-Detect to breast cancer on ultrasonography: A meta-analysis (PRISMA).

BACKGROUND: Computer-aided diagnosis (CAD) systems have shown great potential as an effective auxiliary diagnostic tool in breast imaging. Previous studies have shown that S-Detect technology has a high accuracy in the differential diagnosis of breast masses. However, the application of S-Detect in clinical practice remains controversial, and the results vary among different clinical trials. This meta-analysis aimed to determine the diagnostic accuracy of S-Detect for distinguishing between benign and malignant breast masses.
METHODS: We searched PubMed, Cochrane Library, and CBM databases from inception to April 1, 2021. Meta-analysis was conducted using STATA version 14.0 and Meta-Disc version 1.4 softwares. We calculated the summary statistics for sensitivity (Sen), specificity (Spe), positive, and negative likelihood ratio (LR+/LR-), diagnostic odds ratio(DOR), and summary receiver operating characteristic (SROC) curves. Cochran Q-statistic and I2 test were used to evaluate the potential heterogeneity between studies. Sensitivity analysis was performed to evaluate the influence of single studies on the overall estimate. We also performed meta-regression analyses to investigate potential sources of heterogeneity.
RESULTS: Eleven studies that met all the inclusion criteria were included in the meta-analysis. A total of 951 malignant and 1866 benign breast masses were assessed. All breast masses were histologically confirmed using S-Detect. The pooled Sen was 0.82 (95% confidence interval(CI) = 0.74-0.88); the pooled Spe was 0.83 (95%CI = 0.78-0.88). The pooled LR + was 4.91 (95%CI = 3.75-6.41); the pooled negative LR - was 0.21 (95%CI = 0.15-0.31). The pooled DOR of S-Detect in the diagnosis of breast nodules was 23.12 (95% CI = 14.53-36.77). The area under the SROC curve was 0.90 (SE = 0.0166). No evidence of publication bias was found (t = 0.54, P = .61).
CONCLUSIONS: Our meta-analysis indicates that S-Detect may have high diagnostic accuracy in distinguishing benign and malignant breast masses.

1. Introduction

Breast cancer has become a major threat to women’s health, and its occurrence has recently been increasing.[ Accurate identification of breast cancer and benign masses is important for improving clinical prognosis.[ Developing new diagnostic methods or improving existing diagnostic techniques is the main method to further improve the efficiency of the differential diagnosis of benign and malignant breast masses.

Many new imaging techniques have been developed, such as ultrasound elastography, contrast-enhanced ultrasound, and superb microvascular imaging, all of which have provided more convenience.[ At present, the BI-RADS classification is used as a standard method for ultrasonic imaging to evaluate breast lesions. However, owing to the subjective differences and objective errors of different doctors, the judgment of some atypical breast masses can be easily misdiagnosed.[ In particular, the differential diagnosis of benign and malignant lesions in BI-RADS 4 is still difficult.[

Computer-aided diagnosis (CAD) systems have shown great potential as an effective auxiliary diagnostic tool in breast imaging and have become a popular topic in artificial intelligence and modern medical research.[ Ultrasonic S-Detect(Samsung Medison Co. Ltd., Seoul, South Korea) technology is a computer-aided diagnosis technology that uses a convolutional neural network deep learning algorithm to evaluate breast nodules according to the BI-RADS dictionary and has become one of the most increasingly used CAD systems for the diagnosis of breast cancer.[ The deep learning model is used to automatically detect and analyze the boundary, shape, internal echo, and other nodule information, overcome the interference of human factors, and objectively judge benign and malignant breast nodules.[ Previous studies have shown that S-Detect technology has a high accuracy in the differential diagnosis of breast masses. However, as a new technique, the application of S-Detect in clinical practice remains controversial, and the results vary among different clinical trials. At present, the Spe of the results varies greatly among studies, and there is no meta-analysis or guidance on this technique for the diagnosis of breast cancer. Therefore, the present meta-analysis aimed to determine the accuracy of S-Detect for the differential diagnosis of benign and malignant breast masses.

2. Methods

This study was conducted in accordance with the PRISMA (Preferred Reporting Items for Systematic Reviews and MetaAnalyses) guidelines, the meta-analysis was not registered.

3. Ethics and dissemination

Ethical documents will not be obtained because this study will be conducted based on data from the published literature. We expect that this study will be published in a peer-reviewed journal.

4. Literature search

We searched PubMed, Cochrane Library, and CBM databases from inception to April 1, 2021. The following keywords and MeSH terms were used: [“breast cancer” or “breast neoplasm” or “breast tumor” or “breast nodule “] and [“S-Detect” or “smart detect” or “artificial Intelligence” or “computer aid diagnosis”}. We also performed a manual search to identify additional relevant articles.

5. Selection criteria

The following 4 criteria were required for each study: (1) the study design must be a clinical cohort study or diagnostic test; (2) the study must relate to the accuracy of S-Detect for the differential diagnosis of benign and malignant breast masses, and the final assessments from S-Detect were dichotomized as possibly benign and possibly malignant; (3) all breast masses were histologically confirmed; and(4) published data in the fourfold (2 × 2) tables must be sufficient. If the study did not meet all the inclusion criteria, it was excluded. The most recent publication with the largest sample size was included when the authors published several studies using the same subjects.

6. Data extraction

Relevant data were systematically extracted from all included studies by 2 researchers using a standardized form. The researchers collected the following data: first author’s surname, publication year, language of publication, study design, sample size, number of lesions, source of subjects, and “gold standard”. True positives (TP), true negatives (TN), false positives (FP), and false negatives (FN) in the fourfold (2 × 2) table were also collected.

7. Quality assessment

Methodological quality was independently assessed by 2 researchers using the Quality Assessment of Studies of Diagnostic Accuracy Studies (QUADAS) tool.[ The QUADAS criteria include 14 assessment items. Each item is scored as “yes” (2), “no” (0), or “unclear”(1). The QUADAS score ranged from 0 to 28, and a score ≥ 22 indicated good quality.

8. Statistical analysis

STATA version 14.0 (Stata Corp, College Station, TX, USA) and Meta-Disc version 1.4 (Universidad Complutense, Madrid, Spain) software were used for the meta-analysis. We calculated the pooled summary statistics for sensitivity (Sen), specificity (Spe), positive and negative likelihood ratios (LR+/LR−), and diagnostic odds ratios (DOR) with 95%confidence intervals (CIs). posttest probabilities were calculated using LR + and LR − and plotted on a Fagan nomogram. A summary receiver operating characteristic (SROC) curve and corresponding area under the curve (AUC) were obtained. The threshold effect was assessed using Spearman correlation coefficient. Cochran Q-statistic and I2 test were used to evaluate the potential heterogeneity between studies. If significant heterogeneity was detected(Q test P < .05, I test > 50%), a random-effects model or fixed-effects model was used. We also performed meta-regression analyses to investigate potential sources of heterogeneity. Sensitivity analysis was performed to evaluate the influence of single studies on the overall estimate. We used Begger funnel plot and Egger linear regression test to investigate publication bias.

9. Results

9.1. Characteristics of included studies

Initially, search keywords were used to identify 40 articles. We reviewed the titles and abstracts of all articles and excluded 17 articles. Full texts and data integrity were also reviewed, and 12 articles were further excluded. Finally, 11 studies that met all inclusion criteria were included in this meta-analysis.[ Figure 1 shows the selection process for eligible articles. In total, 1118 malignant and 1595 benign breast nodules were assessed. The study characteristics and methodological quality in Table 1. The QUADAS scores of all included studies were 22.

Figure 1.

Flow chart of literature search and study selection. 11 studies were included in this meta-analysis.

Table 1

Baseline characteristics and methodological quality of all included studies.

First author	Year	Country	Language	Sample size	Age (years)	Instrument	S-Detect 2 × 2 table				QUADAS score
							TP	FP	FN	TN
Xia Q[12]	2021	China	English	40	50.9 ± 13.9	Samsung RS80A	23	1	1	15	22
Kim K[13]	2017	Korea	English	192	46.6 ± 13.3	Samsung RS80A	57	41	15	79	25
Zhou YG[14]	2017	China	Chinese	61	46.5 ± 12.8	Samsung RS80A	16	3	9	33	23
Segni MD[15]	2018	Italy	English	68	21–84	Samsung RS80A	40	7	4	17	24
ChoE[16]	2017	Korea	English	119	48.5 ± 12.2	Samsung RS80A	39	6	15	59	25
ChoiJH[17]	2018	Korea	English	200	49.5 ± 11.8	Samsung RS80A	8	41	4	147	24
Cheng HF[18]	2019	China	Chinese	468	43.3 ± 12.6	Samsung RS80A	145	51	10	262	25
Zhao CY[19]	2022	China	English	757	15–82	Samsung RS80A	273	118	24	342	26
Yan H[20]	2020	China	Chinese	581	43.4 ± 12.2	Samsung RS80A	109	36	61	375	25
Pan JZ[21]	2021	China	Chinese	175	46.6 ± 13.9	Samsung RS80A	70	13	18	74	23
Kim MY[22]	2021	Korea	English	156	46 ± 10	Samsung RS85A	7	22	3	124	27

FN = false negative, FP = false positive, QUADAS = the quality assessment of studies of diagnostic accuracy studies, TN = true negative, TP = true positive.

10. Quantitative data synthesis

A random-effects model was used because there was obvious heterogeneity among the studies. Sensitivity analysis was performed, and none of these caused obvious interference in the results of this meta-analysis(Fig. 2). The pooled Sen was 0.82 (95%CI = 0.74–0.88); the pooled Spe was 0.83 (95%CI = 0.78–0.88)(Fig. 3). There was no significant correlation (R = 0.209, P = .507) between the sensitivity and specificity, indicating that there was no threshold effect. The pooled LR + was 4.91 (95%CI = 3.75–6.41); the pooled negative LR − was 0.21 (95%CI = 0.15–0.31)(Fig. 4). The pooled DOR of S-Detect for the diagnosis of breast masses was 23.12 (95% CI = 14.53–36.77)(Fig. 5). The area under the SROC curve was 0.90 (SE = 0.0166)(Fig. 6). Meta-regression analysis confirmed that none of the factors could explain the potential sources of heterogeneity(Table 2). No evidence of publication bias was observed (Fig. 7). Egger test did not display strong statistical evidence of publication bias (t = 0.54, P = .61). The analysis of the Fagan plot showed that when the pretest probabilities were 25%, 50%, and 75%, the positive posttest probabilities were 62%,83% and 94%, respectively, whereas the negative posttest probabilities were 7%, 18%, and 39%, respectively (Fig. 8).

Figure 2.

Sensitivity analysis. None of them caused obvious interference to the results.

Figure 3.

Forest plots for the sensitivity and specificity of S-Detect for the diagnosis of benign masses.

Figure 4.

Forest plots for the positive and negative likelihood ratio of S-Detect for the diagnosis of benign masses.

Figure 5.

Forest plot of DOR of S-Detect for the diagnosis of benign masses. DOR = diagnostic odds ratio.

Figure 6.

SROC curve for the accuracy of S-Detect in the diagnosis of benign masses. SROC = summary receiver operator characteristic, AUC = area under curve.

Table 2

Meta-regression analyses of potential source of heterogeneity.

Heterogeneity factors	Coefficient	SE	P value	RDOR	95% CI
					UL	LL
Publication year	0.037	0.2298	0.8799	1.04	0.57	1.87
Language	0.269	0.8102	0.7537	0.76	0.10	6.12
Instrument	0.052	1.3506	0.9705	0.95	0.03	30.55
Country	0.449	0.7891	0.5937	0.64	0.08	4.85

LL = lower limit, RDOR = relative diagnostic odds ratio, 95% CI = 95 % confidence interval, SE = standard error, UL = upper limit.

Figure 7.

Begger funnel plot of publication bias on the pooled OR. No publication bias was detected in this meta-analysis.

Figure 8.

Fagan plot analysis for S-Detect in detecting benign masses: (a) pretest probability at 25%; (b) pretest probability at 50%; (c) pretest probability at 75%. The Fagan plot is composed of the left vertical axis representing the pretest probability, the middle vertical axis representing the likelihood ratio, and the right vertical axis representing the posttest probability.

11. Discussion

In recent years, many new breast diagnosis technologies have emerged to assist ultrasound doctors in diagnosis, improve the coincidence rate of diagnosis, and achieve early detection, diagnosis, and treatment.[ However, in the application of new technology, we should understand the influencing factors of the technology itself in order to really achieve the purpose of improving the accuracy. High-resolution ultrasonography plays an important role in the differential diagnosis of breast masses.[ The growing incidence of breast masses also increases the burden on radiologists in diagnosing breast cancers based on ultrasound (US) imaging, which outperforms other imaging modalities in diagnosing breast masses. In recent years, artificial intelligence (AI) has been developed. A new CADs for ultrasound imaging, also known as “S-Detect,” has been recently introduced to improve breast US interpretation and provide assistance in the morphological analysis of breast masses.[ S-Detect is software based on morphological image analysis. It extracts local features of an image to obtain global features.[ The features of malignant breast masses include shape, direction, edge, rear features, and echo pattern, which are more characteristic and easier to identify for the system.[ However, only a few articles have reported the diagnostic performance of S-Detect for breast masses, most of which were published by Korean researchers. To further study the diagnostic value of S-detect in breast ultrasound, more validation sets from different countries are required. Therefore, this study aimed to provide a comprehensive and reliable conclusion regarding the diagnostic accuracy of S-Detect for breast tumors.

In the present meta-analysis, we systematically evaluated the technical performance and accuracy of S-Detect for the differential diagnosis of benign and malignant breast masses. The pooled Sen, Spe, and DOR values of S-Detect for the diagnosis of breast nodules were 0.82, 0.83, and 23.12, respectively. These results are consistent with the potentially high diagnostic accuracy of S-Detect for benign masses, suggesting that S-Detect may be a good tool for the differential diagnosis of benign and malignant benign masses and could predict the prognosis of patients with breast nodules. The threshold effect is usually interpreted as a sudden and radical change in a phenomenon that often occurs after surpassing a quantitative limit. Our findings showed no significant relationship between Sen and Spe in these studies, thus providing no evidence of a threshold effect. Furthermore, our results showed no direct evidence of publication bias. Collectively, our findings strongly suggest that S-Detect is a highly accurate and noninvasive tool for the qualitative diagnosis of benign masses, which is consistent with previous studies.

Despite the demonstrated diagnostic accuracy of S-Detect for benign masses, our study had certain limitations. First, owing to the relatively small sample sizes and low quality of the included studies, there were insufficient data to assess the accuracy of S-Detect. Moreover, the inclusion of studies with only histological confirmation and the retrospective nature of the meta-analysis could have led to subject selection bias. Importantly, the majority of the included studies originated in Asia, which may adversely affect the reliability and validity of our results.

In conclusion, our meta-analysis suggests that S-Detect may have high diagnostic accuracy in distinguishing benign and malignant breast masses. It can be used as a supplement to conventional ultrasonography. However, owing to these limitations, further detailed studies are required to confirm the present findings.

Author contributions

Data curation: Xiaolei Wang.

Investigation: Xiaolei Wang.

Writing – review & editing: Shuang Meng.