Validation of Breast Cancer Models for Predicting the Nonsentinel Lymph Node Metastasis After a Positive Sentinel Lymph Node Biopsy in a Chinese Population.

OBJECTIVES: Over the years, completion axillary lymph node dissection is recommended for the patients with breast cancer if sentinel lymph node metastasis is found. However, not all of these patients had nonsentinel lymph node metastasis on final histology. Some predicting models have been developed for calculating the risk of nonsentinel lymph node metastasis. The aim of our study was to validate some of the predicting models in a Chinese population.
METHOD: Two hundred thirty-six patients with positive sentinel lymph node and complete axillary lymph node dissection were included. Patients were applied to 6 models for evaluation of the risk of nonsentinel lymph node involvement. The receiver-operating characteristic curves were shown in our study. The calculation of area under the curves and false negative rate was done for each model to assess the discriminative power of the models.
RESULTS: There are 105 (44.5%) patients who had metastatic nonsentinel lymph node(s) in our population. Primary tumor size, the number of metastatic sentinel lymph node, and the proportion of metastatic sentinel lymph nodes/total sentinel lymph nodes were identified as the independent predictors of nonsentinel lymph node metastasis. The Seoul National University Hospital and Louisville scoring system outperformed the others, with area under the curves of 0.706 and 0.702, respectively. The area under the curve values were 0.677, 0.673, 0.432, and 0.674 for the Memorial Sloan-Kettering Cancer Center, Tenon, Stanford, and Shanghai Cancer Hospital models, respectively. With adjusted cutoff points, the Louisville scoring system outperformed the others by classifying 26.51% of patients with breast cancer to the low-risk group.
CONCLUSION: The Louisville and Seoul National University Hospital scoring system were found to be more predictive among the 6 models when applied to the Chinese patients with breast cancer in our database. Models developed at other institutions should be used cautiously for decision-making regarding complete axillary lymph node dissection after a positive biopsy in sentinel lymph node.

Introduction

Assessing the axillary lymph node (ALN) status is still an important part of surgery in patients with breast cancer, because it is thought to be one of the most important prognostic factors.[1] Axillary lymph node dissection (ALND) was performed for the staging of axilla in almost all patients with breast cancer till the late 1990s.[2] After the early 1990s, the introduction of the sentinel lymph node (SLN) biopsy helps to achieve a lower morbidity than ALND.[3-5] Sentinel lymph node biopsy can accurately stage the axilla in patients with early breast cancer, and it is widely accepted as a standard approach.[6-8] A complete ALND is recommended if a metastatic SLN is found in patients with breast cancer.[9] However, 30% to 70% of these patients didn’t have non-SLN metastasis on final histology.[10-15] Therefore, the identification of patients who do not have metastatic non-SLN when SLN is positive becomes a problem demanding prompt solution.

Some clinicopathologic features of the primary breast cancer tumor and metastatic SLN are identified as factors that may predict the non-SLNs metastases risk, such as tumor size, lymphovascular invasion, and size of SLN metastasis.[10,15,16] A number of predicting models, including nomograms and scoring systems, have been developed, combining some statistically significant factors.[11-13,17-19] But how well these nomograms/scoring systems will perform in our Chinese patients with breast cancer is still unknown. The aim of this article was to validate several nomograms/scoring systems in a Chinese breast cancer population with positive SLNs. The 6 predicting models that we used are listed as follows: (1) the Memorial Sloan-Kettering Cancer Center (MSKCC) nomogram,[17] (2) the Tenon scoring system,[18] (3) the Louisville scoring system,[13] (4) the Seoul National University Hospital (SNUH) scoring system,[19] (5) the Stanford nomogram,[12] and (6) the Shanghai Cancer Hospital (SCH) nomogram.[11]

Materials and Methods

Patients

From September 2010 to September 2016, data on 236 patients with breast cancer were included at the Guangdong General Hospital (Guangzhou, China). This retrospective study was approved by the institutional ethics committee of Guangdong General Hospital, and requirement for informed consent was waived. The inclusion criteria were (1) no systemic treatment (such as neoadjuvant chemotherapy) before SLN biopsy, (2) identification of metastatic SLN(s), and (3) complete clinical and histological data.

Surgery and SLN Histopathological Evaluation

All patients underwent SLN biopsy, and an SLN is defined as the first lymph node which receives drainage from the primary breast cancer. The patients received a subareolar intradermal injection in 4 parts of the periareola of 2 mL of patent blue, and then 3 minutes of breast massage were done. Sentinel lymph node was any blue-stained node following a blue lymphatic channel. Lymph nodes were marked as sentinel if they were stained blue, otherwise they were marked as non-SLNs. If metastasis in SLN was identified by frozen section (FS), hematoxylin and eosin (H&E) staining, or immunohistochemistry, then the surgery of ALND was carried out. According to the sixth Edition of American Joint Committee on Cancer (AJCC), SLNs metastases were classified into isolated tumor cells or clusters (isolated tumor cell [ITC], ≤0.2 mm), micrometastasis (>0.2, ≤2 mm), and macrometastasis (>2 mm); if there are more than 1 metastatic lymph nodes, then the maximum diameter was recorded. Non-SLNs obtained during ALND were totally submitted immediately, then sectioned serially, and stained with H&E according to the standard procedure introduced by European Institute of Oncology.[20]

Statistical Analysis

Twenty clinicopathological features were studied individually by the presence or absence of metastatic non-SLN: age, ultrasonography result of the axilla, tumor size, operation method, location of primary tumor, multifocality of primary tumor, histological type of primary tumor, histological grade of primary tumor, lymphovascular invasion, estrogen receptor(ER) status, progesterone receptor(PR) status, human epidermal growth factor receptor 2 (HER2) status, Ki 67 status, metastasis detecting method, the number of SLNs excised, the number of metastatic SLNs, the number of negative SLN, proportion of metastatic SLN/total SLN, and size of SLN metastasis and extracapsular extension.

For univariate analysis of clinicopathologic variables for non-SLN metastasis, χ2 test was used for categorical variables, and Kruskal-Wallis rank-sum test was used for ordinal variables. Multivariate analyses were performed using binary logistic regression multivariate analysis to identify the correlated clinicopathologic variables with the non-SLN positivity. P Values were also calculated. The probability of non-SLNs metastases was calculated using the 6 predicting models which we had mentioned before. With an online calculator for MSKCC nomogram which is available at http://nomograms.mskcc.org/breast/BreastAdditionalNonSLNMetastasisPage.aspx, we calculated the risk of non-SLN metastasis for the patients with breast cancer in our database. To calculate the probability for Stanford nomogram, a method of boosted classification and regression trees (CART) were used (available at: http://www.salford-systems.com/cart.php). Other probability was calculated based on the primary literatures.[11,13,18,19]

The mean predicted probabilities of the 6 models predicting non-SLN metastasis in our Chinese patients with breast cancer were compared. The receiver–operating characteristic (ROC) curves were drawn in figure. The calculations of the areas under the ROC curve (AUC) were done for each model. The discrimination probability of each model was quantified with AUC. The 95% confidence intervals (CIs) of AUC values were also calculated for each model. The AUC value ranges from 0 to 1, and it is generally accepted that a considerable discrimination values of AUC are between 0.7 and 0.8; AUC values exceeding 0.8 represent good discrimination.[21]

With the P value <.05 as a cutoff point, the statistical analysis was done. The version 3.3.2 of R software was used (available at: https://cran.r-project.org/).

Results

Clinicopathologic Features and Results of Univariate and Multivariate Analyses

The clinicopathologic features of the 236 patients with breast cancer included in our study are listed (Table 1). The mean age of these patients was 48.37 years (range, 24-77 years). Mean size of the primary tumor was 2.57 cm (range, 0.6-7.0 cm). Among these patients, 105 (44.5%) patients had at least 1 metastatic non-SLN. The mean number of excised SLN was 3.93 (range, 1-16 nodes) and metastatic SLN 1.79 (range, 1-12 nodes). And the mean number of dissected and metastatic non-SLN was 16.93 (range, 1-50 nodes) and 2.50 (range, 0-42 nodes). After the univariate analysis, the parameters that were identified as statistically significant were as follows: primary tumor size, histological grade of primary tumor, lymphovascular invasion, the number of metastatic SLN, the number of negative SLN, and the proportion of metastatic SLNs/total SLNs (P < .05).

Table 1.

Clinicopathologic Features and Univariate Analysis by the Presence or Absence of Metastatic Non-SLN in Our Chinese Patients With Breast Cancer.a

Characteristics	Non-SLN Metastasis; Present, n = 105	Non-SLN Metastasis; Absent, n = 131	Significance, P Value
Age, years, n (%)			.67
≤50	67 (63.8%)	80 (61.1%)
>50	38 (36.2%)	51 (38.9%)
Axillary ultrasonography, n (%)			.087
Low suspicion of LN metastasis	62 (59.0%)	85 (64.9%)
High suspicion of LN metastasis	21 (20.0%)	13 (9.9%)
Not done	22 (21.0%)	33 (25.2%)
Primary tumor size, n (%)			.0015
T1	42 (40.0%)	73 (55.7%)
T2	51 (48.6%)	56 (42.7%)
T3	12 (11.4%)	2 (1.5%)
Operation, n (%)			.24
Conservative	10 (9.5%)	19 (14.5%)
Mastectomy	95 (90.5%)	112 (85.5%)
Location of primary tumor, n (%)			.78
Upper outer	33 (31.4%)	41 (31.3%)
Upper inner	12 (11.4%)	18 (13.7%)
Lower outer	17 (16.2%)	14 (10.7%)
Lower inner	7 (6.7%)	10 (7.6%)
Central or 2 quadrant	36 (34.4%)	48 (36.6%)
Multifocality of primary tumor, n (%)			.65
Yes	8 (7.6%)	8 (6.1%)
No	97 (92.4%)	123 (93.9%)
Histological type of primary tumor, n (%)			.64
Invasive ductal carcinoma	94 (89.5%)	117 (89.3%)
Invasive lobular carcinoma	6 (5.7%)	5 (3.8%)
Other type	5 (4.8%)	9 (6.9%)
Histological grade of primary tumor, n (%)			.05
Ductal, I	2 (1.9%)	8 (6.1%)
Ductal, II	41 (39.0%)	64 (48.9%)
Ductal, III	49 (46.7%)	44 (33.6%)
Unclear	13 (12.4%)	15 (11.5%)
Lymphovascular invasion, n (%)			<.001
Present	50 (47.6%)	34 (26.0%)
Absent	55 (52.4%)	97 (74.0%)
ER status, n (%)			.49
Positive	78 (77.2%)	98 (81.0%)
Negative	23 (22.8%)	23 (19.0%)
Unclear	4 (3.8%)	10 (7.6%)
PR status, n (%)			.46
Positive	75 (74.3%)	95 (78.5%)
Negative	26 (25.7%)	26 (21.5%)
Unclear	4 (3.8%)	10 (7.6%)
Her-2 status, n (%)			.19
Positive	30 (29.1%)	30 (24.6%)
Intermediate	4 (3.9%)	1 (0.8%)
Negative	69 (67.0%)	92 (75.4%)
Unclear	2 (1.9%)	8 (6.1%)
Ki 67, n (%)			.35
Low	23 (22.1%)	36 (27.5%)
High	81 (77.9%)	95 (72.5%)
Unclear	1 (1.0%)	0
Method of detection, n (%)			.25
FS	70 (66.7%)	95 (72.5%)
Routine H&E	12 (11.4%)	18 (13.7%)
Serial H&E	19 (18.1%)	17 (13.0%)
IHC	4 (3.8%)	1 (0.8%)
Sentinel lymph node features
No of SLNs excised, n (%)			.50
1	18 (17.1%)	16 (12.2%)
2	17 (16.2%)	28 (21.4%)
3	19 (18.1%)	16 (12.2%)
4	21 (20.0%)	19 (14.5%)
≥5	30 (28.6%)	42 (32.1%)
Number of metastatic SLN, n (%)			<.001
1	50 (47.6%)	95 (72.5%)
2	27 (26.0%)	23 (17.6%)
3	13 (12.5%)	12 (9.2%)
≥4	15 (14.3%)	1 (0.8%)
Number of nonmetastatic SLN, n (%)			.005
0	38 (36.2%)	23 (17.6%)
1	17 (16.2%)	34 (26.0%)
2	19 (18.1%)	21 (16.0%)
3	16 (15.2%)	18 (13.7%)
≥4	15 (14.3%)	35 (26.7%)
Proportion of metastatic SLN/total SLN, n (%)			.0035
<0.5	35 (33.3%)	64 (48.9%)
Between 0.5 and 1	32 (30.5%)	44 (33.6%)
1	38 (36.2%)	23 (17.6%)
Size of SLN metastasis			.30
ITC	2 (1.9%)	1 (0.8%)
Micrometastasis	2 (1.9%)	7 (5.3%)
Macrometastasis	101 (96.2%)	123 (93.9%)
Extracapsular extension			.36
Present	5 (4.8%)	10 (7.6%)
Absent	100 (95.2%)	121 (92.4%)

Abbreviations: ER, estrogen receptor; FS, frozen section; H&E, hematoxylin and eosin; HER-2, human epidermal growth factor receptor 2; ITC, isolated tumor cell; LN, lymph node; PR, progesterone receptor; SLN, sentinel lymph node.

a n = 236.

After some of the variables were found to be significant (P < .05) in the univariate analysis, the binary logistic regression multivariate analysis was performed. And the results of the multivariate analysis were summarized and shown (Table 2). In the multivariate analysis, only 3 variables were identified as independent predictors of non-SLN metastasis: primary tumor size, the number of metastatic SLN, and the proportion of metastatic SLNs/total SLNs (Table 2).

Table 2.

Results of the Multivariate Analysis of the Risk of Non-SLN Metastasis.

Variable	Coefficient	SE	Wald	P	OR	95% CI
						Lower	Upper
Primary tumor size (versus T1)			7.912	.019
T2	0.421	0.298	1.997	.158	1.523	0.850	2.732
T3	2.191	0.827	7.022	.008	8.942	1.769	45.206
Number of metastatic SLN	0.428	0.147	8.497	.004	1.534	1.151	2.046
Proportion of metastatic SLN/total SLN	1.583	0.487	10.588	.001	4.871	1.877	12.640
Constant	−2.208	0.393	31.528	.000	0.110

Abbreviations: CI, confidence interval; OR, odds ratio; SLN, sentinel lymph node.

Performance of the Models Applied to the Chinese Patients With Breast Cancer in Our Database

Upon considering the missing variables, the number of patients applied to the MSKCC, Tenon, Louisville, SNUH, Stanford, and the SCH model was 194, 227, 230, 180, 236, and 227, respectively (Table 3). The ROC curves of the different models were plotted (Figure 1), and AUCs were listed (Table 3). The AUC of the SNUH and Louisville model was 0.706 and 0.702, respectively, which is considered a good discriminator. The MSKCC, Tenon, and SCH models had AUCs of 0.677, 0.673, and 0.674, respectively. However, the AUC value of the Stanford model was only 0.432 when applied to our patients, suggesting no better than chance (0.50).

Table 3.

The AUCs and FNRs in the 6 Models for Predicting Non-SLNs Metastases in Our Patients With Breast Cancer.

Model	MSKCC	Tenon	Louisville	SNUH	Stanford	SCH
Type	Nomogram	Score (0-7)	Score (0-6)	Score	Nomogram	Nomogram
Patients applied, n	194	227	230	180	236	227
Patients with non-SLNs (+), n	86	101	101	101	105	101
AUC (95% CI)	0.677 (0.601-0.752)	0.673 (0.602-0.743)	0.702 (0.634-0.769)	0.706 (0.630-0.781)	0.432 (0.351-0.477)	0.674 (0.590-0.732)
OCP	P ≤ 10.0%	Score ≤ 3.5	Score ≤ 1.0	Score ≤ 1.5	–	P ≤ 10.0%
Patients under OCP, n	12 (6.19%)	54 (23.79%)	0 (0%)	83 (46.11%)	–	0 (0%)
Patients under OCP with non-SLNs (+), n	2	14	0	30	–	0
FNRa	2.33%	13.86%	0.00%	29.70%	–	0.00%
ACP	P≤20%	score≤3.5	score≤2.0	score≤0.63	–	P≤30.0%
Patients under ACP, n	30 (15.46%)	54 (23.79%)	61 (26.51%)	45 (25.00%)	–	40 (17.62%)
Patients under ACP with non-SLNs (+), n	7	14	13	9	–	10
Adjusted FNRb	8.14%	13.86%	12.87%	8.91%	–	9.90%

Abbreviations: ACP, adjusted cut-off point; AUC, area under the curve; CI, confidence interval; FNR: false negative rate; MSKCC, Memorial Sloan-Kettering Cancer Center; Non-SLNs (+): positive non-SLNs; OCP, original cut-off point; SCH, Shanghai Cancer Hospital; SNUH, Seoul National University Hospital; SLN, SLN, sentinel lymph node.

a FNR = patients with metastatic non-SLNs under OCP/(patients with metastatic non-SLNs in total).

b Adjusted FNR = patients with metastatic non-SLNs under ACP/(patients with metastatic non-SLNs in total).

Figure 1.

The ROCs and AUCs of the six models (MSKCC, Tenon, Louisville, SNUH, Stanford, and SCH model) in our study. AUC denotes area under the curve; MSKCC, Memorial Sloan-Kettering Cancer Center; ROC, receiver-operating characteristic; SCH, Shanghai Cancer Hospital; SNUH, Seoul National University Hospital.

For clinical utility, the ability to classify patients into low-risk group of metastatic non-SLN and false negative rates (FNRs) were compared (Table 3). With the original cutoff points, the SNUH, Tenon, and MSKCC model assigned 46.11%, 23.79%, and 6.19% patients with breast cancer into the low-risk group of metastatic non-SLN, respectively.

As the FNR of ALND when assessing the non-SLN metastasis is close to 5%, this rate is widely accepted as a target value of the predicting models. When applied to our patients, only Louisville scoring system (0%), SCH scoring system (0%), and the MSKCC nomogram (2.33%) have an FNR <5% using the original cutoff points for each model. Two models have an FNR >10%: 13.86% (14 of 101) for Tenon score and 29.70% (30 of 101) for the SNUH score (Table 3), indicating that these 2 models may not be suitable for the Chinese patients with breast cancer in our database. The FNR was not calculated for the Stanford nomogram because this nomogram did not show any discriminative ability (AUC < 0.5).

Sentinel lymph node biopsy has an inherent FNR of 5% to 10%.[6-8] Although the FNR of SLN biopsy is higher than that of ALND, the clinical significance of this difference is diminished by the frequent use of adjuvant systemic therapy in node-negative disease.[7] Given the selection of lower-risk patients (with FNR up to 10%) for SLNB, the rate of axillary recurrence following a negative SLNB is very low. This rate has been reported to be less than that in the population of women undergoing ALND.[7-8] Therefore, although the researchers set a target FNR of 5% when building their models, adjustment to 10% is clinically acceptable when applying the model, which is consistent with the highest FNR of SLNB. False negative rate at 5% and 10% were reported at the same time in some literatures.[11,19] When the FNR for each model were adjusted close to 10%, the Louisville score (26.51%) and SNUH score (25.00%) outperformed the others in assigning patients to the low-risk group, compared to the SCH (17.62%) and MSKCC (15.46%).

Discussion

In patients with breast cancer, the status of ALN is thought to be the most important prognostic factor.[6,22] In order to offer more prognostic information, the ALND has become a standard staging procedure, but it remains controversial how it will benefit the breast cancer cure.[3] As the breast cancer surgery becomes more conservative, SLN biopsy, a minimally invasive way, has gradually replaced the routine ALND for SLN staging.[7-9] As a revolution of the breast cancer surgery, SLN biopsy helps patients with the absence of non-SLNs metastases to avoid ALND.[1,8] However, complete ALND is still the gold standard treatment when metastases are found in SLNs. Many have questioned the need for complete ALND in patients with breast cancer with positive SLN(s). There are a number of studies which show that the only metastatic site is in SLN(s) in almost 30% to 70% of patients with metastatic SLNs[10-15] Among the SLN-positive patients in our database, only 44.5% (105/236) of them had further metastasis in non-SLN, which is similar to some results of other investigators[14,11]

There is increasing interest to figure out what factors may predict the risk of non-SLN metastasis after a positive SLN biospy. Many studies have reported some of these risk factors through their research.[11,13,15,17-19] These predictive factors include 2 kinds of characteristics: primary tumor characteristics and metastatic SLN characteristics, such as detecting method of SLN metastasis,[17] tumor size,[14,17-19] lymphovascular invasion,[12,19,23] metastasis size of SLN,[12,22,24] extracapsular extension,[14,25] the number of metastatic SLNs,[13,19] the number of non-metastatic SLNs,[19,23] and the proportion of positive SLNs[18,24,25] Primary tumor size, the number of metastatic SLN, and the proportion of metastatic SLNs/total SLNs were identified as independent predicting factors for the risk of non-SLN metastasis in our study, which is similar to some of the results of others [12,13,18,19,25,26] but not all.[11,23] By combining different clinicopathological prognostic factors, some predicting models have been developed, allowing to assessing the risk of non-SLN involvement. In our study, we evaluated and compared the performance of 6 predicting models and they do not perform equally well (Table 3).

The MSKCC nomogram, published in 2003, is the most famous and widely validated predicting model in this field.[17] The MSKCC nomogram was tested by many authors, with an AUC varying from 0.58 to 0.84[10-12,14,15,19,24-28] When applied to 194 Chinese patients in our database, the AUC of this model was 0.677, compared to the original result of 0.75. This result suggests that the MSKCC model did not achieve an acceptable discriminative value of AUC in the Chinese patients with breast cancer from our database. A potential weakness of this model might be using the method of detecting SLNs metastases to replace the true size of SLNs metastases, because the detecting methods of SLN metastases vary considerably among different institutions.[12,14,24-26]

The Tenon scoring system was a predicting model published in 2005. Only 3 variables were used to build the model: the metastasis size of SLN, pathologic tumor size, and the proportion of metastatic SLNs/total SLNs.[18] The AUC of the Tenon scoring system was 0.673 in our study. In the Tenon study, 49.3% of the patients were found to have micrometastasis in SLN, while the percentage of patients in our study who had ITC, micrometastasis, and macrometastasis in SLNs are 1.3%, 3.8%, and 94.9%, respectively. The percentage of the population with primary tumor size larger than 20 mm was 25.4% in Tenon study, compared to 51.3% in our study. Furthermore, the rate (25.4%) of the proportion of metastatic SLNs/total SLNs < .5 in the study is lower than that (41.9%) in our study. The differences appeared in the micrometastasis rate of SLNs, T1 tumors, and the proportion of the metastatic SLNs, which might influence the results of AUC between different studies.

The Louisville scoring system was published in 2006, with patients from the United States and Canada.[13] The 3 clinicopathologic parameters that were found to be significant after the multivariate analysis in our study were totally the same as the Louisville study: primary tumor size, the number of metastatic SLN, and the proportion of metastatic SLN/total SLN. The AUC of the Louisville scoring system was 0.702 in our study, compared to the original Louisville result of 0.680. As for clinical utility, the FNR was 0% (0 of 101) for the Louisville scoring system, which is thought to be good for clinical utility. What’s more, when the FNRs for each model were adjusted close to 10%, the Louisville model assigned 26.51% of patients into the low-risk group, which outperformed the others.

In 2008, the SNUH scoring system [19] was developed for predicting non-SLN status, with a Korean population. The parameters included the result of ultrasound, lymphovascular invasion, tumor stage, the number of positive SLN, and negative SLN. The original SNUH AUC were both higher than 0.80 in the training and validation data set, compared to 0.706 in our study. The clinicopathological features of the Chinese patients in our database were similar to that in the SNUH study, except for lymphovascular invasion. Lymphovascular invasion was absent in 32.6% of the patients with SNUH, while 62.2% (112 of 180) of our patients didn’t had lymphovascular invasion. The AUC of the SNUH scoring system was the highest one among the 6 models that we validated, and the SNUH scoring system assigned 46.11% of our individuals into the low-risk group, but the FNR of this model was also the highest one (29.70%). After adjusting FNR to 8.91% (close to 10%), this model assigned 25.00% of our individuals into the low-risk group, similar to the result of the Louisville score system (26.51%).

The Stanford nomogram was developed in 2008 with the patients with breast cancer in Bay Area data set (modeling data set).[12] According to the Stanford study, the original AUC value was 0.83 in the modeling data set, which is considered good. However, the Stanford nomogram had an AUC value of 0.432 in our study, which is no better than chance (0.50). There is no lymphovascular invasion in 40.2% of patients in the modeling data set of the Stanford study, whereas 64.4% of our patients didn’t have the lymphovascular invasion. Furthermore, the percentage of patients in the modeling data set of Stanford nomogram who had ITC, micrometastasis, and macrometastasis was 20.6%, 71.3%, and 8.0%, respectively. And the percentage (3.8%) of patients who had micrometastatic SLN(s) in our study was significantly lower than that (20.6%) in the Stanford modeling data set. The differences between the Stanford nomogram and our study, especially in the percentage of lymphovascular invasion and micrometastatic SLN(s), might be the reasons of bias when applied to our patients. Furthermore, the Stanford model was built with tree method (CART), and the size of SLN metastasis played a key role in determining tree-based model branching. The proportion of size of SLN metastasis had a strong impact on the accuracy of the model when applied to our patients, thus the result of calculation revealed a low AUC value.

The SCH nomogram [11] was developed by Chen et al in 2012, using a Chinese breast cancer population. And this is the validating study of this Chinese nomogram, using a Chinese population well. Although the population that we used was the same as the SCH nomogram, a Chinese population, this model didn’t outperform the others. The original AUC was 0.7105, compared to 0.674 (95% CI 0.590-0.732) in our study. The SCH nomogram included 5 variables: age, primary tumor size, location of the tumor, histological type of the primary tumor, and lymphovascular invasion. According to results of many researches, lymphovascular invasion is identified as a significant predictor for non-SLN metastasis.[11,12,17,19,23,26] But among the models predicting the non-SLN status for patients with breast cancer, hardly any of them included age and tumor location as independent predictors. In our study, age, location of the primary tumor, histological type of the primary tumor, and lymphovascular invasion found no significance in predicting the non-SLN status. The clinicopathologic difference between the 2 studies might explain the low AUC of the SCH nomogram in our study although using a same population. The FNR of the SCH nomogram was 0% (0 of 101). When the FNR was adjusted 9.90% (close to 10%), the SCH model only assigned 17.62% of our individuals into the low risk group, suggesting that this model didn’t perform well in clinical use.

As Chen et al reported,[29] breast cancer has become the leading cause of cancer death in Chinese women younger than 45 years old, and the estimated incidence of breast cancer was 272.4 thousand in total with mortality rate of 70.7 thousand in total. Thus, it is important to figure out whether these existing predicting models are suitable for Chinese patients. Lots of literatures validating these predicting models have been published, but few come from China. What’s more, most of these Chinese articles validated the MSKCC models only,[11,15,27,30] except Chen et al[31] (Table 4). To our knowledge, this is the first validation study of the SCH nomogram, a Chinese nomogram, using a Chinese breast cancer population. But the results of our study showed that the SCH nomogram didn’t perform as expected. In China, most of the patients from rural area couldn’t get early detection and diagnosis of the breast cancer. Thus, the clinicopathologic characteristics of Chinese patients with breast cancer in our database may be very different from other institutions, even the SCH database (Shanghai, China). Thus, it may be difficult to develop a worldwide suitable predicting model for non-SLN metastasis in patients with breast cancer, but all these efforts should be appreciated.

Table 4.

The AUC Results of Articles Validating Different Models for Predicting Non-SLN Metastasis With Positive SLNs in a Chinese Population.

Author	MSKCC	Tenon	Louisville	SNUH	Stanford	SCH
Original	0.75	–	0.680	0.80	0.74	0.7649
Chen et al (2012)[31]	0.64	0.66	0.60	0.61	0.54	–
Chen et al (2012), (SCH model)[11]	0.7105	–	–	–	–	–
Qiu et al (2012)[15]	0.730	–	–	–	–	–
Liu et al (2014)[30]	0.688	–	–	–	–	–
Huang et al (2015)[27]	0.677	–	–	–	–	–
Present	0.677	0.673	0.702	0.706	0.432	0.674

Abbreviations: AUC, area under the curve; MSKCC, Memorial Sloan-Kettering Cancer Center; SCH, Shanghai Cancer Hospital; SNUH, Seoul National University Hospital; SLN, SLN, sentinel lymph node.

According to the results of American College of Surgeons Oncology Group Z0011 trial,[32] the use of sentinel lymph node dissection (SLND) alone compared to ALND did not result in inferior survival for the majority of women with T1 and T2 clinically node-negative breast cancer. Experts had made consensus about it on St Gallen international conference in 2015.[33] But it is still controversial. Most breast surgeons will hardly ever take the risk of avoiding completion axillary dissection even with minimal sentinel lymph node metastases in daily clinical practice. The predicting models can indicate the risk of non-SLN metastasis; thus, it may help clinicians to make a more appropriate surgical plan for the patients, particularly in borderline cases.

Our study had a few limitations. First, this is a retrospective study using a single and small population, and maybe our sample were too less to represent the whole Chinese population. There are more than 2000 patients who did SLN operations in our hospital, but not all of them were suitable for our study. For the patients enrolled, the inclusion criteria were (1) no systemic treatment (such as neoadjuvant chemotherapy) before SLN biopsy, (2) identification of metastatic SLN(s), and (3) complete clinical and histological data. For example, many patients who did SLN operations were SLN negative, so they were excluded. Some did neoadjuvant chemotherapy before SLN biopsy, and they were excluded as well. The strict inclusion criteria ensured that all patients we included can be used to validate the existing predictive models. Second, the AUC values of the most of nomograms were worse than those reported in the original articles, probably because our patients’ characteristics significantly differ from those of the original series for which the models were developed. Especially in validating the Stanford nomograms, low micrometastatic rate population in our study might be responsible for the low AUC.

Conclusion

It should be pointed out that a nomogram or scoring system usually performs best at the institution where it was born, and it remains uncertain whether it would suit for other institutions. Thus, a new predicting model should be validated in more patients outside the facility. To select the most appropriate model from the existing predicting models developing for non-SLN assessment, the analysis of the clinicopathological features for the targeted patients with breast cancer is indispensable. And we should keep in mind that these predictive models are only risk calculators, they should be used with caution for decision-making when regarding complete ALND after a metastatic SLN biopsy.