An artificial neural network model predicting pathologic nodal metastases in clinical stage I-II esophageal squamous cell carcinoma patients.

BACKGROUND: Current preoperative staging for lymph nodal status remains inaccurate. The purpose of this study was to build an artificial neural network (ANN) model to predict pathologic nodal involvement in clinical stage I-II esophageal squamous cell carcinoma (ESCC) patients and then validated the performance of the model.
METHODS: A total of 523 patients (training set: 350; test set: 173) with clinical staging I-II ESCC who underwent esophagectomy and reconstruction were enrolled in this study. Their post-surgical pathological results were assessed and analysed. An ANN model was established for predicting pathologic nodal positive patients in the training set, which was validated in the test set. A receiver operating characteristic (ROC) curve was also created to illustrate the performance of the predictive model.
RESULTS: Of the enrolled 523 patients with ESCC, 41.3% of the patients were confirmed pathologic nodal positive (216/523). The ANN staging system identified the tumour invasion depth, tumour length, dysphagia, tumour differentiation and lymphovascular invasion (LVI) as predictors for pathologic lymph node metastases. The C-index for the ANN model verified in the test set was 0.852, which demonstrated that the ANN model had a good predictive performance.
CONCLUSIONS: The ANN model presented good performance for predicting pathologic lymph node metastasis and added indicators not included in current staging criteria and might help improve the staging strategies. 2020 Journal of Thoracic Disease. All rights reserved.

Introduction

Esophageal cancer (EC) is one of the most common malignant tumours in humans with poor prognosis. The overall 5-year survival rate of patients with EC ranges from 15% to 25% (1). According to the Global Cancer Statistics of 2018, among the 36 cancer diseases in 185 countries surveyed, the incidence of EC ranked seventh among all cancer diseases, and the mortality rate ranked sixth (2). Two common histologic subtypes have marked geographic distributions: squamous cell carcinoma (SCC) and adenocarcinoma (AC). In China, more than 90% of patients with EC are eventually diagnosed with SCC (3).

Treatment strategies for esophageal squamous cell carcinoma (ESCC) include surgery, radiotherapy, chemotherapy and etc. Different treatment strategies depend on the tumour pathologic stages. Precision medicine requires us to classify different states and processes of a disease accurately to achieve personalized treatment strategies for patients. However, due to the inaccuracy of clinical staging, significant percentages of patients were under-staged or over staged (4-6). Under-staged patients who should receive preoperative neoadjuvant chemoradiotherapy underwent surgery directly (7); over-staged patients received unnecessary induction therapy but did not improve long-term outcomes (8). Preoperative clinical staging is essential for developing rational treatment strategies that determine whether patients should receive preoperative neoadjuvant therapy or not. Therefore, in order to obtain the most suitable treatment strategies, it is vital to improve the accuracy of clinical staging.

The new eighth edition of the American Joint Committee on Cancer (AJCC) Cancer Staging system separates the clinical staging system from the pathologic staging system. The new clinical staging system is grouped by histopathologic cell type; the two main cell types of EC (SCC and AC) have independent staging standards. However, current clinical staging based largely on imaging and biomarker information is limited by the resolution and accuracy of each technique and remains coarse and imprecise (mainly when assessing clinical N staging) (9-12). The aim of this study was to construct and validate an artificial neural network (ANN) model to improve the accuracy of clinical N staging in patients with ESCC. We present the following article in accordance with the STROBE reporting checklist (available at http://dx.doi.org/10.21037/jtd-20-1956).

Methods

Study design and settings

This is a retrospective observational study design. A total of 798 patients with clinical staging I–II (cTNM < T3N1M0) EC (eighth edition AJCC) who underwent esophagectomy and reconstruction of the esophageal tract in Jinling Hospital of Nanjing Medical University from January 2017 to April 2019 were retrospectively enrolled in this study. Exclusion criteria included: (I) patients who received neoadjuvant therapy before surgery (n=142); (II) patients who had AC or other cell types (n=133). The remaining 523 cases consisted of 415 men and 108 women.

Collection of data

Preoperative examination for clinical staging included oesophagogastroscopy and biopsy to determine location, histologic subtypes, and grade; endoscopic ultrasound (EUS) and chest and abdominal computed tomography (CT) for cT; EUS fine-needle aspiration (EUS-FNA) for cN (non-peritumoral); positron emission tomography-CT (PET-CT) for cM and cN (9). Clinical N stage was established using a combination of EUS-FNA and PET-CT. Criteria for N+: (I) EUS-FNA positive cases were defined as those whose cytological diagnosis confirmed the presence of malignant cells in the aspirated lymph node consistent with origin from ESCC. All EUS-FNAs were performed with a CookTM 22-gauge needle by a single experienced endosonographer. (II) The criterion of positive lymph nodal metastasis for PET was determined to be more than 2.5 the maximum standardized uptake values (SUVmax) (type of scanner: Philips Gemini TF 16, Philips, The Netherlands; amount of tracer agent: 3.7 MBq; slice thickness of CT: 3 mm/slice). SUVmax for each site was calculated by the conventional formula normalized for body weight. All PET/CT images were evaluated by two experienced nuclear medicine physicians, and the diagnoses were made by consensus.

The pathologic staging was confirmed by pathologic examination of the tumour tissue and lymph node resected during surgery in all patients. All of the patients agreed to participate and provided informed consent. This study was reviewed and approved by the Institutional Review Board of the Jinling Hospital (approval number 2015NZKY-028-03). The study was conducted in accordance with the Declaration of Helsinki (as revised in 2013). A flowchart depicting the process used during the study is shown in .

Figure 1

The flow chart shows the original patient population, excluded patients and details of procedures used in the construction and validation of the ANN model. ANN, artificial neural network; AJCC, American Joint Committee on Cancer; AC, adenocarcinoma; ESCC, esophageal squamous cell carcinoma; DA, diagnostic accuracy; PPV, positive predictive value; NPV, negative predictive value; AUC, area under the ROC curve; ROC, receiver operating characteristic.

ANN modelling and validation

According to the principle of simple random sampling, the 523 cases were divided into two groups (training set, 350 patients; test set, 173 patients). The training set was used to construct the ANN prediction models, and the test set was used to verify and evaluate the accuracy of the ANN models.

A total of 28 variables including the baseline characteristics, perioperative data, and pathologic features of the 523 ESCC patients were collected and summarised in this study. Univariable analysis was performed to assess the candidate variables associated with pathologic lymph node metastasis among the 28 variables; then the candidate variables were input into the input layer to construct the ANN prediction models. The standard feed-forward, back-propagation neural network (BPNN) consisted of three layers composed of logical units: an input layer, a hidden layer, and an output layer. Beginning with an input layer consisting of all of the patients in the training set, the ANN models, similar to the hierarchical synaptic organisation of neurons in the brain, assigned each candidate variable an importance (weight) (13). The hidden layer received the weighted synapses and synthesised the data with the specified activation function; the output layer accepted the hidden layer and calculated the results. In this study, Sigmoid and Softmax were used as the activation functions of the hidden layer and output layer, respectively. The number of neurons in the hidden layer of the ultimate prediction model depended on the predictive performance and clinical relevance of the ANN model.

The diagnostic performance of the ANN model was assessed using the test set. To evaluate the accuracy of the prediction model, we compared the predicted outcomes of the ANN model with the actual observations and then calculated the overall sensitivity (SEN), specificity (SPE), false positive and negative rates, positive predictive value (PPV), negative predictive value (NPV), and diagnostic accuracy (DA) of the model. Receiver operating characteristic (ROC) curves were generated as primary measures of each model’s discriminant ability.

Statistical analysis

We compared the classification and continuous variables between the training set and the test set to ensure that the two groups were statistically comparable. The categorical variables were analysed using the χ2 or Fisher’s exact test, as appropriate. The Shapiro-Wilk test was used to assess whether the continuous data were normally distributed. On the basis of the results of the Shapiro-Wilk test, the continuous variables were compared using Student’s t-test or Mann-Whitney non-parametric tests, as appropriate. Univariable analysis and ANN modelling were performed using SPSS version 21.0 (IBM Corp., Armonk, NY, USA). P values <0.05 were defined as statistically significant.

Results

Study population and characteristics

A total of 523 patients suspected to have limited local-regional ESCC (clinical staging I–II) participated in this study. The pathologic nodal status distribution was as follows: N0: 307 (58.7%), N1: 148 (28.3%), N2: 65 (12.4%) and N3: 3 (0.5%). The 41.3% of patients were confirmed pathologic nodal positive (216/523). The demographics of all patients are illustrated in . According to the results of the univariate analysis, there were no statistical differences between the two sets in the 32 variables enrolled (P>0.05, ), indicating that training set and test set were statistically comparable. Mean dissected lymph node station number in 523 patients were 3.79±1.69; mean number of lymph nodes resected were 21.68±9.67. There is no statistical difference in SUVmax between clinically unrecognized positive lymph nodes and pathologically negative lymph nodes (1.73±0.66 vs. 1.68±0.59, P=0.62).

Table 1

Perioperative characteristics of patients

No.	Variables	All (n=523)	Training (n=350)	Test (n=173)	P
1	Gender				0.69
	Male	415	276	139
	Female	108	74	34
2	Age (years)	64.25±8.01	64.17±7.70	64.4±8.65	0.76
3	Diabetes				0.46
	Yes	39	24	15
	No	484	326	158
4	Charlson comorbidity index				0.84
	1	132	91	41
	2	363	240	123
	3	28	19	9
5	Smoking history				0.99
	Yes	266	178	88
	No	257	172	85
6	Drinking history				0.92
	Yes	219	146	73
	No	304	204	100
7	Symptoms				0.69
	Dysphagia	74	51	23
	Obstruction	449	299	150
8	Preoperative WBC (109/L)	5.86±1.84	5.82±1.76	5.92±2.01	0.55
9	Preoperative lymphocyte (109/L)	1.68±0.57	1.68±0.59	1.69±0.55	0.84
10	Preoperative neutrophil (109/L)	3.63±1.62	3.61±1.55	3.68±1.76	0.60
11	AFP (μg/L)	3.18±1.94	3.14±1.47	3.25±2.65	0.53
12	CEA (μg/L)	2.59±1.95	2.55±1.73	2.66±2.35	0.57
13	CA199 (U/mL)	11.75±9.49	12.13±10.01	10.99±8.31	0.19
14	CA125 (U/mL)	12.19±15.2	11.35±7.69	13.89±24	0.18
15	CA153 (U/mL)	8.73±4.81	8.68±4.70	8.84±5.05	0.71
16	SCC (ng/mL)	2.63±3.01	2.61±3.02	2.68±3.01	0.78
17	NSE (μg/L)	9.48±4.90	9.32±4.25	9.81±6.01	0.28
18	CYFRA21-1 (ng/mL)	4.92±18.10	5.57±22.05	3.59±1.99	0.24
19	Tumour location				0.88
	Upper	27	17	10
	Middle	302	204	98
	Lower	194	129	65
20	ASA score				0.32
	1	494	330	164
	2	28	20	8
	3	1	0	1
21	Type of surgery				0.62
	Open operation	242	157	85
	VATS	208	144	64
	RATS	73	49	24
22	Dissected lymph node station number	3.79±1.69	3.81±1.70	3.76±1.68	0.76
23	Mean number of lymph nodes resected	21.68±9.67	21.72±9.74	21.61±9.57	0.90
24	Method of anastomosis				0.74
	Cervical anastomosis	359	242	117
	Intrathoracic anastomosis	164	108	56
25	Tumour differentiation				0.69
	Well	91	60	31
	Moderate	278	185	93
	Poor	154	105	49
26	Tumour length (cm)	3.47±1.59	3.42±1.52	3.57±1.71	0.29
27	Tumour invasion depth				0.59
	cT1a	43	28	15
	m1	10	6	4
	m2	15	11	4
	m3	18	11	7
	cT1b	78	48	30
	sm1	14	9	5
	sm2	32	23	9
	sm3	32	16	16
	cT2	130	88	42
	cT3	272	186	86
28	Clinical N stage				0.10
	cN0	475	323	152
	cN1	48	27	21
29	Clinical tumour stage				0.83
	I	121	80	41
	II	402	270	132
30	LVI				0.16
	Absence	386	265	121
	Presence	137	85	52
31	Pathologic N status				0.52
	N0	307	210	97
	N1	148	92	56
	N2	65	46	19
	N3	3	2	1
32	Pathologic tumour stage				0.86
	I	183	125	58
	II	149	98	51
	III	187	125	62
	IV	4	2	2

P values refer to the comparisons between training set and test set. WBC, white blood cell; AFP, alpha-fetoprotein; CEA, carcinoembryonic antigen; CA199, carbohydrate antigen 19-9; CA125, carbohydrate antigen 125; CA153, carbohydrate antigen 15-3; SCC, squamous cell carcinoma; NSE, 2-phospho-D-glycerate hydrolase; ASA, American Society of Anaesthesiologists; VATS, video-assisted thoracic surgery; RATS, robot-assisted thoracic surgery; T, tumour classification; N, lymph node classification; m1, Intraepithelial tumour; m2, tumour invading the lamina propria; m3, tumour invading the muscularis mucosa; sm1, tumour invading the most superficial one-third of the submucosa; sm2, tumour invading the middle one-third of the submucosa; sm3, tumour invading deeper than sm2 level; LVI, lymphovascular invasion. Obstruction was defined as patient feels discomfort or pain when swallowing hard foods. Dysphagia was defined as difficulty in swallowing liquids or soft foods.

ANN modelling in the training set

As illustrated in , the univariate analysis was performed among 28 variables to identify the candidate variables using the data from the training set. Of the 28 variables, a total of 10 variables were identified as candidate variables [including age, smoking history, dysphagia, carcinoembryonic antigen (CEA), tumour location, tumour invasion depth, clinical N stage, tumour differentiation, tumour length and lymphovascular invasion (LVI), P<0.20]. The 10 variables were input into the input layer as neurons to construct the ANN prediction model. The eventual ANN prediction model is shown in . When the hidden layer was set to four, the ANN model obtained a good prediction performance. According to the ANN model generated by SPSS, we obtained the normalised importance of each variable (). Tumour invasion depth, tumour length, tumour differentiation, LVI and dysphagia were identified as important predictors for predicting pathologic nodal positive patients among all 10 variables (normalised importance >80%); their normalised importance values were 100%, 89.3%, 83.5%, 83.0% and 80.2%, respectively.

Table 2

Univariate analysis of the factors related to pathological nodal positive

No.	Variables	pN– (n=210)	pN+ (n=140)	P
1	Gender			0.22
	Male	161	115
	Female	49	25
2	Age (years)	63.96±7.43	64.49±8.10	0.06
3	Diabetes			0.49
	Yes	16	8
	No	194	132
4	Charlson comorbidity index			0.90
	1	53	38
	2	145	95
	3	12	7
5	Smoking history			0.08
	Yes	99	79
	No	111	61
6	Drinking history			0.89
	Yes	87	59
	No	123	81
7	Symptoms			<0.001
	Dysphagia	12	39
	Obstruction	198	101
8	Preoperative WBC (109/L)	5.73±1.52	5.94±2.05	0.30
9	Preoperative lymphocyte (109/L)	1.68±0.59	1.68±0.57	0.96
10	Preoperative neutrophil (109/L)	3.53±1.25	3.71±1.90	0.31
11	AFP (μg/L)	3.13±1.42	3.15±1.55	0.90
12	CEA (μg/L)	2.44±1.82	2.75±1.54	0.10
13	CA199 (U/mL)	11.78±9.66	12.74±10.60	0.39
14	CA125 (U/mL)	10.93±7.05	12.08±8.69	0.20
15	CA153 (U/mL)	8.55±4.70	8.90±4.71	0.50
16	SCC (ng/mL)	2.57±3.30	2.66±2.45	0.78
17	NSE (μg/L)	9.19±4.22	9.55±4.32	0.44
18	CYFRA21-1 (ng/mL)	6.42±27.53	4.09±2.85	0.34
19	Tumour location			0.10
	Upper	11	6
	Middle	131	73
	Lower	68	61
20	ASA score			0.98
	1	198	131
	2	12	9
	3	0	0
21	Tumour invasion depth			<0.001
	cT1a	27	1
	m1	6	0
	m2	11	0
	m3	10	1
	cT1b	40	8
	sm1	7	2
	sm2	21	2
	sm3	12	4
	cT2	58	30
	cT3	85	101
22	Clinical N stage			0.09
	cN0	198	125
	cN1	12	15
23	Type of surgery			0.28
	Open operation	101	56
	VATS	83	61
	RATS	26	23
24	Mean number of lymph nodes resected	20.99±9.88	23.00±9.39	0.06
25	Method of anastomosis			0.46
	Cervical anastomosis	146	95
	Intrathoracic anastomosis	64	45
26	Tumour differentiation			<0.001
	Well	55	5
	Moderate	114	71
	Poor	41	64
27	Tumour length (cm)	3.02±1.50	4.12±1.29	<0.001
28	LVI			<0.001
	Absence	183	82
	Presence	27	58

pN–, pathological lymph node negative; pN+, pathological lymph node positive; WBC, white blood cell; AFP, alpha-fetoprotein; CEA, carcinoembryonic antigen; CA199, carbohydrate antigen 19-9; CA125, carbohydrate antigen 125; CA153, carbohydrate antigen 15-3; SCC, squamous cell carcinoma antigen; NSE, 2-phospho-D-glycerate hydrolase; ASA, American Society of Anaesthesiologists; VATS, video-assisted thoracic surgery; RATS, robot-assisted thoracic surgery; T, tumour classification; N, lymph node classification; m1, Intraepithelial tumour; m2, tumour invading the lamina propria; m3, tumour invading the muscularis mucosa; sm1, tumour invading the most superficial one-third of the submucosa; sm2, tumour invading the middle one-third of the submucosa; sm3, tumour invading deeper than sm2 level; LVI, lymphovascular invasion. Obstruction was defined as patient feels discomfort or pain when swallowing hard foods. Dysphagia was defined as difficulty in swallowing liquids or soft foods.

Figure 2

ANN model constructed with training set data. ANN, artificial neural network; LVI, lymphovascular invasion; CEA, carcinoembryonic antigen.

Figure 3

Normalised importance of 10 variables for predicting pathologic nodal positive patients in the ANN model. ANN, artificial neural network; LVI, lymphovascular invasion; CEA, carcinoembryonic antigen.

Validation of the ANN model

We verified the performance of the ANN model in the training and test sets, respectively. The results are summarised in . The ROC curve is illustrated in . The area under the ROC curve (AUC) value for the ANN model verified in the test set was 0.852.

Table 3

Performance of the predictive model for pathological lymph node positive patients verified in the training and test sets

Sample	Predicted	Observed
		pN–	pN+	Value
Training	pN–	156	16	NPV 90.7% (156/172)
	pN+	54	124	PPV 69.7% (124/178)
	Value	SEN 74.3% (156/210)	SPE 88.6% (124/140)	DA 80.0%
Test	pN–	80	16	NPV 83.3% (80/96)
	pN+	17	60	PPV 77.9% (60/77)
	Value	SEN 82.5% (80/97)	SPE 78.9% (60/76)	DA 80.9%

pN–, pathological lymph node negative; pN+, pathological lymph node positive; SEN, sensitivity; SPE, specificity; NPV, negative predictive value; PPV, positive predictive value; DA, diagnostic accuracy.

Figure 4

ROC curve for the ANN model in the test set. ROC, receiver operating characteristic; ANN, artificial neural network.

Discussions

Over the past few decades, despite the continuous updating of the AJCC staging system, the clinical staging of EC patients still lacks sufficient accuracy. Several studies have reported the inaccuracy of the clinical staging. Zhang et al. reported that more than 50% of cT2N0 patients were under-staged based on the 6th edition of the AJCC staging system (7). Worrell et al. evaluated the accuracy of the seventh edition of the AJCC staging system; of 88 AC patients enrolled with clinical stage ≤ T3N1M0, 21 patients were diagnosed with final pathologic stage > T3N1M0 (14). In 2018, a systematic review and meta-analysis reported similar results. Lv et al. collected eight retrospective studies of 2,646 patients with cT2N0M0 (neoadjuvant therapy combined with surgery group, n=961; simple surgery group, n=1,685). The results showed that tumour staging was under-estimated in six studies; further subgroup analysis suggests that N staging was more likely to be under-estimated (30.1% vs. 35.1%) than T staging (6). This was because the current preoperative examination is inaccurate (especially for N staging) and equipment has regional differences. To date, previous studies have found several predictors related with pN+. Gaur et al. found that cT (OR, 5.6; 95% CI, 1.7 to 18.6; P<0.01) and tumour length (OR, 7.0; 95% CI, 2.7 to 18.1; P<0.001) were significant independent factors of nodal metastases in patients with esophageal AC (EAC) (15). Barbetta enrolled 80 cT2N0M0 EAC patients and found vascular invasion was an independent risk factor for nodal disease (16). Samson et al. enrolled 46 ESCC patients (29.3%) and 111 EAC patients (70.7%); LVI (OR, 6.0; 95% CI, 2.9 to 12.5; P<0.001) and tumour grade 3 (OR, 9.4; 95% CI, 1.8 to 48.4; P=0.007) were identified as independent predictors for nodal metastases (11). Currently, data on patients with ESCC are lacking. In this study, we constructed an ANN model for predicting pathologic nodal positive ESCC patients. Based on the subsequent validation, the ANN model was proven to have a good performance and may help clinicians improve the clinical strategies.

The ANN is a statistical model that mimics the structure and function of biological neural networks (17). Compared with traditional statistical techniques (for example, logistic regression and Cox proportional hazards regression), the self-learning ability of the BPNN (by modifying weights and thresholds based on new inputs) makes it possible to approximate any non-linear function with arbitrary precision (18,19). This machine learning statistical method is increasingly used in the medical field, such as for disease diagnosis and the prediction of patient outcomes or complications (20-22). This is also the first study demonstrating the application of BP ANNs for predicting pN+ patients with ESCC.

In this study, 523 patients with ESCC (clinical TNM classification < T3N1M0) were enrolled and randomly assigned to a training group (n=350) and a test group (n=173). 9% (48/523) of the patients were diagnosed with clinical nodal positive preoperatively; whereas 41.3% (216/523) of the patients were confirmed pathologic nodal positive. This result may imply the inaccuracy of the current clinical examination for N staging. Ideally, clinical staging should correctly determine a patient’s lymph node metastasis. To achieve this goal, an ANN model was established based on the baseline characteristics, perioperative data, and pathologic features in the training set (). As illustrated in , tumour invasion depth, tumour length, tumour differentiation, LVI and dysphagia were identified as important factors influencing model decision (the normalised importance values were 100%, 89.3%, 83.5%, 83.0% and 80.2%, respectively). Subsequently, we performed a series of verifications on the constructed prediction model based on the test set. The AUC revealed that the ANN model had good predictive accuracy (SEN, 82.5%; SPE, 78.9%; PPV, 77.9%; NPV, 83.3%; DA, 80.9%; AUC, 0.852).

The eighth edition of the AJCC staging system separates clinical staging from pathologic staging and establishes independent clinical staging strategies for AC and ESCC, respectively (9). The eighth edition of the AJCC cancer clinical staging system is based solely on T staging and N staging and does not take into account tumour differentiation, tumour length, LVI, or symptoms. In this study, the above four indicators were identified as important predictors influencing model decision. We suggest that future clinical staging criterion can consider incorporating the above indicators to make the subgroups more homogeneous.

Several drawbacks limited our study. Because only patients with clinical staging < T3N1M0 were enrolled (patients with advanced stages were excluded), both SEN and accuracy in the detection of nodal metastasis may have been relatively low. The ANN model constructed in this study can be used as a supplement to the current clinical staging system to reduce the false negative rate of preoperative lymph node assessment (if the patient is diagnosed as clinical stage I–II, we can use the model to screen out patients with suspicious N+). This single-centre study is also limited by its retrospective nature and the small number of patients. In addition, the ANN model constructed in this study also lacks external validation. In the future, we will continue to expand the sample size (using clinical databases or collecting multi-centre clinical data) to improve the versatility and accuracy of the model.

Conclusions

In conclusion, tumour invasion depth, tumour length, tumour differentiation, LVI and dysphagia were important factors for predicting pathologic nodal positive. The ANN model integrates these factors and appears suitable for clinical staging stratification for ESCC. The prediction model is based on easily available perioperative indicators and can have potential implications for clinical management.

The article’s supplementary files as