Comparison of prognostic nomograms based on different nodal staging systems in patients with resected gastric cancer.

Purpose: Previous studies addressing the optimal nodal staging system in patients with resected gastric cancer have shown inconsistent results, and the optimal system for development of prognostic nomograms remains unclear. In this study, we compared prognostic nomograms based on the metastatic lymph node (MLN) count, lymph node ratio (LNR), and log odds of metastatic lymph nodes (LODDS) to predict the 5-year overall survival in patients with resected gastric cancer.
Methods: We analysed 15,320 patients with resected gastric cancer in the Surveillance, Epidemiology, and End Results (SEER) database between 1988 and 2010. Missing data were handled using multiple imputation. When assessed as a continuous covariate with restricted cubic splines, each MLN, LNR, and LODDS variable was incorporated into a nomogram with other significant prognosticators to predict the 5-year overall survival. A two-centre Chinese dataset (1,595 cases) was used as external validation data.
Results: The discriminatory abilities of the MLN-, LNR-, and LODDS-based nomograms were comparable (concordance indices: 0.744, 0.741, and 0.744, respectively, in the SEER set, P > 0.152 for all pairwise comparisons; 0.715, 0.712, and 0.713, respectively, in the Chinese set, P > 0.445 for all pairwise comparisons). The discriminatory abilities of the three nomograms were all superior to the American Joint Committee on Cancer (AJCC) TNM classification (concordance indices: 0.713, P < 0.001 for all in the SEER set; and 0.693, P < 0.001 for all in the Chinese set). The discriminatory abilities of the nomograms were comparable regardless of the number of nodes examined. Moreover, decision curve analyses indicated similar net benefits of using the nomograms.
Conclusion: MLN-, LNR-, and LODDS should be considered equally in the development of multivariate prognostic models and nomograms to refine the prediction of survival among patients with resected gastric cancer.

Introduction

Gastric cancer (GC) was the fifth most common malignancy and the third leading cause of cancer-related deaths worldwide, with an especially high incidence in East Asia 1-3. The accuracy of survival prediction for patients with GC is pivotal for postoperative treatment decisions and surveillance. Conventionally, the outcomes of radically resected GC are predicted based on the pathological tumour invasion depth and lymph node status, which are addressed in the American Joint Committee on Cancer (AJCC) TNM classification 4. However, although a minimum of 16 total harvested nodes (THNs) are required by the 7th AJCC TNM classification for accurate nodal staging,3 this system is merely based on the number of metastatic lymph nodes (MLNs) and has been criticized for disregarding the influence of THNs on survival 5, 6. To refine the prediction of prognosis in patients with GC, two ratio-based nodal staging systems have been proposed: the lymph node ratio (LNR) 7, defined as the ratio between MLNs and THNs, and the log odds of metastatic lymph nodes (LODDS) 8, defined as the log of the ratio between the number of MLNs and number of negative lymph nodes (NLNs). However, because of the variation and correctness of the analysis methodology, studies comparing the discriminatory capacities of MLN-, LNR- and LODDS-based staging systems have shown inconsistent results 8-14, and a consensus regarding the optimal system has not yet been reached.

Prognostic nomograms are useful tools that allow for intuitive individual risk evaluation by combining prognostic factors 15. Several nomograms have been developed that incorporate the number of MLNs along with the number of THNs or NLNs and other prognostic factors (e.g., age and depth of invasion) to refine the prediction of outcomes in patients with resected GC 15-20. However, it remains unknown whether using the LNRs or LODDS instead of the MLNs in model development can improve the prognostic performances of these nomograms.

Thus, in the present study, we developed prognostic nomograms incorporating the MLNs, LNR, or LODDS with other significant prognosticators using a population-based cohort from the Surveillance, Epidemiology, and End Results (SEER) database. Thereafter, we performed comparisons of these nomograms among the SEER dataset and an independent Chinese dataset, which allowed for multivariable-adjusted comparisons among the nodal staging systems.

Materials and Methods

Data sets

Using the SEER database, we identified 21,117 patients with GC diagnosed from January 1, 1988 to December 31, 2010. We excluded patients diagnosed after 2010 to allow for a longer median follow-up period. The inclusion criteria were as follows: (1) a single primary gastric carcinoma (ICD-O-3 8010-8231 and 8255-8576) confirmed microscopically; (2) a gastrectomy with at least one node harvested; (3) no radiotherapy at any time; and (4) active follow-up. The following cases were excluded from the analysis: cases with a length of follow-up less than 3 months (3,031 cases), cases of carcinoma in situ (302 cases), and cases of synchronous distant metastasis or unknown status of metastasis (2,204 cases). Stage information was determined according to the 7th AJCC TNM classification. Patients with insufficient information for conversion to the 7th AJCC TNM classification (260 cases) were excluded. The final analytic SEER cohort for the development of nomograms consisted of 15,320 patients.

An independent Chinese cohort consisting of 1,595 patients who had been diagnosed with GC and had undergone curative gastrectomy between 2001 and 2010 at the Sun Yat-Sen University Cancer Centre or Nanfang Hospital was used as a validation data set. The Chinese validation set was collected according to the aforementioned inclusion and exclusion criteria.

Ethical issues

No institutional review was sought because SEER is public-use data; informed consent was waived. The study protocol for the Chinese dataset was approved by the Ethics Committees of the Sun Yat-Sen University Cancer Centre and the Nanfang Hospital.

Statistical analysis

MissForest is a random forest-based method that can efficiently address missing data imputation among multivariate data by producing a single imputed dataset without setting aside test data or performing cross validations 21. For inferring missing data of variables in the SEER set (race: 43 cases, 0.3%; tumour location: 5,237 cases, 34.2%; size: 2,578 cases, 16.8%; and differentiation: 858 cases, 5.6%) and the Chinese dataset (tumour location: 87 cases, 5.5%; size: 35 cases, 2.2%; and differentiation: 178 cases, 11.2%), multiple imputation using missForest19 was performed with the following variables: year of diagnosis (or year of surgery for the Chinese set), patient age, gender, T stage, MLNs, and NLNs.

Overall survival (OS) was the primary outcome of interest. Multivariate Cox regression analyses were used to assess the relationships between clinicopathological factors and OS. To avoid multicollinearity, the MLN, LNR, and LODDS variables were included in three different Cox models 22. Restricted cubic splines were used to examine the functional forms of continuous variables in relation to survival 23. When the relationship was apparently non-linear, transformation using multivariable fractional polynomials was performed 24, and the optimal transformation was obtained based on the Bayesian Information Criteria 25. These methods reduce the bias that can result from empirically selecting cut-off points for continuous variables. For model construction, we began with the following variables: patient age, year of diagnosis, race, tumour location, size, differentiation, T stage, one of the nodal variables (MLNs, LNR, and LODDS), and NLNs (included in the initial MLN-based model only). The final Cox models were obtained using backward stepwise selection (P < 0.050).

Nomograms were generated from the final models based on the MLN, LNR and LODDS variables. Concordance indices (C-indices) were used to compare the capacities of the three nomograms to distinguish patients at low and high risk of death, that is, the discriminative powers of the three nomograms 26. The higher the C-index, the greater the discriminatory capacity of the scheme 26. Additionally, we used calibration to measure how far the nomogram-predictions deviated from the actual outcomes 27. For each nomogram, calibration was performed by reviewing the plot of nomogram-predicted survival probabilities and the Kaplan-Meier-estimated probabilities. A perfect calibration was indicated by the 45-degree reference line 27. Bootstraps with 1,000 resamples were used to quantify the model overfit and calculate Kaplan-Meier estimates. External validation of the nomograms was performed by discrimination and calibration using the Chinese dataset.

In addition to comparisons of the predictive accuracies of the nomograms, we also relied on decision curve analyses, as describe by Vickers et al. 28, to assess the potential of the nomograms for clinical application. Decision curve analyses examine the clinical practical value of a predictive model by quantifying its net benefit according to the threshold probability and the relative weight between false-positive and false-negative results.

For sensitivity analyses, we also constructed Cox regression models based on the MLN, LNR or LODDS variables and repeated the analyses among the imputed Chinese dataset or among patient subsets with complete data for all variables in the SEER or Chinese datasets.

Statistical significance was set as P < 0.050 in a two-tailed test. Statistical analyses were performed using SAS version 9.3 (SAS Institute, Cary, NC, USA), SPSS v.19.0 (SPSS, Chicago, IL, USA), and R v.3.2.3 (http://www.r-project.org).

Results

The clinicopathological characteristics for the SEER set (15,320 cases) and Chinese validation set (1,595 cases) are listed in Table 1 and Table S1, respectively. Patient characteristics for the multiple imputed datasets are listed in Table S2. The median numbers of MLNs and THNs were 1 (interquartile range [IQR]: 0-5) and 11 (IQR: 6-19), respectively, in the SEER dataset, and 2 (IQR: 0-6) and 18 (IQR: 11-26), respectively, in the Chinese dataset. The 5-year Kaplan-Meier OS estimates were 37.6% (95% confidence interval: 37.2%-38.0%) and 61.2% (95% confidence interval: 59.9%-62.5%) in the SEER and Chinese sets, respectively.

Table 1

Characteristics of patients in the SEER set.

Variable	Median (IQR)/N (%)
Year of diagnosis
1988-1991	1528 (10.0%)
1992-1995	2271 (14.8%)
1996-1999	2324 (15.2%)
2000-2003	3429 (22.4%)
2004-2007	3343 (21.8%)
2008-2010	2425 (15.8%)
Age, years	70 (60-78)
Race
White	9874 (64.5%)
Black	1882 (12.3%)
Other	3521 (23.0%)
Unknown	43 (0.3%)
Gender
Female	6147 (40.1%)
Male	9173 (59.9%)
Tumour location
Cardia	3455 (22.6%)
Upper one-third	469 (3.1%)
Middle one-third	1257 (8.2%)
Lower one-third	4902 (32.0%)
Not specified	5237 (34.2%)
Tumour size, cm	4.2 (2.5-6.5)
Unknown	2578 (16.8%)
Tumour differentiation
Poorly or undifferentiated	9940 (64.9%)
Well or moderately differentiated	5036 (32.9%)
Unknown	858 (5.6%)
T stage
T1	3718 (24.3%)
T2	1943 (12.7%)
T3	3856 (25.2%)
T4a	4234 (27.6%)
T4b	1569 (10.2%)
MLNs	1 (0-5)
THNs	11 (6-19)
NLNs	8 (3-14)
LNR	0.08 (0.00-0.50)
LODDS	-1.61 (-2.94-0.00)
Adequate nodal evaluation
No (≤ 15)	9,641 (62.9%)
Yes (> 15)	5,679 (37.1%)

SEER, Surveillance, Epidemiology, and End Results database; IQR, interquartile range; MLNs, metastatic lymph nodes; THNs, total harvested lymph nodes; NLNs, negative lymph nodes; LNR, lymph node ratio; LODDS, log ratio of metastatic lymph nodes.

Table 2 shows the MLN-, LNR-, and LODDS-based final Cox models obtained by stepwise backward selection (P < 0.050). The hazard ratios were significantly higher for white and black race, older age, male gender, advanced T stage, increased MLNs, LNR, and LODDS, and decreased NLNs. However, the year of diagnosis was not significantly associated with OS. In the MLN-based model, there was no significant interaction between MLNs and NLNs ( = 0.104). Prognostic nomograms were developed from the three final models (Figures 1-3).

Table 2

MLN-. LNR-, and LODDS-based prognostic models.

Variable	MLN-based model			LNR-based model			LODDS-based model
	Hazard ratio	P value		Hazard ratio	P value		Hazard ratio	P value
MLNa	1.333 (1.309-1.356)	< 0.001
LNRb				6.528 (6.017-7.082)	< 0.001
LODDS							1.325 (1.300-1.330)	< 0.001
NLNc	0.734 (0.719-0.749)	< 0.001
Race		< 0.001			< 0.001			< 0.001
White	1			1			1
Black	1.151 (1.083-1.224)			1.143 (1.075-1.215)			1.151 (1.085-1.225)
Other	0.874 (0.831-0.919)			0.859 (0.817-0.903)			0.877 (0.834-0.922)
Age	1.020 (1.018-1.022)	< 0.001					1.020 (1.018-1.022)	< 0.001
Aged				4.776 (4.239-5.380)	< 0.001
Gender		< 0.001
Female	1			1			1
Male	1.083 (1.039-1.128)			1.09 (1.048-1.137)			1.082 (1.039-1.128)
Tumour Location		< 0.001			< 0.001			< 0.001
Cardia	1			1			1
Upper one-third	0.898 (0.817-0.987)			0.902 (0.821-0.992)			0.893 (0.813-0.982)
Middle one-third	0.787 (0.735-0.842)			0.787 (0.736-0.843)			0.783 (0.732-0837)
Lower one-third	0.838 (0.798-0.879)			0.845 (0.805-0.887)			0.836 (0.797-0.877)
Tumour sizee	1.128 (1.087-1.171)	< 0.001		1.118 (1.078-1.159)	< 0.001		1.131 (1.091-1.174)	< 0.001
Tumour differentiation		< 0.001			< 0.001			< 0.001
Poorly or undifferentiated	1			1			1
Well or moderately differentiated	0.898 (0.861-0.937)			0.900 (0.863-0.939)			0.897 (0.860-0.935)
T stage		< 0.001			< 0.001			< 0.001
T1	1			1			1
T2	1.403 (1.300-1.515)			1.367 (1.266-1.476)			1.407 (1.303-1.519)
T3	1.835 (1.710-1.968)			1.773 (1.654-1.901)			1.849 (1.725-1.981)
T4a	2.249 (2.093-2.416)			2.220 (2.068-2.384)			2.268 (2.113-2.435)
T4b	3.174 (2.916-3.454)			3.148 (2.893-3.427)			3.194 (2.936-3.475)

MLN, metastatic lymph node; NLN, negative lymph node; LNR, lymph node ratio; LODDS, log ratio of metastatic lymph node.

a. In the MLN-based model, MLN had a non-linear effect on the log hazard of survival. Therefore, loge (MLN+0.5) was included in the model to fit a linear form.

b. In the LNR-based model, LNR had a non-linear effect on the log hazard of survival. Therefore, was included in the model to fit a linear form.

In the MLN-based model, NLN had a non-linear effect on the log hazard of survival. Therefore, loge [(NLN+1)/10] was included in the model to fit a linear form.

c. In the LNR-based model, age had a non-linear effect on the log hazard of survival. Therefore, (Age/100)2 was included in the model to fit a linear form.

d. In the MLN-, LNR-, and LODDS-based models, tumour size had a non-linear effect on the log hazard of survival. Therefore, loge (size/10) was included in the models to fit a linear form.

Figure 1

Nomogram based on the metastatic lymph node (MLN) count for predicting the 5-year overall survival (OS) in patients with resectable gastric cancer. The nomogram is used by summing the points projected on the points scale by each variable. The total points projected on the bottom scales indicate the estimated 5-year OS probabilities. NLN, negative lymph node.

Figure 3

Nomogram based on the log odds of metastatic lymph nodes (LODDS) for predicting the 5-year overall survival (OS) in patients with resectable gastric cancer. The nomogram is used by summing the points projected on the points scale by each variable. The total points projected on the bottom scales indicate the estimated 5-year OS probabilities.

The unadjusted C-indices for OS prediction for the MLN-, LNR-, and LODDS-based nomograms were 0.744, 0.741, and 0.744, and the bootstrap-corrected C-indices were also 0.744, 0.741, and 0.744, respectively, indicating minimal evidence of overfit for all three models. The C-indices among the three nomograms were comparable and were all equal to two decimal places ( > 0.152 for all pairwise comparisons). All three nomograms showed better discriminatory abilities than the 7th AJCC TNM classification (C-index: 0.713, P < 0.001 for all). When the patients were stratified into subgroups according to THNs (1-10, 11-15 and > 15 THNs), the discriminatory capacities of the three nomograms remained similar among all three subgroups (Table S3). Furthermore, calibration plots showed close agreement between the actual 5-year OS probabilities in the patients of the SEER set and the OS predicted from the three nomograms (Figure S1). Similar calibration results were recorded from the examination of the mean deviations between the actual and predicted OS probabilities, and the mean deviations between the actual survival probabilities and those predicted with the MLN-, LNR-, and LODDS-based nomograms were 0.016, 0.014, and 0.014, respectively (Figures S1A-C).

The C-indices for OS prediction for the MLN-, LNR- and LODDS-based nomograms were comparable for the Chinese dataset (0.715, 0.712, and 0.713, respectively, > 0.445 for all pairwise comparisons). All three nomograms showed better discriminatory abilities than the 7th AJCC TNM classification (C-index: 0.693, < 0.001 for all). The calibration was consistent between the actual 5-year OS probabilities and the nomogram-predicted OS among the Chinese set (Figure S2). The mean deviations between the actual and predicted OS probabilities from the three nomograms were also comparable (0.022, 0.017, and 0.023 for the MLN-, LNT-, and LODDS-based nomograms, respectively; Figures S2A-C).

The decision curve analysis indicated that for most of the threshold probabilities for 5-year OS in both the SEER and Chinese sets, the three nomograms yielded a similar net benefit, and all of them achieved a greater net benefit compared with the 7th AJCC TNM classification (Figs. 4A-4B).

Figure 4

Decision curve analyses of the 5-year overall survival (OS) predictions for the training set (A) and the Chinese set (B). The dashed lines indicate the net benefit of using the nomograms based on the metastatic lymph node (MLN) count (green dashed line), lymph node ratio (LNR; yellow dashed line), and log odds of metastatic lymph nodes (LODDS; blue dashed line), and the 7th TNM staging (red dashed line). The assumption that all of the patients will die is shown with a grey line, and the assumption that all of the patients will survive is indicated with a black line. Threshold probability = 1 - predicted 5-year OS.

As determined through sensitivity analyses performed among the imputed Chinese dataset, patients with complete data for all variables in the SEER set, and patients with complete data for all variables in the Chinese set and using the C-index, calibration, and decision curve analysis as metrics, the MLN-, LNR-, and LODDS-based models exhibited comparable performances (data not shown).

Discussion

An accurate prediction of the prognosis of patients with GC is essential for treatment and follow-up planning. Traditionally, the outcomes of resected GC are predicted based on the AJCC TNM classification, which involves tumour invasion depth and nodal status. However, this system does not account for the number of THNs, which reportedly may influence patient survival 29, 30. Two ratio-based nodal staging systems, namely LNR and LODDS, were proposed to refine survival prediction in patients with resected GC. However, studies comparing these nodal staging systems have proven to be contradictory; whereas some studies have found that the LODDS-based system exhibited the best prognostic performance, 8-12 others did not find a predictive superiority of the LODDS system over the MLN or LNR systems 13, 14, 31.

The inconsistencies among these studies are most likely due to differences in the analysis methodology. Because all three nodal staging systems are closely correlated, it is methodologically inappropriate to use stepwise variable selection to determine the best-performing staging system 32, as was performed in the studies by Sun et al. 8 and Liu et al. 14. In addition, it is necessary to compare the prognostic performances of the nodal staging systems based on multivariate models to account for available prognostic information and reduce potential confounding. However, some of the studies that addressed this issue did not report the specific variables that were included in the multivariate models along with the nodal variables 9, 13. In addition, in a SEER-based study which found a significant predictive superiority of the LODDS-based model compared with the MLN-based model in patients with less than 10 THNs, neither the THNs nor the NLNs were included in the MLN-based model,12 which may have weakened the prognostic performance of this model. In the present study, the number of NLNs was actually identified as an independent prognosticator in the MLN-based model. When the NLNs was removed from the model, the C-index decreased significantly from 0.744 to 0.730 (< 0.001).

No consensus exists regarding the ideal cut-off values for the three nodal staging systems, and the discrimination of all three systems reportedly improves when they are treated as continuous covariates 9. In this study, multivariate regression models were fitted when the MLN, LNR, and LODDS variables were assessed as continuous covariates to avoid arbitrarily selecting cut-off points for continuous variables, and prognostic nomograms were developed from these models to provide intuitive illustrations. The discriminatory powers of the MLN-, LNR- and LODDS-based nomograms were comparable among the SEER and Chinese datasets. Additionally, the discriminatory abilities of the nomograms were comparable regardless of the number of THNs.

Metrics concerning prognostic accuracy cannot indicate whether a prognostic model is worth using in clinical practice. For example, a scheme with high specificity but low sensitivity may have the same C-index as one with low specificity but high sensitivity. However, in the case of a disease (e.g., death in patients with GC) for which a false-negative result is much more harmful than a false-positive result; the former scheme would be a poorer choice for clinical use. Decision curve analysis addresses this issue, because it can evaluate the clinical practical value of a staging scheme by quantifying its net benefit according to the threshold probability and the relative weight between false-positive and false-negative results 28. For both the SEER and Chinese sets, the three nomograms yielded a similar net benefit over virtually the entire range of the threshold probabilities for the 5-year OS predictions. Therefore, the nomograms showed similar potential for clinical application, suggesting that they were equivalently useful in decision-making regarding postoperative treatment and surveillance.

Several good nomograms have been developed to refine survival prediction among patients with resected GC 15-20. Among these nomograms, only one included LNR rather than MLN along with other prognostic factors to refine the prediction of outcomes,17 and scarce data have addressed whether LNR and LODDS is better than MLN in development of nomograms. Additionally, it is necessary to confirm the generalizability of a nomogram by externally validation using an independent population. In this study, despite the well-recognized differences in tumor epidemiology (e.g., the percentage of proximal disease), treatment patterns, and treatment outcomes between Eastern and Western countries, 33 the proposed nomograms based on MLN-, LNR-, and LODDS exhibited favorable performance in both the SEER and Chinese datasets. Moreover, by using the Chinese set, we also verified the findings from the SEER set that MLN, LNR, and LODDS exhibited equivalent prognostic value after adjustment for other prognostic factors. Therefore, MLN, LNR, LODDS should be equally considered when developing prognostic nomograms.

The use of the SEER data enabled us to draw sound conclusions pertinent to the general clinical practice based on a large sample of patients with GC, which is not possible in single-institution studies. However, the present study has some limitations that should be noted. First, even though the SEER database is checked regularly for discrepancy and reportedly has 95% accuracy, the possibility of coding errors remains. Additionally, the SEER database does not provide data on other possible prognosticators such as lymphovascular invasion, margin status, and patient comorbidities, which may influence the results. Moreover, due to the retrospective nature of the SEER and Chinese datasets, further verification of our findings using prospective data is needed.

In summary, this study demonstrated the comparable predictive abilities and potential for clinical application of the prognostic nomograms based on MLN-, LNR-, and LODDS. Therefore, MLN-, LNR-, and LODDS are interchangeable in the development of multivariate prognostic models and nomograms to refine the prediction of survival among patients with resected GC.

Supplementary figures and tables.

Click here for additional data file.