Development and validation of a nomogram to predict liver metastasis in patients with pancreatic ductal adenocarcinoma: a large cohort study.

Background: Few studies have explored the relationship between clinicopathological factors of patients with pancreatic ductal adenocarcinoma (PDAC) and liver metastasis. The aim of this study was to develop and validate a nomogram to predict liver metastasis in patients with PDAC. Patients and methods: Patients diagnosed with PDAC between 2004 and 2015 from the Surveillance, Epidemiology, and End Results (SEER) database were retrospectively collected. The nomogram was established based on a logistic regression model. The precision of the nomogram was evaluated and compared using concordance index (C-index), and the area under receiver operating characteristic curve (AUC). The clinical use of nomogram was evaluated by making use of a decision curve analysis (DCA).
Results: A total of 12,644 eligible patients, which were randomly divided into training (n=9,483) and validation cohorts (n=3,161), were included in this study. The nomograms, which were established on the basis of independent predictors, were well calibrated, and demonstrated good discriminative ability, with C-indexes of 0.784 for the training cohort and 0.790 for validation cohort. The values of AUC for training and validation cohort were 0.792 and 0.800, respectively. When other sites of distant metastases were included into this predictive system, the new predictive model demonstrated a better discriminative ability and greater net benefit in predicting liver metastasis in patients with PDAC in both the training and validation cohorts.
Conclusion: Nomograms were constructed to predict liver metastasis in patients with PDAC. Validation revealed excellent discrimination and calibration of the nomograms, suggesting that the nomograms were well calibrated and could serve to improve the prediction of the risks of liver metastasis which can be used to guide the management of patients with PDAC.

Introduction

Pancreatic ductal adenocarcinoma (PDAC) is a lethal disease, ranking 7th and 6th as the leading cause of cancer-related death, in both worldwide and China, respectively.1,2 Mostly because of the absence of specific clinical manifestation, most patients are diagnosed at advanced stages, and as such only 15–20% of the PDAC patients become eligible for potential curative surgical resection at diagnosis.3 Distant metastasis is a major indicator of poor prognosis.4,5 PDAC predisposes the liver to secondary tumors and liver metastasis is the most common form of distant metastasis in these patients.6 Also, it was shown that patients with liver metastasis had worse prognoses than those with lung metastasis or other distant metastases.7 The treatment strategies and prognoses of PDAC are largely determined by the presence or absence of liver metastasis. Moreover, the early detection of liver metastasis signifies an earlier chance to receive more aggressive treatment approaches, which may lead to better survival, compared with the standard chemotherapeutic treatment alone.4,8 In fact, all patients with PDAC are at risk of liver metastasis, therefore it is essential for clinicians to accurately judge the risk of liver metastasis for selection of an optimal therapeutic.

Previous studies have shown some correlation between the clinical and pathological factors for the predisposition of liver metastasis in patients with PDAC.9,10 However, those reports were mainly based on small cohorts of patients and only evaluated fragmentary factors. It would be interesting to study the association between clinicopathological factors and liver metastasis, and to construct effective models to predict the risk of liver metastasis for patients with PDAC. In this study, we aimed to identify the independent factors promoting liver metastasis in PDAC patients and to establish a nomogram to predict the risk of these liver metastases. With this scoring system, we expect clinicians to be able to promptly identify high-risk patients for providing them with an optimal individualized therapeutic regimen.

Patients and methods

Patients

The Surveillance, Epidemiology, and End Results (SEER) database of the National Cancer Institution, which covers approximately 30% of the US population, was used in this study. Data of patients with PDAC diagnosed between 2004 and 2015 were extracted from the SEER database, using the SEER*Stat software version 8.3.5. The study cohort consisted of patients with the following: the International Classification of Diseases for Oncology, Third Edition (ICD-O-3), histology code: 8,140; and the ICD-O-3 site code C25.0, C25.1 and C25.2. The exclusion criteria were as follows: (1) patients with second primary cancer; (2) age at diagnosis younger than 18 years; (3) patients not pathologically diagnosed; (4) patients with missing or incomplete information about clinicopathological features. Three-fourths of the patients were randomly selected to form the training cohort and to develop the nomograms, and the rest of the patients were selected to serve as an internal validation cohort. Institutional review board approval and informed consent are not required for this current study because the SEER research data is publicly available and all patient data are de-identified. All authors have signed the authorization form and received permission from SEER organization to access and use the dataset.

Data collection

The following clinicopathological features used for this study are as follows: age at diagnosis, gender, tumor site, tumor grade, tumor size, tumor-node-metastasis (TNM) stage, lymph node (LN) metastasis, liver metastasis, lung metastasis, bone metastasis and brain metastasis. The 8th edition of the TNM stage was used as the stage system.

Statistical analysis

Statistical analysis was performed using SPSS software version 22 (SPSS Inc., Chicago, IL, USA) and the R version 3.4.2 software (The R Foundation for Statistical Computing, Vienna, Austria. http://www.r-project.org). For comparison between two groups, the Student’s t-test was used for nominal data and the chi-square test was used for categorical variables. The association of the risk of liver metastasis with the clinicopathological factors was evaluated using univariate logistic regression analysis. Variables deemed significant were further analyzed in the multivariate analysis to determine the independent risk factors for liver metastasis. The odds ratio (OR) and associated 95% confidence interval (CI) were also calculated.

A nomogram for predicting the probability of liver metastasis was developed based on the independent risk factors identified in the multivariate analysis. The discrimination and calibration power were two important aspects of the performance of the established nomograms, and they were evaluated by using concordance index (C-index) and calibration curves,11,12 respectively, accompanied by the Hosmer-Lemeshow test.13 Bootstraps with 1,000 resamples were used for the validation of the nomograms and the calibration curves to reduce the overfit bias. In addition, the precision of the prediction for liver metastasis was evaluated using the area under receiver operating characteristic (ROC) curve (AUC). The clinical use of nomogram was evaluated by decision curve analysis (DCA).14 A two-tailed P-value <0.05 was considered as statistically significant.

Results

Patient characteristics

We identified 12,644 eligible patients with PDAC diagnosed from 2004 to 2015 in the SEER database. There were 9,483 patients allocated to the training cohort and 3,161 patients to the validation cohort. The median age of the entire study cohort was 67 years old and 50.8% of the patients were male. The demographic and clinicopathological features of the training and validation cohort are summarized in Table 1. In the present study, the mean and the median follow-up time were 13.8 months and 8 months (range, 1–143 months), respectively. During the whole period of follow-up, 10,689 (84.5%) patients had died and only 1,955 (15.5%) patients were alive. Liver metastasis was present in 1,207 of the 9,483 (12.7%) patients and 440 of 3,161 (13.9%) patients in the training and validation cohorts, respectively. There was no significant difference in liver metastasis rates between these two cohorts (p=0.087). Moreover, the presence of liver metastasis was significantly associated with male gender, carcinoma located at the pancreatic tail, low differentiation histology, larger tumors, more advanced T stage, LN metastasis and distant metastases at other sites.

Table 1

The comparison of demographic and clinicopathological factors between training cohort and validation cohort

Characteristic		Whole cohort	Training cohort	Validation cohort	P-value	Training cohort				Validation cohort
						Liver metastasis		N	P-value	Liver metastasis		N	P-value
Absent	Present	Absent	Present
		12,644	9,483	3,161		8,276	1,207	9,483		2,721	440	3,161
Age	≤65	5,769	4,314	1,455	0.606	3,741	573	4,314	0.146	1,244	211	1,455	0.410
>65	6,875	5,169	1,706	4,535	634	5,169	1,477	229	1,706
Gender	Femal	6,215	4,722	1,493	0.013	4,197	525	4,722	<0.001	1,307	186	1,493	0.027
Male	6,429	4,761	1,668	4,079	682	4,761	1,414	254	1,668
Tumor site	Head	9,197	6,900	2,297	0.995	6,295	605	6,900	<0.001	2,065	232	2,297	<0.001
	Body	1,740	1,304	436		1,028	276	1,304		347	89	436
Tail	1,707	1,279	428	953	326	1,279	309	119	428
Tumor grade	Well	1,346	1,013	333	0.851	959	54	1,013	<0.001	312	21	333	<0.001
	Moderate	5,476	4,123	1,353		3,667	456	4,123		1,190	163	1,353
Poor	5,822	4,347	1,475	3,650	697	4,347	1,219	256	1,475
Tumor size (cm)	≤2	1,341	1,017	324	0.330	953	64	1,017	<0.001	292	32	324	<0.001
	2~4	6,733	5,014	1,719		4,469	545	5,014		1,532	187	1,719
>4	4,570	3,452	1,118	2,854	598	3,452	897	221	1,118
T stage*	I	1,236	952	284	0.333	902	50	952	<0.001	258	26	284	<0.001
	II	5,800	4,324	1,476		3,884	440	4,324		1,324	152	1,476
	III	3,359	2,532	827		2,071	461	2,532		657	170	827
IV	2,249	1,675	574	1,419	256	1,675	482	92	574
LN metastasis	Absent	6,409	4,826	1,583	0.455	4,330	496	4,826	<0.001	1,409	174	1,583	<0.001
Present	6,235	4,657	1,578	3,946	711	4,657	1,312	266	1,578
N stage*	N0	1,972	1,494	478	0.246	1,397	97	1,494	<0.001	449	29	478	<0.001
	N1	3,189	2,421	768		2,299	122	2,421		728	40	768
N2	7,483	5,568	1,915	4,580	988	5,568	1,544	371	1,915
Lung metastasis	Absent	12,241	9,181	3,060	0.999	8,150	1,031	9,181	<0.001	2,683	377	3,060	<0.001
Present	403	302	101	126	176	302	38	63	101
Bone metastasis	Absent	12,523	9,392	3,131	0.993	8,245	1,147	9,392	<0.001	2,710	421	3,131	<0.001
Present	121	91	30	31	60	91	11	19	30
Brain metastasis	Absent	12,632	9,473	3,159	0.742	8,271	1,202	9,473	0.005	2,721	438	3,159	0.019
	Present	12	10	2	5	5	10	0	2	2

Note: *Based on the 8thTNM stage classification.

Abbreviation: LN, lymph node.

Nomogram construction

Univariate analyses were performed to filter risk factors. All these factors were entered into the multivariate logistic regression analyses. After a stepwise removal of variables, age, gender, tumor site, tumor grade, tumor size and LN metastasis remained significant predictors for liver metastasis (Table 2). All of the independent predictors in the training cohort were included for the nomogram construction (model 1, Figure 1). The nomogram demonstrated good accuracy, with C-indexes of 0.764 (95% CI=0.748–0.780) for the training cohort and 0.769 (95% CI=0.737–0.801) for the validation cohort. A patient’s probability of liver metastasis could be easily calculated by adding up the scores of each variable.

Table 2

Univariate and multivariate logistic analyses of liver metastasis in patients PDAC

Characteristic		Model 1						Model 2
		Univariate analysis			Multivariate analysis			Univariate analysis			Multivariate analysis
OR	95% CI	P-value	OR	95%CI	P-value	OR	95% CI	P-value	OR	95% CI	P-value
Age (years)	≤65/>65	0.857	0.766–0.959	0.007	0.794	0.675–0.934	0.005	0.857	0.766–0.959	0.007	0.768	0.650–0.907	0.002
Gender	Female/male	1.310	1.171–1.467	<0.001	1.225	1.042–1.441	0.014	1.310	1.171–1.467	<0.001	1.228	1.041.450	0.015
Tumor site	Head/body/tail	2.119	1.972–2.277	<0.001	1.816	1.642–2.008	<0.001	2.119	1.972–2.277	<0.001	1.714	1.545–1.902	<0.001
Tumor grade	Well/moderate/poor	1.730	1.586–1.888	<0.001	1.697	1.500–1.921	<0.001	1.730	1.586–1.888	<0.001	1.660	1.464–1.883	<0.001
Tumor size (cm)	≤2/2–4/>4	1.911	1.737–2.101	<0.001	1.471	1.221–1.771	<0.001	1.911	1.737–2.101	<0.001	1.422	1.177–1.719	<0.001
T stage*	T1/T2/T3/T4	1.520	1.421–1.626	<0.001	1.109	0.973–1.264	0.121	1.520	1.421–1.626	<0.001	1.108	0.970–1.266	0.131
LN metastasis	Absent/present	1.748	1.552–1.969	<0.001	14.877	7.332–30.187	<0.001	1.748	1.552–1.969	<0.001	14.624	7.029–30.426	<0.001
N stage*	N0/N1/N2	4.263	3.614–5.028	<0.001	0.643	0.413–1.001	0.051	4.263	3.614–5.028	<0.001	0.648	0.411–1.021	0.062
Lung metastasis	Absent/present							5.251	4.253–6.482	<0.001	3.135	2.182–4.505	<0.001
Bone metastasis	Absent/present							6.049	4.130–8.860	<0.001	3.054	1.609–5.794	0.001
Brain metastasis	Absent/present							5.283	1.545–18.071	0.008	1.214	0.061–23.986	0.898

Notes: *Based on the 8thTNM stage classification. Model 1: predictive model consisted of clinical and pathological factors; model 2: predictive model consisted of model 1 and additional distant metastatic sites.

Abbreviations: OR, odds ratio; CI, confidence interval; LN, lymph node.

Figure 1

Nomograms predicting risk of the liver metastasis in patients with pancreatic ductal adenocarcinoma (A, model 1; B, model 2).

Notes: Model 1: predictive model consisted of clinical and pathological factors; model 2: predictive model consisted of model 1 and additional distant metastatic sites.

Abbreviation: LN, lymph node.

Nomogram validation

The calibration curves demonstrated good agreement between the prediction and validation of the probability of liver metastasis in both the training and validation cohort (Figure 2A and B). The Hosmer-Lemeshow test yielded a non-significant P-value of 0.460. Furthermore, the ROC models of liver metastasis regarding the predictive ability were constructed (Figure 2C and D) and the resulting AUC values for the training and validation cohorts were 0.777 and 0.775, respectively.

Figure 2

The calibration plots and ROC curves of model 1 in the training cohort (A and C, respectively) and the validation cohort (B and D, respectively).

Notes: Model 1: predictive model consisted of clinical and pathological factors.

Abbreviation: ROC, receiver operating characteristic.

Incremental predictive value of distant metastases at other sites

Results for the predictive model after the addition of other sites of distant metastasis (model 2) are shown in Table 2. The nomogram (Figure 1B) of model 2 also showed good accuracy with C-indexes of 0.784 (95% CI =0.768–0.800) and 0.790 (95% CI =0.760–0.820) for the training and validation cohort, respectively. The Hosmer-Lemeshow test yielded a non-significant P-value of 0.579. Moreover, the calibration plots for estimating the probabilities of liver metastasis showed fair agreement between the nomogram-predicted probability and the actual observed metastatic status. The values of the AUC of the nomogram for predicting liver metastasis were 0.792 and 0.800, respectively (Figure 3).

Figure 3

The calibration plots and ROC curves of model 2 in the training cohort (A and C, respectively) and validation cohort (B and D, respectively).

Notes: Model 2: predictive model consisted of model 1 and additional distant metastatic sites.

Abbreviation: ROC, receiver operating characteristic.

The discrimination and calibration power of the two models were compared. It was observed that slightly higher C-indexes were obtained for model 2 in both the training and validation cohort as compared with model 1, although there were no significant differences. Moreover, the comparison of the likelihood ratio test indicated that compared with model 1, model 2 had superior discriminatory ability in predicting liver metastasis in both the training and validation cohort (Table 3).

Table 3

Comparison of the C-index values and likelihood ratio tests between two models

Nomogram	C-index	P-value	Log likelihood	Chi-square value	P-value
Training cohort
Model 1	0.764 (0.748–0.780)	0.068	−2532.900	166.980	<0.001
Model 2	0.784 (0.768–0.800)		−2449.400
Validation cohort
Model 1	0.769 (0.737–0.801)	0.111	−672.68	54.893	<0.001
Model 2	0.790 (0.760–0.820)		−645.24

Notes: Model 1: predictive model consisted of clinical and pathological factors; model 2: predictive model consisted of model 1 and additional distant metastatic sites.

Abbreviation: C-index, concordance index.

Clinical performance of two models

The decision curve analysis (DCA) is a novel method that evaluates predictive models from the perspective of clinical consequences. When the score is within the range 0–0.6, using the nomogram to predict liver metastasis adds greater net benefit than the treat-all or treat-none strategies. Moreover, compared with model 1, the increased net benefit of model 2 indicated that model 2 was more accurate in predicting liver metastasis in patients with PDAC in both the training and validation cohorts (Figure 4).

Figure 4

Decision curve analysis for models 1 and 2. The y-axis represents net benefit. The x-axis shows the threshold probability. “All” refers to the assumption that all patients have liver metastasis and “none” to the assumption that no patient had liver metastasis.

Notes: Model 1: predictive model consisted of clinical and pathological factors; model 2: predictive model consisted of model 1 and additional distant metastatic sites.

Discussion

Distant metastasis is a common sign of advanced stages and indicates poor prognosis for patients with PDAC. Liver metastasis is the most common site of metastases,15,16 which also indicates worse outcomes than patients with other sites of distant metastases.7 At present, the standard treatment for metastatic PDAC is chemotherapy, which can only provide limited benefit to increase the patient’s survival. However, it was reported that surgical approaches including liver resection might allow an improved survival in selective groups of PDAC patients with liver metastasis, especially in patients with oligometastasis.4,8 Thus, timely diagnosis of liver metastasis is extremely important, which may support the theoretical estimations and provide evidential recommendations to clinicians when making appropriate treatment decision. Unfortunately, the routine imaging examination, such as computed tomography (CT) or magnetic resonance imaging (MRI), does not show relatively high sensitivities and specificities in diagnosis of liver metastasis, especially minor metastasis in patients with PDAC.17,18 Liver biopsy was also reported to increase the risks for distant metastasis and reduce survival.19 Thus, a noninvasive method which can predict the probability of liver metastasis of patients with PDAC is needed. In this study, we have developed and validated the predictive nomograms using the SEER dataset, which demonstrated significant discrimination and calibration to provide evidence-based and individualized estimation for the probabilities of liver metastasis in patients with PDAC.

This study primarily focused on the clinical features of patients with liver metastasis, illustrated that patients with younger ages were more likely to have liver metastasis than those with older ages. This result was consistent with similar studies.20,21 In this study, it was shown that the probability of liver metastasis was higher in younger patients. Larger proportions of patients with younger ages (38%) had larger tumors than those with older ages (34.5%). Moreover, LN metastasis was more frequently observed in younger patients (55.0%), compared with older patients (47.0%). Compared with patients with older ages, patients with younger ages usually had tumors with higher malignancy or more aggressive histological features, which might ultimately lead to a higher tendency to liver metastasis or other forms of distant metastases.22,23 Consistent with studies of other kinds of tumors, Chang et al reported that early onset patients were much more vulnerable to positive circumferential margins, venous invasion and perineural invasion in colorectal adenocarcinoma.24 Multiple genetic alterations in younger patients also supported the hypothesis that tumor cells in the young were more susceptible to promoting DNA damage than in the old. As a result, young patients could be more likely to develop metastasis.25,26 Gender was associated with liver metastasis and female patients were less likely to have liver metastasis, which was similar to one study conducted by Lin et al.20 The significant association between clinical features and liver metastasis suggested that more focus should be made upon different clinical features of patients when liver metastasis was evaluated.

Apart from the clinical variables, the multivariate logistic analysis also revealed that tumor site, tumor grade, tumor size and LN metastasis were all independent predictors of liver metastasis in patients with PDAC. Recently, Dong et al reported a retrospective study that included 1,787 advanced PDAC patients.10 They demonstrated that both primary tumor location and diameter were significantly associated with the risk of synchronous liver metastasis. Tumor size, a powerful and reliable predictor of both distant metastasis and prognosis,5,27 was found to have an intimate association with the infiltration of the pancreatic cancer cells into the liver. Larger size tumors were more invasive and infiltrated to the peripheral organs or vessels; which may also signify greater tumor burden for patients with PDAC. Consistent with many previous reports,28,29 it was shown that the primary tumor at the body and tail of the pancreas was more prone to have liver metastasis as compared with those occurring at the head of the pancreas. In contrast with tumors located at the head of the pancreas, PDAC of the body and tail were larger or were more frequently diagnosed at advanced stages; possibly due to the lack of obstructive jaundice, which may result in higher risk of liver metastases among these patients. Meanwhile, our nomogram indicated that the magnitude of liver metastasis as tumor grade changed from well to poorly differentiated. Similar with studies which have shown that poor tumor grade was associated with worse prognosis,5,30 tumor grade, an inherent characteristic of tumor, which reflects the tumor biological behavior, was also found to be an independent predictor for liver metastasis in patients with PDAC. Moreover, LN metastasis was shown to weigh more than other pathological factors in predicting liver metastasis. LN metastasis was proved as a common sign before distant metastases,31,32 suggesting that greater attention concerning distant metastases should be paid in PDAC patients with LN metastasis. Regarding the common prognostic biomarkers, such as carbohydrate antigen 19-9 (CA19-9) and CA125, they were shown to be associated with tumor burden and advanced stages in patients with PDAC.27,33,34 In the study of Dong et al,10 it was likely to show that elevated level of CA19-9 was related to synchronous liver metastasis in stage IV PDAC patients. However, CA19-9 level was found not to be a risk factor of metachronous liver metastasis for stage III PDAC patients. Due to the unavailability of CA19-9 and CA125 in SEER database, the predictive value of these two biomarkers in predicting liver metastasis could not be evaluated while we believed that the evaluating of these two factors would increase the predictive power of the established nomograms in the further studies.

Apart from liver metastasis, there were also other sites of distant metastases, even though these patients consisted of only a small proportion of the whole cohort. In this study, over 60.0% of the patients with bone metastases had liver metastasis and 58.3% of the patients with lung metastases were diagnosed with simultaneous liver metastasis. This indicates that there is a high likelihood of multiple metastases being already developed by the time that distant metastases have been detected. This finding in our study was consistent with results from other studies,7,21 which revealed that patients with lung or bone metastases had a significantly higher number of metastatic sites. When other sites of distant metastases were included into this predictive system, lung metastasis was the greatest contributor to the risk of liver metastasis, followed by LN metastasis and bone metastasis. In addition, the new predictive model demonstrated a better discriminative ability to predict liver metastasis.

It is well known that patient counseling and decision are based on the prognosis estimated from the individual risk profiles.35 Nomogram is an important component of modern medical decision-making.27,36 In this study, the nomogram for predicting liver metastasis in patients with PDAC was established and demonstrated clinical reliable discriminatory data with relatively high C-indexes and values of AUC in the training and validation cohort. The calibration plots also demonstrated that the predicted probability of the nomograms corresponded well with the observed rates of liver metastasis. However, a higher value of C-index and perfect calibration does not guarantee that the established nomogram will be clinically useful. Therefore, we performed DCA and showed that the nomogram could be used to derive the net benefit within the derived probabilities.35 In addition, analysis on the basis of data from a population-based cohort made our results more generable than studies from a single center. This nomogram can, therefore, be used to estimate the individualized probabilities of liver metastasis and to guide personalized treatment for patients with PDAC.

There are several limitations in this study that should be noted. The major limitation of the present study is that the variables used to construct the nomograms, only represented some of the clinicopathological features. Some important tumor biomarkers were unavailable in SEER database. Also, we acknowledge that certain additional variables (eg, other pathological factors or molecular biomarkers) might provide potential predictive information. This is the major part of improvement in our future research. Another limitation is that although the established nomogram showed good discrimination and validation, further validation based on large-scale external cohort is needed.

Conclusion

In conclusion, we established a nomogram for predicting the risk of liver metastasis in patients with PDAC in this study. Physicians assess a diverse range of parameters of patients with more objectives and precision for predicting liver metastasis using this nomogram, resulting in reduced health care costs, less radiation exposure and fewer unnecessary diagnostic investigations.