Development and validation of nomograms for prediction of overall survival and cancer-specific survival of patients with Stage IV colorectal cancer.

BACKGROUND: Surgical resection of patients with resectable Stage IV colorectal cancer (CRC) is regarded as first choice if possible. However, its influence on overall survival (OS) has not been thoroughly explored. In this study, we aimed to construct nomograms to help predict 1-, 3- and 5-year OS rate and colorectal cancer-specific survival (CCSS) rate.
METHODS: A total of 2996 cases who underwent primary and metastatic resection were selected in the study from surveillance, epidemiology and end results (SEER) database. About 48 Stage IV CRC patients after resection from the Fudan University Shanghai Cancer Center (FUSCC) were assigned as an independent external validation group. Log-rank and multivariate Cox regression analysis were used. The competing-risks model was used to estimate the cumulative incidence of death. Nomograms were built for prediction of OS and CCSS after surgical resection in patients with Stage IV CRC.
RESULTS: The 1-, 3- and 5-year probabilities of OS were 76.6%, 41.4% and 23.2%, respectively. The 1-, 3- and 5-year colorectal cumulative incidence of death were 23.0%, 54.9% and 71.3%, respectively. The calibration curves for probability of 1-, 3- and 5-year OS and CCSS showed optimal agreement between nomogram prediction and actual observation, and the Harrell's C-indexes for the nomograms to predict OS and CCSS were 0.662 and 0.650, respectively. For FUSCC validation set, the C-index for this model to predict OS was 0.657.
CONCLUSION: Nomograms for prediction of OS and CCSS of patients with Stage IV CRC who underwent primary and metastatic resection were built. Performance of the model was excellent. These nomograms may be helpful for patients and physicians when making a decision.

Introduction

Colorectal cancer (CRC) was predicted to earn the fourth highest rate of incidence among all cancers in 2015 in the USA, with over 130 000 new cases expected (1). Unfortunately, 20% patients with newly diagnosed CRCs have presented with metastatic disease. The most common metastatic sites are the regional lymph nodes, liver and lungs. Since the venous drainage of the intestinal tract is via the portal system, the first site of hematogenous dissemination is usually liver, followed by lungs, bone and many other sites, including (rarely) brain.

Over the past 10 years, with the development of chemotherapy as well as the molecular target agents, in combination with improved surgical techniques, outcomes in patients with metastatic CRC have greatly improved. In an attempt to prolong the survival rate of patients with disseminated CRC, there is consensus that complete (R0) resection is the only curative option in the treatment of patients with CRC liver metastasis (2). What’s more, pulmonary metastasectomy has become a widely accepted practice when metastasis goes to the lung (3).

Five-year overall survival (OS) rates up to 58% have been reported by specialized centers after resection of the liver and primary lesion (4). However, surgery is accompanied with risks of postoperative morbidity and more commonly, high risk of recurrence. Postoperative complications following surgery for CRC are not only associated with poor short-term outcomes, but also with worse long-term outcomes (5). Serious postoperative morbidity includes bile leakage and associated peri-hepatic abscess in liver metastasis resection. As for recurrence, it is reported that ~60% of patients develop early second metastatic recurrence within 3 years after the first liver resection (6). Thus, perioperative care and appropriate surgery candidate selection is of great clinical significance, which would result in lower morbidity rates and better life quality. Therefore, it would be desirable to develop a practical and easy-to-use scoring system to screen in patients with better overall physical conditions to undergo the surgery process and hopefully, enjoy the benefit from it.

Since patients of Stage IV CRC might die of causes unrelated to cancer itself, as a result, OS might fail to accurately represent a patient’s long-term survival rate attributed to CRC. Thus, excluding other causes of death is necessary when estimating colorectal cancer-specific survival (CCSS).

Nomograms have been widely used as practical tools in clinical oncology for quantifying risks by taking important prognostic factors into account and models are built upon that, where overall probability of a specific outcome for any individualized patient can be calculated by adding scores up.

In this study, we aimed to develop a comprehensive nomogram involving a larger population with all situations of metastasis (not just the liver) to estimate the long-term OS and CCSS, thus providing information to patients for reference.

Patients and methods

Screening and data processing

Data were collected from the SEER program of the National Cancer Institute. The criteria of eligible patients were as followed: aged 18–85 years old at diagnosis; known time of diagnosis between 1 January 2004 and 31 December 2012; diagnosis with colorectal carcinoma as the first and only cancer diagnosis; diagnosis confirmed in a patient who is alive and not from a death certificate or autopsy; surgical treatment with both original and metastatic site resected; tumor grade is pathologically confirmed; staging is defined according to American Joint Committee on Cancer (AJCC) Seventh edition and Stage IV is included; histology confirmed as mucinous adenocarcinoma, signet ring cell carcinoma, adenocarcinoma except mucinous adenocarcinoma which was categorized as ‘other adenocarcinoma’ and others. Patients who were diagnosed before 2004 were not included because AJCC got the updated edition at 2004. Additionally, to make sure there is adequate follow-up time, we excluded the patients diagnosed after 2012.

A total of 2996 patients were included after the screening. American Indian/Alaskan Native and Asian/Pacific Islanders were recorded as ‘other’ under race. Age was transformed into categorical variables based on recognized cut off values. For the analysis based on carcinoembryonic antigen (CEA) level, we selected patients for whom the lab result of CEA was coded as positive/elevated or negative/normal in the database. As to create a group of patients with whom the positivity or negativity was known, we excluded patients whose CEA were coded as unknown/undetermined/ equivocal. We classified patients into two categories: (1) positive for CEA and (2) negative for CEA.

There were two external validation sets used to validate the nomogram, 749 patients from SEER data were selected as SEER validation cohort randomly. In addition, 48 CRC patients diagnosed between 2006 and 2018 who underwent both primary and metastatic resection from FUSCC were assigned as FUSCC validation cohort. None of these patients had synchronous benign disease or cancers. Patients without sufficient clinicopathological information, or patients suffered from more than one primary tumors, or those died within 2 months of surgery, were all excluded. Our study was approved by the Fudan University Shanghai Cancer Center (FUSCC) ethics committee.

Statistical analysis and construction of the nomogram

In order to validate a competing-risks nomogram, the selected SEER patients were divided into a training (n = 2247) cohort and a validation (n = 749) cohort randomly. Time expanding from tumor diagnosis to death was used to calculate OS. If date of last contact was after 2012, then date of last follow-up was calculated as 31 December 2012. Log-rank tests were applied to determine univariate prognostic factors. A multivariate Cox proportional hazards model was applied to estimate the independent effects of the univariate prognostic factors on OS. The independent prognostic factors determined by the multivariate analysis were used to construct the nomogram for OS. The cumulative incidence function (CIF) was used to assess the probability of colorectal cancer-specific mortality (CCSM) and death from other causes. CIF was calculated by Gray’s test between category groups. Deaths due to other causes were viewed as competing-risk events, which preclude the possibility of death resulting from CRC. In the Cox regression model analyzing the cause-specific regression, patients who died from other causes were excluded at the end of follow-up. Integrating the associated risk factors, nomograms were developed to predict the OS and cancer-specific survival (CSS) 1, 3 or 5 years after diagnosis. All P values are two-sided, and those <0.05 were considered statistically significant on the basis of the large number of patients.

Validation and calibration of the nomogram

To decrease bias, the nomograms were subjected to internal validation in the cohort and external validation in SEER validation cohort and independent FUSCC cohort, respectively. To create a calibration diagram, the marginal estimate versus model average predictive probability was used. In a perfectly calibrated model, predictive rates would fall on a 45-degree diagonal line. The interpretation of an index of probability of concordance (C-index) between predicted probability and actual outcome was used so as to evaluate the predictive ability and discrimination of the model. The value of the C-index should fall between 0.5 and 1.0, with 0.5 indicating random chance and 1.0 indicating a perfect discriminative ability. Identification of independent prognostic factors was conducted using Stata. The construction, validation and calibration of the nomograms were built on R version 3.1.2 software. The R packages cmprsk and rms and a C-index function for competing-risks model were used for modeling and developing the nomograms.

Results

The patients with Stage IV CRC in training cohort (N = 2247) and surveillance, epidemiology and end results (SEER) validation cohort (N = 749) were collected from the SEER cancer registry program. The SEER database covers ~26% of the US population, and the characteristics of the SEER population are comparable to the general US population. The flow chart for SEER data selection is shown in Fig. 1. The clinicopathological characteristics of selected patients are listed in Table 1. The median survival times of the training cohort and SEER validation cohort were 20 months (9–38 months) and 21 months (7–36 months), respectively. By the end of the last follow-up, 1876 (62.6%) patients of the entire population had died. The clinical pathological characteristics of FUSCC validation cohort are listed in Table 2. The median survival times of FUSCC validation cohort was 33 months (10–60 months). By the end of the last follow-up, 24 (50%) patients of the entire population had died.

Figure 1.

Flow chart for the surveillance, epidemiology and end results data screening.

Table 1.

Patients’ demographics and clinical characteristics SEER 2004–12 (n = 2996)

Variable	All patients		Training cohort		SEER validation cohort
	No.	%	No.	%	No.	%
	2996	100	2247	75	749	25
Race
White	2352	78.5	1779	79.17	573	76.5
Black	387	12.92	282	12.55	105	14.02
Others*	257	8.58	186	8.58	71	9.48
Age
18–39	184	6.14	140	6.23	44	5.87
40–59	1245	41.46	928	41.3	317	42.32
>60	1567	52.3	1179	52.47	388	51.8
Sex
Female	1526	50.93	1134	50.47	392	52.34
Male	1470	49.07	1113	49.53	357	47.66
T
T1	40	1.34	33	1.47	7	0.93
T2	96	3.2	74	3.29	96	3.2
T3	1878	62.68	1390	61.86	488	65.15
T4	982	32.78	750	33.38	232	30.97
N
N0	527	17.59	409	18.2	118	15.75
N1	1129	37.68	843	37.52	286	38.18
N2	1340	44.73	995	44.28	345	46.06
M
M1a	711	23.73	529	23.54	182	24.3
M1b	2285	76.27	1718	76.46	567	75.7
CEA
Negative	709	23.66	540	24.03	169	22.56
Positive	2287	76.34	1707	75.97	580	77.44
Grade
Well differentiated	99	3.3	78	3.47	21	2.8
Moderately differentiated	2077	69.33	1562	69.51	515	68.76
Poorly differentiated	712	23.77	526	23.41	186	24.83
Undifferentiated	108	3.6	81	3.6	27	3.6
Histology
Other Adenocarcinoma	2681	89.49	2010	89.45	671	89.59
Mucinous adenocarcinoma	264	8.81	202	8.99	62	8.28
Signet ring cell carcinoma	51	1.7	35	1.56	16	2.14

aIncluding American Indian or Alaska Native and Asian or Pacific Islander.

Table 2.

Patients’ demographics and clinical characteristics. FUSCC 2006–18 (n = 48)

Variable	FUSCC validation cohort
	No.	%
Total	48	100.00
Race
Othersa	48	100.00
Age
18–39	6	12.50
40–59	24	50.00
>60	18	37.50
Sex
Female	24	50.00
Male	24	50.00
T
T1	0	0.00
T2	4	8.33
T3	19	39.58
T4	25	52.08
N
N0	14	29.17
N1	21	43.75
N2	13	27.08
M
M1a	27	56.25
M1b	21	43.75
CEA
Negative	3	6.25
Positive	45	93.75
Grade
Well/Moderately differentiated	32	66.67
Poorly differentiated/Undifferentiated	16	33.33
Histology
Adenocarcinoma	39	81.25
Mucinous adenocarcinoma	9	18.75
Signet ring cell carcinoma	0	0.00

aAll patients are Chinese.

Factors associated with overall survival

We used Log-rank method to evaluate OS for these patients. The results showed that 1-, 3- and 5-year OS were 76.55%, 41.44% and 23.24%, respectively. In univariate survival analysis, except gender, these variables, including age, race, histology type, CEA level, TMN stage and grade were proved to be significantly correlated with OS (P < 0.001 for all except race and P < 0.05 for race). These prognostic factors that were identified as statistically significant in univariate analysis were further included in the multivariate analysis (Cox proportional hazards model) and were further confirmed to be independently associated with OS (Table 3). These factors were finally included in the nomograms and helped to build nomograms. A weighted total score added up from these variables was used to estimate and predict 1-, 3- and 5-year OS of patients.

Table 3.

Univariate and multivariate survival analysis for colorectal cancer overall survival (OS) in the training cohort. Surveillance, epidemiology and end results 2004–12 (n = 2247)

				Univariate analysis		Multivariate analysis
	1-Year OS (%)	3-Year OS (%)	5-Year OS (%)	Log-rank χ2	P	Hazard ratio	95%CI	P
All	76.55%	41.44%	23.24%
Sex				0.12	0.727
Female	75.76	41.63	23.09
Male	77.35	41.24	23.38
Age				85.7	<0.001
18–39	86.79	52.78	34.68			Ref.
40–59	85.45	50.48	29.49			1.22	0.95–1.56	0.122
>60	68.0	32.69	16.88			2.08	1.64–2.65	<0.001
T				52.44	<0.001
T1	90.30	62.68	37.97			Ref.
T2	88.79	59.41	27.32			1.1	0.63–1.92	0.746
T3	80.16	45.25	25.93			1.26	0.78–2.04	0.347
T4	67.65	31.01	16.83			1.67	1.02–2.71	0.040
N				52.08	<0.001
N0	82.49	55.13	30.07			Ref.
N1	79.78	44.64	27.39			1.22	1.04–1.43	0.014
N2	71.32	33.18	17.13			1.55	1.33–1.81	<0.001
M				23.09	<0.001
M1a	85.41	47.87	26.92			Ref.
M1b	74.12	39.57	22.21			1.44	1.22–1.7	<0.001
CEA				21.52	<0.001
Negative	79.39	51.23	31.73			Ref.
Positive	75.63	38.30	20.46			1.35	1.19–1.54	<0.001
Grade				55.23	<0.001
Well differentiated	82.95	46.53	22.43			Ref.
Moderately differentiated	80.98	45.99	25.44			0.98	0.74–1.29	0.864
Poorly differentiated	63.76	28.61	17.50			1.36	1.02–1.82	0.035
Undifferentiated	66.93	31.62	21.68			1.3	0.87–1.93	0.197
Histology				44.24	<0.001
Other Adenocarcinoma	77.8	43.25	24.48			Ref.
Mucinous adenocarcinoma	70.66	28.09	14.23			1.2	1.01–1.44	0.042
Signet ring cell carcinoma	41.9	14.37	4.79			1.46	1–2.12	0.050
Race				8.96	0.0113
White	76.28	42.38	24.43			Ref.
Black	76.18	33.65	14.75			1.31	1.13–1.54	<0.001
Othera	79.68	43.56	23.56			0.99	0.82–1.2	0.920

aIncluding American Indian or Alaska Native and Asian or Pacific Islander.

Cancer-specific survival and competing-risk analysis

At 1, 3 and 5 years after diagnosis, the cumulative incidences of death resulting from colorectal carcinoma (CIDCC) of the training cohort were 23.0%, 54.9% and 71.2%, respectively, while the cumulative incidences of death resulting from other causes were 2.48%, 4.7% and 6.1%, respectively. Estimates of probabilities of death resulting from CRC and other causes according to patient and tumor characteristics are listed in Table 4. Age and grade status showed significant associations with probability of death. A significant association between TMN stage and cumulative incidence of death was observed only within the CRC death cohort (P < 0.001). There was no statistical relationship between probability of death between different races and gender. All variables significantly correlated with CIDCC were used to build the nomogram to predict 1-, 3- and 5-year CSS rate.

Table 4.

1-, 3- and 5-year cumulative incidences of death among patients in the training cohort. SEER 2004–12 (n = 2247)

	Cumulative incidence of death resulting from colorectal cancer				Cumulative incidence of death resulting from other causes
	1-Year (%)	3-Year (%)	5-Year (%)	P	1-Year (%)	3-Year (%)	5-Year (%)	P
All	23.04	54.94	71.27		2.48	4.75	6.12
Sex				0.741				0.656
Female	24.1	55.0	71.8		2.1	4.5	5.7
Male	21.9	54.9	70.7		2.9	5.0	6.6
Age				<0.001				0.011
18–39	12.4	42.3	60.3		1.5	5.3	5.3
40–59	13.9	47.2	66.8		1.1	2.6	3.9
>60	31.5	62.5	76.0		3.7	6.3	7.9
T				<0.001				0.77
T1	6.5	30.1	50.0		3.2	7.2	12.0
T2	12.4	36.4	66.1		0	4.9	6.9
T3	19.4	51.6	69.2		2.4	4.3	5.5
T4	31.8	64.6	76.9		2.9	5.5	6.8
N				<0.001				0.106
N0	15.7	40.2	61.2		2.9	5.3	9.1
N1	20.2	51.7	67.5		2.1	4.8	5.8
N2	28.4	63.5	78.3		2.7	4.5	5.1
M				<0.001				0.525
M1a	14.7	47.3	67.8		1.9	5.9	5.9
M1b	25.3	56.9	72.4		2.7	4.6	6.0
CEA				<0.001				0.795
Negative	19.6	45.5	62.9		2.3	4.2	5.9
Positive	24.1	57.9	74.0		2.5	4.9	6.2
Grade				<0.001				0.027
Well differentiated	13.1	49.5	65.8		4.0	4.0	11.8
Moderately differentiated	18.5	49.9	68.7		2.4	5.2	6.4
Poorly differentiated	35.9	68.6	78.6		2.7	3.9	4.5
Undifferentiated;	35.1	66.9	76.3		1.2	3.1	3.1
Histology				0.005				0.88
Other Adenocarcinoma	22.0	53.0	69.9		0.025	0.050	0.063
Mucinous adenocarcinoma	28.4	70.6	82.5		0.016	0.016	0.034
Signet ring cell carcinoma	52.4	76.9	86.5		0.057	0.087	0.087
Race				0.258				0.742
White	23.3	54.1	70.1		2.7	4.8	6.2
Black	23.7	62.7	80.3		2.6	4.7	5.4
Othersa	19.7	51.9	69.7		0.6	4.5	6.8

Abbreviation: SEER = Surveillance, Epidemiology and End Results.

aIncluding American Indian or Alaska Native and Asian or Pacific Islander.

Nomogram

The nomogram for predicting 1-, 3- and 5-year OS were built based upon the reduced multivariate models in the training cohort (Fig. 1), while the nomogram for predicting the CSS was built upon competing-risk analysis (Fig. 2). For model validation, an external validation procedure was also adopted. As shown in Table 1, in the training cohort, the internal Harrell’s C-indexes for the nomograms to predict OS and CSS were 0.662 and 0.650, respectively (95% CI 0.647–0.676 and 0.635–0.666). In SEER validation cohort, the external C-indexes of OS and CSS were slightly higher: 0.666 and 0.669 (95% CI 0.639–0.693 and 0.641–0.697). In FUSCC validation cohort, the external C-index of OS was 0.657 (95% CI 0.544–0.770). The tiny difference between the internal and external validation C-index implied that these models were quite satisfactory. The internal calibration plots are presented in Fig. 3, revealing an excellent correlation in OS and CSS between the nomogram predicted and outcome observed. When it comes to the clinical application of this nomogram, we can take a patient who has recently been diagnosed of Stage IV CRC as an example. This patient is an African American who is 63 years old with positive CEA level. Biopsy showed poorly differentiated cells and signet ring cell type. CT showed hepatic metastasis and then the stage was evaluated as T3M1bN2. He wonders the prognosis of his situation and whether he could have both the primary site and liver lesion resected. As his physician, you refer to these two nomograms and add the individual scores up. The result shows that he has got 41.56 points and 41.31 points in OS and CSS nomogram. The predictive 1-year OS rate is slightly lower than 40%, and 1-year CSS rate is a bit lower than 50%. The difference between these two figures could possibly be explained by the serious complications due to surgery and thus, rendering lower OS compared with CSS, which explains why some physicians prefer conservative management rather than radical surgery. Furthermore, there is little hope that he can survive the third year, and 3-year CSS rate is 10%. Through this practical tool, the physician can stratify this patient and give him personal advice.

Figure 2.

Nomograms for (a) predicting 1-, 3- and 5-year overall survival (OS) and (b) colorectal cancer-specific survival (CCS) Instructions for use of the nomogram: First, assign the points of each characteristic of the patient by drawing a vertical line from that variable to the points scale. Then, sum all the points and draw a vertical line from the total points scale to the 1-, 3-, 5-OS or CSS to obtain the probability of death.

Figure 3.

Internal calibration plots. (a) 1-year, (b) 3-year and (c) 5-year overall survival (OS) nomogram calibration curves; (d) 1-year, (e) 3-year and (f) 5-year colorectal cancer-specific survival (CCSS) nomogram calibration curves. The dashed line represents a perfect match between the nomogram-predicted probability (x-axis) and the actual probability calculated by Kaplan–Meier analysis (y-axis). Closer distances from the points to the dashed line indicates better agreement between the predicted and actual outcomes.

Discussion

Patients with late stage CRC has very low median survival and 5-year OS rate if they did not receive any treatment (7). Resection is still considered as the preferred therapy for potential cure by most physicians (8). However, the definition and criteria for ‘resectable’ is still not well determined. There are plenty of researches discussing the extension of resection criteria (9,10). However, surgery could only be considered if, R0 resection is thought possible at the end of treatment. Those are the factors should be taken into consideration regarding whether to be operated on: the anatomical distribution of the disease; the residual functional volume of the metastatic organ; the management of the primary site; the timing and role of neoadjuvant chemotherapy, and whether all lesions can be resected successfully at one setting (11).

Several clinicopathological characteristics were proven independent prognostic factors for both OS and CSS in the present study, including age, histology type, grade, stage and CEA level. One exception is race. Race type has influence on OS, while does not affect CSS. Histology type exerts great effect on the survival rate. From the data on cumulative incidence of death, mucinous adenocarcinoma and signet ring cell cancer have far more percentage of death compared with other type of adenocarcinoma. In 1-year follow-up, the cumulative incidence of death of signet ring type is 52.3% compared with that of the other adenocarcinoma, which is 22.0%. When viewing three-year follow-up, the CIF increases to 76.9% among signet ring cell cancer, and 53.0% and 70.6% in adenocarcinoma and mucinous adenocarcinoma, respectively.

Previous data have described that CEA level has a sensitive indication of worse prognosis and that increase in CEA is an independent predictor of poor survival. In this study, positive CEA had a hazard ratio of 1.35 compared with negative CEA and showed distinct difference in cancer-specific mortality, which coincide with previous study. Fine and Gray modeling approach was adopted to construct models and the nomogram. The main advantage of the sub distribution methodology is that through simply model fitting we can see the direct effect of each covariate on cumulative incidence. The log-rank test and Cox proportional hazards regression were used for calculating the independent prognostic factors of OS. However, hazard ratios as obtained by cause-specific Cox regression analyses do not directly quantify the ability of the single markers to predict the unconditional absolute risk of an event of interest, failing to identify prognostic factors of CSS due to the possibility of biased results (12). Therefore, a competing-risks model was introduced. A competing risk was defined when the occurrence of an event either precludes the occurrence of another event under evaluation or altered the probability of occurrence for the other event (13).

Since many patients who were diagnosed of Stage IV CRC have the concern about whether or not running the risk of surgery in exchange for a longer life, there is great urge to build a scoring system for reference. There are several nomograms already developed associated with CRC, including but not limited to prediction of the oncological prognosis, the short-term outcome of treatments, such as surgery or neoadjuvant chemo/radiotherapy, and the possible future development of CRC (14,5,14–15). Since variables differ substantially between early stage and metastatic CRC, the nomogram usually focuses on only one aspect, either Stage I-III or Stage IV CRC (16).

There have been previous reports of nomograms predicting the prognosis after resection of liver metastases (15,17), which included both synchronous, metachronous liver metastases and a variety of clinical settings, with the C-indexes being ~0.6, relative low ones. Most of these models were built upon a relative small population, for example, 727 cases in Beppu T’s (18) nomogram and 578 cases in Kanemitsu et al's (19) nomogram.

Not until 2015 was the first nomogram predicting the prognosis of Stage IV CRC after curative resection developed. They built nomograms predicting the prognosis of Stage IV patients with metastasis to organs other than the liver, such as the peritoneum, lung or distant lymph nodes. In this study, Kawai (20) enrolled 1133 patients who had Stage IV CRC and underwent curative surgery from January 1997 to December 2007. He built a nomogram to predict disease free survival and OS with C-index of 0.62–0.64. His nomogram of prediction of OS rate included four variables: post-operation CEA level, T and N status and peritoneal dissemination. However, the majority patients are Asians in their study. What’s more, studies conducted in one single institution often do not have sufficient power to figure true prognostic factors for CRC because of complications and short follow-up periods. As to extend the usage of this predictive tool in the US population, we enrolled more candidates, wider variety of races and took two more variables into consideration, histology types and grade. The population-based SEER cancer registries have made the estimation of more prognostic factors based on a larger sample possible that reduce selection biases (21). Results from a population-based cohort are more reliable and are probably to be more applicable to larger population. Cumulative incidence of death resulting from CRC and other causes can be calculated based on the recorded cause of death. In conclusion, the strengths of our present study include the population-based design, long-term follow-up and sufficient sample size.

Despite advantages in our study, there are some limitations in our study. First, most patients once received chemotherapy or radiation, however, the type, number of chemotherapy/radiation regimens received and their response to treatment is not available in SEER database. Second, to ensure a sufficient follow-up period, this study included relatively old cases; what’s more to enroll as many patients as possible, the study period was relatively long. During this period, surgery technique has improved greatly, and bias might result.

In summary, we developed nomograms to estimate the probability of OS and CSS of Stage IV colorectal carcinoma after operation based on a large, population-based cohort with long-term follow-up. The nomograms have satisfactory performance in both the training and validation cohorts, and they are potentially effective tools for predicting the prognosis of Stage IV CRC with surgical resection of both primary and metastatic lesion. They will help clinicians to rule in individuals who could benefit most from the operation, thus providing more individualized treatment strategies.