Contemporary Validation of a Nomogram Predicting Colon Cancer Recurrence, Revealing All-Stage Improved Outcomes.

BACKGROUND: The Memorial Sloan Kettering Cancer Center (MSK) colon cancer recurrence nomogram is a risk calculator that provides patients and clinicians with individualized prediction of recurrence following curative resection of colon cancer. Although validated on multiple separate cohorts, the nomogram requires periodic updating as patient care changes over time. The aim of this study was to evaluate the nomogram's accuracy in a contemporary cohort and modify the tool to reflect improvements in outcome related to advances in colon cancer therapy.
METHODS: A contemporary patient cohort was compiled, including consecutive colon cancer patients undergoing curative resection for stage I-III colon adenocarcinoma at MSK from 2007 to 2014. The nomogram's predictive accuracy was assessed by concordance index and calibration plots of predicted vs actual freedom from recurrence at 5 years after surgery.
RESULTS: Data from a total of 999 eligible patients with complete records were used for validation. Median follow-up among survivors was 37 months. The concordance index was 0.756 (95% confidence interval = 0.707 to 0.805), indicating continued discriminating power, but the calibration plot revealed that the nomogram overestimated recurrence risk. Recalibration of the nomogram by estimating a new baseline freedom-from-recurrence function restored the nomogram's accuracy.
CONCLUSION: The updated nomogram retains the original nomogram's variables but includes a lower baseline estimation of recurrence risk, reflecting improvements in outcomes for all stages of colon cancer, likely resulting from advances in imaging and integration of multiple treatment modalities.

Predictive models such as nomograms have become popular to assess risk of cancer outcomes because they provide a more refined estimate than the TNM staging system of the American Joint Commission on Cancer (AJCC) and the International Union Against Cancer (1). Although TNM staging is easy to implement because it relies only on tumor depth, number of positive lymph nodes, and presence of distant metastasis, its predictive accuracy is limited. Outcomes of patients with cancer of the same TNM stage vary considerably because TNM staging does not account for all prognostic variables (2).

Nomograms provide better predictive accuracy of outcomes, enabling individualized treatment and planning of postoperative surveillance and refining the design of clinical trials (3). One such model is the Memorial Sloan Kettering Cancer Center (MSK) colon cancer recurrence nomogram (published in 2008), which allows clinicians to more precisely predict an individual’s risk of colon cancer recurrence following curative resection (4). Based on a single-institutional cohort treated from 1990 to 2000, the nomogram estimates 5-year and 10-year freedom from recurrence (FFR) by integrating information on nine clinicopathologic features including age, tumor size, preoperative carcinoembryonic antigen, use of adjuvant chemotherapy, and several indicators of tumor invasiveness and spread. The nomogram was intended for easy adoption in clinical practice because it was based on commonly measured variables and is available online (https://www.mskcc.org/nomograms/colorectal). The nomogram was externally validated in cohorts from Australia (2000–2005, n = 134), the United Kingdom (1998–2003, n = 138), and China (1996–2008, n = 985), confirming its broad applicability (5–7).

Throughout the past decade, the management of colon cancer has advanced in terms of clinical staging, surgery, and adjuvant chemotherapy. High-quality imaging, including routine use of 64-slice, contrast-enhanced, helical computed tomography and magnetic resonance imaging, has improved staging by preoperative identification of small metastatic deposits (8). Adoption of a standardized, anatomically based surgical technique has increased the thoroughness of tumor resection and lymphadenectomy (9–12). Following prospective randomized trials demonstrating improved outcome, oxaliplatin-based adjuvant chemotherapy became standard for stage III colon cancer in 2004 (13). A better understanding of stage II risk factors has also led to an expanded use of chemotherapy in node-negative patients (14).

Considering the evolution of colon cancer care, the aim of this study was to assess the validity of the MSK colon cancer recurrence nomogram for a contemporary cohort and update the model to ensure continued predictive accuracy.

Methods

Patient Cohort

After approval from the institutional review board, prospectively maintained institutional databases were queried for patients undergoing curative resection for stage I, II, or III colon adenocarcinoma from 2007 through the end of 2014. All lesions were located from the cecum to the rectosigmoid (>12 cm from the anal verge). Exclusion criteria included history of treatment for malignant tumors within the last 5 years, recurrence prior to surgery, residual tumor following surgery, metastatic disease, and preoperative chemotherapy or radiotherapy.

Demographics, clinicopathologic variables, and follow-up were retrieved and manually reviewed via the electronic medical record, similarly to the development of the original nomogram.

Staging, Surgical Procedure, and Surveillance Protocol

Preoperative staging included contrast-enhanced computed tomography (CT) of the chest, abdomen, and pelvis and colonoscopy. Resection method was complete mesocolic excision with central vascular ligation, performed by specialized colorectal surgeons (9). Surgical approach (open, laparoscopic, or robotic) was determined at the surgeon’s discretion. Adjuvant chemotherapy was administered and surveillance performed in accordance with national guidelines (14). Surveillance included physical examination, interval history, and serum carcinoembryonic antigen testing at 3- to 6-month intervals for the first 3 years and then at 6-month intervals thereafter for a total of 5 years. Imaging, most frequently CT of the chest, abdomen, and pelvis with oral and intravenous contrast, was performed at least annually. Colonoscopy was typically performed at 1 year after surgery and then repeated every 3–5 years based on endoscopic findings. Recurrence was determined based on radiographic evidence (with or without biopsy), colonoscopy, and serum carcinoembryonic antigen.

Nomogram Concordance and Calibration

Recurrence-free time was defined from the date of surgery until the date of recurrence or last follow-up, and patients without recurrence were censored at last follow-up. The risk score (ie, linear predictor) based on the existing nomogram was calculated for each patient in the current dataset. Predictive accuracy was assessed by concordance index (15), which represents the probability that given two randomly selected patients, the patient who recurred first had a higher predicted probability of recurrence. Values are interpreted similarly to the area under the receiver operating characteristics curve, with 0.5 equaling random chance and 1.0 representing correct predictions for all patients (16). The nomogram was calibrated by plotting the nomogram-predicted recurrence-free probability vs actual recurrence-free rates based on Kaplan–Meier estimates.

Recalibration

The nomogram was recalibrated by refitting the nomogram model on the contemporary dataset and estimating a new baseline recurrence-free probability function at 5 years while keeping the linear predictor unchanged (17). The new 5-year predicted probability was verified by comparison with observed Kaplan–Meier estimates in a new calibration plot.

Other Statistical Analysis

Characteristics of patients in the 1990–2000 and 2007–2016 cohorts were compared using the Mann-Whitney U and χ2 tests for continuous and categorical variables, respectively. Recurrence-free time was estimated by Kaplan-Meier method and the statistical significance of between-group differences was assessed by log-rank test. All statistical analyses were performed using SAS version 9.4 (SAS Institute Inc, Cary, NC) and R version 3.2.4 (www.R-project.org). All tests were two sided, and P values less than .05 were considered statistically significant.

Results

The database query identified 1095 consecutive patients treated for stage I–III colon adenocarcinoma from 2007 through 2014 who met the inclusion criteria. Clinicopathologic features of patients in the 1990–2000 and 2007–2014 datasets are shown in Table 1. Compared with the 1990–2000 cohort, patients in the 2007–2014 dataset were slightly younger and had tumors with a higher proportion of moderate to poor (vs well) differentiation, lymphovascular invasion, perineural invasion, T3–4 (vs T1–2) disease, and N1–2 (vs N0) disease. The median number of lymph nodes evaluated was higher in the 2007–2014 cohort compared with the 1990–2000 cohort (22, interquartile range [IQR] = 17–30, vs 14, IQR = 9–21, P < .0001). A similar proportion of stage III patients received adjuvant chemotherapy in both cohorts, whereas a greater proportion of stage II patients received chemotherapy in the 2007–2014 cohort (25% vs 14%, P < .0001). Details of adjuvant chemotherapy administered to patients in the 2007–2014 cohort are shown in Supplementary Table 1 (available online). In the 2007–2014 cohort, 62.6% of stage II and 87.3% of stage III patients receiving adjuvant chemotherapy were given regimens including oxaliplatin.

Table 1.

Clinicopathologic features of patients in the 1990–2000 and 2007–2014 datasets*

Characteristic	1990–2000	2007–2014	P
	(n = 1320)	(n = 1095)
Age, y	69 (59–76)	65 (54–75)	<.0001
Sex			.90
Female	664 (50%)	554 (51%)
Male	656 (50%)	541 (49%)
Location			<.0001
Right	506 (38%)	434 (40%)
Transverse	131 (10%)	132 (12%)
Left	153 (12%)	83 (8%)
Sigmoid	401 (30%)	290 (26%)
Rectosigmoid	129 (10%)	156 (14%)
Differentiation			<.0001
Well	139 (11%)	29 (3%)
Moderate	1011 (77%)	884 (81%)
Poor	165 (13%)	179 (16%)
Lymphovascular invasion	212 (16%)	493 (46%)	<.0001
Perineural invasion	79 (6%)	282 (26%)	<.0001
T-stage			.019
T1	230 (17%)	159 (15%)
T2	247 (19%)	190 (17%)
T3	743 (56%)	630 (58%)
T4	100 (8%)	116 (11%)
N-stage			.003
N0	940 (71%)	711 (65%)
N1	271 (21%)	265 (24%)
N2	109 (8%)	119 (11%)
AJCC stage, fifth edition			.001
I	421 (32%)	286 (26%)
II	520 (39%)	425 (39%)
III	379 (29%)	384 (35%)
Preoperative CEA, ng/mL	3.1 (1.7 – 6.7)	3.1 (2 – 5.9)	.33
	range = 0.2– 798	range = 0.4–210.6
No. of positive nodes (N1/2)	2 (1–4)	2 (1–4)	.576
No. of negative nodes	13 (8–20)	21 (16–29)	<.0001
	range = 0–146	range = 2–78
No. of lymph nodes evaluated	14 (9 – 21)	22 (17 – 30)	<.0001
>12 lymph nodes evaluated	807 (61%)	1064 (97%)	<.0001
Stage I	187 (44%)	271 (95%)	<.0001
Stage II	343 (66%)	417 (98%)	<.0001
Stage III	277 (73%)	376 (98%)	<.0001
Postoperative chemotherapy
Stage I	8 (2%)	0 (0%)	.019
Stage II	72 (14%)	107 (25%)	<.0001
Stage III	320 (85%)	331 (89%)	.14

*Continuous data are presented as n (interquartile range), with ranges below when they differ, and categorical data as n (%). AJCC = American Joint Commission on Cancer; CEA = carcinoembryonic antigen.

Kaplan-Meier estimates for FFR stratified by AJCC stage (eighth edition) are shown in Figure 1; 120 experienced recurrence and 19 died without recurrence. Median follow-up among survivors was 37 months (IQR = 19.2–55.0). A stage-by-stage comparison of FFR between patients in the 1990–2000 and 2007–2014 cohorts is shown in Figure 2. The contemporary cohort had statistically significantly higher FFR in stage II (HR = 0.52, 95% CI = 0.35 to 0.77, P = .0011) and a trend toward higher FFR in stage III (HR = 0.77, 95% CI = 0.58 to 1.02, P = .064).

Figure 1.

Postsurgery freedom from recurrence in 1095 patients undergoing complete resection of nonmetastatic colon cancer from 2007 to 2014 according to American Joint Commission on Cancer substage (eighth edition).

Figure 2.

Postsurgery freedom from recurrence in patients in the 1990–2000 and 2007–2014 cohorts according to American Joint Commission on Cancer stage. CI = confidence interval.

A complete set of data required for nomogram validation was available in 999 of the 1095 patients in the 2007–2014 dataset (Supplementary Table 2, available online). FFR in the validation cohort was similar to that in the entire cohort (Supplementary Table 3, available online); 113 had recurrence and 16 died without recurrence.

The concordance index of the nomogram for predicting 5-year FFR in the contemporary dataset was 0.756 (95% CI = 0.707 to 0.805), indicating good discriminating power to distinguish risk of recurrence among patients in the cohort (18,19). Although the concordance index was high, the calibration curve comparing actual with nomogram-predicted 5-year FFR demonstrated that the nomogram overestimated the risk of recurrence at all risk levels in the 2007–2014 cohort (Figure 3A). We adjusted the nomogram by estimating a new baseline FFR function using the current data but keeping the same linear predictors (Figure 4). After recalibration, the nomogram accurately predicted 5-year FFR (Figure 3B).

Figure 3.

Calibration curve for (A) the original nomogram and (B) the updated nomogram for predicting 5-year freedom from recurrence in the 2007–2014 cohort (n = 999).

Figure 4.

Colon cancer nomogram for freedom from recurrence. Locate the patient’s preoperative level of carcinoembryonic antigen (in nanograms per milliliter) on the CEA axis. Draw a straight line up to the Points axis to determine how many points toward recurrence the patient should receive. Repeat this process for each of the remaining axes, drawing a straight line each time to the Points axis. Sum the points received from each prognostic variable and locate this number on the Total points axis. Draw a straight line from the total points down to the 3- or 5-year freedom from recurrence axis to ascertain the patient’s specific risk of remaining free from recurrence for either 3 or 5 years. CEA = carcinoembryonic antigen; L = left colon; R = right colon; RS = rectosigmoid colon; Sig = sigmoid colon; TC = transverse colon.

Discussion

We have updated the MSK nomogram for predicting recurrence of colon cancer after complete resection using data from a contemporary cohort of patients. Because the original nomogram remained prognostic based on concordance index but overestimated the risk of recurrence, we recalibrated by decreasing the baseline recurrence risk function.

The improvement in outcomes we observed relative to the 1990–2000 period likely results from multiple advances. Enhanced imaging capable of identifying small metastatic deposits improves the accuracy of staging for advanced disease. Multidetector helical CT increased the sensitivity of CT for detecting liver metastasis, from 62%–74% in the 1990s using nonhelical CT (20–22) to 80%–90% (23–25). The addition of liver magnetic resonance imaging to evaluate questionable lesions seen on CT further improves the sensitivity to 95%–97% (23,26), which is particularly helpful for evaluating subcentimeter lesions (27).

Another likely factor contributing to improved outcome is adoption of complete mesocolic excision with central vascular ligation. As described by Hohenberger et al. (9), the procedure is an anatomic-based resection of tumor and locoregional lymphatic drainage that does not breach the visceral fascia, avoiding tumor spread within the peritoneal cavity and ensuring complete lymphadenectomy (28). The Erlangen group reported complete mesocolic excision to be most beneficial for stage III patients, improving 5-year cancer-related survival from 61.7% to 80.9% (29). This is consistent with the present study in that absolute improvement was the most evident in the highest-risk patients. Nonetheless, the Danish Colorectal Cancer Group found that mesocolic excision was independently associated with improved outcome irrespective of stage (10).

A third potential factor is increased lymph node yield, from a median of 14 in 1990–2000 to 22 in 2007–2014, related to increased adoption of more radical lymphadenectomy associated with complete mesocolic excision as well as increased lymph node harvest by pathologists (12,30). Evaluating more lymph nodes increases the likelihood of detecting nodal metastases (31) and may partially explain the higher proportion of stage III patients in the current cohort (35% vs 29% in the 1990–2000 dataset; P = .001). More accurate detection of metastatic lymph nodes also likely resulted in some stage migration from II and III and therefore higher rates of freedom from recurrence both for stage II patients and stage III patients.

Treatment advances also include the addition of oxaliplatin to 5-fluorouracil adjuvant chemotherapy. Most recent reports demonstrate an 11% relative improvement in 5-year disease-free survival in stage III patients treated with FOLFOX4 vs LV5FU (HR = 0.78, 95% CI = 0.65 to 0.93, P = .005) (32). The majority of patients with stage III disease in the 2007–2014 cohort received oxaliplatin-based adjuvant chemotherapy.

Finally, the increase in the proportion of patients with stage II disease receiving adjuvant chemotherapy (from 14% to 25%) may have improved outcomes, although its benefit for these patients is controversial. This change likely reflects increased identification of high-risk features such as lymphovascular invasion and perineural invasion; assessment of these features has become standard since 2000 (30). Both the QUASAR study and a large meta-analysis found only small reductions in recurrence in stage II patients treated with adjuvant chemotherapy (33,34).

An important advantage of nomograms over AJCC staging is that they predict risk along a continuous scale rather than through categories. Despite expansion in the number of AJCC substages to 10 in the current eighth edition, heterogeneity still remains within each substage, as demonstrated by a histogram of estimated risk of recurrence by stage (Supplementary Figure 1, available online). Further, the order of AJCC substages does not correspond to relative risk; patients with stage IIIA colon cancer have better outcomes than those with IIB (Figure 1) (35). The nomogram’s more precise prediction of outcomes can help improve treatment decision making in clinical practice.

The risk recurrence calculator updated herein meets all methodological criteria outlined by the AJCC Precision Medicine Core (36). This is especially notable given that the majority of prognostic tools in colorectal cancer were shown in a recent review to be methodologically deficient (3). Another highly rated tool predicts recurrence in stage III patients and is based on the Adjuvant Colon Cancer End Points (ACCENT) database (37). We compared the concordance index of our nomogram to that of the ACCENT calculator using data from the current cohort. Although evaluated in a small number of patients, the nomograms had similar concordance indices (0.693 for ACCENT, 95% CI = 0.623 to 0.763; and 0.674 for MSK, 95% CI = 0.609 to 0.740) (Supplementary Table 4, available online).

The strengths of our study include the relatively large cohort of patients who underwent a standardized resection procedure. Other strengths include comprehensive histologic assessment by specialized pathologists and the availability of granular clinical and demographic information, including use of chemotherapy. However, the study is subject to the selection bias inherent in observational retrospective studies. Risk models developed in a specialty institution have the potential of limited applicability. However, the MSK Colon Cancer Recurrence Nomogram has been previously validated in a diverse set of cohorts (5–7) and likely remains generalizable. Assessing the model’s performance based on the calibration plot may be overly optimistic, as the same data were used to estimate the new baseline recurrence-free probability function. Further external validation of the recalibrated nomogram with contemporary data (including patients aged older than 75 years) is needed.

In summary, evaluating the MSK recurrence nomogram using a contemporary dataset demonstrated continued discriminating power and allowed an update to the baseline recurrence function. Improvements in outcome are encouraging and likely reflect advancements along the spectrum of care including staging, surgery, and adjuvant chemotherapy. As staging moves to prognostic models and risk calculators rather than classifiers and ordered risk strata (1,3,36), periodic updates are required to maintain predictive accuracy and model validity.

Funding

This work was supported by the National Institutes of Health (P30 CA008748).

Notes

Affiliations of authors: Department of Surgery (TK, YS, IHW, EP, JJS, GMN, JGG, PBP, JGA, MRW) and Department of Epidemiology and Biostatistics (MH, MG) and Department of Medicine (AC, RY, ZKS, NHS, AV, LBS) and Department of Pathology (JS, EV), Memorial Sloan Kettering Cancer Center, New York, NY; Department of Gastroenterological Surgery, Cancer Institute Hospital, Japanese Foundation for Cancer Research, Tokyo, Japan (TK); Division of Digestive and General Surgery, Niigata University Graduate School of Medical and Dental Sciences, Niigata, Japan (YS).

J. Joshua Smith has received travel support from Intuitive Surgical and is an advisor for Endogenesis, Inc. José G. Guillem is a member of the Roche Speaker Bureau. Andrea Cercek has received research funding from AbbVie, Amgen, and Seattle Genetics and serves on the advisory board of Bayer. Neil H. Segal has received research funding from Roche/Genentech, Pfizer, Merck, Bristol-Myers Squibb, MedImmune/AstraZeneca, and Incyte and serves on the advisory boards of Roche/Genentech, Merck, Bristol-Myers Squibb, MedImmune/AstraZeneca, Boehringer Ingelheim, Pfizer, Pieris, PsiOxus, Synlogic, Aduro, Kyn Therapeutics, PureTech Ventures, Horizon Pharma, EMD Serono, Gritstone Oncology, Chugai, TRM Oncology, and IFM Therapeutics. Anna Varghese has received research funding from BioMed Valley, Verastem, Lilly, Bristol-Myers Squibb, and Silenseed. Leonard B. Saltz has received research funding from Taiho Pharmaceuticals.

We gratefully acknowledge Jessica Moore and Arthur Gelmis for editing the manuscript.

Click here for additional data file.