Development and Validation of Nomograms to Predict Overall Survival Outcomes in Serous Ovarian Cancer Patients with Satisfactory Cytoreductive Surgery and Chemotherapy.

OBJECTIVE: Nomograms are statistics-based predictive tools that integrate predictive factors. Herein, a nomogram was developed and validated to predict the overall survival (OS) in serous ovarian cancer (SOC).
METHODS: Primary SOC patients with satisfactory cytoreductive surgery, chemotherapy, and OS ≥1 month were included in this study. A total of 6957 patients from the Surveillance, Epidemiology, and End Results (SEER) database comprised the training group and 1244 patients comprised the external validation group. The nomogram was structured on Cox models and evaluated in both the training and validation groups using consistency index, area under the receiver operating characteristics curve, calibration plots, and risk subgroup classification. Kaplan-Meier curves were plotted to compare the survival outcomes between subgroups. A decision-curve analysis was used to test the clinical value of the nomogram.
RESULTS: Independent factors, including age, tumor grade, and Federation of Gynecology and Obstetrics (FIGO) stage, identified by multivariate analysis in the training cohort, were selected for the nomogram. The consistency indexes for OS were 0.689 in the training cohort and 0.639 in the validation cohort. The calibration curves showed good consistency between predicted and actual 3- and 5-year OS. Significant differences were observed in the survival curves of different risk subgroups. The decision-curve analysis indicated that our nomogram was superior to the American Joint Committee on Cancer (AJCC) staging system.
CONCLUSION: A nomogram was constructed to predict the long-term OS in SOC and verified in Asians. The accurate predictions facilitated personalized treatments and follow-up strategies.

Introduction

Epithelial ovarian cancer (EOC) is one of the most common malignant tumors of the female genital organs. As a subtype of EOC, serous ovarian cancer (SOC) presents a distinct biological profile from other histological types.1 In 2002, Singer et al proposed a two-level classification system for SOC, wherein patients were divided into two subtypes: low-grade ovarian serous cancer (LGOS) and high-grade ovarian serous cancer (HGOS).1 In SOC, the binary theory of low- and high-grade disease is widely accepted because of the obvious differences in tumorigenesis, malignant biological behaviors, and molecular characteristics.2

HGOS is a type II and the most common type of ovarian cancer, accounting for approximately 90% of SOCs.3 No precancerous lesions were detected and were often found with extensive pelvic spread and high malignancy.4 LGOS is a type I ovarian cancer that accounts for approximately 10% of ovarian serous adenocarcinomas.3 These tumors grow slowly and are not sensitive to chemotherapy, but the clinical diagnosis is early, and the prognosis is satisfactory.5

During clinical diagnosis and treatment, the patients often asked gynecological oncologists: “If I have surgery and chemotherapy, how long will I live?” The AJCC staging system for ovarian cancer currently used internationally does not provide a good answer. The doctor has the ability to predict the probability of certain outcomes, which could modify the medical practice models and clinical decisions. Nomograms are statistics-based predictive tools that integrate pivotal predictive factors and are widely used to quantify risks and evaluate the prognosis of several cancer types.4,6,7 However, nomograms for patients with SOC have not yet been developed to the best of our knowledge. In this study, we aimed to construct nomograms using data extracted from the Surveillance, Epidemiology, and End Results (SEER)8 database to predict the prognosis of patients with SOC. Then, the model was verified externally to determine whether it provides accurate predictions of patient survival compared to the currently available staging systems.

Materials and Methods

Data Sources and Extraction

The training cohort of ovarian cancer patients was obtained from the SEER database, consisting of cancer patients’ data from 18 regional registries, covering approximately 34.6% of the total United States population.9 The cases of ovarian cancer (International Classification of Diseases [ICD]-O-3, primary site, C56.9: ovary) diagnosed between 2010 and 2016 were selected. The ICD-O-3 morphology codes “8441–8442 and 8460–8463” were used to identify women with SOC.10 Relevant information was extracted using SEER*Stat software version 8.3.6.

A retrospective study was conducted for the external validation cohort of patients who underwent cytoreductive surgery for ovarian tumors between January 2009 and June 2015 at Guangxi Medical University Tumor Hospital and Guangxi Zhuang Autonomous Region People’s Hospital in Guangxi Province, China. All patients underwent satisfactory cytoreductive surgery with residual tumor diameter <1 cm and paclitaxel combined with platinum chemotherapy.

Inclusion and Exclusion Criteria

The inclusion criteria for this study were as follows: no history of previous anticancer therapy, no history of other malignancies, satisfactory cytoreductive surgery, a pathological diagnosis of SOC (including LGOS and HGOS), the stage was not limited, at least three courses of paclitaxel combined with platinum-based intravenous chemotherapy were administered after surgery, and there was no age or race restriction. The exclusion criteria were as follows: no primary tumor, no histological confirmation, survival time <1 month, no surgery, no chemotherapy, and cases with incomplete information.

Study Design and Ethics

The final analysis consisted of 8201 individuals, of which the training cohort extracted from the SEER database (n=6957) was used for model development, and the remaining validation samples were used for external validation of model (n=1244). This study was approved by the ethics committee, and all patients signed the informed consent before surgery and chemotherapy.All procedures performed in studies involving human participants were in accordance with the ethical standards of the institutional and/or national research committee and with the 1964 Helsinki declaration and its later amendments or comparable ethical standards.

Follow-Up

The clinicopathological information and follow-up data were collected by four gynecologists with unified training. To ensure the accuracy of data entry, two specially trained gynecologists input the same medical records and established a database after checking the doubtful parameters. The SEER program included demographic data, stage of cancer at the time of diagnosis, and treatment information at the time of follow-up.

Variates and Outcomes

Variables were grouped to match with nomograms according to the clinical situation. The following factors were assessed at diagnosis: age (<50, 50–59, 60–69, 70–79, and ≥80 years), grade (LGOS or HGOS), and AJCC stage (I, II, III, or IV). OS was used as the primary endpoint and defined as the time from diagnosis to death or the last follow-up.

Statistical Analysis

We used standard model development and validation methods on training and external validation cohorts.11 The primary endpoint was OS. The categorical variables are shown as frequencies and proportions. The comparisons of clinicopathological characteristics between the training and validation cohorts were performed using the chi-squared test or Fisher’s exact test. Continuous variables were compared using the t-test or Mann–Whitney U-test for variables with an abnormal distribution. Next, univariate and multivariate analysis was conducted via Cox proportional hazard regression models in the training cohort to analyze the different prognostic variables associated with OS and construct a nomogram. Those with a P-value <0.05 by univariate analysis were included in the multivariate analysis.

A nomogram was drawn using the RMS26 package in R version 3.6.2 (; R Foundation, Vienna, Austria). The performance of the nomogram was validated internally and externally in the training and validation cohorts, respectively. The receiver operating characteristic (ROC) was used to evaluate the discriminative abilities of the nomograms. The larger the ROC, the more accurate the prognostic prediction.12 Calibration curves (1000 bootstrap resamples) were generated to test the consistency between the predicted and actual 3- and 5-year OS. Moreover, the whole cohort was regrouped into low- and high-risk groups, according to the median risk score predicted from the nomogram. Kaplan–Meier analysis and the Log rank test were used to explore the survival differences between the risk subgroups.

All statistical analyses were performed using SPSS version 25.0 (IBM Corp., Armonk, NY, USA) and R software version 3.6.2. P-values <0.05 indicated statistical significant.

Results

Patient Characteristics

The patient characteristics in the training (n=6957) and validation (n=1124) cohorts are listed in Table 1. A total of 8201 women with SOC were identified. As shown in Table 1, the majority of the cases (>85%) were diagnosed in women ≥50-years-old. About 84.4% of patients had HGOS, of which 77.8% were at an advanced stage.

Table 1

Demographics and Clinicopathological Characteristics of Patients with SOC

Demographic or Characteristic	All Patients (n=8201)			Training Cohort (n=6957)		Validation Cohort (n=1244)
	n		%	n	%	n	%
Age, years
<50	1079		13.2	1014	14.5	65	5.2
50–59	2163		26.3	1803	25.9	360	28.9
60–69	2813		34.3	2187	31.4	626	50.3
70–79	1576		19.2	1400	20.1	176	14.1
≥80	570		7.0	553	7.9	17	1.5
Grade
LGOS	1276		15.6	728	10.5	548	44.1
HGOS	6925		84.4	6229	89.5	696	55.9
AJCC stage
I	960		11.7	684	9.8	276	22.2
II	864		10.5	703	10.2	161	12.9
III	4418		53.9	3730	53.6	688	55.3
IV	1959		23.9	1840	26.4	119	9.6

Abbreviations: LGOS, low-grade ovarian serous cancer; HGOS, high-grade ovarian serous cancer; AJCC, American Joint Committee on Cancer staging system (7th edition).

Independent Prognostic Factors in the Training Cohort

Univariate analysis of the training cohort showed that age, grade, and AJCC stage were significant risk factors for OS (P<0.05) (Table 2). The independent prognostic factors in the final model were identified by multivariate analysis, wherein the three factors, age, grade, and AJCC stage, were independent and significantly associated with OS (Table 2).

Table 2

Univariate and Multivariate Cox Regression Analysis for OS in the Training Cohort

Variables	Univariate Analysis			Multivariate Analysis
	HR	95% CI	P-value	HR	95% CI	P-value
Age, years
<50	1			1
50–59	1.213	1.029–1.430	0.021	1.142	0.969–1.347	0.113
60–69	1.568	1.342–1.833	<0.001	1.437	1.230–1.680	<0.001
70–79	2.255	1.922–2.646	<0.001	1.959	1.668–2.300	<0.001
≥80	4.014	3.362–4.793	<0.001	3.934	3.292–4.702	<0.001
Grade
LGOS	1			1
HGOS	1.940	1.637–2.300	<0.001	1.393	1.173–1.653	<0.001
AJCC_stage
I	1			1
II	2.261	1.654–3.091	<0.001	2.065	1.509–2.824	<0.001
III	4.280	3.293–5.562	<0.001	4.124	3.169–5.367	<0.001
IV	7.521	5.772–9.800	<0.001	6.909	5.293–9.018	<0.001

Abbreviations: LGOS, low-grade ovarian serous cancer; HGOS, high-grade ovarian serous cancer; AJCC, American Joint Committee on Cancer staging system (7th edition); HR, hazard ratio.

Constructing Nomograms for OS

Prognostic nomograms for predicting 3- and 5-year OS were constructed independently based on the prognostic variables from the training cohort. The complex Cox regression formulas were transformed into visual graphics. The line segment corresponding to each variable was marked with a scale that represented the value range of the variable, and the length reflected the contribution of the factor to the outcome. The nomograms demonstrated that the AJCC stage contributed to the OS of SOC patients. Nomograms are commonly used to estimate the prognosis in oncology. Each variable was assigned a score of 0–100. To easily use the nomograms, one can draw a vertical line upwards from the specific points of each predictor to the “point” lines and then add up the corresponding points.13 Then, a straight line was drawn from the total points axis to the 3- and 5-year survival axis so that the surgeons could predict the probability of 3- and 5-year survival (Figure 1).

Figure 1

Nomograms for predicting the 3- and 5-year overall survival of patients with serous ovarian cancer in the training cohort.

Validation of the Nomograms

The nomograms were externally validated. C-statistic, ROC curves, and calibration curves were used to validate the model. These indexes were used to evaluate the accuracy of prediction models.

Values of the C-Statistic

The C-statistic ranged from 0.5 (absence of discrimination) to 1.0 (perfect discrimination). Typically, if the C-statistic value was >0.6, the model had a good predictive value.

In the training cohort, the C-statistic value for predicting OS was 0.689 (95% confidence interval (CI): 0.677–0.701). In the validation cohort, the C-statistic value for OS was 0.639 (95% CI: 0.601–0.670). Thus, the C-statistic values of the training and validation sets were similar.

ROC Curve Analyses

The ROC curve for OS in the training and validation cohorts was 0.675 and 0.661, respectively (Figure 2). The ROC curve values ranged from 0.5 (absence of discrimination) to 1.0 (perfect discrimination). Typically, if the ROC curve was >0.6, the model had a medium prediction value. The ROC curves of the training and validation sets were similar (Figure 2).

Figure 2

The ROC curve. The ability of the model to be measured by the AUC in the training cohort (A) and in the validation cohort (B).

Calibration Curves

The calibration curves indicated an agreement between the nomogram predictions and the actual survival outcomes in the training and validation cohorts. The x-axis represents the nomogram-predicted probability of each patient’s 3- and 5-year OS rates, while the y-axis represents the actual 3- and 5-year survival rates of each patient. If the blue line completely coincided with the black dashed line, the model was the most ideal. The agreement between the predicted vs observed probabilities was indicated by the calibration curves for 3- and 5-year OS (Figure 3).

Figure 3

The calibration curves for predictions of 3-year (A) and 5-year OS (B) in the training cohort, and 3-year (C) and 5-year OS (D) in the validation cohort.

Risk Stratification of SOC Patients and Decision-Curve Analysis (DCA)

A risk score of each variable was generated from the nomogram, and the total scores were calculated for all patients. The whole cohort was divided into low- and high-risk subgroups based on the median risk score. According to the survival curves shown in Figure 4A, significant differences were observed between the low- and high-risk groups for OS (P<0.001), which implied the risk stratification ability of the nomogram.

Figure 4

Kaplan-Meier cuves. OS for patients stratified by risk stratification system (A). DCA curve of the nomogram and AJCC stage for OS (B).

DCA indicated that the nomogram models make predictions and outperform the AJCC staging system (Figure 4B).

Discussion

In medical research, the tumor risk prediction models are used to predict the future incidence and prognosis of certain tumors. Specifically, the nomograms represent a means to establish a statistical model of the quantitative correlation between multiple risk factors and tumor occurrence and/or prognosis. The purpose of such models was to inform the patients of the risk of onset or prognosis, screen high-risk groups, and help doctors make clinical decisions. In 2003, van Zee et al14 first proposed a nomogram model to predict the risk of non-sentinel lymph node metastasis in sentinel node-positive breast cancer. The advantage is that it provides individualized prognostic risk assessments in the form of intuitive graphics, which have a definite value in clinical practice and provide a reference for individualized clinical decision-making. Such predictive nomograms are widely used in clinical oncology.15–17 However, a lack of external verification was the common limitation of these studies. External validation of cohorts from other countries or prospective randomized clinical trials is essential to confirm a model’s performance.

A subtype of EOC, SOC, accounts for approximately 85% of EOC diagnoses. Therefore, it is important to analyze the performance of nomograms independently with respect to the prognostic prediction of this EOC subtype. In this study, we used the external verification method and analyzed a training cohort of 6957 SOC patients from the SEER database and a validation cohort of 1244 SOC patients from two tertiary institutional hospitals to develop and validate the easy-to-use nomograms for predicting the 3- and 5-year OS. The current study identified the age, grade, and AJCC stage as independent predictors of OS. We also observed that the major predictors of improved OS were young age, early clinical stage, and well-differentiated grade, which is consistent with previous studies.18 In this study, the majority of cases (>85%) were diagnosed in women aged >50-years-old; the older the patient, the worse the prognosis. Typically, older patients were likely to present poor survival outcomes due to low immune responses.19

Several scholars speculated that clinical staging is a critical factor that affects the prognosis of ovarian cancer. In the current study, >70% of patients were diagnosed with advanced (III–IV) ovarian cancer. The later the clinical stage, the lower the 3- and 5-year OS rates. Patients with early Federation of Gynecology and Obstetrics (FIGO) staging could be more thoroughly removed after surgery, as the residual lesions were relatively small, chemotherapy-sensitive, and had a low risk of recurrence and metastasis, suggesting a good prognosis. Patients with late FIGO stage disease have tumor cells in the body that spread widely, making it difficult to implement complete surgical treatment. Patients with poor tolerance to chemotherapy have poor prognoses. Especially after stage IIIA, the later the stage, the higher the risk of death and the worse the prognosis.20

HGOS are the most common subtypes of EOC, and a majority of HGOS patients subsequently develop platinum resistance with relapse, indicating poor prognosis.21 In this study, 84.4% of patients were HGOS. Based on the nomogram, the prognosis of these patients was worse than that of LGOS. Thus, it could be speculated that tumors with low histological grades have a high degree of malignancy, such as rapid disease progression and poor survival and prognosis due to adverse biological behavior (rapid cell proliferation, diffusion, and strong invasion). The higher the degree of tissue differentiation, the slower the proliferation of tumor cells in the body. The weak invasion of body tissues was associated with a low degree of malignancy, slow disease progression, sensitivity to chemotherapy, prolonged survival time, and improved prognosis.

Herein, we validated the accuracy of our nomograms using the C-index, ROC analysis, and calibration curves in both the training and validation cohorts. The C-index and ROC curves were >0.6 for OS in the external verification processes. The calibration curves also demonstrated the optimal performance of the nomograms. These findings indicated a discriminatory performance and calibration of our nomogram. Previous studies have used the SEER database to establish nomograms that could predict the prognosis of EOC. For example, Wang et al developed and internally validated the nomograms to predict OS and cancer-specific survival (CSS) of EOC patients, achieving a C-index of 0.733 for OS and 0.747 for CSS.22 Chen et al constructed and validated the nomograms to predict OS and CSS in patients with ovarian clear cell carcinoma, achieving a C-index of 0.802 (95% CI: 0.773–0.831) for OS and 0.802 (95% CI: 0.769–0.835) for CSS. The nomograms were validated and found to be of good predictive value, which could aid in future clinical decision-making.23

The traditional AJCC staging system does not accurately assess tumor prognosis because it only includes a limited subset of critical prognostic factors. Several studies have reported better performance of nomograms than conventional staging systems and have proposed such models as promising tools for evaluating prognosis. Xu et al developed and validated a nomogram based on log of odds between the number of positive lymph nodes and the number of negative lymph nodes to predict the OS and CSS for EOC patients, achieving a C-index of 0.757 (95% CI: 0.746–0.768) for OS and 0.770 (95% CI: 0.759–0.782) for CSS. Additionally, this model performed favorably compared to the currently used FIGO model with concordance indices of 0.699 (95% CI: 0.688–0.710; P<0.05) and 0.719 (95% CI: 0.709–0.730; P<0.05) for OS and CSS, respectively. This study suggested that the nomogram may be superior to the currently used FIGO staging system for predicting OS and CSS in postoperative EOC patients.24 The nomogram developed in the current study also showed better predictive accuracy for survival compared to the AJCC 7th staging system. It also enabled the risk stratification of patients, thus facilitating personalized treatment plans and follow-up schedules. For example, a patient <50-years-old with AJCC stage IV has poor prognosis based on the AJCC staging system. Therefore, the patient may give up treatment due to the economic burden. However, based on our nomogram, this patient might have a better prognosis with a 5-year OS of >50%, which could promote the patient to cooperate with the doctor’s treatment plan.

Our study has several advantages. First, we developed and validated nomograms using clinically important long-term oncological OS outcomes, which reduced the bias by scoring the model performance using high-quality data based on the large sample sizes from SEER. Second, although some studies have established nomograms to predict the prognosis of EOC, the nomogram model of these studies has only undergone internal validation. In contrast, the present study adopted the method of external verification using cohorts from other countries to confirm the performance. Third, compared to the AJCC-stages, DCA curves in this study showed that our nomograms provided excellent clinical utility. All variables included in the SOC nomogram were obtained easily and applied in clinical practice. Furthermore, we aimed to establish a prediction model for general SOC patients with common characteristics, such that the results were not affected by ethnic and regional differences. Although the SEER database contains information from the US population, our external validation population was from China. The C-statistic and AUC values of the training and validation sets were similar, demonstrating the discriminatory power of the nomogram. It also showed that this nomogram was applicable to regions other than the USA, facilitating clinical application.

Nevertheless, the present study has several limitations. First, the included variables were simple, and detailed data, such as a family history of ovarian cancer, primary tumor diameter, positive lymph nodes, ascites cytological results, location of metastasis, chemotherapy regimens and cycles, sensitivity to chemotherapy, and genetic results, could have been included. Second, this study was based on retrospective data; hence, bias may have been introduced due to some undetected potential factors.

Conclusion

We used the SEER database to construct a nomogram of SOC prognosis by integrating various prognostic factors into a simple and intuitive tool and externally verifying the model in an Asian population to obtain universally applicable unbiased estimates. In the future study, we will include additional variables while constructing accurate predictive models of SOC prognoses.