S-GRAS score performs better than a model from SEER for patients with adrenocortical carcinoma.

Purpose: To externally validate the performance of the S-GRAS score and a model from the Surveillance, Epidemiology, and End Results (SEER) database in a Chinese cohort of patients with adrenocortical carcinoma (ACC).
Methods: We first developed a model using data from the SEER database, after which we retrospectively reviewed 51 ACC patients hospitalized between 2013 and 2018, and we finally validated the model and S-GRAS score in this Chinese cohort.
Results: Patient age at diagnosis, tumor size, TNM stage, and radiotherapy were used to construct the model, and the Harrell's C-index of the model in the training set was 0.725 (95% CI: 0.682-0.768). However, the 5-year area under the curve (AUC) of the model in the validation cohort was 0.598 (95% CI: 0.487-0.708). The 5-year AUC of the ENSAT stage was 0.640 (95% CI: 0.543-0.737), but the Kaplan-Meier curves of stages I and II overlapped in the validation cohort. The resection status (P = 0.066), age (P=0.68), Ki67 (P = 0.69), and symptoms (P = 0.66) did not have a significant impact on cancer-specific survival in the validation cohort. In contrast, the S-GRAS score group showed better discrimination (5-year AUC: 0.683, 95% CI: 0.602-0.764) than the SEER model or the ENSAT stage.
Conclusion: The SEER model showed favorable discrimination and calibration ability in the training set, but it failed to distinguish patients with various prognoses in our institution. In contrast, the S-GRAS score could effectively stratify patients with different outcomes.

Introduction

Adrenocortical carcinoma (ACC) is a rare and aggressive cancer that affects both children and adults, having an overall incidence of 1–2 cases/million per year. In most cases, cancer presents with steroid hormone excess (1). Currently, complete tumor resection is the only curative alternative for ACC, although retrospective studies (2, 3, 4) have revealed the potential of adjuvant therapy in improving the prognosis, such as mitotane. The 5-year and median overall survival (OS) of ACC is about 40% and 22 months, respectively (5, 6). Given the variable prognoses, the management of ACC remains a clinical challenge for surgeons and endocrinologists worldwide, and a precise prognostic assessment is instructive for selecting proper treatment and follow-up strategies.

Many methods have been proposed to evaluate the prognosis of ACC patients. However, limited prognostic tools have been developed and validated in a large cohort due to the scarcity of ACC. Several models, which were developed from the Surveillance, Epidemiology, and End Results (SEER) database comprising more than 1000 ACC patients, have shown promising predictive accuracy in American patients (5, 7, 8). Recently, a scoring system known as the S-GRAS score has been developed from a multicenter cohort comprising 942 patients from the European Network for the Study of Adrenal Tumors (ENSAT) group (9, 10). These tools focus on different clinical variables, but no comparison has been made in the same cohort.

In this study, we developed a model for predicting the survival of ACC patients after surgery using the SEER database, which was consequently compared with the S-GRAS score in an independent single-center Chinese cohort of ACC patients.

Materials and methods

Population and data source

The clinical information of patients who were diagnosed with ACC between 2004 and 2016 was obtained from the SEER database (18 Regs Research Data + Hurricane Katrina Impacted Louisiana Cases) using SEER*Stat 8.3.9.2 software. The ACC diagnostic code (primary site code: 74.0/74.9, ICD-O-3 code: 8370/3) was used as the inclusion criterion. The exclusion criteria were the following: (i) with unknown TNM stage or T0 disease; (ii) with bilateral tumors or unknown laterality; (iii) with unknown cause of death; (iv) without surgery. In addition, all the included patients were diagnosed with ACC and hospitalized in Ruijin Hospital (Shanghai, China) between 2013 and 2018. The detailed information is shown in Fig. 1. Review and analysis of the data from this Chinese cohort of ACC patients were approved by the ethics committee of Ruijin Hospital. All participants provided their written informed consent.

Figure 1

Flowchart illustrating patient selection in this study.

The clinical and histopathological characteristics (TNM stage, age at diagnosis, cancer-related symptoms, resection status of primary tumor, and Ki67 index) were collected through a review of the medical records. Disease monitoring was done through periodical cross-sectional imaging. Survival data, causes of death, and postoperative treatment were collected through outpatient clinical visits or by telephone. The S-GRAS score was calculated as a sum of the following points: tumor stage (1–2 = 0; 3 = 1; 4 = 2), grade (Ki67 index 0–9% = 0; 10–19% = 1; ≥20% = 2 points), resection status (R0 = 0; RX = 1; R1 = 2; R2 = 3), age (<50 years = 0; ≥50 years = 1), symptoms (no = 0; yes = 1), and categorization, which generated four groups (0–1, 2–3, 4–5, and 6–9) (9).

Development of the model from the SEER dataset

Univariable and multivariable Cox proportional hazards regression analyses were applied to select independent prognostic factors for cancer-specific survival (CSS) and develop a Cox regression model. Significant variables identified in the univariable Cox regression analysis were further selected by the backward stepwise method in the multivariate Cox regression model. The concordance index (C-index) was used to evaluate the discriminative ability of the model. Calibration curves were plotted by a bootstrapping method with 1000 resamples so as to assess the difference between the actual survival rate and the survival rate predicted by the model.

Survival risk classification based on the SEER model

In the training dataset, two cut-off values were found, and the patients were classified into three groups by the X-tile software (3.6.2). Then, the group with the most patients was further divided into two groups according to the cut-off, which could meet both of the following demands: (i) each group included more than 30 patients; (ii) the relative risk was kept at as high as possible level. Finally, all patients were classified into four groups. The Kaplan–Meier analysis was used to evaluate the CSS of ACC patients after surgery in these four risk groups.

External validation of the SEER model and the S-GRAS score

The multivariate Cox proportional hazards regression model from the SEER dataset and the S-GRAS score were applied to patients from the Chinese validation cohort. Besides, the Kaplan–Meier analysis was also used to evaluate the GRAS parameters, and a comparison was made by log-rank test. The time-dependent area under the curve (AUC) was used to evaluate the discriminative ability of the model and the S-GRAS score system.

Statistical analysis

All statistical analyses were conducted using the R software version 4.1.2 (https://www.R-project.org). CSS was computed from the date of diagnosis to the date of ACC-related death or the latest follow-up. All the categorical variables were presented with frequencies and proportions, and the continuous variables were presented with medians and interquartile ranges. Tumor size was converted into a categorical variable by X-tile. The chi-square test or Fisher’s exact test was used to compare binary variables and unordered categorical variables between two cohorts. Tumor sizes were compared using the Wilcoxon rank-sum test. Statistical significance was set at 0.05. R software ‘survival’ and ‘rms’ packages were used to develop a Cox proportional hazards regression model. R software ‘ggplot2’, ‘timeROC’, and ‘survival ROC’ package were used to plot receiver-operating characteristic (ROC) curves and calculate AUCs.

Results

Patient demographics

After excluding the patients without complete information, 366 ACC patients from the SEER database were included. Among 81 patients who were hospitalized in Shanghai Ruijin Hospital for ACC between 2013 and 2018, 51 patients with necessary data were finally included in the Chinese single-center validation set of the present study. The patients’ characteristics are shown in Table 1. Ki67, the resection status, and symptoms were not available in the SEER database. The GRAS score was 0 in 2 patients, 1 in 10 patients, 2 in 12 patients, 3 in 15 patients, 4 in 10 patients, and 6 in 2 patients.

Table 1

 Baseline characteristics of the investigated ACC patients in two cohorts.

Characteristics	SEER Training cohort, n  = 366 (%)	Chinese Ruijin validation set, n  = 51 (%)	P value
Age (years)			0.060
<50	153 (41.8)	29 (56.9)
≥50	213 (58.2)	22 (43.1)
Gender			0.410
Male	136 (37.2)	22 (43.1)
Female	230 (62.8)	29 (56.9)
Ethnicity			<0.001
White	296 (80.9)	0 (0.0)
Black	34 (9.3)	0 (0.0)
Other/unknown	36 (9.8)	51 (100.0)
Laterality			0.580
Right	173 (47.3)	22 (43.1)
Left	193 (52.7)	29 (56.9)
T stage			<0.001
T1/T2	202 (55.2)	41 (80.4)
T3/T4	164 (44.8)	10 (19.6)
N stage			0.105
N0	331 (90.4)	50 (98.0)
N1	35 (9.6)	1 (2.0)
M stage			<0.001
M0	294 (80.3)	50 (98.0)
M1	72 (19.7)	1 (2.0)
Tumor size (mm)			<0.001
≤86	134 (36.6)	32 (62.7)
87–75	184 (50.3)	17 (33.3)
>175	48 (13.1)	2 (4.0)
Radiotherapy			<0.001
No	301 (82.2)	30 (58.8)
Yes	65 (17.8)	21 (41.2)
Chemotherapy			0.029
No/unknown	189 (51.6)	18 (35.3)
Yes	177 (48.4)	33 (64.7)
Ki67 index(%)			—
0–9	—	10 (19.6)
10–19	—	17 (33.3)
≥20	—	24 (47.1)
Resection status			—
R0	—	47 (92.2)
Rx	—	2 (3.9)
R1	—	1 (2.0)
R2	—	1 (2.0)
Symptoms			—
No	—	27 (52.9)
Yes	—	24 (47.1)

ACC, adrenocortical carcinoma; SEER ,Surveillance Epidemiology, and End Results database.

Bold indicates statistical significance.

Prognostic performance of the SEER model from the SEER training cohort

Table 2 shows the findings of univariate and multivariate analyses in the SEER training set. Age at diagnosis, tumor size, TNM stage, and radiotherapy was identified as independent predictors for CSS in multivariate analysis. The Cox regression coefficients of all factors included in the model are also displayed in Table 2. The Harrell’s C-index of the model from the SEER cohort was 0.725 (95% CI: 0.682–0.768). The model demonstrated good discriminative ability (Fig. 2A and B) and favorable consistency between the predicted and actual survival (Fig. 2C) in the SEER cohort. The X-tile software was used to classify the patients into four groups, that is, low risk, medium to low risk, medium to high risk, and high risk, according to three linear prediction cut-offs (Fig. 3A and B); these four groups were able to distinguish patients with different prognoses in the SEER cohort (Fig. 3C).

Table 2

 Univariate and multivariate Cox regression analysis of cancer-specific survival in the training cohort.

Characteristics	Univariate Cox hazard analysis			Multivariate Cox hazard analysis
	HR	95% CI	P value	HR	95% CI	P value
Age
≥50 (vs <50)	1.425	1.011–2.008	0.043	1.359	0.957–1.928	0.086
Gender
Female (vs male)	0.770	0.550–1.079	0.129	—	—	—
Ethnicity
Black (vs White)	0.646	0.339–1.231	0.184	—	—	—
Other/unknown (vs White)	0.837	0.439–1.595	0.588	—	—	—
Laterality
Left (vs right)	0.825	0.592–1.150	0.255	—	—	—
T stage
T3/T4 (vs T1/T2)	3.063	2.166–4.331	<0.001	2.479	1.717–3.578	<0.001
N stage
N1 (vs N0)	3.371	2.088–5.443	<0.001	2.576	1.566–4.236	<0.001
M stage
M1 (vs M0)	2.741	1.902–3.951	<0.001	1.808	1.220–2.680	0.003
Tumor size
87–175 (vs ≤86)	1.689	1.137–2.508	0.009	1.391	0.931–2.077	0.108
>175 (vs ≤86)	2.385	1.438–3.955	<0.001	1.846	1.094–3.116	0.022
Radiotherapy
Yes (vs no)	0.589	0.355–0.979	0.041	0.576	0.346–0.959	0.034
Chemotherapy
Yes (vs no/unknown)	1.256	0.901–1.752	0.179	—	—	—

HR, hazard ratio.

Figure 2

Performance of the SEER model in the training cohort. (A) ROC curve of the model in the SEER cohort. (B) Time-dependent AUC curve of the model in the SEER cohort. (C) Calibration curve of the model in the SEER cohort.

Figure 3

Stratification of ACC patients according to the linear prediction of the SEER model. (A) Kaplan–Meier plots categorized by low-risk, medium risk, and high-risk groups according to the optimal linear prediction cut-off. The optimal CSS linear prediction cut-offs were determined as 1.21 and 1.56 by X-tile. (B) Kaplan–Meier plots categorized by two groups in the low risk group in (A). The CSS linear prediction cut-offs in the low-risk group were determined as 0.06. (C) Kaplan–Meier plots categorized by low risk, medium to low risk, medium to high risk, and high risk groups.

Prognostic performance of the S-GRAS parameters

Kaplan–Meier curves for CSS were plotted to see whether the parameters in the S-GRAS score could well differentiate patients with distinct outcomes in the Chinese validation cohort (Fig. 4A, B, C, D, and E). No statistically significant differences were found in CSS between different age (P = 0.68), Ki67 (P = 0.3), symptoms (P = 0.66), and resection status (P = 0.066). The 5-year AUC of the ENSAT stage was 0.640 (95% CI: 0.543–0.737), but the Kaplan–Meier curves of stages I and stage II obviously overlapped in the validation cohort. The median survival time of stage I patients was even shorter than that of stage II patients (65 vs 98 months).

Figure 4

Kaplan–Meier analysis of CSS stratified by each S-GRAS parameter. (A, B, C, D, and E) in the validation cohort.

Comparison between SEER model and S-GRAS score groups

In the validation cohort, ACC patients with different prognoses could not be well distinguished between stage I and stage II–III. However, the SEER model also failed to discriminate between patients at low risk and medium to low risk in the Ruijin cohort (Fig. 5A). In contrast, the S-GRAS score showed better prognostic stratification for patients in our institution (Fig. 5B and C).

Figure 5

Kaplan–Meier analysis of CSS stratified by the four SEER model groups. (A) Four S-GRAS score groups; (B) the Chinese cohort; (C) time-dependent receiver operating characteristic analysis comparing the predictive efficiency of the SEER model and S-GRAS score in the Chinese cohort.

Discussion

ACC is a life-threatening malignancy with a widely varying prognosis in individual cases. Consequently, it is necessary to develop risk-prediction tools for selecting appropriate therapeutic and follow-up strategies. Although previous studies have confirmed the effectiveness of biomarkers in predicting the mortality of ACC patients (11, 12, 13, 14, 15, 16), molecular analysis cannot be used as a routine examination due to high costs. Thus, clinicopathological information still has an important role in prognosis prediction for ACC patients. To the best of our knowledge, there are only two forecasting methodologies currently being widely used.

One of them is the model developed from the SEER database, which generally includes age, tumor stage, and surgery status (5, 7). Based on the SEER database, Kong et al. (5) reported that a model with TNM stage was developed prior to a model with AJCC stage or ENSAT stage, and their final model incorporated age as a continuous variable along with TNM stage, showing good performance in the TCGA database and a Chinese cohort. Whether adjuvant therapy and tumor size are predictors for ACC remains controversial; however, they are not significant predictors for OS or CSS in previous SEER-based studies (5, 7). Nonetheless, in the present study, we found that radiotherapy and tumor size were significant predictors for ACC patients after surgery and were therefore included in the final model. This inconsistency may be attributed to different populations analyzed in each study. For instance, Kong et al. (5) only chose surgical adult patients, Zhang et al. (7) chose all ACC patients, and we chose surgical patients of all age groups.

The other forecasting methodology is the GRAS parameter, which includes the grade (Ki67 index), resection status, age, and symptoms caused by tumors or hormone secretion. The recently proposed S-GRAS score system combines the GRAS parameter and ENSAT stage (9), where the latter is a classical method for predicting the prognosis of ACC, although it was not confirmed as an independent predictor in several cohorts (17, 18). Ki67 and the resection status have been consistently recognized as factors with significant prognostic power in most studies, although an interobserver variation of Ki67 was found in previous studies (9, 18, 19). According to a meta-analysis, the higher mortality risk of cortisol-secreting ACC was determined, while other hormones such as androgen do not appear to be connected with poor prognosis (20). Younger age has also been linked with better survival; however, its prognostic value is still under debate (5, 7, 8, 21, 22).

To the best of our knowledge, this is the first study that compared a Cox regression model developed from the SEER database with the S-GRAS score system in an independent cohort, thus making it possible to assess the generalizability of these two prediction tools. Although this validation cohort was from a single center in China, it is a relatively high-volume cohort due to the rarity of ACC. In the present study, we transferred continuous variables, including age and tumor size, into categorical variables to obtain a more stable model, and we explored the effect of adjuvant therapy in postoperative ACC patients from the SEER database. TNM stage, age, tumor size, and radiotherapy were included in the model in this study. Besides, we examined whether other social factors such as income, marital status, and insurance influenced the patients’ prognoses, finding no significant factors among them. Unlike the case in the training cohort (C-index: 0.725), the SEER model performed unfavorably in the validation cohort, and its 5-year AUC was only 0.598 (95% CI: 0.487–0.708), which may be due to the inherent differences between the Chinese and SEER patient populations. To be specific, compared with the SEER cohort, the validation cohort included Chinese patients with smaller tumors and fewer metastatic diseases, and the patients were more likely to receive radiotherapy or chemotherapy in the validation cohort. Interestingly, the ENSAT stage also performed poorly in this Chinese cohort, although it is often regarded as an independent predictor for the survival of ACC patients. This may be because two patients in stage I group with a high Ki67 index (50%) and positive symptoms died within 2 years after diagnosis. As for GRAS parameters, only the resection status was marginally significant to CSS, while other parameters failed. Nevertheless, the advantaged performance of the S-GRAS score implied that the Ki67 index, age, symptoms, and resection status could provide extra information and reduce the probability of misclassification by the ENSAT stage.

In our institution, we usually use several postoperative therapeutic interventions such as ablation, reoperation, immunotherapy, and targeted therapy to improve the prognosis of ACC patients; however, the diversity and complicacy of the treatments limited further subgroup analysis.

There are several limitations in the present study. First, the validation cohort was from a retrospective study in a single institution, and the number was limited due to the rarity of ACC, which could lead to potential bias in our results. Accordingly, prospective multicenter validation studies are necessary to evaluate better the prognostic value of the SEER model and the S-GRAS score. Second, as the present study only included surgical patients, further studies are required to explore ways for more effective discrimination of ACC patients under different conditions.

Conclusion

In the present study, we compared a model developed from the SEER database with the S-GRAS score in a Chinese cohort of ACC patients. Contrary to previous studies, the SEER model failed to distinguish patients with various prognoses in our institution. In contrast, the S-GRAS score effectively stratified patients with different prognoses. However, due to the relatively limited AUC of the S-GRAS score, further research is required to develop reliable biomarkers for more accurate prediction.

Declaration of interest

The authors declare that there is no conflict of interest that could be perceived as prejudicing the impartiality of the research reported.

Funding

This work did not receive any specific grant from any funding agency in the public, commercial, or not-for-profit sector.

Author contribution statement

All authors listed above have contributed sufficiently to be included as authors. All of the authors had full access to the data for this study and take full responsibility for the integrity of the data and the accuracy of the data analysis. Juping Zhao and Wenhao Lin conceived and designed the study. Danfeng Xu, Fukang Sun, Wei He, and Xin Huang performed the surgeries. Jialing Xie and Juping Zhao made and confirmed the pathological diagnosis. Wenhao Lin, Jiacheng Liu, and Juping Zhao collected and analyzed the data and drafted the manuscript. All of the authors revised the article critically, gave final approval of submission, and agreed to be accountable for all aspects of the work.