Development and Validation of a Prediction Model for Predicting the Prognosis of Postoperative Patients with Hepatocellular Carcinoma.

Purpose: The aims of this study were to identify the prognosis-related risk factors for HCC patients after surgery and to develop a predictive model by analysing the medical records of 152 HCC patients in our hospital. Patients and
Methods: Univariate Cox regression analysis was applied to identify potential risk factors for HCC patients after surgery and to determine independent prognosis-related risk factors by multivariate analysis. Subsequently, a nomogram model was developed based on all independent factors and was validated by a validation set. Calibration and receiver operating characteristic curves were employed to evaluate the accuracy of the model. Finally, decision curve analyses were used to assess its clinical utility.
Results: The univariate Cox regression analysis indicated that the patient's age, sex, grade, different AJCC TNM stages, vascular invasion, lymphatic infiltration, and tumour size were potential prognostic-related risk factors for HCC patients (p < 0.2), and the findings of multivariate analysis revealed that grade, different AJCC TNM stages, vascular invasion, and lymphatic infiltration were independent prognostic-related risk factors for HCC patients (p < 0.05). Subsequently, we constructed a prognosis-related prediction model based on all independent prognostic predictors and validated it with internal and external validation sets. The validation results indicated that the prediction model showed good accuracy (AUC = 0.81, 0.728) and consistency. More importantly, decision curve analysis illustrated that the nomogram model is a practical tool for predicting prognosis.
Conclusion: This study found that grade, different AJCC TNM stages, vascular invasion, and lymphatic infiltration were independent prognosis-related predictors for HCC patients after surgery, and a nomogram model built on these predictors exhibited great accuracy and clinical usefulness.

Introduction

Hepatocellular carcinoma (HCC) is one of the most common malignant tumours worldwide and the third leading cause of cancer-related deaths.1 Epidemiological studies have shown2 that HCC accounts for more than 80% of primary liver cancer cases, and 85% of HCC cases occur in low- and middle-income countries. Chinese epidemiological data indicate that the number of HCC patients in China has increased significantly after the age of 45, and the incidence rate in the population over 65 is approximately 175 per 100,000.3 Since HCC is primarily diagnosed when it is accompanied by liver function impairment, HCC is mostly in the middle and late stages, which results in a poor prognosis for patients. To better guide the treatment and assessment of prognosis of HCC patients, a variety of HCC staging systems have been developed,4–8 but among them, the Barcelona Clinic Liver Cancer (BCLC) staging system approved by the American Association for Study of Liver Disease (AASLD) and the European Association for the Study of Liver Disease (EASL) is the most commonly used HCC staging system.9 The BCLC staging system combines the severity of liver cirrhosis, tumour burden, and patient performance status and has advantages in evaluating prognosis and treatment allocation.10 According to the BCLC staging system, surgical resection is recommended for patients with early HCC (stage 0 and stage A); for patients with intermediate HCC (stage B) and advanced HCC (stage C), transcatheter arterial chemoembolization (TACE) and sorafenib are recommended. However, the main limitation of the BCLC staging system lies in the heterogeneity of the prognosis of patients with different conditions in each stage, especially in patients with advanced stages. In recent years, studies have focused on expanding the indications for surgical resection of HCC in the high-incidence areas of Asia, and it has been proven that surgical resection can improve the survival rate more than other palliative treatments.11 Some studies have found that surgery to treat HCC patients in the middle and late stages has the same effect as TACE.12 According to the 2020 European Society for Medical Oncology (ESMO) guidelines, sorafenib, lenvatinib, and a combination of atezolizumab plus bevacizumab are three different First-line therapeutic options.13 It was reported that there was significantly longer overall survival and progression-free survival in patients treated with atezolizumab plus bevacizumab.14 Additionally, according to the results from early clinical trials, lenvatinib plus pembrolizumab is a potential and novel treatment option for HCC patients.15 However, there are currently no exact predictive biomarkers for guiding treatment decision-making. Likewise, the prognostic evaluation system or prognosis-related predictors concerning HCC patients after surgery are still unclear at present. Given the accuracy and clinical usefulness of the nomogram model for HCC,16 this study aims to analyse the survival of HCC patients undergoing surgical treatment in our Hospital in the past 3 years, identify risk factors related to the prognosis of HCC patients, and build a clinical prediction model based on these factors to provide scientific evidence for the clinical prevention and treatment of HCC.

Materials and Methods

Inclusion and Exclusion Criteria

For the training set and internal validation set, patients with a pathological diagnosis of HCC and surgical treatment were included. We excluded patients who met the following criteria: ① received nonsurgical treatment; ② age at diagnosis older than 80 years old and younger than 35 years old; ③ declined to participate in the study; ④ refused multiple follow-ups; and ⑤ hospitalized patients before January 2018 and after December 2020. For the external validation set from the SEER database, patients with a pathological diagnosis of HCC and surgical treatment were included. We excluded patients with ① nonsurgical treatment or surgical treatment combined with other treatments; ② non-Asians; ③ age older than 80 years old and younger than 35 years old; and ④ cases diagnosed before 2017.

Data Source

This study collected the case data and follow-up results of 152 patients with HCC who underwent surgical treatment in the Department of Hepatobiliary Surgery in our hospital from January 2018 to December 2020. The surgical procedures for HCC were in line with Liu et al.17 The information includes the patient’s age, sex, tumour location, size, whether the tumour has invaded macrovascular invasion vessels, lymph node infiltration, AJCC TNM staging, BCLC staging and the degree of tumour differentiation. The outcome of the patient (mainly including death and recurrence) and the time of survival and recurrence were included in the follow-up data by telephone follow-up and outpatient review follow-up. The collected data of 152 HCC patients were randomly divided into a training set and a validation set at a ratio of 7:3. The training set data were used to construct the prediction model, and the validation set data were used to validate the accuracy and consistency of the prediction model. In addition, this study included 1136 Asian HCC patients who were surgically treated from the National Cancer Institute SEER (Surveillance, Epidemiology, and End Results Program) database () as an external validation set to externally verify the accuracy and consistency of the prediction model. The flow scheme is shown in Figure 1.

Figure 1

Research flow chart.

Development and Validation of the Prognostic Model

All independent prognostic risk factors were included in the multivariate Cox regression analysis to construct a prognostic model, and the model was visualized by a nomogram. To validate the consistency and accuracy of the model for predicting patient prognosis, the internal and external validation sets were used to evaluate the consistency of the model through calibration curves, and the receiver operator characteristic curve (ROC) was applied to validate the accuracy of the model prediction in this study. Finally, decision analysis was utilized to further evaluate the clinical applicability of the model. Decision curve analysis (DCA) conducted by the DecisionCurve package was employed to assess the clinical utility of the nomogram.

Statistical Analysis

Statistical analyses were performed by R software (version 3.63; ). All numerical variables in this study are shown as the mean ± standard deviation, and categorical variables are described by the frequency and percentage. The overall survival (OS) time was calculated from the date of surgery to the date of death from any cause or to the last follow-up date. Overall survival was summarized using Kaplan–Meier curves, and survival differences were assessed with the Log rank test by the survival package. Cox regression analysis was used to identify risk factors related to prognosis with the rms package. In the univariate Cox analysis, factors with P˂0.2 were defined as possible risk factors, and the possible risk factors were included in the multivariate Cox regression analysis. In the multivariate Cox regression analysis, factors with P˂0.05 were defined as independent prognostic risk-related factors.

Results

Basic Characteristics of the Patients

A total of 1288 HCC patients were included in this study, including 152 HCC patients in our hospital and 1136 from the SEER database. The data from our hospital were randomly divided into a training set (n=108) and a validation set (n=48). In addition, 1136 cases from the SEER database were used as the external validation set. The details of the baseline characteristics of HCC patients in this study are shown in Table 1.

Table 1

Baseline Characteristics of HCC Patients

Factors	Mean ± SD/N (%)
	Training Set	Internal Validation Set	External Validation Set
Age	58.40±10.86	57.21±12.05
≥ 50			1011 (89.0%)
< 50			125 (11.0%)
Sex
Male	30 (27.78%)	15 (34.09%)	827 (72.8%)
Female	78 (72.22%)	29 (65.91%)	309 (27.2%)
Tumour size			54.21±10.05
Grade
I	9 (8.33%)	3 (6.82%)	246 (21.65%)
II	22 (20.37%)	10 (22.73%)	590 (51.94%)
III	63 (58.33%)	25 (56.82%)	272 (23.94%)
IV	14 (12.96%)	6 (13.64%)	28 (2.46%)
T stage
T1	38 (35.19%)	12 (27.27%)	695 (61.18%)
T2	18 (16.67%)	9 (20.45%)	279 (24.56%)
T3	34 (31.48%)	14 (31.82%)	116 (10.21)
T4	18 (16.67%)	9 (20.45%)	46 (4.05%)
N stage
N0	102 (94.44%)	39 (88.64%)	1125 (99.03%)
N1	6 (5.56%)	5 (11.36%)	11 (0.97%)
M stage
M0	90 (83.33%)	39 (88.64%)	1112 (97.89%)
M1	18 (16.67%)	5 (11.36%)	24 (2.11%)
BCLC stage
A	83 (76.85%)	36 (81.82%)	–
B	15 (13.89%)	6 (13.64%)	–
C	10 (9.26%)	2 (4.54%)	–
Vascular invasion
+	56 (51.85%)	18 (40.91%)	454 (39.96%)
–	52 (48.15%)	26 (59.09%)	682 (60.04%)
Lymphoid infiltration
+	65 (60.19%)	15 (34.09%)	429 (37.76%)
–	43 (39.81%)	29 (65.91%)	858 (75.53%)

Univariate Cox Regression Analysis

As shown in Table 2, the univariate Cox regression analysis showed that the patient’s age (HR=0.99, p=0.15), grade (HR for III/IV=0.55, p=0.13; HR for II/IV=1.01, p=0.98; HR for I/IV=0.23, p=0.03), different AJCC TNM stages (T stage: HR for T2/T0-1=2.56, p=0.03; HR for T3/T0-1=2.09, p=0.04; HR for T4/T0-1=1.14, p=0.81), vascular invasion (HR for N/Y=0.41, p=0.005), lymphatic infiltration (HR for N/Y=0.16, p=0.000) and tumour size (HR=1.55, p=0.151) were potential risk factors related to the prognosis of HCC patients after surgery. In addition, the results of univariate Cox regression analysis showed that different stages of BCLC (HR for B/A=1.55, p=0.26; HR for C/A=3.11, p=0.99), sex (HR for male/female=0.98, p=0.97), AFP (HR =1.07, p=0.69), EOCG score (HR=1.14, p=0.50), CA199 (HR=1.00, p=0.33), Hb (HR=1.00, p=0.56), WBC count (HR=0.78, p=0.64), ALB (HR=1.01, p=0.70) and tumour location (HR for left liver lobe/right liver lobe=1.52, p=0.30; HR for other/right liver lobe=1.27, p=0.44) were not predictors of prognosis of HCC patients after surgery. These factors were thus excluded from the multivariate Cox regression analysis.

Table 2

Univariate Cox Regression Analysis Results

Factors		N	HR	95% CIs	P
Age		108	0.99	(0.98, 1.02)	0.15
Sex
	Female	45
	Male	63	0.98	(0.98, 1.02)	0.97
Grade
	IV	14
	III	63	0.55	(0.25, 1.19)	0.13
	II	22	1.01	(0.44, 2.32)	0.98
	I	9	0.23	(0.06, 0.88)	0.03
T stage
	T0-1	38
	T2	18	2.56	(1.12, 5.85)	0.03
	T3	34	2.09	(1.05, 4.18)	0.04
	T4	18	1.14	(0.41, 3.18)	0.81
N stage
	N0	102
	N1	6	2.10	(0.75, 5.89)	0.16
M stage
	M0	90
	M1	18	2.44	(1.27, 4.67)	0.007
BCLC stage
	A	83
	B	15	1.55	(0.72, 3.34)	0.26
	C	10	3.11	(0.00, Inf)	0.99
Vascular invasion
	+	56
	-	52	0.41	(0.22, 0.77)	0.005
Lymphoid infiltration
	+	65
	–	43	0.16	(0.06, 0.40)	0.000
Tumour size		108	1.55	(0.85, 2.80)	0.151

Multivariate Cox Regression Analysis

The following are the results of the multivariate Cox regression analysis: patient’s age (HR=0.99, p=0.065), grade (HR for IV/I=15.32, p=0.000; HR for III/I=7.89, p=0.008; HR for II/I=1.86, p =0.396), different AJCC TNM stages (T stage: HR for T2/T0-1=4.55, p=0.000; HR for T3/T0-1=5.81, p=0.00; HR for T4/T0-1=5.60, p=0.004; N stage: HR for N1/N0=3.31, p=0.027; M stage: HR for M1/M0=2.74, p=0.007), vascular invasion (HR for N/Y=0.36, p=0.005), lymphatic infiltration (HR for N/Y=0.12, p=0.000) and tumour size (HR=1.24, p=0.525). Of those, grade, different AJCC TNM stages, vascular invasion, and lymphatic infiltration were independent prognosis-related risk factors for HCC patients after surgery. It has yet to be determined whether the age of patients is an independent prognostic factor. Tumour size (HR=1.24, p=0.525) was not a risk factor related to the prognosis of HCC patients (Table 3).

Table 3

Multivariate Cox Regression Analysis Results

Factors		N	HR	95% CIs	P
Grade
	I	9
	II	22	1.86	(0.44, 7.80)	0.396
	III	63	7.89	(1.68, 17.0)	0.008
	IV	14	15.32	(3.35, 30.06)	0.000
T stage
	T0-1	38
	T2	18	4.55	(1.90, 10.87)	0.000
	T3	34	5.81	(2.45, 13.78)	0.000
	T4	18	5.60	(1.75, 17.85)	0.004
N stage
	N0	102
	N1	6	3.31	(1.14, 9.61)	0.027
M stage
	M0	90
	M1	18	2.74	(1.31, 5.74)	0.007
Vascular invasion
	+	56
	–	52	0.36	(0.17, 0.73)	0.005
Lymphoid infiltration
	+	65
	–	43	0.12	(0.04, 0.31)	0.000

Prognostic Nomogram of Overall Survival Statistical Analysis

In this study, based on the independent prognostic-related risk factors identified in the multivariate Cox regression analysis, a predictive model for the postoperative prognosis of HCC patients was constructed and visualized by a nomogram. As shown in Figure 2, the T stage and M stage in the AJCC TNM staging were the most critical factors affecting the postoperative outcome of HCC patients. Vascular invasion, grade, and lymphoid infiltration were also key prognostic factors for HCC patients after surgery. In addition, the effects of all dependent prognostic factors on the survival of patients were assessed by survival analysis and are shown by Kaplan–Meier curves (Figure 3).

Figure 2

Nomogram for predicting the 3-year and 5-year cancer-specific survival of HCC patients. Prognostic factors comprising age, tumour size, grade, vascular invasion, lymphoid infiltration, and various AJCC stages and scores for each level of every variable on the nomogram were determined by a vertical dot-line from that factor to the point scale. Therefore, a total point was obtained by summing all the values. Finally, the 3-year and 5-year cancer-specific survival for each patient could be estimated based on the total points. *p<0.05, **p<0.01, and ***p<0.001.

Figure 3

Kaplan–Meier curves for patients with HCC according to different independent prognostic factors. Kaplan–Meier curves for patients with HCC according to grade (A), AJCC T stage (B), vascular invasion or not (C), AJCC N stage (D), lymphoid infiltration or not (E), and AJCC M stage (F).

Validation of the Model

As shown in Figure 4, the internal and external validation findings of the prediction results showed that the model predictions for 3-year survival probability and for 5-year survival probability of HCC patients were close to the patients’ actual survival rates, indicating that the model prediction results were highly accurate. In addition, ROC curves were used to assess the discriminative property, and the areas under the curve (AUCs) in the internal and external validation sets were 0.81 and 0.728, respectively, indicating that the model had good discriminative ability (Figure 5A and B). Finally, the clinical usefulness and applicability of the model were evaluated by decision analysis. As shown in Figure 5C and D, the net benefit of the model was higher than the extreme assumption that all patients would have a positive result, indicating that the model has potential clinical usefulness as a nomogram.

Figure 4

Calibration plots of the nomogram model. (A and B) The calibration curves exhibited variance in the probability of 3-year and 5-year cancer-specific survival between the observed probability from Kaplan–Meier curves and the probability predicted by the nomogram in the internal validation set. (C and (D) The calibration curves exhibited a difference in the probability of 3-year and 5-year cancer-specific survival between the observed probability from Kaplan–Meier curves and the probability predicted by the nomogram in the external validation set.

Figure 5

The ROC curves and DCA plots. The ROC curves of the nomogram predicting 3-year cancer-specific survival in the internal validation set (A) and external validation set (B). The decision curves of the nomogram predicting 3- (C) and 5-year (D) cancer-specific survival. The solid purple line assumes that all patients experience cancer-specific death, whereas the red line on the plot assumes that none experience cancer-specific death. The blue line represents the net benefit of using the nomogram.

Discussion

HCC is one of the most common primary tumours, and it is also the most common cause of cancer-related deaths worldwide.1 In 2020 alone, the number of HCC patients increased by 900,000, and the number of deaths exceeded 800,000.18 Epidemiological studies have shown that HCC accounts for 90% of primary liver cancer, and its incidence in the global population increases sharply with patient age, reaching a peak at the age of 70.19–21 Chinese epidemiological studies have shown that 80% of HCC cases in China are caused by hepatitis B virus (HBV) infection, while in developed countries, most HCC cases are related to non-alcoholic fatty liver disease caused by metabolic disorders.22,23 This discrepancy led to differences in domestic and external treatment strategies and their prognostic evaluation. In developed regions, the selection of treatment strategies for HCC patients and the evaluation of patient survival were mostly based on the BCLC staging system. Of note, the prognosis-related predictors concerning HCC patients after surgery are still unclear at present. Therefore, the aim of this study was to identify prognosis-related risk factors for HCC patients and to build a predictive model based on the data of 155 HCC patients undergoing surgical treatment in our hospital’s hepatobiliary surgery unit from January 2018 to December 2020.

Studies have found that tumour macrovascular invasion is an important risk factor for the poor prognosis of HCC patients,24–26 which is consistent with our findings. Our study found that compared with patients with macrovascular invasion, patients without macrovascular invasion had a significantly lower risk of death (HR=0.27, p<0.000), indicating that tumour macrovascular invasion was a key factor leading to poor prognosis of HCC patients after surgery. Of note, the survival analysis findings also confirmed that there was a longer overall survival for HCC patients without macrovascular invasion than for those with macrovascular invasion (p<0.000). On the other hand, the results of our study also indicated that compared with patients with lymphatic infiltration, patients without lymphatic infiltration had a significantly lower risk of death (HR for N/Y=0.24, p=0.02), indicating that lymphatic infiltration of the tumour caused poor prognosis in HCC patients. Likewise, survival analysis found that the overall survival rate was greatly lower in patients with lymphatic infiltration or lymph node metastasis. Studies by Iguchi and others27 have shown that high expression of lncRNA-ATB was significantly related to tumour size, local invasion, lymphatic infiltration and vascular invasion, and the prognosis of HCC patients with high expression of lncRNA-ATB was poor. These findings indicated that the occurrence of lymphatic infiltration was a risk factor for the poor prognosis of HCC patients. In addition, studies from Chen28 and Xu29 have also confirmed that lymphatic infiltration is a risk factor for poor prognosis in HCC patients. In summary, our study confirmed that tumour macrovascular invasion and lymphatic infiltration are independent risk factors for poor prognosis in HCC patients after surgery.

The American Joint Committee on Cancer (AJCC) is currently the most widely used tumour classification system for cancer classification, prognosis definition, and optimal treatment decisions.30 Studies have shown that most malignant tumours have a better prognosis in the early stages of AJCC TNM staging, while the prognosis of patients in the late stages is poor.31,32 The results of this study showed that the prognosis of HCC patients in the T2 stage (HR for T2/T0-1=4.55, p=0.001) and T3 stage (HR for T3/T0-1=5.81, p=0.001) was worse than that in T0-1 stage HCC patients. Additionally, survival analysis indicated that the overall survival rate, compared with HCC patients in the T0-1 stage, was significantly reduced in patients in the T2-4 stages (p<0.001). Similarly, the prognosis of HCC patients in stage N1 (HR for N1/N0=3.31, p=0.027) was worse than that of HCC patients in stage N0. Likewise, the prognosis of HCC patients in stage M1 (HR for M1/M0=2.74, p=0.007) was worse than that of HCC patients in M0 stage. Correspondingly, survival analysis showed that the overall survival rate was significantly lower in patients with N1 or M1 stage disease than in those with N0 or M0 stage disease (p<0.000). In line with our results, the study from Wang et al also confirmed that high ANXA1 expression correlated with advanced TNM stage and poor survival of HCC patients, indirectly indicating that patients with advanced TNM stage often exhibited poorer prognosis.33 In brief, the postoperative prognosis of HCC patients in the late stages of AJCC TNM staging was worse than that in earlier stages.

Regarding the effect of tumour size on the prognosis of patients, thus far, there is still some controversy in the published literature. Some studies have reported that a small tumour size is an independent predictor of poor prognosis in tumour patients.34–36 However, other studies have demonstrated that large tumour size was a high-risk factor for poor outcomes in tumour patients.29–39 Interestingly, the findings of this study revealed that tumour size was not a prognosis-related factor in HCC patients after surgery. In line with this, the results from Yang et al indicated that tumour size was not always correlated with the overall survival of patients, while tumour origin was a critical factor for the prognosis of patients.40 Additionally, evidence indicates that age is a critical factor for the overall survival rate of tumour patients.41–43 According to our results (HR for age = 0.99, p=0.065), however, it is yet to be determined whether age is an independent predictor of outcome for HCC patients. We speculate that this was related to the limited sample size and age range included in this study. Beyond this, Yeşil Çınkır et al found that the survival rate of patients with a platelet/neutrophil ratio (PNR) value > 44.8 was significantly lower than that of patients with a PNR value < 44.8,44 and they also demonstrated that a low systemic inflammation response index (SIRI) value was related to increased survival in patients receiving sorafenib for HCC. In addition, Liang et al developed and validated the HCC risk score based on platelets (PLT), HDL, insulin, HOMA-IR, lymphocyte percentage, neutrophil percentage, sugar/insulin ratio (SIR) and lymphocyte/neutrophil ratio (LNR), which could effectively predict HCC occurrence in cirrhosis patients.45

Of note, it is generally accepted that the TNM staging system is most frequently used to predict the survival of tumour patients. However, the prognostic model has been widely used in the prediction and evaluation of the prognosis of various tumour patients in recent years.46–49 More importantly, the nomogram model showed higher accuracy than the TNM staging system.50 In addition, the nomogram model constructed based on existing clinical data can personally predict the prognosis of patients and has great clinical application value. Therefore, we developed and validated a predictive model based on HCC patients’ medical records and follow-up data and HCC patients’ data from the SEER database for predicting the prognosis of postoperative HCC patients, which showed great accuracy and consistency. The major strength of this model lies in the fact that, apart from the evaluation results of the widely used AJCC TNM staging, the model also includes other pathological characteristics of the patient, such as vascular invasion and lymphoid infiltration.

A few limitations of this study must be acknowledged. First, the total sample size of this study was limited; thus, the sample size in different subgroups was much smaller, which led to decreased statistical power. Second, we collected medical records from one hospital; therefore, the single-centre effects from a single-centre study cannot be excluded. Third, some critical factors, such as the PNR, LNR, and SIRI, were not included in this study. Despite these limitations, this study has strengths. We identified several prognosis-related predictors for HCC patients after surgery based on 108 HCC patients’ medical data and developed and validated a nomogram model for predicting the prognosis of HCC patients. More importantly, all risk factors included in this model are much easier to obtain than those determined in a costly and time-consuming manner.

Conclusion

In summary, this study identified that vascular invasion, lymphoid infiltration, and various AJCC TNM staging states were independent risk factors related to the prognosis of HCC patients. Additionally, we developed and validated a nomogram model based on those predictors that could accurately predict the prognosis of HCC patients after surgery. More importantly, the model could be applied to assist clinicians in assessing the prognosis of HCC patients.