Deep Learning-Based Prediction of Future Extrahepatic Metastasis and Macrovascular Invasion in Hepatocellular Carcinoma.

PURPOSE: For timely treatment of extrahepatic metastasis and macrovascular invasion (aggressive progressive disease [PD]) in hepatocellular carcinoma, models aimed at stratifying the risks of subsequent aggressive PD should be constructed. PATIENTS AND METHODS: After dividing 332 patients from five hospitals into training (n = 236) and validation (n = 96) datasets, non-invasive models, including clinical/semantic factors (ModelCS), deep learning radiomics (ModelD), and both (ModelCSD), were constructed to stratify patients according to the risk of aggressive PD. We examined the discrimination and calibration; similarly, we plotted a decision curve and devised a nomogram. Furthermore, we performed analyses of subgroups who received different treatments or those in different disease stages and compared time to aggressive PD and overall survival in the high- and low-risk subgroups.
RESULTS: Among the constructed models, ModelCSD, combining clinical/semantic factors and deep learning radiomics, outperformed ModelCS and ModelD (areas under the curve [AUCs] for the training dataset: 0.741, 0.815, and 0.856; validation dataset: 0.780, 0.836, and 0.862), with statistical difference per the net reclassification improvement, the integrated discrimination improvement, and/or the DeLong test in both datasets. Besides, ModelCSD had the best calibration and decision curves. The performance of ModelCSD was not affected by treatment types (AUC: resection = 0.839; transarterial chemoembolization = 0.895; p = 0.183) or disease stages (AUC: BCLC [Barcelona Clinic Liver Cancer] stage 0 and A = 0.827; BCLC stage AB &B = 0.861; p = 0.537). Moreover, the high-risk group had a significantly shorter median time to aggressive PD than the low-risk group (training dataset hazard ratio [HR] = 0.108, p < 0.001; validation dataset HR = 0.058, p < 0.001) and poorer overall survival (training dataset HR = 0.357, p < 0.001; validation dataset HR = 0.204, p < 0.001).
CONCLUSION: Our deep learning-based model successfully stratified the risks of aggressive PD. In the high-risk population, current guideline indicates that first-line treatments are insufficient to prevent extrahepatic metastasis and macrovascular invasion and ensure survival benefits, so more therapies may be explored for these patients.

Plain Language Summary

What is Already Known About This Subject?

► Extrahepatic metastasis or macrovascular invasion (aggressive progressive disease [PD]) could hinder treatment of hepatocellular carcinoma and reduce survival.

► Several promising methods have been explored to treat aggressive PD, and these alternatives should be performed as early as possible.

► Deep learning may be used to predict the risk of future aggressive PD and subsequently assist physicians in developing timely interventions.

What are the New Findings?

► Combining clinical/semantic factors and deep learning radiomics, ModelCSD showed good discrimination (AUCs in the training and validation datasets were 0.856 and 0.862) and calibration.

► The performance of ModelCSD was not affected by treatment type or BCLC stages. Moreover, the high-risk group identified by ModelCSD had a significantly shorter median time to aggressive PD and overall survival.

How Might It Impact on Clinical Practice in the Foreseeable Future?

►Based on ModelCSD, early detection, treatment, and prevention can be explored for the high-risk population.

Introduction

According to the latest global cancer statistics, liver cancer has one of the highest incidence and mortality rates, and 75–85% of liver cancer cases are hepatocellular carcinomas (HCCs).1 HCC patients who experience only intrahepatic progressive disease (PD) can undergo repeat liver resection or transarterial chemoembolization (TACE);2,3 however, treatment is more challenging for those who develop extrahepatic metastasis or macrovascular invasion (defined as aggressive PD). Once it occurs, aggressive PD could accelerate the deterioration of liver function and patients’ performance status, subsequently hindering the following treatments. Thus, although oncologists have explored several promising methods for patients with aggressive PD,4–6 these alternatives should be performed as early as possible.7–9 Predicting the risk of aggressive PD can assist in making a more reasonable follow-up plan, consequently providing an additional chance for early detection and treatment for high-risk population. Similarly, in locoregionally advanced nasopharyngeal carcinoma, scores stratifying patients according to the risk of future distant metastasis can be used to predict the patients who will truly benefit from early concurrent chemotherapy.10 Therefore, to perform timely interventions, models are needed to stratify HCC based on the risk of aggressive PD.

Although there are several studies on the mechanisms underlying HCC invasion,11,12 studies calculating integrated scores to predict aggressive PD based on clinical data are limited. Nevertheless, these studies show that microscopic changes occur long before aggressive PD. In addition, microscopic change in the level of cells or tissues (beyond the human visual system) can be analyzed by radiomics,13 which may be used to analyze clinical data and predict aggressive PD. Meanwhile, recently, deep learning radiomics has become a promising method that can advance medicine to a data-driven era;14 additionally, it has provided additional information about gastrointestinal cancer15 and liver disease16 that could not be gleaned from traditional methods. More importantly, researchers have shown the potential advantages of deep learning radiomics in differentiating between HCC and cirrhotic liver tissue.17 Thus, considering that HCC can be diagnosed and treated without biopsy,18,19 non-invasive deep learning radiomics is an ideal way to analyze the microscopic changes in HCC and subsequently stratify patients based on their future risk of developing aggressive PD.

Therefore, in this multicenter study, we constructed models that predict aggressive PD by combining clinical/semantic factors and deep learning radiomics. Through this process, we aimed to assist physicians in providing timely interventions for the high-risk population (Figure 1).

Figure 1

Study design. For hepatocellular carcinoma in early and moderate stages, we aimed to predict future extrahepatic metastasis and macrovascular invasion after treatment with liver resection or transarterial chemoembolization. First, we used clinical/semantic factors to construct ModelCS, used deep learning radiomics to construct ModelD, and combined all of them to construct ModelCSD. Second, we compared the three models using eight parameters (AUC, calibration, etc.) to identify the best model. Finally, we performed the subgroup analysis to test the robustness under different populations, as well as identify the threshold for high- and low-risk subgroups.

Methods

Patients

We recruited patients from five Chinese hospitals (). Patients initially diagnosed with HCC between April 2007 and November 2016 were included and followed-up until December 2019. The inclusion criteria were: 1) HCC diagnosed clinically or pathologically, 2) computed tomography (CT) performed at the time of diagnosis, 3) absence of extrahepatic metastasis or macrovascular invasion at the time of diagnosis, 4) initial treatment with liver resection or TACE per recommended guidelines,18,19 and 5) developed extrahepatic metastasis/ macrovascular invasion during treatment, or with no subsequent aggressive PD for at least one year unless death occurred. In contrast, the exclusion criteria were: 1) irregular follow-up visits and 2) the presence of other cancers. A flow chart of the patient selection process is shown in Figure 2. Next, patients were classified into stages according to the modified Barcelona Clinic Liver Cancer (BCLC) staging system per the latest guidelines (0, A, and B),18,19 Considering the controversy regarding the staging of a single HCC >5 cm, we defined such tumors as “stage AB” as discussed in the European guidelines.19,20

Figure 2

Flowchart showing patient selection for this study.

The study protocol conforms to the ethical guidelines of the 1975 Declaration of Helsinki as reflected in a priori approval by the ethical committee of Zhuhai People’s Hospital (approval number: 2021KT-4). The requirement for informed consent was waived due to the retrospective design of this study. All patient records and information were anonymized and de-identified prior to analysis.

Treatments and Follow-Up

The initial treatment option (liver resection or TACE) was decided by a multidisciplinary team based on the tumor characteristics, liver function, and patients’ treatment intention. For liver resection, the negative margin was confirmed pathologically. For TACE, super-selective embolization with lipiodol, platinum, and doxorubicin was performed under the guidance of digital subtraction angiography. During treatment, antiviral therapy using Entecavir or Tenofovir was administered to HBV-positive patients under the guidance of infectious disease specialist. Follow-up visits (including chest radiography, abdominal CT/magnetic resonance imaging, and necessary laboratory tests) occurred every 4–6 weeks for at least one year; this interval was lengthened 2-fold in the absence of PD.18,19 Patients with symptoms of extrahepatic metastasis underwent additional CT or MRI at the suspected anatomical region.

Endpoints

The primary endpoint was subsequent aggressive PD. Extrahepatic metastasis or macrovascular invasion was confirmed by two independent radiologists with over ten years of work experience via CT or MRI. In contrast, the secondary endpoint was overall survival (OS), which was calculated as the interval between initial treatment and the date of death.

Clinical and Semantic Factors Considered

The clinical factors we considered are listed in Table 1. Considering the potential multicollinearity between Child-Pugh class and BCLC stages, we performed a correlation analysis for them. The correlation coefficient was −0.076, with a p=0.243. Thus, both of them can be candidates for model construction.

Table 1

Baseline Demographics of Patients Included in the Study

	Training Dataset (N=236)	Validation Dataset (N=96)	p-value
Age	55.12±11.83	55.92±12.28	0.582
Sex			0.498
Male	197 (83%)	83 (86%)
Female	39 (17%)	13 (14%)
Initial treatment			0.891
Liver resection	153 (65%)	63 (66%)
TACE	83 (35%)	33 (34%)
HBV infection (N)			0.636
Negative	10 (4%)	3 (3%)
Positive	226 (96%)	93 (97%)
Cirrhosisa			0.763
Negative	99 (42%)	42 (44%)
Positive	137 (58%)	54 (56%)
Child-Pugh class (N)			0.369
A	200 (85%)	85 (89%)
B	36 (15%)	11 (11%)
BCLC stage			0.076
0	23 (10%)	13 (14%)
A	53 (22%)	29 (30%)
AB	94 (40%)	32 (33%)
B	66 (28%)	22 (23%)
Max diameter (mm)	60.65 (10–210)	51.50 (7–171)	0.042
Number of lesions			0.784
1	162 (69%)	66(69%)
2	31 (13%)	16 (17%)
3	19 (8%)	8 (8%)
>3	24 (10%)	6 (6%)
AFP level (ng/mL, N)			0.088
<25	89 (38%)	50 (52%)
25–400	70 (30%)	17 (18%)
>400	77 (32%)	29 (30%)

Note:aRefers to cirrhosis exhibiting morphological changes on computed tomography.

Abbreviations: AFP, alpha fetoprotein; BCLC, Barcelona Clinic Liver Cancer; HBV, hepatitis B virus; TACE, transcatheter arterial chemoembolization.

Additionally, we considered the following: 1) neutrophil-to-lymphocyte ratio; 2) HCC spatial location, including lobe (classified as left, right, or cross-sectional) and surface (whether lesions adjacent to the liver capsule were present); and 3) nine semantic factors, as detailed in our previous study,21 including fusion lesions, invasive shape, HCC capsule integrity, HCC capsule breakthrough, corona enhancement,22 corona with low attenuation, mosaic architecture,22 nodule-in-nodule architecture,22 and CT enhancement ratio of HCC. The following methods were used to assess these semantic factors: 1) semantic factors were assessed by two independent radiologists with over 10 years of experience (J. Z and J. Y); 2) when disagreement occurred, another independent assessment by a third radiologist with over 20 years of experience (X.H) was introduced; the final decision was made by two of the three radiologists. Intra-class correlation for the semantic factors ranged from 0.892 to 0.986.

CT Acquisition and Tumor Segmentation

The CT parameters used at each collaborating hospital are listed in . Since the HCC capsule can be visualized more clearly in the portal phase than in the arterial phase,22 thereby increasing the accuracy of segmentation, we used the portal phase for feature extraction. In case multiple lesions were present, we selected the target lesion according to the modified RECIST assessment depending on the size (ie, lesions with the longest diameter) and suitability for accurate and repetitive measurements.23

Model Construction

“ModelCS” was devised using a combination of candidate clinical and semantic factors, as described previously. Univariate analysis was first used to select the primary characteristics, and a stepwise elimination algorithm using Akaike’s information criterion (the step “AIC” function of the R package “MASS”) was performed to select relevant factors. The selected characteristics were subjected to multivariate logistic regression analysis, whose output was considered in the ModelCS score.

ModelD was constructed using a convolutional neural network model to predict aggressive PD. The model was constructed via 6 ResNet blocks, one flatten layer, and one dropout layer that was added to reduce over-fitting. Softmax was used as the dense layer activation function to determine the probability of aggressive PD. The structure of the model is detailed in Figure 3 and . A rectangular region of interest was created for the HCC lesion by an experienced radiologist on the portal phase contrast-enhanced CT. The rectangular box was expanded by 5 pixels to all sides and resized to 128×128 (pixels×pixels) to provide the input for the model. A deep learning score was calculated for every slice to predict the possibility of aggressive PD. The ModelD score was obtained by averaging all of a patient’s slice scores. Considering the unbalanced sample sizes of the aggressive PD and non-aggressive PD groups, additional positive slices were produced by adding Gaussian noise (mean=0, variance=0.1) to make two labeled slices (in a 1:1 ratio) in the training set. Data augmentations, including width/height-shift, shear, rotation, and zoom, were performed to reduce over-fitting. The model with the best performance in both the training and validation sets was used in the final model.

Figure 3

Illustration of the deep learning model architecture. (A) Structure of convolutional block (C_Block); (B) structure of ResNet block (R_Block); (C) overall structure of the model for aggressive-PD prediction, which contains one C_Block, six R_Block, one flatten layer, one dropout layer, and one dense layer. @16 represents 16 filters, and the number around the cube indicates the feature map size.

Finally, ModelCSD was built using multiple logistic regression, incorporating the scores of ModelCS and ModelD.

Statistical Analysis

Continuous variables were displayed as means (normal distributions) or medians (no normal distributions). Student’s t-test and the Wilcoxon rank-sum test were used to compare the continuous variables between the training and validation datasets. Categorical variables were expressed as percentages, and the Wilcoxon rank-sum test, Pearson’s χ2 test, or Fisher’s exact test was used to compare such variables between the training and validation datasets.

To determine the best model among ModelCS, ModelD, and ModelCSD, we first compared their discrimination ability using receiver operating characteristic (ROC) analysis, incorporating net reclassification improvement (NRI), integrated discrimination improvement (IDI), and the DeLong test, as well as calibration (evaluated using a calibration plot) and decision curve analysis (DCA). Subsequently, we constructed a nomogram using the best model. Next, we tested the performance of the best model using patients who underwent resection versus those who were treated with TACE, comparing the early stage (0 and A) and moderate-stage (AB and B) subgroups. Finally, we used the threshold identified by the best model to subcategorize the patients, and we compared the time to aggressive PD as well as OS using Kaplan-Meier plots and the Log rank test in the high- and low-risk subgroups.

A 2-sided p-value <0.05 was considered statistically significant, and we used the R statistical package () for all statistical analyses.

Role of the Funding Source

The funding sources were not involved in the study design, collection, analysis, interpretation of data, writing of the report, or decision to submit the paper for publication.

Results

Study Population and Baseline Characteristics

We included 332 patients (236 and 96 in the training and validation datasets, respectively). There were 83 patients in the training dataset with aggressive PD (14 with macrovascular invasion, 54 with extrahepatic metastasis, and 15 with both) and 22 with aggressive PD in the validation dataset (three with macrovascular invasion, 11 with extrahepatic metastasis, and eight with both). Regarding BCLC stages, five patients in BCLC 0 stage (training dataset: four; validation dataset: one), 17 patients in BCLC A stage (training dataset: 14; validation dataset: three), 47 patients in BCLC AB stage (training dataset: 39; validation dataset: eight), and 36 patients in BCLC B stage (training dataset: 26; validation dataset: 10) had aggressive PD. Except for maximum diameter, no significant differences in the baseline characteristics were observed between patients in the training and validation datasets (Table 1).

Models’ Construction

Among the investigated clinical and semantic factors, logistic regression analysis revealed that maximum tumor diameter (p=0.049), presence of cirrhosis (p=0.025), fusion lesion (p=0.128, ), HCC capsule (non-intact: p=0.233, intact: p=0.004; ), and nodule-in-nodule architecture (p=0.088, ) were significantly associated with aggressive PD (). The score of ModelCS was calculated using the following formula:

ModelCS = −1.6705 + 0.009 × maximum diameter + 0.7009 × cirrhosis (negative: 0; positive: 1) + 0.5365 × fusion lesion (negative: score=0; positive: score=1) −0.4967 × HCC capsule (non-intact, score = 1) −1.7083 × HCC capsule (intact, score = 2) + 0.6375 × nodule-in-nodule architecture (negative: score=0; positive: score=1).

As mentioned earlier, ModelD was constructed using six ResNet blocks, one flatten layer, and one dropout layer, whereas ModelCSD was constructed by combining ModelCS and ModelD.

Model Comparison

The areas under the curves (AUCs) of each model (ModelCS vs ModelD vs ModelCSD) were 0.741, 0.815, and 0.856, respectively, in the training dataset and 0.780, 0.836, and 0.862, respectively, in the validation dataset (Figure 4A and B, and Table 2). ModelCSD showed significantly better performance per the NRI, the IDI, and/or the DeLong test in both datasets (). Regarding calibration, the performance of ModelCSD was similar to that of ModelD; however, it was better than that of ModelCS (Figure 4C and D). Based on these results, ModelCSD was used as our final model, and we constructed a nomogram using ModelCSD (Figure 5A). We then constructed the DCA, in which ModelCSD still had better performance than ModelCS and ModelD (Figure 5B).

Figure 4

Comparison of the three models. The areas under the curve for ModelCS, ModelD, and ModelCSD were 0.741, 0.815, 0.856, respectively, in the training dataset (A) and 0.780, 0.832, 0.861, respectively, in the validation dataset (B). Good calibrations (C and D).

Table 2

AUC, Sensitivity, and Specificity of the Three Models

	Training Dataset			Validation Dataset
	AUC	SEN	SPE	AUC	SEN	SPE
ModelCS	0.741	0.795	0.601	0.780	0.863	0.662
ModelD	0.815	0.831	0.660	0.832	0.955	0.622
ModelCSD	0.856	0.831	0.732	0.861	0.909	0.770

Abbreviations: AUC, area under the curve; SEN, sensitivity; SPE, specificity.

Figure 5

Nomogram and decision curve of ModelCSD. A nomogram of ModelCSD was constructed (A), and it showed better performance than ModelCS and ModelD in the decision curve (B).

Subgroup and Survival Analysis

On subgroup analysis, ModelCSD showed no significant differences in performance between the resection and TACE subgroups, as determined by both AUCs (resection subgroup: 0.839, TACE subgroup: 0.895, p=0.183, ) and calibration (). Similarly, this model demonstrated equivalent performance in patients in the early (BCLC stage 0 and A) and moderate (BCLC stage AB and B) stage subgroups, in terms of both AUCs (BCLC stage 0 and A: 0.827; BCLC stage AB and B: 0.861; p=0.537; ) and calibration ().

For survival analysis, we used the Youden index to identify the threshold of the ModelCSD-predicted risk of aggressive PD, which was determined to be 0.28. The low-risk subgroups in both datasets had significantly longer times to aggressive PD (training dataset: p<0.001, validation dataset: p<0.001; , and ) and more favorable OS (training dataset: p<0.001, validation dataset: p<0.001; , , and ).

Discussion

In this multicenter study, by combining clinical/semantic factors and deep learning radiomics, we constructed a non-invasive model (ModelCSD) to predict the risk of future aggressive PD in HCC patients. With satisfactory performance, our model could provide valuable information for timely intervention in high-risk populations.

In this study, our ModelCSD not only achieved an AUC of 0.861 in the validation dataset but also a high sensitivity of 0.909, demonstrating its promising capacity in identifying high-risk patients. Meanwhile, the performance of ModelCSD was not influenced by treatment types or disease stages, which further proved its robustness under different conditions. Moreover, ModelCSD could stratify patients according to the time to aggressive PD and OS. Similar to nasopharyngeal carcinoma wherein patients with a high risk of distant metastasis gain additional benefit from an early intervention,10 HCC guidelines recommend that treatment stage migration strategy may be considered in highly selected patients.19,24 More importantly, researchers have suggested that patients with high risk of PD are ideal candidates for adjuvant immunotherapy aimed at eliminating or controlling residual or radiologically occult tumor cells.9 Thus, systematic therapies in advanced stage may be used for patients in early or moderate stages. In fact, researchers have conducted studies to test targeted and immune therapies in early and moderate stage HCC.25,26 Therefore, for the high-risk population identified by ModelCSD (with a predicted risk >0.28), if targeted and immune therapies can be explored as early intervention in the future, they may offer additional survival benefit.

Regarding the candidate factors used for model construction, since routine biopsy is not recommended during HCC management owing to the risk of bleeding and tumor seeding,18 we combined clinical/semantic factors and deep learning radiomics for ModelCSD. Concerning deep learning radiomics, with the current technological advances in artificial intelligence, deep learning provides a novel method with which medical data can be analyzed via advanced healthcare-related algorithms.14 The combination of deep learning and radiomics can provide additional information for other cancers and hepatic diseases.15,16 In our study, deep learning radiomics model (ModelD) alone achieved equal performance as ModelCSD in some aspects (NRI in both datasets and DeLong test in the validation dataset). These results showed the advantages of high-dimensional quantitative information provided by deep learning radiomics.

On the contrary, the better improvements observed in ModelCSD than in ModelD in the remaining aspects (ie, IDI in both datasets) showed that clinical/semantic factors were indispensable for better performance. For the clinical/semantic factors included in ModelCSD, a larger tumor diameter would allow the greater cultivation of highly invasive cells. Moreover, cirrhosis identified by CT was mostly at the severe stage, indicating a long history of liver disease and, consequently, more time for the tumor to become invasive. According to a previous study, HCCs without capsules have greater microvascular invasive potential.27 Similarly, our results showed that two semantic factors were related to aggressive PD: the presence of a fusion lesion showed that HCC can form ≥2 lesions that could grow beyond the adjacent normal tissue and an absent or non-intact HCC allowed access to a greater amount of fibrous tissue that increased tumor growth and invasiveness. In contrast, although traditional clinical factors (such as BCLC stages, treatments, Child-Pugh class, and the number of lesions) were potential prognostic factors, they were more likely to be related to local control rates rather than extrahepatic metastasis and macrovascular invasion. Regarding the comparison of deep learning radiomics models, ModelD had better performance than ModelCS, indicating that quantitative data are more valuable for aggressive PD risk classification. However, the fact that ModelCSD could improve ModelCS in certain aspects also showed that qualitative data are indispensable.

This study had some limitations. First, to control bias and perform deep learning radiomics analysis, this study used relatively strict inclusion criteria, in which patients without enough follow-up, negative margin, or CT images at diagnosis were excluded. This led to a relatively small sample size, and future research encompassing a larger sample size should be performed to test the model in more detailed subgroups, such as BCLC B1 to B4 stages. Second, to ensure the accuracy in segmentation, the region of interest was semi-automatically segmented in this study. In the future, to save labor and time costs in clinical application, an automatic segmentation should be developed based on this study. Third, to control bias, patients initially treated with ablation were not included in our study. Considering that ablation provides equal efficacy to liver resection for single HCC with small diameter,18,19 whether our model could be applied for ablation should be tested further. Fourth, most HCCs in our study were caused by HBV; whether our conclusions are suitable for patients with different etiology, such as Hepatitis C or alcoholic hepatitis, requires further validation.

Conclusion

We constructed a deep learning-based model for future risk of extrahepatic metastasis and macrovascular invasion. In high-risk populations, the current guideline indicates that liver resection and TACE have unsatisfying efficacy in preventing extrahepatic metastasis and macrovascular invasion and lead to decreased overall survival. For these patients, more treatment options such as targeted or immune therapies may be explored.