Prognosis Prediction of Uveal Melanoma After Plaque Brachytherapy Based on Ultrasound With Machine Learning.

INTRODUCTION: Uveal melanoma (UM) is the most common intraocular malignancy in adults. Plaque brachytherapy remains the dominant eyeball-conserving therapy for UM. Tumor regression in UM after plaque brachytherapy has been reported as a valuable prognostic factor. The present study aimed to develop an accurate machine-learning model to predict the 4-year risk of metastasis and death in UM based on ocular ultrasound data.
MATERIAL AND METHODS: A total of 454 patients with UM were enrolled in this retrospective, single-center study. All patients were followed up for at least 4 years after plaque brachytherapy and underwent ophthalmologic evaluations before the therapy. B-scan ultrasonography was used to measure the basal diameters and thickness of tumors preoperatively and postoperatively. Random Forest (RF) algorithm was used to construct two prediction models: whether a patient will survive for more than 4 years and whether the tumor will develop metastasis within 4 years after treatment.
RESULTS: Our predictive model achieved an area under the receiver operating characteristic curve (AUC) of 0.708 for predicting death using only a one-time follow-up record. Including the data from two additional follow-ups increased the AUC of the model to 0.883. We attained AUCs of 0.730 and 0.846 with data from one and three-time follow-up, respectively, for predicting metastasis. The model found that the amount of postoperative follow-up data significantly improved death and metastasis prediction accuracy. Furthermore, we divided tumor treatment response into four patterns. The D(decrease)/S(stable) patterns are associated with a significantly better prognosis than the I(increase)/O(other) patterns.
CONCLUSIONS: The present study developed an RF model to predict the risk of metastasis and death from UM within 4 years based on ultrasound follow-up records following plaque brachytherapy. We intend to further validate our model in prospective datasets, enabling us to implement timely and efficient treatments.

Introduction

Uveal melanoma (UM) is the most common aggressive ocular tumor in adults. The annual incidence rate per million people is 6 in non-Hispanic whites (1) and 0.3–0.6 in Asians (2–4). Although new techniques such as proton beam therapy have been introduced (5), plaque brachytherapy, mainly using iodine-125, remains the dominant option as an eyeball-conserving treatment for UM. In the United States, the ratio of plaque brachytherapy is increasing each year, which accounts for more than 50% recently (6, 7). The same case was also seen in our eye center. However, patients with UM have high mortality with approximately 50% of patients developing metastatic disease and eventually dying within 5 years (8, 9). Therefore, it is important to predict the metastasis risk and long-time survival accurately.

Several factors have been proven to correlate with patient outcomes. These include tumor size and location, as well as related features such as retinal detachment, extrascleral extension, and retinal invasion (10, 11). The most significant factor for melanoma-specific mortality prediction is dependent on tumor-specific genetic alterations and histopathologic factors including epithelioid cell type, monosomy 3 and 6p gain, and loss of BAP-1 gene (12). Gene expression profiling (GEP) of 15 genes was divided into class 1 and class 2 UM, those with the class 2 GEP have a greater rate of metastasis and mortality compared to class 1 GEP. However, fine-needle aspiration is not available in most cases for patients with UM treated by plaque brachytherapy. Therefore, we wish to construct a prediction model with more readily accessible clinical data.

Ultrasonography, a cost, and time-effective non-invasive examination is the most used application for determining the dimensions of a posterior UM. And it is essential throughout follow-up for tumor measurement (13). Tumor regression has commonly been evaluated as a percentage change from initial tumor thickness measured with B-scan ultrasonography. According to the Collaborative Ocular Melanoma Study, a 15% increase in tumor thickness after brachytherapy should be considered as a failure. Many previous studies have shown that such local treatment failure (14–17) and rapid regression of tumors after plaque brachytherapy (18, 19) predict a lousy prognosis.

Previous models based on clinical and demographic characteristics have been developed to predict individual patient prognosis after UM treatment (20–26). To our knowledge, this is the first report that describes a mathematical model for patients with UM after iodine-125 plaque brachytherapy using postoperative follow-up ultrasound data. The present study investigates the prognostic value of dynamic morphometric parameters to predict 4-year survival and metastasis status (Figure 1).

Figure 1

Workflow of the research. Machine learning was carried out according to the clinical and follow-up data from patients to predict the prognosis. Results of metastasis and survival analysis are based on filtered data.

Materials and Methods

Source of Data

This is a retrospective, single-center study conducted in the Beijing Tongren Eye Center. The study population included adult patients that were clinically diagnosed with UM from July 2007 to December 2016. Generally, iodine-125 plaque brachytherapy was used for tumors with a thickness of <10 mm in our center. The standard dose of irradiation was 100 Gy to the apex of the tumor. However, patients who were refractory to other treatments and strongly requested it were also treated by brachytherapy.

Selection of Participants

Patients who were diagnosed with UM at the Beijing Tongren Eye Center and subsequently received brachytherapy were included in this study. The exclusion criteria were: (1) age <18 years, (2) received other therapies, (3) alive and had a follow-up time of fewer than 4 years, (4) the third follow-up time was more than 3 years, (5) filled follow-up time was later than the time of outcome, (6) had metastatic disease at the time of diagnosis. Finally, 454 patients were included to construct the model for predicting death and 424 patients to build the model for predicting metastasis (Figure 2A). Moreover, 177 surviving patients with UM had a follow-up duration ranging from 3 to 4 years. They will be included in the prospective validation of our models in future studies (Figure 2B).

Figure 2

Steps for excluding features and samples to preprocess the input dataset for machine learning. (A) The workflow for excluding some samples. (B) Prospective validation (C) The workflow for excluding some features.

Data Collected

The age, gender, and involved eye were recorded from each patient's record during the initial interview. The presence of subretinal fluid, optic disk involvement, vitreous hemorrhage, ciliary body involvement, tumor thickness, minimum and maximum tumor diameter, tumor shape and position, intraocular pressure and visual acuity, photographs, and ultrasound records were collected from the preoperative medical records. Several strategies, including fundus photography, fluorescein angiography, indocyanine green angiography, standardized echography, and orbital MRI, were conducted to assist diagnosis. Tumors were staged according to the American Joint Committee on Cancer (AJCC) consensus. We excluded duplicate factors and factors that did not differ among groups (Figure 2C).

Ultrasound images were reviewed by two independent radiologists with at least 5 years of experience in interpreting ocular ultrasound images. The radiologists were blinded to the clinical data. When two radiologists failed to reach a consensus through their independent assessment, the image would be reviewed jointly to ultimately achieve agreement. They measured the tumor's thickness from the inner surface of the sclera to the tumor apex and maximum basal diameter. Thickness and the minimum basal diameter were measured from two meridians, along with the maximum basal diameter and perpendicular to it. Representative digitized scans were stored at the time of each diagnostic and follow-up visit.

Missing Value Completion

There were some missing data values due to the loss of clinical data and some missing features. The missforest algorithm (R package missForest) was used to fill in the blank values in the dataset (27). Missforest iteratively filled all features with missing values by predicting missing values from existing values. The order for filling missing values was from features with the fewest missing value to the feature with the most missing values. Moreover, numerical features and nominal features were predicted with Random Forest (RF) regression and classification, respectively. The follow-up information of patients with less than three visits was also filled, and the length of follow-up was less than that of outcome events.

Prediction Model

Machine learning is a powerful tool for mining the hidden relationships in our dataset which included imaging (28–32), genetic (33), clinical (27, 34), multi-modal sensor data (35–37), and other sources (38). RF is a type of ensemble learning method which encapsulates multiple decision trees to vote the classification results. The decision tree is a basic machine learning method that applies tree data structure to recursively split the whole dataset into multiple subsets. Finally, the samples in each leaf node either belong to one class or own more features could be used to be split, namely, the class of each sample can be inferred according to the paths from the root node to leaf nodes in the tree (39–42). In our research, the RF model was used to construct models of whether a patient will survive for more than 4 years and whether the tumor will metastasis within 4 years after plaque brachytherapy. This was done using demographic attributes, clinical features, and follow-up records.

Additionally, all datasets used were imbalanced. Therefore, the most convenient, cost-sensitive method (43) was used to tackle this problem and assist RF in constructing the models. Synthetic Minority Oversampling Technique (SMOTE), the simplest oversampling algorithm, is typically used to enrich the minority in each training set of the internal cohort. Numerical and nominal features are preprocessed differently in terms of measuring the distance of two samples. However, we did not adopt this method because we cannot guarantee the ratio for generating more minority class samples. It will also import some noise into the dataset. The numerical and nominal features were separately oversampled and then merged. The under-sampling method (44) randomly deleted some majority samples in the training set, which was not suitable for our study because the follow-up dataset is precious. We cannot sacrifice the majority class to trade off the minority class. Similar to the multi-objective optimization, the cost-sensitive method (43) leveraged another objective function (cost function) and accuracy function in constructing a machine learning model. The number of trees in RF was primarily set as 500 when experiments were carried out. Four-fold stratified cross-validation was used to evaluate the performance of RF fairly, and the subjects in each fold were independent (a patient owns only one entry of data).

Statistical Analysis

The baseline characteristics of enrolled participants were presented and compared between survivors and non-survivors by applying either Student's t-test, Chi-square test, and Mann-Whitney U-test as appropriate. Continuous variables were characterized as mean (standardized differences [SD]) or median (interquartile range [IQR]), while categorical or ranked data were reported as count and proportion. One-way ANOVA and Kaplan-Meier analysis were used to evaluate tumor regression patterns. All calculations were performed in Statistical Package for the Social Sciences (SPSS) version 26 and GraphPad Prism version 7. Random forest was performed using Python 3.7.3 (Wilmington, DE, United States) and MATLAB R2016a. Accuracy, sensitivity, specificity (32, 45, 46), Receiver Operating Characteristic (ROC) curve, Precision-Recall (PR) curve, and Area under Receiver Operating Characteristic Curve (AUROC) were used to evaluate the performance of models.

Results

Baseline Characteristics

A total of 454 patients with UM treated by plaque brachytherapy were included in the death analysis. 210 (46.3%) were male. UM occurred in 248 right eyes and 206 left eyes. The mean age was 46.3 ± 11.7 years. We used the criteria of the AJCC consensus to determine that there were 41 T1 stage tumors, 213 T2 stage tumors, 137 T3 stage tumors, and 11 T4 stage tumors. In total, 48 of 424 (11.3%) patients developed metastasis during the follow-up period, and 52 of 454 (11.5%) patients died. The baseline characteristics are compared in Supplementary Table 1. The stage, position, ciliary body involvement, subretinal fluid, maximum and minimum basal diameter, and follow-up data were significantly correlated with death and metastasis. The median period was 154 [IQR:124, 251], 402 [IQR:315, 530], 700 [IQR:581, 839] days respectively for the three follow-up visits (Figure 3).

Figure 3

Boxplot of follow-up time. (A) Death analysis. (B) Metastasis analysis.

Evaluation of Model Performance

In our research, we developed a model to predict death 4 years after treatment, (Figures 4A,B) with 70.51% sensitivity, 56.96% specificity, and overall diagnostic accuracy of 58.51% using the first follow-up data. The overall performance of the prediction model was improved when three follow-up records were included. The performance was raised to a sensitivity of 80.45%, a specificity of 83.35%, and overall diagnostic accuracy of 83.02% (Figure 4C, Supplementary Table 2). Due to imbalanced datasets, we used a relatively high cost-sensitive parameter to increase sensitivity. A higher sensitivity means that patients with poor prognoses are more likely to be detected in clinical practice and radical treatments can be undertaken earlier to improve patient outcomes. The maximum basal diameter was the top-ranked preoperative factor related to death within 4 years after surgery (Figure 5). Position, preoperative minimum basal diameter, corrected visual acuity, and intraocular pressure was also clearly correlated with death. In addition, the span of records from the follow-up was remarkably correlated with predicting death. Thus, obtaining data from three follow-ups had the greatest impact on accuracy.

Figure 4

Machine learning result of death and metastasis model. (A) Receiver operating characteristic (ROC) curve. (B) Precision-recall (PR) curve. (C) Boxplot of all metrics for predicting death. (D) Boxplot of all metrics for predicting metastasis.

Figure 5

Relative importance of each factor in the machine learning model.

Moreover, we constructed a model to predict four-year metastasis status (Figures 4A,B), with 66.67% sensitivity, 69.42% specificity, and 69.10% accuracy. We then incorporated additional follow-up information to achieve a sensitivity of 77.08%, a specificity of 79.79%, and overall diagnostic accuracy of 79.48% (Figure 4D, Supplementary Table 2). The model for predicting death did perform better than the one for metastasis. We found that the maximum basal diameter, intraocular pressure, minimum basal diameter were the most critical factors (Figure 5). Similarly, additional follow-up information beyond the first collection was significantly related to successfully predicting metastasis. Tumor thickness recorded in the third follow-up was the most important information.

Regression Pattern

We next investigated investigate the importance of tumor thickness after treatment. We classified the tumor response to brachytherapy into the following four main patterns (47, 48) (Figure 6). Pattern D (decrease) involved at least one follow-up visit, the thickness decreased by at least 15% compared to the preoperative period, and two other visits also showed a decrease in thickness. Pattern S (stable) indicates there was < a 15% change in thickness. Pattern I (increase) is defined by at least one follow-up visit, the thickness increased by at least 15% compared to the preoperative period, and thickness also increased at two other visits. Pattern O (others) indicates an irregular change in thickness. Preoperative tumor sizes of different patterns are listed in Table 1. It was found that the tumor regression rate increased with increasing tumor thickness (P < 0.001) (Figure 7).

Figure 6

Percentage graph of initial tumor thickness vs. duration of follow-up after iodine 125 brachytherapy for patients with uveal melanoma by tumor regression patterns (D vs. S vs. I vs. O).

Table 1

Preoperative tumor size of different patterns.

	D	S	I	O	P-value
Maximum basal diameter, mm	11.7 ± 2.8	12.0 ± 2.5	12.1 ± 3.6	12.5 ± 3.7	0.311
Minimum basal diameter, mm	10.4 ± 2.7	10.5 ± 2.4	10.6 ± 3.2	11.1 ± 2.7	0.409
Maximum thickness, mm	7.6 ± 2.3	6.3 ± 2.0	4.9 ± 2.0	6.2 ± 2.4	<0.001

Figure 7

Boxplot of preoperative maximum thickness of different tumor regression patterns (D vs. S vs. O vs. I). **P ≤ 0.05, ***P < 0.001.

As shown in Figures 8A,B, there is a statistical significance relating metastasis and death (P < 0.001) to different tumor regression patterns. Patterns D/S were associated with a significantly better prognosis than the I/O group. Then, we further categorized the O group into three subtypes: DI (decrease followed by increase), ID (increase followed by decrease), and Z (“zigzag” or alternating measurements). Kaplan-Meier survival analysis revealed that pattern DI was significantly related to a higher death rate (P < 0.001) (Figure 8).

Figure 8

Kaplan-Meier analysis. (A,B) Kaplan-Meier analysis of survival curve (A) and non-metastasis curve (B) for patients with uveal melanoma after iodine 125 plaque brachytherapy by tumor regression patterns (D vs. S vs. I vs. O), expressed in three follow-up visits. (C) Kaplan-Meier analysis of survival curve for patients with uveal melanoma after iodine 125 plaque radiotherapy by tumor regression patterns (DI vs. ID vs. Z), expressed in three follow-up visits.

Discussion

Great changes have taken place in traditional medicine after entry into the era of data. Physiological parameters can be recorded by wearable smart products (such as smart glasses, watches, and bracelets (49, 50), biological parameters can be expressed by gene sequencing (51), and anatomical parameters can be displayed by image data (52). The limits on analysis of such data by humans alone have clearly been exceeded, necessitating an increased reliance on machines. Accordingly, at the same time that there is more dependence than ever on humans to provide healthcare, algorithms are desperately needed to help (53).

Uveal melanoma (UM) is the most common intraocular tumor in adults. Although several treatments are available for patients with UM, more than half of patients end up with distant metastases. Unfortunately, there is currently no effective treatment for the metastatic disease, and the median survival time for metastatic UM is only 12 months (54–56). So risk factors that allow the early prediction of the metastasis and survival time of patients will contribute to the implementation of a more aggressive treatment strategy and improve patient outcomes (57). Additionally, numerous studies have shown that the great majority of patients want to know whether their prognosis is good or bad both before surgery and during follow-up. Although the bad news is particularly upsetting, patients feel a sense of empowerment over their future planning and a reduction in uncertainty and accompanying anxiety (58–60).

Our previous studies, and those of others, have shown that clinical characteristics such as male gender, advanced age, larger tumor size, epithelioid cell type, subretinal fluid, and ciliary body involvement can increase the risk of metastasis and death (10, 11, 61–64). Additionally, the treatment response by tumors can also affect the outcome to some extent. Several studies discovered that local treatment failure, defined by COMS as a 15% increase in tumor thickness after brachytherapy was significantly related to uveal melanoma-related mortality and systemic dissemination (15, 65). Furthermore, Augsburger and Kaiserman (19, 66) found that rapid regression of tumors after plaque brachytherapy indicates an unfavorable prognosis. Also, in other treatment modalities, Christoph et al. (67) reported a non-linear influence of the regression rate of choroidal melanoma as an independent risk factor of metastatic disease after linear accelerator stereotactic fractionated photon radiotherapy. Thus, tumor size change after surgery is significantly correlated with prognosis. In our research, we added this aspect to the construction of the model to determine whether postoperative information could improve model performance for prediction.

Medicine has experienced the recent emergence of artificial intelligence (AI) as a novel tool for analyzing large amounts of data (68). AI has recently achieved high accuracy in recognizing ocular structure. Deep-learning convolutinal neural networks (CNNs) developed by Li Dong et al. (69–71) have shown superior performance in assessing axial length, subfoveal choroidal thickness, and fundus tessellated density with color fundus photographs. In the diagnosis of multiple ocular disorders, AI outperformed human experts with multimodality imaging, including magnetic resonance imaging (MRI), fundus photographs, and fundus fluorescence angiography (FFA). An updated meta-analysis demonstrated that AI-based algorithms are capable of detecting age-related macular degeneration (AMD) in fundus images with a pooled AUC 0.983 (72, 73). Naoya Nezu et al. (74) recently reported that several algorithms predicted the diagnosis of 17 selected intraocular diseases including UM with aqueous humor cytokines, and indicated some new biomarkers facilitating the diagnosis of relevant diseases. In addition, Zhang et al. (75) also justified the effectiveness of deep learning for predicting nBAP1 expression in UM based only on Hematoxylin and eosin (H&E) sections.

Models based on clinical and demographic characteristics are being used to predict the prognosis of individual patients with UM after treatment. Jorge Vaquero-Garcia et al. (24)created an interactive web-based tool for the Prediction of Risk of Metastasis in Uveal Melanoma (PRiMeUM), which provides a tool for assessing the personalized risk for metastasis based on individual and tumor characteristics. The accuracy of the risk prediction was 80% using only chromosomal features, 83% using only clinical features, and 85% using combined clinical and chromosomal information. However, in most eye centers, chromosomal information is not available. Fine-needle aspiration biopsy is an invasive method and may contribute to some related complications such as vision loss, persistent hemorrhage, and even extraocular extension (76). Therefore, most patients being treated by plaque brachytherapy are reluctant to accept this examination.

We previously applied machine learning technology to establish a model to predict whether a patient would die or metastasize within 2 years after initial treatment. This model achieved an overall accuracy of 77.0 and 75.0% with all features (77). Information extracted from B-ultrasound images was additionally applied to machine learning to provide personalized risk prediction. To the best of our knowledge, ours is the first machine learning-based UM prognosis model using follow-up information after surgery. With the increasing availability of follow-up information, the performance of predictive models has improved significantly. The AUC of models increased from 0.708 to 0.883 after two additional follow-up records were added.

Figure 5 shows that follow-up data were remarkably correlated with 4-year survival. This suggests we can provide a more accurate prognostic evaluation for patients by intensive follow-up, which is readily obtained. In our study, tumor treatment response was divided into four patterns. The D pattern of decreasing tumor thickness correlated to the best prognosis, contrary to some previous research (18, 19). It found that early rapid regression of tumors after plaque brachytherapy was associated with an unfavorable outcome for patients with UM. However, a greater regression indicated a better prognosis in our relatively longer postoperative follow-up. In addition, similar to their results, a positive correlation between tumor thickness and regression rate was also found in our research.

Among the patients enrolled for model construction, the 177 surviving patients with UM with follow-up ranging from 3 to 4 years, can validate algorithms in a short time. Additionally, we welcome external datasets, especially with Asian patients, to continue our validation efforts. We hope that a predictive model for Asian patients can be established using factors that are non-invasive and easily available clinically in the future.

Deep learning (DL)-powered ultrasound has begun to be widely used in diagnosing certain diseases and for distinguishing between benign and malignant tumor types (78–80). But it has been used less for determining prognosis. Thus, we have also tried to construct a DL model using B-ultrasound images to predict long-term survival in patients with UM. However, the performance was found to be unsatisfactory. We do plan to undertake additional prospective studies that will incorporate uniform standard ultrasound images and color Doppler flow imaging to gather more prognostic information. In addition, multiple imaging modalities have been used recently with deep learning, including CT and MRI. Using these tools, researchers can attain more specific and informative histology and prognostic information. Compared to ultrasound, MRI provides excellent contrast resolution and multiple tissue-contrasts. Due to the paramagnetic effect, lesions with different melanin contents will present distinct signal intensities in MRI. Furthermore, the use of multiple sequences including dynamic contrast-enhanced (DCE) sequence and diffusion-weighted MR imaging has made it easier to identify intertumor heterogeneity (81, 82). It has been proven that quantitative multiparametric MRI can be used to predict monosomy 3 and UM metastasis (83, 84). Therefore, we propose to adopt DL to automatically extract high-throughput features from multi-modal, multi-channel preoperative MRI to predict the survival time for patients with UM. This will enable us to better develop personalized treatment plans and realize precision medicine.

There are some limitations in our study that should be noted. First, while death is an outcome that can be precisely determined metastasis can only be detected at follow-up visits. Therefore, metastasis may present before the clinical diagnosis, which would affect our model's predictive value for metastasis. Second, due to the retrospective nature of this study, the follow-up interval after surgery in our study was not fixed. This affected the results to some extent. Third, based on the COMS data, post-therapy surveillance relies on decreasing thickness with ultrasound B repeated every 6 months for 2 years and yearly after that (16). But most of our patients can only be checked three times within 3 years. Our results showed that the algorithm's performance could be enhanced with more follow-up visits. Frequent follow-up of patients is advisable, ideally leading to earlier detection of metastasis and timely enrollment into treatment and care. Thus, patients will be strictly followed up in the future to further explore the role of data from follow-up examinations in predicting prognosis.

Conclusions

In conclusion, the present study developed an RF model to predict the risk of UM metastasis and death within 4 years based on ultrasound follow-up records following plaque brachytherapy. We intend to further validate our model in prospective datasets, which can prompt us to implement timely and efficient treatments.

Data Availability Statement

The raw data supporting the conclusions of this article will be made available by the authors, without undue reservation.

Ethics Statement

The studies involving human participants were reviewed and approved by Beijing Tongren Hospital of Capital Medical University. The patients/participants provided their written informed consent to participate in this study.

Author Contributions

YaL and WW contributed to the concept, design of the study, revised the manuscript, and handled the supervision. JL, YC, and YY wrote the manuscript. JL developed the study. YC, YY, KZ, YuL, HZ, LD, and JX participated in the final design of the study. JL, YC, YY, KZ, YuL, and HZ carried out the study. JL, YY, YuL, and HZ collected the data. All authors read and approved the final submitted version of the manuscript.

Funding

This trial is supported by the National Natural Science Foundation of China (82101180); Beijing Natural Science Foundation (7204245); Scientific Research Common Program of Beijing Municipal Commission of Education (KM202010025018); Beijing Municipal Administration of Hospital's Youth Programme (QML20190202); Beijing Dongcheng District Outstanding Talents Cultivating Plan (2018); the Capital Health Research and Development of Special (2020-1-2052); Science & Technology Project of Beijing Municipal Science & Technology Commission (Z201100005520045, Z181100001818003).

Conflict of Interest

KZ is employed by InferVision Healthcare Science and Technology Limited Company. The remaining authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.

Publisher's Note

All claims expressed in this article are solely those of the authors and do not necessarily represent those of their affiliated organizations, or those of the publisher, the editors and the reviewers. Any product that may be evaluated in this article, or claim that may be made by its manufacturer, is not guaranteed or endorsed by the publisher.