Predicting subretinal fluid absorption with machine learning in patients with central serous chorioretinopathy.

BACKGROUND: Machine learning was used to predict subretinal fluid absorption (SFA) at 1, 3 and 6 months after laser treatment in patients with central serous chorioretinopathy (CSC).
METHODS: The clinical and imaging data from 480 eyes of 461 patients with CSC were collected at Zhongshan Ophthalmic Center (ZOC) and Xiamen Eye Center (XEC). The data included clinical features from electronic medical records and measured features from fundus fluorescein angiography (FFA), indocyanine green angiography (ICGA), optical coherence tomography angiography (OCTA), and optical coherence tomography (OCT). A ZOC dataset was used for training and internal validation. An XEC dataset was used for external validation. Six machine learning algorithms and a blending algorithm were trained to predict SFA in patients with CSC after laser treatment. The SFA results predicted by machine learning were compared with the actual patient prognoses. Based on the initial detailed investigation, we constructed a simplified model using fewer clinical features and OCT features for convenient application.
RESULTS: During the internal validation, random forest performed best in SFA prediction, with accuracies of 0.651±0.068, 0.753±0.065 and 0.818±0.058 at 1, 3 and 6 months, respectively. In the external validation, XGBoost performed best at SFA prediction with accuracies of 0.734, 0.727, and 0.900 at 1, 3 and 6 months, respectively. The simplified model showed a comparable level of predictive power.
CONCLUSIONS: Machine learning can achieve high accuracy in long-term SFA predictions and identify the features relevant to CSC patients' prognoses. Our study provides an individualized reference for ophthalmologists to treat and create a follow-up schedule for CSC patients. 2021 Annals of Translational Medicine. All rights reserved.

Introduction

Central serous chorioretinopathy (CSC) is a retinochoroid disease that causes idiopathic serous retinal detachment, which is associated with one or more leakages from the choroid through the defects in the retinal pigment epithelium (RPE) outer blood-retina barrier. It primarily affects relatively young men of working age (1,2). CSC is fairly common, being considered as the fourth most prevalent non-surgical retinopathy associated with subretinal fluid (SRF) leakage in the world (3). Although SRF can resolve spontaneously in some cases, many patients still suffer permanent vision loss or significant clinical sequelae due to incomplete subretinal fluid absorption (SFA) (2).

CSC is usually divided into two categories: acute CSC and chronic CSC, based on the duration of symptoms. Most investigators have employed this incomplete and relatively rudimentary classification of CSC in their clinical studies (2). However, there is no clear consensus regarding the criteria for classification. Our lack of an established classification system necessitates studying the natural disease progression of CSC and its therapeutic management (4,5). There remains an absence of academically recognized treatment guidelines for CSC. Ophthalmologists have to make decisions experientially in the treatment of CSC patients.

To develop more precise care for patients, we have established an individualized management plan based on SFA utilizing big data. SFA is the most concerning issue for clinicians after treatment, and it is the most important prognostic characteristic for patients with CSC (6). The increase or decrease in SFA affects the therapeutic strategy and the follow-up intervals for patients. In our study, we tried to establish an intelligent prediction system to foresee SFA at 1, 3 and 6 months after laser treatment with big data that incorporates medical records and imaging features, which help us to clarify the prognosis of patients with CSC and choose a sensible treatment.

We present the following article in accordance with the TRIPOD reporting checklist (available at http://dx.doi.org/10.21037/atm-20-1519).

Methods

Data collection

CSC was defined as patients with leakage on their fundus fluorescein angiography (FFA), abnormal choroidal circulation such as hyperpermeability, dilated choroidal vessels or other abnormal microangiopathy on indocyanine green angiography (ICGA), and serous retinal detachment (SRD) as confirmed by optical coherence tomography angiography (OCTA) and optical coherence tomography (OCT). A total of 416 eyes in 401 patients and 64 eyes in 60 patients were studied at Zhongshan Ophthalmic Center (ZOC) and Xiamen Eye Center (XEC), respectively, from January 2013 to September 2019. A definite diagnosis was made for all patients based on FFA and ICGA. The patients were followed for 1–6 months after treatment. Study exclusion criteria were as follows: (I) patients with high myopia, as defined as a refractive error (spherical equivalent) <−6.00 diopters, or an axial length >26.5 mm and (II) patients with media opacities or signal strength indexes who were affected. The study was conducted in accordance with the Declaration of Helsinki (as revised in 2013). The study was approved by the ethics board of ZOC (No. 2020KYPJ024) and individual consent for this retrospective analysis was waived.

A total of 6,732 imaging pictures (1,248 FFA, 1,248 ICGA, 1,412 OCTA, and 2,824 OCT images) and 554 imaging pictures (192 FFA and 362 OCT images) were collected from ZOC and XEC, respectively. FFA (Heidelberg Spectralis, Heidelberg, Germany) and ICGA (Heidelberg Spectralis, Heidelberg, Germany) images for each patient were included only at the baseline, including three images from the early, middle and late phases. However, for the OCTA (RTVue XR Avanti with AngioVue; Optovue Inc., Fremont, CA, USA) and OCT (Heidelberg Spectralis, Heidelberg, Germany) follow-up data, data from the baseline, 1, 3, and 6 months after laser treatment were included. Measurement information on the FFA, ICGA, OCTA, and OCT, were extracted from the software of Heidelberg Eye Explorer (version 1.7.1.0) and Optovue (Version 2017.1.0.155). The clinical features (20 clinical features, e.g., the duration of CSC) of these CSC patients were also extracted from electronic medical records (details are provided in Table S1). For therapy information, the data from ZOC included conventional laser (CL) therapy (117 eyes), subthreshold micropulse laser (SML) therapy (80 eyes) and half-dose photodynamic therapy (hd-PDT) (219 eyes). The XEC data included CL therapy (21 eyes), 577nm SML therapy (14 eyes) and hd-PDT (29 eyes). According to type 3 of the TRIPOD statement, we developed prediction models using the dataset from ZOC and evaluated its performance in a separate dataset from XEC (7).

We treated each eye as a separate CSC case during data preprocessing. There were only a few values missing from the ZOC data, and we filled them in with the mean values for other cases. However, all the ICGA and OCTA features were missing for patients collected from XEC data. Considering their clinical significance and importance in the algorithm, we ultimately chose to fill in the missing features in the XEC data with the mean values for the same features of the ZOC data.

Construction of models

All the training and testing approaches were run on a workstation configured with 32-core Intel Xeon E5 CPU and 128 GB RAM. We used Python 3.6.8 in the Ubuntu 16.04 system. The Python libraries we used in this study are as follows: Jupyter (1.1.0), Scikit-Learn (0.19.1), and Pandas (0.20.3). Six separate algorithms were trained, with a total of 165 features and state-of-the-art performance in each adaptive domain, and they are listed as follows: Decision Tree (8), AdaBoost.R2 (9), Gradient Boosting (10), XGBoost (11), random forest (12), and extra-trees (13). While selecting the optimal algorithms, we randomly divided the ZOC data into 10 parts, and calculated the decision tree, AdaBoost.R2, gradient boosting, XGBoost, random forest, and extra-trees by 10-fold cross-validation. We then selected the best three algorithms for the ensemble according to the prediction accuracy. After determining the selected algorithms, to make the most of the existing clinical data, we used all the data from ZOC to retrain the selected algorithms. The workflow is shown in .

Figure 1

Overall Study Workflow. Workflow diagram showing the training overview for the SFA prediction model.

For each of the above algorithms, we used a grid search with cross-validation to select the suitable hyperparameters (14).

The ensemble learning method was applied to obtain a model with good fitting ability and generalization performance for tasks such as classification and regression and model averaging is a common and effective approach (14,15).

Evaluation of models

To evaluate the performance of our models, the accuracy in predicting SFA at 1, 3 and 6 months after laser treatment were validated. The baseline data were used to predict SFA at 1, 3 and 6 months after treatment. To obtain more accurate predictions, when predicting SFA at 3 months after treatment, we trained the model using the baseline and 1-month data; to predict SFA at 6 months after treatment, we trained the models using the baseline, 1-month and 3-month data. A 10-fold cross-validation was applied to evaluate the performance of the models.

Simplified model

We constructed a simplified prediction model using relatively few clinical data and OCT features to make our study more accessible for clinical use. The remaining features were determined according to the relative importance obtained during the establishment of the original algorithms (Figures S1-S6), and the difficulty in imaging feature acquisition. Table S2 shows all the training features of the simplified model. For the simplified model, the training steps are the same as those in the original models.

Statistical analysis

Accuracy (ACC) is to evaluate the predictive effectiveness of the model. Values are shown as means ± SDs.

Results

A total of 480 eyes in 461 patients aged 28 to 71 years old (43.56±6.64) were addressed during our study. The demographic information for the training and validation datasets are shown in . shows the accuracies of predicting SFA during all the algorithm tasks. Among the original models, random forest had the best performance in the internal validation, and XGBoost performed best at external validation. For the simplified models, gradient boosting had the best performance for internal validation, and the blending algorithm performed best at external validation.

Table 1

Patient demographics

Variable	1 M prediction			3 M prediction			6 M prediction
	ZOC data	XEC data		ZOC data	XEC data		ZOC data	XEC data
Patients	401 (63 females)	60 (11 females)		308 (46 females)	30 (5 females)		244 (37 females)	19 (2 females)
Eyes	416	64		322	33		258	20
Age (years)	43.19±6.44	43.86±7.06		42.87±6.44	43.21±7.51		42.96±6.48	41.70±6.73
VA (logMAR)	0.28±0.21	0.29±0.16		0.28±0.21	0.27±0.16		0.28±0.22	0.28±0.17

Visual acuity (VA) values are presented as the means ± standard deviations at baseline in different groups (in logarithm of minimum angle of resolution [logMAR] units). ZOC, Zhongshan Ophthalmic Center; XEC, Xiamen Eye Center.

Table 2

Accuracy of the subretinal fluid absorption predictions during internal and external validation tests

Variable	1 M (ACC, %)		3 M (ACC, %)		6 M (ACC, %)
	Baseline		Baseline + 1 M		Baseline + 1 M + 3 M
Algorithm learner
Internal validation
Decision tree	0.563±0.054		0.712±0.050		0.767±0.095
Adaboost	0.603±0.066		0.749±0.089		0.748±0.057
Gradient boosting	0.623±0.054		0.755±0.052*		0.791±0.072
XGBoost	0.628±0.045		0.752±0.056		0.810±0.059
Random forest	0.651±0.068*		0.753±0.065		0.818±0.058*
Extra-trees	0.645±0.044		0.740±0.059		0.795±0.079
Blending algorithm	0.647±0.067		0.749±0.058		0.810±0.066
External validation
Decision tree	0.563		0.515		0.800
AdaBoost	0.719		0.576		0.750
Gradient boosting	0.703		0.697		0.850
XGBoost	0.734*		0.727*		0.900*
Random forest	0.703		0.636		0.900*
Extra-trees	0.734*		0.636		0.900*
Blending algorithm	0.703		0.697		0.900*
Simplified model
Internal validation
Decision tree	0.536±0.053		0.687±0.048		0.764±0.069
AdaBoost	0.613±0.069		0.725±0.072		0.779±0.057
Gradient boosting	0.630±0.057		0.780±0.043*		0.818±0.074*
XGBoost	0.625±0.049		0.768±0.066		0.811±0.074
Random forest	0.634±0.048		0.762±0.067		0.811±0.063
Extra-trees	0.635±0.038*		0.737±0.057		0.814±0.081
Blending algorithm	0.632±0.056		0.759±0.047		0.811±0.070
External validation
Decision tree	0.563		0.515		0.850
AdaBoost	0.563		0.636		0.800
Gradient boosting	0.609		0.727		0.900*
XGBoost	0.578		0.697		0.900*
Random forest	0.672*		0.667		0.900*
Extra-trees	0.641		0.667		0.900*
Blending algorithm	0.656		0.758*		0.900*

*, the best learners in all cases. ACC, accuracy of the SFA prediction at 1, 3 and 6 months after laser treatment compared with the ground truth. The results were stratified according to the follow-up periods and the points input into the algorithms.

During the internal validation, random forest performed best in predicting SFA, with accuracies of 0.651±0.068, 0.753±0.065 and 0.818±0.058 at 1, 3 and 6 months, respectively. In the external validation, XGBoost performed best in predicting SFA, with accuracies of 0.734, 0.727, and 0.900 at 1, 3 and 6 months, respectively. The simplified model showed a comparable level of predictive power. In the internal validation, gradient boosting performed best at predicting SFA, with accuracies of 0.630±0.057, 0.780±0.043 and 0.818±0.074 at 1, 3 and 6 months, respectively. During the external validation, the blending model performed best at predicting SFA, with accuracies of 0.656, 0.758, and 0.900 at 1, 3 and 6 months, respectively.

In the cross-validation, random forest achieved high-accuracy predictions with areas under the curve (AUCs) ranging from 0.35 to 0.80 for 1 month, from 0.63 to 0.89 for 3 months, and from 0.92 to 1.00 for 6 months. In the external validation, XGBoost provided high-accuracy predictions with AUCs ranging from 0.76 to 1.00 for 1 month, from 0.27 to 0.72 for 3 months, and from 0.98 to 1.00 for 6 months (). The simplified model exhibited an analogous prediction accuracy with AUCs ranging from 0.57 to 0.81 for 1 month, from 0.23 to 0.97 for 3 months, and from 0.88 to 1.00 for 6 months in the cross-validation, and AUCs ranging from 0.72 to 1.00 for 1 month, from 0.27 to 0.66 for 3 months, and from 0.98 to 1.00 for 6 months in the external validation (). The distributions of prediction results and ground truth in each task are revealed in the confusion matrixes (CM) of and . As shown in the receiver operating characteristic curve (ROC curve) and the CM, the SFA prediction at 6 months after treatment was the most accurate. Figures S1-S6 show the importance of features in the SFA predictions at 1, 3 and 6 months.

Figure 2

Prediction performance in the internal and external validation tests on the full model. Panels A, B, and C, CM of the classification in the internal validation test. Panels D, E, and F, ROC of the internal validation test. Panels G, H, and I, CM of the classification in the external validation test. Panels J, K, and L, ROC of the external validation test. CM, confusion matrix; ROC, receiver operating characteristic curve.

Figure 3

Prediction performance in the internal and external validation tests on the simplified model. Panels A, B, and C, CM of the classification in the internal validation test. Panels D, E, and F, ROC of the internal validation test. Panels G, H, and I, CM of the classification in the external validation test. Panels J, K, and L, ROC of the external validation test. CM, confusion matrix; ROC, receiver operating characteristic curve.

Discussion

To our knowledge, no study has previously generated a machine learning model for predicting SFA in CSC patients. Our prediction models can foresee the patient’s condition six months in advance. However, recent predictions within three months are relatively imprecise. Generally, short-term targets should be more accurate than long-term targets for applying artificial intelligence in disease prognosis prediction (16). This may be due to the lower limitation of the courses in the inclusion criteria; as the baseline data for CSC patients vary greatly, and the short-term consistency of patient prognosis is low. Usually, at six months after treatment, the SFA rates of CSC patients were much higher than at one and three months, which makes prediction tasks easier (17,18). Last but not least, for prognosis predictions of complex fundus disease, we still need to accumulate follow-up data to improve our models.

By predicting SFA in patients, we can better understand the progression of CSC, choose cost-effective therapies and manage the follow-up more efficiently. The increasing use of FFA, ICGA, OCTA, and OCT in studying CSC has greatly improved our understanding of its pathogenesis and imaging characteristics. However, there is still no clear consensus regarding the criteria for classification and the guidelines for treatment. Our predictive system provides a reference for clinicians to choose therapies. We can input the information related to the patients and different therapies, and then the models will give us the probability of presenting SFA in six months. This is a new strategy for mining big data, and it enables us to achieve precise treatment without considering the specific classification of the disease. Based on the prediction models established in this study, we could choose a more efficient method such as hd-PDT in patients with consistent SRF, and we can also choose a more economical therapy such as SML for patients with SRF that is easily absorbed.

More than helping clinicians choose reasonable therapies, the models also help define the factors that are relevant to the CSC prognosis. In the analysis of feature importance, we found that in addition to the therapies chosen by clinicians, the baseline characteristics on the retina and lifestyle also exert a significant impact on SFA, including the SRF height, the central macula thickness (CMT), the double-layer sign (DLS), and the scores on the Pittsburgh Sleep Quality Index and Hamilton Anxiety Scale (19,20). The findings can help us analyze the relevant factors that lead to different prognoses in CSC patients with the same treatment and guide patients to pay attention to influencing these factors in their daily life.

To make our study applicable to different scenarios, we constructed a simplified prediction model according to the relative importance obtained in the original models and the accessibility of the imaging features; the model is trained with only 11 clinical features and OCT features. This advantage can be further appreciated in applications by hospitals in underdeveloped areas without FFA, ICGA, and OCTA. In a clinical setting, FFA, ICGA, and OCTA are not necessary at certain stages, in most cases.

Limitations

There are some limitations in our present study. To improve the accuracy of the short-term predictions within three months, we need to incorporate the duration of the disease into the inclusion criteria. More data on CSC patients are necessary to improve the accuracy of the SFA prediction models. In addition, data for external validation is unicentric, and more real-world tests are needed for improved accuracy.

Conclusions

In summary, our study showed that multidimensional patterns of clinical and imaging features are predictive factors of SFA in CSC patients. The prediction models provide us with a whole new strategy to counsel patients from an individual-based perspective, and they serve as references for ophthalmologists who can choose efficient therapies and make follow-up schedules.

The article’s supplementary files as