A radiomics-based model for predicting local control of resected brain metastases receiving adjuvant SRS.

PURPOSE: Adjuvant radiosurgery to the cavities of surgically resected brain metastases provides excellent local tumor control while reducing the risk of deleterious cognitive decline associated with whole brain radiotherapy. A subset of these patients, however, will develop disease recurrence following radiosurgery. In this study, we sought to assess the predictive capability of radiomic-based models, as compared with standard clinical features, in predicting local tumor control.
METHODS: We performed a retrospective chart review of patients treated with adjuvant radiosurgery for resected brain metastases at the "Institution" from 2009 to 2019. Shape, intensity and texture based radiomics features of the cavities were extracted from the pre-radiosurgery treatment planning MRI scans and trained using a gradient boosting technique with K-fold cross validation.
RESULTS: In total, 71 cavities from 67 treated patients were included for analysis. The 6 and 12 month local control estimates were 86% and 76%, respectively. The 6 and 12 month overall survival was 78% and 55%, respectively. Thirty-six patients developed intracranial failures outside of the surgical cavity. The predictive model for local control trained on imaging features from the whole cavity achieved an area-under-the-curve (AUC) of 0.73 on the validation set versus an AUC of 0.40 for the clinical features.
CONCLUSIONS: Here we report a single institutional experience using radiomic-based predictive modeling of local tumor control following adjuvant Gamma Knife radiosurgery for resected brain metastases. We found the radiomics features to provide more robust predictive models of local control rates versus clinical features alone. Such techniques could potentially prove useful in the clinical setting and warrant further investigation.

Introduction

Metastatic disease comprises the most common brain tumors in adults and, by some estimates, affects nearly a quarter of all patients with cancer [1]. Due to the historically perceived limited efficacy of systemic therapy in crossing the blood–brain barrier, surgery and radiotherapy have traditionally formed the backbone of central nervous system (CNS) directed therapy for treatment of brain metastases. Prior clinical work has demonstrated that the addition of whole brain radiotherapy (WBRT) following surgery improves CNS control and decreases the risk of neurologic death [2]. The use of WBRT, however, is not without the risk of deleterious cognitive side effects [3]. With improved systemic disease control and survival observed with modern targeted therapies [4], there has been a growing interest in minimizing such toxicity. Recent trials comparing the use of WBRT versus stereotactic radiosurgery (SRS) as adjuvant treatment for resected brain metastases have demonstrated similar survival rates but generally better cognitive function among the patients receiving SRS [5]. This has led to a paradigm shift, establishing SRS as a less toxic standard of care alternative to conventional WBRT following tumor resection.

A theoretical drawback for focal therapies such as SRS is the risk of missing residual microscopic tumor, leading to subsequent local disease relapse. With the anticipated increased utilization of postoperative SRS following the publication of the NCCTG N107C/CEC3 trial, there is a pressing need to identify factors associated with risk of local recurrence in order to improve target delineation and to guide treatment planning. Prior studies have suggested correlations between the risk of local failure and a host of tumor, patient and treatment related factors including tumor histology [6], preoperative tumor size [7], [8], cavity volume [9], planning target volume margin [10], time interval between surgery and radiotherapy [11], [12], prescribed dose [9], [11], [13] and plan conformality indices [14]. In light of these previous reports, a recently published guideline for postoperative cavity SRS has attempted to standardize target volume contours by identifying high risk structures through consensus expert opinion [15].

Much of the previous work examining the correlation between imaging based features and local disease recurrence following cavity directed SRS has focused predominantly on basic shape descriptors such as cavity diameter or volume. The relatively recent advent of high throughput feature extraction from medical images, known as radiomics, has enabled acquisition of a magnitude of higher order elements from the image datasets. In this work, we sought to investigate whether the use of these radiomics features would better predict local control following cavity SRS as compared with clinical features alone. To test this hypothesis, we conducted a retrospective study in a population of patients who received postoperative Gamma Knife SRS at the “Institution”.

Methods

Patients

Medical records were reviewed from patients who received Gamma Knife SRS at the “Institution” from 2009 to 2019. Selection criteria included the presence of one or more surgical cavities treated with Gamma Knife SRS and available imaging from the day of the Gamma Knife procedure as well as at least one follow up that included imaging. Patients were excluded if they received prior or concurrent WBRT to avoid confounding variables when analyzing local and distant intracranial control. Patients who received SRS to additional intracranial metastases at the same Gamma Knife session were included provided that the additional lesions were not directly adjacent to the surgical cavity. This study was approved by the “Institution’s” Institutional Review Board.

Treatment

All patients received SRS using the Gamma Knife 4C platform (Elekta, Stockholm, Sweden) at the “Institution”, typically within several weeks of surgery. Each patient had a contrast enhanced T1-weighted MRI performed on the day of Gamma Knife using either a 1.5 T or 3 T Siemens scanner (Erlange, Germany) with 1.0-mm isotropic voxels, TR/TE = 9.4/4.8 ms. The cavity was delineated by the treating radiation oncologist and neurosurgeon and included the cavity surface without an additional volumetric expansion. Follow up imaging was typically performed 2 to 3 months after SRS with subsequent imaging intervals determined at the discretion of the treating physician.

Feature extraction

The cavity contours and dose information were extracted from the treatment planning system for each patient. The images and volumes were visually assessed for accuracy and artifact. Contours were checked and adjusted by a practicing radiation oncologist to ensure uniformity of the contours and complete coverage of the cavity (Fig. 1). Images were histogram matched to the mean image of all patients to reduce the impact of overall signal level variations between patient scans. The images and cavity regions-of-interest data was then imported into an open source radiomics software package (PyRadiomics) [16] for radiomics feature extraction. A bin width of 18 was chosen to generate histogram bin counts between 40 and 100, as recommended by Welch et al. [17]. A total of 107 radiomics features were extracted for each cavity contour, including 32 shape and first order features and 75 texture features.

Fig. 1

MRI images in the same patient: (a) Pre-resection contrast-enhanced T1 MRI of the metastasis, (b) Treatment planning contrast enhanced T1 MRI of the cavity including the contour used for radiomics feature extraction, (c) Follow-up contrast enhanced T1 MRI of recurrent disease following resection and adjuvant SRS.

The texture features used in the predictive modeling quantify the second and higher order statistical distribution of voxel values and fall into several categories. Grey-level co-occurrence features (N = 24) are derived from the co-occurrence matrix that characterizes the number of times a certain voxel value appears in the region of interest. Grey-level size zone features (N = 16) characterize the presence of regions of identical voxel values within a region of interest. Grey-level run length features (N = 16) are derived from the run-length matrix that characterizes a region of interest by the number of consecutive voxels of the same grey-level. Neighboring grey-tone difference features (N = 5) describe the statistical distribution relating a voxel value to the average voxel value of the surrounding voxels. Finally, grey-level dependence matrix features (N = 14) describe the dependence of individual voxel values on the voxels around them as described in Sun et al. [18].

Local control and survival analysis

The primary endpoint was local control which was defined as the absence of disease recurrence either within the surgical cavity or the abutting brain parenchyma. Local recurrence was defined based upon histologic confirmation or radiologic assessment by an experienced neuroradiologist. Imaging changes that were indeterminate for tumor recurrence versus radiation necrosis were followed on subsequent studies and, if there was unequivocal evidence of progressive tumor, the date of the initial imaging changes was used as the date of tumor recurrence (Fig. 1). Secondary endpoints included overall survival as measured from the time of Gamma Knife SRS to death from any cause and distant intracranial failure which was defined as recurrent intracranial disease in brain sites outside of the treatment volume. Survival curves were estimated using the Kaplan-Meier method. A P-value less than 0.05 was considered statistically significant.

Predictive modeling

Two predictive models, using the binary local control variable as target, were trained using patient data: 1) a model using clinical factors extracted from patient charts and the treatment planning system, and 2) a model using radiomics features derived from patient imaging. Feature selection and model training steps in the predictive modeling pipeline were performed in Python using the scikit-learn package [19]. To prevent data leakage and to estimate the generalization error, K-Fold cross validation with K = 5 was used. In K-fold cross validation, the data is randomly split with balanced class stratification into a training and validation set K times. Cross-validation is well known to estimate the error rate of a predictor [20]. Features are normalized, selected, and the model is trained on each of the training sets and then tested on the validation set. This process is repeated for each of the folds. Model performance is generalized by averaging the performance statistics calculated on each of the folds.

To reduce the dimensionality of the data to limit model overfitting, feature selection was performed by recursive feature elimination on a logistic regression model using local control as the target variable. This algorithm considers sets of features trained on a logistic regression model, ranks the contributions of individual features, and repeats with only the strongest performing features. The total number of features that were selected in this way for inclusion in the predictive model were tuned within the cross-validation scheme to balance model complexity and performance on the validation set [21].

A gradient boosting classifier was trained using the features selected by the recursive feature elimination algorithm. Ensemble classification methods such as gradient boosting classifiers are often employed in small datasets and in datasets with class imbalances because they combine the performance of several weak classifiers and resist overfitting [22]. To limit overfitting, decision trees were limited to a single decision node (stumps). Accuracy, sensitivity, specificity, F1 scores, and receiver-operating-characteristic (ROC) curves were calculated for the model.

The gradient boosting classifier trained with radiomics features was used to predict the class probabilities for each patient. The R package “cutpointr” was used to find the ideal cutpoint in the distribution of class probabilities between the groups of “low” and “high” risk of local recurrence based on the sum of sensitivity and specificity of the cutpoint on the outcome of local recurrence [23]. Survival curves for these two groups were plotted using the Kaplan-Meier method and tested for significance with the log-rank test. Feature extraction and predictive modeling source code can be found on the project’s GitHub repository.

Results

A total of 71 treated cavities from 67 patients were identified and included in the current study. The majority (N = 68) of the acquired scans were performed at 1.5 T. Four patients received treatment for more than one cavity with SRS. Three of the four patients had simultaneous surgical resection of two metastases and underwent treatment with SRS to both cavities at the same session. The remaining patient had SRS delivered to a single cavity and approximately 2 years later developed a distant intracranial metastasis which was resected and followed by SRS.

The most common primary tumor was melanoma (n = 18) followed by non-small cell lung (n = 15), breast (n = 13) and renal cell (n = 8) carcinomas. The remaining patients included those with colorectal (n = 3), ovarian (n = 2), small cell lung cancer (n = 2), sebaceous (n = 2), endometrial (n = 2), head and neck (n = 2), esophageal (n = 1), gastric (n = 1), and bladder (n = 1) primary histologies. All patients had received single fraction SRS to the resection cavity at a median of 24 days following surgery. The median prescribed dose was 18 Gy delivered to the 50% isodose line. The median conformity index was 1.6 with a median gradient index of 2.8. The median maximum cavity diameter was 3.7 cm with a median volume of 6.2 cc. The median follow-up was 478 days following SRS. Patient, tumor, and treatment characteristics are summarized in Table 1.

Table 1

Clinical characteristics.

Variable	Number of Patients
Sex (female/male)	37/30
Age – Median (Range)	60 (27–83)
Histology
NSCLC	15
Melanoma	18
Breast Carcinoma	13
Renal Carcinoma	8
Other*	17
	Median (range)
Time to SRS (Days)	24 (6–55)
Prescription Dose (Gy)	18 (15–22)
Conformity Index	1.6 (1.2–3.5)
Gradient Index	2.8 (2.0–3.9)
Cavity Volume (cc)	6.2 (0.2–31.6)
Cavity Diameter (cm)	3.7 (1.2–6.3)
Follow-up duration (days)	478 (13–2523)

*Other included: Bladder (1), Colorectal (3), Endometrial (2), Esophageal (2), Head & Neck (2), Gastric (1), Ovarian (2), SCLC (2), Sebaceous Cell (2).

Rates for 6 and 12 month overall survival following SRS were 78% and 55%, respectively. We observed eighteen local failures with a median time to failure of 183 days (range: 39–1055). The 6 and 12 month local control estimates were 86% and 76%, respectively (Fig. 2a). A total of 36 patients developed distant intracranial failures beyond the surgical cavity (Fig. 2b). The patients who developed a local recurrence were also more likely to develop distant intracranial failure (78%, n = 14/18) compared to those with controlled cavities (42%, n = 22/53) (p = 0.006).

Fig. 2

Kaplan-Meier plots for a) local progression-free survival for the entire cohort, b) intracranial progression-free survival for the cohort.

Of the radiomics features extracted from the whole cavity contours, six were selected by the logistic regression recursive feature elimination process. These six radiomics features and all seven clinical features included in the predictive model were listed in Table 2.

Table 2

Variable selection.

Clinical Features	Radiomics Features
Time to SRS (days)	Grey-Level Dependence Matrix: Large Dependence High Gray-Level Emphasis
Primary Tumor Organ	Grey-Level Run Length Matrix: Long Run High Gray-Level Emphasis
Dose (Gy)	Shape: Maximum 3D Diameter
Age (years)	First Order: Uniformity
Conformity Index	Grey-Level Size Zone Matrix: Large Area High Gray-Level Emphasis
Cavity Volume	Neighboring Grey-Tone Difference Matrix: Strength
Gradient Index

The gradient boosting classifier trained on the clinical features to predict local control had an average accuracy of 0.54 and an average class-weighted F1 score of 0.51. The average area-under-the-curve (AUC) of this model was 0.40. The classifier trained on the radiomics features extracted from whole-cavity contours had an average accuracy of 0.76 on the validation sets. The average AUC of the classifier was 0.73, and the average class-weighted F1 score was 0.72. The ROC curves are shown in Fig. 3.

Fig. 3

ROC curves for the models generated using the radiomics features (solid line) and clinical features (dashed line).

The survival curves for the “low” and “high” risk of local recurrence groups (Fig. 4) generated from the class probabilities predicted by the radiomics-based model demonstrate separation where patients in the “high” risk group showing relatively poorer freedom from local recurrence times. However these do not rise to the level of significance (p = 0.06).

Fig. 4

Kaplan-Meier curves for the “low” and “high” risk of local recurrence groups as predicted by the gradient boosting classifier and delineated by cutpoint analysis.

Discussion

We conducted a retrospective study to evaluate outcomes in patients receiving adjuvant SRS to the surgical cavity for resected metastatic CNS disease and to evaluate clinical and radiomics features for their predictive value. Prior work has demonstrated mixed correlation of clinical features such as cavity diameter or volume with local control [7], [9], [12], [14], [23], [24]. These previous studies, however, have not evaluated additional image features and their predictive capability on local tumor control.

The area is felt to be at highest risk of harboring microscopic residual disease following resection in the wall of the cavity and the immediate surrounding brain parenchyma. We hypothesized that the cavity texture characteristics, rather than the standard reference cavity diameter or volume, would be a more robust predictor of local tumor recurrence. Our radiomics based model demonstrated marked improvement in predictive capabilities over the model trained with clinical features alone.

In comparison to other studies, the target delineation for the patients examined in this study consisted of the cavity wall without additional volumetric expansions. This led to smaller target volumes than utilized in either the N107C trial or the volumes proposed by Soliman et al. [5], [15]. Nevertheless, we observed 12 month local control and overall survival rates of 76% and 55%, respectively, which compare favorably with previous trials and other retrospective series derived from similar patient populations [10], [11], [12]. In light of these numbers, our study supports the body of literature demonstrating the efficacy of SRS alone without WBRT in this patient population. We did observe a significantly higher risk of distant failure among patients who also developed local recurrences although this finding could potentially be attributed to a sign of more aggressive disease as opposed to a causal relationship.

A potential limitation of our current study is the relatively small number of local failures during the follow up interval. This limitation primarily manifests itself as an increase in model volatility. This may also explain the lack of a statistically significant difference in our identified high and low risk patient populations using our radiomics model (Fig. 4).

Another potential limitation is the variability in MRI acquisition between patients. A large proportion of the acquired images were obtained at a 1.5 T field strength. While we cannot completely disregard the possible impact of field strength on our selected radiomics features, other groups have previously shown that radiomics features are robust to such alterations [24]. Furthermore, while field strength varied among our selected patients, the resolution of the images was uniform within the cohort with 1 mm isotropic voxel sizes, thus limiting variances in imaging acquisition as a potential confounding factor. Finally, the histogram matching we performed helped to eliminate the overall signal level variations related to field strength signal-to-noise ratio differences.

Conclusion

In conclusion, our radiomics features-based model demonstrated performance gains over clinical features alone for predicting local recurrence following adjuvant radiosurgery to resected brain metastases in this single institutional cohort. Additionally, our study supports the findings from several other groups demonstrating a lack of a strong interaction between the cavity size and the risk of subsequent local failure following adjuvant SRS. Identifying patients at risk of developing local recurrence remains a challenge, and the utilization of radiomics features to improve recurrence prediction should be validated in a large prospective cohort to fully assess the robustness of these novel models.

Funding

This work was supported in part by a Grant-in-Aid of Research, Artistry and Scholarship through the University of Minnesota, grant 416184, and by the National Institutes of Health’s National Center for Advancing Translational Sciences, grants TL1R002493 and UL1TR002494. The content is solely the responsibility of the authors and does not necessarily represent the official views of the National Institutes of Health’s National Center for Advancing Translational Sciences.

Declaration of Competing Interest

The authors declare that they have no known competing financial interests or personal relationships that could have appeared to influence the work reported in this paper.