A functional artificial neural network for noninvasive pretreatment evaluation of glioblastoma patients.

BACKGROUND: Pretreatment assessments for glioblastoma (GBM) patients, especially elderly or frail patients, are critical for treatment planning. However, genetic profiling with intracranial biopsy carries a significant risk of permanent morbidity. We previously demonstrated that the CUL2 gene, encoding the scaffold cullin2 protein in the cullin2-RING E3 ligase (CRL2), can predict GBM radiosensitivity and prognosis. CUL2 expression levels are closely regulated with its copy number variations (CNVs). This study aims to develop artificial neural networks (ANNs) for pretreatment evaluation of GBM patients with inputs obtainable without intracranial surgical biopsies.
METHODS: Public datasets including Ivy-GAP, The Cancer Genome Atlas Glioblastoma (TCGA-GBM), and the Chinese Glioma Genome Atlas (CGGA) were used for training and testing of the ANNs. T1 images from corresponding cases were studied using automated segmentation for features of heterogeneity and tumor edge contouring. A ratio comparing the surface area of tumor borders versus the total volume (SvV) was derived from the DICOM-SEG conversions of segmented tumors. The edges of these borders were detected using the canny edge detector. Packages including Keras, Pytorch, and TensorFlow were tested to build the ANNs. A 4-layered ANN (8-8-8-2) with a binary output was built with optimal performance after extensive testing.
RESULTS: The 4-layered deep learning ANN can identify a GBM patient's overall survival (OS) cohort with 80%-85% accuracy. The ANN requires 4 inputs, including CUL2 copy number, patients' age at GBM diagnosis, Karnofsky Performance Scale (KPS), and SvV ratio.
CONCLUSION: Quantifiable image features can significantly improve the ability of ANNs to identify a GBM patients' survival cohort. Features such as clinical measures, genetic data, and image data, can be integrated into a single ANN for GBM pretreatment evaluation.

This study establishes a noninvasive pretreatment evaluation of GBM patients.

Artificial neural network (ANN) performs deep learning with 4 inputs.

ANN facilitates GBM prognosis prediction and can guide treatment planning.

Pretreatment evaluation is critical for personalized GBM treatment planning, particularly for those frail and elderly patients. Artificial neural network (ANN) provides a unique opportunity to integrate critical information from different clinical aspects to facilitate the decision-making process right at the diagnosis of the GBM. This study provides a model of ANN that uses 4 simple inputs, that is, CUL2 copy number, Karnofsky Performance Scale (KPS), age at GBM diagnosis, and surface versus volume (SvV) ratio of T1 MRI, all of which are obtainable without intracranial interventions. This study offers a new perspective in how to incorporate recent development of machine learning (ML) in better cancer patient care.

Glioblastoma (GBM) is an aggressive form of tumor in the central nervous system (CNS), with less than 5% of patients surviving for 5 years following initial diagnosis.[1] While concurrent and adjuvant chemoradiotherapy constitutes standardized treatment for GBM after the Stupp et al. clinical trial in 2005, finding methods of refining treatment planning are urgently needed for better clinical outcomes.[2] Furthermore, treatment for elderly and frail GBM patients is not standardized as they were excluded in the Stupp et al. trial.[2] Therefore, pretreatment evaluation is extremely valuable in guiding the selection of treatment options.

Recent advances on GBM radiomics as well as machine learning (ML) applications in neuroimaging analysis promises a personalized treatment strategy that can lead to the paradigm shift for GBM management.[3,4] Aspects of deep learning (DL), a subfield of ML concerned with multi-layered ANNs, stand to benefit GBM research in particular given that applications in incorporating medical image processing and genetic data mining.[4-6] However, acquisition of genetic data, including RNA-sequence (RNA-Seq), isocitrate dehydrogenase (IDH) mutation, and cytosine-phosphate-guanine (CpG) island methylator phenotype (G-CIMP) status, requires intracranial biopsy which carries a significant risk of perioperative complication.[7] Therefore, a noninvasive method is urgently needed in the presurgical evaluation of GBM patients, particularly for those elderly and frail.[8,9]

We recently demonstrated that expression levels of CUL2, which encodes the cullin2 scaffold protein in the cullin2-RING E3 ligase (CRL2) complex, can predict GBM radiosensitivity and prognosis.[10] CRL2 is the E3 ligase that degrades critical proteins such as hypoxia-inducible factor 1 (HIF-1) α that is involved in GBM neovascularization, and epidermal growth factor receptor (EGFR) that is involved in GBM progression.[10]CUL2 copy number variation (CNV) dictates expression levels, suggesting that models could rely on CNVs measured through noninvasive methods rather than expression levels to predict patients’ survival rates.[10,11] In this study, we attempted to integrate quantifiable image data, genetic information such as CUL2 CNVs, and clinical data such as Karnofsky Performance Scale (KPS) in a functional ANN for noninvasive presurgical evaluation GBM patients.

Materials and Methods

Public Datasets

Datasets including genetic data regarding CUL2 CNVs and expression levels, clinical data indicating patient demographic and overall survival (OS), and T1 magnetic resonance imaging (MRI) images enabled this study. Such datasets include The Cancer Genome Atlas Glioblastoma (TCGA-GBM), the Ivy Glioblastoma Atlas Project (Ivy-GAP), and the Chinese Glioma Genome Atlas (CGGA).[12-14] Clinical and genetic data from the TCGA-GBM were acquired through the Xena platform (https://xena.ucsc.edu/).[15] TCGA-GBM images were made available through The Cancer Imaging Archive (TCIA).[16] This also includes skull-stripped and co-registered segmentations of TCGA-GBM images made available by Bakas et al., as well as DICOM-SEG conversions of these segmented images created by Beers et al.[16] Images from the Ivy-GAP dataset were also acquired through TCIA, with clinical and genetic data made available by Puchalski et al.[13]

T1 MRI Image Segmentation

While images considered in this research included post-gadolinium T1-weighted DICOM images from the original TCGA-GBM dataset, the features used in modeling were derived from the most voluminous DICOM-SEG conversions of images in the segmented image dataset for each patient. Python packages for processing image files in the DICOM, DICOM-SEG, and NIfTI file formats include pydicom, pydicom_seg, and nibabel, respectively.

Image Analyses

To explore the value of image features in forecasting patient outcomes, a ratio comparing the surface area of tumor borders to the total volume of borders was derived from the DICOM-SEG conversions of segmented tumors. The edges of these borders can be detected using the canny edge detector such as the one available from the Skimage, or Scikit-image, Python package. Finding the sum of the pixels forming these edges gives a close approximation of surface area. Tumor border volume can be calculated by summing up all pixels within the tumor borders. Calculating the tumor surface area versus volume (SvV) ratio between the two is then as simple as dividing the surface area by volume (SvV = Surface Area/Volume). This results in a value unique to each patient that is indicative of tumor border regularity.[17]

Kaplan–Meier (K–M) Survival Analysis

K–M survival analyses for GBM with differential CUL2 copy numbers, KPS rankings, age at GBM diagnosis, surface area, and SvV ratios were conducted using the Lifelines library. The data set is split into two at the mean value of any of respective attributes above. Any value above the mean for that attribute is placed in the upper group, while any value below the mean is placed in the lower group. Therefore, the number of patients in either group varies from attribute to attribute. The number of patients in any one group is labeled next to the legend for that group on its graph. Log-Rank analyses for P-values smaller than .05 were considered statistically significant.

ANN of GBM Genomics and Clinical Features

Four different ANNs were created for the purpose of predicting patients’ OS cohort and to test whether measuring patient CUL2 copy numbers would yield similar results to CUL2 expression levels. Results are determined by how often a neural network can identify a patient’s OS cohort based on clinical data. Four different loss functions were used to test on all 4 of our neural networks, resulting in 16 different results to compare. The loss functions we chose were Binary Crossentropy, Mean Absolute Error, Mean Error Squared, and Categorical Crossentropy. The packages used for creating these neural networks include Keras, Pytorch, and TensorFlow. After extensive testing, a 4-layer model (8-8-8-2), with a binary output, was built. The first layer consists of 8 nodes and takes number of inputs based on the information we are passing to it. The groups of information being passed to our neural networks are as follows: Baseline (Age, KPS, longest dimension), Expression (CUL2 expression, Age, KPS, longest dimension), CNV (CUL2 copy numbers, Age, KPS, longest dimension), and Feature Data (CUL2 copy numbers, Age, SvV). Each neural network consequently requires different numbers of inputs. The second and third layer feature 8 nodes and employ a ReLU activation function. Finally, the output layer is binary.

The output layer of the ANN splits each dataset into OS cohorts. Patients who are assigned a “1” survived longer, and those who did not are assigned a “0.” The data are split to be approximately 55%–45% in favor of “0” targets, with the exception of the feature dataset which is split approximately 60%–40% in favor of “0” targets. This is to ensure a balance of training and testing datasets with the relevant features. Figure 3A is a schematic overview of the architecture of the ANN.

Figure 3.

Parameterizing artificial neural networks (ANNs) based on CUL2 copy number variations (CNVs), image features and clinical data. (A) A schematic overview of the 4-layer neural networks with the Feature Data with inputs of CUL2 copy numbers, age at glioblastoma (GBM) diagnosis, and SvV. (B) Kaplan–Meier analysis of the test set of GBM cases assigned by the Feature Data-based neural network to group “1” and “0”. (C) Average performance of 4 neural networks based on 4 set of inputs including baseline (age, Karnofsky Performance Scale [KPS], longest dimension), expression (CUL2 expression, age, KPS, longest dimension), CNV (CUL2 copy numbers, age, KPS, longest dimension), and feature data (CUL2 copy numbers, age, KPS, SvV).

Results

CUL2 CNVs and GBM Genotype

Copy number loss of chromosome (chr) 10 is one of the initial driver events of non-cytosine-phosphate-guanine (CpG) island methylator phenotype (non-G-CIMP) GBM tumorigenesis.[18] In GBM patients (n = 576) derived from TCGA dataset, we found that copy number loss of CUL2, located in chr10p11 (chr10p), is related with non-G-CIMP, IDH1 wildtype (IDH1WT) GBM (Figure 1A). IDH1 mutations status and G-CIMP positivity are key genotypic biomarkers critical for GBM classification, prognosis, and treatment planning.[19] Therefore, these results indicate that CUL2 copy numbers can reflect the fundamental genetic background of GBM. CUL2 CNVs have a strong positive correlation with its expression levels in GBM (Figure 1A). We went further to study the correlation of CUL2 CNVs and expression in other cell types derived from both cranial and extracranial tissue. We found linear correlation between CUL2 CNVs and expression levels in cancer cell lines derived from cranial (eg, glioma, medulloblastoma, meningioma) and extracranial (eg, B-/T-cell leukemia, multiple myeloma, small cell lung cancer [SCLC], pancreatic cancer, soft tissue, breast cancer, thyroid cancer) tissues (Figure 1B). Notably, the linear correlation between CUL2 CNVs and expressions levels in pancreatic cancer and SCLC exhibit a coefficient of determination (R2) (0.71 and 0.78, respectively) and may be described by equations (y = 1.021*x+ 3.4467 and y = 1.288*x + 3.7066, respectively). These relationships prove similar to those in glioma cancer cells (R2 = 0.71, y = 1.15*x + 3.6955) (Figure 1C–E). This elucidates the valuable linear relationship between CUL2 CNVs and expression levels. CUL2 levels in extracranial tissue may also reflect expression levels in central neural system (CNS) malignancies.

Figure 1.

CUL2 copy number variations (CNVs) and expression levels. (A) Glioblastoma (GBM) cases (n = 576) from TCGA-GBM dataset were aligned based on the CUL2 CNVs in the xena platform. Corresponding data regarding CUL2 expression levels, cytosine-phosphate-guanine (CpG) island methylator phenotype (G-CIMP) and IDH1 mutation status were color coded. Gray color indicates cases with no available data for that particular genetic marker. (B) The correlation between CUL2 CNV and expression levels in different cell lines. Data derived from Broad Institute Cancer Cell Line Encyclopedia (CCLE). (C–E) The correlation between CUL2 CNV and expression levels in cell lines of glioma, pancreatic cancer, and small cell lung cancer.

GBM T1 MRI Segmentation and Tumor Surface Regularity

Many radiomic features serve as predictors for GBM OS rates.[20] These include attributes such as first order statistics, texture features, and data describing the shape of the tumor. One method of systematically measuring shape involves finding the SvV of tumor borders.[21] Using the most voluminous binary segmentation masks attainable from DICOM-SEG (DSO) conversions for the TCGA-GBM segmentation dataset for each patient, the surface areas of tumor borders were approximated by summing the number of pixels belonging to edges of 2D slices determined via canny edge detection (Figure 2A). The volumes of segmented tumors masks were then approximated by summing up all pixels of each slice (Figure 2A). Applying this approach on the segmented image dataset creates surface area (S), volume (V), and SvV ratio data for 102 patients, 98 of these patients have CUL2 CNV data. We found the correlation of S and V with equation S = 3.5887*V0.7228 (R2 = 0.8346) (Figure 2C). Calculation of partial derivative of the equation gets the equation ∂s/∂v = 2.59391*v-0.2772 (Figure 2B,C). These results indicate that SvV ratios provide unique feature to GBM patients (Figure 2B,C).

Figure 2.

Glioblastoma (GBM) T1 image segmentation. (A) Example of binary segmentation masks attainable from DICOM-SEG (DSO) conversions for the TCGA-GBM dataset. The surface area of tumor borders calculated by summing up edges of 2D slices derived with a canny edge detector. The volumes were approximated by summing up all pixels of each slice. (B) GBM cases (n = 84) were aligned based on increasing volume. The surface area, surface versus volume (SvV), and partial derivative ∂s/∂v were aligned accordingly. (C) Relationship between surface area and volume of GBM based on segmented images (n = 84).

Segmented Image Features

To further investigate the clinical relevance of proposed segmentation methods, we studied the prognostic value of quantifiable image features in GBM. These features include the total surface area, tumor volume and SvV ratio. In Kaplan–Meier analyses where median values are used as cutoffs, we found no difference in OS between groups with High and Low values for surface, volume and SvV values (Supplementary Figure 1). However, given that each parameter in the MRI images is unique for individual GBM patient, we tested these image features as additional inputs in our neural network to evaluate their predictive value.

Constructed ANNs Indicate Likely GBM Survival Cohort

The chosen loss function may afford or disallow optimal ANN performance depending on the nature of the problem.[22] To find an appropriate loss function for the classification of a patient’s OS cohort, we tested 4 different loss functions on 4 neural networks. This resulted in 16 different results to compare. The loss functions we tested included Binary Crossentropy, Mean Absolute Error, Mean Error Squared, and Categorical Crossentropy (Supplementary Figure 2).[22] The structure of the ANN we chose is a 4-layer model (8-8-8-2), with a binary output (Figure 3A). Due to limited cases with image data to calculate SvV, we used the longest dimension of GBM tumor mass as an input in certain ANNs. The first layer consists of 8 nodes with sets of inputs as follows: Baseline dataset (age, KPS, longest dimension), Expression dataset (CUL2 expression, age, KPS, longest dimension), CNV dataset (CUL2 copy numbers, age, KPS, longest dimension), and Feature Data dataset (CUL2 copy numbers, age, SvV) (Supplementary Figure 2). The second and third layer are also 8 nodes with a rectified linear activation function (ReLU). The last layer features binary output. Patients who are assigned a “1” will likely exhibit greater OS, and those who are assigned a “0” will not (Figure 3A). As expected, the Baseline neural network with fewer features performed poorest in all 4 trials (Supplementary Figure 2). We found the CNV neural network slightly outperformed the Expression neural network (Supplementary Figure 2, Figure 3C). Therefore, CUL2 expression levels or CUL2 copy numbers will yield similar results, with around 75% accuracy as displayed on the average of the 4 loss functions combined in a single visual (Figure 3C). Employing image feature data as inputs afforded the greatest performance of about 80%–85% (Figure 3C).

To achieve these results and identify the set of inputs that produce the best performing ANN, we performed a set number of trials for each of our 16 neural networks. Each of the ANNs were run 1000 times for a total of 16,000 trials and 16 ANNs. The best performing model for each input was saved for the purpose of distribution and to view the model’s structure and weights. To reiterate, the best performing ANN employed the Feature Data dataset with CUL2 copy numbers, Age, and SvV to allow a consistent 80%–85% accuracy across all loss functions (Supplementary Figure 2, Figure 3C). The Kaplan–Meier survival analysis for the test set of 20 GBM patients in the Feature Data group was presented (P = .0005; Figure 3B).

Discussion

We previously demonstrated that one may evaluate CUL2 gene expression levels and CNVs to predict radiosensitivity and overall survival (OS) in GBM patients.[10] This study explored the potential value of this concept to the training of ANNs and deep learning (DL) of clinical information, T1 MRI-based imaging features, and CUL2 copy numbers in pretreatment evaluation. In our best performing ANN, we consistently identified the OS cohort of GBM patients with accuracies of 80%–85% using CUL2 copy numbers, patients’ ages at GBM diagnosis, and SvV ratio in segmented images as inputs. All these inputs are objective quantifiable parameters that can be obtained without any intracranial biopsy which carries significant risk of causing serious permanent morbidity (5%).[7] Therefore, our model provides a unique tool for noninvasive presurgical evaluation of GBM patients regarding prognosis, which will be of great value for treatment planning.

We found the usage of tumor border surface area and volume led to improved accuracy for each ANN model. This not only suggests that the uniqueness of these measurements, but also illustrates the general value of quantifiable features derived from image processing in anticipating treatment outcomes. In addition to including other measurements of shape indicating surface regularity, future models could also potentially integrate first order statistics regarding gray level intensity, textural features, and a myriad of radiomic features with demonstrable impact on GBM survival. Artificial intelligence-based automated image segmentation will play a critical role in this aspect.

Processing medical images such as T1 images involves numerous challenges, but several approaches exist to extract hundreds of potentially useful features of each category. Image processing solutions that employ convolutional neural network architectures such as 3D U-Net to automate tumor segmentation allow the systematic tumor segmentation, though atlas. Ultimately, additional image features derived using DL solutions stand to improve the already promising model by introducing more noninvasive data alongside CUL2 CNVs, further indicating the value of combining such measurements to predictions concerning treatment response.

Although we built a promising model for noninvasive presurgical evaluation of GBM patients, several steps may be performed to further test and enhance these findings. Firstly, the incorporation of additional image features would likely improve the predictive value of our existing neural networks. Secondly, solely leveraging data from select prior studies for both training and testing may limit generalizability. Clinical trial and additional testing of the model on other cohorts would assist in validating whether the given ANNs can guide RT treatment planning in future contexts. Furthermore, the reliance on subjective clinical parameters may introduce unnecessary risk. Developing neural networks that perform consistently with only imaging data and patients’ age would markedly improve the value of such models. Finally, the value of the discussed model is limited by the binary output layer. Future models developed with additional data could support a higher resolution of output and bolster treatment planning.

In summary, the previously established relationship between CUL2 CNVs and clinically significant attributes such as radiosensitivity prompted the testing of ANNs capable of aiding in pretreatment evaluation. Noninvasive clinical, genetic, and image data were leveraged in training to afford the ability to identify a patient’s OS cohort with accuracies of 80%–85%. The best performing model consistently achieved these scores by training with CUL2 CNVs, patient age, and tumor surface area versus volume. Further testing with additional cohorts and the introduction of additional image features may improve generalizability, enhance output resolution, and afford greater accuracy. Nonetheless, the given model stands to both assist GBM pretreatment evaluation and demonstrate the value of DL in the relevant decision-making processes.

Click here for additional data file.