Recognition of moyamoya disease and its hemorrhagic risk using deep learning algorithms: sourced from retrospective studies.

Although intracranial hemorrhage in moyamoya disease can occur repeatedly, predicting the disease is difficult. Deep learning algorithms developed in recent years provide a new angle for identifying hidden risk factors, evaluating the weight of different factors, and quantitatively evaluating the risk of intracranial hemorrhage in moyamoya disease. To investigate whether convolutional neural network algorithms can be used to recognize moyamoya disease and predict hemorrhagic episodes, we retrospectively selected 460 adult unilateral hemispheres with moyamoya vasculopathy as positive samples for diagnosis modeling, including 418 hemispheres with moyamoya disease and 42 hemispheres with moyamoya syndromes. Another 500 hemispheres with normal vessel appearance were selected as negative samples. We used deep residual neural network (ResNet-152) algorithms to extract features from raw data obtained from digital subtraction angiography of the internal carotid artery, then trained and validated the model. The accuracy, sensitivity, and specificity of the model in identifying unilateral moyamoya vasculopathy were 97.64 ± 0.87%, 96.55 ± 3.44%, and 98.29 ± 0.98%, respectively. The area under the receiver operating characteristic curve was 0.990. We used a combined multi-view conventional neural network algorithm to integrate age, sex, and hemorrhagic factors with features of the digital subtraction angiography. The accuracy of the model in predicting unilateral hemorrhagic risk was 90.69 ± 1.58% and the sensitivity and specificity were 94.12 ± 2.75% and 89.86 ± 3.64%, respectively. The deep learning algorithms we proposed were valuable and might assist in the automatic diagnosis of moyamoya disease and timely recognition of the risk for re-hemorrhage. This study was approved by the Institutional Review Board of Huashan Hospital, Fudan University, China (approved No. 2014-278) on January 12, 2015.

Chinese Library Classification No. R445; R743.34; Q-334

Introduction

Moyamoya disease (MMD) is a chronic cerebrovascular disease that is characterized by progressive stenosis and occlusion of the supraclinoid internal carotid artery (ICA) and its proximal branches and abnormal collateral vessels at the base of the brain, both with unknown etiologies (Suzuki and Kodama, 1983; Su et al., 2019). The pathological angioarchitecture of MMD involves bilateral ICA, whereas patients with unilateral moyamoya vasculopathy are generally categorized as quasi-MMD or moyamoya syndrome (Research Committee on the Pathology and Treatment of Spontaneous Occlusion of the Circle of Willis and Health Labour Sciences Research Grant for Research on Measures for Infractable Diseases, 2012). Nevertheless, the disease and syndrome are often investigated together because of their common clinical presentations and surgical strategies (Scott and Smith, 2009). Intracranial hemorrhage is a major clinical manifestation of moyamoya among adults, and several morphological changes such as fragile moyamoya vessels and saccular aneurysms in the circle of Willis are thought to be the main causes (Kuroda and Houkin, 2008). Published studies indicate that untreated hemorrhagic MMD presents with high rebleeding rates (Kobayashi et al., 2000; Kang et al., 2019). Therefore, risk factors of hemorrhage remain after the initial bleeding and recognition of these features is crucial for predicting future rebleeding events.

Recently, machine learning has been widely recognized as a powerful tool for discovering hidden information that may not be expressed explicitly. Commonly used algorithms in neuroscience include the support vector machine, Bayesian algorithm, and artificial neural network (Lo et al., 2013; Fukuda et al., 2014; Wang, 2014). Models that incorporate these algorithms often need a manually labeled dataset based on human experience and recognition. Thus, some crucial but unknown information may be omitted. The convolutional neural network (CNN) is a deep learning algorithm inspired by biological neuronal responses and is designed to extract information automatically (Cireşan et al., 2013). The application of CNNs in image recognition and facial recognition is considered a landmark because the efficiency of object classification and detection is very high (Ciregan et al., 2012; Krizhevsky et al., 2017). Additionally, CNNs have achieved state-of-the-art accuracies in joint prediction from multi-view images of three-dimensional shapes (Su et al., 2015).

In the present study, we first applied a pre-trained deep residual neural network (ResNet) to detect hemispheric moyamoya vasculopathy after learning the relevant features of the ICA as seen on digital subtraction angiography (DSA) (He et al., 2016). Next, to detect hemorrhagic risk in moyamoya, we applied a combined multi-view CNN (MV-CNN-C) algorithm that integrated individual clinical characteristics and DSA features of all intracranial vessels on the side of the brain with a history of hemorrhage. The models were finally assessed through cross-validation.

Participants and Methods

Participants

The inclusion criteria were as follows: (1) Aged between 18 and 68 years; (2) Diagnosis confirmed by DSA and in accordance with published guidelines (Research Committee on the Pathology and Treatment of Spontaneous Occlusion of the Circle of Willis and Health Labour Sciences Research Grant for Research on Measures for Infractable Diseases, 2012); (3) No evidence of other cerebrovascular diseases, brain tumor, brain trauma, or any medical history of neurosurgery. From January 2017 to September 2019, 460 eligible adult patients with moyamoya (418 MMD and 42 moyamoya syndrome) were retrospectively identified using data from the Department of Neurosurgery, Huashan Hospital located at Fudan University in China. The bilateral intracranial vessels in cases of MMD and the unilateral intracranial vessels in cases of moyamoya syndrome with vasculopathy were collected as positive samples.

Additionally, 500 adult patients with unruptured unilateral intracranial aneurysms were selected from the hospital’s database and their contralateral intracranial vessels were used as negative samples. The above 500 patients were involved after being screened through the following exclusion criteria: (1) Those aged over 68 years; (2) Those with aneurysms located in either the anterior communicating artery or posterior circulation which may interfere with feature recognition; (3) Those with evidence of any obvious abnormalities in the hemisphere contralateral to the aneurysms.

All patients in our database were diagnosed independently by two senior neurosurgeons as routine procedures. If a consensus was not reached, the whole treatment team discussed the case together and came to a final consensus. This study was conducted in accordance with the Declaration of Helsinki after approval by the Institutional Review Board of Huashan Hospital, Fudan University (approval No. 2014-278). All participants or their legal guardians provided informed consent.

Diagnosis modeling

Referring to the definition and diagnostic criteria (Research Committee on the Pathology and Treatment of Spontaneous Occlusion of the Circle of Willis and Health Labour Sciences Research Grant for Research on Measures for Infractable Diseases, 2012), we examined unilateral ICA angiography for moyamoya vasculopathy. Thus, dynamic raw DSA data from the unilateral ICA were defined as a sample. Considering the usual clinical practice and algorithm simplification, only images in the anteroposterior position view were used. In total, we collected 878 positive samples (bilateral hemispheres from the 418 patients with MMD and unilateral hemispheres from the 42 patients with moyamoya syndrome) and 500 negative samples.

First, all right hemisphere samples were flipped horizontally to align with the left hemisphere samples. Then, randomized crops and rotations were applied to the input images to improve the robustness of the small displacements and orientations. The brightness, contrast, saturation, and hue of the images were adjusted, with adjustment factors being randomly selected from within the interval [0.9, 1.1]. Transformation procedures were applied before each epoch of model training.

A deep ResNet-152 model (CVPR 2016, Las Vegas, NV, USA) was initiated by ImageNet (CVPR 2009, Miami, FL, USA) with pre-trained weights. This model was fine-tuned using 30 epochs and a minibatch size of 32. The learning rate was 0.001 with an exponential decay factor of 0.1 for every seven epochs. A five-fold cross-validation strategy was applied to avoid sampling bias. Thus, the samples were divided into five sets with equal positive/negative sample ratios, and four sets were used for training, and the remaining set was used for validation. Then, the whole procedure was repeated five times until all sets had been used once for validation. The final result was the mean of the five individual validations. Afterward, the model was evaluated by calculating the sensitivity and specificity, as well as the area under the receiver operating characteristic (ROC) curve.

Hemorrhagic-risk modeling

Dataset construction

The natural history of hemorrhagic MMD indicates that rebleeding episodes frequently occur in the original hemisphere at different sites (Houkin et al., 1996; Saeki et al., 1997; Ryan et al., 2012; Kang et al., 2019). Thus, hemispheres with bleeding remain at high risk for future bleeding, which might not be attributed to a single feature. Thus, the features of these hemorrhagic hemispheres should be learned. In total, we obtained 126 positive samples with prior bleeding episodes (ipsilateral hemispheres from 118 cases of MMD and 8 cases of moyamoya syndrome) and 634 negative samples without any history of intracranial bleeding (bilateral hemispheres from 300 MMD cases and 34 ipsilateral hemispheres from cases of moyamoya syndrome). The contralateral hemispheres from cases of hemorrhagic MMD were excluded considering their unclear involvement in some intraventricular hemorrhage episodes and obscure basis for grouping.

For each sample, demographic characteristics (age, gender, and risk factors of hypertension, smoking, and drinking) were collected for modeling, as well as dynamic DSA raw data of the ICA, external carotid artery, and vertebra-basilar artery in both anteroposterior and lateral position views. Afterward, all temporal DSA images of the same artery and position were integrated into one combined image, and all combined images of the same hemisphere were stored together.

Development of the CNN algorithms

We proposed the MV-CNN to extract image features, and its architecture can be seen in . Feature maps of the input images were extracted by two feed-forward densely connected convolutional blocks (Huang et al., 2017). The dense block comprised five convolutional layers, all of which were forwardly connected with each other to reduce information loss and gradient vanishing. Each input image generated 256 feature maps, the aggregation of which was learnable (Su et al., 2015). Instead of direct max-pooling, all feature maps were jointly reweighted by the corresponding adaptive importance vector, which was learned from feature maps by a two-layer squeeze and excitation block (Hu et al., 2018). The reweighted feature maps were encoded by two shared dense blocks to generate the final image feature vector X ∈ R4416×1. The feature vector X was followed by a fully connected layer with Softmax activation, which outputted a prediction of risks R = σ(βT X), where β ∈ R4416×2 was the weights vector of the fully connected layer and σ was the Softmax activation function. To provide a loss function for backpropagation, the risks were inputted to a cross-entropy layer for calculating the negative log likelihood: (β, X, Y) = –Σ αtyi log(σ(βTxi)), where xi and yi were the image feature vector and its corresponding risk annotation. Specifically, we introduced a weighted factor αt as the inverse class frequency, which was 634/(126 + 634) for positive samples and 126/(126 + 634) for negative samples. Thus, the positive and negative samples contributed equally to the total loss. A stochastic gradient descent with momentum was used to minimize the negative log likelihood via backpropagation that optimized model weights and biases. The momentum was set to 0.9 and the initial learning rate was set to 0.01, which was applied with an exponential decay factor of 0.1. The feature extraction layers for each input image were initialized by the ImageNet pre-trained weights, whereas the dense blocks that followed were initialized (Glorot and Bengio, 2010; Lo et al., 2013). Models were trained for 200 epochs in which the size of the minibatch was four. Batch normalization and dropout were used to mitigate overfitting (Ciregan et al., 2012; Ioffe and Szegedy, 2015).

Methodological process of DSA features extraction (A, B) and the architecture of a combined deep multi-view convolutional neural network (C).

Image feature maps of the input images were extracted by two feed-forward densely connected convolutional blocks. The dense block was composed of five convolutional layers, and all the layers were forwardly connected with each other to reduce the information loss and gradient vanish. A two-layer Squeeze and Excitation block was used to learn from feature maps. A gradient boosting decision tree method was finally proposed to integrate image with clinical features of demographic and vascular risk factors. AP: Anteroposterior position; DSA: digital subtraction angiography; ECA: external carotid artery; GBDT: gradient boosting decision tree; ICA: internal carotid artery; LAT: lateral position; S.E.: Squeeze and Excitation block; VA: vertebra-basilar artery.

Afterward, the gradient-boosting decision-tree method was used to integrate images with clinical features and development of the MV-CNN-C algorithm (“C” for combined; ). As a multivariant tree-boosting method, the gradient-boosting decision tree is often used to integrate and analyze multiple factors. It produces the prediction from the linear ensemble of multiple decision trees by iteratively reducing the training residuals, which enables the quick capture of the differentiated feature combinations. To normalize the features, the aggregation feature vectors were Z-normalized and the major components of both the clinical and image features were extracted via the singular value decomposition.

We compared performance of the MV-CNN-C with the MV-CNN to evaluate the significance of clinical features. Two other basic CNN models (vanilla CNN and MV-CNN-NA) were also constructed and compared. The vanilla CNN architecture is similar to that of DenseNet, except that the number of nodes in the last fully connected layer is changed to three, whereas the input is the concatenate of all the input images along the width axis (Huang et al., 2017). The MV-CNN-NA architecture is similar to that of the MV-CNN in that the separate feature vectors from each input image are combined directly by concatenation instead of by the squeeze-and-excitation block.

Model training and validation

Samples were randomly assigned to nonoverlapping training (80%) and validation (20%) sets. For MMD with bilateral samples, they were simultaneously assigned to the same training or validation set. This ensures that no data from any patient were represented in both training and validation sets at the same time, and avoids overfitting and optimistic estimates of generalization accuracy. The randomized assignment was repeated five times until all sets had been used for validation. Afterward, the final validation result was generated as the mean of the five individual validations.

Statistical analysis

The models were trained using PyTorch 0.4.0 () under python 3.5 (https://www.python.org/downloads/release/python-350/) on servers equipped with Intel(R) Core (TM) i7-6800K CPU @ 3.40 GHz CPUs, 64 GB RAM, and dual NVIDIA GTX 1080Ti graphic cards. DSA data were obtained from the Philips and GE X-ray intensifiers in Huashan Hospital, Fudan University (Philips UNIQ Clarity FD20/20, Philips, Amsterdam, Netherlands; GE Innova IGS 630; GE, Boston, MA, USA).

Results

Diagnosis modeling of MMD

The clinical and image characteristics of all participants in this study are summarized in . Patients with MMD, moyamoya syndrome, and intracranial aneurysm did not show significant differences in age, gender, or current smoking or drinking status (P > 0.05). Significant differences were found for hypertension (P < 0.001). Most patients with MMD exhibited a Suzuki stage of III or IV, which was similar in patients with moyamoya syndrome (P > 0.05). Additionally, hemorrhage rates did not differ significantly between MMD and moyamoya syndrome (P > 0.05).

DSA image features extracted through the ResNet-152 model for one sample is shown in . After repeating training and validation five times, the average accuracy of the proposed method was 97.64 ± 0.87%, with sensitivity and specificity of 96.55 ± 3.44% and 98.29 ± 0.98%, respectively. The quality of the model was also evaluated by a ROC, and the area under the ROC curve reached 0.990 ().

Example of image feature extraction from the digital subtraction angiography.

(A, F) The anteroposterior position view of the left internal carotid artery in randomly selected negative (A) and positive samples (F). (B–E, G–J) Based on this view of moyamoya, intensive features have been color-coded to be bright.

The ROC for diagnosis modeling of moyamoya disease which was constructed by the ResNet-152 algorithm.

The orange curve shows that value increases quickly to nearly 1.0, which shows that algorithm performance was good and the area under receiver operating characteristic curve was high. ResNet: Residual neural network; ROC: receiver operating characteristic.

Hemorrhagic risk modeling of MMD

The baseline characteristics of adult moyamoya with an episode of prior bleeding are shown in . To determine whether the model that we constructed had any advantages, we compared the performance of the MV-CNN-C algorithm with that of the MV-CNN, vanilla CNN, and MV-CNN-NA (). The results indicated that the MV-CNN-C reached the highest mean classification accuracy and precision, implying that the gradient boosting decision-tree algorithm (vs. the MV-CNN), the SE block (vs. the MV-CNN-NA), and the two-way input structure (vs. the vanilla CNN) all contributed to the improved performance of the MV-CNN-C.

Classification accuracy (A) and precision (B) among the MV-CNN, MV-CNN-C, MV-CNN-NA, and vanilla CNN models.

CNN: Convolutional neural network; MV-CNN: multi-view convolutional neural network; MV-CNN-C: combined multi-view convolutional neural network; MV-CNN-NA: multi-view convolutional neural network with squeeze and excitation block inapplicable.

As an example, deep features extracted from the fully connected layer of the MV-CNN-C for 16 positive and 16 negative samples were converted to 64 × 69 matrices, which revealed an obvious difference in features between samples (). After repeating training and validation five times, the mean accuracy of the proposed method was 90.69 ± 1.58%, and the sensitivity and specificity were 94.12 ± 2.75% and 89.86 ± 3.64%, respectively.

The 64 × 69 matrices of 16 randomly selected positive samples with prior bleeding episodes (A) and 16 negative samples without prior bleeding episodes (B).

The matrices were converted from features that were extracted from the fully connected layer of the MV-CNN-C model. The values ranged from 0 to 1, and darker colors indicate larger values. MV-CNN-C: Combined multi-view convolutional neural network.

Discussion

Here, we proposed a series of deep MV-CNN algorithms as a reliable, automatic, and objective tool for detecting cases of moyamoya disease/syndrome and for evaluating the clinical risk of hemorrhage. We developed a ResNet-152 model to extract image features related to moyamoya, resulting in improved diagnostic efficacy and automation, and laying a solid foundation for the detection of hemorrhagic risk. An MV-CNN-C model was then proposed to integrate both clinical and image features and generate a hemorrhagic risks classifier. Finally, the classifier was evaluated using a cross-validation strategy.

Referring to the natural history of hemorrhagic MMD, Kang et al. (2019) reported that rebleeding events occurred in 36.7% of patients who received conservative treatment. Additionally, Morioka et al. (2003b) revealed a rebleeding rate of 61.1% in another hemorrhagic MMD cohort. Therefore, hemorrhagic risk remains high after an initial bleeding episode and should be recognized and prevented. Furthermore, the sites of rebleeding have been reported to vary from the initial site, but are often in the same side (Houkin et al., 1996; Saeki et al., 1997; Kuroda and Houkin, 2008; Kang et al., 2019). Thus, we conclude that the hemisphere in which bleeding initially occurs in moyamoya remains at high risk of future rebleeding, and this might not be attributed to a single risk factor.

Of all the clinical characteristics reported in studies of hemorrhagic MMD, smoking is the only one that has been related to rebleeding, while hypertension has been proved irrelevant (Yoshida et al., 1999; Morioka et al., 2003a; Kang et al., 2019). Nevertheless, here we included all these clinical factors to avoid missing any relevant features. Previous studies have provided several morphological features via angiography, including fragile moyamoya vessels (Kuroda and Houkin, 2008), brand extension of anterior choroidal artery-posterior communicating artery (Morioka et al., 2003a; Jiang et al., 2014), and cerebral aneurysms developed from shift circulation (Kawaguchi et al., 1996). Although these features are deemed to be important references for clinicians, their use is limited and controversial. For example, fragile moyamoya vessels and some circulation-related aneurysms may vanish gradually as the disease progresses to high Suzuki stages. Thus, we are not sure whether hemorrhagic risks increase as the disease worsens. For brand extension of the anterior choroidal artery-posterior communicating artery, the diameters of the vessels should be calculated and correlated with hemorrhagic risks. Additionally, most of the aforementioned studies focused on univariant analysis of a single factor and generated a paradoxical result that is difficult to refer to in clinical practice. Thus, all these features should be considered simultaneously, quantified, and weighted to generate a practical risk recognition model. However, this is difficult based on the present clinical experience.

Compared with other intelligent models, the CNN-based deep learning model has several advantages. First, the CNN learning paradigm differs from the feature engineering paradigm in that the predictive features are adaptively transformed from the input images instead of manually designed and extracted, thus relieving clinicians of the burden of manual input and saving time. Second, the network is composed of non-linear transformations and learnable filter kernels that summarize the high-level semantic features from the low-level morphological textures. It does not use prior knowledge, which may include some hidden errors, but instead generates knowledge directly derived from the medical image data. Finally, elements of the structure, such as the number and size of filters, type of convolutional layers and blocks, loss functions, hyper-parameters, and even numbers of input pathways can be customized to be suitable for any classification, detection, segmentation, or other artificial intelligence tasks. This makes the CNN model much more flexible than the traditional models.

In the first stage, the ResNet model we proposed has a novel network structure called the residual unit, which consists of a primary path of several convolution layers and an alternative path that short-circuits the input to the output of the residual unit. This structure of the residual unit helps to counter the problem of vanishing and exploding gradients and results in many more layers and better performance than other deep learning networks (Deng et al., 2009). In the second stage, the MV-CNN-C model we developed can simultaneously extract a large amount of information from different dimensions, making it suitable for this application.

This study had several limitations. First, because developing and stabilizing deep learning models requires large amounts of data, the model in this study still needs to be improved by including a larger training set. Second, an independent testing set may provide a more convincing result. Third, features extracted from the deep learning algorithm are difficult to explain in terms of medical significance, and relevant clinical work of deep learning is needed in cooperation with clinicians and engineers.

In summary, the deep learning algorithms we proposed have been shown to be valuable and could assist in automatic diagnosis of MMD and timely recognition of the risk for rebleeding. We are establishing a national database to help build a better deep learning model through an ongoing multi-center study of MMD (A Multi-Center Registry Study of Chinese Adult Moyamoya Disease).

Table 1

Clinical and image characteristics of the participants

	Moyamoya disease (n = 418)	Moyamoya syndrome (n = 42)	Intracranial aneurysm (n = 500)	P-value
Age (yr)	44.5±9.6	44.2±12.9	45.7±7.6	0.169
Male	197 (47.1)	22 (52.4)	205 (41.0)	0.097
History of risk factors
Hypertension	66 (15.8)	8 (19.0)	189 (37.8)	< 0.001
Current smoking	78 (18.7)	7 (16.7)	109 (21.8)	0.42
Current drinking	81 (19.4)	9 (21.4)	130 (26.0)	0.058
Hemorrhagic type	118 (28.2)	8 (19.0)	–	0.203
Unilateral Suzuki stage
I	0	0	–	–
II	63 (7.5)	3 (7.1)	–	1
III	379 (45.3)	17 (40.5)	–	0.634
IV	221 (26.4)	14 (33.3)	–	0.371
V	172 (20.6)	8 (19.0)	–	1
VI	1 (0.1)	0	–	1

Age is expressed as the mean ± SD, and other data are expressed as number (percentage).

Table 2

Baseline characteristics of adult moyamoya with prior bleeding episodes

	Moyamoya disease (n = 118)	Moyamoya syndrome (n = 8)
Age (yr)	43.2±9.1	49.4±6.0
Male	49 (41.5)	4 (50.0)
History of risk factors
Hypertension	13 (11.0)	1 (12.5)
Current smoking	23 (19.5)	1 (12.5)
Current drinking	21 (17.8)	2 (25.0)
Type of bleeding
IVH	65 (55.1)	5 (62.5)
ICH	15 (12.7)	0
ICH & IVH	38 (32.2)	3 (37.5)

Age is expressed as the mean ± SD, and other data are expressed as number (percentage). ICH: Intracranial hemorrhage; IVH: intraventricular hemorrhage.