Postsurgical functional outcome prediction model using deep learning framework (Prediction One, Sony Network Communications Inc.) for hypertensive intracerebral hemorrhage.

BACKGROUND: Reliable prediction models of intracerebral hemorrhage (ICH) outcomes are needed for decision-making of the treatment. Statistically making such prediction models needs a large number of samples and time-consuming statistical analysis. Deep learning (DL), one of the artificial intelligence, is attractive, but there were no reports on DL-based functional outcome prediction models for ICH outcomes after surgery. We herein made a functional outcome prediction model using DLframework, Prediction One (Sony Network Communications Inc., Tokyo, Japan), and compared it to original ICH score, ICH Grading Scale, and FUNC score.
METHODS: We used 140 consecutive hypertensive ICH patients' data in our hospital between 2012 and 2019. All patients were surgically treated. Modified Rankin Scale 0-3 at 6 months was defined as a favorable outcome. We randomly divided them into 100 patients training dataset and 40 patients validation dataset. Prediction One made the prediction model using the training dataset with 5-fold cross-validation. We calculated area under the curves (AUCs) regarding the outcome using the DL-based model, ICH score, ICH Grading Scale, and FUNC score. The AUCs were compared.
RESULTS: The model made by Prediction One using 64 variables had AUC of 0.997 in the training dataset and that of 0.884 in the validation dataset. These AUCs were superior to those derived from ICH score, ICH Grading Scale, and FUNC score.
CONCLUSION: We easily and quickly made prediction models using Prediction One, even with a small single-center dataset. The accuracy of the DL-based model was superior to those of previous statistically calculated models. Copyright:

INTRODUCTION

Hypertensive intracerebral hemorrhage (ICH) is responsible for 10–30% of all strokes, and it is a significant cause of all stroke-related morbidity and mortality.[16] Only 20% of ICH patients regain functional independence within 3 months after the onset.[24] Surgical hematoma evacuation or conservative therapy are the main treatments for ICH, but the role of surgery for ICH patients remains unclear.

Theoretically, surgical hematoma evacuation prevents herniation by reducing the intracranial pressure. It also decreases the pathophysiological impact of the ICH on surrounding tissue. However, the Surgical Trial in Intracerebral Hemorrhage (STICH) showed that patients with spontaneous supratentorial ICH showed no overall benefit from the early surgery when compared to the initial conservative therapy, though 24% of patients in the conservative group finally underwent surgery.[36] The STICH II trial showed that early surgery did not increase the morbidity and disability at 6 months and may have a bit of survival advantage for ICH patients without intraventricular hematoma.[35] The STICH II trial’s continuation report reported that only patients with a Glasgow Coma Scale (GCS) score 10–13 or a large ICH was likely to benefit from surgery.[10] The STICH and STICH II trials could not exhibit comprehensive benefit for the functional outcome over conservative treatment.[31] As a study on minimally invasive treatments, Minimally Invasive Surgery Plus Rt-PA for ICH Evacuation (MISTIE) trial was done, and it demonstrated no functional benefit for the MISTIE procedure in selected patients; however, a subgroup analysis showed improvement of the 1-year outcomes in patients with an increased hematoma removal rate (≤ 15 mL residual hematoma after the surgery).[13] Besides, an endoscopic evacuation arm of MISTIE II, called the Intraoperative Stereotactic Computed Tomography (CT)-Guided Endoscopic Surgery, also demonstrated the safety and effectiveness of the chronic neurological outcome.[50] However, after all, the role of surgery for ICH has not apparent, although these prospective studies were performed.

We practice according to the Japanese Guidelines for the Management of Stroke 2009[47] and 2015.[48] The guidelines suggest us to perform surgery depending on the patients’ neurological status and radiological findings. However, it is difficult to determine whether to perform the surgical treatment or not because the surgery’s role is unclear, as described above. Furthermore, not only the neurological status and radiological findings but also patients’ age, background, and their comorbidities have large effects on the outcomes, so we should determine the treatment strategy after a comprehensive evaluation and discussion with their families. Therefore, we need a reliable functional outcome prediction model to determine whether to perform surgery for ICH or not.

Previously, many studies tried to make the prediction model for ICH outcomes, including the original ICH score,[15] ICH Grading Scale,[42] Essen-ICH score,[52] max-ICH score,[43] the simplified ICH score,[3] ICH functional outcome score,[19] modified ICH score,[2] ICH outcome project score,[11] and the functional outcome (FUNC) score.[41] Validation studies and meta-analyses on their utilities have also been done, and their pooled area under the curve (AUC) of the receiver operating curve for functional outcome or mortality varied from 0.76 to 0.85 [Table 1].[9] These statistically-made prediction models or scoring systems need a large number of samples over thousands, so these studies tend to be country-initiated or academic association-initiated research. However, the larger the sample size, the less detailed information is available, such as comorbidities, use of antithrombotic drugs, or laboratory test data, and the more there are missing data. Furthermore, the treatment strategies vary from hospital to hospital, and patient backgrounds differ depending on countries and regions. Therefore, these prediction scores work as the greatest common denominator worldwide but not necessarily applicable to the respective hospital.[21]

Table 1:

Previously reported ICH outcome prediction scales.

Deep learning (DL), one of the machine learning, is recently attractive. DL is starting to be used in the neurosurgical situations in decision-making for spinal canal stenosis,[1] predicting outcomes after subarachnoid hemorrhage,[21] automated diagnosis of primary headache,[26] predicting the occurrence of stroke[25] and ambulance transport,[51] pathological diagnosis[33] or radiomics studies of brain tumors.[4,34] However, there are no reports on the DL-based outcome prediction of ICH, though studies using other machine learning methods, such as decision tree, random forest, support vector machine, and XGBoost, have been reported.[12,37,53]

We hypothesized that we could make a good prediction model for our own hospital using DL, even with a small dataset with detailed variables. Therefore, we herein produced the DL-based functional outcome prediction model using DL framework, Prediction One (Sony Network Communications Inc., Tokyo, Japan)[44] with our ICH dataset and compared the utility of the model made by Prediction One to other statistically-made scores, including original ICH score,[15] ICH Grading Scale,[42] and FUNC score,[41] for functional prediction at 6 months. If we could make an excellent functional outcome prediction model, it would be beneficial for decision-making for whether to perform surgical hematoma removal or not. This is the first report to use DL to predict functional outcomes after surgery for ICH.

MATERIALS AND METHODS

Study population

We retrospectively retrieved data from medical records of all the consecutive 140 hypertensive ICH patients admitted between 2012 and 2019 and surgically treated at our institution. Patients who did not undergo surgical treatment, those without GCS score at admission nor the outcome data at 6 months, were excluded from the study. The detail of the dataset is available online.[23] The diagnosis of ICH was based on the clinical history and the presence of ICH on CT. The inclusion criteria for the study were as follows; (1) patients with ICH at the basal ganglia or subcortex, (2) patients designated for surgical treatment according to the Japanese Guidelines for the Management of Stroke 2015[48] and 2009[47] (described in detail in the General management section) and treated by surgical hematoma evacuation with craniotomy or endoscopically, and (3) the interval between onset and hematoma removal was <24 h. The exclusion criteria were as follows; (1) ICHs due to the tumor, trauma, aneurysm, arteriovenous malformation, and hemorrhage after infarction, and (2) patients who had a thalamic or caudate head hemorrhage with an intraventricular hemorrhage treated by the flexible neuroendoscope to only remove the intraventricular hematoma. There were no patients with preoperative cognitive impairment. Our hospital’s research ethics committee approved this study, and we gained written informed consent for this study from all of the patients, the legally authorized representative of the patients, or next of kin of the deceased patients. All methods were carried out in accordance with relevant guidelines and regulations (Declaration of Helsinki).

General management

During admission and in the acute phase, patients were first administered nicardipine to maintain the normal systolic blood pressure at under 140 mmHg. The prothrombin time of patients undergoing anticoagulation therapy was normalized by the administration of Vitamin K and/or fresh frozen plasma. Then, a surgical indication was made following the Japanese Guidelines for the Management of Stroke 2009[47] and 2015.[48] Both versions describe the same surgical indications, which are as follows: patients with hematoma at the basal ganglia which was more than 30 mL, and who were neurologically deteriorating were designated for surgery. Patients with superficial lobar hemorrhage within 1 cm of the cortical surface and with disturbance of consciousness or moderate neurological deficits were also designated for surgery. Patients with a small hemorrhage and without severe neurological symptoms that could be treated by conservation or patients with cardiopulmonary arrest on arrival did not undergo any surgical treatment. Rehabilitation and nutritional support were started immediately after the operation, and steps were undertaken to prevent and treat the complications. Antithrombotic agents were discontinued postoperatively for several days depending on the patients’ condition and comorbidities.

Hematoma removal with craniotomy was performed primarily from 2012 to 2013 and the endoscopic hematoma removal began in 2013. We gradually transitioned from craniotomy to endoscopic hematoma removal as a first-choice treatment between 2014 and 2015. During this period, patients who received antithrombotic drugs and displayed apparent extravasation on the contrast-enhanced CT image were likely to undergo a craniotomy. Since 2015, endoscopic procedures have been routinely performed in our hospital regardless of age, comorbidities, presence of antithrombotic drugs, and extravasation on the contrast-enhanced CT image. However, a craniotomy was still performed when the endoscope was unavailable due to reasons such as cleaning or the unavailability of the medical staff in the operating room (i.e., weekends and holidays). We performed craniotomy under general anesthesia but endoscopic hematoma removal under local anesthesia. The details of each surgical procedure and anesthesia method were described in our previous reports.[22,24]

Clinical variables

We collected data regarding physiological symptoms at admission for patients included in this study, that is, year, age, sex, height, weight, preoperative GCS score, National Institutes of Health Stroke Scale score, systolic blood pressure, administration of antithrombotic drugs, history of smoking and massive alcohol intake (over 450 g ethanol intake/week), and comorbidities (history or present treatment by a clinician for hypertension, diabetes mellitus, dyslipidemia, cardiovascular diseases, previous stroke, cancer, hepatic cirrhosis, chronic kidney diseases, or orthopedic disease). We also measured serum total protein, albumin, total bilirubin, aspartate aminotransferase, alanine transaminase, lactate dehydrogenase, alkaline phosphatase, gamma-glutamyl transpeptidase, glucose, hemoglobin A1c, sodium, potassium, chlorine, blood urea nitrogen, creatinine, C-reactive protein, uric acid, triglycerides, total cholesterol, high-density lipoprotein cholesterol, and low-density lipoprotein cholesterol levels. We also investigated whole and differential white blood cell counts, hemoglobin level, platelet count, prothrombin time-international normalized ratio, activated partial thromboplastin time, and D-dimer level.

We determined the location of the hematoma (subcortex, basal ganglia, or cerebellum) and measured hematoma volume by ABC/2 methods. We also checked the presence of intraventricular hematoma and the apparent destruction of the pyramidal tract. We observed the primary motor area, radiate corona, posterior limb of the internal capsule, and cerebral peduncle for potential destruction. The obvious destruction of these areas indicated that the pyramidal tract was apparently destroyed; while the equivocal one was absent. Moreover, the temporal muscle thickness and area[5-7,22,24,27,30,46] were measured on the head CT at admission as an indicator of sarcopenia[40] and nutrition[14,38] using the method reported by Katsuki et al.[27-30] We used the Aquilion ONE (Canon Medical Systems Corporation, Tochigi, Japan) to take CT images of 0.5 × 0.5 × 1.0 mm voxels. The slice thickness was reconstructed to 5 mm. The window width was adjusted to 300 and the window level to 20 for temporal muscle measurement.

We also investigated the treatment strategy (hematoma evacuation with craniotomy or endoscopic hematoma removal with or without neuronavigation). To evaluate the outcomes, modified Rankin Scale (mRS) scores at 6 months after the treatment of all 140 patients were collected by either personal outpatient interviews, reports from the rehabilitation hospital or home doctor, or interviews over the telephone, once the ethical approval was obtained for the study. We dichotomized mRS scores into favorable (mRS 0–3) or poor (mRS 4–6).

Making prediction model by prediction one

We used Prediction One framework to make the prediction model. We divided our 140 patients’ data randomly into 100 patients training dataset and 40 patient’s validation dataset. Prediction One read the 100 patients’ data with 64 variables, and automatically divided them into five-fold cross-validation datasets. Prediction One automatically adjusted and optimized the variables that are easy to be processed statistically and mathematically and select appropriate algorithms with ensemble learning. The missing values were automatically compensated, and Prediction One made the best prediction model by artificial neural network with internal five-fold cross-validation. The details are trade secrets and could not be provided.

We let the Prediction One framework make a functional outcome prediction model using 100 patients training dataset using 64 variables described above. The AUC of the model and strong variables with the weights were automatically calculated. Then, we performed the model’s validation using the 40 patients datasets, and calculated AUC, accuracy, precision, recall, and F value, which were used to evaluate the prediction model made by artificial intelligence.

Prediction using original ICH score, ICH Grading Scale, and FUNC score

We also investigated the original ICH score,[15] ICH Grading Scale,[42] and FUNC score[41] of the 100 patients in the training dataset and 40 patients in the validation dataset, respectively.

We evaluated 3 scores’ AUCs calculated using the sum scores and the outcome, and compared them to AUCs of the model made by Prediction One.

Statistical analysis

Results are shown as median (interquartile range). The difference between the training dataset and the external validation dataset was tested using the Mann–Whitney U-test, Fisher’s exact test, Pearson’s Chi-square test, or Kruskal–Wallis test, appropriately. A two-tailed P < 0.05 was considered statistically significant. We calculated AUCs and their P values using SPSS software version 24.0.0 (IBM, New York, USA).

RESULTS

Clinical characteristics

The clinical characteristics of the 140 ICH patients (59 women and 81 men) are summarized in [Table 2]. The median (interquartile range) age was 74 (63–82), GCS score 10 (8–13), and hematoma volume 103 (65–158) mL. Hematoma evacuation with craniotomy was performed for 67 patients and endoscopic hematoma removal for 73. The median mRS was 4 (2–5) at 6 months, and 48 patients (34%) were independent in their ADLs. Preoperative GCS score, ratio of dyslipidemia patients, and gamma-glutamyl transpeptidase level were significantly different between training and validation dataset, despite the fact that we randomly divided them.

Table 2:

Characteristics of the datasets.

Model development and validation

Prediction One produced the functional outcome prediction model using the 100 patients with 64 variables in <4 min. The AUC of the model was 0.997 (95% confident interval [CI] 0.989–1.000). The model’s accuracy, precision, recall, and F value were 0.810, 1.000, 0.769, and 0.869, respectively. Its AUC for the validation dataset were 0.884 (95% CI 0.753– 1.000) with 80.0% accuracy [Table 3].

Table 3:

Models for functional prediction at 6 months.

The stronger variables and their weights of the model are listed in [Table 4]. Alanine transaminase, C-reactive protein, %eosinophil, lactate dehydrogenase, uric acid, %neutrophil, diabetes mellitus, triglycerides, preoperative GCS score, and platelet count had large effects on the outcome in order. The presence of intraventricular hematoma, age, hematoma volume, and hematoma location (supra or infratentorial hematoma) was not so important in our model.

Table 4:

Stronger variables of each model made by Prediction One.

Comparison to original ICH score, ICH Grading Scale, and FUNC score

We calculated original ICH score, ICH Grading Scale, and FUNC score in the training and validation dataset, respectively. The AUCs of these scores in the training dataset were 0.610 (95% CI 0.493–0.728), 0.777 (95% CI 0682–872), and 0.764 (95% CI 0.670–0.858), respectively. Those in the validation dataset were 0.755 (95% CI 0.595–0.915), 0.806 (95% CI 0.658–0.955), and 0.688 (95% CI 0.509–0.876), respectively. These AUCs were all inferior to the models made by Prediction One [Table 3].

DISCUSSION

We made the postsurgical functional outcome prediction model using the DL framework, Prediction One. We created the model with a high prediction rate using a small dataset (n = 100) with several missing data. It would be reliable for the functional prediction in our own hospital with 80% accuracy. Furthermore, this is the first report on creating a functional outcome prediction model of postoperative ICH patients using DL.

Advantages of DL

Conventional time and cost-consuming statistical analysis need laborious standardization of variables like a logarithmic transformation to increase the prediction model’s accuracy. It also requires the arbitrary selection of variables based on the previous studies, and multivariate analysis needs 10 folds number of samples against the variables.[39] Therefore, there is a risk that variables that might be important cannot be included in the statistical analysis, or that even the multivariate analysis cannot be performed in a small hospital with a small dataset. Furthermore, in statistical analysis, when there is missing data, we should do multiple imputations or list wise deletions, which also affects accuracy. However, DL has the potential to overcome these problems. DL develops beneficial models with less effort or time using the small dataset, without time-consuming variable optimization nor arbitrarily choosing variables because the DL framework automatically does these processes. Furthermore, the number of variables used in the DL framework is not limited, and DL sometimes finds interesting variables as important that have not been taken into account in the previously reported statistical models. Furthermore, the DL framework automatically substitutes appropriate values instead of the missing ones, and calculates the best prediction model without our statistical trial and error.

We then review these benefits of DL in our study. Conventionally, we could have used only ten variables for statistical analysis due to the small sample size of the training dataset (n = 100). Furthermore, the dataset contains several missing data. However, we could use 64 variables for making the prediction model by Prediction One, and make a good prediction model from the small dataset. We did not need to perform variable optimization nor manipulations for the missing values. Furthermore, some unexpected serological test results such as alanine transaminase, C-reactive protein, %eosinophil, lactate dehydrogenase, and uric acid levels were judged to be more important among many other previously reported important factors, such as age, hematoma volume, and hematoma location. We usually think that some variables, such as age, hematoma volume, and preoperative GCS, largely affect the outcomes, with common sense. However, the DL framework treats many variables equally and without preconceptions, revealing the very important factors that were not expected.

Besides, the time needed for creating each model was <4 min. Finally, the models achieved high accuracy with the AUC of 0.997 in the training dataset and that of 0.884 in the validation dataset. Putting it bluntly, our study showed that our DL-based prediction model, even made from the small dataset, can predict the ICH patients’ outcomes surgically treated in our hospital with higher accuracy than other scores, which was made from the large cohort study.

Recent study on artificial intelligence and ICH

Andrew reported that decision tree and random forests could be useful to predict 3 months functional outcomes.[12] Independent predictors selected by the algorithms as important included hematoma volume at hospital admission, hematoma expansion, intraventricular hemorrhage, total ICH Score, and GCS. Different from DL, in which the weights of the variables are comprehensively calculated, decision tree, and random forest can suggest us a few critical and important factors in order, so they are helpful for clinicians to make decisions simply. Nie reported that random forest had better performance in predicting in-hospital mortality for cerebral hemorrhage patients in intensive care units compared to other machine learning methods.[37] They suggested the possibility of the dreamlike efficient medicine. All the selected variables in their model were initial clinical data and electronic monitoring data that can be automatically obtained by the monitor or can be simply evaluated, such as age, gender, and GCS score. Therefore, their model can be completed by nurses or assistants, thereby significantly reducing the burden of clinical work for doctors. Xu et al. reported an outcome prediction model using CT radiomics with random forest and XGBoost.[53] It is worth noting that they suggested the possibility of making a prediction model only from CT images. In the future, these attempts can be completed by artificial intelligence monitoring instruments, achieving full automation.

Our study is the first attempt to make a prediction model using DL, not other machine learning methods. DL treats variables comprehensively, so it could not present particularly important factors as in a decision tree or XGBoost, and Prediction One could not treat CT radiomics. However, the AUC of the model in the training dataset of 0.997 and that in the validation dataset 0.884 were much higher compared to the other machine learning methods in these previous studies. It is also a strong point that we could easily create a prediction model in <4 min without vigorous effort except for collecting data. Each artificial intelligence algorithm has its own merits and demerits, and it is necessary to consider which method is better in the future.

Future outlook

Despite the easiness, advantages, and future potential of DL, the majority of medical staff cannot treat DL frameworks.[45] As simple DL frameworks like Prediction One are being developed, there is a need for an active interest in using them to benefit medical staff and patients. Our study is just one example but suggested the utility of the DL framework. DL-based tailormade and efficient medicine, depending on each patient and hospital, would be performed as the DL framework becomes more popular. DL framework can produce predicting models specific to individual centers that would be based on their own unique experience in managing ICH patients. Furthermore, with modern electronic medical records, the clinical variables and clinical outcome data could be automatically fed to the DL framework, leading to progressive improvement in predictions over time. This evolutionary prediction will be a benefit to patients, health-care providers, and hospital managers. Furthermore, the big data have been stored, such as Miyagi medical and welfare information network,[17] Tohoku Medical Megabank,[8,20] Japanese Stroke Databank,[32] or the Japan Neurosurgical Database.[18] When these data are open for researchers, it will spur competition to develop further prognostic models using such big data, like Kaggle competition.[49] In the future, various data would be evaluated at once, including neurological and physiological information, from video systems at the outpatient, inpatient, surgery, rehabilitation, chronological information from the monitoring system, radiological information, laboratory test results, and any other information. The paradigm shift will come when we can know the optimal treatment if these data will be shared worldwide, although treatment strategies differ in each hospital and doctor.

Limitation of this study

First, we did not use other scores, such as Essen-ICH score,[52] max-ICH score,[43] the simplified ICH score,[3] ICH functional outcome score,[19] modified ICH score,[2] or ICH outcome project score,[11] because we did not have data to be used for these scores. Second, we dichotomized the outcome as mRS 0–3 or 4–6, but other standards such as functional independent measure could be used as detailed outcomes. Third, the prediction model derived from our own data cannot be applied to other institutions, and the training and validation dataset must be updated to keep up with advances in medical science and changes in surgical techniques. Fourth, DL can treat images, sentences, and chronological data, and this is the very strong point compared to other algorithms, but we did not use these advantages. Prediction One cannot treat radiological images or videos at this time. Fifth, it is unknown why serological test results such as alanine transaminase, C-reactive protein, %eosinophil, lactate dehydrogenase, and uric acid levels were judged to be important among many other previously reported important factors. We should be very careful in interpreting the results. Sixth, we did not investigate patients treated conservatively, so this study would be helpful only in determining the surgical indication.

CONCLUSION

We easily and quickly made the functional outcome prediction model using Prediction One framework, and it is superior to other prediction scores, such as the original ICH score, ICH Grading Scale, and FUNC score, which were statistically calculated with a large cohort. Even with a small single-center dataset, containing missing data, prognostic models made by the DL framework can be useful at the institution and may be beneficial for us to determine the surgical indication.