Development of a convolutional neural network to detect abdominal aortic aneurysms.

Objective: We sought to train a foundational convolutional neural network (CNN) for screening computed tomography (CT) angiography (CTA) scans for the presence of infrarenal abdominal aortic aneurysms (AAAs) for future predictive modeling and other artificial intelligence applications.
Methods: From January 2015 to January 2020, a HIPAA (Health Insurance and Accountability Act)-compliant, institutional review board-approved, retrospective clinical study analyzed contrast-enhanced abdominopelvic CTA scans from 200 patients with infrarenal AAAs and 200 propensity-matched control patients with non-aneurysmal infrarenal abdominal aortas. A CNN was trained to binary classification on the input. For model improvement and testing, transfer learning using the ImageNet database was applied to the VGG-16 base model. The image dataset was randomized to sets of 60%, 10%, and 30% for model training, validation, and testing, respectively. A stochastic gradient descent was used for optimization. The models were assessed by testing validation accuracy and the area under the receiver operating characteristic curve.
Results: Preliminary data demonstrated a nonrandom pattern of accuracy and detectability. Iterations (≤10) of the model characteristics generated a final custom CNN model reporting an accuracy of 99.1% and area under the receiver operating characteristic curve of 0.99. Misjudgments were analyzed through review of the heat maps generated via gradient weighted class activation mapping overlaid on the original CT images. The greatest misjudgments were seen in small aneurysms (<3.3 cm) with mural thrombus. Conclusions: Preliminary data from a CNN model have shown that the model can accurately screen and identify CTA findings of infrarenal AAAs. This model serves as a proof-of-concept to proceed with potential future directions to include expansion to predictive modeling and other artificial intelligence-based applications.

Abdominal aortic aneurysm (AAA) rupture represents a life-threatening disease., Conventional management relies on aneurysm repair, which can be performed by open surgery or endovascular repair. Societal recommendations have provided guidance for the management of AAAs and the decision to intervene depends on the risk of intervention versus the risk of growth and rupture., Recent advances in medical imaging technology have led to the development of software allowing for the analysis of AAAs. However, most current industry-based software available are not automated and require human intervention to initiate aorta localization and measurement of vessel diameters, nor are they able to provide automatic quantitative analysis of the AAA anatomic characteristics such as vessel calcification or the presence of intraluminal thrombus.

Artificial intelligence (AI) corresponds to the ability of a computer to perform tasks commonly associated with human thought. A version of AI termed “machine learning” allows us to discover patterns and make decisions from large data sets without the need for programmed instructions or assumptions. Convolutional neural networks (CNNs) have gained attention in the medical community for solving computer-based visual tasks, including image analysis, object identification, categorization, and segmentation. The application of AI, specifically CNNs, has been investigated in a wide range of medical fields, including imaging and biologic analysis, and could potentially lead to the development of new approaches for the diagnosis, prognosis, or treatment of patients.,

The ever growing medical databank, in the form of clinical, genomic, imaging, and pooled registry data, is only likely to continue to exponentially increase as each year passes. As these data increase, the potential application of analyzing all these data to improve patient care will improve. Thus, the future of medicine is likely to be even more data dependent, with the synergy between medicine and AI technology becoming more pronounced. In the era of personalized medicine and big data analytics, AI AAA analytical programs would be useful for vascular surgery with the potential ability to compute large and heterogeneous AAA imaging and biomarker data sets and to identify patterns even if their relationships are complex and nonlinear.6, 7, 8, 9, 10

The goal of our report was to describe the method and algorithms used to train a robust foundational CNN, independent of human manual input, to identify the presence of infrarenal AAAs from computed tomography (CT) angiography (CTA) scans, which might allow for future predictive modeling and other AI applications.

Methods

The local institutional review board approved this HIPAA (Health Insurance and Accountability Act)-compliant study and waived the requirement for written informed consent. A retrospective review of the hospital’s electronic medical records system was performed via an internal radiology database (mPower Clinical Analytics; Nuance Communications, Inc, Burlington, MA) that contains report and protocol information for CTA abdomen and pelvis examinations. A total of 398 CTA examinations of the abdomen and pelvis were performed in which the archived dictation reported the presence of an aortic AAA between January 2015 and January 2020. These examinations were manually reviewed for the presence of infrarenal AAAs (diameter >3.0 cm in the axial plane), and 68 were excluded because of a ruptured aneurysm, the absence of an infrarenal AAA, prior repair of an infrarenal AAA, image nonavailability in the PACS (picture archiving and communication system), and/or protocol errors (ie, absence of intravenous contrast material). After exclusion of repeated examinations performed on the same patient at ≥2 different time points, 200 CTA scans containing infrarenal AAAs were identified. The concurrent presence of thoracic and/or iliac aneurysmal disease and/or dissection did not preclude the inclusion of CTA scans in the data set. Subsequently, demographic and clinical data such as patient date of birth, patient gender, the presence or absence of hypertension, a history of tobacco use, and scanner type were collected from the medical records system.

For the development of a propensity-matched control group, a query of the electronic medical records identified 4821 CTA scans of the abdomen and pelvis performed between January 2015 and January 2020. From these examinations, with matching by demographics and comorbidities similar to those of the study group, 200 propensity-matched nonaneurysmal aorta control CTA examinations were selected.

All CTAs were performed using a GE Medical Systems Revolution EVO (GE Healthcare, Milwaukee, WI), GE Medical Systems Discovery CT750 HD (GE Healthcare), GE Medical Systems LightSpeed VCT (GE Healthcare), GE Medical Systems LightSpeed16 (GE Healthcare), GE Medical Systems LightSpeed Ultra (GE Healthcare), Siemens Somatom Definition AS (Siemens Healthcare, Erlangen, Germany), or Siemens Biograph128 (Siemens Healthcare) scanners.

Axial reconstructions from all selected CT scans were exported in noncompressed JPEG format at preset window widths and levels. The axial reconstructions from all CT scans were reviewed manually to confirm diagnostic image quality. The axial reconstructions from each CT examination containing an infrarenal AAA were sorted to a set exclusively containing AAAs. A total of 6175 axial images containing infrarenal AAAs were sorted, and a total of 100,249 axial non-AAA images were sorted.

The aneurysm set was randomized to 60% training (n = 3705), 10% validation (n = 618), and 30% testing (n = 1852) subsets. A numerically balanced non-AAA set was generated through sampling of non-AAA axial reconstruction images at fixed intervals. The balanced non-AAA set was randomized to 60% training (n = 3705), 10% validation (n = 618), and 30% testing (n = 1852) subsets. The training and validation subsets were used for model hyperparameter tuning. The test subsets were used for evaluation of model performance.

CNN architecture

The VGG-16 neural network architecture was selected for development of an AAA detection system owing to its robust performance in a variety of image recognition tasks.11, 12, 13, 14, 15 Transfer learning was applied to the neural network using ImageNet, a pretrained CNN developed using >14 million manually labeled images in >20,000 categories. All axial reconstruction images were resized to 512 × 512 pixels. The window widths and levels were kept at the preset export values. Image augmentation was applied to the training set during model development using imgaug (version 0.2.5).17, 18, 19, 20

The model was trained for 40 epochs with batch sizes of 15 to stable convergence of the loss function in the validation set. To address class imbalance, the majority class (non-AAA) was undersampled to the same size as the minority class (infrarenal AAA)., Model development and analysis were performed using Keras (version 2.4.3), TensorFlow (version 2.4.1), imgaug (version 0.2.5), Scipy (version 1.2.1), NumPy (version 1.8.2), scikit-learn (version 0.23.1), and Matplotlib (version 3.2.2). All experiments were performed using a computer equipped with an NVIDIA Quadro P5000 GU (graphical processing unit) with 16 GB of GDDR5 (graphics double data rate 5 synchronous dynamic random-access memory) video memory.

Assessment of model performance and statistical analysis

Following tuning of the model hyperparameters, the model was assessed for overall diagnostic accuracy at the image level. Statistical calculations for patient demographic data and risk factors were performed using GraphPad Prism, version 8.4.3 (GraphPad Software, San Diego, CA). The loss and accuracy of the training and validation groups were plotted by epoch to observe for stable convergence of model performance. Overall accuracy (number of correctly classified images per number of total classified images) was calculated from the test set of images to assess overall model performance. The area under the receiver operating characteristic curve (AUC) and F1 score were also used to evaluate model performance within the test set of images. A confusion matrix (two-by-two) table was generated from the test set of images. The sensitivity, specificity, positive predictive value, and negative predictive value were calculated from the classification results of the test set. Gradient weighted class activation maps (heat maps) were generated for analysis of misjudgments., Misjudged images were manually reviewed to characterize the aneurysm size, presence or absence and extent of mural thrombus, concurrent presence of an iliac or a pararenal or suprarenal aneurysm on the misjudged axial images, and/or the presence of tortuosity resulting in the appearance of an AAA that would not otherwise be classified on a curvilinear reformatted image. The plots and figures were generated using Matplotlib and converted to vector graphic format in Visio Professional 2019 (Microsoft, Redmond, WA) or OmniGraffle Pro, version 7.18.1 (The Omni Group, Seattle, WA).

Results

Between January 2015 and January 2020, 200 patients with AAAs and 200 propensity-matched control patients with non-AAA aortas were identified. The demographics of both groups (AAA vs non-AAA) were equally matched in age (73.2 years vs 72.1 years; P = .359), male gender (71.5% vs 72.0%; P = .999), tobacco use (79.9% vs 76.5%; P = .891), and a history of hypertension (86.3% vs 82.7%; P = .878; Table).

Table

Patient characteristics used to develop convolutional neural network (CNN) to detect abdominal aortic aneurysms (AAAs)

Characteristic	AAA group (n = 200)	Non-AAA group (n = 200)	P value
Age, years	73.2 ± 10.2	72.1 ± 12.1	.359
Male gender	143 (71.5)	144 (72.0)	.999
Tobacco use	160 (79.9)	153 (76.5)	.891
Hypertension	172 (86.3)	165 (82.7)	.878

Data presented as mean ± standard deviation or number (%).

The VGG-16 neural network architecture was trained using conventional training (60%) and validation (10%) image sets, including 3705 AAA training images and 3705 non-AAA training images and 618 AAA validation images and 618 non-AAA validation images (Fig 1). During the training and optimization iterations of the VGG-16 CNN, the model demonstrated significant improvement in overall performance. As the number of epochs increased, the number of times the weights were changed in the neural network decreased, which, in turn, resulted in stable convergence with optimal fitting for model performance. In addition, an appropriate reduction in the loss of function and appropriate increase in the overall accuracy occurred (Fig 2).

Fig 1

Flowchart of study process depicting patient selection and study design. AAA, Abdominal aortic aneurysm; CTA, computed tomography angiography.

Fig 2

During training (train) and optimization of the VGG-16 convolutional neural network (CNN), the model demonstrated significant improvement in overall performance. A, In an early version of the CNN model, as the number of epochs increased, the loss of function and accuracy curves demonstrated suboptimal fitting for model performance. B, In the optimized CNN model, as the number of epochs increased, an appropriate reduction occurred in the loss of function, with an increase in overall accuracy, demonstrating optimal fitting for model performance. Val, Validation.

The final trained version of the new CNN model was tested with conventional testing (30%) image sets: 1852 AAA training images and 1852 non-AAA aorta images. The final custom CNN model demonstrated a sensitivity of 98.9% (95% confidence [CI], 98.28%-99.30%), specificity of 99.3% (95% CI, 98.80%-99.62%), accuracy of 99.1% (95% CI, 98.72%-99.36%), and AUC of 0.99 (Fig 3).

Fig 3

The final custom convolutional neural network (CNN) model demonstrated high diagnostic accuracy. A, The results of model testing in the confusion matrix (two-by-two) table. B, The model demonstrated an accuracy of 99.1% (95% confidence interval [CI], 98.72%-99.36%) and area under the receiver operating characteristic (ROC) curve (AUC) of 0.99.

Misjudgments were analyzed through a review of the heat maps generated via gradient weighted class activation mapping overlaid on the original CT images. The highest number of misjudgments was seen for small AAAs (<3.3 cm) with mural thrombus (Fig 4).

Fig 4

Analysis of judgments through review of heat maps generated via gradient weighted class activation mapping overlaid on original computed tomography (CT) images. A, A correct judgment by the custom convolutional neural network (CNN) that identified the abdominal aortic aneurysm (AAA). B, A misjudgment of a relatively small size aneurysm and presence of mural clot contributed to a false-negative diagnosis.

Discussion

The implementation of AI in medicine is undergoing continuous evolution. Some of the early AI studies in vascular surgery were implemented to assess the predictive nature of clinical markers for patient outcomes. Turton et al demonstrated that their neural network correctly predicted the outcomes for 82.5% of individual cases using four highly significant independent predictors of AAA mortality: preoperative hypotension, intraperitoneal rupture, preoperative coagulopathy, and preoperative cardiac arrest. Wise et al also reported a neural network-based predictive model that could accurately predict for high mortality risk from attempted repair of ruptured AAAs.

Recently, the evolution of the CNN has gained attention in imaging-based surgical subspecialties for solving computer-based visual tasks. The integration of imaging and biologic analysis could potentially lead to the development of revolutionary approaches for patient management., Imaging is an integral component of the diagnosis, surveillance, and management of AAAs. CT imaging has remained the mainstay technique for planning and monitoring AAAs because it provides a rapid and accessible method to examine the extent, morphology, and pathology of AAAs.,

In the vascular surgery literature, previous studies examined the role of semiautomated AAA image analysis, focusing on image segmentation. Image segmentation is the process of separating a digital image into multiple sets (groups of pixels). The pixels are tagged such that the pixels with the same tag share similar characteristics. This allows for manual labeling of regions of interest, analysis of sets of shapes, and creation of three-dimensional images. de Bruijne et al described a semiautomated method for aneurysm sac segmentation that used manual segmentation of the first slice, with automated detection of the contour in subsequent slices, allowing for rapid processing of the entire volume of the AAA. Other investigators have reported similar computer-based techniques.29, 30, 31 Although these were significant advances in programming techniques, these programs all require some baseline manual input. Manual segmentation is also laborious and time-consuming, requires a trained and experienced operator, and is subject to inter- and intraoperator variations.

Automated nonsegmented techniques would be an invaluable tool for patient management because it would reduce the analysis time, alleviate the burden of performing repetitive tasks, and improve reproducibility. Recently, Lareyre et al described a fully automated pipeline to characterize AAAs, including the presence of intraluminal thrombus and calcifications. This rapid method was tested on a set of 40 patients with CTA images and demonstrated a good correlation with the results obtained from manual segmentation by human experts. Although this was a very robust model that produced excellent output information, it remains a programmed model with defined parameters and does not harness the power of machine learning. Mohammadi et al designed a CNN classifier for the aorta that used a Hough transform circles algorithm to classify a group of 120 aorta patches according to their diameter, with an accuracy of 98.33% and a detection rate of 98.62%.

We have described a fully automated, novel, trained CNN model that demonstrated robust accuracy of 99.1% (95% CI, 98.72%-99.36%) and an AUC of 0.99. The CNN model was tested using 3600 images from 400 patients in two propensity-matched cohorts. Our results were derived from real-world, unaltered, nonsegmented images that had been obtained using various acquisition methods, contrast agents, resolution, with differing concomitant comorbid pathologies, and noise and artifacts. With this robust CNN, we have demonstrated a proof-of-concept model that can be used for a variety of potential future applications, including the prediction of growth and rupture, determination of long-term prognosis, and, most importantly, the merging of clinical biomarkers and imaging data to develop more personalized therapeutic management plans for this complex patient population.

Additional work will continue to determine the limitations of our model and increase its accuracy moving forward. Complex aneurysm pathology, including, but not limited to, thrombus, calcifications, and adjacent and/or contiguous aneurysms, will need to be analyzed to optimize the algorithm. In the present feasibility model, a few patients in our propensity-matched control group had had peripheral artery disease. We attempted to match the pertinent demographic and comorbidity factors to the study group as closely as possible to achieve our objective for our study: to create a de novo foundational CNN model that can accurately delineate AAAs in the general population for real world, nonbiased applicability. Future investigations are required to improve the predictive accuracy when confounding imaging variables are present.

In the era of personalized medicine and big data sciences, machine learning will likely play a vital role in analyses that humans would have difficulty in providing. Precision medicine provides the opportunity for personalized healthcare management to individuals or groups of patients using large datasets of disease profiles, diagnostic or prognostic information, and treatment responses. Before the application of machine learning to image analysis, human interpretation was required to transform an image into a binary or categorical variable for analysis. However, with the use of CNN, we have the ability to objectively break down an image into large biostatistical datasets. The tidal wave of medical data in the form of clinical, genomic, imaging, and pooled registry data is only likely to exponentially increase as precision and personalized medicine matures. Thus, the future of medicine is likely to be even more data dependent, with the synergy between medicine and AI technology becoming more pronounced.

The present study had several limitations. First, this was a retrospective, single-center study with a limited number of queried imaging studies that had met the exclusion criteria (ie, ruptured aneurysm, prior repair of an infrarenal AAA, and/or protocol errors [absence of intravenous contrast material, timing issues]). However, with the development of our training technique and method, we have the tools to continue to optimize our protocol to train the CNN to potentially mitigate these variables. Second, the sample size was underpowered, particularly for a subanalysis according to aneurysm size and morphology, quantification of mural thrombus, or quantification of aortic calcifications. The present model was trained as a binary classifier on a modest GPU with limited computation power. However, our group will be moving our future projects to a dedicated AI core laboratory with a 10-fold increase in processing power. This will allow for a more robust analysis of exponentially larger data sets with multiple categorical and continuous imaging variables. Third, the present model is not 100% accurate, demonstrating a <1% misjudgment rate. These errors had mostly occurred with aneurysms <3.3 cm in size with mural thrombus. These findings likely resulted from difficulties in resolving pixel groups with <3 mm of spatial resolution; a task that is difficult for radiologists to reliably reproduce without image manipulation such as imaging magnification and window level changes.

Conclusions

We have reported preliminary data from a CNN model that can accurately screen and identify CTA findings of infrarenal AAAs. This model serves as a proof-of-concept to proceed with potential future directions, including expansion to predictive modeling and other AI-based applications.