Lucas W Remedios1, Sneha Lingam2, Samuel W Remedios3,4, Riqiang Gao1, Stephen W Clark5, Larry T Davis5,6, Bennett A Landman1,6,7. 1. Department of Computer Science, Vanderbilt University, Nashville, TN, USA. 2. School of Medicine, Vanderbilt University, Nashville, TN, USA. 3. Department of Computer Science, Johns Hopkins University, Baltimore, MD, USA. 4. Department of Radiology and Imaging Sciences, National Institutes of Health, Bethesda, MD, USA. 5. Department of Neurology, Vanderbilt University Medical Center, Nashville, TN, USA. 6. Department of Radiology and Radiological Sciences, Vanderbilt University Medical Center, Nashville, TN, USA. 7. Department of Electrical Engineering, Vanderbilt University, Nashville, TN, USA.
Abstract
PURPOSE: Artificial intelligence diagnosis and triage of large vessel occlusion may quicken clinical response for a subset of time-sensitive acute ischemic stroke patients, improving outcomes. Differences in architectural elements within data-driven convolutional neural network (CNN) models impact performance. Foreknowledge of effective model architectural elements for domain-specific problems can narrow the search for candidate models and inform strategic model design and adaptation to optimize performance on available data. Here, we study CNN architectures with a range of learnable parameters and which span the inclusion of architectural elements, such as parallel processing branches and residual connections with varying methods of recombining residual information. METHODS: We compare five CNNs: ResNet-50, DenseNet-121, EfficientNet-B0, PhiNet, and an Inception module-based network, on a computed tomography angiography large vessel occlusion detection task. The models were trained and preliminarily evaluated with 10-fold cross-validation on preprocessed scans (n = 240). An ablation study was performed on PhiNet due to superior cross-validated test performance across accuracy, precision, recall, specificity, and F1 score. The final evaluation of all models was performed on a withheld external validation set (n = 60) and these predictions were subsequently calibrated with sigmoid curves. RESULTS: Uncalibrated results on the withheld external validation set show that DenseNet-121 had the best average performance on accuracy, precision, recall, specificity, and F1 score. After calibration DenseNet-121 maintained superior performance on all metrics except recall. CONCLUSIONS: The number of learnable parameters in our five models and best-ablated PhiNet directly related to cross-validated test performance-the smaller the model the better. However, this pattern did not hold when looking at generalization on the withheld external validation set. DenseNet-121 generalized the best; we posit this was due to its heavy use of residual connections utilizing concatenation, which causes feature maps from earlier layers to be used deeper in the network, while aiding in gradient flow and regularization.
PURPOSE: Artificial intelligence diagnosis and triage of large vessel occlusion may quicken clinical response for a subset of time-sensitive acute ischemic stroke patients, improving outcomes. Differences in architectural elements within data-driven convolutional neural network (CNN) models impact performance. Foreknowledge of effective model architectural elements for domain-specific problems can narrow the search for candidate models and inform strategic model design and adaptation to optimize performance on available data. Here, we study CNN architectures with a range of learnable parameters and which span the inclusion of architectural elements, such as parallel processing branches and residual connections with varying methods of recombining residual information. METHODS: We compare five CNNs: ResNet-50, DenseNet-121, EfficientNet-B0, PhiNet, and an Inception module-based network, on a computed tomography angiography large vessel occlusion detection task. The models were trained and preliminarily evaluated with 10-fold cross-validation on preprocessed scans (n = 240). An ablation study was performed on PhiNet due to superior cross-validated test performance across accuracy, precision, recall, specificity, and F1 score. The final evaluation of all models was performed on a withheld external validation set (n = 60) and these predictions were subsequently calibrated with sigmoid curves. RESULTS: Uncalibrated results on the withheld external validation set show that DenseNet-121 had the best average performance on accuracy, precision, recall, specificity, and F1 score. After calibration DenseNet-121 maintained superior performance on all metrics except recall. CONCLUSIONS: The number of learnable parameters in our five models and best-ablated PhiNet directly related to cross-validated test performance-the smaller the model the better. However, this pattern did not hold when looking at generalization on the withheld external validation set. DenseNet-121 generalized the best; we posit this was due to its heavy use of residual connections utilizing concatenation, which causes feature maps from earlier layers to be used deeper in the network, while aiding in gradient flow and regularization.
Authors: John Muschelli; Natalie L Ullman; W Andrew Mould; Paul Vespa; Daniel F Hanley; Ciprian M Crainiceanu Journal: Neuroimage Date: 2015-04-07 Impact factor: 6.556
Authors: Sunil A Sheth; Victor Lopez-Rivera; Arko Barman; James C Grotta; Albert J Yoo; Songmi Lee; Mehmet E Inam; Sean I Savitz; Luca Giancardo Journal: Stroke Date: 2019-09-24 Impact factor: 7.914
Authors: Ameer E Hassan; Victor M Ringheanu; Rani R Rabah; Laurie Preston; Wondwossen G Tekle; Adnan I Qureshi Journal: Interv Neuroradiol Date: 2020-08-26 Impact factor: 1.610
Authors: Robert C Rennert; Arvin R Wali; Jeffrey A Steinberg; David R Santiago-Dieppa; Scott E Olson; J Scott Pannell; Alexander A Khalessi Journal: Neurosurgery Date: 2019-07-01 Impact factor: 4.654