BACKGROUND: Advanced machine learning methods can aid in the identification of dementia risk using neuroimaging-derived features including FDG-PET. However, to enable the translation of these methods and test their usefulness in clinical practice, it is crucial to conduct independent validation on real clinical samples, which has yet to be properly delineated in the current literature. OBJECTIVE: In this paper, we present our efforts to enable such clinical translational through the evaluation and comparison of two machine-learning methods for discrimination between dementia of Alzheimer's type (DAT) and Non-DAT controls. METHODS: FDG-PET-based dementia scores were generated on an independent clinical sample whose clinical diagnosis was blinded to the algorithm designers. A feature-engineered approach (multi-kernel probability classifier) and a non-feature-engineered approach (3D convolutional neural network) were analyzed. Both classifiers were pre-trained on cognitively normal subjects as well as subjects with DAT. These two methods provided a probabilistic dementia score for this previously unseen clinical data. Performance of the algorithms were compared against ground-truth dementia rating assessed by experienced nuclear physicians. RESULTS: Blinded clinical evaluation on both classifiers showed good separation between the cognitively normal subjects and the patients diagnosed with DAT. The non-feature-engineered dementia score showed higher sensitivity among subjects whose diagnosis was in agreement between the machine-learning models, while the feature-engineered approach showed higher specificity in non-consensus cases. CONCLUSION: In this study, we demonstrated blinded evaluation using data from an independent clinical sample for assessing the performance in DAT classification models in a clinical setting. Our results showed good generalizability for two machine-learning approaches, marking an important step for the translation of pre-trained machine-learning models into clinical practice.
BACKGROUND: Advanced machine learning methods can aid in the identification of dementia risk using neuroimaging-derived features including FDG-PET. However, to enable the translation of these methods and test their usefulness in clinical practice, it is crucial to conduct independent validation on real clinical samples, which has yet to be properly delineated in the current literature. OBJECTIVE: In this paper, we present our efforts to enable such clinical translational through the evaluation and comparison of two machine-learning methods for discrimination between dementia of Alzheimer's type (DAT) and Non-DAT controls. METHODS: FDG-PET-based dementia scores were generated on an independent clinical sample whose clinical diagnosis was blinded to the algorithm designers. A feature-engineered approach (multi-kernel probability classifier) and a non-feature-engineered approach (3D convolutional neural network) were analyzed. Both classifiers were pre-trained on cognitively normal subjects as well as subjects with DAT. These two methods provided a probabilistic dementia score for this previously unseen clinical data. Performance of the algorithms were compared against ground-truth dementia rating assessed by experienced nuclear physicians. RESULTS: Blinded clinical evaluation on both classifiers showed good separation between the cognitively normal subjects and the patients diagnosed with DAT. The non-feature-engineered dementia score showed higher sensitivity among subjects whose diagnosis was in agreement between the machine-learning models, while the feature-engineered approach showed higher specificity in non-consensus cases. CONCLUSION: In this study, we demonstrated blinded evaluation using data from an independent clinical sample for assessing the performance in DAT classification models in a clinical setting. Our results showed good generalizability for two machine-learning approaches, marking an important step for the translation of pre-trained machine-learning models into clinical practice.
Authors: Yiming Ding; Jae Ho Sohn; Michael G Kawczynski; Hari Trivedi; Roy Harnish; Nathaniel W Jenkins; Dmytro Lituiev; Timothy P Copeland; Mariam S Aboian; Carina Mari Aparici; Spencer C Behr; Robert R Flavell; Shih-Ying Huang; Kelly A Zalocusky; Lorenzo Nardo; Youngho Seo; Randall A Hawkins; Miguel Hernandez Pampaloni; Dexter Hadley; Benjamin L Franc Journal: Radiology Date: 2018-11-06 Impact factor: 29.146