Ravi Garg1, Elissa Oh1, Andrew Naidech1, Konrad Kording2, Shyam Prabhakaran3. 1. Department of Neurology, Northwestern University, Feinberg School of Medicine, Chicago, Illinois. 2. University of Pennsylvania, Philadelphia, Pennsylvania. 3. Department of Neurology, Pritzker School of Medicine, University of Chicago, Chicago, Illinois. Electronic address: shyam1@uchicago.edu.
Abstract
OBJECTIVE: The manual adjudication of disease classification is time-consuming, error-prone, and limits scaling to large datasets. In ischemic stroke (IS), subtype classification is critical for management and outcome prediction. This study sought to use natural language processing of electronic health records (EHR) combined with machine learning methods to automate IS subtyping. METHODS: Among IS patients from an observational registry with TOAST subtyping adjudicated by board-certified vascular neurologists, we analyzed unstructured text-based EHR data including neurology progress notes and neuroradiology reports using natural language processing. We performed several feature selection methods to reduce the high dimensionality of the features and 5-fold cross validation to test generalizability of our methods and minimize overfitting. We used several machine learning methods and calculated the kappa values for agreement between each machine learning approach to manual adjudication. We then performed a blinded testing of the best algorithm against a held-out subset of 50 cases. RESULTS: Compared to manual classification, the best machine-based classification achieved a kappa of .25 using radiology reports alone, .57 using progress notes alone, and .57 using combined data. Kappa values varied by subtype being highest for cardioembolic (.64) and lowest for cryptogenic cases (.47). In the held-out test subset, machine-based classification agreed with rater classification in 40 of 50 cases (kappa .72). CONCLUSIONS: Automated machine learning approaches using textual data from the EHR shows agreement with manual TOAST classification. The automated pipeline, if externally validated, could enable large-scale stroke epidemiology research.
OBJECTIVE: The manual adjudication of disease classification is time-consuming, error-prone, and limits scaling to large datasets. In ischemic stroke (IS), subtype classification is critical for management and outcome prediction. This study sought to use natural language processing of electronic health records (EHR) combined with machine learning methods to automate IS subtyping. METHODS: Among IS patients from an observational registry with TOAST subtyping adjudicated by board-certified vascular neurologists, we analyzed unstructured text-based EHR data including neurology progress notes and neuroradiology reports using natural language processing. We performed several feature selection methods to reduce the high dimensionality of the features and 5-fold cross validation to test generalizability of our methods and minimize overfitting. We used several machine learning methods and calculated the kappa values for agreement between each machine learning approach to manual adjudication. We then performed a blinded testing of the best algorithm against a held-out subset of 50 cases. RESULTS: Compared to manual classification, the best machine-based classification achieved a kappa of .25 using radiology reports alone, .57 using progress notes alone, and .57 using combined data. Kappa values varied by subtype being highest for cardioembolic (.64) and lowest for cryptogenic cases (.47). In the held-out test subset, machine-based classification agreed with rater classification in 40 of 50 cases (kappa .72). CONCLUSIONS: Automated machine learning approaches using textual data from the EHR shows agreement with manual TOAST classification. The automated pipeline, if externally validated, could enable large-scale stroke epidemiology research.
Authors: Dane Gunter; Paulo Puac-Polanco; Olivier Miguel; Rebecca E Thornhill; Amy Y X Yu; Zhongyu A Liu; Muhammad Mamdani; Chloe Pou-Prom; Richard I Aviv Journal: Neuroradiology Date: 2022-08-01 Impact factor: 2.995
Authors: Nathan A Shlobin; Ammad A Baig; Muhammad Waqas; Tatsat R Patel; Rimal H Dossani; Megan Wilson; Justin M Cappuzzo; Adnan H Siddiqui; Vincent M Tutino; Elad I Levy Journal: World Neurosurg Date: 2021-12-08 Impact factor: 2.210
Authors: Wyliena Guan; Darae Ko; Shaan Khurshid; Ana T Trisini Lipsanopoulos; Jeffrey M Ashburner; Lia X Harrington; Natalia S Rost; Steven J Atlas; Daniel E Singer; David D McManus; Christopher D Anderson; Steven A Lubitz Journal: Stroke Date: 2020-12-10 Impact factor: 7.914
Authors: B Jiang; G Zhu; Y Xie; J J Heit; H Chen; Y Li; V Ding; A Eskandari; P Michel; G Zaharchuk; M Wintermark Journal: AJNR Am J Neuroradiol Date: 2021-01-07 Impact factor: 3.825
Authors: M D Li; M Lang; F Deng; K Chang; K Buch; S Rincon; W A Mehan; T M Leslie-Mazwi; J Kalpathy-Cramer Journal: AJNR Am J Neuroradiol Date: 2020-12-17 Impact factor: 3.825