Dane Gunter1, Paulo Puac-Polanco2, Olivier Miguel2, Rebecca E Thornhill3, Amy Y X Yu4, Zhongyu A Liu4, Muhammad Mamdani5, Chloe Pou-Prom6, Richard I Aviv7,8. 1. The Ottawa Hospital Research Institute, Ottawa, ON, Canada. 2. Department of Radiology, Radiation Oncology and Medical Physics, University of Ottawa, The Ottawa Hospital Civic Campus Room C110, 1053 Carling Ave, Ottawa, ON, ON K1Y 4E9, Canada. 3. Division of Medical Physics, Department of Radiology, Radiation Oncology and Medical Physics, University of Ottawa, Ottawa, ON, Canada. 4. Department of Medicine (Neurology), University of Toronto, Sunnybrook Health Sciences Centre, Toronto, ON, Canada. 5. Department of Medicine, Unity Health Toronto, University of Toronto, Toronto, ON, Canada. 6. Unity Health Toronto, Toronto, ON, Canada. 7. The Ottawa Hospital Research Institute, Ottawa, ON, Canada. raviv@toh.ca. 8. Department of Radiology, Radiation Oncology and Medical Physics, University of Ottawa, The Ottawa Hospital Civic Campus Room C110, 1053 Carling Ave, Ottawa, ON, ON K1Y 4E9, Canada. raviv@toh.ca.
Abstract
PURPOSE: Data extraction from radiology free-text reports is time consuming when performed manually. Recently, more automated extraction methods using natural language processing (NLP) are proposed. A previously developed rule-based NLP algorithm showed promise in its ability to extract stroke-related data from radiology reports. We aimed to externally validate the accuracy of CHARTextract, a rule-based NLP algorithm, to extract stroke-related data from free-text radiology reports. METHODS: Free-text reports of CT angiography (CTA) and perfusion (CTP) studies of consecutive patients with acute ischemic stroke admitted to a regional stroke center for endovascular thrombectomy were analyzed from January 2015 to 2021. Stroke-related variables were manually extracted as reference standard from clinical reports, including proximal and distal anterior circulation occlusion, posterior circulation occlusion, presence of ischemia or hemorrhage, Alberta stroke program early CT score (ASPECTS), and collateral status. These variables were simultaneously extracted using a rule-based NLP algorithm. The NLP algorithm's accuracy, specificity, sensitivity, positive predictive value (PPV), and negative predictive value (NPV) were assessed. RESULTS: The NLP algorithm's accuracy was > 90% for identifying distal anterior occlusion, posterior circulation occlusion, hemorrhage, and ASPECTS. Accuracy was 85%, 74%, and 79% for proximal anterior circulation occlusion, presence of ischemia, and collateral status respectively. The algorithm confirmed the absence of variables from radiology reports with an 87-100% accuracy. CONCLUSIONS: Rule-based NLP has a moderate to good performance for stroke-related data extraction from free-text imaging reports. The algorithm's accuracy was affected by inconsistent report styles and lexicon among reporting radiologists.
PURPOSE: Data extraction from radiology free-text reports is time consuming when performed manually. Recently, more automated extraction methods using natural language processing (NLP) are proposed. A previously developed rule-based NLP algorithm showed promise in its ability to extract stroke-related data from radiology reports. We aimed to externally validate the accuracy of CHARTextract, a rule-based NLP algorithm, to extract stroke-related data from free-text radiology reports. METHODS: Free-text reports of CT angiography (CTA) and perfusion (CTP) studies of consecutive patients with acute ischemic stroke admitted to a regional stroke center for endovascular thrombectomy were analyzed from January 2015 to 2021. Stroke-related variables were manually extracted as reference standard from clinical reports, including proximal and distal anterior circulation occlusion, posterior circulation occlusion, presence of ischemia or hemorrhage, Alberta stroke program early CT score (ASPECTS), and collateral status. These variables were simultaneously extracted using a rule-based NLP algorithm. The NLP algorithm's accuracy, specificity, sensitivity, positive predictive value (PPV), and negative predictive value (NPV) were assessed. RESULTS: The NLP algorithm's accuracy was > 90% for identifying distal anterior occlusion, posterior circulation occlusion, hemorrhage, and ASPECTS. Accuracy was 85%, 74%, and 79% for proximal anterior circulation occlusion, presence of ischemia, and collateral status respectively. The algorithm confirmed the absence of variables from radiology reports with an 87-100% accuracy. CONCLUSIONS: Rule-based NLP has a moderate to good performance for stroke-related data extraction from free-text imaging reports. The algorithm's accuracy was affected by inconsistent report styles and lexicon among reporting radiologists.
Authors: Amy Y X Yu; Jessalyn K Holodinsky; Charlotte Zerna; Lawrence W Svenson; Nathalie Jetté; Hude Quan; Michael D Hill Journal: Stroke Date: 2016-05-12 Impact factor: 7.914
Authors: M D Li; M Lang; F Deng; K Chang; K Buch; S Rincon; W A Mehan; T M Leslie-Mazwi; J Kalpathy-Cramer Journal: AJNR Am J Neuroradiol Date: 2020-12-17 Impact factor: 3.825
Authors: Charlene Jennifer Ong; Agni Orfanoudaki; Rebecca Zhang; Francois Pierre M Caprasse; Meghan Hutch; Liang Ma; Darian Fard; Oluwafemi Balogun; Matthew I Miller; Margaret Minnig; Hanife Saglam; Brenton Prescott; David M Greer; Stelios Smirnakis; Dimitris Bertsimas Journal: PLoS One Date: 2020-06-19 Impact factor: 3.240
Authors: Arlene Casey; Emma Davidson; Michael Poon; Hang Dong; Daniel Duma; Andreas Grivas; Claire Grover; Víctor Suárez-Paniagua; Richard Tobin; William Whiteley; Honghan Wu; Beatrice Alex Journal: BMC Med Inform Decis Mak Date: 2021-06-03 Impact factor: 2.796
Authors: Amy Y X Yu; Zhongyu A Liu; Chloe Pou-Prom; Kaitlyn Lopes; Moira K Kapral; Richard I Aviv; Muhammad Mamdani Journal: JMIR Med Inform Date: 2021-05-04