BACKGROUND: Reliably abstracting outcomes from free-text electronic health records remains a challenge. While automated classification of free text has been a popular medical informatics topic, performance validation using real-world clinical data has been limited. The two main approaches are linguistic (natural language processing [NLP]) and statistical (machine learning). The authors have developed a hybrid system for abstracting computed tomography (CT) reports for specified outcomes. OBJECTIVES: The objective was to measure performance of a hybrid NLP and machine learning system for automated outcome classification of emergency department (ED) CT imaging reports. The hypothesis was that such a system is comparable to medical personnel doing the data abstraction. METHODS: A secondary analysis was performed on a prior diagnostic imaging study on 3,710 blunt facial trauma victims. Staff radiologists dictated CT reports as free text, which were then deidentified. A trained data abstractor manually coded the reference standard outcome of acute orbital fracture, with a random subset double-coded for reliability. The data set was randomly split evenly into training and testing sets. Training patient reports were used as input to the Medical Language Extraction and Encoding (MedLEE) NLP tool to create structured output containing standardized medical terms and modifiers for certainty and temporal status. Findings were filtered for low certainty and past/future modifiers and then combined with the manual reference standard to generate decision tree classifiers using data mining tools Waikato Environment for Knowledge Analysis (WEKA) 3.7.5 and Salford Predictive Miner 6.6. Performance of decision tree classifiers was evaluated on the testing set with or without NLP processing. RESULTS: The performance of machine learning alone was comparable to prior NLP studies (sensitivity = 0.92, specificity = 0.93, precision = 0.95, recall = 0.93, f-score = 0.94), and the combined use of NLP and machine learning showed further improvement (sensitivity = 0.93, specificity = 0.97, precision = 0.97, recall = 0.96, f-score = 0.97). This performance is similar to, or better than, that of medical personnel in previous studies. CONCLUSIONS: A hybrid NLP and machine learning automated classification system shows promise in coding free-text electronic clinical data.
BACKGROUND: Reliably abstracting outcomes from free-text electronic health records remains a challenge. While automated classification of free text has been a popular medical informatics topic, performance validation using real-world clinical data has been limited. The two main approaches are linguistic (natural language processing [NLP]) and statistical (machine learning). The authors have developed a hybrid system for abstracting computed tomography (CT) reports for specified outcomes. OBJECTIVES: The objective was to measure performance of a hybrid NLP and machine learning system for automated outcome classification of emergency department (ED) CT imaging reports. The hypothesis was that such a system is comparable to medical personnel doing the data abstraction. METHODS: A secondary analysis was performed on a prior diagnostic imaging study on 3,710 blunt facial trauma victims. Staff radiologists dictated CT reports as free text, which were then deidentified. A trained data abstractor manually coded the reference standard outcome of acute orbital fracture, with a random subset double-coded for reliability. The data set was randomly split evenly into training and testing sets. Training patient reports were used as input to the Medical Language Extraction and Encoding (MedLEE) NLP tool to create structured output containing standardized medical terms and modifiers for certainty and temporal status. Findings were filtered for low certainty and past/future modifiers and then combined with the manual reference standard to generate decision tree classifiers using data mining tools Waikato Environment for Knowledge Analysis (WEKA) 3.7.5 and Salford Predictive Miner 6.6. Performance of decision tree classifiers was evaluated on the testing set with or without NLP processing. RESULTS: The performance of machine learning alone was comparable to prior NLP studies (sensitivity = 0.92, specificity = 0.93, precision = 0.95, recall = 0.93, f-score = 0.94), and the combined use of NLP and machine learning showed further improvement (sensitivity = 0.93, specificity = 0.97, precision = 0.97, recall = 0.96, f-score = 0.97). This performance is similar to, or better than, that of medical personnel in previous studies. CONCLUSIONS: A hybrid NLP and machine learning automated classification system shows promise in coding free-text electronic clinical data.
Authors: Keith J Dreyer; Mannudeep K Kalra; Michael M Maher; Autumn M Hurier; Benjamin A Asfaw; Thomas Schultz; Elkan F Halpern; James H Thrall Journal: Radiology Date: 2004-12-10 Impact factor: 11.105
Authors: Alexander Turchin; Nikheel S Kolatkar; Richard W Grant; Eric C Makhni; Merri L Pendergrass; Jonathan S Einbinder Journal: J Am Med Inform Assoc Date: 2006-08-23 Impact factor: 4.497
Authors: Margaret Mahan; Daniel Rafter; Hannah Casey; Marta Engelking; Tessneem Abdallah; Charles Truwit; Mark Oswood; Uzma Samadani Journal: PLoS One Date: 2020-07-01 Impact factor: 3.240
Authors: Robert W Grundmeier; Aaron J Masino; T Charles Casper; Jonathan M Dean; Jamie Bell; Rene Enriquez; Sara Deakyne; James M Chamberlain; Elizabeth R Alpern Journal: Appl Clin Inform Date: 2016-11-09 Impact factor: 2.342
Authors: Siddharth Biswal; Zarina Nip; Valdery Moura Junior; Matt T Bianchi; Eric S Rosenthal; M Brandon Westover Journal: Conf Proc IEEE Eng Med Biol Soc Date: 2015
Authors: Kabir Yadav; Efsun Sarioglu; Hyeong Ah Choi; Walter B Cartwright; Pamela S Hinds; James M Chamberlain Journal: Acad Emerg Med Date: 2016-01-14 Impact factor: 3.451