Debbie A Travers1, Stephanie W Haas. 1. Department of Emergency Medicine, School of Medicine, CB 7594, University of North Carolina at Chapel Hill, Chapel Hill, NC 27599-7594, USA. dtravers@med.unc.edu
Abstract
OBJECTIVES: Emergency Medical Text Processor (EMT-P) version 1, a natural language processing system that cleans emergency department text (e.g., chst pn, chest pai), was developed to maximize extraction of standard terms (e.g., chest pain). The authors compared the number of standard terms extracted from raw chief complaint (CC) data with that for CC data cleaned with EMT-P and evaluated the accuracy of EMT-P. METHODS: This cross-sectional observation study included CC text entries for all emergency department visits to three tertiary care centers in 2001. Terms were extracted from CC entries before and after cleaning with EMT-P. Descriptive statistics included number and percentage of all entries (tokens) and all unique entries (types) that matched a standard term from the Unified Medical Language System (UMLS). An expert panel rated the accuracy of the CC-UMLS term matches; inter-rater reliability was measured with kappa. RESULTS: The authors collected 203,509 CC entry tokens, of which 63,946 were unique entry types. For the raw data, 89,337 tokens (44%) and 5,081 types (8%) matched a standard term. After EMT-P cleaning, 168,050 tokens (83%) and 44,430 types (69%) matched a standard term. The expert panel reached consensus on 201 of the 222 CC-UMLS term matches reviewed (kappa=0.69-0.72). Ninety-six percent of the 201 matches were rated equivalent or related. Thirty-eight percent of the nonmatches were found to match UMLS concepts. CONCLUSIONS: EMT-P version 1 is relatively accurate, and cleaning with EMT-P improved the CC-UMLS term match rate over raw data. The authors identified areas for improvement in future EMT-P versions and issues to be resolved in developing a standard CC terminology.
OBJECTIVES: Emergency Medical Text Processor (EMT-P) version 1, a natural language processing system that cleans emergency department text (e.g., chst pn, chest pai), was developed to maximize extraction of standard terms (e.g., chest pain). The authors compared the number of standard terms extracted from raw chief complaint (CC) data with that for CC data cleaned with EMT-P and evaluated the accuracy of EMT-P. METHODS: This cross-sectional observation study included CC text entries for all emergency department visits to three tertiary care centers in 2001. Terms were extracted from CC entries before and after cleaning with EMT-P. Descriptive statistics included number and percentage of all entries (tokens) and all unique entries (types) that matched a standard term from the Unified Medical Language System (UMLS). An expert panel rated the accuracy of the CC-UMLS term matches; inter-rater reliability was measured with kappa. RESULTS: The authors collected 203,509 CC entry tokens, of which 63,946 were unique entry types. For the raw data, 89,337 tokens (44%) and 5,081 types (8%) matched a standard term. After EMT-P cleaning, 168,050 tokens (83%) and 44,430 types (69%) matched a standard term. The expert panel reached consensus on 201 of the 222 CC-UMLS term matches reviewed (kappa=0.69-0.72). Ninety-six percent of the 201 matches were rated equivalent or related. Thirty-eight percent of the nonmatches were found to match UMLS concepts. CONCLUSIONS: EMT-P version 1 is relatively accurate, and cleaning with EMT-P improved the CC-UMLS term match rate over raw data. The authors identified areas for improvement in future EMT-P versions and issues to be resolved in developing a standard CC terminology.
Authors: Richard S Hopkins; Catherine C Tong; Howard S Burkom; Judy E Akkina; John Berezowski; Mika Shigematsu; Patrick D Finley; Ian Painter; Roland Gamache; Victor J Del Rio Vilas; Laura C Streichert Journal: Public Health Rep Date: 2017 Jul/Aug Impact factor: 2.792
Authors: Debbie Travers; Stephanie W Haas; Anna E Waller; Todd A Schwartz; Javed Mostafa; Nakia C Best; John Crouch Journal: AMIA Annu Symp Proc Date: 2013-11-16
Authors: Richard T Griffey; Jesse M Pines; Heather L Farley; Michael P Phelan; Christopher Beach; Jeremiah D Schuur; Arjun K Venkatesh Journal: Ann Emerg Med Date: 2014-10-16 Impact factor: 5.721
Authors: Hannah Eyre; Alec B Chapman; Kelly S Peterson; Jianlin Shi; Patrick R Alba; Makoto M Jones; Tamára L Box; Scott L DuVall; Olga V Patterson Journal: AMIA Annu Symp Proc Date: 2022-02-21
Authors: Arthur S Hong; Hannah Chang; D Mark Courtney; Hannah Fullington; Simon J Craddock Lee; John W Sweetenham; Ethan A Halm Journal: JCO Oncol Pract Date: 2021-01-08