OBJECTIVES: Left ventricular ejection fraction (EF) is a key component of heart failure quality measures used within the Department of Veteran Affairs (VA). Our goals were to build a natural language processing system to extract the EF from free-text echocardiogram reports to automate measurement reporting and to validate the accuracy of the system using a comparison reference standard developed through human review. This project was a Translational Use Case Project within the VA Consortium for Healthcare Informatics. MATERIALS AND METHODS: We created a set of regular expressions and rules to capture the EF using a random sample of 765 echocardiograms from seven VA medical centers. The documents were randomly assigned to two sets: a set of 275 used for training and a second set of 490 used for testing and validation. To establish the reference standard, two independent reviewers annotated all documents in both sets; a third reviewer adjudicated disagreements. RESULTS: System test results for document-level classification of EF of <40% had a sensitivity (recall) of 98.41%, a specificity of 100%, a positive predictive value (precision) of 100%, and an F measure of 99.2%. System test results at the concept level had a sensitivity of 88.9% (95% CI 87.7% to 90.0%), a positive predictive value of 95% (95% CI 94.2% to 95.9%), and an F measure of 91.9% (95% CI 91.2% to 92.7%). DISCUSSION: An EF value of <40% can be accurately identified in VA echocardiogram reports. CONCLUSIONS: An automated information extraction system can be used to accurately extract EF for quality measurement.
OBJECTIVES: Left ventricular ejection fraction (EF) is a key component of heart failure quality measures used within the Department of Veteran Affairs (VA). Our goals were to build a natural language processing system to extract the EF from free-text echocardiogram reports to automate measurement reporting and to validate the accuracy of the system using a comparison reference standard developed through human review. This project was a Translational Use Case Project within the VA Consortium for Healthcare Informatics. MATERIALS AND METHODS: We created a set of regular expressions and rules to capture the EF using a random sample of 765 echocardiograms from seven VA medical centers. The documents were randomly assigned to two sets: a set of 275 used for training and a second set of 490 used for testing and validation. To establish the reference standard, two independent reviewers annotated all documents in both sets; a third reviewer adjudicated disagreements. RESULTS: System test results for document-level classification of EF of <40% had a sensitivity (recall) of 98.41%, a specificity of 100%, a positive predictive value (precision) of 100%, and an F measure of 99.2%. System test results at the concept level had a sensitivity of 88.9% (95% CI 87.7% to 90.0%), a positive predictive value of 95% (95% CI 94.2% to 95.9%), and an F measure of 91.9% (95% CI 91.2% to 92.7%). DISCUSSION: An EF value of <40% can be accurately identified in VA echocardiogram reports. CONCLUSIONS: An automated information extraction system can be used to accurately extract EF for quality measurement.
Authors: Joseph L Goulet; Joseph Erdos; Sue Kancir; Forrest L Levin; Steven M Wright; Stanlie M Daniels; Lynnette Nilan; Amy C Justice Journal: Med Care Date: 2007-01 Impact factor: 2.983
Authors: Meryl Bloomrosen; Justin Starren; Nancy M Lorenzi; Joan S Ash; Vimla L Patel; Edward H Shortliffe Journal: J Am Med Inform Assoc Date: 2011 Jan-Feb Impact factor: 4.497
Authors: Brandon K Bellows; Joanne LaFleur; Aaron W C Kamauu; Thomas Ginter; Tyler B Forbush; Stephen Agbor; Dylan Supina; Paul Hodgkins; Scott L DuVall Journal: J Am Med Inform Assoc Date: 2013-11-07 Impact factor: 4.497
Authors: Benjamin J Lerman; Rita A Popat; Themistocles L Assimes; Paul A Heidenreich; Sherry M Wren Journal: JAMA Date: 2019-02-12 Impact factor: 56.272