Justine H Ryu1, Andrew J Zimolzak2,3. 1. Department of Internal Medicine, Boston University School of Medicine, Boston, MA. 2. Michael E. DeBakey VA Medical Center, Houston, TX. 3. Baylor College of Medicine, Department of Internal Medicine, Houston, TX.
Abstract
PURPOSE: Serum protein electrophoresis (SPEP) is a clinical tool used to screen for monoclonal gammopathy, thus it is a critical tool in the evaluation of patients with multiple myeloma. However, SPEP laboratory results are usually returned as short text reports, which are not amenable to simple computerized processing for large-scale studies. We applied natural language processing (NLP) to detect monoclonal gammopathy in SPEP laboratory results and compared its performance at multiple hospitals using both a rules-based manual system and a machine-learning algorithm. METHODS: We used the data from the VA Corporate Data Warehouse, which comprises data from 20 million unique individuals. SPEP reports were collected from July to December 2015 at 5 Veterans Affairs Medical Centers. Of these reports, we annotated the presence or absence of monoclonal gammopathy in 300 reports. We applied a machine learning-based NLP and a manual rules-based NLP to detect monoclonal gammopathy in SPEP reports at each of the hospitals, then applied the model from 1 hospital to each of the other hospitals. RESULTS: The learning system achieved an area under the receiver operating characteristic curve of 0.997, and the rules-based system achieved an accuracy of 0.99. When a model trained on 1 hospital's data was applied to a different hospital, however, accuracy varied greatly, and the learning-based models performed better than the rules-based model. CONCLUSION: Binary classification of short clinical texts such as SPEP reports may be a particularly attractive target on which to train highly accurate NLP systems.
PURPOSE: Serum protein electrophoresis (SPEP) is a clinical tool used to screen for monoclonal gammopathy, thus it is a critical tool in the evaluation of patients with multiple myeloma. However, SPEP laboratory results are usually returned as short text reports, which are not amenable to simple computerized processing for large-scale studies. We applied natural language processing (NLP) to detect monoclonal gammopathy in SPEP laboratory results and compared its performance at multiple hospitals using both a rules-based manual system and a machine-learning algorithm. METHODS: We used the data from the VA Corporate Data Warehouse, which comprises data from 20 million unique individuals. SPEP reports were collected from July to December 2015 at 5 Veterans Affairs Medical Centers. Of these reports, we annotated the presence or absence of monoclonal gammopathy in 300 reports. We applied a machine learning-based NLP and a manual rules-based NLP to detect monoclonal gammopathy in SPEP reports at each of the hospitals, then applied the model from 1 hospital to each of the other hospitals. RESULTS: The learning system achieved an area under the receiver operating characteristic curve of 0.997, and the rules-based system achieved an accuracy of 0.99. When a model trained on 1 hospital's data was applied to a different hospital, however, accuracy varied greatly, and the learning-based models performed better than the rules-based model. CONCLUSION: Binary classification of short clinical texts such as SPEP reports may be a particularly attractive target on which to train highly accurate NLP systems.
Authors: Jillian Tate; Grahame Caldwell; James Daly; David Gillis; Margaret Jenkins; Sue Jovanovich; Helen Martin; Richard Steele; Louise Wienholt; Peter Mollee Journal: Ann Clin Biochem Date: 2012-03-08 Impact factor: 2.057
Authors: S Vincent Rajkumar; Meletios A Dimopoulos; Antonio Palumbo; Joan Blade; Giampaolo Merlini; María-Victoria Mateos; Shaji Kumar; Jens Hillengass; Efstathios Kastritis; Paul Richardson; Ola Landgren; Bruno Paiva; Angela Dispenzieri; Brendan Weiss; Xavier LeLeu; Sonja Zweegman; Sagar Lonial; Laura Rosinol; Elena Zamagni; Sundar Jagannath; Orhan Sezer; Sigurdur Y Kristinsson; Jo Caers; Saad Z Usmani; Juan José Lahuerta; Hans Erik Johnsen; Meral Beksac; Michele Cavo; Hartmut Goldschmidt; Evangelos Terpos; Robert A Kyle; Kenneth C Anderson; Brian G M Durie; Jesus F San Miguel Journal: Lancet Oncol Date: 2014-10-26 Impact factor: 41.316