Erel Joffe1, Emily J Pettigrew2, Jorge R Herskovic3, Charles F Bearden4, Elmer V Bernstam5. 1. School of Biomedical Informatics, The University of Texas Health Science Center at Houston, Texas Department of Hematology and Bone Marrow Transplantation, Tel Aviv Medical Center, Tel Aviv Israel. 2. Department of Computer Science, Rice University, Houston, Texas. 3. School of Biomedical Informatics, The University of Texas Health Science Center at Houston, Texas The University of Texas, M.D. Anderson Cancer Center, Houston, Texas. 4. School of Biomedical Informatics, The University of Texas Health Science Center at Houston, Texas. 5. School of Biomedical Informatics, The University of Texas Health Science Center at Houston, Texas Department of Internal Medicine, Medical School, The University of Texas Health Science Center at Houston, Houston, Texas Elmer.v.bernstam@uth.tmc.edu.
Abstract
INTRODUCTION: Automatically identifying specific phenotypes in free-text clinical notes is critically important for the reuse of clinical data. In this study, the authors combine expert-guided feature (text) selection with one-class classification for text processing. OBJECTIVES: To compare the performance of one-class classification to traditional binary classification; to evaluate the utility of feature selection based on expert-selected salient text (snippets); and to determine the robustness of these models with respects to irrelevant surrounding text. METHODS: The authors trained one-class support vector machines (1C-SVMs) and two-class SVMs (2C-SVMs) to identify notes discussing breast cancer. Manually annotated visit summary notes (88 positive and 88 negative for breast cancer) were used to compare the performance of models trained on whole notes labeled as positive or negative to models trained on expert-selected text sections (snippets) relevant to breast cancer status. Model performance was evaluated using a 70:30 split for 20 iterations and on a realistic dataset of 10 000 records with a breast cancer prevalence of 1.4%. RESULTS: When tested on a balanced experimental dataset, 1C-SVMs trained on snippets had comparable results to 2C-SVMs trained on whole notes (F = 0.92 for both approaches). When evaluated on a realistic imbalanced dataset, 1C-SVMs had a considerably superior performance (F = 0.61 vs. F = 0.17 for the best performing model) attributable mainly to improved precision (p = .88 vs. p = .09 for the best performing model). CONCLUSIONS: 1C-SVMs trained on expert-selected relevant text sections perform better than 2C-SVMs classifiers trained on either snippets or whole notes when applied to realistically imbalanced data with low prevalence of the positive class.
INTRODUCTION: Automatically identifying specific phenotypes in free-text clinical notes is critically important for the reuse of clinical data. In this study, the authors combine expert-guided feature (text) selection with one-class classification for text processing. OBJECTIVES: To compare the performance of one-class classification to traditional binary classification; to evaluate the utility of feature selection based on expert-selected salient text (snippets); and to determine the robustness of these models with respects to irrelevant surrounding text. METHODS: The authors trained one-class support vector machines (1C-SVMs) and two-class SVMs (2C-SVMs) to identify notes discussing breast cancer. Manually annotated visit summary notes (88 positive and 88 negative for breast cancer) were used to compare the performance of models trained on whole notes labeled as positive or negative to models trained on expert-selected text sections (snippets) relevant to breast cancer status. Model performance was evaluated using a 70:30 split for 20 iterations and on a realistic dataset of 10 000 records with a breast cancer prevalence of 1.4%. RESULTS: When tested on a balanced experimental dataset, 1C-SVMs trained on snippets had comparable results to 2C-SVMs trained on whole notes (F = 0.92 for both approaches). When evaluated on a realistic imbalanced dataset, 1C-SVMs had a considerably superior performance (F = 0.61 vs. F = 0.17 for the best performing model) attributable mainly to improved precision (p = .88 vs. p = .09 for the best performing model). CONCLUSIONS: 1C-SVMs trained on expert-selected relevant text sections perform better than 2C-SVMs classifiers trained on either snippets or whole notes when applied to realistically imbalanced data with low prevalence of the positive class.
Authors: Erel Joffe; Ofer Havakuk; Jorge R Herskovic; Vimla L Patel; Elmer Victor Bernstam Journal: J Am Med Inform Assoc Date: 2012-06-28 Impact factor: 4.497
Authors: Angus Roberts; Robert Gaizauskas; Mark Hepple; George Demetriou; Yikun Guo; Ian Roberts; Andrea Setzer Journal: J Biomed Inform Date: 2009-01-23 Impact factor: 6.317
Authors: Kirk Roberts; Mary Regina Boland; Lisiane Pruinelli; Jina Dcruz; Andrew Berry; Mattias Georgsson; Rebecca Hazen; Raymond F Sarmiento; Uba Backonja; Kun-Hsing Yu; Yun Jiang; Patricia Flatley Brennan Journal: J Am Med Inform Assoc Date: 2017-04-01 Impact factor: 4.497
Authors: Majid Afshar; Cara Joyce; Anthony Oakey; Perry Formanek; Philip Yang; Matthew M Churpek; Richard S Cooper; Susan Zelisko; Ron Price; Dmitriy Dligach Journal: AMIA Annu Symp Proc Date: 2018-12-05