Yadan Fan1, Rui Zhang2,3. 1. Institute for Health Informatics, University of Minnesota, Minneapolis, MN, USA. 2. Institute for Health Informatics, University of Minnesota, Minneapolis, MN, USA. zhan1386@umn.edu. 3. Department of Pharmaceutical Care & Health Systems, College of Pharmacy, University of Minnesota, Minneapolis, MN, USA. zhan1386@umn.edu.
Abstract
BACKGROUND: Despite widespread use, the safety of dietary supplements is open to doubt due to the fact that they can interact with prescribed medications, leading to dangerous clinical outcomes. Electronic health records (EHRs) provide a potential way for active pharmacovigilance on dietary supplements since a fair amount of dietary supplement information, especially those on use status, can be found in clinical notes. Extracting such information is extremely significant for subsequent supplement safety research. METHODS: In this study, we collected 2500 sentences for 25 commonly used dietary supplements and annotated into four classes: Continuing (C), Discontinued (D), Started (S) and Unclassified (U). Both rule-based and machine learning-based classifiers were developed on the same training set and evaluated using the hold-out test set. The performances of the two classifiers were also compared. RESULTS: The rule-based classifier achieved F-measure of 0.90, 0.85, 0.90, and 0.86 in C, D, S, and U status, respectively. The optimal machine learning-based classifier (Maximum Entropy) achieved F-measure of 0.90, 0.92, 0.91 and 0.88 in C, D, S, and U status, respectively. The comparison result shows that the machine learning-based classifier has a better performance, which is more efficient and scalable especially when the sample size doubles. CONCLUSIONS: Machine learning-based classifier outperforms rule-based classifier in categorization of the use status of dietary supplements in clinical notes. Future work includes applying deep learning methods and developing a hybrid system to approach use status classification task.
BACKGROUND: Despite widespread use, the safety of dietary supplements is open to doubt due to the fact that they can interact with prescribed medications, leading to dangerous clinical outcomes. Electronic health records (EHRs) provide a potential way for active pharmacovigilance on dietary supplements since a fair amount of dietary supplement information, especially those on use status, can be found in clinical notes. Extracting such information is extremely significant for subsequent supplement safety research. METHODS: In this study, we collected 2500 sentences for 25 commonly used dietary supplements and annotated into four classes: Continuing (C), Discontinued (D), Started (S) and Unclassified (U). Both rule-based and machine learning-based classifiers were developed on the same training set and evaluated using the hold-out test set. The performances of the two classifiers were also compared. RESULTS: The rule-based classifier achieved F-measure of 0.90, 0.85, 0.90, and 0.86 in C, D, S, and U status, respectively. The optimal machine learning-based classifier (Maximum Entropy) achieved F-measure of 0.90, 0.92, 0.91 and 0.88 in C, D, S, and U status, respectively. The comparison result shows that the machine learning-based classifier has a better performance, which is more efficient and scalable especially when the sample size doubles. CONCLUSIONS: Machine learning-based classifier outperforms rule-based classifier in categorization of the use status of dietary supplements in clinical notes. Future work includes applying deep learning methods and developing a hybrid system to approach use status classification task.
Entities:
Keywords:
Clinical notes; Dietary supplements; Machine learning-based classification; Natural language processing; Rule-based method; Use status
Authors: Andrew I Geller; Nadine Shehab; Nina J Weidle; Maribeth C Lovegrove; Beverly J Wolpert; Babgaleh B Timbo; Robert P Mozersky; Daniel S Budnitz Journal: N Engl J Med Date: 2015-10-15 Impact factor: 91.245
Authors: Stéphane M Meystre; Youngjun Kim; Julia Heavirland; Jenifer Williams; Bruce E Bray; Jennifer Garvin Journal: Stud Health Technol Inform Date: 2015