A M Cohen1, W R Hersh, K Peterson, Po-Yin Yen. 1. Department of Medical Informatics and Clinical Epidemiology, School of Medicine, Oregon Health & Science University, 3181 S.W. Sam Jackson Park Road, Mail Code BICC, Portland, OR 97239-3098, USA. cohenaa@ohsu.edu
Abstract
OBJECTIVE: To determine whether automated classification of document citations can be useful in reducing the time spent by experts reviewing journal articles for inclusion in updating systematic reviews of drug class efficacy for treatment of disease. DESIGN: A test collection was built using the annotated reference files from 15 systematic drug class reviews. A voting perceptron-based automated citation classification system was constructed to classify each article as containing high-quality, drug class-specific evidence or not. Cross-validation experiments were performed to evaluate performance. MEASUREMENTS: Precision, recall, and F-measure were evaluated at a range of sample weightings. Work saved over sampling at 95% recall was used as the measure of value to the review process. RESULTS: A reduction in the number of articles needing manual review was found for 11 of the 15 drug review topics studied. For three of the topics, the reduction was 50% or greater. CONCLUSION: Automated document citation classification could be a useful tool in maintaining systematic reviews of the efficacy of drug therapy. Further work is needed to refine the classification system and determine the best manner to integrate the system into the production of systematic reviews.
OBJECTIVE: To determine whether automated classification of document citations can be useful in reducing the time spent by experts reviewing journal articles for inclusion in updating systematic reviews of drug class efficacy for treatment of disease. DESIGN: A test collection was built using the annotated reference files from 15 systematic drug class reviews. A voting perceptron-based automated citation classification system was constructed to classify each article as containing high-quality, drug class-specific evidence or not. Cross-validation experiments were performed to evaluate performance. MEASUREMENTS: Precision, recall, and F-measure were evaluated at a range of sample weightings. Work saved over sampling at 95% recall was used as the measure of value to the review process. RESULTS: A reduction in the number of articles needing manual review was found for 11 of the 15 drug review topics studied. For three of the topics, the reduction was 50% or greater. CONCLUSION: Automated document citation classification could be a useful tool in maintaining systematic reviews of the efficacy of drug therapy. Further work is needed to refine the classification system and determine the best manner to integrate the system into the production of systematic reviews.
Authors: Yindalon Aphinyanaphongs; Ioannis Tsamardinos; Alexander Statnikov; Douglas Hardin; Constantin F Aliferis Journal: J Am Med Inform Assoc Date: 2004-11-23 Impact factor: 4.497
Authors: Ida Sim; Samson W Tu; Simona Carini; Harold P Lehmann; Brad H Pollock; Mor Peleg; Knut M Wittkowski Journal: J Biomed Inform Date: 2013-11-13 Impact factor: 6.317
Authors: Zhengyi Deng; Kanhua Yin; Yujia Bao; Victor Diego Armengol; Cathy Wang; Ankur Tiwari; Regina Barzilay; Giovanni Parmigiani; Danielle Braun; Kevin S Hughes Journal: JCO Clin Cancer Inform Date: 2019-08
Authors: Byron C Wallace; Thomas A Trikalinos; Joseph Lau; Carla Brodley; Christopher H Schmid Journal: BMC Bioinformatics Date: 2010-01-26 Impact factor: 3.169