Duy Duc An Bui1, Guilherme Del Fiol2, John F Hurdle2, Siddhartha Jonnalagadda3. 1. Department of Biomedical Informatics, University of Utah, Salt Lake City, UT, USA; Division of Health and Biomedical Informatics, Northwestern University, Chicago, IL, USA. Electronic address: duy.bui@utah.edu. 2. Department of Biomedical Informatics, University of Utah, Salt Lake City, UT, USA. 3. Division of Health and Biomedical Informatics, Northwestern University, Chicago, IL, USA.
Abstract
OBJECTIVES: Extracting data from publication reports is a standard process in systematic review (SR) development. However, the data extraction process still relies too much on manual effort which is slow, costly, and subject to human error. In this study, we developed a text summarization system aimed at enhancing productivity and reducing errors in the traditional data extraction process. METHODS: We developed a computer system that used machine learning and natural language processing approaches to automatically generate summaries of full-text scientific publications. The summaries at the sentence and fragment levels were evaluated in finding common clinical SR data elements such as sample size, group size, and PICO values. We compared the computer-generated summaries with human written summaries (title and abstract) in terms of the presence of necessary information for the data extraction as presented in the Cochrane review's study characteristics tables. RESULTS: At the sentence level, the computer-generated summaries covered more information than humans do for systematic reviews (recall 91.2% vs. 83.8%, p<0.001). They also had a better density of relevant sentences (precision 59% vs. 39%, p<0.001). At the fragment level, the ensemble approach combining rule-based, concept mapping, and dictionary-based methods performed better than individual methods alone, achieving an 84.7% F-measure. CONCLUSION: Computer-generated summaries are potential alternative information sources for data extraction in systematic review development. Machine learning and natural language processing are promising approaches to the development of such an extractive summarization system. Copyright Â
OBJECTIVES: Extracting data from publication reports is a standard process in systematic review (SR) development. However, the data extraction process still relies too much on manual effort which is slow, costly, and subject to human error. In this study, we developed a text summarization system aimed at enhancing productivity and reducing errors in the traditional data extraction process. METHODS: We developed a computer system that used machine learning and natural language processing approaches to automatically generate summaries of full-text scientific publications. The summaries at the sentence and fragment levels were evaluated in finding common clinical SR data elements such as sample size, group size, and PICO values. We compared the computer-generated summaries with human written summaries (title and abstract) in terms of the presence of necessary information for the data extraction as presented in the Cochrane review's study characteristics tables. RESULTS: At the sentence level, the computer-generated summaries covered more information than humans do for systematic reviews (recall 91.2% vs. 83.8%, p<0.001). They also had a better density of relevant sentences (precision 59% vs. 39%, p<0.001). At the fragment level, the ensemble approach combining rule-based, concept mapping, and dictionary-based methods performed better than individual methods alone, achieving an 84.7% F-measure. CONCLUSION: Computer-generated summaries are potential alternative information sources for data extraction in systematic review development. Machine learning and natural language processing are promising approaches to the development of such an extractive summarization system. Copyright Â
Authors: Kaveh G Shojania; Margaret Sampson; Mohammed T Ansari; Jun Ji; Steve Doucette; David Moher Journal: Ann Intern Med Date: 2007-07-16 Impact factor: 25.391
Authors: Jim Hunter; Yvonne Freer; Albert Gatt; Robert Logie; Neil McIntosh; Marian van der Meulen; Francois Portet; Ehud Reiter; Somayajulu Sripada; Cindy Sykes Journal: AMIA Annu Symp Proc Date: 2008-11-06
Authors: Lawrence Mbuagbaw; Daeria O Lawson; Livia Puljak; David B Allison; Lehana Thabane Journal: BMC Med Res Methodol Date: 2020-09-07 Impact factor: 4.615