Dina Demner-Fushman1, Yassine Mrabet1, Asma Ben Abacha1. 1. Lister Hill National Center for Biomedical Communications, National Library of Medicine, National Institutes of Health, Bethesda, Maryland, USA.
Abstract
OBJECTIVE: Consumers increasingly turn to the internet in search of health-related information; and they want their questions answered with short and precise passages, rather than needing to analyze lists of relevant documents returned by search engines and reading each document to find an answer. We aim to answer consumer health questions with information from reliable sources. MATERIALS AND METHODS: We combine knowledge-based, traditional machine and deep learning approaches to understand consumers' questions and select the best answers from consumer-oriented sources. We evaluate the end-to-end system and its components on simple questions generated in a pilot development of MedlinePlus Alexa skill, as well as the short and long real-life questions submitted to the National Library of Medicine by consumers. RESULTS: Our system achieves 78.7% mean average precision and 87.9% mean reciprocal rank on simple Alexa questions, and 44.5% mean average precision and 51.6% mean reciprocal rank on real-life questions submitted by National Library of Medicine consumers. DISCUSSION: The ensemble of deep learning, domain knowledge, and traditional approaches recognizes question type and focus well in the simple questions, but it leaves room for improvement on the real-life consumers' questions. Information retrieval approaches alone are sufficient for finding answers to simple Alexa questions. Answering real-life questions, however, benefits from a combination of information retrieval and inference approaches. CONCLUSION: A pilot practical implementation of research needed to help consumers find reliable answers to their health-related questions demonstrates that for most questions the reliable answers exist and can be found automatically with acceptable accuracy. Published by Oxford University Press on behalf of the American Medical Informatics Association 2019. This work is written by US Government employees and is in the public domain in the US.
OBJECTIVE: Consumers increasingly turn to the internet in search of health-related information; and they want their questions answered with short and precise passages, rather than needing to analyze lists of relevant documents returned by search engines and reading each document to find an answer. We aim to answer consumer health questions with information from reliable sources. MATERIALS AND METHODS: We combine knowledge-based, traditional machine and deep learning approaches to understand consumers' questions and select the best answers from consumer-oriented sources. We evaluate the end-to-end system and its components on simple questions generated in a pilot development of MedlinePlus Alexa skill, as well as the short and long real-life questions submitted to the National Library of Medicine by consumers. RESULTS: Our system achieves 78.7% mean average precision and 87.9% mean reciprocal rank on simple Alexa questions, and 44.5% mean average precision and 51.6% mean reciprocal rank on real-life questions submitted by National Library of Medicine consumers. DISCUSSION: The ensemble of deep learning, domain knowledge, and traditional approaches recognizes question type and focus well in the simple questions, but it leaves room for improvement on the real-life consumers' questions. Information retrieval approaches alone are sufficient for finding answers to simple Alexa questions. Answering real-life questions, however, benefits from a combination of information retrieval and inference approaches. CONCLUSION: A pilot practical implementation of research needed to help consumers find reliable answers to their health-related questions demonstrates that for most questions the reliable answers exist and can be found automatically with acceptable accuracy. Published by Oxford University Press on behalf of the American Medical Informatics Association 2019. This work is written by US Government employees and is in the public domain in the US.
Entities:
Keywords:
artificial intelligence; consumer health questions; deep learning; natural language processing; question answering
Authors: Asma Ben Abacha; Yassine Mrabet; Mark Sharp; Travis R Goodwin; Sonya E Shooshan; Dina Demner-Fushman Journal: Stud Health Technol Inform Date: 2019-08-21
Authors: Halil Kilicoglu; Asma Ben Abacha; Yassine Mrabet; Sonya E Shooshan; Laritza Rodriguez; Kate Masterton; Dina Demner-Fushman Journal: BMC Bioinformatics Date: 2018-02-06 Impact factor: 3.169
Authors: Colleen E Crangle; Colin Bradley; Paul F Carlin; Robert J Esterhay; Roy Harper; Patricia M Kearney; Vera J C McCarthy; Michael F McTear; Eileen Savage; Mark S Tuttle; Jonathan G Wallace Journal: PLoS One Date: 2018-11-16 Impact factor: 3.240
Authors: Travis R Goodwin; Dina Demner-Fushman; Kyle Lo; Lucy Lu Wang; Hoa T Dang; Ian M Soboroff Journal: Sci Data Date: 2022-07-21 Impact factor: 8.501