Gezheng Xu1,2, Wenge Rong3,4, Yanmeng Wang5, Yuanxin Ouyang3,4, Zhang Xiong3,4. 1. State Key Laboratory of Software Development Environment, Beihang University, No.37 Xueyuan Road, Beijing, 100191, China. xugezheng@buaa.edu.cn. 2. Sino-French Engineer School, Beihang University, No.37 Xueyuan Road, Beijing, 100191, China. xugezheng@buaa.edu.cn. 3. State Key Laboratory of Software Development Environment, Beihang University, No.37 Xueyuan Road, Beijing, 100191, China. 4. School of Computer Science and Engineering, Beihang University, No.37 Xueyuan Road, Beijing, 100191, China. 5. Ping An Technology, Xinyuannanlu No.3, Beijing, 100027, China.
Abstract
BACKGROUND: Biomedical question answering (QA) is a sub-task of natural language processing in a specific domain, which aims to answer a question in the biomedical field based on one or more related passages and can provide people with accurate healthcare-related information. Recently, a lot of approaches based on the neural network and large scale pre-trained language model have largely improved its performance. However, considering the lexical characteristics of biomedical corpus and its small scale dataset, there is still much improvement room for biomedical QA tasks. RESULTS: Inspired by the importance of syntactic and lexical features in the biomedical corpus, we proposed a new framework to extract external features, such as part-of-speech and named-entity recognition, and fused them with the original text representation encoded by pre-trained language model, to enhance the biomedical question answering performance. Our model achieves an overall improvement of all three metrics on BioASQ 6b, 7b, and 8b factoid question answering tasks. CONCLUSIONS: The experiments on BioASQ question answering dataset demonstrated the effectiveness of our external feature-enriched framework. It is proven by the experiments conducted that external lexical and syntactic features can improve Pre-trained Language Model's performance in biomedical domain question answering task.
BACKGROUND: Biomedical question answering (QA) is a sub-task of natural language processing in a specific domain, which aims to answer a question in the biomedical field based on one or more related passages and can provide people with accurate healthcare-related information. Recently, a lot of approaches based on the neural network and large scale pre-trained language model have largely improved its performance. However, considering the lexical characteristics of biomedical corpus and its small scale dataset, there is still much improvement room for biomedical QA tasks. RESULTS: Inspired by the importance of syntactic and lexical features in the biomedical corpus, we proposed a new framework to extract external features, such as part-of-speech and named-entity recognition, and fused them with the original text representation encoded by pre-trained language model, to enhance the biomedical question answering performance. Our model achieves an overall improvement of all three metrics on BioASQ 6b, 7b, and 8b factoid question answering tasks. CONCLUSIONS: The experiments on BioASQ question answering dataset demonstrated the effectiveness of our external feature-enriched framework. It is proven by the experiments conducted that external lexical and syntactic features can improve Pre-trained Language Model's performance in biomedical domain question answering task.
Entities:
Keywords:
Biomedical question answering; Feature fusion; NER; POS; Pre-trained language model