The Deep Learning-Based Recommender System "Pubmender" for Choosing a Biomedical Publication Venue: Development and Validation Study.

BACKGROUND: It is of great importance for researchers to publish research results in high-quality journals. However, it is often challenging to choose the most suitable publication venue, given the exponential growth of journals and conferences. Although recommender systems have achieved success in promoting movies, music, and products, very few studies have explored recommendation of publication venues, especially for biomedical research. No recommender system exists that can specifically recommend journals in PubMed, the largest collection of biomedical literature.
OBJECTIVE: We aimed to propose a publication recommender system, named Pubmender, to suggest suitable PubMed journals based on a paper's abstract.
METHODS: In Pubmender, pretrained word2vec was first used to construct the start-up feature space. Subsequently, a deep convolutional neural network was constructed to achieve a high-level representation of abstracts, and a fully connected softmax model was adopted to recommend the best journals.
RESULTS: We collected 880,165 papers from 1130 journals in PubMed Central and extracted abstracts from these papers as an empirical dataset. We compared different recommendation models such as Cavnar-Trenkle on the Microsoft Academic Search (MAS) engine, a collaborative filtering-based recommender system for the digital library of the Association for Computing Machinery (ACM) and CiteSeer. We found the accuracy of our system for the top 10 recommendations to be 87.0%, 22.9%, and 196.0% higher than that of MAS, ACM, and CiteSeer, respectively. In addition, we compared our system with Journal Finder and Journal Suggester, which are tools of Elsevier and Springer, respectively, that help authors find suitable journals in their series. The results revealed that the accuracy of our system was 329% higher than that of Journal Finder and 406% higher than that of Journal Suggester for the top 10 recommendations. Our web service is freely available at https://www.keaml.cn:8081/.
CONCLUSIONS: Our deep learning-based recommender system can suggest an appropriate journal list to help biomedical scientists and clinicians choose suitable venues for their papers. ©Xiaoyue Feng, Hao Zhang, Yijie Ren, Penghui Shang, Yi Zhu, Yanchun Liang, Renchu Guan, Dong Xu. Originally published in the Journal of Medical Internet Research (http://www.jmir.org), 24.05.2019.

Introduction

Background

With the fast-growing research activities, more biomedical papers are being published in thousands of journals worldwide. For example, PubMed Central (PMC) has 5.2 million papers and 7409 journals covering biomedical and life sciences [1]. Although these publications play a major role in disseminating research outcome, the growth of journal publications imposes a challenge for selections of appropriate publication venues. It is vital that authors submit to the right journal that meets the journal scope and provides sound reviews. It is equally important that they reach their intended audience and obtain a large number of citations [2]. However, researchers are unfamiliar with all the journals related to their work for choosing the most suitable one for submitting a paper. Moreover, different publication scopes of journals and research interests of reviewers and editors may affect the decision of a submitted manuscript. If the submitted paper cannot meet the interests of a publication venue and its editors and reviewers, it may lead to rejection, delay, or less readership. An appropriate recommender system can help solve this problem.

Recommender systems have been proven to serve as an effective method for decision making in many areas such as music, movies, and information media choices [3-6]. The well-known techniques of recommender systems are content-based recommendation [7,8], collaborative filtering recommendation [4,9], and hybrid recommendation [6,10]. Content-based recommender systems recommend an item to a user based on a description of the item. Collaborative filtering methods and hybrid methods may outperform the content-based recommendations by applying user data, if available. However, after the user privacy issue of Facebook in 2018 and the introduction of European Union General Data Protection Regulation, user data are no longer easy to obtain. Moreover, in many domains, especially in material recommendation, there are no user data available for collaborative filtering methods at the beginning [11], which is regarded as a cold-start problem. Content-based recommendations do not need any user information and are more suitable for solving these problems [12].

Based on the content-based recommendation strategy, several attempts have been made to create recommender systems for medical applications and scientific literature. Using geotagged mobile search logs, Agarwal et al [13] adopted a Random Forest model to predict medical visits. Using topic, writing style, author information, citation information, abstract, and title as information items, latent Dirichlet allocation [14] and k-nearest neighbor [15] were used to classify the scientific literature for recommendation [2,12,16,17]. Luong et al [18] used the coauthors’ network as advanced information to recommend a publication venue. Beel et al [19] conducted a literature survey on recommender systems, exploring their methods, evaluation measurements, and datasets. For most of these recommender systems, the high-dimensional and sparse matrix computation is a critical problem [20].

Because of the mismatches caused by ambiguity in text comparisons, the content-based recommendation approach may cause a high error rate [21]. Recently, due to the ability of discovering intricate structures and deep semantics in high-dimensional data, deep learning methods have succeeded in many areas and recently been proposed to build recommender systems for both collaborative filtering and content-based approaches. Hinton et al [22] proposed restricted Boltzmann machines for modeling tabular or count data as a collaborative filtering model on the Netflix data set. McAuley et al [23] proposed an image-based recommendation, which adopted a deep learning model to extract image features. Van den Oord et al [24] applied a deep convolutional neural network (CNN) to predict latent factors from music audios for music recommendation. Wang et al [11] proposed a collaborative deep learning model to jointly perform deep representation learning for the content information and collaborative filtering of a rating matrix. However, to the best of our knowledge, these deep learning techniques have not been used in any biomedical literature recommender system.

Most current venue recommendation studies focus on computer science and technology, but not on the biomedical field. Biomedical sciences are highly interdisciplinary and often link to engineering, medicine, biology, physics, psychology, etc, thereby serving more journals and more diverse topics than any other field. Hence, the development of a recommender system is more essential and challenging for the biomedical sciences than any other discipline. Furthermore, previous recommender systems were based on shallow machine learning methods and social networks. They were generally keyword-based methods and did not take semantics into account. In addition, the few existing systems only focus on journals under a certain organization, such as Elsevier, IEEE, and Springer, instead of PubMed.

Aim

In contrast to our previous study on computer science publication recommendations using conventional machine learning approaches [12], we proposed a deep learning–based recommender system for biomedical publication venues, named Pubmender. Due to the copious vocabulary of biomedical literature, the traditional vector space model can lead to high-dimensional and sparse problems. To address this issue, dimensionality reduction methods are needed before learning the pattern. Moreover, initializing text matrix by pretrained word embedding is more beneficial for training neural networks than random initialized embedding [25]. Accordingly, we applied a word2vec model for our study instead of using the conventional vector space model employed in our previous publication venue recommender system. In addition, deep learning models are able to learn multiple-level abstract representations of data with syntactic and semantic information, since more abstract concepts can be constructed with multiple processing layers [26]. We applied the deep learning approach to provide recommendations of journals for biomedical researchers. Unlike shallow learning, the state-of-the-art embedding method and deep CNN in Pubmender were trained from 837,882 papers in 1130 biomedical journals. This method can help researchers find a variety of choices, without being limited to their own knowledge of journals.

Methods

Pubmender System

Figure 1 shows the architecture and workflow of our Pubmender system. It consists of user interface, data preprocessing, abstract representation, classification, and ranking phase.

Figure 1

Architecture of our Pubmender system. CNN: convolutional neural network; ISSN: International Standard Serial Number.

The user interface obtains the input data (an abstract submitted by a user) and presents the recommendation results to the user. The data acquirement is followed by data preprocessing and information extraction. At the start of our deep learning model, the abstract representation phase converts an abstract to a vector. The original abstract vector is a concatenation of pretrained word vectors. Subsequently, deep CNN is applied to train the model to achieve high-level abstract representation. A three-layer fully connected network with a softmax operation is applied to classify papers based on the obtained abstract vectors. The recommendation list of the top N journals obtained from the ranking phase is presented to the user.

Data Preprocessing Methods

The data were downloaded from the File Transfer Protocol service of PubMed Central (PMC) [27], containing 1,534,649 papers. Based on the journal list of PMC, we selected normal journals deposited under full participation or the US National Institutes of Health portfolio mode, excluding records labeled “Predecessor,” “No New Content,” and “Now Select.” Papers from Jan 2007 to Apr 2017 were selected. Papers with no abstracts or with fewer than 200 characters in abstracts were deleted. Journals containing fewer than 100 papers were also removed. Finally, 880,165 papers in the XML format from 1130 journals were used in our study.

Each PMC file is a semistructured XML document and contains various tags, such as