Literature DB >> 35659292

Exploiting document graphs for inter sentence relation extraction.

Hoang-Quynh Le1, Duy-Cat Can2, Nigel Collier3.   

Abstract

BACKGROUND: Most previous relation extraction (RE) studies have focused on intra sentence relations and have ignored relations that span sentences, i.e. inter sentence relations. Such relations connect entities at the document level rather than as relational facts in a single sentence. Extracting facts that are expressed across sentences leads to some challenges and requires different approaches than those usually applied in recent intra sentence relation extraction. Despite recent results, there are still limitations to be overcome.
RESULTS: We present a novel representation for a sequence of consecutive sentences, namely document subgraph, to extract inter sentence relations. Experiments on the BioCreative V Chemical-Disease Relation corpus demonstrate the advantages and robustness of our novel system to extract both intra- and inter sentence relations in biomedical literature abstracts. The experimental results are comparable to state-of-the-art approaches and show the potential by demonstrating the effectiveness of graphs, deep learning-based model, and other processing techniques. Experiments were also carried out to verify the rationality and impact of various additional information and model components.
CONCLUSIONS: Our proposed graph-based representation helps to extract ∼50% of inter sentence relations and boosts the model performance on both precision and recall compared to the baseline model.
© 2022. The Author(s).

Entities:  

Keywords:  Convolutional neural network; Deep learning; Graph; Multiple paths; Relation extraction

Mesh：

Year:  2022        PMID： 35659292      PMCID： PMC9166375          DOI： 10.1186/s13326-022-00267-3

Source DB:  PubMed          Journal:  J Biomed Semantics


Background

Relation extraction (RE) is the task of discovering semantic connections between entities [1]. RE plays a vital intermediate step in a variety of natural language processing (NLP) and information extraction applications in the biomedical domain. Its applications range from precision medicine [2], adverse drug reactions identification [3, 4], drug abuse events extraction [5], major life events extraction [6, 7] to building question answering systems [8, 9] and clinical decision support system [10]. Most previous RE studies followed the assumption that if two entities were related, they would belong to a single sentence and therefore ignored relationships expressed across sentence boundaries [11-15]. I.e., the task of RE aims to classify the semantic relationship between an entity pair e1 and e2 in a given sentence S into a pre-defined relation class including ‘not-relate’. However, relationships between entities are often expressed across sentence boundaries or otherwise require a broader context to disambiguate [16-18]. For example, 30% of relations in the Biocreative V Chemical-Disease Relation (BC5 CDR) dataset [19] are only expressed across sentence boundaries, such as in the following excerpt expressing complicated inter sentence relations. “ Case report: acute unintentional carbachol intoxication.</span> <button onclick="translate_abc(this)" style="border:none;outline:none;color:#5577AA;font-size:10px;margin-bottom:0px;" title="Translate into Chinese"> <span class="glyphicon glyphicon-transfer"></span> </button></pxy> <pxy><span>…</span> <button onclick="translate_abc(this)" style="border:none;outline:none;color:#5577AA;font-size:10px;margin-bottom:0px;" title="Translate into Chinese"> <span class="glyphicon glyphicon-transfer"></span> </button></pxy> <pxy><span>< S1> Carbachol concentrations in serum and urine on day 1 and 2 of hospital admission were analysed by HPLC-mass spectrometry.</span> <button onclick="translate_abc(this)" style="border:none;outline:none;color:#5577AA;font-size:10px;margin-bottom:0px;" title="Translate into Chinese"> <span class="glyphicon glyphicon-transfer"></span> </button></pxy> <pxy><span>< S2> RESULTS: Minutes after oral administration, the patient developed , and , and finally collapsed.</span> <button onclick="translate_abc(this)" style="border:none;outline:none;color:#5577AA;font-size:10px;margin-bottom:0px;" title="Translate into Chinese"> <span class="glyphicon glyphicon-transfer"></span> </button></pxy> <pxy><span>< S3> , cholinergic symptoms and occurred. …”</span> <button onclick="translate_abc(this)" style="border:none;outline:none;color:#5577AA;font-size:10px;margin-bottom:0px;" title="Translate into Chinese"> <span class="glyphicon glyphicon-transfer"></span> </button></pxy> <pxy><span>(PMID: 16740173)</span> <button onclick="translate_abc(this)" style="border:none;outline:none;color:#5577AA;font-size:10px;margin-bottom:0px;" title="Translate into Chinese"> <span class="glyphicon glyphicon-transfer"></span> </button></pxy> <pxy><span>In which, chemical ‘carbachol’ is annotated to the Chemical-induced Disease (CID) relations with four diseases ‘nausea’, ‘hypotension’, ‘bradycardia’ and ‘asystole’. All of them are inter sentence relations: ‘carbachol’ only appears in the title and Sentence 1 while ‘nausea’ and ‘hypotension’ appear in Sentence 2 and ‘bradycardia’ and ‘asystole’ only appear in Sentence 3. These problems are exacerbated by the document- (rather than sentence-) level annotation, which is very common in the biological text [<a title="" data-container="body" data-toggle="popover" data-placement="right" data-html="true" data-trigger="hover click" data-content="17. Verga P, Strubell E, McCallum A. Simultaneously self-attending to all mentions for full-abstract biological relation extraction. <i>Proceedings of the Annual Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies (NAACL HLT)</i>. 2018 <a target='_blank' style='cursor:pointer;' href='si.php?db=pubmed&id='><span class='glyphicon glyphicon-share-alt'></span></a>">17</a>].</span> <button onclick="translate_abc(this)" style="border:none;outline:none;color:#5577AA;font-size:10px;margin-bottom:0px;" title="Translate into Chinese"> <span class="glyphicon glyphicon-transfer"></span> </button></pxy> <pxy><span>Thus, the research community has gained an interest in devising methods to move beyond single sentences and extract semantic relations that span sentences. I.e., the task of inter sentence RE aims to identify the semantic relationship between a pair of entity mentions e1 and e2 in a given document D that contains several sentences S1,S2,...S. The extraction of inter sentence relations is much more difficult than intra sentence relations [<a title="" data-container="body" data-toggle="popover" data-placement="right" data-html="true" data-trigger="hover click" data-content="20. Zheng W, Lin H, Li Z, Liu X, Li Z, Xu B, Zhang Y, Yang Z, Wang J. An effective neural model extracting document level chemical-induced disease relations from biomedical literature. <i>J Biomed Inform</i>. 2018 <a target='_blank' style='cursor:pointer;' href='si.php?db=pubmed&id=29746916'><span class='glyphicon glyphicon-share-alt'></span></a>">20</a>]. In some datasets, the involved entities of an inter sentence relation are marked in specific locations (example includes BB3 corpus [<a title="" data-container="body" data-toggle="popover" data-placement="right" data-html="true" data-trigger="hover click" data-content="21. Deléger L, Bossy R, Chaix E, Ba M, Ferré A, Bessieres P, Nédellec C. Overview of the bacteria biotope task at bionlp shared task 2016. <i>Proceedings of the 4th BioNLP Shared Task Workshop</i>. 2016 <a target='_blank' style='cursor:pointer;' href='si.php?db=pubmed&id='><span class='glyphicon glyphicon-share-alt'></span></a>">21</a>]). DocRed dataset [<a title="" data-container="body" data-toggle="popover" data-placement="right" data-html="true" data-trigger="hover click" data-content="22. Yao Y, Ye D, Li P, Han X, Lin Y, Liu Z, Liu Z, Huang L, Zhou J, Sun M. Docred: A large-scale document-level relation extraction dataset. <i>Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics</i>. 2019 <a target='_blank' style='cursor:pointer;' href='si.php?db=pubmed&id='><span class='glyphicon glyphicon-share-alt'></span></a>">22</a>] annotates the relations and entities together with their corresponding supporting sentences. The inter sentence relation extraction problem becomes much more difficult in the datasets that a relation explores entities at the document level rather than that at the specific mentions. I.e., since several mentions of an entity appear in different locations in the text, we face the difficulty in locating which sentences containing the supporting evidence of a relation. This problem becomes more severe in the biomedical domain since biomedical documents often contain sentences with a long and more complex structure compared with that in the general domain. Moreover, many relations are expressed implicitly. When working with multiple sentences, extracting valuable information, and then understanding the contexts of entity pairs becomes much more difficult. There is a multitude of different relation types in the biomedical domain and potentially any pair of entities in the document could be related. For example, although BC5 CDR corpus is only annotated with CID relations, many pairs of entities can have therapeutic relations.</span> <button onclick="translate_abc(this)" style="border:none;outline:none;color:#5577AA;font-size:10px;margin-bottom:0px;" title="Translate into Chinese"> <span class="glyphicon glyphicon-transfer"></span> </button></pxy> <pxy><span>These characteristics lead to some challenges and require different approaches than those usually applied in intra sentence relation extraction. Despite some initial results, there are still limitations of recent approaches for inter sentence RE. The end-to-end model proposed in [<a title="" data-container="body" data-toggle="popover" data-placement="right" data-html="true" data-trigger="hover click" data-content="23. Le H-Q, Tran M-V, Dang TH, Ha Q-T, Collier N. Sieve-based coreference resolution enhances semi-supervised learning model for chemical-induced disease relation extraction. <i>Database</i>. 2016 <a target='_blank' style='cursor:pointer;' href='si.php?db=pubmed&id='><span class='glyphicon glyphicon-share-alt'></span></a>">23</a>] resolved intra sentence relation classification partly by using a multi-pass sieve coreference resolution module. It has the drawback of strongly depending on the appearances of antecedent and anaphor representations of entities in the text since there are many inter sentence relations not expressed through anaphor. Another approach processes consecutive sentences as longer sentences. Examples include a Support Vector Machine (SVM)-based model with a very rich feature set [<a title="" data-container="body" data-toggle="popover" data-placement="right" data-html="true" data-trigger="hover click" data-content="24. Peng Y, Wei C-H, Lu Z. Improving chemical disease relation extraction with rich features and weakly labeled data. <i>J Cheminformatics</i>. 2016 <a target='_blank' style='cursor:pointer;' href='si.php?db=pubmed&id='><span class='glyphicon glyphicon-share-alt'></span></a>">24</a>], a hybrid model of the convolutional neural network, and maximum entropy (ME) [<a title="" data-container="body" data-toggle="popover" data-placement="right" data-html="true" data-trigger="hover click" data-content="25. Gu J, Sun F, Qian L, Zhou G. Chemical-induced disease relation extraction via convolutional neural network. <i>Database (Oxford)</i>. 2017 <a target='_blank' style='cursor:pointer;' href='si.php?db=pubmed&id='><span class='glyphicon glyphicon-share-alt'></span></a>">25</a>] and a long short-term memory network (LSTM) and convolutional neural network model that learns document-level semantic representations [<a title="" data-container="body" data-toggle="popover" data-placement="right" data-html="true" data-trigger="hover click" data-content="20. Zheng W, Lin H, Li Z, Liu X, Li Z, Xu B, Zhang Y, Yang Z, Wang J. An effective neural model extracting document level chemical-induced disease relations from biomedical literature. <i>J Biomed Inform</i>. 2018 <a target='_blank' style='cursor:pointer;' href='si.php?db=pubmed&id=29746916'><span class='glyphicon glyphicon-share-alt'></span></a>">20</a>]. Since inter sentence RE requires information from all local, non-local, syntactic, and semantic dependencies, several previous studies tried to build a representation for the whole document such as biaffine Relation Attention Networks (BRANs) [<a title="" data-container="body" data-toggle="popover" data-placement="right" data-html="true" data-trigger="hover click" data-content="17. Verga P, Strubell E, McCallum A. Simultaneously self-attending to all mentions for full-abstract biological relation extraction. <i>Proceedings of the Annual Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies (NAACL HLT)</i>. 2018 <a target='_blank' style='cursor:pointer;' href='si.php?db=pubmed&id='><span class='glyphicon glyphicon-share-alt'></span></a>">17</a>] and the labeled edge graph convolutional neural network model on a document-level graph [<a title="" data-container="body" data-toggle="popover" data-placement="right" data-html="true" data-trigger="hover click" data-content="18. Sahu SK, Christopoulou F, Miwa M, Ananiadou S. Inter-sentence relation extraction with document-level graph convolutional neural network. <i>Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics</i>. 2019 <a target='_blank' style='cursor:pointer;' href='si.php?db=pubmed&id='><span class='glyphicon glyphicon-share-alt'></span></a>">18</a>].</span> <button onclick="translate_abc(this)" style="border:none;outline:none;color:#5577AA;font-size:10px;margin-bottom:0px;" title="Translate into Chinese"> <span class="glyphicon glyphicon-transfer"></span> </button></pxy> <pxy><span>The novel approach we present in this paper draws inspiration from related works that explore the consecutive sentences for the inter sentence relation extraction. The construction of document subgraphs is also used to leverage both local and non-local information effectively. We then construct a deep neural architecture based on a shared-weight convolutional neural network (swCNN) with an improved attention mechanism to explore the information of multiple paths on the document subgraph. The experimental results on the BC5 CDR benchmark dataset show potential and are comparable to state-of-the-art approaches. The investigation of the impact of different components and information on the final performance provides insights showing that the graph-based representation, swCNN model, instance merging/weighting technique and distant supervision learning are useful. It also leads us to conclude that the knowledge-based information, coreference information and attention mechanism are still promising areas for future research.</span> <button onclick="translate_abc(this)" style="border:none;outline:none;color:#5577AA;font-size:10px;margin-bottom:0px;" title="Translate into Chinese"> <span class="glyphicon glyphicon-transfer"></span> </button></pxy> <h1>Materials and methods</h1> <pxy><span>We present this section in four main parts: the overview of our evaluated dataset; the overall picture of the proposed architecture and three main components in detail; additional techniques to improve model performance; and experimental configuration.</span> <button onclick="translate_abc(this)" style="border:none;outline:none;color:#5577AA;font-size:10px;margin-bottom:0px;" title="Translate into Chinese"> <span class="glyphicon glyphicon-transfer"></span> </button></pxy> <h2><span>Dataset </span> <button onclick="translate_abc(this)" style="border:none;outline:none;color:#5577AA;font-size:10px;margin-bottom:0px;" title="Translate into Chinese"> <span class="glyphicon glyphicon-transfer"></span> </button> </h2> <pxy><span>Our experiments were conducted on the BioCreative V Chemical-Disease Relation dataset [<a title="" data-container="body" data-toggle="popover" data-placement="right" data-html="true" data-trigger="hover click" data-content="19. Wei C-H, Peng Y, Leaman R, Davis AP, Mattingly CJ, Li J, Wiegers TC, Lu Z. Assessing the state of the art in biomedical relation extraction: overview of the biocreative v chemical-disease relation (cdr) task. <i>Database</i>. 2016 <a target='_blank' style='cursor:pointer;' href='si.php?db=pubmed&id='><span class='glyphicon glyphicon-share-alt'></span></a>">19</a>]. This corpus contained a total of 1500 PubMed articles that were separated into three subsets, each of 500 for the training, development and test set (the details are shown in Table 1). This dataset is annotated with chemicals, diseases and the chemical-induced disease relationships at abstract-level. Relation annotations are asserted for both within and across sentence boundaries. Following the data survey of BioCreative [<a title="" data-container="body" data-toggle="popover" data-placement="right" data-html="true" data-trigger="hover click" data-content="26. . . <i></i>. <a target='_blank' style='cursor:pointer;' href='si.php?db=pubmed&id='><span class='glyphicon glyphicon-share-alt'></span></a>">26</a>], about 30% of total instances are inter sentence relationships.</span> <button onclick="translate_abc(this)" style="border:none;outline:none;color:#5577AA;font-size:10px;margin-bottom:0px;" title="Translate into Chinese"> <span class="glyphicon glyphicon-transfer"></span> </button></pxy> <div class="xtable"><div class="fig"><b>Table 1</b><p><span>Summary of the BioCreative V CDR dataset</span> <button onclick="translate_abc(this)" style="border:none;outline:none;color:#5577AA;font-size:10px;margin-bottom:0px;" title="Translate into Chinese"> <span class="glyphicon glyphicon-transfer"></span> </button></p><table xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink" xmlns:ali="http://www.niso.org/schemas/ali/1.0/" frame="hsides" rules="groups"><thead><tr><th align="left">Subset</th><th align="left">Abs</th><th align="left" colspan="3">Disease</th><th align="left" colspan="3">Chemical</th><th align="left">CID</th></tr><tr><th align="left"></th><th align="left"></th><th align="left">Ment</th><th align="left">ID</th><th align="left">IAA</th><th align="left">Ment</th><th align="left">ID</th><th align="left">IAA</th><th align="left"></th></tr></thead><tbody><tr><td align="left">Training</td><td align="left">500</td><td align="left">4182</td><td align="left">1965</td><td align="left">0.8600</td><td align="left">5203</td><td align="left">1467</td><td align="left">0.9523</td><td align="left">1038</td></tr><tr><td align="left">Development</td><td align="left">500</td><td align="left">4244</td><td align="left">1865</td><td align="left">0.8742</td><td align="left">5347</td><td align="left">1507</td><td align="left">0.9577</td><td align="left">1012</td></tr><tr><td align="left">Test</td><td align="left">500</td><td align="left">4424</td><td align="left">1988</td><td align="left">0.8875</td><td align="left">5385</td><td align="left">1435</td><td align="left">0.9630</td><td align="left">1066</td></tr></tbody></table><p><span>Abs Abstracts, Ment Mentions, CID Chemical-induced disease relations</span> <button onclick="translate_abc(this)" style="border:none;outline:none;color:#5577AA;font-size:10px;margin-bottom:0px;" title="Translate into Chinese"> <span class="glyphicon glyphicon-transfer"></span> </button></p></div></div><pxy><span>Summary of the BioCreative V CDR dataset</span> <button onclick="translate_abc(this)" style="border:none;outline:none;color:#5577AA;font-size:10px;margin-bottom:0px;" title="Translate into Chinese"> <span class="glyphicon glyphicon-transfer"></span> </button></pxy> <pxy><span>Abs Abstracts, Ment Mentions, CID Chemical-induced disease relations</span> <button onclick="translate_abc(this)" style="border:none;outline:none;color:#5577AA;font-size:10px;margin-bottom:0px;" title="Translate into Chinese"> <span class="glyphicon glyphicon-transfer"></span> </button></pxy> <h2><span>Model overview </span> <button onclick="translate_abc(this)" style="border:none;outline:none;color:#5577AA;font-size:10px;margin-bottom:0px;" title="Translate into Chinese"> <span class="glyphicon glyphicon-transfer"></span> </button> </h2> <pxy><span>Figure 1 illustrates our proposed model for extracting the semantic relation at the abstract level, which contain four main phases: (i) Firstly, we construct a document subgraph to represent the relationship between entity pairs. (ii) In order to represent an instance by a set of paths, we apply several advanced techniques for finding, merging and choosing the relevant paths between entity pairs. (iii) In the next step, the advanced attention mechanism and several types of linguistic information are applied to explore the information from the document subgraphs more effectively. (iv) Lastly, to exploit these enriched representations effectively, we develop a shared weight Convolutional Neural Network model (swCNN).</span> <button onclick="translate_abc(this)" style="border:none;outline:none;color:#5577AA;font-size:10px;margin-bottom:0px;" title="Translate into Chinese"> <span class="glyphicon glyphicon-transfer"></span> </button></pxy> <div class="fig"><img src="https://www.ncbi.nlm.nih.gov/pmc/articles/PMC9166375/bin/13326_2022_267_Fig1_HTML.jpg" style="width:99%;" /><b>Fig. 1</b><p><span>Proposed model for inter sentence relation classification. Red dotted and striped nodes indicate two types of disease. Blue filled nodes indicate one type of chemical</span> <button onclick="translate_abc(this)" style="border:none;outline:none;color:#5577AA;font-size:10px;margin-bottom:0px;" title="Translate into Chinese"> <span class="glyphicon glyphicon-transfer"></span> </button></p></div><pxy><span>Proposed model for inter sentence relation classification. Red dotted and striped nodes indicate two types of disease. Blue filled nodes indicate one type of chemical</span> <button onclick="translate_abc(this)" style="border:none;outline:none;color:#5577AA;font-size:10px;margin-bottom:0px;" title="Translate into Chinese"> <span class="glyphicon glyphicon-transfer"></span> </button></pxy> <h2><span>Document subgraph construction </span> <button onclick="translate_abc(this)" style="border:none;outline:none;color:#5577AA;font-size:10px;margin-bottom:0px;" title="Translate into Chinese"> <span class="glyphicon glyphicon-transfer"></span> </button> </h2> <pxy><span>As we noted above two entities that participate in a relation may belong to different sentences. Dependency trees are often used to extract local dependencies of semantic relations in intra sentence relation extraction. However, such dependencies are not adequate for inter sentence RE since sentences have different dependency trees that are not connected. Because of this limitation, using the shortest dependency path to extract the local dependencies of semantic relations is not adequate for inter sentence RE.</span> <button onclick="translate_abc(this)" style="border:none;outline:none;color:#5577AA;font-size:10px;margin-bottom:0px;" title="Translate into Chinese"> <span class="glyphicon glyphicon-transfer"></span> </button></pxy> <pxy><span>To overcome these limitations, we construct a graph for consecutive sentences based on their dependency trees, called the . In this graph, the nodes correspond to words and edges represent the connection between them. We make two assumptions: (i) the distance of two participating entities in a relation should not be too far (experimentally, two entities should be within five consecutive sentences). If two entities are too far apart, the method’s effectiveness would be reduced, or this pair may be ignored. (ii) The title of the abstract is a special sentence that is related to every sentence in the abstract in a certain manner. Because of this assumption, the title is always used together with the abstract sentences to generate each subgraph.</span> <button onclick="translate_abc(this)" style="border:none;outline:none;color:#5577AA;font-size:10px;margin-bottom:0px;" title="Translate into Chinese"> <span class="glyphicon glyphicon-transfer"></span> </button></pxy> <pxy><span>Creating a document subgraph is a three-step process:</span> <button onclick="translate_abc(this)" style="border:none;outline:none;color:#5577AA;font-size:10px;margin-bottom:0px;" title="Translate into Chinese"> <span class="glyphicon glyphicon-transfer"></span> </button></pxy> <pxy><span>Generate the dependency tree for each sentence. All directed dependency labels are kept in the subgraphs as local dependency information.</span> <button onclick="translate_abc(this)" style="border:none;outline:none;color:#5577AA;font-size:10px;margin-bottom:0px;" title="Translate into Chinese"> <span class="glyphicon glyphicon-transfer"></span> </button></pxy> <pxy><span>Merge the dependency trees of the sentences in each sliding window into a document subgraph.</span> <button onclick="translate_abc(this)" style="border:none;outline:none;color:#5577AA;font-size:10px;margin-bottom:0px;" title="Translate into Chinese"> <span class="glyphicon glyphicon-transfer"></span> </button></pxy> <pxy><span>The sliding window of size w indicates the number of consecutive sentences that we use to create the document subgraphs. w=1 indicates a single sentence, i.e. the model only extracts the intra sentence relations. With w=j, each j sentences are used to create a subgraph. Since two entity mentions can appear in different sentences, an unrestricted selection of text spans would risk generating many unexpected examples and lead to an explosion of computing space (see Instance merging.) We, therefore, limit w to 5, i.e., all relations with two entities that are not within 5 consecutive sentences are ignored. After this phase, each abstract will consist of several subgraphs.</span> <button onclick="translate_abc(this)" style="border:none;outline:none;color:#5577AA;font-size:10px;margin-bottom:0px;" title="Translate into Chinese"> <span class="glyphicon glyphicon-transfer"></span> </button></pxy> <pxy><span>Create virtual edges for subgraphs. By using dependency trees, we already have local dependency information. In this step, we try to link new virtual edges by using several additional information:</span> <button onclick="translate_abc(this)" style="border:none;outline:none;color:#5577AA;font-size:10px;margin-bottom:0px;" title="Translate into Chinese"> <span class="glyphicon glyphicon-transfer"></span> </button></pxy> <pxy><span>NEXT-SENT edges connect root nodes in dependency trees of two consecutive sentences. They bring sequential non-local dependency information.</span> <button onclick="translate_abc(this)" style="border:none;outline:none;color:#5577AA;font-size:10px;margin-bottom:0px;" title="Translate into Chinese"> <span class="glyphicon glyphicon-transfer"></span> </button></pxy> <pxy><span>TITLE edges are created between two dependency tree roots of the Title and the first sentence in the sliding window. They provide non-local dependency information.</span> <button onclick="translate_abc(this)" style="border:none;outline:none;color:#5577AA;font-size:10px;margin-bottom:0px;" title="Translate into Chinese"> <span class="glyphicon glyphicon-transfer"></span> </button></pxy> <pxy><span>COREFERENCE edges link an anaphoric expression to its antecedent if identified by the multi-pass sieves coreference resolution method [<a title="" data-container="body" data-toggle="popover" data-placement="right" data-html="true" data-trigger="hover click" data-content="23. Le H-Q, Tran M-V, Dang TH, Ha Q-T, Collier N. Sieve-based coreference resolution enhances semi-supervised learning model for chemical-induced disease relation extraction. <i>Database</i>. 2016 <a target='_blank' style='cursor:pointer;' href='si.php?db=pubmed&id='><span class='glyphicon glyphicon-share-alt'></span></a>">23</a>]. These edges show the semantic relation between terms. We divide this connection type into three specific types: (i) COREF-sent: anaphor and antecedent belong to two normal sentences, (ii) COREF-to-title: anaphor is in a normal sentence and antecedent is in the Title, (iii) COREF-from-title: anaphor is in the Title and antecedent is in a normal sentence.</span> <button onclick="translate_abc(this)" style="border:none;outline:none;color:#5577AA;font-size:10px;margin-bottom:0px;" title="Translate into Chinese"> <span class="glyphicon glyphicon-transfer"></span> </button></pxy> <pxy><span>KB-CTD edges are created between head nodes of two entities if they are annotated as having relation ‘M’ in the Comparative Toxicogenomics Database (CTD)1. We call it knowledge-based information.</span> <button onclick="translate_abc(this)" style="border:none;outline:none;color:#5577AA;font-size:10px;margin-bottom:0px;" title="Translate into Chinese"> <span class="glyphicon glyphicon-transfer"></span> </button></pxy> <pxy><span>These virtual edges are undirected and labeled by their names. We give a realistic example of the document subgraph in Additional file 1: Appendix A.</span> <button onclick="translate_abc(this)" style="border:none;outline:none;color:#5577AA;font-size:10px;margin-bottom:0px;" title="Translate into Chinese"> <span class="glyphicon glyphicon-transfer"></span> </button></pxy> <pxy><span>Using the subgraphs already constructed, this module finds all possible paths between two entities in each graph. We perform a breadth-first search on a graph to find all possible paths between two entities. The graph we constructed is quite complex, moreover, the complexity increases with the sliding window size w and the number of new virtual edges. A traversal in breadth-first order on such a large graph with cycles is resource-consuming (even if we never go back to the passed nodes to avoid the infinite issue).</span> <button onclick="translate_abc(this)" style="border:none;outline:none;color:#5577AA;font-size:10px;margin-bottom:0px;" title="Translate into Chinese"> <span class="glyphicon glyphicon-transfer"></span> </button></pxy> <pxy><span>To overcome this risk, we use two thresholds:</span> <button onclick="translate_abc(this)" style="border:none;outline:none;color:#5577AA;font-size:10px;margin-bottom:0px;" title="Translate into Chinese"> <span class="glyphicon glyphicon-transfer"></span> </button></pxy> <pxy><span>Maximum depth md: The maximum number of nodes traveling from the beginning node.</span> <button onclick="translate_abc(this)" style="border:none;outline:none;color:#5577AA;font-size:10px;margin-bottom:0px;" title="Translate into Chinese"> <span class="glyphicon glyphicon-transfer"></span> </button></pxy> <pxy><span>the Maximum number of path k: The maximum number of paths that we collect.</span> <button onclick="translate_abc(this)" style="border:none;outline:none;color:#5577AA;font-size:10px;margin-bottom:0px;" title="Translate into Chinese"> <span class="glyphicon glyphicon-transfer"></span> </button></pxy> <pxy><span>Nearly all previous studies in relation extraction consider co-occurring entity pairs with known relations as positive instances for training. This assumption is reasonable for intra sentence relations, but the inter sentence problem presents a new challenge since this strategy would risk generating too many wrong examples. It is because a document has a relation between two entities does not mean that all spans of text contain these entities show that relation. Quirk and Poon [<a title="" data-container="body" data-toggle="popover" data-placement="right" data-html="true" data-trigger="hover click" data-content="16. Quirk C, Poon H. Distant supervision for relation extraction beyond the sentence boundary. <i>Proceedings of the Fifteenth Conference on European Chapter of the Association for Computational Linguistics (Volume 1, Long Papers)</i>. 2017 <a target='_blank' style='cursor:pointer;' href='si.php?db=pubmed&id='><span class='glyphicon glyphicon-share-alt'></span></a>">16</a>] tackled this problem when an entity pair co-occurs in a large text span, and also co-occur in a smaller text span that overlaps with the larger one. In such cases, if there is a relation between the pair, most likely it is expressed in the smaller text span when the entities are closer to each other. To reduce the unexpected noise from the large text span, we apply a restriction of generating paths called ‘minimal span’ [<a title="" data-container="body" data-toggle="popover" data-placement="right" data-html="true" data-trigger="hover click" data-content="16. Quirk C, Poon H. Distant supervision for relation extraction beyond the sentence boundary. <i>Proceedings of the Fifteenth Conference on European Chapter of the Association for Computational Linguistics (Volume 1, Long Papers)</i>. 2017 <a target='_blank' style='cursor:pointer;' href='si.php?db=pubmed&id='><span class='glyphicon glyphicon-share-alt'></span></a>">16</a>]. I.e., only the minimal span is chosen to generate the paths between two entities. A co-occurring entity pair has the minimal span if there does not exist another overlapping co-occurrence of the same pair. Since each abstract can have several subgraphs, in this phase, we receive several sets of paths.</span> <button onclick="translate_abc(this)" style="border:none;outline:none;color:#5577AA;font-size:10px;margin-bottom:0px;" title="Translate into Chinese"> <span class="glyphicon glyphicon-transfer"></span> </button></pxy> <h3><span>Instance merging </span> <button onclick="translate_abc(this)" style="border:none;outline:none;color:#5577AA;font-size:10px;margin-bottom:0px;" title="Translate into Chinese"> <span class="glyphicon glyphicon-transfer"></span> </button> </h3> <pxy><span>Figure 2 illustrates the instance merging technique. Firstly, we address two unexpected problems while generating the instance from the document subgraph. In Fig. 2-A, a pair of entities appear several times at different positions in an abstract. Because the BC5 CDR corpus has relations annotated at the abstract-level, all of these co-occurrences are treated as positive examples for the CID relation. In fact, only a few of them actually refer to the CID relation. This may cause much noise during training.</span> <button onclick="translate_abc(this)" style="border:none;outline:none;color:#5577AA;font-size:10px;margin-bottom:0px;" title="Translate into Chinese"> <span class="glyphicon glyphicon-transfer"></span> </button></pxy> <div class="fig"><img src="https://www.ncbi.nlm.nih.gov/pmc/articles/PMC9166375/bin/13326_2022_267_Fig2_HTML.jpg" style="width:99%;" /><b>Fig. 2</b><p><span>Examples of two unexpected problems while generating the instance from document subgraph</span> <button onclick="translate_abc(this)" style="border:none;outline:none;color:#5577AA;font-size:10px;margin-bottom:0px;" title="Translate into Chinese"> <span class="glyphicon glyphicon-transfer"></span> </button></p></div><pxy><span>Examples of two unexpected problems while generating the instance from document subgraph</span> <button onclick="translate_abc(this)" style="border:none;outline:none;color:#5577AA;font-size:10px;margin-bottom:0px;" title="Translate into Chinese"> <span class="glyphicon glyphicon-transfer"></span> </button></pxy> <pxy><span>The example in Fig. 2-B shows the problem of unexpected instance repetition, especially when we widen the window to create subgraphs. In this example, we can generate three identical training instances, i.e., the training patterns of this instance are produced three times, changing the actual frequency of the representation in the training data. This issue may then lead the model to give this instance a higher priority (more important weight). We give a realistic example of these problems below:</span> <button onclick="translate_abc(this)" style="border:none;outline:none;color:#5577AA;font-size:10px;margin-bottom:0px;" title="Translate into Chinese"> <span class="glyphicon glyphicon-transfer"></span> </button></pxy> <pxy><span>“<Title> of human erythrocytes induced by tamoxifen is related to disruption of membrane structure.</span> <button onclick="translate_abc(this)" style="border:none;outline:none;color:#5577AA;font-size:10px;margin-bottom:0px;" title="Translate into Chinese"> <span class="glyphicon glyphicon-transfer"></span> </button></pxy> <pxy><span>…</span> <button onclick="translate_abc(this)" style="border:none;outline:none;color:#5577AA;font-size:10px;margin-bottom:0px;" title="Translate into Chinese"> <span class="glyphicon glyphicon-transfer"></span> </button></pxy> <pxy><span>< S1> TAM induces of erythrocytes as a function of concentration.</span> <button onclick="translate_abc(this)" style="border:none;outline:none;color:#5577AA;font-size:10px;margin-bottom:0px;" title="Translate into Chinese"> <span class="glyphicon glyphicon-transfer"></span> </button></pxy> <pxy><span>< S2> The extension of is variable with erythrocyte samples, but 12.5 microM TAM induces total of all tested suspensions.</span> <button onclick="translate_abc(this)" style="border:none;outline:none;color:#5577AA;font-size:10px;margin-bottom:0px;" title="Translate into Chinese"> <span class="glyphicon glyphicon-transfer"></span> </button></pxy> <pxy><span>< S3> Despite inducing extensive erythrocyte lysis, TAM does not shift the osmotic fragility curves of erythrocytes.</span> <button onclick="translate_abc(this)" style="border:none;outline:none;color:#5577AA;font-size:10px;margin-bottom:0px;" title="Translate into Chinese"> <span class="glyphicon glyphicon-transfer"></span> </button></pxy> <pxy><span>< S4> The hemolytic effect of TAM is prevented by low concentrations of alpha-tocopherol (alpha-T) and alpha-tocopherol acetate (alpha-TAc) (inactivated functional hydroxyl) indicating that TAM-induced is not related to oxidative membrane damage.</span> <button onclick="translate_abc(this)" style="border:none;outline:none;color:#5577AA;font-size:10px;margin-bottom:0px;" title="Translate into Chinese"> <span class="glyphicon glyphicon-transfer"></span> </button></pxy> <pxy><span>< S5> This was further evidenced by absence of oxygen consumption and hemoglobin oxidation both determined in parallel with TAM-induced . …”</span> <button onclick="translate_abc(this)" style="border:none;outline:none;color:#5577AA;font-size:10px;margin-bottom:0px;" title="Translate into Chinese"> <span class="glyphicon glyphicon-transfer"></span> </button></pxy> <pxy><span>(PMID: 10704919)</span> <button onclick="translate_abc(this)" style="border:none;outline:none;color:#5577AA;font-size:10px;margin-bottom:0px;" title="Translate into Chinese"> <span class="glyphicon glyphicon-transfer"></span> </button></pxy> <pxy><span>Tackled with a title and 5 sentences as shown above and a sliding window size w=3, we have 42 valid pairs of CID: TAM-Hemolysis. Each entity pair can potentially be described by up to 15 paths. As a result, if each pair CID: TAM-Hemolysis is considered as a positive instance, we may have too many ‘similar’ positive instances. The same problem also appears for negative instances. To solve this problem, we propose a technique called , in which, we extract all possible dependency paths between a pair of entity mentions and merge them into a single set for this entity pair. To reduce overlapping training instances, we remove the repeated paths (i.e., if several paths are totally identical, only one is kept).</span> <button onclick="translate_abc(this)" style="border:none;outline:none;color:#5577AA;font-size:10px;margin-bottom:0px;" title="Translate into Chinese"> <span class="glyphicon glyphicon-transfer"></span> </button></pxy> <h3><span>Choosing top −k paths </span> <button onclick="translate_abc(this)" style="border:none;outline:none;color:#5577AA;font-size:10px;margin-bottom:0px;" title="Translate into Chinese"> <span class="glyphicon glyphicon-transfer"></span> </button> </h3> <pxy><span>After the instance merging phase, we have a set of several paths to represent a pair of entities. Some of them are useful, but others may be noise.</span> <button onclick="translate_abc(this)" style="border:none;outline:none;color:#5577AA;font-size:10px;margin-bottom:0px;" title="Translate into Chinese"> <span class="glyphicon glyphicon-transfer"></span> </button></pxy> <pxy><span>Prior works on intra sentence relation extraction often explored the single shortest path between two entities [27, 28]. Applying these traditional approaches for inter sentence relation classification problem raises many problems. Firstly, we cannot take advantage of all the local and global features since they may appear in different paths; secondly, the shortest path may not the ‘best’ path.</span> <button onclick="translate_abc(this)" style="border:none;outline:none;color:#5577AA;font-size:10px;margin-bottom:0px;" title="Translate into Chinese"> <span class="glyphicon glyphicon-transfer"></span> </button></pxy> <pxy><span>In contrast to these previous approaches, we propose to consider a as a novel representation for an entity pair. To reduce noise and model complexity, we only choose the top-k best paths. This leads to the problem of how to choose advantageous paths. In this work, we implement two strategies to choose the top-k paths:</span> <button onclick="translate_abc(this)" style="border:none;outline:none;color:#5577AA;font-size:10px;margin-bottom:0px;" title="Translate into Chinese"> <span class="glyphicon glyphicon-transfer"></span> </button></pxy> <pxy><span>Top-k shortest dependency paths, this strategy was also used by [<a title="" data-container="body" data-toggle="popover" data-placement="right" data-html="true" data-trigger="hover click" data-content="16. Quirk C, Poon H. Distant supervision for relation extraction beyond the sentence boundary. <i>Proceedings of the Fifteenth Conference on European Chapter of the Association for Computational Linguistics (Volume 1, Long Papers)</i>. 2017 <a target='_blank' style='cursor:pointer;' href='si.php?db=pubmed&id='><span class='glyphicon glyphicon-share-alt'></span></a>">16</a>].</span> <button onclick="translate_abc(this)" style="border:none;outline:none;color:#5577AA;font-size:10px;margin-bottom:0px;" title="Translate into Chinese"> <span class="glyphicon glyphicon-transfer"></span> </button></pxy> <pxy><span>Top-k paths with the highest number of repetitions.</span> <button onclick="translate_abc(this)" style="border:none;outline:none;color:#5577AA;font-size:10px;margin-bottom:0px;" title="Translate into Chinese"> <span class="glyphicon glyphicon-transfer"></span> </button></pxy> <pxy><span>To explore the information in this novel representation, we cannot use our previous models. Instead, a new deep learning architecture capable of simultaneously processing multiple paths was proposed, based on the swCNN.</span> <button onclick="translate_abc(this)" style="border:none;outline:none;color:#5577AA;font-size:10px;margin-bottom:0px;" title="Translate into Chinese"> <span class="glyphicon glyphicon-transfer"></span> </button></pxy> <h2><span>Path representation </span> <button onclick="translate_abc(this)" style="border:none;outline:none;color:#5577AA;font-size:10px;margin-bottom:0px;" title="Translate into Chinese"> <span class="glyphicon glyphicon-transfer"></span> </button> </h2> <pxy><span>Before inputting to the model, each component in the dependency paths must be transformed into an embedding vector. In order to have an informative representation, we take advantage of various linguistic information along the dependency path, from the original dependency tree and other resources.</span> <button onclick="translate_abc(this)" style="border:none;outline:none;color:#5577AA;font-size:10px;margin-bottom:0px;" title="Translate into Chinese"> <span class="glyphicon glyphicon-transfer"></span> </button></pxy> <pxy><span>The with directions are proven more effective than the dependency relations without directions for the relation extraction task [<a title="" data-container="body" data-toggle="popover" data-placement="right" data-html="true" data-trigger="hover click" data-content="27. Xu K, Feng Y, Huang S, Zhao D. Semantic relation classification via convolutional neural networks with simple negative sampling. <i>Proceedings of the 2015 Conference on Empirical Methods in Natural Language Processing</i>. 2015 <a target='_blank' style='cursor:pointer;' href='si.php?db=pubmed&id='><span class='glyphicon glyphicon-share-alt'></span></a>">27</a>]. However, treating the dependency relations with the opposite direction as two separate relations can induce two vectors for the same relation. We represent the dependency relations with two discrete components: represents the dependency relation type among 72 labels; and is the direction of the dependency relation, i.e. from left-to-right or vice versa on the Shortest Dependency Path (SDP). The final representation d of dependency relation is obtained through a nonlinear transformation as follow:</span> <button onclick="translate_abc(this)" style="border:none;outline:none;color:#5577AA;font-size:10px;margin-bottom:0px;" title="Translate into Chinese"> <span class="glyphicon glyphicon-transfer"></span> </button></pxy> <pxy><span>where the d and d vectors are generated by looking up the embedding matrices and respectively; W and b are trainable parameters of the network.</span> <button onclick="translate_abc(this)" style="border:none;outline:none;color:#5577AA;font-size:10px;margin-bottom:0px;" title="Translate into Chinese"> <span class="glyphicon glyphicon-transfer"></span> </button></pxy> <pxy><span>For token representation, we utilize two types of embeddings to represent the word information in different aspects, including:</span> <button onclick="translate_abc(this)" style="border:none;outline:none;color:#5577AA;font-size:10px;margin-bottom:0px;" title="Translate into Chinese"> <span class="glyphicon glyphicon-transfer"></span> </button></pxy> <pxy><span>Pre-trained fastText embeddings [<a title="" data-container="body" data-toggle="popover" data-placement="right" data-html="true" data-trigger="hover click" data-content="29. Bojanowski P, Grave E, Joulin A, Mikolov T. Enriching word vectors with subword information. <i>Trans Assoc Comput Linguist</i>. 2017 <a target='_blank' style='cursor:pointer;' href='si.php?db=pubmed&id='><span class='glyphicon glyphicon-share-alt'></span></a>">29</a>] learn the word representation based on its external context and n-gram sub-word information. Each token in the input paths is transformed into a vector by looking up the embedding matrix , where dim is the word embedding dimension, and V is the vocabulary of all words we consider.</span> <button onclick="translate_abc(this)" style="border:none;outline:none;color:#5577AA;font-size:10px;margin-bottom:0px;" title="Translate into Chinese"> <span class="glyphicon glyphicon-transfer"></span> </button></pxy> <pxy><span>POS tag embeddings captures (dis)similarities between grammatical properties of words and their syntactic structural roles within a sentence. We concatenate the part-of-speech (POS) tag information into the token representation vector. We randomly initialize the embeddings matrix for 56 OntoNotes 5.0 version of the Penn Treebank POS tags. Each POS tag label is then represented as a corresponding vector .</span> <button onclick="translate_abc(this)" style="border:none;outline:none;color:#5577AA;font-size:10px;margin-bottom:0px;" title="Translate into Chinese"> <span class="glyphicon glyphicon-transfer"></span> </button></pxy> <pxy><span>We concatenate two embedding vectors of each token and transform them into the final token embedding as follow:</span> <button onclick="translate_abc(this)" style="border:none;outline:none;color:#5577AA;font-size:10px;margin-bottom:0px;" title="Translate into Chinese"> <span class="glyphicon glyphicon-transfer"></span> </button></pxy> <pxy><span>Each token t is concatenated with the corresponding attentive augmented information from its child nodes on the original dependency tree proposed by Can et al. [<a title="" data-container="body" data-toggle="popover" data-placement="right" data-html="true" data-trigger="hover click" data-content="30. Can D-C, Le HQ, Ha QT, Collier N. A richer-but-smarter shortest dependency path with attentive augmentation for relation extraction. <i>Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long and Short Papers)</i>. 2019 <a target='_blank' style='cursor:pointer;' href='si.php?db=pubmed&id='><span class='glyphicon glyphicon-share-alt'></span></a>">30</a>]. Given a token t, the attentive augmented information is calculated using the token itself and the set of its M child nodes. Word embedding and POS tag embedding are concatenated to form token embedding vector t while the dependency relation from a direct ancestor is added to form a child node representation c. The position embeddings d is also used to reflect the relative distance from the i-th child to its parent on the original sentence.</span> <button onclick="translate_abc(this)" style="border:none;outline:none;color:#5577AA;font-size:10px;margin-bottom:0px;" title="Translate into Chinese"> <span class="glyphicon glyphicon-transfer"></span> </button></pxy> <pxy><span>Two sequential attention layers on the children of a token are used to produce children context vectors. A simple self-attentive network is applied to child nodes where the attention weights are calculated based on the concatenation of themselves with parent information and distance from the parent. I.e.,</span> <button onclick="translate_abc(this)" style="border:none;outline:none;color:#5577AA;font-size:10px;margin-bottom:0px;" title="Translate into Chinese"> <span class="glyphicon glyphicon-transfer"></span> </button></pxy> <pxy><span>where is the base distance embedding; W and b are weight and bias term.</span> <button onclick="translate_abc(this)" style="border:none;outline:none;color:#5577AA;font-size:10px;margin-bottom:0px;" title="Translate into Chinese"> <span class="glyphicon glyphicon-transfer"></span> </button></pxy> <pxy><span>A distance-based heuristic attentive layer is applied on the self-attentive children context vector to keep track of how close each child is to the target token, as follow:</span> <button onclick="translate_abc(this)" style="border:none;outline:none;color:#5577AA;font-size:10px;margin-bottom:0px;" title="Translate into Chinese"> <span class="glyphicon glyphicon-transfer"></span> </button></pxy> <pxy><span>where f(d)=βd2 with β=−0.03 is a heuristically chosen weighting function.</span> <button onclick="translate_abc(this)" style="border:none;outline:none;color:#5577AA;font-size:10px;margin-bottom:0px;" title="Translate into Chinese"> <span class="glyphicon glyphicon-transfer"></span> </button></pxy> <pxy><span>Afterward, to capture the relevant and essential information from the output of the multi-attention layer and preserve the integrity of the word information, K kernel filters are applied to each child’s attentive vector to produce K features from each child. The final augmented information a is captured by a max-pooling layer, i.e.,</span> <button onclick="translate_abc(this)" style="border:none;outline:none;color:#5577AA;font-size:10px;margin-bottom:0px;" title="Translate into Chinese"> <span class="glyphicon glyphicon-transfer"></span> </button></pxy> <pxy><span>where W is the weight of K kernel filters; and b is bias term.</span> <button onclick="translate_abc(this)" style="border:none;outline:none;color:#5577AA;font-size:10px;margin-bottom:0px;" title="Translate into Chinese"> <span class="glyphicon glyphicon-transfer"></span> </button></pxy> <pxy><span>Finally, this concatenation is transformed into an X-dimensional vector to form the representation of the token, i.e.,</span> <button onclick="translate_abc(this)" style="border:none;outline:none;color:#5577AA;font-size:10px;margin-bottom:0px;" title="Translate into Chinese"> <span class="glyphicon glyphicon-transfer"></span> </button></pxy> <pxy><span>where W and b are trainable parameters of the network.</span> <button onclick="translate_abc(this)" style="border:none;outline:none;color:#5577AA;font-size:10px;margin-bottom:0px;" title="Translate into Chinese"> <span class="glyphicon glyphicon-transfer"></span> </button></pxy> <h2><span>Shared-weight convolutional neural network </span> <button onclick="translate_abc(this)" style="border:none;outline:none;color:#5577AA;font-size:10px;margin-bottom:0px;" title="Translate into Chinese"> <span class="glyphicon glyphicon-transfer"></span> </button> </h2> <pxy><span>Convolutional Neural Networks (CNNs) [<a title="" data-container="body" data-toggle="popover" data-placement="right" data-html="true" data-trigger="hover click" data-content="31. LeCun Y, Boser B, Denker JS, Henderson D, Howard RE, Hubbard W, Jackel LD. Backpropagation applied to handwritten zip code recognition. <i>Neural Comput</i>. 1989 <a target='_blank' style='cursor:pointer;' href='si.php?db=pubmed&id='><span class='glyphicon glyphicon-share-alt'></span></a>">31</a>] are good at capturing the n-gram features in the flat structure and have also been proved effective in many natural language processing tasks including relation classification [14, 17]. The typical structure of a shared-weight CNN (swCNN) is quite similar to the original CNN that is comprising convolution, pooling, fully-connected layers and softmax. The novel point is the ability to share weight between several convolutions, leading to the ability to process multiple data instances at once.</span> <button onclick="translate_abc(this)" style="border:none;outline:none;color:#5577AA;font-size:10px;margin-bottom:0px;" title="Translate into Chinese"> <span class="glyphicon glyphicon-transfer"></span> </button></pxy> <pxy><span>Figure 3 illustrates the overall architecture of our swCNN model, which is comprised of two main components: multi-path representation and classification. Given a set of multiple k paths as input, each path is converted into a separated embedding matrix. A shared-weight convolution with relu activation layer is followed to capture convolved features from these embedding matrices simultaneously. The essential features are gathered using a filter-wise pooling layer before being classified by a fully connected layer with softmax classification.</span> <button onclick="translate_abc(this)" style="border:none;outline:none;color:#5577AA;font-size:10px;margin-bottom:0px;" title="Translate into Chinese"> <span class="glyphicon glyphicon-transfer"></span> </button></pxy> <div class="fig"><img src="https://www.ncbi.nlm.nih.gov/pmc/articles/PMC9166375/bin/13326_2022_267_Fig3_HTML.jpg" style="width:99%;" /><b>Fig. 3</b><p><span>Diagram illustrating of a swCNN architecture</span> <button onclick="translate_abc(this)" style="border:none;outline:none;color:#5577AA;font-size:10px;margin-bottom:0px;" title="Translate into Chinese"> <span class="glyphicon glyphicon-transfer"></span> </button></p></div><pxy><span>Diagram illustrating of a swCNN architecture</span> <button onclick="translate_abc(this)" style="border:none;outline:none;color:#5577AA;font-size:10px;margin-bottom:0px;" title="Translate into Chinese"> <span class="glyphicon glyphicon-transfer"></span> </button></pxy> <pxy><span>In the embeddings layer, each component in the dependency path (i.e., token or dependency relation) is represented by a d-dimensional vector where d is the desired number of embedding dimensions as described in the previous section ‘Path representation’.</span> <button onclick="translate_abc(this)" style="border:none;outline:none;color:#5577AA;font-size:10px;margin-bottom:0px;" title="Translate into Chinese"> <span class="glyphicon glyphicon-transfer"></span> </button></pxy> <pxy><span>After the embeddings layer, the input multiple paths are transformed into:</span> <button onclick="translate_abc(this)" style="border:none;outline:none;color:#5577AA;font-size:10px;margin-bottom:0px;" title="Translate into Chinese"> <span class="glyphicon glyphicon-transfer"></span> </button></pxy> <pxy><span>In general, let us define the vector x as the concatenation of m tokens and m−1 dependency relation between them. I.e.,</span> <button onclick="translate_abc(this)" style="border:none;outline:none;color:#5577AA;font-size:10px;margin-bottom:0px;" title="Translate into Chinese"> <span class="glyphicon glyphicon-transfer"></span> </button></pxy> <pxy><span>In the convolution layer, we apply N filters with region size r to these embedding matrices simultaneously. These filters move by dependency unit to keep the dependency information between tokens. Since the same filters are used for all matrices, our model can extract information from them at the same time, as well as suppress increases in the number of weight parameters then reduce the computational complexity. The filter-wise pooling step converges all outputs of a filter to a single element by choosing the essential feature from all CNN features. This architecture helps swCNN to use the information on multiple paths simultaneously, and from there, selects the truly outstanding features. I.e., the convolutional layer computes an element f of the convolved feature vector f as follows:</span> <button onclick="translate_abc(this)" style="border:none;outline:none;color:#5577AA;font-size:10px;margin-bottom:0px;" title="Translate into Chinese"> <span class="glyphicon glyphicon-transfer"></span> </button></pxy> <pxy><span>where and are the weight matrix and bias vector of the convolutional layer.</span> <button onclick="translate_abc(this)" style="border:none;outline:none;color:#5577AA;font-size:10px;margin-bottom:0px;" title="Translate into Chinese"> <span class="glyphicon glyphicon-transfer"></span> </button></pxy> <pxy><span>At the classification phase, we have the number of features equal to the number of filters we used. They then are flattened into a feature vector and put through the softmax to decide the final prediction. I.e., the output f of the convolutional layer is then fed to a softmax classifier to predict a (K+1)-class distribution over labels :</span> <button onclick="translate_abc(this)" style="border:none;outline:none;color:#5577AA;font-size:10px;margin-bottom:0px;" title="Translate into Chinese"> <span class="glyphicon glyphicon-transfer"></span> </button></pxy> <pxy><span>where W and b are the parameters of the network to be learned.</span> <button onclick="translate_abc(this)" style="border:none;outline:none;color:#5577AA;font-size:10px;margin-bottom:0px;" title="Translate into Chinese"> <span class="glyphicon glyphicon-transfer"></span> </button></pxy> <pxy><span>The proposed model can be stated as a parameter tuple θ=(W,b). To compute the model parameters θ, we define the training objective for a data sample as:</span> <button onclick="translate_abc(this)" style="border:none;outline:none;color:#5577AA;font-size:10px;margin-bottom:0px;" title="Translate into Chinese"> <span class="glyphicon glyphicon-transfer"></span> </button></pxy> <pxy><span>where y∈{0,1}( indicates the one-hot vector represented ground truth; and λ is a regularization coefficient.</span> <button onclick="translate_abc(this)" style="border:none;outline:none;color:#5577AA;font-size:10px;margin-bottom:0px;" title="Translate into Chinese"> <span class="glyphicon glyphicon-transfer"></span> </button></pxy> <h2><span>Additional techniques </span> <button onclick="translate_abc(this)" style="border:none;outline:none;color:#5577AA;font-size:10px;margin-bottom:0px;" title="Translate into Chinese"> <span class="glyphicon glyphicon-transfer"></span> </button> </h2> <h3><span>Ensemble mechanism </span> <button onclick="translate_abc(this)" style="border:none;outline:none;color:#5577AA;font-size:10px;margin-bottom:0px;" title="Translate into Chinese"> <span class="glyphicon glyphicon-transfer"></span> </button> </h3> <pxy><span>Overfitting is one of the most notable problems of deep learning models. It happens when the neural network is very good at learning its training set, but cannot generalize beyond the training set (known as the generalization problem). The ensemble method [<a title="" data-container="body" data-toggle="popover" data-placement="right" data-html="true" data-trigger="hover click" data-content="32. Kowsari K, Jafari Meimandi K, Heidarysafa M, Mendu S, Barnes L, Brown D. Text classification algorithms: A survey. <i>Information</i>. 2019 <a target='_blank' style='cursor:pointer;' href='si.php?db=pubmed&id='><span class='glyphicon glyphicon-share-alt'></span></a>">32</a>] is one of the most effective paradigms to reduce variance and helps to avoid overfitting as well as improve the stability and accuracy of the model. Moreover, random initialization is demonstrated to have an impact on the model’s performance on unseen data, i.e. training model instances may perform substantially better (or worse) than the averaged results [17, 28, 33]. An ensemble mechanism was found to reduce variability whilst yielding better performance than the averaging mechanism [<a title="" data-container="body" data-toggle="popover" data-placement="right" data-html="true" data-trigger="hover click" data-content="17. Verga P, Strubell E, McCallum A. Simultaneously self-attending to all mentions for full-abstract biological relation extraction. <i>Proceedings of the Annual Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies (NAACL HLT)</i>. 2018 <a target='_blank' style='cursor:pointer;' href='si.php?db=pubmed&id='><span class='glyphicon glyphicon-share-alt'></span></a>">17</a>].</span> <button onclick="translate_abc(this)" style="border:none;outline:none;color:#5577AA;font-size:10px;margin-bottom:0px;" title="Translate into Chinese"> <span class="glyphicon glyphicon-transfer"></span> </button></pxy> <pxy><span>In this paper, we use a strict majority vote – a simple but effective ensemble method that has been successfully used in some related works [28, 33]. Our ensemble system runs the model 20 times and uses the strict majority vote to obtain the final results.</span> <button onclick="translate_abc(this)" style="border:none;outline:none;color:#5577AA;font-size:10px;margin-bottom:0px;" title="Translate into Chinese"> <span class="glyphicon glyphicon-transfer"></span> </button></pxy> <h3><span>Distant supervision learning </span> <button onclick="translate_abc(this)" style="border:none;outline:none;color:#5577AA;font-size:10px;margin-bottom:0px;" title="Translate into Chinese"> <span class="glyphicon glyphicon-transfer"></span> </button> </h3> <pxy><span>Distant supervision learning is proved its good impact on the relation classification by utilizing the knowledge base in some research [17, 23, 24]. In this work, we continue to apply distant supervision learning to the proposed subgraph models.</span> <button onclick="translate_abc(this)" style="border:none;outline:none;color:#5577AA;font-size:10px;margin-bottom:0px;" title="Translate into Chinese"> <span class="glyphicon glyphicon-transfer"></span> </button></pxy> <pxy><span>In order to take advantage of the available resources, we do not rebuild the distant data ourselves. Instead, we use the CTD-Pfizer dataset [<a title="" data-container="body" data-toggle="popover" data-placement="right" data-html="true" data-trigger="hover click" data-content="34. Davis AP, Wiegers TC, Roberts PM, King BL, Lay JM, Lennon-Hopkins K, Sciaky D, Johnson R, Keating H, Greene N. A ctd–pfizer collaboration: manual curation of 88 000 scientific articles text mined for drug–disease and drug–phenotype interactions. <i>Database</i>. 2013 <a target='_blank' style='cursor:pointer;' href='si.php?db=pubmed&id='><span class='glyphicon glyphicon-share-alt'></span></a>">34</a>] that has been successfully applied in [17, 24]. Since this data does not contain entity annotations, we used Dnorm [<a title="" data-container="body" data-toggle="popover" data-placement="right" data-html="true" data-trigger="hover click" data-content="35. Leaman R, Islamaj Doğan R, Lu Z. Dnorm: disease name normalization with pairwise learning to rank. <i>Bioinformatics</i>. 2013 <a target='_blank' style='cursor:pointer;' href='si.php?db=pubmed&id=23969135'><span class='glyphicon glyphicon-share-alt'></span></a>">35</a>] and tmChem [<a title="" data-container="body" data-toggle="popover" data-placement="right" data-html="true" data-trigger="hover click" data-content="36. Leaman R, Wei C-H, Lu Z. tmchem: a high performance approach for chemical named entity recognition and normalization. <i>J Cheminformatics</i>. 2015 <a target='_blank' style='cursor:pointer;' href='si.php?db=pubmed&id='><span class='glyphicon glyphicon-share-alt'></span></a>">36</a>] tools to annotate the entities. This dataset contains 18,410 documents with 33,224 CID pairs (15,439 unique).</span> <button onclick="translate_abc(this)" style="border:none;outline:none;color:#5577AA;font-size:10px;margin-bottom:0px;" title="Translate into Chinese"> <span class="glyphicon glyphicon-transfer"></span> </button></pxy> <h2><span>Experimental configuration and model’s hyper parameters </span> <button onclick="translate_abc(this)" style="border:none;outline:none;color:#5577AA;font-size:10px;margin-bottom:0px;" title="Translate into Chinese"> <span class="glyphicon glyphicon-transfer"></span> </button> </h2> <pxy><span>Our model was implemented using Python version 3.5 and TensorFlow v1.15.02. The dependency tree is generated using spaCy3. To generate the document subgraph, we set the maximum depth of md=15 and the maximum number of paths k=150 for the breadth-first search algorithm of pathfinding phase. Widening w more than 5 as it may bring a lot of noise information and cause a computational burden. Therefore, we limit the size of the sliding window w lower than 5, i.e., exclude all entity pairs that are apart more than 5 consecutive sentences. Heuristically, we choose top-k path with k=3 for each entity pair.</span> <button onclick="translate_abc(this)" style="border:none;outline:none;color:#5577AA;font-size:10px;margin-bottom:0px;" title="Translate into Chinese"> <span class="glyphicon glyphicon-transfer"></span> </button></pxy> <pxy><span>The shared weight CNN employs the Adam optimizer [<a title="" data-container="body" data-toggle="popover" data-placement="right" data-html="true" data-trigger="hover click" data-content="37. . . <i></i>. <a target='_blank' style='cursor:pointer;' href='si.php?db=pubmed&id='><span class='glyphicon glyphicon-share-alt'></span></a>">37</a>] and uses Glorot random uniform [<a title="" data-container="body" data-toggle="popover" data-placement="right" data-html="true" data-trigger="hover click" data-content="38. Glorot X, Bengio Y. Understanding the difficulty of training deep feedforward neural networks. <i>Proceedings of the International Conference on Artificial Intelligence and Statistics (AISTATS’10). Society for Artificial Intelligence and Statistics, volume 9 of JMLR Proceedings</i>. 2010 <a target='_blank' style='cursor:pointer;' href='si.php?db=pubmed&id='><span class='glyphicon glyphicon-share-alt'></span></a>">38</a>] initialization. The mini-batch training size is set to 128. Surveying the data has shown an undesirable consequence of the subgraph representation. That is an unexpected increase in negative data. For intra sentence problem, the ratio of positive and negative is about 1:2. But using the subgraph this ratio is 1:2.95, 1:3.53, 1:3.85, 1:4.05 and 1:4.20 respectively for window sizes 1, 2, 3, 4 and 5 (note that the title is always connected to the first sentence in sliding window). This leads to an imbalanced data problem, which may negatively influence system performance caused by the bias to the negative label. To minimize the impact of this problem, we assign the class weights to give priority to the minor classes (positive). At this time, we cannot learn this weight automatically. Therefore, we set them heuristically as 3:1 for positive:negative.</span> <button onclick="translate_abc(this)" style="border:none;outline:none;color:#5577AA;font-size:10px;margin-bottom:0px;" title="Translate into Chinese"> <span class="glyphicon glyphicon-transfer"></span> </button></pxy> <pxy><span>We fine-tuned our deep learning model using training and development subsets (as described in Table 1). The optimized model’s hyper-parameters in detail are shown in Table 2. For the final results, we use these configurations to run the training process 100 times and report the average results of 100 runs. The training time for each run is about 17.5 hours. The prediction time for the BC5 test set using the trained model is about 2 minutes.</span> <button onclick="translate_abc(this)" style="border:none;outline:none;color:#5577AA;font-size:10px;margin-bottom:0px;" title="Translate into Chinese"> <span class="glyphicon glyphicon-transfer"></span> </button></pxy> <div class="xtable"><div class="fig"><b>Table 2</b><p><span>Tuned hyper-parameter of the proposed model</span> <button onclick="translate_abc(this)" style="border:none;outline:none;color:#5577AA;font-size:10px;margin-bottom:0px;" title="Translate into Chinese"> <span class="glyphicon glyphicon-transfer"></span> </button></p><table xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink" xmlns:ali="http://www.niso.org/schemas/ali/1.0/" frame="hsides" rules="groups"><thead><tr><th align="left" colspan="2">Information</th><th align="left">Configuration</th><th align="left">Parameters</th></tr></thead><tbody><tr><td align="left">Dependency embeddings</td><td align="left">Dependency type</td><td align="left">LUT <inline-formula id="IEq16"><alternatives><tex-math id="M53">\documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$\mathbf {W}^{e}_{typ}$\end{document}</tex-math><mml:math id="M54"><mml:msubsup><mml:mrow><mml:mi mathvariant="bold">W</mml:mi></mml:mrow><mml:mrow><mml:mtext mathvariant="italic">typ</mml:mtext></mml:mrow><mml:mrow><mml:mi>e</mml:mi></mml:mrow></mml:msubsup></mml:math><inline-graphic href="13326_2022_267_Article_IEq16.gif"></inline-graphic></alternatives></inline-formula> size 72×150</td><td align="left">10800</td></tr><tr><td align="left"></td><td align="left">Dependency direction</td><td align="left">LUT <inline-formula id="IEq17"><alternatives><tex-math id="M55">\documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$\mathbf {W}^{e}_{dir}$\end{document}</tex-math><mml:math id="M56"><mml:msubsup><mml:mrow><mml:mi mathvariant="bold">W</mml:mi></mml:mrow><mml:mrow><mml:mtext mathvariant="italic">dir</mml:mtext></mml:mrow><mml:mrow><mml:mi>e</mml:mi></mml:mrow></mml:msubsup></mml:math><inline-graphic href="13326_2022_267_Article_IEq17.gif"></inline-graphic></alternatives></inline-formula> size 2×150</td><td align="left">300</td></tr><tr><td align="left">Token embeddings</td><td align="left">FastText embeds</td><td align="left">Pre-trained 300−dim vector</td><td align="left">−</td></tr><tr><td align="left"></td><td align="left">Character embeddings</td><td align="left">LUT <inline-formula id="IEq18"><alternatives><tex-math id="M57">\documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$\mathbf {W}^{e}_{c}$\end{document}</tex-math><mml:math id="M58"><mml:msubsup><mml:mrow><mml:mi mathvariant="bold">W</mml:mi></mml:mrow><mml:mrow><mml:mi>c</mml:mi></mml:mrow><mml:mrow><mml:mi>e</mml:mi></mml:mrow></mml:msubsup></mml:math><inline-graphic href="13326_2022_267_Article_IEq18.gif"></inline-graphic></alternatives></inline-formula> size 85×50</td><td align="left">4250</td></tr><tr><td align="left"></td><td align="left"></td><td align="left">biLSTM with 50 units</td><td align="left">40400</td></tr><tr><td align="left"></td><td align="left">POS tag</td><td align="left">LUT <inline-formula id="IEq19"><alternatives><tex-math id="M59">\documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$\mathbf {W}^{e}_{t}$\end{document}</tex-math><mml:math id="M60"><mml:msubsup><mml:mrow><mml:mi mathvariant="bold">W</mml:mi></mml:mrow><mml:mrow><mml:mi>t</mml:mi></mml:mrow><mml:mrow><mml:mi>e</mml:mi></mml:mrow></mml:msubsup></mml:math><inline-graphic href="13326_2022_267_Article_IEq19.gif"></inline-graphic></alternatives></inline-formula> size 57×50</td><td align="left">2850</td></tr><tr><td align="left"></td><td align="left">WordNet embeds</td><td align="left">Fixed spare 45−dim vector</td><td align="left">−</td></tr><tr><td align="left">Augmented information</td><td align="left">Base distance embeds</td><td align="left">32−dim vector</td><td align="left">32</td></tr><tr><td align="left"></td><td align="left">Self attention score</td><td align="left"><bold>W</bold><sup><italic>e</italic></sup>,<bold>b</bold><sup><italic>e</italic></sup> transform from 832 dim to scalar</td><td align="left">833</td></tr><tr><td align="left"></td><td align="left">Heuristic attention</td><td align="left">Linear</td><td align="left">−</td></tr><tr><td align="left"></td><td align="left">Kernel filters</td><td align="left">100 filters size 832×1</td><td align="left">83300</td></tr><tr><td align="left" colspan="2">Shared weight-CNN</td><td align="left">128 filters each region-size (1,2,3)</td><td align="left">2056320</td></tr><tr><td align="left">Classifier</td><td align="left">Fully-connected MLP</td><td align="left">Do not use</td><td align="left">−</td></tr><tr><td align="left"></td><td align="left">Softmax</td><td align="left">2 classes</td><td align="left">768</td></tr><tr><td align="left" colspan="3">Total number of parameters</td><td align="left">2199853</td></tr></tbody></table><p><span>Embed: Embedding, Dim: Dimension</span> <button onclick="translate_abc(this)" style="border:none;outline:none;color:#5577AA;font-size:10px;margin-bottom:0px;" title="Translate into Chinese"> <span class="glyphicon glyphicon-transfer"></span> </button></p></div></div><pxy><span>Tuned hyper-parameter of the proposed model</span> <button onclick="translate_abc(this)" style="border:none;outline:none;color:#5577AA;font-size:10px;margin-bottom:0px;" title="Translate into Chinese"> <span class="glyphicon glyphicon-transfer"></span> </button></pxy> <pxy><span>Embed: Embedding, Dim: Dimension</span> <button onclick="translate_abc(this)" style="border:none;outline:none;color:#5577AA;font-size:10px;margin-bottom:0px;" title="Translate into Chinese"> <span class="glyphicon glyphicon-transfer"></span> </button></pxy> <pxy><span>We also apply some techniques to overcome overfitting, including max-norm regularization for Gradient descent [<a title="" data-container="body" data-toggle="popover" data-placement="right" data-html="true" data-trigger="hover click" data-content="11. . . <i></i>. <a target='_blank' style='cursor:pointer;' href='si.php?db=pubmed&id='><span class='glyphicon glyphicon-share-alt'></span></a>">11</a>]; adding Gaussian noise [<a title="" data-container="body" data-toggle="popover" data-placement="right" data-html="true" data-trigger="hover click" data-content="13. Quan C, Hua L, Sun X, Bai W. Multichannel convolutional neural network for biological relation extraction. <i>BioMed Res Int</i>. 2016 <a target='_blank' style='cursor:pointer;' href='si.php?db=pubmed&id='><span class='glyphicon glyphicon-share-alt'></span></a>">13</a>] with the mean of 0.001 to the input embeddings; applying dropout [<a title="" data-container="body" data-toggle="popover" data-placement="right" data-html="true" data-trigger="hover click" data-content="39. Srivastava N, Hinton G, Krizhevsky A, Sutskever I, Salakhutdinov R. Dropout: A simple way to prevent neural networks from overfitting. <i>J Mach Learn Res</i>. 2014 <a target='_blank' style='cursor:pointer;' href='si.php?db=pubmed&id='><span class='glyphicon glyphicon-share-alt'></span></a>">39</a>] at 0.5 after all embedding layers and CNN layers; and using early stopping technique [<a title="" data-container="body" data-toggle="popover" data-placement="right" data-html="true" data-trigger="hover click" data-content="40. Caruana R, Lawrence S, Giles CL. Overfitting in neural nets: Backpropagation, conjugate gradient, and early stopping. <i>Advances in Neural Information Processing Systems 13, Papers from Neural Information Processing Systems (NIPS)</i>. 2000 <a target='_blank' style='cursor:pointer;' href='si.php?db=pubmed&id='><span class='glyphicon glyphicon-share-alt'></span></a>">40</a>].</span> <button onclick="translate_abc(this)" style="border:none;outline:none;color:#5577AA;font-size:10px;margin-bottom:0px;" title="Translate into Chinese"> <span class="glyphicon glyphicon-transfer"></span> </button></pxy> <h1>Results</h1> <pxy><span>We present this section in four main parts: the contribution of proposed virtual edges; the effectiveness of subgraph windows sizes, the ablation test results of the model components; and the comparison between our results and other state-of-the-art models.</span> <button onclick="translate_abc(this)" style="border:none;outline:none;color:#5577AA;font-size:10px;margin-bottom:0px;" title="Translate into Chinese"> <span class="glyphicon glyphicon-transfer"></span> </button></pxy> <h2><span>Effect of the injected virtual edges in the document subgraph </span> <button onclick="translate_abc(this)" style="border:none;outline:none;color:#5577AA;font-size:10px;margin-bottom:0px;" title="Translate into Chinese"> <span class="glyphicon glyphicon-transfer"></span> </button> </h2> <pxy><span>We study the contribution of injecting virtual edges on the system performance by ablating each of them in turn from the graph and afterward evaluating the model with the sliding window size w=2 and top-3 shortest paths for each entity pair (k=3). We compare these experimental results by the changes of Precision (P), Recall (R) and F1-measure in Table 3 and Fig. 4.</span> <button onclick="translate_abc(this)" style="border:none;outline:none;color:#5577AA;font-size:10px;margin-bottom:0px;" title="Translate into Chinese"> <span class="glyphicon glyphicon-transfer"></span> </button></pxy> <div class="xtable"><div class="fig"><b>Table 3</b><p><span>Ablation test results for added virtual edges in the document subgraph</span> <button onclick="translate_abc(this)" style="border:none;outline:none;color:#5577AA;font-size:10px;margin-bottom:0px;" title="Translate into Chinese"> <span class="glyphicon glyphicon-transfer"></span> </button></p><table xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink" xmlns:ali="http://www.niso.org/schemas/ali/1.0/" frame="hsides" rules="groups"><thead><tr><th align="left"></th><th align="left">Precision</th><th align="left">Recall</th><th align="left">F1</th></tr></thead><tbody><tr><td align="left"><bold>Full connection</bold></td><td align="left"><italic>61.25</italic></td><td align="left"><italic>61.26</italic></td><td align="left"><italic>61.25</italic></td></tr><tr><td align="left">Without TITLE</td><td align="left">62.24</td><td align="left"><bold>55.58</bold></td><td align="left"><bold>58.72</bold></td></tr><tr><td align="left">Without NEXT-SENT</td><td align="left"><bold>60.98</bold></td><td align="left"><bold>58.79</bold></td><td align="left"><bold>59.86</bold></td></tr><tr><td align="left">Without COREF-sent</td><td align="left">63.80</td><td align="left"><bold>60.01</bold></td><td align="left">61.85</td></tr><tr><td align="left">Without COREF-to-title</td><td align="left">62.60</td><td align="left"><bold>60.89</bold></td><td align="left">61.73</td></tr><tr><td align="left">Without COREF-from-title</td><td align="left"><bold>60.88</bold></td><td align="left">64.27</td><td align="left">62.53</td></tr><tr><td align="left">Without KB-CTD</td><td align="left">64.28</td><td align="left"><bold>59.84</bold></td><td align="left">61.98</td></tr></tbody></table><p><span>Results are reported in %</span> <button onclick="translate_abc(this)" style="border:none;outline:none;color:#5577AA;font-size:10px;margin-bottom:0px;" title="Translate into Chinese"> <span class="glyphicon glyphicon-transfer"></span> </button></p><p><span>Decreased results are highlighted in bold</span> <button onclick="translate_abc(this)" style="border:none;outline:none;color:#5577AA;font-size:10px;margin-bottom:0px;" title="Translate into Chinese"> <span class="glyphicon glyphicon-transfer"></span> </button></p></div></div><div class="fig"><img src="https://www.ncbi.nlm.nih.gov/pmc/articles/PMC9166375/bin/13326_2022_267_Fig4_HTML.jpg" style="width:99%;" /><b>Fig. 4</b><p><span>Ablation test results for virtual edges of the document subgraph. The vertical axis shows the performance in %. Experiments are conducted with 3 shortest paths</span> <button onclick="translate_abc(this)" style="border:none;outline:none;color:#5577AA;font-size:10px;margin-bottom:0px;" title="Translate into Chinese"> <span class="glyphicon glyphicon-transfer"></span> </button></p></div><pxy><span>Ablation test results for virtual edges of the document subgraph. The vertical axis shows the performance in %. Experiments are conducted with 3 shortest paths</span> <button onclick="translate_abc(this)" style="border:none;outline:none;color:#5577AA;font-size:10px;margin-bottom:0px;" title="Translate into Chinese"> <span class="glyphicon glyphicon-transfer"></span> </button></pxy> <pxy><span>Ablation test results for added virtual edges in the document subgraph</span> <button onclick="translate_abc(this)" style="border:none;outline:none;color:#5577AA;font-size:10px;margin-bottom:0px;" title="Translate into Chinese"> <span class="glyphicon glyphicon-transfer"></span> </button></pxy> <pxy><span>Results are reported in %</span> <button onclick="translate_abc(this)" style="border:none;outline:none;color:#5577AA;font-size:10px;margin-bottom:0px;" title="Translate into Chinese"> <span class="glyphicon glyphicon-transfer"></span> </button></pxy> <pxy><span>Decreased results are highlighted in bold</span> <button onclick="translate_abc(this)" style="border:none;outline:none;color:#5577AA;font-size:10px;margin-bottom:0px;" title="Translate into Chinese"> <span class="glyphicon glyphicon-transfer"></span> </button></pxy> <pxy><span>This experiment presents an exciting view of the contributions for each type of virtual edge in the document subgraph. When removing NEXT-SENT from the graph, the results decrease in terms of all Precision, Recall and F1. The same results appear when we remove TITLE.</span> <button onclick="translate_abc(this)" style="border:none;outline:none;color:#5577AA;font-size:10px;margin-bottom:0px;" title="Translate into Chinese"> <span class="glyphicon glyphicon-transfer"></span> </button></pxy> <pxy><span>In addition, although the COREF-sent, COREF-to-title and KB-CTD help to find some more correct relations, it brings too many false-positive results and leads to worse Precision (removing them boosts the Precision but gives a bit lower Recall).</span> <button onclick="translate_abc(this)" style="border:none;outline:none;color:#5577AA;font-size:10px;margin-bottom:0px;" title="Translate into Chinese"> <span class="glyphicon glyphicon-transfer"></span> </button></pxy> <pxy><span>Using the COREF-from-title connection also reduce F1, but because it adversely affects heavily Recall whist only gives a minimal contribution to Precision.</span> <button onclick="translate_abc(this)" style="border:none;outline:none;color:#5577AA;font-size:10px;margin-bottom:0px;" title="Translate into Chinese"> <span class="glyphicon glyphicon-transfer"></span> </button></pxy> <pxy><span>These experimental results have raised a challenge that if we want to use the information about coreference and knowledge-bases, we need some additional methods to increase the quality of the information obtained. We left this problem for further work. Therefore, in the next experiments, we only use two connections NEXT-SENT and TITLE.</span> <button onclick="translate_abc(this)" style="border:none;outline:none;color:#5577AA;font-size:10px;margin-bottom:0px;" title="Translate into Chinese"> <span class="glyphicon glyphicon-transfer"></span> </button></pxy> <h2><span>Effect of different sliding window size w for training and testing </span> <button onclick="translate_abc(this)" style="border:none;outline:none;color:#5577AA;font-size:10px;margin-bottom:0px;" title="Translate into Chinese"> <span class="glyphicon glyphicon-transfer"></span> </button> </h2> <pxy><span>We describe the change of the model’s performance with different sizes of the sliding window in Fig. 5. The larger w helps to increase Recall but leads to a worse Precision. This is an easy-to-explain result because with a larger w we will get more paths, but more noise. The equilibrium point of Precision and Recall gives the highest F1 result at w=2, in detail, we have Precision=61.25%, Recall=61.26% and F1=61.25%.</span> <button onclick="translate_abc(this)" style="border:none;outline:none;color:#5577AA;font-size:10px;margin-bottom:0px;" title="Translate into Chinese"> <span class="glyphicon glyphicon-transfer"></span> </button></pxy> <div class="fig"><img src="https://www.ncbi.nlm.nih.gov/pmc/articles/PMC9166375/bin/13326_2022_267_Fig5_HTML.jpg" style="width:99%;" /><b>Fig. 5</b><p><span>The change of results with different size of sliding window. The vertical axis shows the performance in % while the horizontal axis shows the size of w. Only fastText word embedding is used to represent words. Experiments are conducted with 3 shortest paths</span> <button onclick="translate_abc(this)" style="border:none;outline:none;color:#5577AA;font-size:10px;margin-bottom:0px;" title="Translate into Chinese"> <span class="glyphicon glyphicon-transfer"></span> </button></p></div><pxy><span>The change of results with different size of sliding window. The vertical axis shows the performance in % while the horizontal axis shows the size of w. Only fastText word embedding is used to represent words. Experiments are conducted with 3 shortest paths</span> <button onclick="translate_abc(this)" style="border:none;outline:none;color:#5577AA;font-size:10px;margin-bottom:0px;" title="Translate into Chinese"> <span class="glyphicon glyphicon-transfer"></span> </button></pxy> <pxy><span>More importantly, this statement also raises an idea to take advantage of a large w but minimize the impact on Precision at the lowest level that whether we use the different window sizes for training and testing. The larger window size for training helps to collect new patterns in the text. The smaller window size for testing helps to reduce noise and narrow the allowed distance between two entities. To demonstrate this idea, grid search experiments with k=3 were conducted, the results are shown in Table 4.</span> <button onclick="translate_abc(this)" style="border:none;outline:none;color:#5577AA;font-size:10px;margin-bottom:0px;" title="Translate into Chinese"> <span class="glyphicon glyphicon-transfer"></span> </button></pxy> <div class="xtable"><div class="fig"><b>Table 4</b><p><span>Results of the document subgraph with different sizes of the sliding window for training and testing</span> <button onclick="translate_abc(this)" style="border:none;outline:none;color:#5577AA;font-size:10px;margin-bottom:0px;" title="Translate into Chinese"> <span class="glyphicon glyphicon-transfer"></span> </button></p><table xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink" xmlns:ali="http://www.niso.org/schemas/ali/1.0/" frame="hsides" rules="groups"><thead><tr><th align="left"><italic>w</italic> for training</th><th align="left"><italic>w</italic> for testing</th><th align="left">Precision</th><th align="left">Recall</th><th align="left">F1</th></tr></thead><tbody><tr><td align="left">1</td><td align="left">1</td><td align="left"><bold>65.34</bold></td><td align="left">55.50</td><td align="left">60.02</td></tr><tr><td align="left"></td><td align="left">2</td><td align="left">62.20</td><td align="left">57.22</td><td align="left">59.61</td></tr><tr><td align="left"></td><td align="left">3</td><td align="left">61.47</td><td align="left">58.27</td><td align="left">59.83</td></tr><tr><td align="left"></td><td align="left">4</td><td align="left">61.92</td><td align="left">54.86</td><td align="left">58.18</td></tr><tr><td align="left"></td><td align="left">5</td><td align="left">57.13</td><td align="left">59.76</td><td align="left">58.42</td></tr><tr><td align="left">2</td><td align="left">1</td><td align="left">61.95</td><td align="left">60.19</td><td align="left">61.06</td></tr><tr><td align="left"></td><td align="left">2</td><td align="left">61.25</td><td align="left">61.26</td><td align="left">61.25</td></tr><tr><td align="left"></td><td align="left">3</td><td align="left">61.97</td><td align="left">60.30</td><td align="left">61.12</td></tr><tr><td align="left"></td><td align="left">4</td><td align="left">61.30</td><td align="left">58.52</td><td align="left">59.88</td></tr><tr><td align="left"></td><td align="left">5</td><td align="left">60.99</td><td align="left">59.36</td><td align="left">60.16</td></tr><tr><td align="left">3</td><td align="left">1</td><td align="left">61.05</td><td align="left">61.74</td><td align="left">61.39</td></tr><tr><td align="left"></td><td align="left">2</td><td align="left">60.65</td><td align="left">61.74</td><td align="left">61.19</td></tr><tr><td align="left"></td><td align="left">3</td><td align="left">60.70</td><td align="left">61.88</td><td align="left">61.28</td></tr><tr><td align="left"></td><td align="left">4</td><td align="left">62.30</td><td align="left">59.47</td><td align="left">60.85</td></tr><tr><td align="left"></td><td align="left">5</td><td align="left">61.10</td><td align="left">59.81</td><td align="left">60.45</td></tr><tr><td align="left">4</td><td align="left">1</td><td align="left">60.30</td><td align="left">64.01</td><td align="left">62.10</td></tr><tr><td align="left"></td><td align="left">2</td><td align="left">57.88</td><td align="left"><bold>65.98</bold></td><td align="left">61.67</td></tr><tr><td align="left"></td><td align="left">3</td><td align="left">58.31</td><td align="left">65.27</td><td align="left">61.59</td></tr><tr><td align="left"></td><td align="left">4</td><td align="left">58.40</td><td align="left">63.86</td><td align="left">61.01</td></tr><tr><td align="left"></td><td align="left">5</td><td align="left">59.97</td><td align="left">61.71</td><td align="left">60.83</td></tr><tr><td align="left">5</td><td align="left">1</td><td align="left">61.15</td><td align="left">63.76</td><td align="left">62.43</td></tr><tr><td align="left"></td><td align="left">2</td><td align="left">60.13</td><td align="left">65.89</td><td align="left"><bold>62.88</bold></td></tr><tr><td align="left"></td><td align="left">3</td><td align="left">58.56</td><td align="left">65.79</td><td align="left">61.96</td></tr><tr><td align="left"></td><td align="left">4</td><td align="left">58.64</td><td align="left">62.42</td><td align="left">60.47</td></tr><tr><td align="left"></td><td align="left">5</td><td align="left">57.92</td><td align="left">62.36</td><td align="left">60.06</td></tr></tbody></table><p><span>Results are reported in %</span> <button onclick="translate_abc(this)" style="border:none;outline:none;color:#5577AA;font-size:10px;margin-bottom:0px;" title="Translate into Chinese"> <span class="glyphicon glyphicon-transfer"></span> </button></p><p><span>The highest result in each column is highlighted in bold</span> <button onclick="translate_abc(this)" style="border:none;outline:none;color:#5577AA;font-size:10px;margin-bottom:0px;" title="Translate into Chinese"> <span class="glyphicon glyphicon-transfer"></span> </button></p></div></div><pxy><span>Results of the document subgraph with different sizes of the sliding window for training and testing</span> <button onclick="translate_abc(this)" style="border:none;outline:none;color:#5577AA;font-size:10px;margin-bottom:0px;" title="Translate into Chinese"> <span class="glyphicon glyphicon-transfer"></span> </button></pxy> <pxy><span>Results are reported in %</span> <button onclick="translate_abc(this)" style="border:none;outline:none;color:#5577AA;font-size:10px;margin-bottom:0px;" title="Translate into Chinese"> <span class="glyphicon glyphicon-transfer"></span> </button></pxy> <pxy><span>The highest result in each column is highlighted in bold</span> <button onclick="translate_abc(this)" style="border:none;outline:none;color:#5577AA;font-size:10px;margin-bottom:0px;" title="Translate into Chinese"> <span class="glyphicon glyphicon-transfer"></span> </button></pxy> <pxy><span>The results have verified the effectiveness of the proposed ideas. With the larger w for training size, we have better Recall but worse Precision. For each training window size, the smaller w for testing always brings better F1 than the larger w. The best F1 archived with w=5 for training and w=2 for testing, increase 1.34% compared to the best results of using the same window size for training and testing.</span> <button onclick="translate_abc(this)" style="border:none;outline:none;color:#5577AA;font-size:10px;margin-bottom:0px;" title="Translate into Chinese"> <span class="glyphicon glyphicon-transfer"></span> </button></pxy> <h2><span>Contribution of the model components </span> <button onclick="translate_abc(this)" style="border:none;outline:none;color:#5577AA;font-size:10px;margin-bottom:0px;" title="Translate into Chinese"> <span class="glyphicon glyphicon-transfer"></span> </button> </h2> <pxy><span>We further investigate the contribution of each component in Table 5, which shows changes in F1 when ablating each component from the proposed model.</span> <button onclick="translate_abc(this)" style="border:none;outline:none;color:#5577AA;font-size:10px;margin-bottom:0px;" title="Translate into Chinese"> <span class="glyphicon glyphicon-transfer"></span> </button></pxy> <div class="xtable"><div class="fig"><b>Table 5</b><p><span>Ablation test results for various components of the document subgraph based model</span> <button onclick="translate_abc(this)" style="border:none;outline:none;color:#5577AA;font-size:10px;margin-bottom:0px;" title="Translate into Chinese"> <span class="glyphicon glyphicon-transfer"></span> </button></p><table xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink" xmlns:ali="http://www.niso.org/schemas/ali/1.0/" frame="hsides" rules="groups"><thead><tr><th align="left">Component removed/changed</th><th align="left">Precision</th><th align="left">Recall</th><th align="left">F1</th><th align="left">Change of F1</th></tr></thead><tbody><tr><td align="justify"><bold>Full model</bold></td><td align="left"><bold>60.13</bold></td><td align="left"><bold>65.89</bold></td><td align="left"><bold>62.88</bold></td><td align="justify"></td></tr><tr><td align="justify">Without subgraph</td><td align="left">57.68</td><td align="left">55.16</td><td align="left">56.39</td><td align="justify">-6.49</td></tr><tr><td align="justify">Without TITLE</td><td align="left">61.12</td><td align="left">54.12</td><td align="left">57.41</td><td align="justify">-5.47</td></tr><tr><td align="justify">Without NEXT-SENT</td><td align="left">62.36</td><td align="left">58.33</td><td align="left">60.28</td><td align="justify">-2.60</td></tr><tr><td align="justify">Without instance merging technique</td><td align="left">52.40</td><td align="left">69.26</td><td align="left">59.66</td><td align="justify">-3.22</td></tr><tr><td align="justify">Without swCNN and top-<italic>k</italic> paths</td><td align="left">59.92</td><td align="left">62.19</td><td align="left">61.03</td><td align="justify">-1.84</td></tr><tr><td align="justify">Choose top-<italic>k</italic> by highest frequency (instead of length)</td><td align="left">58.56</td><td align="left">66.96</td><td align="left">62.48</td><td align="justify">-0.40</td></tr><tr><td align="justify">Use w=2 for both training and testing (instead of different <italic>w</italic>)</td><td align="left">61.25</td><td align="left">61.26</td><td align="left">61.25</td><td align="justify">-1.62</td></tr><tr><td align="justify">Without using class weight</td><td align="left">59.60</td><td align="left">65.92</td><td align="left">62.60</td><td align="justify">-0.28</td></tr><tr><td align="justify">Without attention mechanism</td><td align="left">59.13</td><td align="left">64.85</td><td align="left">61.86</td><td align="justify">-1.02</td></tr></tbody></table><p><span>Results are reported in %</span> <button onclick="translate_abc(this)" style="border:none;outline:none;color:#5577AA;font-size:10px;margin-bottom:0px;" title="Translate into Chinese"> <span class="glyphicon glyphicon-transfer"></span> </button></p><p><span>Column ‘Change of F1’ shows the decrease of F1 when removing/changing components from the model</span> <button onclick="translate_abc(this)" style="border:none;outline:none;color:#5577AA;font-size:10px;margin-bottom:0px;" title="Translate into Chinese"> <span class="glyphicon glyphicon-transfer"></span> </button></p><p><span>Highest result in each column is highlighted in bold</span> <button onclick="translate_abc(this)" style="border:none;outline:none;color:#5577AA;font-size:10px;margin-bottom:0px;" title="Translate into Chinese"> <span class="glyphicon glyphicon-transfer"></span> </button></p></div></div><pxy><span>Ablation test results for various components of the document subgraph based model</span> <button onclick="translate_abc(this)" style="border:none;outline:none;color:#5577AA;font-size:10px;margin-bottom:0px;" title="Translate into Chinese"> <span class="glyphicon glyphicon-transfer"></span> </button></pxy> <pxy><span>Results are reported in %</span> <button onclick="translate_abc(this)" style="border:none;outline:none;color:#5577AA;font-size:10px;margin-bottom:0px;" title="Translate into Chinese"> <span class="glyphicon glyphicon-transfer"></span> </button></pxy> <pxy><span>Column ‘Change of F1’ shows the decrease of F1 when removing/changing components from the model</span> <button onclick="translate_abc(this)" style="border:none;outline:none;color:#5577AA;font-size:10px;margin-bottom:0px;" title="Translate into Chinese"> <span class="glyphicon glyphicon-transfer"></span> </button></pxy> <pxy><span>Highest result in each column is highlighted in bold</span> <button onclick="translate_abc(this)" style="border:none;outline:none;color:#5577AA;font-size:10px;margin-bottom:0px;" title="Translate into Chinese"> <span class="glyphicon glyphicon-transfer"></span> </button></pxy> <pxy><span>The F1 reductions illustrate the contributions of all proposals to the final result. However, the level of contribution is varied among the different components. The document subgraph has proven its superiority by boosting the F1 by 6.49%, in which the Recall increases 10.73%. Both TITLE and NEXT-SENT connections have shown a significant influence on model performance. The interesting observation TITLE edges seem to play a leading role: eliminating it reduces the F1 by 5.47%. NEXT-SENT information also plays an essential role since removing it reduces F1 by 2.60%. Our proposed instance merging technique also has a significant contribution, without using it, F1 increases 3.22%. The shared-weight CNN on top-k paths demonstrated its good influence on the results by boosting F1 by 1.84%. Another experiment on using alternative methods for choosing top-k paths (by their repetitions frequencies instead of the shortest length) seems not suitable since it leads to a slight reduction in F1. As discussed above, the use of difference w for training and testing also brings a reduction of F1. Adding class weight and attention technique helps to improve F1 for 0.28% and 1.02%, respectively.</span> <button onclick="translate_abc(this)" style="border:none;outline:none;color:#5577AA;font-size:10px;margin-bottom:0px;" title="Translate into Chinese"> <span class="glyphicon glyphicon-transfer"></span> </button></pxy> <h2><span>Comparison to existing models </span> <button onclick="translate_abc(this)" style="border:none;outline:none;color:#5577AA;font-size:10px;margin-bottom:0px;" title="Translate into Chinese"> <span class="glyphicon glyphicon-transfer"></span> </button> </h2> <pxy><span>We compare the performance of our model against nine competitors. The first three models are capable of predicting intra sentence relations only, the next six models have the ability to extract inter sentence relations:</span> <button onclick="translate_abc(this)" style="border:none;outline:none;color:#5577AA;font-size:10px;margin-bottom:0px;" title="Translate into Chinese"> <span class="glyphicon glyphicon-transfer"></span> </button></pxy> <pxy><span>Zhou et al. (2016) [<a title="" data-container="body" data-toggle="popover" data-placement="right" data-html="true" data-trigger="hover click" data-content="41. Zhou H, Deng H, Chen L, Yang Y, Jia C, Huang D. Exploiting syntactic and semantics information for chemical–disease relation extraction. <i>Database</i>. 2016 <a target='_blank' style='cursor:pointer;' href='si.php?db=pubmed&id='><span class='glyphicon glyphicon-share-alt'></span></a>">41</a>] proposed the hybridDNN model that consists of a feature-based model, a tree kernel-based model and a neural network model.</span> <button onclick="translate_abc(this)" style="border:none;outline:none;color:#5577AA;font-size:10px;margin-bottom:0px;" title="Translate into Chinese"> <span class="glyphicon glyphicon-transfer"></span> </button></pxy> <pxy><span>Panyam et al. (2018) [<a title="" data-container="body" data-toggle="popover" data-placement="right" data-html="true" data-trigger="hover click" data-content="42. Panyam NC, Verspoor K, Cohn T, Ramamohanarao K. Exploiting graph kernels for high performance biomedical relation extraction. <i>J Biomed Semant</i>. 2018 <a target='_blank' style='cursor:pointer;' href='si.php?db=pubmed&id='><span class='glyphicon glyphicon-share-alt'></span></a>">42</a>] used an enhanced dependency parse graph of a sentence with Approximate Subgraph Matching (ASM) kernel to classify CID relation.</span> <button onclick="translate_abc(this)" style="border:none;outline:none;color:#5577AA;font-size:10px;margin-bottom:0px;" title="Translate into Chinese"> <span class="glyphicon glyphicon-transfer"></span> </button></pxy> <pxy><span>MASS [<a title="" data-container="body" data-toggle="popover" data-placement="right" data-html="true" data-trigger="hover click" data-content="28. Le H-Q, Can D-C, Vu ST, Dang TH, Pilehvar MT, Collier N. Large-scale exploration of neural relation classification architectures. <i>Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing</i>. 2018 <a target='_blank' style='cursor:pointer;' href='si.php?db=pubmed&id='><span class='glyphicon glyphicon-share-alt'></span></a>">28</a>] (stands for ‘Man for All Seasons’ model) is a large-scale neural relation classification architecture that has been applied on six benchmark datasets.</span> <button onclick="translate_abc(this)" style="border:none;outline:none;color:#5577AA;font-size:10px;margin-bottom:0px;" title="Translate into Chinese"> <span class="glyphicon glyphicon-transfer"></span> </button></pxy> <pxy><span>UET-CAM system [<a title="" data-container="body" data-toggle="popover" data-placement="right" data-html="true" data-trigger="hover click" data-content="23. Le H-Q, Tran M-V, Dang TH, Ha Q-T, Collier N. Sieve-based coreference resolution enhances semi-supervised learning model for chemical-induced disease relation extraction. <i>Database</i>. 2016 <a target='_blank' style='cursor:pointer;' href='si.php?db=pubmed&id='><span class='glyphicon glyphicon-share-alt'></span></a>">23</a>] is a Support vector machine (SVM) -based model. It uses multi-pass sieves for the coreference resolution to extract inter sentence relations.</span> <button onclick="translate_abc(this)" style="border:none;outline:none;color:#5577AA;font-size:10px;margin-bottom:0px;" title="Translate into Chinese"> <span class="glyphicon glyphicon-transfer"></span> </button></pxy> <pxy><span>SVM-based model of Peng et al. (2016) [<a title="" data-container="body" data-toggle="popover" data-placement="right" data-html="true" data-trigger="hover click" data-content="24. Peng Y, Wei C-H, Lu Z. Improving chemical disease relation extraction with rich features and weakly labeled data. <i>J Cheminformatics</i>. 2016 <a target='_blank' style='cursor:pointer;' href='si.php?db=pubmed&id='><span class='glyphicon glyphicon-share-alt'></span></a>">24</a>] uses a rich feature set and rule-based output, enhanced by distant supervision learning.</span> <button onclick="translate_abc(this)" style="border:none;outline:none;color:#5577AA;font-size:10px;margin-bottom:0px;" title="Translate into Chinese"> <span class="glyphicon glyphicon-transfer"></span> </button></pxy> <pxy><span>CNN+ME [<a title="" data-container="body" data-toggle="popover" data-placement="right" data-html="true" data-trigger="hover click" data-content="25. Gu J, Sun F, Qian L, Zhou G. Chemical-induced disease relation extraction via convolutional neural network. <i>Database (Oxford)</i>. 2017 <a target='_blank' style='cursor:pointer;' href='si.php?db=pubmed&id='><span class='glyphicon glyphicon-share-alt'></span></a>">25</a>] is the hybrid model of the maximum entropy model for inter sentence relation classification and the CNN model for intra sentence relation classification.</span> <button onclick="translate_abc(this)" style="border:none;outline:none;color:#5577AA;font-size:10px;margin-bottom:0px;" title="Translate into Chinese"> <span class="glyphicon glyphicon-transfer"></span> </button></pxy> <pxy><span>An LSTM-CNN model that learns document-level semantic representations by processing consecutive sentences as a sequence of sentences [<a title="" data-container="body" data-toggle="popover" data-placement="right" data-html="true" data-trigger="hover click" data-content="20. Zheng W, Lin H, Li Z, Liu X, Li Z, Xu B, Zhang Y, Yang Z, Wang J. An effective neural model extracting document level chemical-induced disease relations from biomedical literature. <i>J Biomed Inform</i>. 2018 <a target='_blank' style='cursor:pointer;' href='si.php?db=pubmed&id=29746916'><span class='glyphicon glyphicon-share-alt'></span></a>">20</a>].</span> <button onclick="translate_abc(this)" style="border:none;outline:none;color:#5577AA;font-size:10px;margin-bottom:0px;" title="Translate into Chinese"> <span class="glyphicon glyphicon-transfer"></span> </button></pxy> <pxy><span>Biaffine Relation Attention Network (BRAN) takes advantage of the state-of-the-art attention tool Transformer [<a title="" data-container="body" data-toggle="popover" data-placement="right" data-html="true" data-trigger="hover click" data-content="17. Verga P, Strubell E, McCallum A. Simultaneously self-attending to all mentions for full-abstract biological relation extraction. <i>Proceedings of the Annual Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies (NAACL HLT)</i>. 2018 <a target='_blank' style='cursor:pointer;' href='si.php?db=pubmed&id='><span class='glyphicon glyphicon-share-alt'></span></a>">17</a>].</span> <button onclick="translate_abc(this)" style="border:none;outline:none;color:#5577AA;font-size:10px;margin-bottom:0px;" title="Translate into Chinese"> <span class="glyphicon glyphicon-transfer"></span> </button></pxy> <pxy><span>The labeled edge graph convolutional neural network model on a document-level graph [<a title="" data-container="body" data-toggle="popover" data-placement="right" data-html="true" data-trigger="hover click" data-content="18. Sahu SK, Christopoulou F, Miwa M, Ananiadou S. Inter-sentence relation extraction with document-level graph convolutional neural network. <i>Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics</i>. 2019 <a target='_blank' style='cursor:pointer;' href='si.php?db=pubmed&id='><span class='glyphicon glyphicon-share-alt'></span></a>">18</a>]. The graph is constructed using various inter- and intra sentence dependencies to capture local and non-local dependency information.</span> <button onclick="translate_abc(this)" style="border:none;outline:none;color:#5577AA;font-size:10px;margin-bottom:0px;" title="Translate into Chinese"> <span class="glyphicon glyphicon-transfer"></span> </button></pxy> <pxy><span>Table 6 summarizes the performance of our model and some comparative models. In which, the results of comparative models are reported both with and without using any additional enhancements.</span> <button onclick="translate_abc(this)" style="border:none;outline:none;color:#5577AA;font-size:10px;margin-bottom:0px;" title="Translate into Chinese"> <span class="glyphicon glyphicon-transfer"></span> </button></pxy> <div class="xtable"><div class="fig"><b>Table 6</b><p><span>The performance of document subgraph-based model and some comparative models</span> <button onclick="translate_abc(this)" style="border:none;outline:none;color:#5577AA;font-size:10px;margin-bottom:0px;" title="Translate into Chinese"> <span class="glyphicon glyphicon-transfer"></span> </button></p><table xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink" xmlns:ali="http://www.niso.org/schemas/ali/1.0/" frame="hsides" rules="groups"><thead><tr><th align="left" colspan="2">Method/model</th><th align="left">Precision</th><th align="left">Recall</th><th align="left">F1</th></tr></thead><tbody><tr><td align="left" colspan="5"><italic>NOT having the ability to extract inter sentence relations</italic></td></tr><tr><td align="left">hybridDNN (Zhou et al., 2016 [<xref ref-type="bibr" rid="CR41">41</xref>])</td><td align="left">Syntactic features</td><td align="left">62.15</td><td align="left">47.28</td><td align="left">53.70</td></tr><tr><td align="left"></td><td align="left">+ Context</td><td align="left">62.39</td><td align="left">47.47</td><td align="left">53.92</td></tr><tr><td align="left"></td><td align="left">+ Position</td><td align="left">62.86</td><td align="left">47.47</td><td align="left">54.09</td></tr><tr><td align="left">ASM (Panyam et al., 2018 [<xref ref-type="bibr" rid="CR42">42</xref>])</td><td align="left">Dependency graph</td><td align="left">49.00</td><td align="left">67.40</td><td align="left">56.80</td></tr><tr><td align="left">MASS (Le et al., 2018 [<xref ref-type="bibr" rid="CR28">28</xref>])</td><td align="left">Multi channel CNN-LSTM</td><td align="left">58.90</td><td align="left">54.90</td><td align="left">56.90</td></tr><tr><td align="left"></td><td align="left">+ Ensemble</td><td align="left">56.80</td><td align="left">57.90</td><td align="left">57.30</td></tr><tr><td align="left"></td><td align="left">+ Post processing</td><td align="left">52.80</td><td align="left">71.10</td><td align="left">60.60</td></tr><tr><td align="left" colspan="5"><italic>Having the ability to extract inter sentence relations</italic></td></tr><tr><td align="left">UET-CAM (Le et al., 2016 [<xref ref-type="bibr" rid="CR23">23</xref>])</td><td align="left">SVM + coreference</td><td align="left">53.41</td><td align="left">49.41</td><td align="left">51.60</td></tr><tr><td align="left"></td><td align="left">+ Data</td><td align="left">57.63</td><td align="left">60.23</td><td align="left">58.90</td></tr><tr><td align="left">SVM (Peng et al., 2016 [<xref ref-type="bibr" rid="CR24">24</xref>])</td><td align="left">SVM + Rich feature set</td><td align="left">64.24</td><td align="left">52.06</td><td align="left">57.51</td></tr><tr><td align="left"></td><td align="left">+ Data</td><td align="left"><bold>65.59</bold></td><td align="left">56.94</td><td align="left">61.01</td></tr><tr><td align="left">CNN+ME (Gu et al., 2017 [<xref ref-type="bibr" rid="CR25">25</xref>])</td><td align="left">Hybrid model</td><td align="left">60.90</td><td align="left">59.50</td><td align="left">60.20</td></tr><tr><td align="left"></td><td align="left">+ Post-processing</td><td align="left">55.70</td><td align="left">68.10</td><td align="left">61.30</td></tr><tr><td align="left">LSTM-CNN (Zheng et al., 2018 [<xref ref-type="bibr" rid="CR20">20</xref>])</td><td align="left">Sequence of sentences</td><td align="left">24.00</td><td align="left">52.00</td><td align="left">32.80</td></tr><tr><td align="left"></td><td align="left">+ Entity replacing</td><td align="left">54.30</td><td align="left">65.90</td><td align="left">59.50</td></tr><tr><td align="left">BRAN (Verga et al., 2018 [<xref ref-type="bibr" rid="CR17">17</xref>])</td><td align="left">CNN + abstract attention</td><td align="left">55.60</td><td align="left">70.80</td><td align="left">62.10</td></tr><tr><td align="left"></td><td align="left">+ Data</td><td align="left">64.00</td><td align="left">69.20</td><td align="left">66.20</td></tr><tr><td align="left"></td><td align="left">+ Ensemble</td><td align="left">65.40</td><td align="left">71.80</td><td align="left">68.40</td></tr><tr><td align="left">Graph CNN (Sahu et al., 2019 [<xref ref-type="bibr" rid="CR18">18</xref>])</td><td align="left">Document-level Graph</td><td align="left">52.80</td><td align="left">66.00</td><td align="left">58.60</td></tr><tr><td align="left">Our results</td><td align="left">Document subgraph</td><td align="left">60.13</td><td align="left">65.89</td><td align="left">62.88</td></tr><tr><td align="left"></td><td align="left">+ Data</td><td align="left">62.95</td><td align="left"><bold>75.16</bold></td><td align="left">68.52</td></tr><tr><td align="left"></td><td align="left">+ Ensemble</td><td align="left">64.79</td><td align="left">74.05</td><td align="left"><bold>69.11</bold></td></tr></tbody></table><p><span>Results are reported in %</span> <button onclick="translate_abc(this)" style="border:none;outline:none;color:#5577AA;font-size:10px;margin-bottom:0px;" title="Translate into Chinese"> <span class="glyphicon glyphicon-transfer"></span> </button></p><p><span>Highest result in each column is highlighted in bold</span> <button onclick="translate_abc(this)" style="border:none;outline:none;color:#5577AA;font-size:10px;margin-bottom:0px;" title="Translate into Chinese"> <span class="glyphicon glyphicon-transfer"></span> </button></p></div></div><pxy><span>The performance of document subgraph-based model and some comparative models</span> <button onclick="translate_abc(this)" style="border:none;outline:none;color:#5577AA;font-size:10px;margin-bottom:0px;" title="Translate into Chinese"> <span class="glyphicon glyphicon-transfer"></span> </button></pxy> <pxy><span>Results are reported in %</span> <button onclick="translate_abc(this)" style="border:none;outline:none;color:#5577AA;font-size:10px;margin-bottom:0px;" title="Translate into Chinese"> <span class="glyphicon glyphicon-transfer"></span> </button></pxy> <pxy><span>Highest result in each column is highlighted in bold</span> <button onclick="translate_abc(this)" style="border:none;outline:none;color:#5577AA;font-size:10px;margin-bottom:0px;" title="Translate into Chinese"> <span class="glyphicon glyphicon-transfer"></span> </button></pxy> <pxy><span>Our model yields very competitive results when compared to other state-of-the-art models that have taken into account the inter sentence relationships. Compare to the original model without any additional enhancements, our model gives the best results with 62.88%.</span> <button onclick="translate_abc(this)" style="border:none;outline:none;color:#5577AA;font-size:10px;margin-bottom:0px;" title="Translate into Chinese"> <span class="glyphicon glyphicon-transfer"></span> </button></pxy> <pxy><span>Applying distant supervision learning and ensemble technique, our model still achieves the best result among competing models. The distant data helps to improves our F1 by 5.64% with the best hyper-parameter settings (this data also helped to boost the F1 by 5.9% in Peng et al. (2016) [<a title="" data-container="body" data-toggle="popover" data-placement="right" data-html="true" data-trigger="hover click" data-content="24. Peng Y, Wei C-H, Lu Z. Improving chemical disease relation extraction with rich features and weakly labeled data. <i>J Cheminformatics</i>. 2016 <a target='_blank' style='cursor:pointer;' href='si.php?db=pubmed&id='><span class='glyphicon glyphicon-share-alt'></span></a>">24</a>] and 4.1% in BRAN (Verga et al., 2018) [<a title="" data-container="body" data-toggle="popover" data-placement="right" data-html="true" data-trigger="hover click" data-content="17. Verga P, Strubell E, McCallum A. Simultaneously self-attending to all mentions for full-abstract biological relation extraction. <i>Proceedings of the Annual Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies (NAACL HLT)</i>. 2018 <a target='_blank' style='cursor:pointer;' href='si.php?db=pubmed&id='><span class='glyphicon glyphicon-share-alt'></span></a>">17</a>]) The ensemble technique helps BRAN to boost the F1 for 2.2% whist it only helps our model for 0.6% more.</span> <button onclick="translate_abc(this)" style="border:none;outline:none;color:#5577AA;font-size:10px;margin-bottom:0px;" title="Translate into Chinese"> <span class="glyphicon glyphicon-transfer"></span> </button></pxy> <pxy><span>We also show the detailed results for intra- and inter sentence relation extraction in Table 7. In which, we exclude all inter sentence relations when evaluating intra sentence relation extraction results and vice versa.</span> <button onclick="translate_abc(this)" style="border:none;outline:none;color:#5577AA;font-size:10px;margin-bottom:0px;" title="Translate into Chinese"> <span class="glyphicon glyphicon-transfer"></span> </button></pxy> <div class="xtable"><div class="fig"><b>Table 7</b><p><span>The detailed results of the document subgraph-based model</span> <button onclick="translate_abc(this)" style="border:none;outline:none;color:#5577AA;font-size:10px;margin-bottom:0px;" title="Translate into Chinese"> <span class="glyphicon glyphicon-transfer"></span> </button></p><table xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink" xmlns:ali="http://www.niso.org/schemas/ali/1.0/" frame="hsides" rules="groups"><thead><tr><th align="left"></th><th align="left">Precision</th><th align="left">Recall</th><th align="left">F1</th></tr></thead><tbody><tr><td align="left">Full result</td><td align="left">64.79</td><td align="left">74.05</td><td align="left">69.11</td></tr><tr><td align="left">intra sentence relation result <sup><italic>†</italic></sup></td><td align="left">72.91</td><td align="left">85.73</td><td align="left">78.80</td></tr><tr><td align="left">inter sentence relation result <sup><italic>‡</italic></sup></td><td align="left">46.12</td><td align="left">47.28</td><td align="left">46.69</td></tr></tbody></table><p><span>Results are reported in %</span> <button onclick="translate_abc(this)" style="border:none;outline:none;color:#5577AA;font-size:10px;margin-bottom:0px;" title="Translate into Chinese"> <span class="glyphicon glyphicon-transfer"></span> </button></p><p><span>Only evaluated on Intra- or inter sentence relations</span> <button onclick="translate_abc(this)" style="border:none;outline:none;color:#5577AA;font-size:10px;margin-bottom:0px;" title="Translate into Chinese"> <span class="glyphicon glyphicon-transfer"></span> </button></p></div></div><pxy><span>The detailed results of the document subgraph-based model</span> <button onclick="translate_abc(this)" style="border:none;outline:none;color:#5577AA;font-size:10px;margin-bottom:0px;" title="Translate into Chinese"> <span class="glyphicon glyphicon-transfer"></span> </button></pxy> <pxy><span>Results are reported in %</span> <button onclick="translate_abc(this)" style="border:none;outline:none;color:#5577AA;font-size:10px;margin-bottom:0px;" title="Translate into Chinese"> <span class="glyphicon glyphicon-transfer"></span> </button></pxy> <pxy><span>Only evaluated on Intra- or inter sentence relations</span> <button onclick="translate_abc(this)" style="border:none;outline:none;color:#5577AA;font-size:10px;margin-bottom:0px;" title="Translate into Chinese"> <span class="glyphicon glyphicon-transfer"></span> </button></pxy> <h2><span>Error analysis </span> <button onclick="translate_abc(this)" style="border:none;outline:none;color:#5577AA;font-size:10px;margin-bottom:0px;" title="Translate into Chinese"> <span class="glyphicon glyphicon-transfer"></span> </button> </h2> <pxy><span>We studied model outputs to analyze system errors and improvements as shown in Table 8. For further analysis, we use the output of RbSP- an advanced intra sentence relation extraction model [<a title="" data-container="body" data-toggle="popover" data-placement="right" data-html="true" data-trigger="hover click" data-content="30. Can D-C, Le HQ, Ha QT, Collier N. A richer-but-smarter shortest dependency path with attentive augmentation for relation extraction. <i>Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long and Short Papers)</i>. 2019 <a target='_blank' style='cursor:pointer;' href='si.php?db=pubmed&id='><span class='glyphicon glyphicon-share-alt'></span></a>">30</a>]- for comparison, its results are shown in column ‘Comparative model’. The full versions of the abstracts that used in Table 8 are given in Additional file 2: Appendix B.</span> <button onclick="translate_abc(this)" style="border:none;outline:none;color:#5577AA;font-size:10px;margin-bottom:0px;" title="Translate into Chinese"> <span class="glyphicon glyphicon-transfer"></span> </button></pxy> <div class="xtable"><div class="fig"><b>Table 8</b><p><span>Examples of errors on the BC5 CDR test set</span> <button onclick="translate_abc(this)" style="border:none;outline:none;color:#5577AA;font-size:10px;margin-bottom:0px;" title="Translate into Chinese"> <span class="glyphicon glyphicon-transfer"></span> </button></p><table xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink" xmlns:ali="http://www.niso.org/schemas/ali/1.0/" frame="hsides" rules="groups"><thead><tr><th align="left">#</th><th align="left">PMID</th><th align="left">Chemical-Disease</th><th align="left">Golden label</th><th align="left">RbSP <sup><italic>†</italic></sup></th><th align="left">SGM <sup><italic>‡</italic></sup></th><th align="left">Type</th><th align="left">Effect</th><th align="left">Error type</th></tr></thead><tbody><tr><td align="left">1</td><td align="left">2131034</td><td align="left">D003561–D020258</td><td align="left">CID</td><td align="left">NONE</td><td align="left">CID</td><td align="left">Intra</td><td align="left"><bold>Better</bold></td><td align="left">FN → TP</td></tr><tr><td align="left">2</td><td align="left">18801087</td><td align="left">D000638–D009369</td><td align="left">NONE</td><td align="left">CID</td><td align="left">NONE</td><td align="left">Intra</td><td align="left"><bold>Better</bold></td><td align="left">FP → TN</td></tr><tr><td align="left">3</td><td align="left">44072</td><td align="left">C024986–D001145</td><td align="left">CID</td><td align="left">CID</td><td align="left">NONE</td><td align="left">Intra</td><td align="left"><italic>Worse</italic></td><td align="left">TP → FN</td></tr><tr><td align="left">4</td><td align="left">15265979</td><td align="left">D005947–D006529</td><td align="left">NONE</td><td align="left">NONE</td><td align="left">CID</td><td align="left">Intra</td><td align="left"><italic>Worse</italic></td><td align="left">TN → FP</td></tr><tr><td align="left">5</td><td align="left">1655018</td><td align="left">D000305–D006528</td><td align="left">CID</td><td align="left">NONE</td><td align="left">NONE</td><td align="left">Intra</td><td align="left">−</td><td align="left">FN</td></tr><tr><td align="left">6</td><td align="left">35781</td><td align="left">D010423–D002375</td><td align="left">NONE</td><td align="left">CID</td><td align="left">CID</td><td align="left">Intra</td><td align="left">−</td><td align="left">FP</td></tr><tr><td align="left">7</td><td align="left">7644931</td><td align="left">D017239–D018771</td><td align="left">CID</td><td align="left">−</td><td align="left">CID</td><td align="left">Inter</td><td align="left"><bold>Better</bold></td><td align="left">FN → TP</td></tr><tr><td align="left">8</td><td align="left">10327032</td><td align="left">D005472–D008107</td><td align="left">NONE</td><td align="left">−</td><td align="left">CID</td><td align="left">Inter</td><td align="left"><italic>Worse</italic></td><td align="left">TN → FP</td></tr><tr><td align="left">9</td><td align="left">2710809</td><td align="left">D001712–D003680</td><td align="left">CID</td><td align="left">−</td><td align="left">−</td><td align="left">Inter</td><td align="left">−</td><td align="left">FN</td></tr><tr><td align="left">10</td><td align="left">11745287</td><td align="left">D016190–D015431</td><td align="left">CID</td><td align="left">−</td><td align="left">NONE</td><td align="left">Inter</td><td align="left">−</td><td align="left">FN</td></tr><tr><td align="left">11</td><td align="left">10087562</td><td align="left">D004280–D008133</td><td align="left">NONE</td><td align="left">CID</td><td align="left">CID</td><td align="left">Intra</td><td align="left"><bold>Worse</bold><sup><bold>∗</bold></sup></td><td align="left">FN</td></tr><tr><td align="left">12</td><td align="left">24464946</td><td align="left">D015251–D006331</td><td align="left">NONE</td><td align="left">−</td><td align="left">CID</td><td align="left">Inter</td><td align="left"><bold>Worse</bold><sup><bold>∗</bold></sup></td><td align="left">TN → FP</td></tr></tbody></table><p><span>The re-implemented intra sentence RbSP model (Can et al. [30]) - without subgraph model in Table 5</span> <button onclick="translate_abc(this)" style="border:none;outline:none;color:#5577AA;font-size:10px;margin-bottom:0px;" title="Translate into Chinese"> <span class="glyphicon glyphicon-transfer"></span> </button></p><p><span>subgraph model’s prediction</span> <button onclick="translate_abc(this)" style="border:none;outline:none;color:#5577AA;font-size:10px;margin-bottom:0px;" title="Translate into Chinese"> <span class="glyphicon glyphicon-transfer"></span> </button></p><p><span>*Errors due to the imperfect annotation</span> <button onclick="translate_abc(this)" style="border:none;outline:none;color:#5577AA;font-size:10px;margin-bottom:0px;" title="Translate into Chinese"> <span class="glyphicon glyphicon-transfer"></span> </button></p><p><span>CID Chemical-induced disease, NONE Unrelated, ‘ −’: Cannot generate path, TP True Positive, TN True Negative, FP False Positive, FN False Negative</span> <button onclick="translate_abc(this)" style="border:none;outline:none;color:#5577AA;font-size:10px;margin-bottom:0px;" title="Translate into Chinese"> <span class="glyphicon glyphicon-transfer"></span> </button></p><p><span>Cases where the SBM model gives correct results are highlighted in bold</span> <button onclick="translate_abc(this)" style="border:none;outline:none;color:#5577AA;font-size:10px;margin-bottom:0px;" title="Translate into Chinese"> <span class="glyphicon glyphicon-transfer"></span> </button></p></div></div><pxy><span>Examples of errors on the BC5 CDR test set</span> <button onclick="translate_abc(this)" style="border:none;outline:none;color:#5577AA;font-size:10px;margin-bottom:0px;" title="Translate into Chinese"> <span class="glyphicon glyphicon-transfer"></span> </button></pxy> <pxy><span>The re-implemented intra sentence RbSP model (Can et al. [<a title="" data-container="body" data-toggle="popover" data-placement="right" data-html="true" data-trigger="hover click" data-content="30. Can D-C, Le HQ, Ha QT, Collier N. A richer-but-smarter shortest dependency path with attentive augmentation for relation extraction. <i>Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long and Short Papers)</i>. 2019 <a target='_blank' style='cursor:pointer;' href='si.php?db=pubmed&id='><span class='glyphicon glyphicon-share-alt'></span></a>">30</a>]) - without subgraph model in Table 5</span> <button onclick="translate_abc(this)" style="border:none;outline:none;color:#5577AA;font-size:10px;margin-bottom:0px;" title="Translate into Chinese"> <span class="glyphicon glyphicon-transfer"></span> </button></pxy> <pxy><span>subgraph model’s prediction</span> <button onclick="translate_abc(this)" style="border:none;outline:none;color:#5577AA;font-size:10px;margin-bottom:0px;" title="Translate into Chinese"> <span class="glyphicon glyphicon-transfer"></span> </button></pxy> <pxy><span>*Errors due to the imperfect annotation</span> <button onclick="translate_abc(this)" style="border:none;outline:none;color:#5577AA;font-size:10px;margin-bottom:0px;" title="Translate into Chinese"> <span class="glyphicon glyphicon-transfer"></span> </button></pxy> <pxy><span>CID Chemical-induced disease, NONE Unrelated, ‘ −’: Cannot generate path, TP True Positive, TN True Negative, FP False Positive, FN False Negative</span> <button onclick="translate_abc(this)" style="border:none;outline:none;color:#5577AA;font-size:10px;margin-bottom:0px;" title="Translate into Chinese"> <span class="glyphicon glyphicon-transfer"></span> </button></pxy> <pxy><span>Cases where the SBM model gives correct results are highlighted in bold</span> <button onclick="translate_abc(this)" style="border:none;outline:none;color:#5577AA;font-size:10px;margin-bottom:0px;" title="Translate into Chinese"> <span class="glyphicon glyphicon-transfer"></span> </button></pxy> <pxy><span>The former part (Examples # 1−6) shows the effect of the graph-based model on intra sentence relations. It helps find some more intra sentence (Example # 1−2) relations since graph-based representation enriches many useful patterns for training. However, it also causes new noises (Example # 3−4), i.e., some examples are properly correctly labeled by the comparative model, but wrongly by the graph-based model. Example # 5−6 are errors that are not improved.</span> <button onclick="translate_abc(this)" style="border:none;outline:none;color:#5577AA;font-size:10px;margin-bottom:0px;" title="Translate into Chinese"> <span class="glyphicon glyphicon-transfer"></span> </button></pxy> <pxy><span>The latter part (examples # 7−10) focuses on the inter-relation extraction, these relations occupy about 30% of the instances in BC5 CDR corpus and cannot be extracted by the intra sentence model. Example #7 provides an improvement, as the graph model extracts the inter sentence relation correctly. In the case of producing false-positive results (Example #8), the graph-based model is penalized since turning a true negative into a false positive. Moreover, the graph model still misses many cases (Examples # 9−10).</span> <button onclick="translate_abc(this)" style="border:none;outline:none;color:#5577AA;font-size:10px;margin-bottom:0px;" title="Translate into Chinese"> <span class="glyphicon glyphicon-transfer"></span> </button></pxy> <pxy><span>These errors can be attributed to the limitations of our model, including (a) Many errors seem attributable to the parser. Example #9 is the case that we cannot generate any dependency path between two participated entities. The comprehensive analysis shows that our document subgraph representation with w=2 covers only ∼ 93% of total instances in test data (98% intra sentence relations and 87% inter sentence relations), in the remaining cases, we cannot generate any path between two entities. (b) The information in the path may still be insufficient or redundant to make the correct prediction. (c) The graph-based representation brings many noises. New virtual edge also brings confusion, and instance merging with top-k path choosing may lead to the missing of the useful paths. (d) The overfitting problem (leading to wrong prediction – FP) and (e) limited generalisation power in predicting new relations (FN).</span> <button onclick="translate_abc(this)" style="border:none;outline:none;color:#5577AA;font-size:10px;margin-bottom:0px;" title="Translate into Chinese"> <span class="glyphicon glyphicon-transfer"></span> </button></pxy> <pxy><span>Finally, we found some errors caused by the imperfect gold annotation (gold missing relation or gold false relation). Example #11 shows the case that our model finds a correct relation while gold standard annotation does not include. Another annotation errors (Example #12) come from the hierarchy manner. BC5 CDR corpus only annotates relations between the most specific entities, i.e., excludes the relations that involve entities that are more general than other entities already participated in the CID relation of each abstract [<a title="" data-container="body" data-toggle="popover" data-placement="right" data-html="true" data-trigger="hover click" data-content="26. . . <i></i>. <a target='_blank' style='cursor:pointer;' href='si.php?db=pubmed&id='><span class='glyphicon glyphicon-share-alt'></span></a>">26</a>].</span> <button onclick="translate_abc(this)" style="border:none;outline:none;color:#5577AA;font-size:10px;margin-bottom:0px;" title="Translate into Chinese"> <span class="glyphicon glyphicon-transfer"></span> </button></pxy> <h1>Discussion</h1> <pxy><span>In this work, we present a novel representation for a sequence of adjacent sentences in a document (namely document sub-graph). The graph is constructed using various types of information to capture local and non-local features. Knowledge-based information is also used to expropriate the manual realistic information to the model.</span> <button onclick="translate_abc(this)" style="border:none;outline:none;color:#5577AA;font-size:10px;margin-bottom:0px;" title="Translate into Chinese"> <span class="glyphicon glyphicon-transfer"></span> </button></pxy> <pxy><span>We also propose an instance merging mechanism and using a set of multiple paths for representing the relationship between entities pair. Our proposed model outperforms all comparative models in experiments on BC5 CDR corpus without using external knowledge resources and additional enhancements. Comparing the full model performance, our model still achieves comparable results when compared with the current state-of-the-art model (Verga’s BRAN model) [<a title="" data-container="body" data-toggle="popover" data-placement="right" data-html="true" data-trigger="hover click" data-content="17. Verga P, Strubell E, McCallum A. Simultaneously self-attending to all mentions for full-abstract biological relation extraction. <i>Proceedings of the Annual Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies (NAACL HLT)</i>. 2018 <a target='_blank' style='cursor:pointer;' href='si.php?db=pubmed&id='><span class='glyphicon glyphicon-share-alt'></span></a>">17</a>].</span> <button onclick="translate_abc(this)" style="border:none;outline:none;color:#5577AA;font-size:10px;margin-bottom:0px;" title="Translate into Chinese"> <span class="glyphicon glyphicon-transfer"></span> </button></pxy> <pxy><span>When compared with the related work, the highlight of our proposed model is the use of document graphs with different train-test window sizes. To the best of our knowledge, most other studies approach in the direction of seeking relationships in one or several consecutive sentences [20, 24, 25, 28, 42]. Our model solves the problem of extracting relations in the whole document. This idea is similar to the study of Verga et al. [<a title="" data-container="body" data-toggle="popover" data-placement="right" data-html="true" data-trigger="hover click" data-content="17. Verga P, Strubell E, McCallum A. Simultaneously self-attending to all mentions for full-abstract biological relation extraction. <i>Proceedings of the Annual Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies (NAACL HLT)</i>. 2018 <a target='_blank' style='cursor:pointer;' href='si.php?db=pubmed&id='><span class='glyphicon glyphicon-share-alt'></span></a>">17</a>], but they are in the direction of using the attention mechanism to find important information in the text. Instead, we build extract the information on the graph in a linguistic-based manner.</span> <button onclick="translate_abc(this)" style="border:none;outline:none;color:#5577AA;font-size:10px;margin-bottom:0px;" title="Translate into Chinese"> <span class="glyphicon glyphicon-transfer"></span> </button></pxy> <pxy><span>From the perspective of model usage in real-world applications, while graph building and model training are time consuming, they can be done offline. New data processing time is not fast enough to process big data but can be used to extract relations from small and medium datasets in reasonable time. Another problem when applying the model is processing full text. Through research and data survey, the abstract contains the basic information of the article. Basically, it is necessary to investigate more closely because the characteristics of full text and abstract are quite different. For example, with full text processing, window size of 5 may not be enough, two related entities may be very far apart. Extracting the relationship in full text will need some extra processing steps. We leave these problems for the future work.</span> <button onclick="translate_abc(this)" style="border:none;outline:none;color:#5577AA;font-size:10px;margin-bottom:0px;" title="Translate into Chinese"> <span class="glyphicon glyphicon-transfer"></span> </button></pxy> <pxy><span>We also investigated the results in detail to figure out our limitations for future improvements.</span> <button onclick="translate_abc(this)" style="border:none;outline:none;color:#5577AA;font-size:10px;margin-bottom:0px;" title="Translate into Chinese"> <span class="glyphicon glyphicon-transfer"></span> </button></pxy> <pxy><span>Firstly, coreference and discourse resolutions should be analyzed carefully to find a suitable and more effective approach for application.</span> <button onclick="translate_abc(this)" style="border:none;outline:none;color:#5577AA;font-size:10px;margin-bottom:0px;" title="Translate into Chinese"> <span class="glyphicon glyphicon-transfer"></span> </button></pxy> <pxy><span>Secondly, the valuable information coming from knowledge bases needs to be used more reasonably instead of being integrated directly into graphs.</span> <button onclick="translate_abc(this)" style="border:none;outline:none;color:#5577AA;font-size:10px;margin-bottom:0px;" title="Translate into Chinese"> <span class="glyphicon glyphicon-transfer"></span> </button></pxy> <pxy><span>Thirdly, our model’s results resolutely depend on the performance of the dependency parser. This problem leads to the limitation that we must deal with many cascade errors from the processing step. We are planning to use another parser, which is specially built for the biomedical domain.</span> <button onclick="translate_abc(this)" style="border:none;outline:none;color:#5577AA;font-size:10px;margin-bottom:0px;" title="Translate into Chinese"> <span class="glyphicon glyphicon-transfer"></span> </button></pxy> <pxy><span>Lastly, the ensemble mechanism should be improved to have higher results. However, run the graph-based models for many times is quite a time-consuming work; this approach needs an adaptation to be more suitable for the graph-based model.</span> <button onclick="translate_abc(this)" style="border:none;outline:none;color:#5577AA;font-size:10px;margin-bottom:0px;" title="Translate into Chinese"> <span class="glyphicon glyphicon-transfer"></span> </button></pxy> <h1>Conclusions</h1> <pxy><span>In this paper, we present a novel representation for a sequence of consecutive sentences in a document (namely document subgraph). The graph is constructed using various types of information to capture local and non-local features. We also propose an instance merging mechanism and use a set of multiple paths for representing the relationship between entity pairs. To explore the information in the document subgraph, we construct a deep neural architecture based on a shared-weight convolutional neural network.</span> <button onclick="translate_abc(this)" style="border:none;outline:none;color:#5577AA;font-size:10px;margin-bottom:0px;" title="Translate into Chinese"> <span class="glyphicon glyphicon-transfer"></span> </button></pxy> <pxy><span>The interesting analysis is that not all the types of new edges in the graph are useful for inter sentence rela- tion extraction. Only connections of title-sentences and between consecutive sentences are useful. In addition, all components and techniques that we applied in the proposed model show their contributions to the performance at a different level.</span> <button onclick="translate_abc(this)" style="border:none;outline:none;color:#5577AA;font-size:10px;margin-bottom:0px;" title="Translate into Chinese"> <span class="glyphicon glyphicon-transfer"></span> </button></pxy> <pxy><span>In experiments on BioCreative V CDR corpus, without using any external knowledge resources and additional enhancements, our proposed model outperforms all comparative models. We also investigated the results in detail to figure out our limitations for future improvement. The experimental results and error analysis help us to prioritize the future work.</span> <button onclick="translate_abc(this)" style="border:none;outline:none;color:#5577AA;font-size:10px;margin-bottom:0px;" title="Translate into Chinese"> <span class="glyphicon glyphicon-transfer"></span> </button></pxy> <pxy><span>Additional file 1 Appendix A. Example of a document subgraph.</span> <button onclick="translate_abc(this)" style="border:none;outline:none;color:#5577AA;font-size:10px;margin-bottom:0px;" title="Translate into Chinese"> <span class="glyphicon glyphicon-transfer"></span> </button></pxy> <pxy><span>Additional file 2 Appendix B. The full versions of the abstracts that used in Table 8. Examples of errors on the BC5 CDR test set.</span> <button onclick="translate_abc(this)" style="border:none;outline:none;color:#5577AA;font-size:10px;margin-bottom:0px;" title="Translate into Chinese"> <span class="glyphicon glyphicon-transfer"></span> </button></pxy> </div> <div class="tab-pane fade" id="refx"> <div style="padding-top:15px;"> <span style="font-size:13px;"> <span class="glyphicon glyphicon-stats"> </span>   16 in total</span></div> <div style="padding-top:4px;"> </div> <span class="literature_info"></span> <h2><span class="s2"></span> <a href="si.php?db=pubmed&id=29746916" target="_blank" style="cursor:pointer;"><span style="font-weight:500;font-size:15px;color:#337AB7;">1.  An effective neural model extracting document level chemical-induced disease relations from biomedical literature.</span></a></h2> <span class="author">Authors:  Wei Zheng; Hongfei Lin; Zhiheng Li; Xiaoxia Liu; Zhengguang Li; Bo Xu; Yijia Zhang; Zhihao Yang; Jian Wang </span><br> <span class="journal">Journal:  J Biomed Inform </span>      <span class="year">Date:  2018-05-08 </span>      <span class="year">Impact factor: 6.317 </span><br><hr style="padding:0px;margin:10px;margin-left:0px;" /><h2><span class="s2"></span> <a href="si.php?db=pubmed&id=27161011" target="_blank" style="cursor:pointer;"><span style="font-weight:500;font-size:15px;color:#337AB7;">2.  BioCreative V CDR task corpus: a resource for chemical disease relation extraction.</span></a></h2> <span class="author">Authors:  Jiao Li; Yueping Sun; Robin J Johnson; Daniela Sciaky; Chih-Hsuan Wei; Robert Leaman; Allan Peter Davis; Carolyn J Mattingly; Thomas C Wiegers; Zhiyong Lu </span><br> <span class="journal">Journal:  Database (Oxford) </span>      <span class="year">Date:  2016-05-09 </span>      <span class="year">Impact factor: 3.451 </span><br><hr style="padding:0px;margin:10px;margin-left:0px;" /><h2><span class="s2"></span> <a href="si.php?db=pubmed&id=27081156" target="_blank" style="cursor:pointer;"><span style="font-weight:500;font-size:15px;color:#337AB7;">3.  Exploiting syntactic and semantics information for chemical-disease relation extraction.</span></a></h2> <span class="author">Authors:  Huiwei Zhou; Huijie Deng; Long Chen; Yunlong Yang; Chen Jia; Degen Huang </span><br> <span class="journal">Journal:  Database (Oxford) </span>      <span class="year">Date:  2016-04-14 </span>      <span class="year">Impact factor: 3.451 </span><br><hr style="padding:0px;margin:10px;margin-left:0px;" /><h2><span class="s2"></span> <a href="si.php?db=pubmed&id=27630201" target="_blank" style="cursor:pointer;"><span style="font-weight:500;font-size:15px;color:#337AB7;">4.  Sieve-based coreference resolution enhances semi-supervised learning model for chemical-induced disease relation extraction.</span></a></h2> <span class="author">Authors:  Hoang-Quynh Le; Mai-Vu Tran; Thanh Hai Dang; Quang-Thuy Ha; Nigel Collier </span><br> <span class="journal">Journal:  Database (Oxford) </span>      <span class="year">Date:  2016-07 </span>      <span class="year">Impact factor: 3.451 </span><br><hr style="padding:0px;margin:10px;margin-left:0px;" /><h2><span class="s2"></span> <a href="si.php?db=pubmed&id=28053977" target="_blank" style="cursor:pointer;"><span style="font-weight:500;font-size:15px;color:#337AB7;">5.  Multichannel Convolutional Neural Network for Biological Relation Extraction.</span></a></h2> <span class="author">Authors:  Chanqin Quan; Lei Hua; Xiao Sun; Wenjun Bai </span><br> <span class="journal">Journal:  Biomed Res Int </span>      <span class="year">Date:  2016-12-07 </span>      <span class="year">Impact factor: 3.411 </span><br><hr style="padding:0px;margin:10px;margin-left:0px;" /><h2><span class="s2"></span> <a href="si.php?db=pubmed&id=28415073" target="_blank" style="cursor:pointer;"><span style="font-weight:500;font-size:15px;color:#337AB7;">6.  Chemical-induced disease relation extraction via convolutional neural network.</span></a></h2> <span class="author">Authors:  Jinghang Gu; Fuqing Sun; Longhua Qian; Guodong Zhou </span><br> <span class="journal">Journal:  Database (Oxford) </span>      <span class="year">Date:  2017-01-01 </span>      <span class="year">Impact factor: 3.451 </span><br><hr style="padding:0px;margin:10px;margin-left:0px;" /><h2><span class="s2"></span> <a href="si.php?db=pubmed&id=29382397" target="_blank" style="cursor:pointer;"><span style="font-weight:500;font-size:15px;color:#337AB7;">7.  Exploiting graph kernels for high performance biomedical relation extraction.</span></a></h2> <span class="author">Authors:  Nagesh C Panyam; Karin Verspoor; Trevor Cohn; Kotagiri Ramamohanarao </span><br> <span class="journal">Journal:  J Biomed Semantics </span>      <span class="year">Date:  2018-01-30 </span><hr style="padding:0px;margin:10px;margin-left:0px;" /><h2><span class="s2"></span> <a href="si.php?db=pubmed&id=23969135" target="_blank" style="cursor:pointer;"><span style="font-weight:500;font-size:15px;color:#337AB7;">8.  DNorm: disease name normalization with pairwise learning to rank.</span></a></h2> <span class="author">Authors:  Robert Leaman; Rezarta Islamaj Dogan; Zhiyong Lu </span><br> <span class="journal">Journal:  Bioinformatics </span>      <span class="year">Date:  2013-08-21 </span>      <span class="year">Impact factor: 6.937 </span><br><hr style="padding:0px;margin:10px;margin-left:0px;" /><h2><span class="s2"></span> <a href="si.php?db=pubmed&id=24288140" target="_blank" style="cursor:pointer;"><span style="font-weight:500;font-size:15px;color:#337AB7;">9.  A CTD-Pfizer collaboration: manual curation of 88,000 scientific articles text mined for drug-disease and drug-phenotype interactions.</span></a></h2> <span class="author">Authors:  Allan Peter Davis; Thomas C Wiegers; Phoebe M Roberts; Benjamin L King; Jean M Lay; Kelley Lennon-Hopkins; Daniela Sciaky; Robin Johnson; Heather Keating; Nigel Greene; Robert Hernandez; Kevin J McConnell; Ahmed E Enayetallah; Carolyn J Mattingly </span><br> <span class="journal">Journal:  Database (Oxford) </span>      <span class="year">Date:  2013-11-28 </span>      <span class="year">Impact factor: 3.451 </span><br><hr style="padding:0px;margin:10px;margin-left:0px;" /><h2><span class="s2"></span> <a href="si.php?db=pubmed&id=29373599" target="_blank" style="cursor:pointer;"><span style="font-weight:500;font-size:15px;color:#337AB7;">10.  Drug drug interaction extraction from the literature using a recursive neural network.</span></a></h2> <span class="author">Authors:  Sangrak Lim; Kyubum Lee; Jaewoo Kang </span><br> <span class="journal">Journal:  PLoS One </span>      <span class="year">Date:  2018-01-26 </span>      <span class="year">Impact factor: 3.240 </span><br><hr style="padding:0px;margin:10px;margin-left:0px;" /> <div style="padding:0px;"> </div> <span class="more_ref_info"> <a href="javascript:void(0);" style="cursor:pointer;"> <div class="alert alert-info" style="padding-left:0px;width:90%"><center>View more</center></div></a> </span> </div> </div> </div> <script type="text/javascript"> $('.more_ref_info a').click(function() { $(".more_ref_info").html('<div class="alert alert-info" style="padding:8px;padding-left:0px;width:90%"> <div class="three-bounce"> Loading  <div class="bounce1"></div> <div class="bounce2"></div> <div class="bounce3"></div> </div> </div>'); $.post('codes/reference/ref.php',{pn:'2',idx:'35659292'},function(data) { $(".more_ref_info").html(data); }) }); $('.more_cite_info a').click(function() { $(".more_cite_info").html('<div class="alert alert-info" style="padding:8px;padding-left:0px;width:90%"> <div class="three-bounce"> Loading  <div class="bounce1"></div> <div class="bounce2"></div> <div class="bounce3"></div> </div> </div>'); $.post('codes/reference/cite.php',{pn:'2',idx:'35659292'},function(data) { $(".more_cite_info").html(data); }) }); </script> <script type="text/javascript"> $(document).ready(function(){ $(".con").html('<br><br><div style="width:280px;"><div class="spinner"><div class="double-bounce1"></div><div class="double-bounce2"></div></div></div>'); $.post('codes/translate/IF.php',{db:'pubmed',id:'',lang:'en'},function(data) { $(".con").html(data); }); }); $('.search_IF a').click(function(e) { $(".con2").html(''); $(".con").html('<br><div style="width:380px;"><center><font color="#87CEEB"><b>Please waiting ...</b></font></center><br><div class="spinner"><div class="double-bounce1"></div><div class="double-bounce2"></div></div></div>'); $.post('codes/translate/IF.php',{db:'pubmed',id:'',lang:'en'},function(data) { $(".con").html(data); }); }); </script> <script type="text/javascript"> $('.dx_button').click(function() { loading.showLoading({ type:1, tip:"Loading" }) $.post('codes/translate/download_dx.php',{pmid:'35659292'},function(data) { eval('var data='+data); if(data.ti==1){ loading.hideLoading(); window.open('tmpe/35659292.pdf') } }) }) </script> <script type="text/javascript"> $(document).ready(function(){ var t=new Date().getTime(); var id=getCookie('w_id'); if(window.XMLHttpRequest){ var xhr=new XMLHttpRequest(); }else{ var xhr=new ActiveXObject('Microsoft.XMLHTTP'); } xhr.open('GET','src/php/index.php?p='+t); xhr.send(); xhr.onreadystatechange=function(){ if(xhr.readyState==4){ if(xhr.status==200){ if(!(xhr.responseText=='' && id!='' && id!=0)){ var n_val=getCookie('name_val'); //alert(n_val); $.post('codes/translate/download.php',{doi:'10.1186/s13326-022-00267-3',user:n_val},function(data) { $(".d_button").html(data); }); } } } } }); </script> <script> $(document).ready(function(){ $("#Chemical_id").change(function() { if($("#Chemical_id").is(":checked")) { $(".Chemical").addClass("Chemical_desc"); $(".Chemical_desc4").addClass("Chemical_desc3"); $(".Chemical").bind('click',function(e){ e.preventDefault(); var namea = $(this).text(); $(document).ready(function(){ $("#myModal_annotation").modal("show") }); $(".annotation_alert").html('<br><div class="spinner"><div class="double-bounce1"></div><div class="double-bounce2"></div></div>'); $.get('codes/geo/annotation.php',{pmid:'35659292',namea:namea,typea:'Chemical',query:'',db:'pubmed'},function(data) { $(".annotation_alert").html(data); }) }) }else{ $(".Chemical_desc4").removeClass("Chemical_desc3"); $(".Chemical").removeClass("Chemical_desc"); $(".Chemical").unbind(); } }) $(".population").addClass("population_desc"); $("#population_id").change(function() { if($("#population_id").is(":checked")) { $(".population_desc4").addClass("population_desc3"); $(".population").addClass("population_desc"); }else{ $(".population_desc4").removeClass("population_desc3"); $(".population").removeClass("population_desc"); } }) $(".interventions").addClass("interventions_desc"); $("#interventions_id").change(function() { if($("#interventions_id").is(":checked")) { $(".interventions_desc4").addClass("interventions_desc3"); $(".interventions").addClass("interventions_desc"); }else{ $(".interventions_desc4").removeClass("interventions_desc3"); $(".interventions").removeClass("interventions_desc"); } }) $(".outcomes").addClass("outcomes_desc"); $("#outcomes_id").change(function() { if($("#outcomes_id").is(":checked")) { $(".outcomes_desc4").addClass("outcomes_desc3"); $(".outcomes").addClass("outcomes_desc"); }else{ $(".outcomes_desc4").removeClass("outcomes_desc3"); $(".outcomes").removeClass("outcomes_desc"); } }) }) </script> <div class="col-sm-4" style=""> <div id="myNav"> <span class="con"> </span> </div> <span class="con2"></span> <span class="con3"></span> </div> </div> </div> <script type="text/javascript"> function translate_xyz(btnObj){ var x = btnObj.previousElementSibling.innerHTML; $(".con2").html(''); $(".con").html('<br><div style="width:380px;"><center><font color="#87CEEB"><b>正在翻译中 ...</b></font></center><br><div class="spinner"><div class="double-bounce1"></div><div class="double-bounce2"></div></div></div>'); $.post('codes/translate/translate_content.php',{content:x,to_lang:'en'},function(data) { $(".con").html(data); }); } function translate_abc(btnObj){ var x = btnObj.previousElementSibling.innerHTML; $(".con2").html(''); $(".con").html('<br><div style="width:380px;"><center><font color="#87CEEB"><b>正在翻译中 ...</b></font></center><br><div class="spinner"><div class="double-bounce1"></div><div class="double-bounce2"></div></div></div>'); $.post('codes/translate/translate_content.php',{content:x,to_lang:'zh'},function(data) { $(".con").html(data); }); } </script> <script type="text/javascript"> $(document).ready(function(){ loading.hideLoading(); }); $('.tab_b a').click(function() { initial_url_paras = window.location.href.split("?"); initial_url = initial_url_paras[0]; paras = initial_url_paras[1]; paras_array = paras.split("&"); for(let ii=0;ii<paras_array.length;ii++){ current_para_array = paras_array[ii].split("="); if(current_para_array[0]=="db"){dbx=current_para_array[1]} if(current_para_array[0]=="id"){idx=current_para_array[1]} } $(".ax2").html(' <div style="background-color:#d9edf7;padding:1px;padding-left:6px;margin-left:4px;font-size:12px;"><table><tr> <td>跳转中 ... </td> <td> <div class="three-bounce" style="min-height:22px;"> <div class="bounce1"></div> <div class="bounce2"></div> <div class="bounce3"></div> </div></td></tr></table> </div>'); window.location.href = 'si.php?db=' + dbx + '&id=' + idx; }) </script> <div class="modal fade" id="myModal_annotation" tabindex="-1" role="dialog" aria-labelledby="myModalLabel" aria-hidden="true"> <div class="modal-dialog" style="width:300px;"> <div class="modal-content"> <div class="modal-body"> <button type="button" class="close" data-dismiss="modal" aria-hidden="true">× </button> <span class="annotation_alert"></span> </div> </div> </div> </div> <br> <script type="text/javascript" src="src/js/child_nav.js"></script> <div id="autoHeightDiv"></div> <div class="footLineGray" style="border:none;"></div> <div class="lineWhite" style="border:none;"></div> <div class="webFoot"> <div class="foot middle" style="text-align:center;padding-right:10px;;padding-top:19px;background:white;border:none;"> 北京卡尤迪生物科技股份有限公司 © 2022-2023. </div> </div> <script> $(function () { $("[data-toggle='tooltip']").tooltip({html : true }); }); $(function() { $('#rct_show_id').click(function() { $('.rct_class').show() $('.entity_class').hide() $('#rct_show_id').hide() $('#rct_hide_id').show() }) $('#rct_hide_id').click(function() { $('.rct_class').hide() $('.entity_class').show() $('#rct_show_id').show() $('#rct_hide_id').hide() }) }) </script> <script> $(function () { $("[data-toggle='popover']").popover({html:true,trigger:'hover click'}); }); </script> <script type="text/javascript" src="src/js/child_nav.js"></script> <script type="text/javascript" src="src/js/clickx.js"></script> <script src="end.js"></script> </body> </html>