Text-mining and information-retrieval services for molecular biology.

Text-mining in molecular biology -- defined as the automatic extraction of information about genes, proteins and their functional relationships from text documents -- has emerged as a hybrid discipline on the edges of the fields of information science, bioinformatics and computational linguistics. A range of text-mining applications have been developed recently that will improve access to knowledge for biologists and database annotators.

The use of large-scale experimental techniques and bioinformatic tools has increased the pace at which biologists produce relevant information. This also promotes the growth of the scientific literature, which contains information on those experimental results in the form of free text that is structured in a way that makes it straightforward for humans to read but more difficult for computers to interpret automatically. As a consequence, there is increasing interest in methods that can handle collections of biological texts. Such methods include systems that efficiently retrieve and classify documents in response to complex user queries, and beyond this, systems that carry out a deeper analysis of the literature to extract specific associations, such as protein-protein interactions and protein functions. This deeper analysis is called text-mining. The complex and concise nature of the scientific literature means that the use of text-mining tools developed for generic texts is often impractical; a set of freely available text-mining applications adapted to the needs of biology have been developed, however, and some of them are now available for practical use. In parallel, a number of strategies for evaluating text-mining applications have appeared, with the goal of assessing and improving the field by providing datasets that can be used for training and testing applications.

Finding relevant articles

Throughout the last decade, the amount of electronically accessible textual material has been growing exponentially. Internet-based technologies exploit the availability of these large collections of documents for the development of information-retrieval systems. Currently, biologists and bioinformaticians take advantage of those tools, not only when searching generic documents such as news articles using search engines such as Alta Vista [1] and Google [2], but especially when querying publications specific to biomedicine, for example those stored in PubMed [3,4]. The range of community-wide genome projects, for which Internet-based information exchange is crucial, together with the heavy use of biology databases through web-based tools, means that natural language processing (NLP) techniques could be useful. NLP is based on the use of computers to process language, and it includes techniques developed to provide the basic methodology required for automatically extracting relevant functional information from unstructured data, such as scientific publications. Information retrieval and NLP systems are soon likely to become important not only for extracting information but also for assisting in various aspects of research such as the discovery of new facts, the interpretation of findings, and the design of experiments.

One of the first steps when handling textual data is the extraction of relevant documents from a large collection. This process is commonly known as information retrieval. In the case of indexed web pages, powerful search engines such as Google [2] return a ranked list of documents relevant to a given user search. There are two basic search strategies: query-based and document-based searches. In query-based searches, documents are returned that contain certain user-specified combinations of keywords. As some words - 'stop words' such as 'and', 'if' and 'the' - are found at a high frequency within most documents and thus display a low information content, they are often excluded during the retrieval process. Keywords may be combined by Boolean operators, such as AND, OR and NOT. The second type of retrieval, document-based searching, aims to return a ranked list of documents similar to a given query document as a whole, rather than to a combination of a few keywords. The most widely used retrieval tool in molecular biology is Entrez [3,4], the PubMed information retrieval system provided at the US National Center for Biotechnology Information (NCBI) [5]. It supports basic keyword and Boolean query-based searches, as well as document-based searches to return all abstracts that are similar to a given document. The popular search engine Google [2] has recently incorporated a search tool specific to the academic literature, Google Scholar [6,7], for the retrieval of scientific articles, reports and books. The ranking of the returned hits is mainly based on the extent to which documents are connected by citations and web links. Other scientific literature databases and search engines include Crossref Search [8], which enables searches of the full content provided by a set of publishers, and the Nature Publishing Group search engine [9], which allows advanced search strategies.

Although these tools are useful for many tasks, it is time-consuming to use them for efficient searches and article selection, and such functions must be repeated periodically to keep knowledge up-to-date. As PubMed already contains over 15 million citations of biomedical articles [4] and is steadily growing (more than 450,000 articles are added every year [10]), services that periodically retrieve relevant articles and automatically alert the user have been implemented. Among those systems, known as selective dissemination of information (SDI) services, are My NCBI (formerly PubMed Cubby) [4,11], BioMail [12] and PubCrawler [13,14] (these and other services described in this article are listed in Table 1). These, together with some commercial tools, have been evaluated independently [15], showing that the combined use of different SDI systems results in useful automated searching.

Table 1

Biomedical text-mining resources, servers and programs

Name	Description	URL	Published reference or URL*
Abbreviation Server	Biomedical abbreviation server		[35]
AbGene	Protein name tagger		[29]
ABNER	Protein/Gene/DNA/RNA/cell tagger		[31]
AliasServer	Protein alias handler		[37]
ARGH	Biomedical acronym resolver		[88]
ARROWSMITH	Extended MEDLINE search tool		[84]
BioMail	PubMed updating and alerting service		[12]
BioRAT	Biology information extraction tool		[81]
BITOLA	Literature-based biomedical discovery system		[86]
Chilibot	Relationship extraction		[57]
CrossRef Search	Full content search engine		[8]
GAPSCORE	Protein name tagger		[23]
Geisha	Text-mining tool to assist microarray analysis		[67]
GeneScene	Information extraction for regulatory pathways		[59]
GOAnnotator	Annotation extraction from literature		[51]
Google Scholar	Scholar literature search engine		[6]
iHOP	Information on hyperlinked proteins		[40]
iProLINK	Protein annotation and tagging		[55]
KAT	Annotate proteins from scientific references		[52]
KeX	Protein name tagger		[33]
KinasePathway database	Tool for extraction of protein, gene and compound interactions from text		[46]
MedBlast	Document retrieval for sequences		[63]
MedMiner	Extraction of sentences relevant to genes		[69]
microGENIE	Text-mining for microarrays		[76]
My NCBI	PubMed updating and alerting service		[11]
NDPG	Scores the literature based coherence of gene clusters	None	[66]
NLProt	Protein name tagger		[25]
NPG search engine	Nature Publishing Group search engine		[9]
PreBIND	Classifier of protein interaction documents		[44]
PubCrawler	PubMed updating and alerting service		[13]
PubGene	Text-mining tool for microarrays		[72]
PubMatrix	Multiplex literature mining tool		[74]
PubMed Entrez	Biomedical citation retrieval system		[3]
Relationship Extractor	Biomedical relationship extractor		[90]
SAWTED	Text-enhanced remote homolog detector		[61]
Scopus	Scientific literature database and search		[93]
Textpresso	C. elegans literature information retrieval and extraction tool		[48]
XplorMed	Explores bibliographic MEDLINE searches		[91]
Yapex	Protein name tagger		[27]

An overview of some of the available text-mining, information-extraction, information-retrieval and selective dissemination of information services currently available. *References to articles describing each tool are given; where no article has been published, the reference is to the URL.

The first step in text mining: identification of biological entities

Biological research is name-centered: proteins are referred to in free text by their names or symbols rather than using the unambiguous identifiers provided by annotation databases (such as SwissProt accession numbers [16]). Identifying mentions of proteins and genes unambiguously within free text is a fundamental step for the later extraction of functional attributes of these entities. Unfortunately this is a difficult process, partly because of the complex nature and usage of gene and protein names. Genes and proteins maybe referred to in free text in a range of different ways: as full names (for example, porin), as symbols (the Saccharomyces cerevisiae gene POR1), and also through typographical variants (POR-1). Many genes also have several synonyms (such as OMP2 for POR1), or the gene name may be ambiguous [17] and refer to words that also have a different meanings depending on the context (for example, big brain, the full name for the Drosophila melanogaster gene bib, could also be an anatomical description). Furthermore, it has been suggested that errors in gene names might be introduced automatically by certain applications in bioinformatics [18].

In the NLP field, the identification of entities in free text is known as named-entity recognition (NER). To identify biological entities such as genes, proteins and drugs automatically and unambiguously within free text, over 50 information-extraction and text-mining tools have recently been implemented, and two community-wide evaluations have been carried out [19,20]. The top left of Figure 1 shows nine existing NER applications for biology that are provided via an online server or are directly downloadable. Note that the average recovery of biological entities from free text by 15 NER tools was 80%, and the results had an accuracy of 80% [21]; these figures are significantly lower than in the case of entities found in documents from fields such as economics, which demonstrates the complex nature of protein names.

Figure 1

An overview of biological natural language processing (BioNLP) and text-mining applications for biology. The major topics are represented by the inner circle of seven approaches, and the corresponding applications are given in the outer layers of boxes. Most of the tools are available online or for download. Some applications could be classified into multiple topics; they are shown here associated with one of their most significant topics. For instance, most of the text-mining applications (that is, the applications that are not simply for article retrieval) have integrated modules for named entity recognition (NER), and selective dissemination of information (SDI) services often use automated Boolean queries for article retrieval. References and URLs for each application, where available, are given in Table 1.

Proteins and genes are characterized within biological databases through unique identifiers; each identifier is associated with its corresponding protein or nucleotide sequence and functional descriptions. The automatic recognition of entities such as genes and proteins in free text is insufficient if it is not linked to the corresponding database identifiers. Distinguishing between the use of protein names and protein-family names constitutes a serious obstacle in the task of highlighting protein entities in free text, as text passages sometimes refer to the general properties of protein families and at other times to the properties of individual proteins.

Different research communities have addressed the issue of named-entity recognition in biology in different ways. The NLP community has typically tried to identify names by analyzing the syntactic structure of sentences, making use of information about parts of speech in a sentence and the syntactic roles of words, whereas bioinformaticians have instead explored the identification of variants of the names contained in databases, even adapting standard bioinformatics algorithms such as BLAST to the problem of protein-name identification [22]. Neither of these two strategies seems to be efficient by itself, and many intermediate combinations are therefore appearing, including the following examples. GAPSCORE [23,24] is an easy-to-use online tool for detecting protein and gene names within free text (a 'protein tagger'). The text to be analyzed can be pasted into an online form and submitted to the server, which returns a list of the words observed in the document and a statistical quality score that indicates how probable it is that the each word represents a gene or protein name. Another online protein tagger is NLProt, developed at Columbia University [25,26]. NLProt is based on a machine learning technique called support vector machines (SVMs) and allows protein identification either in a submitted text or in the text corresponding to a list of submitted PubMed article identifiers. Additional protein taggers include Yapex [27,28], also available online, and three downloadable tools, AbGene [29,30], ABNER [31,32] and KEX [33,34]. Abbreviations or acronyms are often used as a shorter form to refer to gene names in articles; the Abbreviation Server [35,36] developed at Stanford University allows a similar search strategy to that used by GAPSCORE to be applied to biomedical abbreviations such as gene symbols. Finally, the AliasServer [37,38] helps in linking the various aliases of a given gene through different biological databases for various species.

One of the main challenges when linking protein names to database entries is distinguising between proteins that have the same names but belong to different genomes - a process called inter-species gene disambiguation. This is especially cumbersome in the case of mouse and human genes; the same gene symbol is often used in both species and both names are often mentioned in the same textual passage. The complex nature of protein- and gene-name identification is reinforced further by the dynamic nature of gene-name usage and name creation, with official gene names being changed and new synonyms being created [39]; it is clear that static approaches and dictionaries will not be sufficient for solving the problem.

One step further: mining interactions and relations

Although the identification of biological entities is a crucial step, in practice it is the extraction of associations between proteins and their functional features that poses an interesting biological problem. Several systems have been constructed for extracting annotations of genes and proteins automatically and for detecting protein-protein interactions and regulatory pathways. Protein-protein interactions have attracted particular interest in the light of recent developments in high-throughput proteomics. One system that extracts annotations and detects interactions is the iHOP system that we have implemented at the Spanish National Biotechnology Center [40]. This facilitates the direct linking of information in the INTACT [41] protein-interaction database with corresponding bibliographic references (Figure 2). As well as highlighting direct associations between genes and functional descriptions, iHOP also includes advanced search modes for discovery and visualization of literature-based protein-interaction networks for a range of organisms, including human, mouse and yeast [42]. The basic approach followed by iHOP is protein-centric: it arranges relevant sentences from the literature around protein names, and the use of co-citation of protein names in each sentence facilitates navigation through the dispersed literature relevant to a particular protein. As a result, users can successively explore the functions of related proteins by building virtual protein-relation networks (Figure 2c). The iHOP system is based on the ideas previously developed for the SUISEKI knowledge-discovery system [43].

Figure 2

Basic steps in the use of the iHOP text-mining tool [40], illustrated with screenshots [42]. For a given query (for example, the protein symbols (a) Wnt-1 or (b) LEF-1), all the sentences mentioning the name are retrieved from PubMed. These sentences also contain mentions of other proteins, which are highlighted and which might show associations with the query protein (see the magnified area in (b)). Functional terms (such as 'target' and 'complexes' and interaction verbs (such as 'activated' and 'stabilizes') are in bold. (c) By clicking on the 'Gene model' link in the left panel in (a,b), interaction networks of proteins that co-occur in sentences with the query proteins can be displayed.

Some other text-mining applications include PreBIND [44,45], developed to assist in the extraction of protein-protein interactions; the KinasePathway database text-mining system, which extracts interactions between proteins, genes and compounds [46,47]; and Textpresso [48,49], an information-retrieval and extraction tool developed for the Caenorhabditis elegans literature in the context of the model-organism database WormBase [50]. Textpresso defines 33 categories of word describing entities or relationships - such as genes, pathways, or regulation - and integrates this 'Textpresso Ontology' with a text-mining system for searching the C. elegans literature. Among the text-mining services available online that focus on automatic annotation extraction are GOAnnotator, which provides associations between protein names and Gene Ontology terms [51]; KAT [52,53], a system for deriving terms relevant to annotations such as SwissProt keywords and Gene Ontology terms [54] from PubMed abstracts for a given query protein; and the iProLINK tool [55,56], which performs automated extraction of annotations for given protein names and provides information related to the organisms in which proteins are found and the protein families of which they are members. Figure 1 and Table 1 provide an overview of the different systems currently available.

A system with a special focus on the extraction of relationships between genes, proteins and other information is Chilibot ([57,58]; user registration is required before running queries); it allows searches using gene symbols and keywords, and the color-coded output provides information about gene-expression levels when available. The extraction of complex relationships can be handled by GeneScene [59,60], a toolkit that provides visualization and navigation facilities for exploring regulatory networks; the tool currently provides information only on the literature on yeast and on the p53 tumor suppressor and the AP1 transcription factor.

Some attempts have been made to merge text-mining methods and bioinformatic methods involving sequence analysis into a single system. The integration of functional information extracted by NLP algorithms with standard bioinformatic methods such as sequence-comparison techniques has been exploited by the Structure Assignment With Text Description (SAWTED) system [61,62], which can be tested online. It combines a document-comparison algorithm called a 'Vector-cosine model' with the PSI-BLAST sequence retrieval method, which is especially useful for detecting sequences that are distantly related. Another strategy that makes use of sequence information and free text is MedBlast [63,64]; using the web-based interface of MedBlast, for a given query sequence and optional additional keywords the system returns articles related to the protein corresponding to the query sequence.

Text mining and large gene collections

Technical advances in molecular biology mean that large collections of genes are nowadays often studied simultaneously using genomic approaches. Using conventional information retrieval to link these genes with the associated literature is not efficient, and a large list of irrelevant documents can be returned. For example, microarray experiments result in groups of genes with particular expression patterns; to interpret these groups in terms of the underlying biological meaning, information is needed not only on each individual gene but also on commonalities among the whole group. The functional information is commonly extracted from databases such as SwissProt [16] or GO [65], which in turn are nourished by extracting relevant functional features from the literature.

A number of text-mining methods have been developed for linking groups of genes found in microarrays and other experiments directly and automatically with information contained in biomedical article databases. The neighbor divergence per gene (NDPG) approach [66] uses the literature to score the functional coherence of gene clusters. GEISHA [67,68] automatically mines the literature for functional terms associated with gene groups and carries out a statistical analysis of the significance of those terms. Among the available online tools for assisting in interpreting microarray data are MedMiner [69,70], which can be used to filter and organize information from free text obtained from automatic PubMed [4] and GeneCard [71] searches and PubGene [72,73] which has additional visualization capabilities for displaying network information and pathway mapping. The analysis of frequency matrices of term cooccurrences of two lists of keywords is the basis of the PubMatrix system [74,75], which can be used online after registering. Finally, microGENIE [76] enables semi-automatic queries of very large collections of genes (UniGene and SwissProt gene names and GenBank accession numbers) in PubMed to speed up the retrieval of relevant articles. It is important to realize that existing text-mining technologies in biology are focused on identification and linking of functional information of proteins in free text, they are currently not providing automatically generated summaries of biologically relevant information.

Towards knowledge discovery

The field of 'BioNLP' - text mining and information extraction for molecular biology - is very recent, but the existing applications are improving steadily. This is partly because of newly available resources, such as collections of annotated documents suitable for training new systems (for example, the GENIA [77] corpus and the BioCreative [19] corpus). The improvement also reflects the effect of community-wide assessments such as the BioCreative contest [19] and the KDD challenge cup [78], which enable evaluation of the efficiency of different methodologies, and the genomics track of the Text Retrieval Conference (TREC) workshops [79,80], a forum for developing solutions to information-retrieval and document-classification tasks in biology. The development of controlled, computer-readable vocabularies (ontologies), dictionaries, and functional keywords (Gene Ontology concepts [54] and SwissProt keywords [16]) defining relevant biological aspects of proteins have also been valuable for text-mining tools. Because of the restricted availability of full-text articles most of the existing text-mining systems for biology are centered on the analysis of abstracts, but changes in publishing policy and increasing access to repositories of whole articles make mining of full text a likely development in the near future. Some initiatives in this direction have been started already, for example the BioRAT system [81,82], which processes full-text articles so as to identify target facts.

Perhaps the most likely future developments will be the construction of networks and interactions for discovering new relationships through intermediate entities, followed by the proposal of new functions - this process is referred to as 'knowledge discovery'. Several exploratory attempts have been made to develop knowledge-discovery systems, but they are not yet of general practical use. Our SUISEKI system [83], for instance, extracts indirect relationships between proteins through associations with intermediate proteins in text. Two online tools that directly address the difficulty of making knowledge-discovery practical are ARROWSMITH [84,85] and BITOLA [86,87]. ARROWSMITH [84,85] aims to discover indirect relations between two entities that are not directly connected in the literature; the indirect relationship can be a substance or disease condition. BITOLA [86,87] is a biomedical discovery-support system with a focus on the discovery of disease candidate genes, taking advantage of Medical Subject Heading (MeSH) terms.

Undoubtedly, the development of text-mining applications specific for biology is the only way to cope with the increasing amount of free textual data produced in this field. The increasing interest of users in efficiently retrieving and extracting relevant information, the need to keep up with new discoveries described in the literature or in biological databases, and the demands posed by the analysis of high-throughput experiments, are the underlying forces motivating the development of text-mining applications in molecular biology. Those technologies should provide the foundation for future knowledge-discovery tools able to identify previously undiscovered associations, something that will assist in the formulation of models of biological systems.