GeneSetDB: A comprehensive meta-database, statistical and visualisation framework for gene set analysis.

Most "omics" experiments require comprehensive interpretation of the biological meaning of gene lists. To address this requirement, a number of gene set analysis (GSA) tools have been developed. Although the biological value of GSA is strictly limited by the breadth of the gene sets used, very few methods exist for simultaneously analysing multiple publically available gene set databases. Therefore, we constructed GeneSetDB (http://genesetdb.auckland.ac.nz/haeremai.html), a comprehensive meta-database, which integrates 26 public databases containing diverse biological information with a particular focus on human disease and pharmacology. GeneSetDB enables users to search for gene sets containing a gene identifier or keyword, generate their own gene sets, or statistically test for enrichment of an uploaded gene list across all gene sets, and visualise gene set enrichment and overlap using a clustered heat map.

Introduction

With the rapid development of high-throughput measurement technologies such as next generation sequencing and microarrays, biologists can easily analyse cells and tissues on a whole-genome scale. To make biological sense of the results of these analyses, biologists usually need to comprehensively interpret the biological meaning of gene lists. These gene lists may for example represent mRNAs co-regulated by a drug or experimental condition. Determining whether the members of a gene list share biological features has been made more important by the recent realisation that the expression of transcription factor target sets [1] or RNAs encoding proteins of similar function [2,3], are more often correlated than would be expected by chance. These correlated groups of functionally related RNAs appear to be tissue-specific and conserved across evolution [4,5]. For these reasons, genomics researchers find it valuable to analyse gene sets as well as individual genes [6,7] and gene set analysis (GSA) is frequently employed when interpreting genomic data. GSA statistically assesses whether experimentally identified gene lists have a larger intersection with biologically relevant gene sets than expected due to chance.

A number of GSA tools have been proposed and used successfully over the past decade [7]. However, most of these tools focus on identifying or visualising statistically significant gene set enrichment in one gene set database at a time. There are a fewer published reports of simultaneous analysis of multiple biologically distinct gene sets databases and their cross-visualisation. This is despite the fact that the gene sets employed critically limit the results of any GSA, and affect these results just as much as the statistical analysis methodologies used [7]. At present, Gene Ontology (GO) [8] is used as a “gold standard” gene set by many GSA tools [7]. However, many biologists would prefer to perform GSA using a single meta-database that allowed the statistically robust interrogation of GO and many other types of gene sets databases simultaneously, followed by a cross-visualisation of the results. A few GSA meta-databases (some embedded within specific GSA tools) have already been generated, like ConceptGen [9], DAVID [10], GATHER [11], GeneSigDB [12], MSigDB [13], and WhichGenes [14]. These are very useful resources/tools for GSA. In addition to these tools, there are several commercial products that allow variations of GSA but due to their cost these are not available to many academic researchers. However there remain no databases that are specifically designed for GSA and provide a searchable interface with full coverage of the available pathway, medical and pharmacological datasets.

Therefore, in order to allow comprehensive GSA across multiple databases of different types, we have constructed GeneSetDB, a comprehensive meta-database integrating 26 public databases. GeneSetDB allows users to identify and download the intersection between an individual gene or a gene list and gene sets in 26 databases. Moreover it allows users to statistically analyse the degree of enrichment of their gene list in gene sets and cross-visualise this enrichment in a clustered heatmap based on the overlap between the enriched gene sets.

Materials and methods

Gene set building

Data was downloaded from each source database with permission. Source databases were classified into five subclasses based on the database content: Pathway, Disease/Phenotype, Drug/Chemical, Genes Regulation and Gene Ontology (Table 1). Since different gene/protein identifiers are used in each database, Entrez gene ID was used as a representative identifier in GeneSetDB. The Bioconductor (http://www.bioconductor.org/) or biomaRt [38] bioinformatic resources were used in this identifier conversion. GeneSetDB is based primarily on human data; however, it supports mouse and rat gene lists by using the information in NCBI HomoloGene.

Table 1

Sources databases included in GeneSetDB.

Subclass Name	Sources database	Reference/URL
Pathway	Biocarta	http://www.biocarta.com
	EHMN	[15]
	HumanCyc	[16]
	INOH	[17]
	NetPath	[18]
	PID	[19]
	Reactome	[20]
	SMPDB	[21]
	Wikipathways	[22]
Disease/Phenotype	CancerGenes	[23]
	HPO	[24]
	KEGG Disease	[25]
	MethCancerDB	[26]
	MethyCancer	[27]
	MPO	[28]
	SIDER	[29]
Drug/Chemical	CTD	[30]
	DrugBank	[31]
	MATADOR	[32]
	STITCH	[33]
	T3DB	[34]
Gene Regulation	MicroCosm Targets	[35]
	miRTarBase	[36]
	Rel/NF-κB target genes	http://bioinfo.lifl.fr/NF-KB
	TFactS	[37]
GO	Gene Ontology	[8]

Enrichment analysis

In general, enrichment analysis/overrepresentation analysis is the statistical assessment of whether input gene list has a larger intersection with biologically relevant gene sets than expected by chance. GeneSetDB uses the hypergeometric distribution to calculate the probability of overrepresentation (shown as P-value). The calculation of this P-value is followed by multiple testing correction use the Benjamini and Hochberg method [39], the result of which is shown as a false discovery rate (FDR). Gene sets with less than 10 or more than 500 genes are not used in the enrichment analysis. The reference (background) gene set was the set of Entrez gene IDs that have at least one annotation in the union of the gene sets used in the analysis (e.g. Subclass Pathway, GO, etc.). The gene sets shown on the results page can be filtered based on FDR. GeneSetDB allows the use of several types of identifier for the input gene list, including official gene symbols and commercial microarray probe IDs. Each input identifier is converted to an Entrez gene ID using the Bioconductor or biomaRt resources [38]. GeneSetDB allows visualisation of gene set overlap with the submitted gene list in a clustered heatmap. The heatmap colors show the proportion of overlap between the gene sets.

Implementation

All gene sets are stored in a MySQL database management system, and the web interface is implemented using Apache, PHP, Javascript and HTML. The statistical package R is used for statistical calculations and for drawing clustered heatmaps. To use GeneSetDB users can paste gene lists into the web interface or upload a gene list text file. Results can be downloaded to the user’s computer as a file or visualised in tables through a web browser. GeneSetDB is scriptable over the internet to allow its use by bioinformaticians from within other software such as R (see the Help page on GeneSetDB web site for mode details). The database structure and analysis scheme is visually described in Fig. 1.

Fig. 1

Database structure and analysis scheme. The gene sets are downloaded from source databases and deposited into a MySQL database. All gene identifiers of both the source databases and the input gene list are converted into Entrez Gene ID using Bioconductor or biomaRt.

Results

Currently, GeneSetDB contains nine Pathway databases, seven Disease/Phenotype databases, five Drug/Chemical databases, four Gene Regulation databases and GO (Table 1). Table 2 shows the breadth of coverage of GeneSetDB compared to examples of currently available gene set meta-databases. This comparison suggests that GeneSetDB may cover a wider variety of biomedical features (especially human disease pathways and pharmacology) than some other databases. GeneSetDB also allows researchers to assemble their own original gene sets from these pre-existing gene sets, similar to the tools Gene Set Builder [40] and WhichGenes [14]. When performing enrichment analysis with GeneSetDB, users can choose combinations of one or more database subclasses (Pathway, Disease/Phenotype, Drug/Chemical, Gene Regulation and Gene Ontology), or any of the 26 individual databases as backend annotation depending on their research question (Fig. 2). Users can also generate a heatmap based on the intersection between the enriched gene sets with one another.

Table 2

Feature comparison between GeneSetDB and existing databases.

Feature	GeneSetDB	ConceptGen [9]	DAVID [10]	MSigDB [13]	WhichGenes [14]
Pathway database #	9	3	6	7	3
Disease/Phenotype database #	7	1	2	2a	4
Drug/Chemical database #	5	1	0	1a	1
Whole data downloadable	Yes	No	Yes	Yes	No
Making of original gene set	Yes	No	No	Yes	Yes
Gene/gene set intersection map	No	Yes	Yes	Yes	No
Gene set/gene set intersection map	Yes	No	No	No	No
Organismsb	Hs, Mm, Rn	Hs, Mm, Rn	Over 65 000 species	Dr, Hs, Mm, Mmu, Rn	Hs, Mm

Including original curated datasets.

Dr: Danio rerio, Hs: Homo sapiens, Mm: Mus musculus, Mmu: Macaca mulatta, Rn: Rattus norvegicus.

Fig. 2

Top screen of GeneSetDB. Users can query gene names or biological terms in “gene/gene set search” and “gene list in enrichment analysis” modes. Users can also select a subclass of gene sets and database names if they wish to conduct a focused analysis. In the enrichment analysis mode, GeneSetDB allows the use of several gene identifier types (e.g. commercial probe IDs). FDR can be used to filter the gene sets shown in the results.

Description of the gene/gene set search function

GeneSetDB can be queried by gene name (e.g. FOXL2) or by a descriptive term, such as a disease, drug name or general biological term (e.g. melanoma, gleevec, cell cycle). The user can select subclass of database if their interest is specific (Fig. 2). For example, if they want to identify drugs targeting a specific gene, they can select the “Drug/Chemical” subclass in gene/gene set search. GeneSetDB provides three search functions: “Exact match”, “Phrase search”, and “Similar pattern search”, which allow for broader keywords’ matches. These functions are available only for Gene/Gene Set Search. The retrieved result lists the identifier and types of gene set, their sources and their constituent genes (Fig. 3). A gene set name is hyperlinked to the original data source if the original database’s identifier is available. The result of gene/gene set searches can be downloaded as tables as well as viewed as tables in a web browser window. This enables users to make and save original gene sets, which can be used as the backend annotation in their own enrichment analysis.

Fig. 3

Gene/gene set search result. GeneSetDB shows subclass of gene set, gene set name, source database, the number of genes in corresponding gene set and gene names (first 10 genes). User can see all gene names in a downloadable text file. A gene set name is hyperlinked to the original database if the original database’s identifier is available.

The enrichment analysis function: an example where GeneSetDB is used to identify drugs related to the disease Systemic Lupus Erythematosus

New drugs are urgently needed for the treatment of autoimmune diseases like Systemic Lupus Erythematosus (SLE) [41]. Therefore we used the enrichment analysis function in GeneSetDB to identify drugs that target molecules and molecular pathways associated with SLE. We used a gene list from Alcorta et al.’s Affymetrix microarray analysis, which compared mRNA abundance in the leukocytes of patients with SLE to mRNA abundance in the leukocytes of healthy donors [42]. This experiment identified 205 SLE-associated Affymetrix probe sets, which we found mapped to 153 unique mRNAs using the NetAffix Annotation Update 32 (Supplementary Table 1). These 153 SLE-associated genes were investigated by enrichment analysis using the “Drug/Chemical” subclass of GeneSetDB. This identified 10 drug/chemical-associated gene sets for which the 153 SLE signature genes were significantly enriched (FDR < 1e-05; Fig. 4). Seven out of these 10 drugs/chemicals were related to autoimmune disease (data not shown). GeneSetDB generated a heatmap with clustering based on the proportion of overlapping genes between the enriched gene sets (Fig. 4). This heatmap contained two major clusters – one cluster included 2-xylene, ethylbenzene, trichloroethylene and toluene, which are environmental factors triggering SLE while the other cluster included chlorophyllin, plant extracts, polyinosinic-polycytidylic acid (poly(I:C)) and Zidovudine, which are associated with host response to viral infections. This type of analysis may potentially be useful for generating hypotheses for drug repositioning projects.

Fig. 4

Enrichment analysis result of SLE signature genes. Enrichment analysis identifies the gene sets meeting a user-assigned FDR and generates a clustered heatmap based on the overlap proportion of genes between the gene sets. The overlap between the top 30 gene sets ranked by FDR, and between these gene sets and the submitted gene list, are shown in a clustered heatmap.

The enrichment analysis function: an example where GeneSetDB is used to identify the biological meaning of gene lists generated in human endothelial cell apoptosis microarray experiments

We applied GeneSetDB to a list of 79 Affymetrix probe sets that were differentially expressed when endothelial cells (EC) were induced to undergo apoptosis [43]. When this probe set list was analysed by our research group several years ago using a combination of the GO and manual annotation from the published literature, genes encoding proteins important for apoptosis, cellular stress, cell cycle, transcription, coagulation, and proteosomal degradation were identified [43]. When the gene list was re-analysed in September 2011 using the GATHER web tool [11] with default settings, additional associations were identified with cell signalling, NF-κB-related gene sets, and others (Supplementary Tables 2a–d). The GATHER web tool provided extremely useful information and was an easy-to-use screen for biological associations for this 79 probe list. When this gene list was re-analysed using GeneSetDB, we observed enrichment for many of the gene sets previously identified using manual annotation and GATHER analysis. However, GeneSetDB also identified additional biologically relevant gene set enrichment, especially related to drugs, molecular pathways and diseases (Supplementary Table 3). For example, as found with the previous manual literature and the GATHER analysis, GeneSetDB identified regulated activity of the NF-κB1 transcription. However, GeneSetDB highlighted several additional gene sets associated with upstream regulation of NF-κB activity including: TAK1, RANKL, TNF-α, Interleukin 1 and TNF-like death receptors. GeneSetDB also identified several drugs associated with EC proliferation and apoptosis including NF-κB activation inhibitor, thalidomide (a drug known to regulate apoptosis and the growth of new blood vessels [44]) and Vinblastine (a drug known to affect EC apoptosis [45]). Additional gene sets associated with canonical signalling pathways and diseases of blood vessels were also identified by GeneSetDB including the vascular endothelial growth factor (VEGF) pathway (VEGF is an important endothelial cell survival factor [46]), and abnormal neo-vascularisation.

Heatmap clustering using the “Pathway” subclass of GeneSetDB divided gene sets associated with EC apoptosis into two major groups, relevant to the cell cycle and relevant to signal transduction (Supplementary Fig. 1). In summary, GeneSetDB analysis provided detailed new leads about the molecules involved in EC apoptosis, diseases that may share these molecular relationships, and drugs that may modulate the apoptotic process.

Discussion

We suggest that GeneSetDB is a useful addition to available GSA tools, for both experimental and computational biologists. It is equally applicable to several omics technologies such as: expression microarrays, RNAseq and ChIP on Chip. It enables researchers to simultaneously compare experimentally-derived or in silico-derived gene lists to gene sets from 26 databases that cover five biological categories. Users can also choose to use only a subset of the gene sets contained in GeneSetDB, according to their research purposes, in order to focus on the most relevant results for their particular analysis, and in order to speed up the statistical calculations. GeneSetDB also includes novel clustered heatmap visualisation capabilities that facilitate visual understanding of the enrichment analysis. Computational biologists can script their access to GeneSetDB, and download the meta-gene sets related to their in silico work or to benchmark or validate additional GSA tools. We show that GeneSetDB identifies biologically relevant information about two gene lists that could not be identified using other currently available tools, and that it compliments these tools. However, there are limitations to using GeneSetDB. Since GeneSetDB particularly focuses on human disease and pharmacology, it may not be suitable for the study of microbiology or plant biology. Users having interest in such fields should consider addition GSA tools, which cover wide range of organisms (e.g. DAVID or others listed in [7]). As for the functional aspects, GeneSetDB uses the hypergeometric distribution to calculate the significance of overrepresentation. This type method requires pre-selection of genes (e.g. differentially expressed genes between disease and control samples), and the results of course depend on the criterion used for pre-selection. If users do not want to pre-select genes, Gene Set Enrichment Analysis (GESA), which does not require pre-selection of genes [47], integrated with MSigDB will be especially useful.

In conclusion, although GSA is nowadays a standard omics analysis tool, the wide range of gene set databases that are now publically available is poorly exploited due to our limited ability to analyse these databases together, especially databases related to drugs and human disease. This limitation is especially important for biological researchers who do not have access to expensive commercial tools, and for researchers who focus on drugs/diseases/phenotypes. GeneSetDB was designed a meta-gene set database for use in GSA to overcome these limitations. It is freely available for academic purposes and will be actively maintained. In future work we plan to integrate GeneSetDB with additional gene sets including TRANSFAC and mutation/polymorphism databases.

Availability and requirements

All gene sets, analysis functions and downloadable data in GeneSetDB are available, for academic purposes only, at: http://genesetdb.auckland.ac.nz/haeremai.html.