Literature DB >> 25679508

Integrative multi-omics module network inference with Lemon-Tree.

Eric Bonnet1, Laurence Calzone1, Tom Michoel2.   

Abstract

Module network inference is an established statistical method to reconstruct co-expression modules and their upstream regulatory programs from integrated multi-omics datasets measuring the activity levels of various cellular components across different individuals, experimental conditions or time points of a dynamic process. We have developed Lemon-Tree, an open-source, platform-independent, modular, extensible software package implementing state-of-the-art ensemble methods for module network inference. We benchmarked Lemon-Tree using large-scale tumor datasets and showed that Lemon-Tree algorithms compare favorably with state-of-the-art module network inference software. We also analyzed a large dataset of somatic copy-number alterations and gene expression levels measured in glioblastoma samples from The Cancer Genome Atlas and found that Lemon-Tree correctly identifies known glioblastoma oncogenes and tumor suppressors as master regulators in the inferred module network. Novel candidate driver genes predicted by Lemon-Tree were validated using tumor pathway and survival analyses. Lemon-Tree is available from http://lemon-tree.googlecode.com under the GNU General Public License version 2.0.

Entities:  

Mesh:

Year:  2015        PMID: 25679508      PMCID: PMC4332478          DOI: 10.1371/journal.pcbi.1003983

Source DB:  PubMed          Journal:  PLoS Comput Biol        ISSN: 1553-734X            Impact factor:   4.475


This is a PLOS Computational Biology Software Article

Introduction

Recent years have witnessed a dramatic increase in new technologies for interrogating the activity levels of various cellular components on a genome-wide scale, including genomic, epigenomic, transcriptomic, and proteomic information [1]. It is generally acknowledged that integrating these heterogeneous datasets will provide more biological insights than performing separate analyses. For instance, in 2005, Garraway and colleagues combined SNP-based genetic maps and expression data to identify a novel transcription factor involved in melanoma progression [2]. More recently, international consortia such as The Cancer Genome Atlas (TCGA) or the International Cancer Genome Consortium (ICGC) have launched large-scale initiatives to characterize multiple types of cancer at different levels (genomic, transcriptomic, epigenomic, etc.) on several hundreds of samples. These integrative studies have already led to the identification of novel cancer genes [3, 4]. Among the many ways to approach the challenge of data integration, module network inference is a statistically well-grounded method which uses probabilistic graphical models to reconstruct modules of co-regulated genes (or other biomolecular entities) and their upstream regulatory programs and which has been proven useful in many biological case studies [5, 6]. The module network model was introduced as a method to infer regulatory networks from large-scale gene expression compendia [5] and has subsequently been extended to integrate eQTL data [7, 8], regulatory prior data [9], microRNA expression data [10], clinical data [11], copy number variation data [12] or protein interaction networks [13]. The original module network learning algorithm depended on a greedy heuristic, but subsequent work has extended this with alternative heuristics [14], Gibbs sampling [15] and ensemble methods [16]. Module network inference can be combined with gene-based network reconstruction methods [17, 18] and recently a method has been developed to reconstruct module networks across multiple species simultaneously [19]. This methodological and algorithmic work has complemented studies that were solely focused on applying module network methods to provide new biological and biomedical insights [20-27]. Although the success of the module network method is indisputable, the various methodological innovations have been made available in a bewildering array of tools, written in a variety of programming languages, and, when source code has been released, it has never been with an OSI compliant license (Table 1). Here we present Lemon-Tree, a ‘one-stop shop’ software suite for module network inference based on previously validated algorithms where a community of developers and users can implement, test and use various methods and techniques. We benchmarked Lemon-Tree using large-scale datasets of somatic copy-number alterations and gene expression levels measured in glioblastoma samples from The Cancer Genome Atlas and found that Lemon-Tree compares favorably with existing module network softwares and correctly identifies known glioblastoma oncogenes and tumor suppressors as master regulators in the inferred module network. Novel candidate driver genes predicted by Lemon-Tree were validated using pathway enrichment and survival analysis.
Table 1

Survey of module networks software tools, in chronological order by their first release date.

SoftwareLanguageI/OSourceDataURLYear
GenomicaJavagnom http://genomica.weizmann.ac.il 2003
GeronemoJavagnom, e http://ai.stanford.edu/~koller/index.html 2006
LemoneJava/Matlabcyes 1 m, mi http://bioinformatics.psb.ugent.be/software/details/Lemone 2007
LirnetMatlabcyes 2 m, e http://homes.cs.washington.edu/~suinlee/lirnet 2009
CONEXICJavacnom, c http://www.c2b2.columbia.edu/danapeerlab/html/conexic.html 2010
PMNUnix binarycnom, p http://www.compbio.cs.huji.ac.il/PMN 2010
ARBORETUMCcyes 2 m-s http://pages.discovery.wisc.edu/~sroy/arboretum 2013
MERLINCcyes 2 m http://pages.discovery.wisc.edu/~sroy/merlin 2013
Lemon-TreeJavacyes 3 m, mi, e, c, any http://lemon-tree.googlecode.com 2014

I/O: g, graphical user interface; c, command line. Supported data integration: m, mRNA; mi, microRNA; e, eQTL; c, CNV; p, protein interactions; m-s, mRNA multiple species; any, any combination of discrete or continuous data types measured on the same samples.

(1) Not OSI compliant.

(2) No license provided.

(3) GPL license.

I/O: g, graphical user interface; c, command line. Supported data integration: m, mRNA; mi, microRNA; e, eQTL; c, CNV; p, protein interactions; m-s, mRNA multiple species; any, any combination of discrete or continuous data types measured on the same samples. (1) Not OSI compliant. (2) No license provided. (3) GPL license.

Design and Implementation

Lemon-Tree is a platform-independent command-line tool written in Java which implements previously validated algorithms for model-based clustering [15] and module network inference [16]. The principal design difference between Lemon-Tree and other module network softwares (e.g. Genomica [5] or CONEXIC [12]) consists of the separation of module learning and regulator assignment. We have previously shown that running a two-way clustering algorithm until convergence, and thereafter identifying the regulatory programs that give rise to the inferred condition clusterings for each gene module results in higher module network model likelihoods and reduced computational cost compared to the traditional approach of iteratively updating gene modules and regulator assignments [14, 16]. Hence Lemon-Tree is run as a series of tasks, where each task represents a self-contained step in the module network learning and evaluation process and the output of one task forms the input of another (a work flow representation of the different steps is illustrated in Fig. 1):
Fig 1

Flow chart for integrative module network inference with Lemon-Tree.

This figure shows the general workflow for a typical integrative module network inference with Lemon-Tree. Blue boxes indicate the pre-processing steps that are done using third-party software such as R or user-defined scripts. Green boxes indicate the core module network inference steps done with the Lemon-Tree software package. Typical post-processing tasks (orange boxes), such as GO enrichment calculations, can be performed with Lemon-Tree or other tools. The Lemon-Tree task names are indicated in red (see main text for more details).

Flow chart for integrative module network inference with Lemon-Tree.

This figure shows the general workflow for a typical integrative module network inference with Lemon-Tree. Blue boxes indicate the pre-processing steps that are done using third-party software such as R or user-defined scripts. Green boxes indicate the core module network inference steps done with the Lemon-Tree software package. Typical post-processing tasks (orange boxes), such as GO enrichment calculations, can be performed with Lemon-Tree or other tools. The Lemon-Tree task names are indicated in red (see main text for more details). Task “ganesh” Run one or more instances of a model-based Gibbs sampler [15] to simultaneously infer co-expression modules and condition clusters within each module from a gene expression data matrix. Task “tight_clusters” Build consensus modules of genes that systematically cluster together in an ensemble of multiple “ganesh” runs. Consensus modules are reconstructed by a novel spectral edge clustering algorithm which identifies densely connected sets of nodes in a weighted graph [28], with edge weight defined here as the frequency with which pairs of genes belong to the same cluster in individual “ganesh” runs. Details about the tight clustering algorithm are provided in S1 Text. Task “regulators” Infer an ensemble of regulatory programs for a set of modules and compute a consensus regulator-to-module score. Regulatory programs take the form of a decision tree with the (expression level of) regulators at its internal nodes. The regulator score takes into account the number of trees a regulator is assigned to, with what score (posterior probability), and at which level of the tree [16]. An empirical distribution of scores of randomly assigned regulators is provided to assess significance. Regulator data need not come from the same data that was used for module construction but can be any continuous or discrete data type measured on the same samples. When multiple regulator types are considered, the “regulators” task is run once for each of them. Task “experiments” For a fixed set of gene modules, cluster conditions separately for each module using a model-based Gibbs sampler [15] and store the resulting hierarchical condition trees in a structured XML file. Task “split_reg” Assign regulators to a given range of one or more modules. This task allows parallelization of the “regulators” task and needs the output of the “experiments” task as an input. Task “figures” Draw publication-ready visualizations for a set of modules in postscript format, consisting of a heatmap of genes in each module, organized according to a consensus clustering of the samples, plus heatmaps of its top-scoring regulators, separated according to the regulator type (cf. S1 Fig.). Task “go_annotation” Calculate gene ontology enrichment for each module using the BiNGO [29] library. While a typical run of Lemon-Tree will apply tasks “ganesh”, “tight_clusters” and “regulators” in successive order, the software is designed to be flexible. For instance, the “tight_clusters” task can be equally well applied to build consensus clusters from the output of multiple third-party clustering algorithms, regulators can be assigned to the output of any clustering algorithm, etc. To facilitate this interoperability with other tools, input/output is handled via plain text files with minimal specification, the only exception being the storage of the regulatory decision trees which uses an XML format. Tasks also permit customization by changing the value of various parameters. We have purposefully provided default values for all parameters, based on our experience accrued over many years of developing and applying the software to a great variety of datasets from multiple organisms, and avoided mentioning any parameter settings in the Tutorial such that first-time users are presented with a simple workflow. Detailed instructions on how to integrate or extend (parts of) Lemon-Tree and a complete overview of all parameters and their default values are provided on the project website (http://lemon-tree.googlecode.com/).

Results

Benchmark between Lemon-Tree and CONEXIC

We compared the performance of Lemon-Tree with CONEXIC (COpy Number and Expression In Cancer), a state-of-the-art module network algorithm designed to integrate matched copy number (amplifications and deletions) and gene expression data from tumor samples [12]. The general framework is the same for the algorithms, with modules of co-expressed genes associated to a list of regulators assigned via a probabilistic score. However, the probabilistic techniques used to build the modules and to assign regulators are different. We ran the two programs on the same large-scale reference data set to evaluate these differences. We used Gene Ontology (GO) enrichment and a reference network of protein-protein interactions to compare the co-expressed modules and the regulatory programs. We downloaded gene expression and copy number glioblastoma datasets from the TCGA data portal [3] and we built an expression data matrix of 250 samples and 9,367 genes. We limited the number of samples for this benchmark study in order to save computational time. For the candidate regulators, we selected the top 1,000 genes that were significantly amplified or deleted as input genes for both CONEXIC and Lemon-Tree. To run CONEXIC, we followed the instructions of the manual and more specifically used the recommended bootstrapping procedure to get robust results. For Lemon-Tree, we generated an ensemble of two-way clustering solutions that were assembled in one robust solution by node clustering. Then we assigned the regulators using the same input list as with CONEXIC. A global score was calculated for each regulator and for each module and we selected the top 1% regulators as the final list (see S1 Text). The total run-time for the two software programs on the benchmark dataset was quite similar, with a small advantage for Lemon-Tree (S5 Table). To compare the Gene Ontology (GO) categories between Lemon-Tree and CONEXIC, we built a list of all common categories for a given p-value threshold and converted the corrected p-values to −log10(p-value) scores. We selected the highest score for each GO category and we counted the number of GO categories having a higher score for Lemon-Tree or CONEXIC, and calculated the sum of scores for each GO category and each software. The results shown in Fig. 2 indicate that Lemon-Tree clusters have a higher number of GO categories with lower p-values than CONEXIC (Fig. 2A), and that globally the p-values are lower for Lemon-Tree clusters (Fig. 2B). To benchmark the regulators’ assignment of each software, we used a scoring scheme developed by Jornsten et al. [30]. For a given interaction distance in a reference protein-protein interaction network, we calculated the relative enrichment of known interactions in the networks inferred by Lemon-Tree and CONEXIC with respect to known interactions in networks where edges have been randomly re-assigned (see S1 Text). Fig. 2C shows the relative enrichments for interaction distances ranging from 1 (direct interaction) to 4. The Lemon-Tree inferred network is enriched for short or direct paths, a desired characteristic for well-estimated networks [30].
Fig 2

Comparison between Lemon-Tree and CONEXIC.

Gene Ontology (GO) enrichment of the co-expressed gene clusters, indicated by counting the number of GO categories having a lower p-value (A) and by comparing the sum of the quantity -log10(p-value) (B) for different global p-value cutoff levels (x-axis). (C) Relative enrichment of inferred interactions by Lemon-Tree and CONEXIC to known molecular protein-protein interactions (PPI), for increasing interaction distances.

Comparison between Lemon-Tree and CONEXIC.

Gene Ontology (GO) enrichment of the co-expressed gene clusters, indicated by counting the number of GO categories having a lower p-value (A) and by comparing the sum of the quantity -log10(p-value) (B) for different global p-value cutoff levels (x-axis). (C) Relative enrichment of inferred interactions by Lemon-Tree and CONEXIC to known molecular protein-protein interactions (PPI), for increasing interaction distances. These results are consistent with a previous study conducted on bacteria and yeast data, where we showed a better performance in terms of enrichment in functional categories and known regulatory interactions of the algorithms underlying the Lemon-Tree software over Genomica (a software tool on which CONEXIC is based) [17]. Taken together, these results show that Lemon-Tree compares favorably with state-of-the-art module network inference algorithms.

Integrative analysis of TCGA glioblastoma expression and copy-number data

Lemon-Tree can be used to integrate various types of ‘omics’ data and generate new biological and biomedical insights. Here, we exemplify how to integrate copy-number and expression data for a large dataset of glioblastoma tumor samples and show that the results are enriched in known key players of canonical tumor pathways as well as novel candidates. Malignant gliomas are the most common subtype of primary brain tumors and are very aggressive, highly invasive and neurologically destructive. Glioblastoma multiforme (GBM) is the most malignant form of gliomas, and despite intense investigation of this disease in the past decades, most patients with GBM die within approximately 15 months of diagnosis [31]. Somatic copy-number alterations (SCNA) are extremely common in cancer and affect a larger fraction of the genome than any other types of somatic genetic alterations. They have critical roles in activating oncogenes and inactivating tumor suppressor genes, and their study has suggested novel potential therapeutic strategies [32, 33]. However, distinguishing the alterations that drive cancer development from the passenger SCNAs that are acquired over time during cancer progression is a critical challenge. Here we use the module network framework implemented in the Lemon-Tree software tool to build a module network relating genes located in regions that are significantly amplified or deleted to modules of co-expressed genes. In other words, the module network selects and prioritizes copy-number altered genes that might play a role (direct or indirect) for clusters of co-expressed genes, performing important biological functions in glioblastoma. The resulting module network is used to prioritize SCNA genes that are amplified or deleted, and to provide novel hypotheses regarding drivers of glioblastoma. We downloaded data from the TCGA project portal [3] and we selected 484 glioblastoma tumor samples from different patients (representing 91% of the available samples). We selected 7,574 gene expression profiles and generated an ensemble of two-way clustering solutions that were assembled in one robust solution by node clustering, resulting in a set of 121 clusters composed of 5,423 genes (S1 Text and S1 Table). We assembled a list of genes amplified and deleted in glioblastoma tumors from the most recent GISTIC run of the Broad Institute TCGA Copy Number Portal on glioblastoma samples. GISTIC [34] is the standard software tool used for the detection of peak regions significantly amplified or deleted in a number of samples from copy-number profiles. We also included in the list a number of key genes amplified or deleted from previous studies [34-36]. The final list is composed of 353 amplified and 2,007 deleted genes (with all genes present on sex chromosomes excluded). We did not use extremely stringent statistical thresholds for the selection, to avoid the exclusion of potentially interesting candidates. From this list we built SCNA gene copy-number profiles using TCGA data and used those profiles as candidate regulators for the co-expressed gene clusters. We assigned regulators independently for amplified and deleted genes, and we selected the top 1% highest scoring regulators as the final list (a cutoff well above assignment of regulators expected by chance), with 92 amplified and 579 deleted selected genes (S1 Text; S2 and S3 Tables). The resulting glioblastoma module network is composed of 121 clusters of co-expressed genes, together with associated prioritized lists of high-scoring SCNA genes (associated to amplified and deleted regions). More than 60% of the clusters have a significant Gene Ontology (GO) enrichment (corrected p-value < 0.05, Table 2 and S4 Table). Several of those enriched clusters can be related to the hallmarks of cancers, ten distinctive and complementary capabilities that have been defined as the fundamental biological capabilities acquired during tumor development [37, 38]. For instance, we have 11 clusters enriched for GO categories related to cell cycle processes and regulation (p-value < 0.05), with three of them having very strong enrichment (corrected p-values 4×10−18, 6×10−24 and 9×10−71, Table 2). The cell cycle is deregulated in most cancers and is at the heart of the “sustaining proliferative signaling” hallmark. Eight clusters are enriched for categories related to immune response, with two of them displaying strong enrichment (corrected p-values 6×10−33 and 6×10−45, Table 2). Most tumor lesions contain immune cells present at various degrees of density. Intense recent research has shown that this immune response is linked to two phenomena. First, it is obviously an attempt by the immune system to eradicate the tumor, but secondly, there is now a large body of evidence showing that immune cells also have strong tumor-promoting effects, and both aspects are categorized as part of the hallmarks of cancer [38]. For instance, microglia are a type of glial cells that act as macrophages of the brain and the spinal cord and thus act as the main form of immune response in the central nervous system. They constitute the dominant form of glioma tumor infiltrating immune cells, and they might promote tumor growth by facilitating immunosuppression of the tumor microenvironment [39]. The development of blood vessels (angiogenesis) is another crucial hallmark of cancer, providing sustenance in oxygen and nutrients and a way to evacuate metabolic wastes and carbon dioxide [38]. Glioblastoma multiforme is characterized by a striking and dramatic induction of angiogenesis [31]. There are seven clusters enriched for GO categories related to angiogenesis and blood vessel development, with two of them having strong enrichment (corrected p-values 4×10−6 and 9×10−16, Table 2). A recent large-scale integrative study of hundreds of glioblastoma samples has shown that chromatin modifications could potentially have high biological relevance for this type of tumor [40]. Interestingly, we have a cluster highly enriched in chromatin assembly and organization (corrected p-value 5×10−17 and 9×10−24, Table 2). Taken together, these results show that the clusters of co-expressed genes in the module network are representative of the molecular functions and biological processes involved in tumor in general and more specifically in glioblastoma.
Table 2

GO enrichment for glioblastoma modules.

GroupModule numberModule nb of genesCorrected p-valueGO category
Cell Cycle1859×10−71 cell cycle phase
2×10−67 cell cycle process
6×10−63 mitotic cell cycle
11606×10−24 cell cycle phase
6×10−24 mitotic cell cycle
33364×10−18 cell cycle phase
1×10−17 mitotic cell cycle
Immune response31456×10−45 immune response
6×10−45 immune system process
1×10−26 inflammatory response
4×10−23 innate immune response
141276×10−33 response to type I interferon
8×10−24 innate immune response
26547×10−6 defense response
9×10−6 immune response
48371×10−6 immune system process
Vasculature27404×10−16 vasculature development
2×10−15 blood vessel development
7×10−13 angiogenesis
37813×10−10 extracellular matrix organization
9×10−6 blood vessel development
Chromatin modifications70129×10−24 chromatin assembly
8×10−24 nucleosome assembly
5×10−17 chromatin organization

Selection of clusters of co-expressed genes from the glioblastoma module network highly enriched for GO categories related to cancer hallmarks. Enriched categories are grouped into broader functional groups. Only a subset of the GO categories are displayed in this table. The full list is available as S1 Table.

Selection of clusters of co-expressed genes from the glioblastoma module network highly enriched for GO categories related to cancer hallmarks. Enriched categories are grouped into broader functional groups. Only a subset of the GO categories are displayed in this table. The full list is available as S1 Table. In the glioblastoma module network, we inferred a list of amplified and deleted SCNA genes linked to one or more clusters of co-expressed genes. Some of those SCNA genes are highly connected, representing potential master copy-number regulators for module activity. To identify and analyze those SCNA hub genes, we calculated for each high-scoring regulator the sum of the scores obtained in each module, and ranked them by decreasing score for amplified (Table 3) and deleted (Table 4) genes. Among these genes, we find many well-known oncogenes and tumor supressors that are frequently amplified, deleted or mutated in glioblastoma. Those genes include EGFR, PDGFRA, FGFR3, PIK3CA, MDM4, CDKN2A/B, PTEN and are all members of the core alterated pathways in glioblastoma controlling key phenotypes such as proliferation, apoptosis and angiogenesis (Fig. 3, [3,35,40,41]). Those genes and pathways are also frequently impaired in many other types of tumors [42-44]. In addition, we find in those lists of hub genes a number of interesting new candidates, both in amplified and deleted genes, that have not been associated with glioblastoma before. To better visualize the importance and role of both the well-known and novel SCNAs prioritized by Lemon-Tree, we represent those that are part of the three core pathways altered in glioblastoma as a network with edges representing activation or inhibition relationships, together with their levels of gene gains and losses in glioblastoma samples (Fig. 3).
Table 3

High-scoring amplified gene hubs detected by Lemon-Tree.

SymbolPathwayBandNmSum score% amp.M-listP-list
CHIC24q1232588419xx
EGFREGFR signalling7p11.224518487xx
INSREGFR signalling19p13.215391839xx
ASAP1Membrane cytoskeleton interactions, cell motility8q24.2116311911
MYCNRegulation of transcription2p24.32130288x
C1orf1011q4419298017x
RHOBRho protein signal transduction2p24.11927317
KRIT1Small GTPase mediated signal transduction7q21.211224283
CCNE1Regulation of cell cycle19q1214198036xx
SDCCAG81q4314197317x
ADCY8Intracellular signal transduction8q24.2212194911
PDGFRACell proliferation, signal transduction4q1210187418xx
DDX1Regulation of translation2p24.31617638
MDM4p53 regulation1q32.19138527xx
mir-4283-27q11.2110137480
PRDM2Regulation of transcription1p36.218132315
FGFR3Cell growth4p16.3510318xx
SCIMPImmune response, signal transduction17p13.2810228
GSDMCEpithelial cell proliferation and apoptosis8q24.21891911
COL4A1Angiogenesis13q3427435x
PIK3CACell signalling, cell growth3q26.3774317x

List of the top 20 amplified genes ordered by decreasing sum of score values. Nm: number of modules in which the gene is selected as a high-scoring regulator. % amp.: percentage of samples in which the gene is classified as low-level gain or high-level amplification (according to GISTIC putative calls). M-list: presence in a list of genes frequently mutated in cancer, compiled from [42–44]. P-list: presence in a list of genes recurrently amplified or deleted in 11 cancer types [33].

Table 4

High-scoring deleted genes detected by Lemon-Tree.

SymbolPathwayBandNb modulesSum score% del.M-listP-list
PAOXPolyamine homeostasis, apoptosis10q26.354793789xx
CDKN2ANegative regulation of cell proliferation9p21.331478575x
mir-320122q13.3221303037x
mir-3405q35.335303010x
mir-60410p11.2349293082x
mir-93810p11.2345292182
C9orf539p21.329289775x
ATAD110q23.3155243388
KIAA012514q32.3330211728x
mir-548q10p1335201781
OMGCell adhesion17q11.221169713x
EVI2B17q11.219162913x
KRTAP5-611p15.518156421
SRGAP1Cell migration12q14.220139714
KLLNCell cycle arrest, apoptosis10q23.3134137488x
FLT4Protein tyrosine kinase signalling5q35.312102210x
EFCAB4AMetabolic process11p15.53396423
HBD11p15.43896420
DMRTA2Regulation of transcription1p32.3289265
TBC1D3012q14.31579113
ART5Protein glycosylation11p15.41178521
FAM19A522q13.32474537x
EVI2A17q11.21770913x
ARID212q12568114x
WDR3710p15.32161481
MOB2Death receptor signalling11p15.51559923
PTENEGFR signalling, AKT pathway10q23.311959389xx
MUC4Cell matrix adhesion, transport3q291058811
IDI1Isoprenoids synthesis10p15.132356981
CSMD18p23.2856612x
CDKN2BNegative regulation of cell proliferation9p21.31956575x

List of top 30 deleted genes ordered by decreasing sum of score values. % del.: percentage of samples in which the gene is classified as single-copy loss or deep loss (according to GISTIC putative calls). Nm, M-list and P-list: see Table 3.

Fig 3

Glioblastoma signaling pathway alterations for top hub regulators.

Copy number alterations for a selection of predicted hub regulators are indicated for canonical glioblastoma signaling pathways p53, RB and RTK/PI3K. Genes selected by the algorithm are indicated in black boxes, while light grey boxes depict genes that were not selected by the algorithm but are key factors for the pathway. Purple hexagons indicate phenotypes. Percentage of copy gain or loss is indicated by value and by color shades of red for gene gains and green for gene losses. The values are taken from GISTIC putative calls for low-levels gains or single-copy losses on 563 glioblastoma samples (data from the Broad institute).

Glioblastoma signaling pathway alterations for top hub regulators.

Copy number alterations for a selection of predicted hub regulators are indicated for canonical glioblastoma signaling pathways p53, RB and RTK/PI3K. Genes selected by the algorithm are indicated in black boxes, while light grey boxes depict genes that were not selected by the algorithm but are key factors for the pathway. Purple hexagons indicate phenotypes. Percentage of copy gain or loss is indicated by value and by color shades of red for gene gains and green for gene losses. The values are taken from GISTIC putative calls for low-levels gains or single-copy losses on 563 glioblastoma samples (data from the Broad institute). List of the top 20 amplified genes ordered by decreasing sum of score values. Nm: number of modules in which the gene is selected as a high-scoring regulator. % amp.: percentage of samples in which the gene is classified as low-level gain or high-level amplification (according to GISTIC putative calls). M-list: presence in a list of genes frequently mutated in cancer, compiled from [42-44]. P-list: presence in a list of genes recurrently amplified or deleted in 11 cancer types [33]. List of top 30 deleted genes ordered by decreasing sum of score values. % del.: percentage of samples in which the gene is classified as single-copy loss or deep loss (according to GISTIC putative calls). Nm, M-list and P-list: see Table 3. Within the list of amplified gene hubs (Table 3), we find a number of genes that have been rarely or never associated before with glioblastoma. INSR is a gene encoding for the insulin receptor, a transmembrane receptor activated by insuline and IGF factors, member of the tyrosine receptor kinase family, and playing a key role in glucose homeostasis. INSR is selected as a high-scoring regulator in 15 modules and ranked in third position in the list of amplified gene hubs. It is found to be amplified as low-level gain or higher in 39% of the samples (Table 3). Beyond its well-known role in glucose homeostasis, INSR stimulates cell proliferation (Fig. 3) and migration and is often aberrantly expressed in cancer cells [45]. Consequently, amplification of INSR in glioblastoma may enhance proliferation. MYCN encodes a transcription factor (N-myc) highly expressed in fetal brain and critical for normal brain development. It is also a well-known proto-oncogene, and amplification of N-myc is associated with poor outcome in neuroblastoma [46]. MYCN is amplified as low-level gain or higher in 8% of the glioblastoma samples and is connected to 21 modules (Table 3). MYCN is part of the RB signaling pathway, and is also strongly connected to the RTK / PI3K and p53 pathways (Fig. 3), with a direct influence on proliferation. For that reason, its amplification may also favor proliferation in glioblastoma. KRIT1 (also known as CCM1) is a gene crucial for maintaining the integrity of the vasculature and for normal angiogenesis. Loss of function of this gene is responsible for vascular malformations in the brain known as cerebral cavernous malformations [47, 48]. It is amplified as low-level gain or higher in 83% of the glioblastoma samples and it is listed in the top 10 hubs in our list (Table 3). The consequences of KRIT1 amplification are not completely clear, but we may hypothesize that it is required for proper angiogenesis development, which is a hallmark of glioblastoma [31], and that it may also help decrease apoptosis (Fig. 3). In the list of putative deleted genes, PAOX (polyamine oxidase) is ranked first, with a connection to 54 modules and the highest sum of scores value. It is classified as single loss (GISTIC call value of -1 or lower) in 89% of the samples. This is a very high value, comparable to the value obtained for the classical tumor suppressor CDKN2A (75%, Table 4). Amine oxidases are involved in the metabolism of polyamines, regulating their intracellular concentrations and elimination. The products of this metabolism (e.g. hydrogen peroxyde) are cytotoxic and have been considered as a cause for apoptotic cell death. Amine oxidases are considered as biological regulators for cell growth and differentiation, and a primary involvement of amine oxidases in cancer growth inhibition and progression has been demonstrated [49]. Therefore, PAOX might have a tumor suppressor activity and its deletion in many glioblastoma samples could provide a selective advantage to glioblastoma tumor cells. Interestingly, amino acids metabolism is not part of the standard alterated pathways in glioblastoma (explaining why we did not represent PAOX on Fig. 3), but targeting this pathway could lead to novel therapeutic treatments [50]. KLLN encodes a nuclear transcription factor and shares a bidirectional promoter with PTEN. It is activated by p53 and is involved in S phase arrest and apoptosis [51]. Recent studies show that KLLN has a tumor supressor effect and is associated with worse prognosis in prostate and breast carcinomas [52, 53]. Consequently, the loss of KLLN that is observed in 88% of the glioblastoma samples (Table 4) would help the development of tumor cells by decreasing apoptosis and favoring proliferation (Fig. 3). To assess the biological relevance of the amplified and deleted gene hubs in the module network, we analyzed the prognosis value of the top gene hubs by survival analysis, using the clinical data available for TCGA samples (survival time and status of the patient). We constructed Kaplan-Meier estimates using GISTIC putative calls to define genes having single or deep copy loss (i.e. GISTIC call value ≤-1) and genes having low-level gains or high-level amplifications (i.e. GISTIC call value ≥1. The differences between groups were formally tested and a total of 3 amplified genes and 18 deleted genes from the lists displayed in Tables 3 and 4 have significant p-values < 0.05 (Fig. 4 and S6 Table). Interestingly, among those genes we find the well-known glioblastoma oncogene EGFR and tumor suppressors CDKN2A and PTEN, but also novel candidates such as KRIT1 and PAOX described in the previous paragraph. Glioblastoma patients having copy-number alterations for those genes have a worse survival prognostic. This indicates the biological relevance of those genes that may be used as biomarkers.
Fig 4

Kaplan-Meier survival curves for a selection of top hub glioblastoma genes predicted by the Lemon-Tree algorithm.

The top three panels are genes having low-levels gains or high-level amplifications (magenta) compared to normal (blue), the bottom three panels are genes having single-copy loss or homozygous deletions (green) compared to normal (blue). All genes display significant differences between the groups (p < 0.05, see S6 Table for a full list of p-values). Patient with putative gene gains or losses have significantly worse prognosis (lower values on the y-axis). The x-axis on all figures represent the time in number of days.

Kaplan-Meier survival curves for a selection of top hub glioblastoma genes predicted by the Lemon-Tree algorithm.

The top three panels are genes having low-levels gains or high-level amplifications (magenta) compared to normal (blue), the bottom three panels are genes having single-copy loss or homozygous deletions (green) compared to normal (blue). All genes display significant differences between the groups (p < 0.05, see S6 Table for a full list of p-values). Patient with putative gene gains or losses have significantly worse prognosis (lower values on the y-axis). The x-axis on all figures represent the time in number of days.

Availability and Future Directions

The Lemon-Tree software is hosted at Google Code (http://lemon-tree.googlecode.com/). The source code, executables and documentation can be downloaded with no restrictions and no registration, and are released under the terms of the GNU General Public License (GPL) version 2.0. Developers and users can join the project by contacting the authors and there is a mailing list for discussions and news about module networks and the project. A step-by-step tutorial to learn how to install and use the software is available on the wiki section, together with the corresponding data sets. In the future, we intend to extend Lemon-Tree’s support for explicitly modelling causal relations between regulator types and to incorporate complementary algorithms available in the literature for integrating gene-based methods, physical interactions and cross-species data. Firstly, the current version of Lemon-Tree is able to associate co-expression modules to multiple ‘regulator’ types (e.g. expression regulators, structural DNA variants, phenotypic states, etc.) by assigning each of those independently as regulators of a module. We will extend the software with Bayesian methods to account for possible causal relations between regulator types, e.g. when the association between a module and expression regulator can be partly explained by a structural DNA variant. Secondly, a key long-term objective of the Lemon-Tree project is to provide a general open-source repository for module network inference tools with a consistent user interface. As a first step, the current version of Lemon-Tree implements algorithms previously developed by our group [14-17]. In the future, we intend to extend it with complementary algorithms developed by other groups, including algorithms to combine the strengths of module network methods with gene-based methods [18], to incorporate physical protein-protein or protein-DNA interactions as a prior in the regulator assignment procedure [13] or to infer module networks from multiple species simultaneously [19]. A document detailing guidelines to implement new functions in the Lemon-Tree Java codebase is available on the project wiki.

Complete list of clusters and genes for the glioblastoma dataset.

(XLS) Click here for additional data file.

Top regulators (1% cutoff level) for copy-number profiles (amplified genes).

(XLS) Click here for additional data file.

Top regulators (1% cutoff level) for copy-number profiles (deleted genes).

(XLS) Click here for additional data file.

GO enrichment for glioblastoma clusters (corrected p-value <0.05).

(XLS) Click here for additional data file.

Running time comparison between Lemon-Tree and CONEXIC on the benchmark dataset.

(XLS) Click here for additional data file.

Survival curves p-values for amplified and deleted genes.

(XLS) Click here for additional data file.

Example heatmap figure generated by Lemon-Tree showing co-expression module 18 in the glioblastoma dataset and copy-number profiles of amplified and deleted genes predicted to be causal regulators of module 18.

(PDF) Click here for additional data file.

Additional details about the tight clustering algorithm and data analyses performed.

(PDF) Click here for additional data file.

Copy of the Lemon-Tree source code, test dataset and software tutorial.

(ZIP) Click here for additional data file.
  53 in total

Review 1.  Next-generation genomics: an integrative approach.

Authors:  R David Hawkins; Gary C Hon; Bing Ren
Journal:  Nat Rev Genet       Date:  2010-07       Impact factor: 53.242

Review 2.  Amine oxidases in apoptosis and cancer.

Authors:  Antonio Toninello; Paola Pietrangeli; Umberto De Marchi; Mauro Salvi; Bruno Mondovì
Journal:  Biochim Biophys Acta       Date:  2005-09-29

3.  Killin is a p53-regulated nuclear inhibitor of DNA synthesis.

Authors:  Yong-Jig Cho; Peng Liang
Journal:  Proc Natl Acad Sci U S A       Date:  2008-04-02       Impact factor: 11.205

4.  International network of cancer genome projects.

Authors:  Thomas J Hudson; Warwick Anderson; Axel Artez; Anna D Barker; Cindy Bell; Rosa R Bernabé; M K Bhan; Fabien Calvo; Iiro Eerola; Daniela S Gerhard; Alan Guttmacher; Mark Guyer; Fiona M Hemsley; Jennifer L Jennings; David Kerr; Peter Klatt; Patrik Kolar; Jun Kusada; David P Lane; Frank Laplace; Lu Youyong; Gerd Nettekoven; Brad Ozenberger; Jane Peterson; T S Rao; Jacques Remacle; Alan J Schafer; Tatsuhiro Shibata; Michael R Stratton; Joseph G Vockley; Koichi Watanabe; Huanming Yang; Matthew M F Yuen; Bartha M Knoppers; Martin Bobrow; Anne Cambon-Thomsen; Lynn G Dressler; Stephanie O M Dyke; Yann Joly; Kazuto Kato; Karen L Kennedy; Pilar Nicolás; Michael J Parker; Emmanuelle Rial-Sebbag; Carlos M Romeo-Casabona; Kenna M Shaw; Susan Wallace; Georgia L Wiesner; Nikolajs Zeps; Peter Lichter; Andrew V Biankin; Christian Chabannon; Lynda Chin; Bruno Clément; Enrique de Alava; Françoise Degos; Martin L Ferguson; Peter Geary; D Neil Hayes; Thomas J Hudson; Amber L Johns; Arek Kasprzyk; Hidewaki Nakagawa; Robert Penny; Miguel A Piris; Rajiv Sarin; Aldo Scarpa; Tatsuhiro Shibata; Marc van de Vijver; P Andrew Futreal; Hiroyuki Aburatani; Mónica Bayés; David D L Botwell; Peter J Campbell; Xavier Estivill; Daniela S Gerhard; Sean M Grimmond; Ivo Gut; Martin Hirst; Carlos López-Otín; Partha Majumder; Marco Marra; John D McPherson; Hidewaki Nakagawa; Zemin Ning; Xose S Puente; Yijun Ruan; Tatsuhiro Shibata; Michael R Stratton; Hendrik G Stunnenberg; Harold Swerdlow; Victor E Velculescu; Richard K Wilson; Hong H Xue; Liu Yang; Paul T Spellman; Gary D Bader; Paul C Boutros; Peter J Campbell; Paul Flicek; Gad Getz; Roderic Guigó; Guangwu Guo; David Haussler; Simon Heath; Tim J Hubbard; Tao Jiang; Steven M Jones; Qibin Li; Nuria López-Bigas; Ruibang Luo; Lakshmi Muthuswamy; B F Francis Ouellette; John V Pearson; Xose S Puente; Victor Quesada; Benjamin J Raphael; Chris Sander; Tatsuhiro Shibata; Terence P Speed; Lincoln D Stein; Joshua M Stuart; Jon W Teague; Yasushi Totoki; Tatsuhiko Tsunoda; Alfonso Valencia; David A Wheeler; Honglong Wu; Shancen Zhao; Guangyu Zhou; Lincoln D Stein; Roderic Guigó; Tim J Hubbard; Yann Joly; Steven M Jones; Arek Kasprzyk; Mark Lathrop; Nuria López-Bigas; B F Francis Ouellette; Paul T Spellman; Jon W Teague; Gilles Thomas; Alfonso Valencia; Teruhiko Yoshida; Karen L Kennedy; Myles Axton; Stephanie O M Dyke; P Andrew Futreal; Daniela S Gerhard; Chris Gunter; Mark Guyer; Thomas J Hudson; John D McPherson; Linda J Miller; Brad Ozenberger; Kenna M Shaw; Arek Kasprzyk; Lincoln D Stein; Junjun Zhang; Syed A Haider; Jianxin Wang; Christina K Yung; Anthony Cros; Anthony Cross; Yong Liang; Saravanamuttu Gnaneshan; Jonathan Guberman; Jack Hsu; Martin Bobrow; Don R C Chalmers; Karl W Hasel; Yann Joly; Terry S H Kaan; Karen L Kennedy; Bartha M Knoppers; William W Lowrance; Tohru Masui; Pilar Nicolás; Emmanuelle Rial-Sebbag; Laura Lyman Rodriguez; Catherine Vergely; Teruhiko Yoshida; Sean M Grimmond; Andrew V Biankin; David D L Bowtell; Nicole Cloonan; Anna deFazio; James R Eshleman; Dariush Etemadmoghadam; Brooke B Gardiner; Brooke A Gardiner; James G Kench; Aldo Scarpa; Robert L Sutherland; Margaret A Tempero; Nicola J Waddell; Peter J Wilson; John D McPherson; Steve Gallinger; Ming-Sound Tsao; Patricia A Shaw; Gloria M Petersen; Debabrata Mukhopadhyay; Lynda Chin; Ronald A DePinho; Sarah Thayer; Lakshmi Muthuswamy; Kamran Shazand; Timothy Beck; Michelle Sam; Lee Timms; Vanessa Ballin; Youyong Lu; Jiafu Ji; Xiuqing Zhang; Feng Chen; Xueda Hu; Guangyu Zhou; Qi Yang; Geng Tian; Lianhai Zhang; Xiaofang Xing; Xianghong Li; Zhenggang Zhu; Yingyan Yu; Jun Yu; Huanming Yang; Mark Lathrop; Jörg Tost; Paul Brennan; Ivana Holcatova; David Zaridze; Alvis Brazma; Lars Egevard; Egor Prokhortchouk; Rosamonde Elizabeth Banks; Mathias Uhlén; Anne Cambon-Thomsen; Juris Viksna; Fredrik Ponten; Konstantin Skryabin; Michael R Stratton; P Andrew Futreal; Ewan Birney; Ake Borg; Anne-Lise Børresen-Dale; Carlos Caldas; John A Foekens; Sancha Martin; Jorge S Reis-Filho; Andrea L Richardson; Christos Sotiriou; Hendrik G Stunnenberg; Giles Thoms; Marc van de Vijver; Laura van't Veer; Fabien Calvo; Daniel Birnbaum; Hélène Blanche; Pascal Boucher; Sandrine Boyault; Christian Chabannon; Ivo Gut; Jocelyne D Masson-Jacquemier; Mark Lathrop; Iris Pauporté; Xavier Pivot; Anne Vincent-Salomon; Eric Tabone; Charles Theillet; Gilles Thomas; Jörg Tost; Isabelle Treilleux; Fabien Calvo; Paulette Bioulac-Sage; Bruno Clément; Thomas Decaens; Françoise Degos; Dominique Franco; Ivo Gut; Marta Gut; Simon Heath; Mark Lathrop; Didier Samuel; Gilles Thomas; Jessica Zucman-Rossi; Peter Lichter; Roland Eils; Benedikt Brors; Jan O Korbel; Andrey Korshunov; Pablo Landgraf; Hans Lehrach; Stefan Pfister; Bernhard Radlwimmer; Guido Reifenberger; Michael D Taylor; Christof von Kalle; Partha P Majumder; Rajiv Sarin; T S Rao; M K Bhan; Aldo Scarpa; Paolo Pederzoli; Rita A Lawlor; Massimo Delledonne; Alberto Bardelli; Andrew V Biankin; Sean M Grimmond; Thomas Gress; David Klimstra; Giuseppe Zamboni; Tatsuhiro Shibata; Yusuke Nakamura; Hidewaki Nakagawa; Jun Kusada; Tatsuhiko Tsunoda; Satoru Miyano; Hiroyuki Aburatani; Kazuto Kato; Akihiro Fujimoto; Teruhiko Yoshida; Elias Campo; Carlos López-Otín; Xavier Estivill; Roderic Guigó; Silvia de Sanjosé; Miguel A Piris; Emili Montserrat; Marcos González-Díaz; Xose S Puente; Pedro Jares; Alfonso Valencia; Heinz Himmelbauer; Heinz Himmelbaue; Victor Quesada; Silvia Bea; Michael R Stratton; P Andrew Futreal; Peter J Campbell; Anne Vincent-Salomon; Andrea L Richardson; Jorge S Reis-Filho; Marc van de Vijver; Gilles Thomas; Jocelyne D Masson-Jacquemier; Samuel Aparicio; Ake Borg; Anne-Lise Børresen-Dale; Carlos Caldas; John A Foekens; Hendrik G Stunnenberg; Laura van't Veer; Douglas F Easton; Paul T Spellman; Sancha Martin; Anna D Barker; Lynda Chin; Francis S Collins; Carolyn C Compton; Martin L Ferguson; Daniela S Gerhard; Gad Getz; Chris Gunter; Alan Guttmacher; Mark Guyer; D Neil Hayes; Eric S Lander; Brad Ozenberger; Robert Penny; Jane Peterson; Chris Sander; Kenna M Shaw; Terence P Speed; Paul T Spellman; Joseph G Vockley; David A Wheeler; Richard K Wilson; Thomas J Hudson; Lynda Chin; Bartha M Knoppers; Eric S Lander; Peter Lichter; Lincoln D Stein; Michael R Stratton; Warwick Anderson; Anna D Barker; Cindy Bell; Martin Bobrow; Wylie Burke; Francis S Collins; Carolyn C Compton; Ronald A DePinho; Douglas F Easton; P Andrew Futreal; Daniela S Gerhard; Anthony R Green; Mark Guyer; Stanley R Hamilton; Tim J Hubbard; Olli P Kallioniemi; Karen L Kennedy; Timothy J Ley; Edison T Liu; Youyong Lu; Partha Majumder; Marco Marra; Brad Ozenberger; Jane Peterson; Alan J Schafer; Paul T Spellman; Hendrik G Stunnenberg; Brandon J Wainwright; Richard K Wilson; Huanming Yang
Journal:  Nature       Date:  2010-04-15       Impact factor: 49.962

5.  Transcription regulatory networks in Caenorhabditis elegans inferred through reverse-engineering of gene expression profiles constitute biological hypotheses for metazoan development.

Authors:  Vanessa Vermeirssen; Anagha Joshi; Tom Michoel; Eric Bonnet; Tine Casneuf; Yves Van de Peer
Journal:  Mol Biosyst       Date:  2009-07-17

6.  Krev1 interaction trapped-1/cerebral cavernous malformation-1 protein expression during early angiogenesis.

Authors:  Ozlem Guzeloglu-Kayisli; Umit A Kayisli; Nduka M Amankulor; Jennifer R Voorhees; Ozgun Gokce; Michael L DiLuna; Maxwell S H Laurans; Guven Luleci; Murat Gunel
Journal:  J Neurosurg       Date:  2004-05       Impact factor: 5.115

7.  Combined microarray analysis uncovers self-renewal related signaling in mouse embryonic stem cells.

Authors:  Hao Zhu; He Yang; Markus R Owen
Journal:  Syst Synth Biol       Date:  2008-05-08

8.  Learning a prior on regulatory potential from eQTL data.

Authors:  Su-In Lee; Aimée M Dudley; David Drubin; Pamela A Silver; Nevan J Krogan; Dana Pe'er; Daphne Koller
Journal:  PLoS Genet       Date:  2009-01-30       Impact factor: 5.917

9.  Comparative analysis of module-based versus direct methods for reverse-engineering transcriptional regulatory networks.

Authors:  Tom Michoel; Riet De Smet; Anagha Joshi; Yves Van de Peer; Kathleen Marchal
Journal:  BMC Syst Biol       Date:  2009-05-07

10.  Integrated module and gene-specific regulatory inference implicates upstream signaling networks.

Authors:  Sushmita Roy; Stephen Lagree; Zhonggang Hou; James A Thomson; Ron Stewart; Audrey P Gasch
Journal:  PLoS Comput Biol       Date:  2013-10-17       Impact factor: 4.475

View more
  37 in total

1.  Machine Learning for Integrating Data in Biology and Medicine: Principles, Practice, and Opportunities.

Authors:  Marinka Zitnik; Francis Nguyen; Bo Wang; Jure Leskovec; Anna Goldenberg; Michael M Hoffman
Journal:  Inf Fusion       Date:  2018-09-21       Impact factor: 12.975

2.  Omics, big data and machine learning as tools to propel understanding of biological mechanisms and to discover novel diagnostics and therapeutics.

Authors:  Nikolaos Perakakis; Alireza Yazdani; George E Karniadakis; Christos Mantzoros
Journal:  Metabolism       Date:  2018-08-08       Impact factor: 8.694

3.  Construction of an integrated gene regulatory network link to stress-related immune system in cattle.

Authors:  Elham Behdani; Mohammad Reza Bakhtiarizadeh
Journal:  Genetica       Date:  2017-08-20       Impact factor: 1.082

4.  Unsupervised discovery of phenotype-specific multi-omics networks.

Authors:  W Jenny Shi; Yonghua Zhuang; Pamela H Russell; Brian D Hobbs; Margaret M Parker; Peter J Castaldi; Pratyaydipta Rudra; Brian Vestal; Craig P Hersh; Laura M Saba; Katerina Kechris
Journal:  Bioinformatics       Date:  2019-11-01       Impact factor: 6.937

Review 5.  Machine learning: its challenges and opportunities in plant system biology.

Authors:  Mohsen Hesami; Milad Alizadeh; Andrew Maxwell Phineas Jones; Davoud Torkamaneh
Journal:  Appl Microbiol Biotechnol       Date:  2022-05-16       Impact factor: 4.813

6.  Network Analysis of Microarray Data.

Authors:  Alisa Pavel; Angela Serra; Luca Cattelani; Antonio Federico; Dario Greco
Journal:  Methods Mol Biol       Date:  2022

7.  Graph- and rule-based learning algorithms: a comprehensive review of their applications for cancer type classification and prognosis using genomic data.

Authors:  Saurav Mallik; Zhongming Zhao
Journal:  Brief Bioinform       Date:  2020-03-23       Impact factor: 11.622

8.  Identification of Key Components in Colon Adenocarcinoma Using Transcriptome to Interactome Multilayer Framework.

Authors:  Ehsan Pournoor; Zaynab Mousavian; Abbas Nowzari Dalini; Ali Masoudi-Nejad
Journal:  Sci Rep       Date:  2020-03-19       Impact factor: 4.379

Review 9.  Network approaches to systems biology analysis of complex disease: integrative methods for multi-omics data.

Authors:  Jingwen Yan; Shannon L Risacher; Li Shen; Andrew J Saykin
Journal:  Brief Bioinform       Date:  2018-11-27       Impact factor: 11.622

10.  Mergeomics 2.0: a web server for multi-omics data integration to elucidate disease networks and predict therapeutics.

Authors:  Jessica Ding; Montgomery Blencowe; Thien Nghiem; Sung-Min Ha; Yen-Wei Chen; Gaoyan Li; Xia Yang
Journal:  Nucleic Acids Res       Date:  2021-07-02       Impact factor: 16.971

View more

北京卡尤迪生物科技股份有限公司 © 2022-2023.