iPiDA-LTR: Identifying piwi-interacting RNA-disease associations based on Learning to Rank.

Piwi-interacting RNAs (piRNAs) are regarded as drug targets and biomarkers for the diagnosis and therapy of diseases. However, biological experiments cost substantial time and resources, and the existing computational methods only focus on identifying missing associations between known piRNAs and diseases. With the fast development of biological experiments, more and more piRNAs are detected. Therefore, the identification of piRNA-disease associations of newly detected piRNAs has significant theoretical value and practical significance on pathogenesis of diseases. In this study, the iPiDA-LTR predictor is proposed to identify associations between piRNAs and diseases based on Learning to Rank. The iPiDA-LTR predictor not only identifies the missing associations between known piRNAs and diseases, but also detects diseases associated with newly detected piRNAs. Experimental results demonstrate that iPiDA-LTR effectively predicts piRNA-disease associations outperforming the other related methods.

This is a PLOS Computational Biology Methods paper.

Introduction

Piwi-interacting RNA (piRNA) with 24–31 nucleotides in length is a class of small RNAs interacting with Piwi-subfamily Argonaute proteins [1-3]. Early studies found that piRNAs mainly located in germ stem cells on drosophila and mouse, and regulated germ stem cell proliferation [4-6]. With the fast development of biotechnology and computing techniques [7, 8], more and more piRNAs were discovered, and the corresponding functions were also detected, including stem cell proliferation, gene expression, and heterochromatin formation, etc [9-12].

As more and more piRNA functions were detected, many evidences indicated that dysfunction and abnormal expression of piRNAs are closely associated with the emergence and development of diseases [13-17]. Therefore, the identification of associations between piRNAs and diseases is important for diagnosis and treatment of diseases [18, 19]. Currently, it mainly focused on biological experimental methods and computational methods. For biological experiments methods, Cabral et al. indicated that piRNAs play a role in the process of translational research of gastric cancer as potential biomarkers [20]. Krishnan et al. identified eight non-redundant piRNAs as breast cancer markers [21]. Roy et al. studied the reciprocal expression between piRNAs and the corresponding targets, and provided a novel insight into the role of piRNAs in Alzheimer’s disease [22]. Although biological experimental methods are highly reliable, it takes substantial time and resources. Some computational methods have been proposed for identifying the associations between non-coding RNAs and diseases, such as miRNA-disease associations [23], circRNA-disease associations [24], etc. In this regard, computational methods are proposed to predict piRNA-disease associations, which can serve as powerful auxiliary tools to save time and cost compared with biological experiments. For example, Wei et al. proposed the first computational predictor for identifying piRNA-disease associations based on the positive unlabelled learning algorithm, and established the first web server [25]. A convolutional neural network was utilized to extract association features between piRNAs and diseases, and then the Support Vector Machine was employed to construct the predictor [26]. Although computational methods have been proposed, they mainly aim at the application scenario of identifying missing associations between known piRNAs and diseases. However, more and more newly detected piRNAs were detected [27-29]. Therefore, the application scenario of identifying piRNA-disease associations of newly detected piRNAs is very important to investigate piRNA functions and disease pathogenesis.

In recent years, information retrieval (IR) becomes a widely used technology, whose ultimate goal is to rank documents based on the relevance to certain topics [30, 31]. As an successful algorithm in information retrieval, Learning to Rank (LTR) [32, 33] has been successfully applied to web page retrieval employed by Google [34], Yahoo [35], Microsoft [36], etc. Compared with traditional IR methods, the advantage of LTR is that it integrates component methods so as to automatically rank documents associated with query from multiple perspectives [30]. LTR has been applied in identifying circRNA-disease associations [24], detecting protein remote homology [37], predicting protein-phenotype associations [38], drug–target binding affinity prediction [39], etc. The core concept of LTR is to calculate the relevance score f(q, d) between query q and document d. Therefore, this task is particularly similar with identification of piRNA-disease associations (see ). PiRNAs and diseases can be treated as queries and documents, respectively. Learning to Rank not only identifies associations between known piRNAs and diseases, but also ranks diseases associated with newly detected piRNAs.

In this study, we propose a new predictor, named iPiDA-LTR, to predict associations between piRNAs and diseases, which has the following advantages. iPiDA-LTR predictor combines component methods and Learning to Rank, which cannot only identify missing associations between known piRNAs and diseases, but also can identify diseases associated with newly detected piRNAs. Experimental results indicated that iPiDA-LTR is promising to identify piRNA-disease associations. A web server of iPiDA-LTR is constructed to identify diseases associated with query piRNAs, which can be accessed at http://bliulab.net/iPiDA-LTR.

Materials and methods

Materials

To imitate two application scenarios, we construct two types of datasets based on piRDisease v1.0 database [40] collecting 7939 piRNA-disease associations with 4796 piRNAs and 28 diseases. Firstly, a standard dataset is constructed following [25], which can be represented as: where represents 5002 piRNA-disease associations from [25]. and contain 4350 piRNAs and 21 diseases from , respectively. and contain known piRNA-disease associations and unknown piRNA-disease associations, respectively. Specifically, piRNA-disease associations contained in are labelled as 1, otherwise 0. To avoid overfitting problem, is further divided into a benchmark dataset and an independent dataset. The benchmark dataset is used to adjust parameters and train model via cross-validation, and the independent dataset is employed to evaluate the performance of different methods.

For the first application scenario: predicting associations between known piRNAs and known diseases

Benchmark dataset and independent dataset are constructed as: where we randomly select 20% associations from and to construct and , respectively, and then the remaining associations in and are used to construct and , respectively. Obviously, represents benchmark dataset, which is used to optimize parameters and train models, and then trained models are used to identify unknown associations in .

For the second application scenario: predicting the associations between newly detected piRNAs and known diseases

To imitate the second application scenario, we randomly select 80% and 20% piRNAs from as known piRNA set and newly detected piRNA set , respectively, based on which benchmark dataset and independent dataset are constructed as: where and represent benchmark dataset and independent dataset, respectively. PiRNAs contained in and belong to and , respectively. Detailed information of , , and is shown in . The datasets can be obtained at http://bliulab.net/iPiDA-LTR/dataset/.

Method overview

In this study, a novel ranking framework, named iPiDA-LTR, is proposed to solve two application scenarios. The workflow of iPiDA-LTR is shown in with three steps: (a) Association feature extraction; (b) Component methods; (c) Ranking diseases associated with query piRNAs.

The workflow of iPiDA-LTR predictor.

(i) Association feature extraction: piRNA sequences and disease ontology are used to calculate piRNA sequence similarities and disease semantic similarities by combining them and piRNA-disease associations to construct association features and labels; (ii) Component methods: four methods are used to train models with benchmark dataset, and then trained models are utilized to calculate association scores of query piRNAs; (iii) Ranking diseases associated with query piRNAs: association scores of samples in the benchmark dataset are used to train LambdaMART model, and then trained LambdaMART model is employed to rank diseases associated with query piRNAs.

Association feature extraction

PiRNA sequence similarities

The piRNA similarities play a vital role in RNA-disease association identification [24-26], and piRNA sequence similarities have been applied to piRNA-disease association identification [25, 26]. Many methods have been proposed to calculate sequence similarities [41-43]. For example, Smith-Waterman algorithm has been successfully applied to multiple sequence analysis tasks, including RNA sequence similarity analysis [25, 26, 44], protein sequence analysis [45, 46], etc. In this study, we employ Smith-Waterman algorithm [41, 44] to calculate piRNA sequence similarities: where S(p, p) is similarity between piRNA p and piRNA p. SW(p, p) represents local alignment score between piRNA p and piRNA p based on Smith-Waterman algorithm.

Disease semantic similarities

The disease semantic similarity calculation is a key component in RNA-disease association identification. The disease ontology [47] has been applied to RNA-disease association identification so as to calculate disease semantic similarities [48-53]. Disease ontology organized by the directed acyclic graph (DAG) provides a hierarchical structure of the complex disease parent node [47]. Similar diseases share similar hierarchical structure in DAG of disease ontology. Therefore, DAG of disease ontology helps to measure similarity between two diseases. In this study, we use DAG of disease ontology to calculate disease semantic similarities [54, 55]: where S(m, n) is similarity between disease m and disease n. T represents the node set containing the ancestor nodes of k and itself. S(i) is the semantic value of node i to node n.

Association features and labels

The association feature between disease d and the query piRNA p is: where is the association features of piRNA p and disease d. S(p,:) and S(d,:) represent pth row and dth row in the S and S, respectively. If piRNA p is associated with disease d, the label of is equal to 1, otherwise 0.

Component methods

In this study, we select two types of component methods to calculate association scores, including machine learning methods and collaborative filtering (CF). For machine learning methods, Random Forest (RF) [56-60], Logistic Regression method (LR) [61], and Support Vector Machine (SVM) [62-64] are employed, treating the identification of piRNA-disease association as a classification problem. CF is a recommendation algorithm [65, 66], which utilizes guilt-by-association assumption to identify piRNA-disease association focusing on local information. In this study, association features of benchmark dataset (see ) are used to train machine learning models, and then used to calculate association scores for dataset. Finally, association features between piRNA p and disease d can be represented as: where represents association features of piRNA p and disease d. , , and are association scores between piRNA p and disease d calculated by CF, LR, RF and SVM, respectively.

Ranking diseases associated with query piRNAs

In this study, we employ Learning to Rank (LTR) to solve the problem of identifying potential piRNA-disease associations motivated by information retrieval [24, 37, 38, 67]. LTR is generally classified into three categories, including ListWise, PairWise and PointWise [68]. In this study, a ListWise method LambdaMART [32] is selected to obtain high quality of top-ranked diseases, which has been applied in identifying circRNA-disease associations [24], detecting protein remote homology [37], predicting protein-phenotype associations [38] and drug–target binding affinity prediction [39]. The number of trees, the truncation level k, shrinkage and the number of leaves are the four main parameters. The truncation level of k influences the quality of top-ranked results by Normalized Discounted Cumulative Gain (NDCG), which can be formulated as [32]: where k represents the truncation level. IDCG@k is the value of DCG@k in the best optimal ranking results. If a query piRNA is associated with disease located in position i, rel is equal to 1, otherwise 0. To obtain the final ranking results, association scores calculated by for training set are used to train LambdaMART model, and the trained LambdaMART model is employed to rank diseases associated with query piRNAs based on association scores of query piRNAs.

Results and discussion

Evaluation criteria

In this study, the benchmark dataset is employed to optimize the parameters of the models, and the independent dataset is used to evaluate the performance of predictors. How to evaluate the ranking quality and prediction performance is crucial for identifying piRNA-disease associations. Because iPiDA-LTR predictor treats the identification of piRNA-disease associations as an information retrieval ranking task, we employ three important ranking criteria to evaluate the rank quality of different predictors: Normalized Discounted Cumulative Gain (NDCG), Mean Average Precision (MAP) and ROCk. Besides, Area Under the ROC Curve (AUC) and Area Under the Precision-Recall Curve (AUPR) are also used to measure comprehensive performance [69-72]. The average values of these criteria for all query piRNAs are calculated to evaluate performance of predictors.

The effect of parameters for identifying piRNA-disease associations

iPiDA-LTR predictor mainly contains the following four parameters: the number of trees, the truncation level k, shrinkage and the number of leaves. Due to the large number of combinations of the four parameters, we fix three parameters in turns, and then find the local optimal values of the remaining parameters according to AUPR. The influences of different combinations of parameters for iPiDA-LTR on dataset and dataset are shown in Figs and , respectively, from which we can see that the final optimized combinations of four parameters on iPiDA-LTR predictor on dataset and dataset are (120, 14, 0.22, 3) and (30, 15, 0.10, 29), respectively.

The predictive results of iPiDA-LTR predictor on dataset via five-fold cross-validation.

(a) The truncation level k, shrinkage and the number of leaves are assigned as 10, 0.10, and 10 respectively, which are RankLib’s default values (https://sourceforge.net/p/lemur/wiki/RankLib/), and the optimal value of the number of trees is 120; (b) The number of trees, shrinkage and the number of leaves are fixed as 120, 0.10 and 10 respectively, and the truncation level k is optimized as 14; (c) The number of trees, the truncation level k and the number of leaves are fixed as 120, 14 and 10 respectively, and the shrinkage is optimized as 0.22; (d) The number of trees, the truncation level k and shrinkage are 120, 14 and 0.22 respectively, and the number of leaves is set as 3.

(a) The truncation level k, shrinkage and the number of leaves are assigned as 10, 0.10, and 10 respectively, which are the RankLib’s default values (https://sourceforge.net/p/lemur/wiki/RankLib/), and the optimal value of the number of trees is 30; (b) The number of trees, shrinkage and the number of leaves are fixed as 30, 0.10 and 10 respectively, and the truncation level k is optimized as 15; (c) The number of trees, the truncation level k and the number of leaves are fixed as 30, 15 and 10 respectively, and the shrinkage is optimized as 0.10; (d) The number of trees, the truncation level k and shrinkage are 30, 15 and 0.1 respectively, and the number of leaves is optimized as 29.

Complementary analysis for component methods

In this study, iPiDA-LTR incorporates two types of component methods, including machine learning methods (LR, RF and SVM) and collaborative filtering (CF). LR, RF and SVM are obtained by python package Scikit-learn [73]. For LR’s parameters, max_iter and solver are assigned as 300 and liblinear, respectively. For RF’s parameters, n_estimators, max_leaf_nodes, n_jobs and max_features are assigned as 80, 10, -1 and 0.2, respectively. For SVM’s parameters, kernel and probability are assigned as linear and True, respectively. We analyze the impact of different types of component methods to identify associations between piRNAs and diseases, and the results are shown in Tables and , from which we can see the followings: (i) iPiDA-LTR predictor outperforms iPiDA-LTR-ML predictor on dataset and dataset; (ii) The iPiDA-LTR obviously outperforms iPiDA-LTR-ML in terms of ranking criteria (NDCG@5 and ROC1), especially for the second application scenario (see ). Machine learning methods based on classification algorithms focus on global predictive performance, and collaborative filtering can identify special piRNA-related diseases focusing on local predictive performance. Therefore, machine learning methods and collaborative filtering are complementary. It is not surprising that iPiDA-LTR predictor obtains the best performance compared with iPiDA-LTR-ML, because iPiDA-LTR shares the advantages of these two types of methods.

The usage frequencies of component methods measure the contribution of component methods for iPiDA-LTR. shows the usage frequencies of component methods on iPiDA-LTR, from which we can see that each component method is frequently used, indicating that they are important for iPiDA-LTR. Tables and and show that component methods are complementary, and iPiDA-LTR combines them leading to better performance for identifying piRNA-disease associations.

The usage frequencies of component methods, calculated by Apache commons-math3 library and RankLib library.

(a) shows the usage frequencies of component methods on dataset via five-fold cross-validation; (b) shows the usage frequencies of component methods on dataset via five cross-validation.

Comparison with related methods

In this section, the two state-of-the-art predictors including iPiDi-PUL predictor [25] and iPiDA-sHN predictor [26] are compared with iPiDA-LTR predictor, and the results are shown in Tables and , from which we can see that iPiDA-LTR is better than the other methods, indicating that iPiDA-LTR is more suitable for identifying piRNA-disease associations. Researchers tend to focus on the top ranked predicted associations in practical application scenarios. Therefore, we analyze the quality of the predicted results (see ), from which we can see that iPiDA-LTR outperforms the other predictors in terms of ROC1-ROC10. It is not surprising because the loss function of LambdaMART NDCG mainly focuses on the top-ranked predictive known associations (see ).

The comparison results of different methods.

(a) and (b) are based on dataset and dataset, respectively.

Case study

To illustrate the predictive performance of iPiDA-LTR predictor for the identification of associations between new piRNAs and diseases, two query piRNAs, including piR-hsa-23210 and piR-hsa-15023, are selected as query piRNAs from dataset, respectively. The remaining piRNAs in are used to train iPiDA-LTR model, and then the trained iPiDA-LTR model is employed to predict diseases associated with piR-hsa-15023 and piR-hsa-23210.

The predicted results of piR-hsa-23210 and piR-hsa-15023 are shown in Tables and respectively, from which we can see the followings: (i) The evidences for the top five predicted piR-hsa-23210-associated diseases are supported by PubMed (https://pubmed.ncbi.nlm.nih.gov/). For example, the target gene of piR-hsa-23210 is SMC5, which plays crucial roles in the process of human spermatogenesis, such as on the synaptonemal complex between synapsed chromosomes, and in the development of spermatogonial cells [74]. Roy et al. found that piR-33044 (piR-hsa-23210) is significantly abnormal expression in Alzheimer Disease [22]. (ii) Four diseases in have been proved to be associated with piR-hsa-15023. For example, Busch et al. found that piR-hsa-15023 is down-regulated in renal cell carcinoma [75]. piR-hsa-15023 showed a significantly differentially expression in gastric adenocarcinoma and non-malignant stomach tissue [76]. Therefore, these results demonstrated that iPiDA-LTR predictor is an effective approach to identify associated diseases for newly detected query piRNAs.

Conclusion

In this study, we treat the task of piRNA-disease associations as a search task based on Learning to Rank [32, 68], where piRNA and disease are regarded as query and document, respectively. The following conclusions can be drawn: (i) iPiDA-LTR can effectively handle with two types of application scenarios compared with the other state-of-the-art methods, especially for the identification of diseases associated with newly detected piRNAs, which is important for studying the pathogenesis of disease and the function of piRNAs; (ii) iPiDA-LTR incorporates component methods into Learning to Rank so as to improve the predictive performance; (iii) The corresponding web server of iPiDA-LTR is freely accessed at http://bliulab.net/iPiDA-LTR/. Although iPiDA-LTR effectively predicts piRNA-disease associations, it only integrates basic machine learning methods and collaborative filtering. In future studies, we will integrate the other state-of-the-art methods and features to improve piRNA-disease associations. The LTR-based framework discussed in this study is a general framework, which would have many other applications in bioinformatics, such as protein function prediction, remote homology detection, etc.

Table 1

The detailed statistical information of , , and .

Datasets	PiRNAs	Diseases	Known*	Unknown#
Sbena	4350	21	4002	69079
Sinda	4311	21	1000	17269
Sbenp	3480	21	3999	69081
Sindp	870	21	1003	17267

The associations between piRNAs and diseases have been validated by experiments

The associations between piRNAs and diseases without experiment validations.

Table 2

The comparison results of predictors based on Learning to Rank integrating different component methods via five-fold cross-validation on dataset.

	AUC	AUPR	NDCG@5	MAP	ROC1	ROC3	ROC5
iPiDA-LTR-ML a	0.9511	0.9003	0.9492	0.9305	0.8678	0.9407	0.9503
iPiDA-LTR b	0.9543	0.9111	0.9545	0.9379	0.8822	0.9457	0.9538

a The component methods include RF, LR and SVM

b The component methods include RF, LR, SVM and CF.

Table 3

The comparison results of predictors based on Learning to Rank integrating different component methods via five-fold cross-validation on and dataset.

	AUC	AUPR	NDCG@5	MAP	ROC1	ROC3	ROC5
iPiDA-LTR-ML a	0.9544	0.6243	0.7722	0.7299	0.5103	0.7779	0.8478
iPiDA-LTR b	0.9558	0.6695	0.7884	0.7581	0.5725	0.7918	0.8515

a The component methods include RF, LR and SVM

b The component methods include RF, LR, SVM and CF.

Table 4

The comparison results between iPiDA-LTR and two state-of-the-art predictors on dataset.

	AUC	AUPR	NDCG@5	MAP
iPiDi-PUL	0.9153	0.8511	0.9190	0.8847
iPiDA-sHN	0.8042	0.7023	0.8198	0.7705
iPiDA-LTR	0.9521	0.8987	0.9472	0.9283

Note: iPiDi-PUL and iPiDA-sHN are reproduced, and their parameters are set as the optimized values reported in [25] and [26], respectively.

Table 5

The comparison results between iPiDA-LTR and two state-of-the-art dataset.

	AUC	AUPR	NDCG@5	MAP
iPiDi-PUL	0.9413	0.6154	0.7736	0.7110
iPiDA-sHN	0.8015	0.3702	0.4875	0.4583
iPiDA-LTR	0.9623	0.6780	0.8067	0.7697

Note: iPiDi-PUL and iPiDA-sHN are reproduced, and their parameters are set as the optimized values reported in [25] and [26], respectively.

Table 6

The top five piR-hsa-23210 associated diseases and relevant evidences.

Rank	disease name	Evidence
1	Cardiovascular diseases (CDC, CF, CCS) cardicregeneration	PMID: 28289238
2	Renal Cell Carcinoma	PMID: 25998508
3	Alzheimer Disease	PMID: 28127595
4	Male Infertility	PMID: 24855106
5	Gastric Cancer	PMID: 25779424

Note: the evidences can be found at https://pubmed.ncbi.nlm.nih.gov/.

Table 7

The top five piR-hsa-15023 associated diseases and relevant evidences.

Rank	disease name	Evidence
1	Cardiovascular diseases (CDC, CF, CCS) cardicregeneration	PMID: 28289238
2	Renal Cell Carcinoma	PMID: 26071182
3	Alzheimer Disease	PMID: 28127595
4	Rheumatoid Arthritis	None
5	Gastric Cancer	PMID: 25779424

Note: the evidences can be found at https://pubmed.ncbi.nlm.nih.gov/.