Computational Prediction of Ubiquitination Proteins Using Evolutionary Profiles and Functional Domain Annotation.

BACKGROUND: Ubiquitination, as a post-translational modification, is a crucial biological process in cell signaling, apoptosis, and localization. Identification of ubiquitination proteins is of fundamental importance for understanding the molecular mechanisms in biological systems and diseases. Although high-throughput experimental studies using mass spectrometry have identified many ubiquitination proteins and ubiquitination sites, the vast majority of ubiquitination proteins remain undiscovered, even in well-studied model organisms.
OBJECTIVE: To reduce experimental costs, computational methods have been introduced to predict ubiquitination sites, but the accuracy is unsatisfactory. If it can be predicted whether a protein can be ubiquitinated or not, it will help in predicting ubiquitination sites. However, all the computational methods so far can only predict ubiquitination sites.
METHODS: In this study, the first computational method for predicting ubiquitination proteins without relying on ubiquitination site prediction has been developed. The method extracts features from sequence conservation information through a grey system model, as well as functional domain annotation and subcellular localization.
RESULTS: Together with the feature analysis and application of the relief feature selection algorithm, the results of 5-fold cross-validation on three datasets achieved a high accuracy of 90.13%, with Matthew's correlation coefficient of 80.34%. The predicted results on an independent test data achieved 87.71% as accuracy and 75.43% of Matthew's correlation coefficient, better than the prediction from the best ubiquitination site prediction tool available.
CONCLUSION: Our study may guide experimental design and provide useful insights for studying the mechanisms and modulation of ubiquitination pathways. The code is available at: https://github.com/Chunhuixu/UBIPredic_QWRCHX.

INTRODUCTION

As a well-known post-translational modification (PTM), ubiquitination is crucial in proteome dynamics and various signaling pathways in the cells [1]. Ubiquitination is an enzymatic PTM in which ubiquitin (a small regulatory protein) [2] is attached to a lysine residue of the targeting protein [3]. Ubiquitination marks proteins for degradation through the proteasome [4], alters their cellular location [5], and regulates protein interactions [5]. It is involved in signal transduction [6], apoptosis, endocytosis, gene transcription, DNA repair, and replication, intracellular trafficking, virus budding [3], cellular transformation, immune response, and inflammatory response [7].

Due to its importance and complexity, the identification of ubiquitination proteins and ubiquitination sites is highly valuable. However, experimental identification is time-consuming and expensive [8] particularly because the ubiquitination process is dynamic, rapid and reversible [9-11]. Hence, computational predictions become an important and practical alternative. A number of computational methods were developed based on the traditional machine learning method for predicting lysine ubiquitination sites, including Radivojac’s UbPred [12], Cai’s mRMR model [13], Zhao’s ensemble classifier [14], and Chen’s CKSAAP approach [15], but they can only identify ubiquitination sites with limited accuracies. It is not practical to predict ubiquitination proteins through these methods since the false-positive rates would be too high to be useful. Recently, deep learning method tools, such as MusiteDeep-Capsule [16] have become competitive due to the advancing of computing resources, our results show good performance even when compared with it, also, it has a better performance than three machine learning-based tools, UbiProber [17], UbiSite [18] and PDM-PUB [19]. To the best of our knowledge, so far no computational method has been developed to predict whether an uncharacterized protein is able to be ubiquitinated or not. The present study was initiated in an attempt to address this problem for the first time. If it can be predicted whether an uncharacterized protein can be ubiquitinated or not, it will be valuable for the prediction itself and helpful for identifying ubiquitination sites. Although this problem has not been addressed, some explorations have been made on a related study, i.e. to predict whether a protein can be phosphorylated or not [20, 21]. A method has been presented for identifying human phosphorylated proteins by incorporating evolutionary information into a general pseudo amino acid composition (PseAAC) model through a grey system [21-23]. It is believed that the formulation and approach can be also used to predict ubiquitination proteins. Ubiquitination is much less frequent than phosphorylation, as ubiquitination is a much more “expensive” biological operation than phosphorylation. The smaller group of ubiquitination proteins may have even stronger common features than phosphorylation proteins so that a whole sequence-based method may work better in predicting ubiquitination proteins. Furthermore, other gene features will be used, such as Gene Ontology (GO) [24], a structured repository of concepts (GO Terms) related to gene functions for the prediction, none of which was used in predicting phosphorylation sites.

In this study, a novel computational method, was developed to predict ubiquitination proteins for a query amino acid sequence on the basis of its evolutionary information through a grey system model [25] and K Nearest Neighbour (KNN) scores calculated with the fuzzy distance by using its Functional Domain Annotation (FDA) and subcellular localization. There are two major feature sets in this study: one set includes 80 sequence grey model features extracted from the sequence evolution information and another contains the features calculated by KNN scores based on FDAs. To thoroughly evaluate the proposed model, it was trained and tested with different datasets and cross-validations methods. In addition, the distribution of the above-mentioned features in predicted ubiquitination proteins was analyzed and it provided some hypotheses for distinguishing ubiquitination proteins from non-ubiquitination ones.

MATERIALS AND METHODS

Benchmark Dataset

The dataset used was extracted from Uniprot at http://www.ebi.ac.uk/uniprot [26]. The version of protein data used in the current study was released on May 2017. The positive dataset containing 1906 known ubiquitination proteins was generated through the following queries in the UniProt advanced search: “annotation: (type:crosslnk ubiquitin) length: [50 TO *] AND reviewed: yes.” Three hundred of 1906 proteins were separated as an independent test dataset so that the remaining 1606 positive proteins were kept in training and validation. For the negative dataset, we started from all reviewed proteins (~550,000 totals) and performed a filtering process by using CD-HIT-2D [27] with a threshold of 70%, after this step, there were 320,096 negative proteins left. To conduct balanced training, these samples were randomly taken to form three negative datasets, in which the number of samples was the same as the given positive dataset. At this time, a 300 proteins negative independent dataset was randomly selected and isolated for testing. There is no overlap between training and testing datasets. For the annotation information, we extracted the 8 types UniProt annotations of ‘Subcellular localization (SL) [28]’ and FDAs of ‘GO [29]’, ‘Pfam [30]’, ‘Smart [31]’, ‘PROSITE [32]’, ‘SUPFAM [33]’, ‘InterPro [34]’, and ‘PRINTS [35]’ for all the proteins in the datasets. SL was reorganized by the UniProt build-in hierarchical subcellular localization table.

Incorporate Extracted Features into the General Pseudo amino Acid Composition

It is known that most traditional machine-learning algorithms, such as Neural Network [36], Covariant Discriminant [37], Support Vector Machine [38], K Nearest Neighbor [39], and Random Forest [40], can only handle vector but not sequence samples. To formulate a biological sequence of a variable length into a discrete model or a vector, yet still considerably keep its sequence pattern or inherent characteristics, researchers formulated the protein sequence or peptides using pseudo amino acid composition (PseAAC) [21], encoding method [41] or other approaches [42]. Here, a model following the general form of PseAAC [43] has been proposed, which formulates a protein P as (Eq. 1):

where T is a transpose operator, the subscript is an integer, and its value as well as the components P, … depend on the extraction of the desired information from the amino acid sequence of P as described below.

Vectorization of Sequence Profile through a Grey System Model

From the evolutionary viewpoint, all the protein sequences have been evolved from a very limited number of ancestral species. Their evolution involves mutations of single residues, as well as insertions and deletions of residues, gene duplication, and gene fusion. With these changes accumulated for a long period of time, many similarities between the original and evolved amino acid sequences have gradually disappeared, but they may still share some common features, such as belonging to the same type of protein [44], residing in a same subcellular location [45], or having a similar biological function [46]. It is assumed that ubiquitination proteins have evolutionary relationships that are reflected in some common attributes encoded in sequence profiles, i.e. the Position Specific Scoring Matrix (PSSM), as described below. The sequence profile by a L×20 matrix as P is given as:

where L is the number of residues of the protein, represents the profile score in the i-th (i = 1,2, ⋯, L) position of the protein having amino acid type j (j = 1,2, ⋯, 20) during the evolution. Here, the numerical codes 1, 2, …, 20 are used to denote the 20 natural amino acid types according to the alphabetical order of their single-character codes. The L×20 scores in Eq. 2 were generated by using PSI-BLAST [47] to search the UniProtKB/Swiss-Prot database through three iterations with 0.001 as the E value cutoff for protein P. Following the same approach as done by Lin et al. [48], PSSM was encoded to a fixed-size vector with a total of quantities by using the first-order grey model, or a total of quantities by the second-order model. Thus, the components of a given protein sample can be represented by Eq. 3 or Eq. 3’ (which is called SeqEvo Descriptor), for the first-order and the second-order models, respectively.

where are the parameters of the first-order model for the th amino acid (); are the parameters of the second-order model for the th amino acid ().

KNN Score based on FDA and Subcellular Localization

A set of complementary FDAs and subcellular localization were used as features of ubiquitination proteins, including (1) UniProt annotations of subcellular localization and FDAs of ‘GO’, ‘Pfam’, ‘Smart’, ‘PROSITE’, ‘SUPFAM’, ‘InterPro’, and ‘PRINTS’, as well as GO annotations with categories of molecular function, biological process, and cellular component; (2) Pfam, a large collection of protein families generated using hidden Markov models; (3) Smart [31], a collection of protein domains and domain architectures; (4) PROSITE [32], a database of protein families, domains and functional sites; (5) SUPFAM, a database of protein structural and functional annotation; (6) InterPro, a resource of protein families, domains and important sites; (7) PRINTS, a collection of sequence “fingerprints” and protein families. Studies have shown that ubiquitination proteins often share the same subcellular localization [49]. Hence, subcellular localizations were also used as a feature for predicting ubiquitination proteins. These features were used based on a KNN algorithm as follows:

Step 1. For a query protein sequence, find its nearest neighbors in the whole set, including positive and negative samples, according to local sequence similarity. For proteins and , let:

Which represents the j-th feature of FDA of and , respectively. j =1,2,…,7,8 represents ‘GO’, ‘Pfam’, ‘Smart’, ‘PROSITE’, ‘SUPFAM’, ‘InterPro’, ‘PRINTS’ or ‘subcellular localization’, respectively, and the distance Dist ()between is defined as follows in Eq. (4):

where and represent the “union” and “intersection” in the set theory, and || || is the operator acting on the set therein to count the number of its elements.

Step 2. A corresponding KNN feature is then extracted by calculating the KNN score, represented by the percentage of positive neighbors (ubiquitination proteins) in its nearest neighbors.

Step 3. To take advantage of different properties of neighbors with various similarity cutoffs, Steps 1 and 2 were repeated for different values to obtain multiple features for the ubiquitination protein predictor. In this study, based on empirical trials, by default, was chosen to be 0.1%, 0.4%, 0.7%, …, 14.5% and 14.8%; then the number of features is 50, i.e. 50 KNN scores were extracted as features for predicting ubiquitination proteins. For the j-th member of FDA, the protein P can be formulated as (Eq. 5):

where are the ratios of positive neighbors to the whole samples at 0.1%, 0.4%...14.8% of the training data set size, respectively. Hence, a query protein sequence can be formulated with seven 50-dimension vectors, i.e., by using the FDA database and a 60- or 80- dimension vector for each i.e., or . These digital representations are used as the input of query protein for the prediction model.

Algorithm

Random Forest has been used as the main classifier of the predictor. The workflow (Fig. ) illustrates how our classifier works. In the proposed model, the first step is to input the query amino acid sequence with its FDAs. The next step is to generate two sets of features of a given protein as described above, where the annotation features are encoded into a distance matrix based on the KNN-score extraction, and PSI-BLAST was used to generate the PSSM and then transform into the SeqEvo Descriptor. In the last step, two types of features are assembled to enter the machine learning classifier as input for training.

Method Evaluation

To evaluate the prediction performance of our method, a 5-fold cross-validation test was performed following several widely-accepted measurements: (1) overall accuracy (ACC), the ratio of true positive and true negative sample among all the samples; (2) Mathew’s correlation coefficient or MCC; (3) sensitivity (SN), the ratio between true positive and positive samples; and (4) specificity (SP), the ratio between true negative and negative samples; (5) precision (Pre), the ratio of true positive among the sum of true positive and false positive. As mentioned in Section 2.1, there were 3 sets of negative data, and 3 sets of training data were constructed whose positive datasets were the same. Then training and 5-fold cross-validation were performed 3 times, then all these measurements were calculated from the average of 3 training sets. Furthermore, Receiver Operating Characteristic (ROC) curves were calculated and plotted based on specificities and sensitivities. The Areas under ROC curves (AUCs) were also calculated based on the trapezoidal approximation.

RESULTS

Investigating the Performances of PSSM-Grey Feature

As mentioned above in the introduction, the PSSM-Grey feature is mainly based on the evolutionary conservation of proteins. To determine whether ubiquitination proteins and non-ubiquitination proteins have distinct evolutionary conservation patterns, the error bar of their four components , (Equation 3') was compared between ubiquitination proteins and non-ubiquitination proteins (Fig. S1). The result indicates that the ubiquitination proteins and non-ubiquitination proteins have some different patterns in the four parameters, but not very significant. and were applied in three general machine learning algorithms (Table ). The performance of SVM and RF was better than KNN, but there is a little difference between SVM and RF. The performance of was overall similar to that of .

The GO enrichment network of the training dataset A, B, C indicates the positive datasets of H. sapiens, followed by M. musculus and A. thaliana, D, E, F indicate the negative datasets with the same order of species. The network was generated using Cytoscape [44], packaged in Metascape, with p-value < 0.01, minimum count 3, and enrichment factor > 1.5. A Kappa score of 4 was used as the similarity metric when performing hierarchical clustering on the enriched terms and then sub-trees with similarity > 0.3 were considered a cluster. Each node represents an enriched cluster and colored by its cluster ID as shown in the legend. The edge indicates the number of shared proteins between two-term nodes.

GO Enrichment Analysis

To confirm the classification results, the gene set GO enrichment analysis was performed using Metascape [50]. Here, the analysis of 1906 positive data on three species: Homo sapiens (393 GO terms), Mus musculus (390 GO terms) and Arabidopsis thaliana (277 GO terms) has been perfomed. For 10,000 negative dataset, the numbers of GO annotations was 1059, 1034 and 639, respectively. It shows that in all three species, most ubiquitination proteins belong to a small number of biological annotation terms with small p-values, which indicates that annotations could be useful features for our machine learning approach. Figs. ( and ( show that for H. sapiens and M. musculus, the positive datasets are more centered in the same functional GO term groups, with many linked edges this may be due to the commonness of mammals. The negative datasets for these two species are clustered in independent groups with fewer edges. Such a pattern is less obvious in A. thaliana.

Fig. (2)

GO enrichment analysis of the training dataset. A, B, C indicate the positive datasets of H. Sapiens, followed by M. Musculus and A. Thaliana, D, E, F indicate the negative datasets with same order of species.

Fig. (3)

GO enrichment analysis network visualization of the training dataset.

Subcellular Localization Enrichment Analysis

To study the relationships between ubiquitination proteins and subcellular localization, the enrichment analysis of subcellular localization for this dataset has been performed (Fig. ). As shown in Fig. (, 49.8% of the positive data were labeled with the nucleus location, and 44.3% were localized in the cytoplasm, which is significantly different from the negative data (8.1% and 26.5%, respectively). Also, it is noted that 94.4% positive proteins have more than two localization annotations and 61.1% for negative proteins, especially, for positive data which is localized in the nucleus, around half of them (460 of 950) share other localizations. Some earlier studies have shown that the ubiquitin-related enzymes are highly localization- specific [24], and hence the subcellular localization could be an informative feature to predict ubiquitination proteins.

Fig. (4)

Distribution of subcellular localization of the training data.

Investigating the Performances of KNN Score of Features

It was found that 5,184 GO terms were involved in the training dataset, of which 3,012 appeared in the set of ubiquitination proteins, 3,565 appeared in non-ubiquitination proteins, and only 1,393 GO terms were shared by both positive and negative datasets. Hence, the functional properties of the two groups are significantly different, as consistently shown in Figs. ( and ). Following this idea, the KNN scores of ubiquitination proteins were compared with those of non-ubiquitination proteins on all the FDA features (Fig. S2). Overall, ubiquitination proteins gained obvious larger KNN scores which are greater than 0.5 (i.e., with significant information content as prediction feature; the larger, the more significant) on GO and subcellular localization, and a slightly larger score greater than 0.5 in the Smart, SUPFAM, and InterPro.

Specifically, for ubiquitination proteins, the average KNN scores of GO with different sizes of nearest neighbors were within 0.5 - 0.8, and for non-ubiquitination proteins, the average KNN scores were within 0.2 - 0.4. For subcellular localization, the average KNN scores of ubiquitination proteins were in the range of 0.5 - 0.7, while those of non-ubiquitination proteins fluctuated around 0.4. For Smart, SUPFAM, and InterPro, there was no clear gap between the ubiquitination proteins and non-ubiquitination proteins, especially with the growth of KNN cutoffs. Subsequently, the eight types of features were tested on the three datasets with KNN, RF and SVM algorithms on the training dataset, and the mean performance of these three algorithms is listed in Table . The best results of accuracy for RF, SVM, and KNN are 0.88 (using InterPro), 0.85 (using Pfam) and 0.85 (using Pfam), respectively. The Random Forest algorithm has the best performance on all the features of accuracy, where the accuracy is between 0.70 - 0.88. Hence, Random Forest has been selected as our classifier.

Performance of the Proposed Model

Since the combined features generated a high-dimensional vector output, the Relief method [25] can be used to rank the values of the underlying features. To evaluate the performance of our method for different features, 5-fold cross-validation has been performed (Tables and for training and testing, respectively). In general, the evaluation result was the best when all features were included reaching the accuracy of 87.71% and MCC of 75.43%. The performance of the proposed models was further illustrated by the ROC analysis (Fig. ), especially using AUC (Area Under the Curve). The AUC value is a number between 0 and 1, and the greater the AUC value, the better is the predictor. The AUC value of the proposed model is 0.8507 (Go), 0.8509 (PFAM), 0.8502 (SMART), 0.8495 (PROSITE), 0.8513 (SUPFAM), 0.8501 (INTERPRO), 0.8497 (PRINTS), 0.8476 (Subcellular localization), 0.9396 (GreyPssm) and 0.9598 (All).

Testing Data Performance and Comparison with Ubiquitination Site Prediction

A balanced independent test dataset was used to evaluate our model in comparison with ubiquitination site prediction tools. The results were based on five-fold cross-validation, with the model including all features together for our tool. The results were compared with a deep learning ubiquitination site prediction tool, MusiteDeep-Capsule [16]; a machine learning approach based tool, UbiProber [17]; an SVM based tool, UbiSite [18]; and a Bayesian Discriminant Method based tool, BDM-PUB [19]. Since existing prediction tools are designed for site prediction, their site prediction results were transformed to ubiquitinated protein results by using the following strategy: if any site from a given protein was predicted as ‘positive’ or ‘ubiquitinated’, the whole protein was labeled as ‘positive’ as well; if multiple sites were predicted as positive, the max predicted score was picked from them for generating the Receiver Operator Curve (ROC). Their pre-trained models were used with default parameters to conduct the prediction comparison with our method. For MusiteDeep-Capsule (another in-house tool), the same test dataset was used to train the models and the same independent test dataset was used to perform the comparison. For other tools, they do not provide customized model training, therefore their pre-trained model was used to predict for the same testing dataset. The results are shown in Fig. (, in which our tool showed a better performance than other tools; in particular, our tool has a significantly lower false-positive discovery rate and a higher true positive discovery rate than other tools.

Fig. (6)

ROC curves to show the performance comparison with other prediction tools. AUC indicates the area under the curve.

DISCUSSION

In order to detect ubiquitination proteins, a method was developed based on the Random Forest algorithm using the sequence conservation information, as well as the information of ‘GO’, ‘Pfam’, ‘Smart’, ‘PROSITE’, ‘SUPFAM’, ‘InterPro’, ‘PRINTS’ and subcellular localization of the query protein. The features only incorporate the sequence conservation using a grey system model and KNN scores based on protein annotation databases. This method achieved an overall accuracy of 90.03%, MCC of 80.13%, Sn of 87.94%, Sp of 92.13% and Precision of 91.78%, which indicates that this method reflects the sequence patterns well, containing the ubiquitination sites. Since our method could do the prediction without relying on sequence profiles, it can scan a batch of unknown proteins very efficiently. In addition, our method showed better performance than the existing tools for the protein level prediction of ubiquitination. The user may apply our predictor to select potential candidates before doing the site prediction or the lab work.

CONCLUSION

Our study may guide experimental design and provide useful insights for studying the mechanisms and modulation of ubiquitination pathways. The comparison results indicate that we have an advantage in ubiquitination prediction at the protein level. It may improve the sensitivity when conducting the ubiquitination site prediction if our method is applied first to remove the false positive samples. In addition, it may help accelerate the expensive and time-consuming process of identifying ubiquitination proteins with known annotations.

Table 1

Performance comparison of PSSM-Grey by a 5-fold cross-validation.

%	PSSM - Grey (80)				PSSM - Grey (60)
	ACC	MCC	SN	SP	ACC	MCC	SN	SP
KNN	79.99	60.52	86.63	73.35	80.43	61.37	86.79	74.08
SVM	88.68	77.60	84.75	92.61	87.62	75.28	86.00	89.24
RF	86.21	72.44	87.24	85.19	86.19	72.41	87.24	85.15
Average	84.96	70.19	86.20	83.72	84.75	69.68	86.68	82.82

Note: The abbreviations in the table are: Accuracy (ACC), Matthews Correlation Coefficient (MCC), Sensitivity (SN) and Specificity (SP).

Table 2

A comparison of eight features with different algorithms.

%	ACC	MCC	SN	SP
Algorithm	RF SVM KNN	RF SVM KNN	RF SVM KNN	RF SVM KNN
GO	82 81 80	64 62 60	83 79 79	81 83 81
Pfam	87 85 85	75 70 70	83 80 83	92 90 87
Smart	73 72 70	50 50 42	50 49 60	95 95 81
PROSITE	79 78 76	60 59 55	64 61 67	94 94 86
SUPFAM	75 73 74	52 48 50	60 56 60	90 90 88
InterPro	88 83 83	77 67 67	86 79 81	91 88 86
PRINTS	70 70 69	49 49 45	41 41 43	99 99 95
SL*	80 78 77	59 57 54	76 78 76	83 79 77

Note: The abbreviations in the table are: Accuracy (ACC), Matthews Correlation Coefficient (MCC), Sensitivity (SN) and Specificity (SP). Three algorithms, Random Forest (RF), Support Vector Machine (SVM) and KNN (K-Nearest Neighbor) were applied. * indicates Subcellular Localization (SL).

Table 3

A comparison of eight features performance in the training data.

%	ACC	MCC	SN	SP	Precision	Recall
1 GO	82.18	64.39	83.29	81.08	81.52	83.29
2 Pfam	87.35	75.01	82.88	91.83	91.03	82.88
3 Smart	72.62	50.45	50.49	94.75	90.59	50.49
4 PROSITE	78.69	60.24	63.50	93.89	91.23	63.50
5 SUPFAM	75.01	52.30	60.43	89.59	85.34	60.43
6 InterPro	88.42	76.91	86.25	90.59	90.16	86.25
7 PRINTS	70.07	49.06	41.31	98.83	97.24	41.31
8 SL	79.64	59.49	76.00	83.28	82.02	76.00
9 PSSM	86.19	72.40	87.34	85.04	85.37	87.34
Feature(1-8)	89.74	79.53	87.85	91.63	91.30	87.85
Feature(1-9)	90.13	80.34	87.99	92.28	91.93	87.99

Note: The abbreviations in the table are: Accuracy (ACC), Matthews Correlation Coefficient (MCC),

Sensitivity (SN) and Specificity (SP). “Feature(1-8)” indicates that the first 8 features were applied, and “Feature(1-9)” means that all features were applied.

Table 4

A comparison of eight features performance in the test data.

-	ACC	MCC	SN	SP	Precision	Recall
1 GO	77.45	57.40	91.63	63.27	71.51	91.63
2 Pfam	82.39	65.06	78.10	86.67	85.49	78.10
3 Smart	72.61	47.70	56.80	88.43	83.13	56.80
4 PROSITE	78.14	56.86	71.05	85.23	82.79	71.05
5 SUPFAM	72.71	45.78	66.80	78.63	75.81	66.80
6 InterPro	80.16	60.40	81.76	78.56	79.31	81.76
7 PRINTS	70.78	47.54	46.54	95.03	90.37	46.54
8 SL	74.35	49.35	80.46	68.24	72.06	80.46
9 PSSM	85.72	71.45	86.21	85.23	85.40	86.21
Feature(1-8)	86.27	72.57	87.25	85.29	85.58	87.25
Feature(1-9)	87.71	75.43	87.91	87.52	87.57	87.91

Note: The abbreviations in the table are: Accuracy (ACC), Matthews Correlation Coefficient (MCC),

Sensitivity (SN) and Specificity (SP). “Feature(1-8)” indicates that the first 8 features were applied, and “Features(1-9)” means that all features were applied.