A Robust Manifold Graph Regularized Nonnegative Matrix Factorization Algorithm for Cancer Gene Clustering.

Detecting genomes with similar expression patterns using clustering techniques plays an important role in gene expression data analysis. Non-negative matrix factorization (NMF) is an effective method for clustering the analysis of gene expression data. However, the NMF-based method is performed within the Euclidean space, and it is usually inappropriate for revealing the intrinsic geometric structure of data space. In order to overcome this shortcoming, Cai et al. proposed a novel algorithm, called graph regularized non-negative matrices factorization (GNMF). Motivated by the topological structure of the GNMF-based method, we propose improved graph regularized non-negative matrix factorization (GNMF) to facilitate the display of geometric structure of data space. Robust manifold non-negative matrix factorization (RM-GNMF) is designed for cancer gene clustering, leading to an enhancement of the GNMF-based algorithm in terms of robustness. We combine the l 2 , 1 -norm NMF with spectral clustering to conduct the wide-ranging experiments on the three known datasets. Clustering results indicate that the proposed method outperforms the previous methods, which displays the latest application of the RM-GNMF-based method in cancer gene clustering.

1. Introduction

With the progressive implementation of human whole-genome and microarray technologies, it is possible to simultaneously observe the expressions of numerous genes in different tissue samples. By analyzing gene expression data, genes with varying expressions in tissues and their relationships may be identified to figure out the pathogenic mechanism of cancers based on genetic changes [1]. Recently, cancer classification based on gene expression data has become a hot research topic in bioinformatics.

Due to the fact that the analysis of genome-wide expression patterns can provide unique perspectives into the structure of genetic networks, the clustering technique has been used to analyze gene expression data [2,3]. Cluster analysis is the most widespread statistical techniques for analyzing massive gene expression data. Its major task is to classify genes with similar expressions to discover groups of genes with identical features or similar biological functions, in order that people can acquire a deeper understanding about the essence of many biological phenomena such as gene functions, development, cancer, and pharmacology [4].

Currently, it has been shown that non-negative matrix factorization (NMF) [5,6] is superior to the hierarchical clustering (HC) and self-organizing map (SOM) [7] in the application of cancer samples in clustering gene expression data. Over the past few years, the NMF-based method has been used for the gene expressions of statistically analyzing data for clustering [8,9,10,11,12,13]. The main idea is to approximately factorize a non-negative data matrix into a product of two non-negative matrices, which makes sure that all elements of the matrices are non-negative. Therefore, the appearance of NMF has attracted considerable attention. Recently, various variants based on the original NMF have been developed by modifying the objective function or the constraint conditions [14,15,16]. For instance, Cai et al. proposed graph regularized non-negative matrix factorization (GNMF), giving forth to the neighboring geometric structure. It illustrates the nearest neighbor graph that preserves the neighborhood information of high-dimensional space in low-dimensional space. The GNMF reveals the intrinsic geometrical structure by incorporating a Laplacian regularization term [17], which is effective for solving clustering problems. After that, the sparse NMF [18] was proposed with sparse constraints upon the basis matrices and coefficient matrices factored by the NMF so that the sparseness may be reflected from data. The non-smooth NMF [19] can realize the global or local sparseness [20] by making basis and encoding matrices sparse simultaneously. For the sake of enhancing the robustness of the GNMF-based method in gene clustering, we propose improved robust manifold non-negative matrix factorization (RM-GNMF) by making use of the combination of -norm and spectral clustering with Laplacian regularization, leading to the internal geometry of data representations. It facilitates the display purposes of intrinsic geometric structure of the cancer gene data space.

This paper is organized as follows. In Section 2, we give a brief review on the NMF and the GNMF. In Section 3, we propose an improved RM-GNMF algorithm. In Section 4, we give the experimental results comparing with the previous methods. Finally, the conclusions are drawn in Section 5.

2. The NMF-Based and GNMF-Based Method

The NMF-based method [5] is a linear and non-negative approximate data description for non-negative matrices. We consider an original decomposed matrix X of size , where m represents data characteristics and n represents the number of samples. Based on the NMF method, the matrix X is decomposed into two non-negative matrices and , i.e., where . For a given decomposition , sample m can be divided into r classes according to matrix . Each sample is placed in the highest metagene expression level in the sample, meaning that if is the largest in column j, then sample j is assigned to class i .

Using the square of the Euclidean distance between X and [21], we have the objective function of the NMF method

According to the iterative update algorithms [6], the NMF-based method is performed on the basis of multiplicative update rules of W and H given by

In order to overcome the limitation of the NMF-based method, Cai et al. [17] proposed the GNMF-based method, in which an affinity graph is generated to encode the geometrical information followed by a matrix factorization with respect to the graph structure. In contrast to the NMF-based method, it has a regular graph constraint, which preserves the advantage of the local sparse representation of the NMF-based method and preserves the similarity between the original data points after dimensionality reduction.

There are several weighting schemes, such as zero-one weighting, heat kernel weighting, and Gaussian weighting [17]. In what follows, we consider the zero-one weighting described as

Based on the weight matrix Q, we obtain the objective function of the GNMF method given by where denotes the trace of matrices and . D is a diagonal matrix whose entries are column or row sums of Q with [22]. The regularization parameter can be used for the smoothness control of the new representation. By the iterative algorithms to minimize the objective function , we achieve the updating rules

3. The RM-GNMF-Based Method for Gene Clustering

So far, we have described the NMF-based method and GNMF-based method. In what follows, we seek RM-GNMF gene clustering by making use of the combination of -norm and spectral clustering with the Laplacian regularization.

3.1. The -Norm

The -norm of a matrix was initially employed as a rotational invariant -norm [23], which was usually used for multi-task learning [24,25] and tensor factorization [26]. Instead of using -norm-based loss function that is sensitive to outliers, we resort to the -norm-based loss function and regularization [23], which is convergence-proved.

For the sake of getting over the drawbacks of the NMF-based method and enhancing the robustness of the GNMF-based method, we employ the -norm for the matrix factorization in the RM-GNMF-based method. For a non-negative matrix X of size , the -norm of matrice X is defined as where data vectors are arranged in columns, and the -norm calculates the -norm for column vectors first. Subsequently, the matrix factorization assignment becomes

3.2. Spectral-Based Manifold Learning to Constrained GNMF

The spectral method is a classical method of analysis and algebra in mathematics. It is widely used in low dimensional representation and clustering problems of high dimensional data [27,28]. A relational matrix describing the similarity of the pair of data points is defined according to the given sample dataset, and the eigenvalues and eigenvectors of the matrices are calculated. After that, the appropriate eigenvectors are selected and the low dimensional embedding of the data is obtained. The degree matrices are defined on a given graph, such as an adjacency matrix of the graph, a Laplacian matrix, and so on [22].

Based on the spectrum of the matrices with respect to the graph, spectral theory further reveals the information contained in the graph , and establishes the connection between the discrete space and the continuous space through the techniques of geometry, analysis, and algebra. It has a wide range of applications in manifold learning. In this section, the p-nearest neighbor method can be used for establishing the relationship between each data point and its neighborhood.

For a data matrix , we treat each column of X as a data point and each data point as a vertex, respectively. The p-nearest-neighbor graph G can be constructed with n vertices. Then the symmetric weight matrix is generated, where the element denotes the weight of the edge joining vertices i and j and the value of is given by where denotes the set of p-nearest neighbors of . It is obvious that the matrix Q represents the affinity between the data points.

There is an assumption about manifold. Namely, if two data points and are close in the intrinsic geometric structure of the data distribution, then their presentations under a new basis will be close [29]. Therefore, we define the relationship as follows where and denote the mappings of and , respectively. The degree matrix D is a diagonal matrix given by . Obviously, is the sum of all the similarities related with . Then, the graph Laplacian matrix is given by

The graph embedding can be written as

In the RM-GNMF-based method, we combine the GNMF-based method with the spectral clustering, resulting in the -norm constrained GNMF as follows where is the regularization parameter. We resort to the augmented Lagrange multiplier (ALM) method to solve the above problem.

For an auxiliary variable , we rewrite the in Equation (13) as satisfying the constraints and . Then, we define the augmented Lagrangian function satisfying the constraint , where is the step size of update and is the Lagrangian multiplier.

To optimize Equation (15), we rewrite the objective function to get the following task satisfying the constraint .

3.3. Computation of Z

For the given W and H, we can solve Z in Equation (15) by using the update of Z related to the following issue

We need the following Lemma to solve Z in Equation (15). Please see the Appendix for a detailed proof.

Given a matrix and the i-th column of

3.4. Computation of W and H

For the given other parameters, we solve the optimal W. The update of W amounts to solve

Let . The problem in Equation (20) can be rewritten as

If the partial derivative of W is set to be 0, we obtain

Then, we derive the optimal H. Taking , the update problem of H can be expressed as satisfying the constraint . We have

Therefore, the optimal can be achieved by counting eigenvectors of

3.5. Updating of and

The update standard of and can be described as follows where p is the nearest neighbor graph parameter. The detailed process of the RM-GNMF-based method is listed in Algorithm 1.

4. Results and Discussion

In this section, we evaluate the performance of the proposed method on the three gene expression datasets. We compare the RM-GNMF-based method with the NMF-based method [6], the -NMF-based method [23], the LNMF-based method [20], and the GNMF-based method [17].

4.1. Datasets

In order to evaluate the performance of proposed RM-GNMF algorithm, the clustering experiment was conducted on several gene expressions datasets of cancer patients. Three classical genetic datasets were used in the experiment, including leukemia [1], colon, and GLI_85 [30]. These gene expression datasets are downloaded from: http://featureselection.asu.edu/datasets.php.The colon cancer datasets consist of the gene expression profiles of 2000 genes for 62 tissue samples among which 40 are colon cancer tissues and 22 are normal tissues. The leukemia datasets consist of 7129 genes and 72 samples (47 ALL and 25 AML).

A brief description of experimental datasets is described in Table 1.

Table 1

Statistics of three gene expression datasets.

Data Sets	Instances	Features	Classes
Colon	62	2000	2
GLI_85	85	22,283	2
Leukemia	72	7070	2

More detailed information on these datasets can be found in the relevant references, and these datasets are available for download from the reference website.

4.2. Evaluation Metrics

For the sake of evaluating the clustering results, we use the clustering accuracy and normalized mutual information (NMI) to demonstrate the performance of the proposed algorithm.

Clustering accuracy can be calculated as where is the cluster label of , and is the true class label i-th sample, n denotes the total number of samples, and is a delta function. If , we obtain , where is the mapping function that maps the cluster label into the actual label . Otherwise, we have . We can find the best mapping by the Kuhn–Munkres method [31]. NMI can be described as where is the size of the i-th cluster and is the size of the j-th class, is the number of data between the intersections, and N denotes the number of clusters. We perform 100 experiments under each target feature dimension, taking the mean of the accurate and NMI values as the experimental results.

4.3. Parameter Selection

The RM-GNMF-based method involves two essential parameters, i.e., the regularization parameter and the regularity coefficient determining the penalty for infeasibility.

We set the parameters and in the range of and . We use the cross-validation method to get the best parameter values and . In order to intuitively to analyze the influence of parameters and of the RM-GNMF-based method on the accuracy of clustering, Figure 1 shows the variation on clustering accuracy when the two parameters are modified. The three subgraphs in Figure 1 correspond to three gene expression datasets respectively. As can be viewed in Figure 1, the parameter can get higher ACC. With the change of regular parameter , the change of ACC is relatively flat, and the clustering accuracy is higher when the value of is smaller. Therefore, we set in the follow-up experiments.

Figure 1

Influence of parameters on clustering accuracy.

4.4. Clustering Results

In Table 2, we demonstrate the clustering results on the colon, GLI_85 and leukemia datasets, respectively. Reported is the mean of clustering results from 100 runs of different NMF methods together.

Table 2

Clustering results on different datasets. NMF: non-negative matrix factorization; GNMF: graph regularized non-negative matrices factorization; RM-GNMF: robust manifold non-negative matrix factorization; NMI: normalized mutual information.

Methods	Colon		GLI_85		Leukemia
	ACC	NMI	ACC	NMI	ACC	NMI
NMF	0.6290	0.0110	0.6088	0.1906	0.6389	0.0193
L_2,1-NMF	0.5323	0.0048	0.6088	0.1916	0.6328	0.0258
LNMF	0.6129	0.0181	0.5294	0.0011	0.6250	0.0306
GNMF	0.6290	0.0110	0.6000	0.1584	0.6389	0.0193
RM-GNMF	0.6613	0.0220	0.7529	0.1925	0.6528	0.0369

It can be found that the RM-GNMF-based method outperforms the original NMF-based method, while the RM-GNMF-based method achieves the best performance compared with the other three datasets. The clustering accuracies of the RM-GNMF-based method are , and for the colon, GLI_85, and leukemia datasets, respectively.

Our tests on several gene expression profiling datasets of cancer patients consistently indicate that the RM-GNMF-based method achieves significant improvements in comparison with the NMF-based method, the -NMF-based method, the LNMF-based method, and the GNMF-based method, in terms of cancer prediction accuracy.

As shown in Figure 2, the RM-GNMF-based method always gives birth to better clustering results than other NMF-based method using the three original datasets.

Figure 2

Clustering results on different datasets.

To demonstrate the robustness of our approach to data changes, we add uniform noise onto the three gene expression datasets. A disturbed matrix is generated by adding independent uniform noise, defined as follows: where Y is the original matrix, r is a random number generated by a uniform distribution on the interval , and is the maximum expression of Y .

The experimental results with noise added are shown in Figure 3. It can be seen that the clustering result of RM-GNMF algorithm is still stable with the addition of noise, which shows that RM-GNMF algorithm is robust.

Figure 3

Influence of noise on clustering accuracy.

In order to verify the results obtained from the algorithms in the experiments, we import the clustering result of the comparison methods into STAC web platform to perform the statistical test (http://tec.citius.usc.es/stac/). We selected the Friedman test of non-parametric multiple groups; the significance level is . The analysis results obtained are presented in Table 3 and Table 4.

Table 3

Friedman test (significance level of 0.05).

Statistic	p-Value	Result
7.00000	0.01003	H0 is rejected

Table 4

Ranking.

Rank	Algorithm
1.33333	LNMF
2.33333	NMF
2.66667	L_2,1-NMF
3.66667	GNMF
5.00000	RM-GNMF

From the above test results it can be concluded that is rejected. Hence, we believe that the clustering results of five algorithms are significantly different.

5. Conclusions

We have proposed the RM-GNMF-based method with the -norm and spectral-based manifold learning. This algorithm is suitable for cancer gene expression data clustering with an elegant geometric structure. Our tests on several gene expression profiling datasets of cancer patients consistently indicate that the RM-GNMF-based method achieves significant improvements in comparison with the NMF-based method, the -NMF-based method, the LNMF-based method, and the GNMF-based method, in terms of cancer prediction accuracy and robustness.