Deterministic sensing matrices in compressive sensing: a survey.

Compressive sensing is a sampling method which provides a new approach to efficient signal compression and recovery by exploiting the fact that a sparse signal can be suitably reconstructed from very few measurements. One of the most concerns in compressive sensing is the construction of the sensing matrices. While random sensing matrices have been widely studied, only a few deterministic sensing matrices have been considered. These matrices are highly desirable on structure which allows fast implementation with reduced storage requirements. In this paper, a survey of deterministic sensing matrices for compressive sensing is presented. We introduce a basic problem in compressive sensing and some disadvantage of the random sensing matrices. Some recent results on construction of the deterministic sensing matrices are discussed.

1. Introduction

Consider a scenario that x ∈ ℝ is a vector we wish to recover. Let y ∈ ℝ  (m ≪ n) be a linear measurements of the vector x, which is given by where A is the measurement matrix or sensing matrix. Because this system is underdetermined, the recovery problem of the vector x from the measurement vector y is an ill-posed problem. However, two papers by Donoho [1] and Candès et al. [2] gave us a breakthrough by exploiting sparsity in recovery problems. The authors show that a sparse signal can be reconstructed from very few measurements by solving via l 0-minimization l 1-minimization or adopting a strategy between ((P) and ((P) The sufficient conditions for having the solution of ((P) to coincide with that of ((P) are dependent on either mutual coherence or Restricted Isometry Property (RIP). These conditions are closely related to each other and play an important role in the construction of sensing matrices. Consider A = [a 1 ⋯ a ] is an m × n sensing matrix we investigate. Then its mutual coherence is defined as

Lemma 1 (see [3])

For an m × n sensing matrix A, the Welch bound is given by

The existence and uniqueness of the solution can be guaranteed as soon as the measurement matrix A satisfies the RIP of order k; that is, The smallest value of δ is called the Restricted Isometry Constant (RIC). A strict condition δ 2 < 1 also guarantees exact solution via l 0-minimization. However, the problem ((P) remains NP-hard; that is, it cannot be solved in practice. For 0 < q ≤ 1, there is no numerical scheme to compute solutions with minimal l -norm as well. Furthermore, the problem ((P) is a convex optimization problem, and in fact, it can be formulated as a linear optimization problem. Then solving via l 1-minimization is efficient with high probability. Hence, most researchers are interested in the recovery via l 1-minimization.

There are two common ways to solve these problems. First, we can exactly recover x via l 1-minimization by solving the problem ((P) or ((P) which is given as The second method is using greedy algorithms for l 0-minimization, such as Matching Pursuit (MP), Orthogonal Matching Pursuit (OMP), or their modifications [4-8].

However, in order to ensure unique and stable reconstruction, the sensing matrix A must satisfy some criteria. One of the well-known criteria is 2k-RIP. More attention has been paid to random sensing matrices generated by identical and independent distributions (i.i.d.) such as Gaussian, Bernoulli, and random Fourier ensembles, to name a few. Their applications have been shown in medical images processing [9], geophysical data analysis [10], communications [11, 12], and other various signal processing problems. Even though random sensing matrices ensure high probability in reconstruction, they also have many drawbacks such as excessive complexity in reconstruction, significant space requirement for storage, and no efficient algorithm to verify whether a sensing matrix satisfies RIP property with small RIC value. Hence, exploiting specific structures of deterministic sensing matrices is required to solve these problems of the random sensing matrices.

Recently, several deterministic sensing matrices have been proposed. We can classify them into two categories. First are those matrices which are based on coherence [13-15]. Second are those matrices which are based on RIP or some weaker RIPs [16-20]. In this paper, we introduce some highlighted results such as deterministic construction of sensing matrices via algebraic curves over finite fields in term of coherence and chirp sensing matrices, second-order Reed-Muller codes, binary Bose-Chaudhuri-Hocquenghem (BCH) codes, and the sensing matrices with statistical RIP in terms of the RIP.

The rest of this paper is organized as follows. Section 2 introduces some random sensing matrices and their practical disadvantages. In Section 3, we present some highlighted results in terms of deterministic constructions. Section 4 concludes this paper.

2. Random Sensing Matrices and Their Drawbacks

Recall that x ∈ ℝ is the vector we want to recover. Because the number of measurements is much smaller than its dimension (m ≪ n), we cannot find a linear identity reconstruction map; that is, unique solution does not exist for all x in ℝ. However, if we assume that the signal x belongs to a certain subset Σ ⊂ ℝ which is the set of all k-sparse vectors as for each index set T⊆{1,…, n}, the k-best approximation is given by where ||·|| can be any norm in R . As we noted above, the use of randomly generated sensing matrices has become powerful in compressive sensing. For an upper bound where c is a positive constant, the i.i.d. Gaussian matrix achieves the k-RIP as well, which guarantees to recover sparse signals with high probability [21, 22]. The condition in (7) is also known to hold for the symmetric Bernoulli distribution case and changed to k ≤ cm/(log⁡⁡n)6 for the Fourier measurements [23]. For noiseless recovery, it can be stated as follows.

Theorem 2 (see [24])

If z ∈ ℝ is a k-sparse vector and the sensing matrix A satisfies then x is the unique minimizer to ((P).

In practice, the original signals may be affected by noise, so the recovered signals are not exact, and rather they are almost sparse instead. Hence, some modified criteria were proposed as follows.

Theorem 3 (see [25])

Suppose that x ∈ Σ and the noise e = A z − y satisfies ||e||2 ≤ ϵ. If the sensing matrix A has RIP such that then x* which is the output of the reconstruction algorithm applied to x via ((P) will obey where the constant c depends on sparsity k.

A new result on RIC was proposed by Candès as follows.

Theorem 4 (see [25])

Given x ∈ Σ(x) and an upper bound of noise ||e||2 = ||A z−y||2 ≤ ϵ, if the sensing matrix A has RIP such that then any solution x* of ((P) obeys where c 1 and c 2 are two positive constants depending on sparsity k.

Several inequalities in terms of RIC have been discovered, such as in [26], in [27], or in [28], to name a few. In sum, we can obtain stable and unique solution by using tools from random sensing matrices.

Random matrices are easy to construct and ensure high probability reconstruction. However, they also have many drawbacks. First, storing random matrices requires a lot of storage. Second, there is no efficient algorithm verifying RIP condition. So far, it is not a good approach because of its lack of efficiency. The recovery problems may be difficult when the dimension of the signal becomes large, and we have to construct a measurement matrix that satisfies RIP with a small δ , such as Theorem 4.

3. Deterministic Sensing Matrices

3.1. Chirp Sensing Matrices

A discrete chirp of length m has the form where r is the chirp rate and ω is the base frequency. The full chirp sensing matrix A of size m × m 2 can be written as Each matrix U   (t = 1,…, ω) is an m × m matrix with columns given by the chirp signals having a fixed chirp rate r with base frequency ω varying from 0 to m − 1. For instance, given k = 2 and r, m, l ∈ {0,1}, we obtain Hence, we get the 2 × 4 deterministic sensing matrix A as Note that when r = r 1 = 0, the matrix U corresponding to chirp rate r 1 becomes the Discrete Fourier Transform (DFT) matrix.

Most of the sensing chirp matrices admit a fast reconstruction algorithm which reduces the complexity to O(kmlog⁡⁡m).

3.2. Second-Order Reed-Muller Sensing Matrices

The second-order Reed-Muller code is given as follows: where P is a p × p binary symmetric matrix, b is a p × 1 binary vector in ℤ 2, and w(b) is the weight of b, that is, number of bit-1 entries. In practice, the matrices P are set as all-zero matrices or the matrices with zero diagonals. Thus, there are only 2 matrices P satisfying this condition, which are {P 1,…, P 2}, and the functions {ϕ (a)} are real valued. The set forms a basis of ℤ 2 . The inner product on ℱ is defined as follows. For any two vectors ϕ and ϕ in ℱ , where q = rank⁡(P − P′). The deterministic sensing matrix has the form where U is the unitary matrix corresponding to ℱ   (i = 1,…, 2). Note that if we set m = 2 and n = 2, we get an m × n sensing matrix A RM. For instance, let p = 2; then There are only 22(2−1)/2 = 2 binary symmetric matrices P of size 2 × 2 satisfying the condition. These are Hence, we get the deterministic sensing matrix A as Reconstruction algorithms using the second-order Reed-Muller sensing matrices can outperform the standard compressive sensing using random matrices via l 1-minimization, especially when the original signal is not sparse and the noise is present. Moreover, the nesting of the Delsarter-Goethals sets of the Reed-Muller codes is still feasible if the dimension of the original signal is large [17, 29].

3.3. Binary BCH Matrices

Denote n as a divisor of 2 − 1 for some integer p ≥ 3, and γ ∈ GF(2) as a primitive nth root of unity and assume that p is the smallest integer for which n divides 2 − 1. If we set α = γ (2, then α has order n. Define a code 𝒞 over GF(2) by The BCH code 𝒞 is defined by In other words, if we denote 𝒩 as the null-space of the above matrix of GF(2), then the BCH code 𝒞 = 𝒩∩𝔽 2.

An example of binary matrices formed by BCH code is given as follows. Let n = 15, p = 4, and t = 1, and let γ be a primitive element of GF(24) = GF(16) satisfying γ 4 + γ + 1 = 0. Then α = γ (2 = γ (2 = γ. The BCH code is the set of 15 tuples that lie in the null space of the matrix Since satisfies g(α 3) = 0, we have [1 0 0 0 1 0 1 1 1 0 0 0 0 0 0] as a codeword in the BCH code. The binary matrix is obtained as follows: Since BCH code is cyclic, we can describe it in terms of a generator polynomial which is the smallest degree polynomial having zeros α, α 3,…, α 2,…, α 2. The advantages of these matrices are deterministic construction, simplicity in sampling process, and reduced computational complexity compared with the DeVoice's binary sensing matrices. However, the generated matrices formed by BCH codes do not acchieve the RIP constraint yet.

3.4. Sensing Matrices with Statistical Restricted Isometry Property

In [18], the authors proposed some weaker Statistical Restricted Isometry Properties (StRIPs) defined as follows.

Definition 5 (StRIP)

An m × n matrix A is called StRIP of order k with constant δ and probability 1 − ϵ if with respect to a uniform distribution of vector x among all k-sparse vectors in R of the same norm.

Definition 6 (UStRIP)

An m × n matrix A is called UStRIP of order k with constant δ and probability 1 − ϵ if it satisfies StRIP and with probability exceeding 1 − ϵ with respect to a uniform distribution of vector x among all k-sparse vectors in R of the same norm.

These constructions allow recovery methods for which expected performance is sublinear in n and quadratic in m, compared to the superlinear in n of the BP or the MP algorithms. The criteria are simple; however, when satisfied by a deterministic sensing matrix, they guarantee successful recovery in an exponentially small fraction of k-sparse signals. The authors also showed that such sensing matrices satisfying these aforementioned properties could be constructed by chirps, second-order Reed-Muller codes, and BCH codes [16-18].

3.5. Deterministic Construction of Sensing Matrices via Algebraic Curves over Finite Fields

In [14], DeVore used polynomials over finite field 𝔽 to construct binary sensing matrices of size p 2 × p , where p is a primer number. Let {a 0, a 1,…, a } be a subset of 𝔽 , and let P be the set of all polynomials of degree no more than r on 𝔽 as There are n = p such polynomials. Any polynomial P ∈ P can be described as a mapping The set of (x, P(x)) is a subset of 𝔽 × 𝔽 . We order the element of 𝔽 × 𝔽 as (0,0), (0,1),…, (p − 1, p − 1). For given 𝔽 , the set 𝔽 × 𝔽 has m = p 2 of order pairs. For each P ∈ P , we denote v as the vector indexed on 𝔽 × 𝔽 which is defined by where for i, j = 0,1,…, p − 1.

Theorem 7

Let A 0 be the m × n matrices with columns v , P ∈ P with these columns ordered lexicographically with respect to the coefficients of the polynomial. Then the matrix satisfies the RIP of order k < p/r + 1 with RIC value δ = (k − 1)r/p.

There are several deterministic constructions of sensing matrices via algebraic curves over finite fields called algebraic geometry codes [30-33]. Goppa's code is one of well-known results which contain many linear codes with many good parameters. Hence, these kinds of sensing matrices are good candidates in reconstruction issues using compressive sensing.

3.6. Binary Sensing Matrices Generated by Unbalanced Expander Graphs

In [20], a large class of deterministic sensing matrices based on unbalanced expander graphs, that is, the combinatorial structures, was proposed. Denoting [n] = {1 ⋯ n}, these bipartite graphs are formalized through the following definitions.

Definition 8

A bipartite graph with n left vertices, m right-vertices, and left-degree d is specified by a function Γ : [n]×[d]→[m], where Γ(x, y) denotes the yth neighbor of x. For a set S ⊂ [N], we write Γ(S) to denote its set of neighbors {Γ(x, y) : x ∈ S,  y ∈ [D]}.

Definition 9

A bipartite graph Γ : [n]×[d]→[m] is a (K, A) expander if for every set S ⊂ [n] of size k, we have |Γ(S)|≥A · K.

They constructed a large class of binary and sparse matrices satisfying a different form of the RIP property called RIP-p as If the sensing matrix A is an adjacency matrix of high-quality unbalanced expander, then the RIP-p holds for 1 ≤ p ≤ 11 + O(1)/log⁡(n).

Theorem 10 (see [19])

Consider any m × n matrix A 0 which is the adjacency matrix of an (k, ϵ) unbalanced expander G = (A, B, E), |A | = n, |B | = m with left degree d, such that 1/ϵ and d < n. Then the scaled matrix satisfies the RIP with RIC value δ = cϵ for some positive constant c > 1.

This approach utilizes sparse matrices interpreted as adjacency matrices of sparsity to recover an approximation to the original signal. The new property RIP-q suffices to guarantee exact recovery algorithms.

4. Concluding Remarks

In this paper, various deterministic sensing matrices have been investigated and presented in terms of coherence and RIP. The advantages of these matrices, in addition to their deterministic constructions, are the simplicity in sampling and recovery process as well as small storage requirement. It can be possible to make further improvement in both reconstruction efficiency and accuracy using these deterministic matrices in compressive sensing, particularly when some a priori information on location of nonzero components is available.