Design of Sparse FIR Decision Feedback Equalizers in MIMO Systems Using Hybrid l₁/l₂ Norm Minimization and the OMP Algorithm.

In this paper, a novel scheme using hybrid l₁/l₂ norm minimization and the orthogonal matching pursuit (OMP) algorithm is proposed to design the sparse finite impulse response (FIR) decision feedback equalizers (DFE) in multiple input multiple output (MIMO) systems. To reduce the number of nonzero taps for the FIR DFE while ensuring its design accuracy, the problem of designing a sparse FIR DFE is transformed into an l₀ norm minimization problem, and then the proposed scheme is used to obtain the sparse solution. In the proposed scheme, a sequence of minimum weighted l₂ norm problems is solved using the OMP algorithm. The nonzero taps positions can be corrected with the different weights in the diagonal weighting matrix which is computed through the hybrid l₁/l₂ norm minimization. The simulation results verify that the sparse FIR MIMO DFEs designed by the proposed scheme get a significant reduction in the number of nonzero taps with a small performance loss compared to the non-sparse optimum DFE under the minimum mean square error (MMSE) criterion. In addition, the proposed scheme provides better design accuracy than the OMP algorithm with the same sparsity level.

1. Introduction

In high-speed wireless communication systems, the multipath effect in channels will cause severe intersymbol interference (ISI). The orthogonal frequency division multiplexing (OFDM) system [1,2,3], with the presence of the orthogonal subcarriers and the cyclic prefix, has a strong ability to combat the ISI and generally requires simple equalization. However, compared with OFDM, the single-carrier communication has the advantage of low peak-to-average power ratio of the transmitted signal and insensitivity to carrier frequency offsets [4]. Therefore, the single-carrier communication is widely applied in many communication systems, such as in underwater acoustic communication [5,6,7] and vehicular communication systems [8]. One of the widely used methods to reduce the ISI in single-carrier systems is employing the finite impulse response (FIR) decision feedback equalizers (DFE) with a large number of taps to filter the received signals. However, the design complexity of FIR DFEs grows proportional to the square of the number of nonzero taps. Especially in multiple input multiple output (MIMO) systems, the design complexity can be much higher because the number of DFE taps grows proportional to the number of multiple antennas [9]. Therefore, to reduce the design complexity, it is necessary to design the FIR DFE with only a few nonzero taps (i.e., sparse FIR DFE). In [10], the number of nonzero taps is reduced by choosing only a few significant taps of the minimum mean square error (MMSE) equalizer which is not sparse in general. Nevertheless, the non-sparse MMSE solution must be calculated first which increases the design complexity. Therefore, how to determine the positions and values of nonzero taps directly is one of the main difficulties to design the sparse FIR DFE.

In recent years, some new ways of designing sparse FIR DFE have arisen with the research and application of sparse representation and compressed sensing theory [11,12,13,14,15,16]. Among them, the compressed sensing framework for sparse signal processing is proposed in [11]. A greedy search method to determine the nonzero taps positions of a single input single output (SISO) system is presented in [12]. The l1 norm minimization method to design sparse filters is proposed in [13], but the modeling without wireless channel is quite different from the sparse equalizer design. In [14], a sparse adaptive equalizer design approach using the stochastic gradient pursuit algorithm based on compressed sensing is proposed, but the channel impulse response in the approach is assumed to be sparse. The new design frameworks for sparse FIR MIMO equalizers based on the orthogonal matching pursuit algorithm (OMP) and sparse approximation of a vector using different dictionaries are exploited in [15,16], respectively. They achieve better performance than the method in [10] with the same number of nonzero taps. Furthermore, for the pursuit of both sparse promotion and improved accuracy, the sparse signals reconstruction algorithms inspired by l1 and weighted l1 regularization schemes are proposed in [17,18]; the joint smoothed l0 norm algorithm for direction-of-arrival estimation in MIMO radar is proposed in [19].

We aim to reduce the number of nonzero taps for the FIR DFE while ensuring its design accuracy by applying the OMP algorithm to hybrid l1/l2 norm minimization. The OMP algorithm is a popular greedy algorithm that determines a nonzero tap position for each iteration [20]. However, there is no correction for the selected nonzero taps positions in the previous iterations, which will reduce the design accuracy for the sparse FIR DFE. The hybrid l1/l2 norm minimization is proposed in [21] to make the solution for data fitting less sensitive to large residuals and more robust, and it has been applied in many fields such as geophysics, image processing and machine learning. Within this context, we propose a scheme using hybrid l1/l2 norm minimization and OMP algorithm. In the proposed scheme, a minimum weighted l2 norm problems is solved using the OMP algorithm in each iteration, and the nonzero taps positions can be corrected with the iteratively updated weights in the diagonal weighting matrix, then the weighting matrix is updated through the hybrid l1/l2 norm minimization.

In this paper, we transform the problem of designing a sparse FIR MIMO DFE into an l0 norm minimization problem, then use the proposed scheme to solve the l0 norm minimization problem and obtain the sparse solution. It is verified through simulations that, when compared with the non-sparse optimum MMSE DFE, the sparse FIR MIMO DFEs designed by the proposed scheme obtain a significant reduction in the number of nonzero taps with a small performance loss. In addition, the proposed scheme provides better design accuracy than the OMP algorithm with the same nonzero taps percentage.

2. System Model

Consider a baseband equivalent discrete signal model in the MIMO systems with n transmit (input) antennas and n receive (output) antennas. Assuming that the channel is a linear time-invariant Rayleigh fading channel with an oversampling factor of l, the output of the MIMO channel at time k, i.e., the received samples from the n receive antennas which are grouped into a ln × 1 vector can be formed as: where is the l × 1 output vector of the j-th receive antennas, is the ln × nth MIMO channel matrix, is the input vector of the transmit antennas at time (). The parameter ν is the maximum length of all the subchannel impulse response. is an additive white Gaussian noise vector of the j-th receive antennas with mean 0 and variance σ2. Considering a block of N received vector , (1) can be rewritten as [9] where

Assuming that the transmit signals are independent with the average power , the input autocorrelation matrix can be expressed as where is the identity matrix, and the noise autocorrelation matrix can be expressed as

Furthermore, the output-input crosscorrelation and the output autocorrelation matrices are as follows

3. Design of Sparse FIR DFE Using Hybrid l1/l2 Norm Minimization and OMP Algorithm

In the FIR MIMO DFE model, in order to eliminate ISI and inter-antenna interference, then recover the input signal of the i-th transmit antenna, the feedforward filter with ln taps processes the received signals , and the feedback filter with n taps convolves with the previous decision signals, then the result combined both together is applied to the decision device to obtain current decision signal. If the previous decision signals before time k are correct, the DFE error sample at time k for the i-th input antenna is given by where is the transmit signal at time () by the i-th input antenna, and is an integer representing the decision delay, usually . Using to represent the number of DFE taps for simplicity, so is an vector. Then the equalizer mean square error (MSE) for the i-th input antenna is expressed as where , , and is the ()-th column of . Performing the Cholesky decomposition , where is an lower triangular matrix, (9) can be rewritten as

From (10) it shows that the different choice of can only change , and remains unchanged. Obviously, . Therefore, when , achieves the optimum solution under the MMSE criterion as is usually non-sparse with too much complexity to compute directly. Therefore, in order to reduce the computational complexity, considering to reduce the number of the DFE nonzero taps at the cost of a small increase in the equalizer MSE, i.e., finding the sparse solution for when , the sparse FIR MIMO DFE design problem can be transformed into the l0 norm minimization problem as follows [15]

In general, finding the optimal solution of the above l0 norm minimization problem is NP-hard. One of the approaches to computing the suboptimal solution for the approximation of (12) is greedy algorithms. To design the sparse solution for the FIR MIMO DFE, in this paper, we propose the scheme based on the OMP algorithm which is a widely-used greedy algorithm.

The OMP algorithm selects an atom from the matrix (i.e., the column vector of the matrix ) that is most correlated with the current residual, then solves a constraint least squares problem to obtain all the selected atoms coefficients and updates the residual in each iteration. There is no correction for the selected nonzero taps positions in OMP algorithm, which will reduce the design accuracy for the sparse FIR DFE. To solve the problem, in this paper, we propose the scheme using hybrid l1/l2 norm minimization and OMP algorithm.

According to the hybrid l1/l2 norm minimization, at the ()-th iteration, the proposed scheme will reweight the residual vector with an iteratively updated diagonal weighting matrix , and minimize the weighted l2 norm

Then solve the above problem using the OMP algorithm to find a sparse solution. is updated from the current residual through the hybrid l1/l2 norm minimization. The diagonal entries of are as follows where is the n-th entry of the current residual and is determined by the sparse solution. Substituting (14) into (13), minimizing (13) is equivalent to minimizing

For an arbitrary value of , the approximation of is which suggests behaves like an l2 norm for small residuals and like an l1 norm for large residuals. Therefore, the hybrid l1/l2 norm minimization will make the sparse solution more robust and accurate by decreasing the sensitivity of the solution to large residuals.

The procedures of the proposed scheme for finding the sparse solution of the FIR MIMO DFE are as follows:

Initialization: Set the initial residual , define the index set , and initialize the diagonal weighting matrix for the first iteration.

Main iteration: Set k = 1 and perform the following steps:

Step 1: Reweight the residual vector and formulate the problem

Set and normalize the column vectors of the matrix as follows where . Set and , then (14) becomes

Step 2: Solve (16) with the OMP algorithm. Set the equalizer nonzero taps initial index set , initialize the residual . In the t-th () iteration, the process is as follows:

(1) Calculate the correlation between the residual and the column vector of the matrix , and find the index to the vector which is most correlated with , i.e., ;

(2) Update the nonzero taps index set , and then update the set of column vectors corresponding to ;

(3) Compute the sparse solution , and update the residual ;

(4) Check the value of t. If t = k, update the sparse solution and residual , then proceed to Step 3, else set t = t + 1 and go to (1).

Step 3: Check the stopping rule whether . If met, get the sparse solution for the FIR DFE, and exit the algorithm; else update the diagonal weighting matrix whose diagonal entries are determined by (14). From [22], set . Then set k = k + 1 and go to Step 1.

4. Simulation Results

In the following, the simulation results are provided to verify the performance of the proposed scheme for sparse FIR MIMO DFE design in two different channels. As a comparison, the performance of the optimum MMSE DFE and the OMP method in [15] are presented respectively.

Two different multipath Rayleigh fading channels, namely ITU (International Telecommunication Union) Vehicular A channel [23] and an exponential decay channel, are simulated. ITU Vehicular A channel has a sparse power delay profile (PDP) with 6-paths spanning about 13 symbol periods. The exponential decay channel has a non-sparse PDP as with 13-paths. The simulation system uses binary phase shift keying (BPSK) modulation. The decision delay is set to be , which is optimum when N = ν for DFEs [24]. The small increase in the equalizer MSE of the sparse DFE compared with the MMSE DFE is determined by the maximum tolerable performance loss as , where is the MSE for the MMSE DFE. We define the signal-to-noise (SNR) as , set the oversampling factor l = 2 and MIMO DFEs with n = 2 and n = 2.

Figure 1 plots the nonzero taps percentage for the sparse SISO DFEs and MIMO DFEs versus the performance loss using the proposed scheme respectively. It is shown that the number of nonzero taps for DFE is reduced with the increase of . When = 0.4 dB, the nonzero taps for both the sparse SISO DFEs and MIMO DFEs are reduced by more than 70% and 60% in the two different channels respectively. Therefore, comparing the ITU Vehicular A channel with sparse PDP, the FIR DFEs needs more nonzero taps to eliminate the ISI caused by the multipath effect in the exponential decay channel with more paths. Furthermore, in MIMO systems, the DFEs needs more nonzero taps to eliminate both ISI and inter-antenna interference, as shown in Figure 1, MIMO DFEs has a larger percentage of nonzero taps when compared with SISO DFEs.

Figure 1

Nonzero taps percentage for the sparse SISO DFEs and MIMO DFEs versus with SNR = 20 dB and N = 30.

Figure 2 gives a single example of the SISO DFE by the three design methods in ITU Vehicular A channel. Note that only the nonzero taps are marked, and the tap indices below 25 are insignificant to show. It is shown that there are many taps whose values are close to zero in the non-sparse MMSE DFE. Compared with the MMSE DFE, most of the taps for sparse FIR DFEs designed by the OMP method and the proposed scheme are zero, and the tap positions selected by the two methods are not identical.

Figure 2

Single example of the SISO DFE by the three design method in Vehicular A channel with = 0.6 dB, SNR = 20 dB and N = 25.

To verify the performance of the proposed scheme, we compare the bit error rate (BER) and the normalized mean square error (NMSE) for the three design methods. Figure 3 and Figure 4 compare the BER performance and the equalizer NMSE respectively in ITU Vehicular A channel. The sparse DFEs designed by the proposed scheme and the OMP algorithm are constrained to have the same sparsity level in the simulation. The simulation results show that the maximum SNR loss for the SISO DFEs and MIMO DFEs designed by the proposed scheme are only 0.4 dB and 0.6 dB when the number of nonzero taps is reduced by more than 83% and 80%, respectively. That is, the BER performance and the NMSE for the sparse DFEs designed by the proposed scheme are similar to the MMSE DFE performance. As shown in Figure 3, the BER performance for the sparse DFEs designed by the proposed scheme outperform the OMP algorithm with the same sparsity level. Besides, by comparing the NMSE of the two methods, it is known that the design accuracy of the proposed scheme is superior to the OMP method. Considering that the MIMO system has a certain diversity gain, the simulation results in Figure 3 show that the BER performance of the MIMO DFEs have a significant improvement than the SISO DFEs.

Figure 3

BER versus SNR for SISO DFE and MIMO DFE in the Vehicular A channel with = 0.6 dB and N = 50.

Figure 4

NMSE versus SNR for SISO DFE and MIMO DFE in the Vehicular A channel with = 0.6 dB and N = 50.

To verify the performance of the proposed scheme in different channels, Figure 5 and Figure 6 show the simulation results in the exponential decay channel. Comparing with the OMP method, the results also show that the proposed scheme has better BER performance and smaller NMSE with the same sparsity level. Therefore, the proposed scheme provides excellent performance in different channel conditions.

Figure 5

BER versus SNR for SISO DFEs and MIMO DFEs in the exponential decay channel with = 0.6 dB and N = 50.

Figure 6

NMSE versus SNR for SISO DFEs and MIMO DFEs in the exponential decay channel with = 0.6 dB and N = 50.

5. Conclusions

In this paper, a novel scheme using hybrid l1/l2 norm minimization and OMP algorithm has been proposed to design the sparse FIR MIMO DFE. The proposed scheme solves a of minimum weighted l2 norm problem using OMP algorithm in each iteration, then the weights weighting matrix is updated through the hybrid l1/l2 norm minimization. The simulation results demonstrate that the sparse FIR MIMO DFEs designed by the proposed scheme achieve a significant reduction in the number of nonzero taps with a small performance loss compared to the non-sparse MMSE DFE. Meanwhile, the proposed scheme provides better design accuracy than the OMP algorithm with the same sparsity level.