A study on the empirical distribution of the scaled Hankel matrix eigenvalues.

The empirical distribution of the eigenvalues of the matrix XX(T) divided by its trace is evaluated, where X is a random Hankel matrix. The distribution of eigenvalues for symmetric and nonsymmetric distributions is assessed with various criteria. This yields several important properties with broad application, particularly for noise reduction and filtering in signal processing and time series analysis.

Introduction

Consider a one-dimensional series Y = (y1, … , y) of length N. Mapping this series into a sequence of lagged vectors with size L, X1, … , X, with X = (y1, …, y+−1) ∊ R provides the trajectory matrix , where L(2 ⩽ L ⩽ N/2) is the window length and K = N − L + 1;The trajectory matrix is a Hankel matrix as has equal elements on the antidiagonals i + j = const. The importance of and its corresponding singular values can be seen in different areas including time series analysis [1], [2], biomedical signal processing [3], [4], mathematics [5], econometrics [6] and physics [7]. However, the distribution of eigenvalues/singular values and their closed form has not been studied adequately [8]. For recent work on the generalized eigenvalues of Hankel random matrices see Naronic article [9]. For the eigenvalue distributions of beta-Wishart matrices which is a special case of random matrix see Edelman and Plamen study [10].

Furthermore, such Hankel matrix naturally appears in multivariate analysis and signal processing, particularly in Singular Spectrum Analysis, where each of it column represents the L-lagged vector of observations in R [11], [12]. Accordingly, the aim was to determine the accurate dimension of the system, that is the smallest dimension with which the filtered series is reconstructed from a noisy signal. In this case, the main analysis is based on the study of the eigenvalues and corresponding eigenvectors. If the signal component dominates the noise component, then the eigenvalues of the random matrix have a few large eigenvalues and many small ones, suggesting that the variations in the data takes place mainly in the eigenspace corresponding to these few large eigenvalues. Note that the number of correct singular values, r, for filtering and noise reduction, is increased with the increased L which makes the comparison among different choices (L, r) more difficult. Furthermore, despite the fact that several approaches have been proposed to identify the values of r [13], due to a lack of substantial theoretical results, none of them consider the distribution of singular values of . Here, we study the empirical distribution of singular values of for different situations considering various criteria. Accordingly, the theoretical results on the eigenvalues of XX divided by its trace with a new view is considered in Main results. The empirical results using simulated data are presented in The empirical distribution of ζi. Some conclusions and recommendations for future research are drawn in Conclusion.

Main results

The singular values of are the square root of the eigenvalues of the L by L matrix XX, where X is the conjugate transpose. For a fixed value of L and a series with length N, the trace of matrix XX, , where ‖‖ denotes the Frobenius norm, and are the eigenvalues of XX. Note that the increase of sample size N leads to the increase of which makes the situation more complex. To overcome this issue, we divide XX by its trace , which provides the following properties.

Let ζ , where are the eigenvalues of :

0 ⩽ ζ ⩽ … ⩽ ζ1 ⩽ 1,

,

ζ1 ⩾ 1/L,

ζ ⩽ 1/L.

The first two properties are simply obtained from matrix algebra and thus not provided here. The outermost inequalities are attained as equalities when, for example, y = 1 for all i. To prove the third property, the first two properties are used as follows. The second part confirms ζ1 + ζ2 + … + ζ = 1. Thus, using the first property, ζ1 ⩾ ζ (i = 2, … , L), we obtain ζ1 + ζ1 + … + ζ1 = Lζ1 ⩾ 1 ⇒ ζ1 ⩾ 1/L. Similarly, for the fourth property, it is straightforward to show that ζ + ζ + … + ζ = Lζ ⩽ 1 ⇒ ζ ⩽ 1/L, since ζ ⩽ ζ(i = 1, 2, … , L − 1), and ∑ ζ = 1. Note also that if y = 1 and y = 0 for i ≠ L then ζ1 = …, ζ = 1/L. Rational number theory can also aid us to provide more informative inequalities (for more information see [14]). □

Let us now evaluate the empirical distribution of ζ. In doing so, a series of length N from different distributions, is generated m times. For consistency and comparability of the results, a fixed value of L, here 10, is used for all examples and case studies throughout the paper. For point estimation and comparing the mean value of eigenvalues, the average of each eigenvalue in m runs is used; as defined before, i = 1, … , L, and m is the number of the simulated series. Here we consider eight different cases that can be seen in real life examples:where α = 1, β = 2, φ = 2πt/12, ϑ = 2πt/5, and t is the time which is used to generate the linear trend series.

White Noise; WN.

Uniform distribution with mean zero; U(−α, α).

Uniform distribution; U(0, α).

Exponential distribution; Exp(α).

β + Exp(α).

β + t.

Sine wave series; sin(φ).

β + sin(φ) + sin(ϑ),

The effect of N

In this section, we consider the effect of the sample size, N on . Fig. 1 demonstrates for different values of N for cases ((a)–(c)) considered in this study. In Fig. 1, has a decreasing pattern for different values of N. It can be seen that, for a large N,  → 1/10 for cases (a) and (b). Thus, increasing N clearly affects the values of for the white noise (a) and uniform distribution (b). However, there is no obvious effect on ζ for other cases. For example, for case (c), is approximately equal to 0.8 for different values of N, and is less than 1/10 (see Fig. 1 (right)).

Fig. 1

The plot of , ( = 1, … , 10) for different values of N for cases ((a)–(c)).

Although the pattern of for the uniform distribution (c) is similar to exponential case (d), but for case (c), is greater than comparing to the case (d), whilst other are smaller. It has been observed that has similar patterns for cases ((c), … , (f)). The values of for cases (a) and (b), where Y generated from a symmetric distribution, are approximately the same. The results clearly indicate that increasing N does not have a significant influence on the mean of for all cases except (a) and (b). As a result, if Y is generated from WN or U(−1, 1), then increasing N will affect the value of significantly.

The patterns of

Let us now consider the patterns of for N = 105. For the white noise distribution (a) and trend series (f), has different pattern. It is obvious that, for the white noise series, converges asymptotically to 1/10, whilst for the trend series is approximately equal to 1, and tends to zero. Similar results were obtained for the uniform distributions, cases (b) and (c), respectively.

Both samples generated from exponential distribution have similar patterns for . However, it is noticed that adding an intercept β to the exponential distribution, increases the value of and decreases other . The results indicate that and , whilst, other for sine wave (g). It also indicates that, for sine case (h), ζ(i = 1, … , 5) are not zero, whereas other tend to zero. It was noticed that the value of for sine wave (h) is greater than its value for sine case (g), whilst the value of is less.

The empirical distribution of ζi

The distribution of ζ was assessed for different values of L. It was observed that the histograms of ζ are similar for different values of L (the results are not presented here). Therefore, for graphical aspect, and visualization purpose, L = 10 is considered here. The results are provided only for ζ1, ζ5 and ζ10, for the cases ((a), … , (d)), as similar results are observed for other ζ. Fig. 2 shows histogram of ζ(i = 1, 5, 10) for L = 10, and m = 5000 simulations. It appears that the histogram of ζ1, is skewed to the right for samples taken from WN (a) and uniform distributions (b), whilst for the data generated from the uniform (c) and exponential (d) distributions, might be symmetric. For the middle ζ, the histogram might be symmetric for the four cases (the results only provided for ζ5), whilst the distribution of ζ10, is skewed to the left.

Fig. 2

The histograms of ζ1, ζ5, and ζ10 for cases ((a), … , (d)).

For cases, exponential distribution (e), trend series (f), and sine wave series (g) and complex series (h), we have standardized ζ to have conveying information about their distributions. Fig. 3 shows the density of ζ (i = 1, 2, 3, 5, 6, 10) for those cases. It is clear that ζ1 has different histogram for these cases, and also different from what was achieved for the white noise and uniform distributions with zero mean. Remember that, if Y generated from a symmetric distribution, like case (a) and (b), ζ1 has a right skewed distribution. Moreover, it is interesting that ζ10 has a negative skewed distribution for all cases except the trend series and sine cases ((g) and (h)).

Fig. 3

The density of ζ, i = 1, … , 6, 10 for cases ((e), … , (h)).

Additionally, it should be noted that, for sine series (g), both ζ1 and ζ2 have similar distributions, whereas other ζ have right skewed distributions. It is obvious that the distribution of ζ for sine series (h) becomes skewed to the right for ζ (i = 6, … , 10). Remember that the sine wave (h) was generated from an intercept and two pure sine waves. This means that the components related to the first five eigenvalues create the sine series (h). The results confirm that adding even an intercept alone will change the pattern of ζ. Note that an intercept can be considered as a trend in time series analysis. Generally, if we add more non stochastic components to the noise series, for instance trend, harmonic and cyclical components, then the first few eigenvalues are related to those components and as soon as we reach the noise level the pattern of eigenvalues will be similar to those found for the noise series.

Usually every harmonic component with a different frequency produces two close eigenvalues (except for frequency 0.5 which provides one eigenvalues). It will be clearer if N, L, and K are sufficiently large [15]. In practice, the eigenvalues of a harmonic series are often close to each other, and this fact simplifies the visual identification of the harmonic components [15]. Thus, the results obtained here are very important for signal processing and time series techniques where noise reduction and filtering matter.

Generally, it is not easy to judge visually if ζ has a symmetric distribution, thus it is necessarily to consider other criteria like statistical test. We calculate the coefficient of skewness which is a measure for the degree of symmetry in the distribution of a variable. Table 1 represents the coefficient of skewness for ζ for all cases. Bulmer [16] suggests that; if skewness is less than −1 or greater than +1, the distribution is highly skewed; if skewness is between −1 and −1/2 or between +1/2 and +1, the distribution is moderately skewed, and finally if skewness is between 1/2 and +1/2, the distributions approximately symmetric. Therefore, we can say that, for instance, the distribution of ζ1 for cases ((c), … , (f)), and ζ5 for all cases might be symmetric.

Table 1

The coefficient of skewness for ζ, (i = 1, … , 10), for all cases.

	Coefficient of Skewness of ζi, i = 1, … , 10
	WN	U(−1, 1)	U(0, 1)	Exp(1)	2 + Exp(1)	sin(φ)	2 + sin(φ) + sin(ϑ)	2 + t
ζ1	0.991	0.450	0.005	−0.003	−0.126	0.186	−0.764	0.466
ζ2	0.692	0.733	0.428	0.330	0.230	−0.186	0.273	−0.544
ζ3	0.461	0.502	0.224	0.280	0.154	0.691	0.025	0.995
ζ4	0.401	0.234	0.075	0.092	0.154	0.623	−0.096	0.781
ζ5	0.099	0.021	0.055	0.077	0.153	0.624	−0.045	0.915
ζ6	−0.140	−0.130	−0.001	0.071	0.154	0.649	0.775	0.835
ζ7	−0.37	−0.230	−0.041	−0.102	0.145	0.690	0.632	1.020
ζ8	−0.503	−0.460	−0.033	−0.139	0.110	0.855	0.716	1.135
ζ9	−0.577	−0.520	−0.162	−0.226	0.021	1.970	1.020	1.484
ζ10	−0.810	−0.790	−0.371	−0.480	−0.036	1.880	1.459	2.030

D’Agostino–Pearson normality test [17] is applied here to evaluate this issue properly. It is also known as the omnibus test because it uses the test statistics for both the skewness and kurtosis to come up with a single p-value and quantify how far from Gaussian the distribution is in terms of asymmetry and shape. The p-value of D’Agostin test was significant, greater than 0.05 for ζ1, for cases ((c), … , (f)), whereas, it is less than 0.05 for other cases ((a), (b), (g), (h)). Therefore, we accept the null hypothesis that the data of ζ1 for cases ((c), … , (f)) are not skewed and as a result are symmetric. Moreover, ζ5 has a symmetric distribution for all cases, except the trend series and sine waves. The distribution of ζ(i = 2, 4), for the exponential case (d) is symmetric, whereas skewed for the exponential case with intercept (e).

In terms of the distribution of ζ for the trend series and sine wave (g), the distributions of ζ=1,2 are totally different to the distributions of other ζ, which becomes skewed distribution. Note that the distribution of ζ (i = 1, 2) for the trend series is symmetric, whilst skewed for sine wave (g). For sine series (h), the distribution of ζ (i = 1, … , 5) is different from the distribution of ζ (i = 6, … , 10). It is obvious from the figure that ζ (i = 6, … , 10) has a right skewed distribution.

Conclusions

The pattern of the eigenvalues of the matrix , generated from different distributions was studied, and several properties were introduced. We have considered symmetric, nonsymmetric distributions, trend and sine wave series. The results indicate that for a large sample size N, ζ; N → 1/L for the symmetric distributions (the white noise and the uniform distributions with zero mean), whilst this convergence has not been observed for other cases. The results also indicate that, for the symmetric cases, the pattern of the first eigenvalue is skewed, whilst it can be symmetric for the trend and nonsymmetrical distributions. Furthermore, for all cases under this study, the distribution of the middle ζ, for L = 10, can be symmetric except the pattern of ζ5 for the trend case and both sine series. It is found that the last eigenvalue has a positive skewed distribution, for all cases except the trend series and sine waves. For future research, the theoretical distribution of the matrix is of our interest.

Furthermore, we aim to evaluate the applicability of the results found here for noise reduction of the chaotic series. Additionally, we are applying the properties obtained here as extra criteria for filtering series with complex structure. We may also consider a test to evaluate the k largest eigenvalues, to decide whether the distribution of the eigenvalues can resemble the particular distribution of the eigenvalues. In addition, the distribution of the smallest eigenvalue is as well of great interest, for example, because its behavior is used to prove its convergence to the circular law. Accordingly, the study of the local properties of the spectrum as well as the related distribution is of interest.

Conflict of Interest

The authors have declared no conflict of interest.

Compliance with Ethics Requirements

This article does not contain any studies with human or animal subjects.