Bayesian Input Design for Linear Dynamical Model Discrimination.

A Bayesian design of the input signal for linear dynamical model discrimination has been proposed. The discrimination task is formulated as an estimation problem, where the estimated parameter indexes particular models. As the mutual information between the parameter and model output is difficult to calculate, its lower bound has been used as a utility function. The lower bound is then maximized under the signal energy constraint. Selection between two models and the small energy limit are analyzed first. The solution of these tasks is given by the eigenvector of a certain Hermitian matrix. Next, the large energy limit is discussed. It is proved that almost all (in the sense of the Lebesgue measure) high energy signals generate the maximum available information, provided that the impulse responses of the models are different. The first illustrative example shows that the optimal signal can significantly reduce error probability, compared to the commonly-used step or square signals. In the second example, Bayesian design is compared with classical average D-optimal design. It is shown that the Bayesian design is superior to D-optimal design, at least in this example. Some extensions of the method beyond linear and Gaussian models are briefly discussed.

1. Introduction

Discrimination of various dynamical models of the same process has a wide area of applications, especially in multiple-model fault detection and isolation [1,2,3,4] and, in many other estimation and control problems [5,6,7], it is necessary to choose the most likely dynamical model from a finite set. The discrimination task can be formulated as an estimation problem, where the estimated parameter indexes particular models. The problem can also be considered as a finite-dimensional approximation of more general identification tasks [8]. As the error probability or variance of the estimator of usually depends on the input signal, it is important to select a signal that minimizes error probability or maximizes a utility function that encodes the purpose of the experiment. Selection of an input signal that maximizes a utility function is strongly related to optimal experimental design [9].

Experimental design methods can be divided into classical and Bayesian. The classical methods, also called optimal experimental design, typically use various functionals of the Fisher information matrix as a utility function. These methods are widely described in the literature and work well if the model is linear in its parameters (see [8,10,11,12] and the review article [9]). Unfortunately, in typical identification tasks, the solution of model equation and the covariance depends non-linearly on , even if the model equation is linear. This implies that the information matrix and the utility function depend on the parameter to be estimated. Therefore, only locally-optimal design can be obtained [13]. To obtain more robust methods, an averaging over the prior parameter distribution or minimax design [8,14] (Section 6.1) are commonly used, but these methods are not fully Bayesian.

Bayesian optimal design uses the utility function, a functional of the posterior distribution (see [15,16] and the review articles [13,17,18]). The most commonly used utility functions are mutual information between parameters and model output, Kullback-Leibler divergence between the prior and posterior distributions, and the determinant of the posterior covariance matrix [13,16]. In contrast to classical methods, in Bayesian design the utility function does not depend on the parameters to be estimated. Hence, the method can cope with non-linear problems. The utility function, which is suitable for model discrimination, is the error probability of the MAP estimator of [19]. Such a utility function is generally difficult to calculate (see [19]), but the result of Feder & Merhav [20] implies that the error probability of the MAP estimator is upper-bounded by some decreasing function of mutual information between and the output of the system. Hence, the maximization of mutual information creates the possibility of reducing the error probability, provided that appropriate estimator is used. However, the most serious problem that inhibits the development of this idea is great computational complexity in calculating the mutual information.

The main contribution of this article is a fully-Bayesian (in the terminology of [13]) method for finding an input signal that maximizes the mutual information between and the system output. Maximization of information or, equivalently, maximization of the output entropy has been proposed by many authors (see, e.g., [13,15,17,18,21,22,23]), but the mutual information is very hard to compute and the problem is often intractable. To overcome this serious difficulty, instead of mutual information the lower bound, given by Kolchinsky & Tracey [24], has been used. This is a pairwise-distance based entropy estimator and it it useful here, since it is differentiable, tight, and asymptotically reaches the maximum possible information (see [24] (Sections 3.2, 4, 6)). Maximization of such a lower bound, under the signal energy (i.e., the square of the signal norm) constraints, is much simpler, gives satisfactory solutions, and allows for practical implementation of the idea of maximizing information. This is illustrated with examples. Moreover, it is shown that, for certain cases, this problem reduces to a solution of a certain eigenproblem.

The article is organized as follows. In Section 2, the estimation task is formulated and the upper bound of the error probability and the lower bound of the mutual information are given. In Section 2.1, a selection between two models is discussed and an exact solution is given. Design of input signals with small energy, which is required in some applications, is described in Section 2.2. In Section 2.3, the large energy limit is discussed. An application to linear dynamical systems with unknown parameters is given in Section 3. An example of finding the most likely model among three stochastic models with different structures is given in Section 4. Comparison with classical D-optimal design is performed in Section 5. The article ends with conclusions and references.

2. Maximization of Mutual Information between the System Output and Parameter

Let us consider a family of linear models where , , and . The matrices are bounded. The parameter is unknown. The prior distribution of is given by

The random variable Z is conditionally normal (i.e., ), where the covariance matrices are given a priori and , for all . The variable U is called the input signal. In all formulas below, the input signal U is a deterministic variable. The set of admissible signals is given by

Under these assumptions, and after applying Bayes rule:

The entropies of Y and and the conditional entropies are defined as The mutual information between and Y is defined as (see [25] (pp. 19, 250)) As and , then is given by The MAP estimator of is defined as The error probability of is given by (see [20])

It follows from Fano’s inequality ([25] (p. 38)), that is lower bounded by an increasing function in . Feder & Merhav [20] proved that . As and does not depend on U, then the maximization of creates the possibility of reducing , and the optimal signal is given by

To overcome the problems associated with the calculation of , we will use its lower bound.

(Information bounds). For all where

According to (4), is finite Gaussian mixture. For such mixtures, the information bounds are known. A detailed proof, based on Chernoff -divergence, is given in [24] (Section 4). □

Let

Feder & Merhav [20] (see Theorem 1 and Equation (14)) proved that there exists an increasing, continuous, and convex function , such that

As and , then . The function is increasing, continuous, concave, and it follows from (18) that . Hence, . Taking we obtain the result. □

Now, the approximate solution of (12) is given by

As is smooth and is compact, (19) is well-defined.

2.1. Selection between Two Models

Suppose that takes only two values, 1 and 2, with prior probabilities and , respectively. It’s easy to check, by direct calculation, that

Equation (20) implies that the maximization of is equivalent to the maximization of . On the basis of (15), we have the following optimization: task

The solution of (21) is the eigenvector of corresponding to its largest eigenvalue; that is,

2.2. Small Energy Limit

In many practical applications, the energy of an excitation signal must be small. The second order Taylor expansion of (14) gives where

If the value of (see (3)) is small, then the last term in (23) can be omitted. As the second term does not depend on U, we have the following optimization task:

The solution of (26) is the eigenvector of Q corresponding to its largest eigenvalue.

2.3. Large Energy Limit

We will investigate asymptotic behaviour of when . On the basis of Lemma 1, the condition guarantees that . It is also possible that , for some U. Such signals are weakly informative and they cannot generate the maximum information, even if their amplitude tends to infinity. Let denote the unit ball in and let be the Lebesgue measure on . The set of weakly informative signals is defined as

On the basis of Lemma 1 and (28), , where . Since is positive-definite and then, on the basis of (16), the matrix has at least one positive eigenvalue. Hence, and . The condition implies that . Hence, has at least one positive eigenvalue, which is possible only if . □

As a conclusion, we have the following result.

Let

By the assumption, and from Theorem 1, the set is non-empty. If , then and, from Lemma 1, we get . Now, Lemma 2 implies that , for sufficiently large . □

3. Application to Linear Dynamical Systems

Consider, now, the family of linear systems where the prior distribution of is given by (2) and , and . The variables are mutually independent. The initial condition is zero. The solution of (29) with initial condition has the form

If we denote , , , , and , then (31) and (30) can be rewritten as where the matrices , , , and follow forms (30) and (31). The variables W and V are independent, where and . Substituting (32) into (33) we get Equation (1), where , . The conditional density of Z has the form , where the covariance matrix is given by

Hence, the results of Section 2 can be applied to the dynamical system (29) and (30).

4. Example

In some fault detection and isolation problems [3,4], there is a need to determine which of the known models of the process is the most adequate. It is, therefore, important to find a signal that emphasizes the differences between these various models. As an example of this type of problem, let us consider three stochastic continuous-time models: where , , , , w is a standard Wiener process, and

The step responses of these models are similar and they are difficult to experimentally distinguish from each other if the noise level is significant. The observation equation has the form where , , , and is the first component of . If and , then, after discretization, the state and the output are described by (29) and (30), with appropriate matrices , , , , and . The matrices and are calculated by using (31)–(34). Let us observe that, although the orders of the systems are different, the size of both and is always . We are interested in the maximization of . The solutions of (19) and (26) with a uniform prior, , and steps, are shown in the upper part of Figure 1. The step responses and the optimal responses are shown in the bottom part of Figure 1.

Figure 1

(Top) Numerical solution of (19) and the small energy approximation (26), for . (Bottom) Step responses and optimal responses of all systems.

Let us observe that, in contrast to the step signal, the optimal signal clearly distinguishes the systems—although the energy of all input signals was the same and equal to N.

Let denote the normalized step and square signal with period of three, respectively, and let denote the optimal signal. To check the validity of the results, the error probabilities , , and were estimated by Monte Carlo simulation with trials and steps. The results are shown in Figure 2. It was observed that the optimal signal gives an error probability several thousand times smaller than the step or square signal with the same energy. The second observation is that initially increased with . To explain this, let us note that Inequality (17) does not guarantee that is decreasing function of . Hence, it is possible that increases in certain directions, although Theorem 2 guarantees that tends to zero, provided that signal norm tends to infinity.

Figure 2

Error probability of the MAP estimator for the optimal signal (.), step signal (+), and square (*) signal with period of three. The number of steps is . The error probability has been estimated by a Monte Carlo method with trials. Standard error bars were multiplied by factor of 10 for better visibility.

5. Comparison with the Average D-Optimal Design

Classical methods of signal design for parameter identification use various functionals of the Fisher information matrix as a utility function. One of the most popular is D-optimal design, which consists of finding a signal that maximizes the determinant of the information matrix (see [10,11,12] and the review article [9]). These methods are well-suited to models that are linear in their parameters. Unfortunately, in typical identification and discrimination tasks, the output is a non-linear function of the parameters and the information matrix depends on unknown parameters to be identified. One of the possibilities for avoiding this problem is the averaging of the utility function over the prior parameter distribution. This method is called average D-optimal design (see [14,26] and [9] (Sections 5.3.5 and 6), for details). The Bayesian design, described in the previous sections, will be compared with the average D-optimal design. To that end, let us consider a finite family of linear models (see also [12] (pp. 91–93)) where , , , , and . The prior distribution of is uniform (i.e., ). The state space representation of (40) has the form which is consistent with (29) and (30). The Fisher information matrix is given by where and , , denote the sensitivity of the output to changes in parameters a and b, respectively. The derivatives and fulfil the sensitivity equations with zero initial conditions. The average D-optimal design consists in finding a signal U that maximizes the expectation of the determinant of the information matrix (see [9] (Sections 5.3.5 and 6), [11,14] (Chapter 6), and [12] for details). Hence, the utility function to be maximized has the form with the energy constraints given by (3). Maximization of the utility function (46) has been performed for various signal energies and the error probability of the MAP estimator was estimated by Monte Carlo with trials. The same procedure was repeated using Bayesian design for (41) and (42). The results are shown in Figure 3. The error rate of Bayesian method is significantly smaller when compared to D-optimal design, at least in this example. In particular, the signal shown in the upper-right part of Figure 3 gives an error probability approximately three times smaller than that of D-optimal signal, although the energy of both signals was the same.

Figure 3

Error probability of theMAP estimator (see Section 2), as a function of signal norm and the exemplary input signals (top right) generated by D-optimal and Bayesian methods. The error probability was calculated by a Monte Carlo method with trials. Standard error bars were multiplied by a factor of 10 for better visibility.

6. Possible Extensions of the Results

In this section, we will briefly discuss some possible extensions of the results to an infinite set of parameters and beyond linear and Gaussian models.

6.1. Non-Linear Models

Although the article refers to linear models, it is possible to extend the results to non-linear models of the form where the conditional density of variable Z is given by and , for all , . Under these assumptions, the density of Y still remains a Gaussian mixture and the information lower bound takes the form where

6.2. Non-Gaussian Models

If is non-Gaussian distribution, then it is possible, on the basis of Equation (10) in [24], to construct an information lower bound of the form where is the Chernoff -divergence and . Unfortunately, calculation of (52) is difficult if is large.

6.3. Infinite Set of Parameters

Let us consider following model: where and . If we assume that prior density of is Gaussian; that is, then and can be approximated by a finite Gaussian mixture where and . It’s possible to calculate weights and nodes in (56) by using multidimensional quadrature. If is large, then an appropriate sparse grid should be used. To illustrate the method, we will show only a simple, second-order quadrature with points.

The approximate value of the integral where and

Direct calculation. □

Application of Lemma 3 to (56) gives , . Now, since (56) is a Gaussian mixture, the results of Section 2 can be utilized and the information lower bound takes the form where and are given by (15), (16), and (58), respectively, and , . The approximate solution of (12) can be found by maximization of (59) with the constraints (3).

7. Discussion and Conclusions

An effective Bayesian design method for linear dynamical model discrimination has been developed. The discrimination task is formulated as an estimation problem, where the estimated parameter indexes particular models. To overcome the computational complexity, instead of , its lower bound (), proposed by Kolchinsky & Tracey [24], was used as a utility function. This bound is especially useful, as it is differentiable, tight, and reaches the maximum available information . It has been proved, on the basis of the results of Feder & Merhav [20], that the error probability of the MAP estimator (see Lemma 2) is upper bounded by . The maximization of has been considered under the signal energy constraint, but other kinds of constraints can also be easily implemented. It was shown that the maximization of , in the case of two parameters, is equivalent to maximization of a quadratic form on the sphere (see also [3]). Next, the small energy limit was analyzed. It was proved that the solution is given by an eigenvector corresponding to maximal eigenvalue of some Hermitian matrix. This result can serve as starting point for numerical maximization of . If the energy of the signal tends to infinity, then almost all (in the sense of the Lebesgue measure) signals generate maximum information, provided that the impulse responses of the models are pairwise different. Under these conditions, it was proved that of the MAP estimator tends to zero.

An example of discrimination of three stochastic models with different structures was given. It is easy to observe, from Figure 1, that, in contrast to the step signal, the optimal signal clearly distinguished the systems, although the energy of both signals was the same. The of the MAP estimator was calculated by Monte Carlo simulation. It was observed that the square signal gave an error probability several thousand times greater than the optimal signal with the same energy. Hence, we conclude that the error probability and the accuracy of MAP estimator depends very strongly on the excitation signal. Although Theorem 2 implies that for almost all U, there exist signals such that is locally increasing. This is the case of a high-frequency square signal, as illustrated in Figure 2.

It was shown, in Section 5 (see Figure 3), that of the MAP estimator corresponding to Bayesian design was a few times smaller than generated by D-optimal design, at least in the analyzed example. This result suggests that Bayesian design can be applied to non-linear problems and that it is superior to classical D-optimal design.

Some extensions of the results to the infinite set of parameters and beyond linear and Gaussian assumptions were briefly discussed in Section 6. Extension to non-linear models seems to be easy, but the non-Gaussian case is difficult and a deeper analysis is required. The case of an infinite set of parameters was discussed in Section 6.3. It was shown that the measurement density can be approximated by a finite Gaussian mixture, after which the results of Section 2 could be directly applied. The general conclusion of this analysis is that the information bounds can be easily constructed, as long as the measurement density is a mixture of Gaussian distributions.

An analytical gradient formula must be provided for the effective numerical maximization of . The matrix inversions and the determinants in (15) and (16) should be calculated by SVD. To reduce the computational complexity and required memory resources, the symmetries appearing in (15) and (16), and the fact that , should be utilized. The determinants in (15) and the matrices can be calculated off-line, but the matrices may require too much memory if N and r are large. Therefore, was calculated on-line.

Applications of the presented methods in dual control problems [5] are expected as part of our future work. An additional area in applications is the issue of automated testing of dynamical systems.