A bayesian generative model for surface template estimation.

3D surfaces are important geometric models for many objects of interest in image analysis and Computational Anatomy. In this paper, we describe a Bayesian inference scheme for estimating a template surface from a set of observed surface data. In order to achieve this, we use the geodesic shooting approach to construct a statistical model for the generation and the observations of random surfaces. We develop a mode approximation EM algorithm to infer the maximum a posteriori estimation of initial momentum μ, which determines the template surface. Experimental results of caudate, thalamus, and hippocampus data are presented.

1. Introduction

3D surfaces are important geometric models for many objects of interest in image analysis and Computational Anatomy. For example, they are often used to represent the shape of 3D objects, the surface of human faces, and the boundaries of brain structures or of other human organs. Most data analysis methods in this domain are template-centered, and a proper estimation of a template plays an important role to obtain high quality results. This paper is devoted to the description of statistically supported template estimation method which is adapted to surface data sets.

Our approach will be to build a generative statistical shape model in which the template is a parameter. We will then estimate it using maximum likelihood. This model relies on the very natural setting for which an observed surface is a noisy version of a random deformation of the template. This is the most generic and most basic approach of the deformable template paradigm, even if we add a small refinement by including a prior distribution on the template, based on what we will call a hypertemplate. Even with this global scheme which is fairly simple, we will see that implementing it in the context of surfaces will constitute a significant theoretical and numerical challenge.

At the exception of the recent work of [1], this approach significantly differs from what has been mostly proposed in the literature, in which most of the methods compute templates as averages over specific common parametrizations of the surfaces (using, for example, the sphere as a parameter space [2]). Parametric representations, however, are limited by the fact that, because they are defined a priori and independently for each object, they cannot be assumed to suitably align important features in a given data set of surfaces (i.e., give similar coordinates to similar features in the surfaces). This usually results in oversmoothed template surfaces (which is the equivalent of getting blurry template images in the case of image averaging). In [1], a similar diffeomorphic transformation model is used, but, as we will see, our Bayesian construction will provide a well-specified template whereas [1] needs to rely to topologically unconstrained approximations to end up with a manageable template.

In addition to the references above, there have been several publications addressing the issue of shape averaging over a dataset, although most of them work with 3D volume data or landmark points set. In several cases, the average is based on metric properties of the space of shapes [3-7], and the template is computed as an intrinsic average, minimizing the sum of square distances to each element of the set (Fréchet mean). Such methods have been implemented in the context of diffeomorphic deformation models (which are also models of interest in the present paper) for landmark matching [8], for 3D average digital atlas construction [9], and to quantify variability in heart geometry [10]. Other definitions of the average, adapted to situations in which the data is corrupted by noise, have been proposed [11-13], based on variational approaches (but not relying on a generative statistical model).

Our approach to build a generative statistical shape model is reminiscent of the construction developed in [14] for linear models of deformations, and in [15] for large diffeomorphic deformations in 3D volume image averaging. Adapting these ideas to surfaces will however require new algorithms and numerical developments.

In order to present our model, we need to first provide some background materials and notation, describing in particular the geodesic shooting equations that we will use to generate deformed surfaces. We will then introduce a random statistical model describing the generation and the observations of random surfaces. We then develop a Mode Approximation EM algorithm for surface averaging, to estimate the template from observations. In the optimization part, we derive and implement a new variational scheme, which is also applicable to surface matching, providing an alternative approach to the one originally proposed in [16, 17]. Finally, we present and discuss experimental results on caudate, thalamus, and hippocampus data.

2. EPDiff for Surface Evolution

We will base our random shape model on the so-called EPDiff equation, which describes the evolution of deformable structures (like images, surfaces, or landmarks) under the action of groups of diffeomorphisms. It is a geodesic equation for a Riemannian metric on diffeomorphisms, and describes a momentum conservation law in the associated Hamiltonian system. The reader interested by the theory behind this equation can refer to [18-20], but most of this background will not be needed for the present paper, in which we will only use the specific form of the equations for surface evolution. The term EPDiff comes from its determination as an Euler-Poincaré equation in the group of diffeomorphisms, as introduced in [21]. One of its main interests here is that it provides a numerically stable, easily described, Hamiltonian evolution over diffeomorphisms, which will constitute an ideal support for our shape models.

The EPDiff equations describe the combined time evolution of a diffeomorphism, denoted ϕ(t, ·) and of what can be interpreted as a momentum, denoted p(t, ·). The initial conditions are always ϕ(0, x) = x for ϕ, and some initial value, p 0, for p. This initial momentum will be a key component of the statistical model that will be built later on.

Let us start with the simplest form of the equation, which assumes that p 0 is a vector-valued function over ℝ, that is, p 0 : ℝ → ℝ. It involves a smoothing kernel, K, defined on ℝ3 × ℝ3, a typical choice being

Letting ∇1 K denote the gradient of K with respect to its first variable, the corresponding EPDiff equation is

Here, the notation a · b refers to the usual dot product between vectors in ℝ3. If K is smooth enough, this system has solutions for arbitrary large t, and the mapping x ↦ ϕ(t, x) is a diffeomorphism at all times.

The interesting fact about these equations is that they can have singular variants that are described in a similar way and have the same existence properties. The simplest way to relate the variants to the previous equation is to replace the Lebesgue's measure in the integrals by another, possibly singular, measure. For example, taking a surface S 0 in ℝ3, we can use the volume form on S 0 as a reference measure and obtain the equations (in which p 0 and p(t, ·) need only to be defined on S 0)

where d ω is the volume form on S 0. Note that the first equation is defined for x ∈ ℝ3, but it suffices to use it for x ∈ S 0 to obtain an equation for the evolving surface

We can write a discrete form of the equations by replacing d y by a sum of Dirac measures (at points x 1,…, x in ℝ3), which gives, letting a (t) = p(t, x ),

Similarly to (3), the first equation is valid for all x ∈ ℝ3, but it suffices to solve it for x = x , l = 1,…, L to obtain the evolution of the point set

Also, (5) can be seen as a discretization of (3) in which x 1,…, x are the vertices of a triangulation of S 0, and a (t) = p(t, x )δ σ (x ), where δ σ (x ) is the area of a surface element around x .

The evolution of point sets is the most important from a practical point of view, since it is an ODE that can be easily implemented. Assuming a radial kernel K(x, y) = γ(||x−y||2) like in (1), and denoting γ = γ(||x −x ||2), and γ ′ = γ′(||x −x ||2), (5) can be rewritten as

Once the initial position of the vertices, x(0) = (x 1(0),…, x (0)), and the initial momentum, a(0) = (a 1(0),…, a (0)), are provided, the evolution of the point set is uniquely determined.

3. Generative Model for Surface Observation

3.1. Random Triangulated Surfaces

If a triangulated template surface T with vertices x ( is given, and we solve, until time t = 1, (5) initialized with x(0) = x ( and a random initial momentum a(0) = α, the displaced vertices provide a random perturbation of the initial surface that will be denoted by T . This is stated in the following definition.

Definition 1

Let T be a triangulated surface with vertices x ( = (x 1 (,…, x (). Let α ∈ (ℝ3) be a collection of L vectors in ℝ3. Let (x(t), a(t)) be the solution of (5) with initial condition x(0) = x ( and a(0) = α. One defines T to be the triangulated surface with vertices x ( = x(1) and the same topology as T.

By letting α be random, we build T as a random deformation of T. This will form the “ideal”, unobserved, surface, of which only a noisy version is observable (the noise process will be described in the next section).

Following [15], we will use a Bayesian formulation in which T is itself represented as a random deformation T 0, : = (T 0), where T 0 is a fixed surface that we will call the hypertemplate, and μ is a prior initial momentum shooting from T 0 to T (same notation as in Definition 1). One of the main interests of using a hypertemplate is to fix the topology of T so that it belongs, by construction, to the same class of objects as T 0.

So, if N surfaces are observed, we need to model the probability distribution of the prior momentum, μ (starting at T 0), which specifies T = T 0, and of N deformation momenta α (1),…, α ( which specify the surfaces T ,…, T . We now provide a statistical model for the joint probability distribution of μ, α (1),…, α (.

We first introduce some notation. Letting K be the kernel introduced in the previous section to define the geodesic shooting equations, we let Γ be the 3L by 3L matrix formed with the 3 by 3 blocks K(x (, x ()I d ℝ. We define, for a triangulated surface T with L vertices x (, and α ∈ ℝ3,

We define the joint distribution of μ, α (1),…, α ( on ℝ3 × (ℝ3) by

where λ is a fixed parameter regulating the weight on the hypertemplate.

There is a technical difficulty here, which is that one must make sure that this probability can be normalized (Z exists), which requires that the exponential is integrable. That this is true is not straightforward, and we have not been able to find a proof that works with any choice of the kernel K. One way to deal with this is to introduce a constant A (which can be chosen arbitrarily large so that it does not interfere with the algorithms that will follow), and add to the model the constraint that ||μ|| is smaller than A . Under such an assumption, one obtains (after integrating out the α's)

This is finite, since, for any given μ, the transformation x ( → x ( is the restriction of a diffeomorphism to the vertices of T 0 (as seen from (5)). This implies that Γ is nonsingular, and its determinant is bounded away from 0 when μ is restricted to a compact space.

In fact, the choice A = ∞ can be proved to be acceptable for a large class of kernels. Those are kernels for which the smallest eigenvalue of Γ decreases at a speed which is at most polynomial in the minimal distance, h , between the vertices. A list of kernels satisfying this property can be found in [22]. For such kernels, we find that (L being fixed) log det Γ = O(log h ). Just sketching the argument here, one can prove, using elementary properties of dynamical systems, that h = O(exp (−C||μ||) for some constant C, so that the log determinant in (9) is linear in ||μ|| and Z is well defined, even with A = ∞. For very smooth kernels, including the Gaussian, bounds on the smallest eigenvalue of Γ are much worse (with a decay which is exponential in (−1/h 2)), and the previous argument does not work. Since the bounds in [22] hold uniformly with respect to the number of points, a polynomial bound may still be valid for a fixed L, although we were unable to discover it.

Notice that, conditionally to the template, the momenta α are independent and follow a Gaussian distribution with inverse covariance matrix given by Γ. An example of simulated random deformations obtained using such a model is provided in Figure 1.

Figure 1

Random deformation of a template caudate surface (left). The six surfaces following the template are independent realization of the model described in (9).

3.2. Observation and Noise

The second part of our generative model is to describe the observation process, which takes an ideal surface T generated according to the model above, and returns the noisy observable.

Modeling such a noise process is a tricky issue. Obvious choices (like adding noise to the vertices of T ) do not work because one cannot assume that the observed surfaces are discretized consistently with the template. In this paper, we will work around this issue by assimilating the observation of a surface that of a singular Gaussian process.

For this, we consider that surfaces in ℝ3 are not observable directly, but only via their action on test functions, that we will call sensors. We define a sensor to be a smooth vector field w over ℝ3 (typically with small support). Given an oriented surface S, define

where N is the normal to S. The real number, (S, w) is the measurement of S through the sensor w.

Now, modeling noisy surfaces will result in assuming that, given any w, the measurement (S, w) is a random variable. We will assume that it is Gaussian, and more generally, that, given m sensors, w 1,…, w , the random vector ((S, w ), j = 1,…, m) is Gaussian.

S, via its action on sensors, is therefore modeled as a Gaussian random field. Given an ideal surface T , we will assume that its mean is given by

and the process is thereafter uniquely characterized by its covariance operator

We will assume that this covariance is associated to a symmetric operator L obs so that

(The apparently redundant parameter σ 2, which could have been included in L obs, appears here because it can be easily estimated by the algorithm, with the operator L obs remaining constant.)

To finalize our model, it remains to describe how S is discretized, that is, to make explicit a finite family of sensors through which S is measured. Let (z , j ∈ J) form a regular grid of points in Ω. Let γ be a radial basis function (a Gaussian, e.g.,) and define, for j ∈ J and d = 1,2, 3

where e is the dth vector of the standard basis of ℝ3 (this therefore specifies 3|J| sensors). The resulting observed variables are

where N ( = N · e is the dth coordinate of N . These variables are, by assumption, jointly Gaussian, with means m = (T , w ) and covariance matrix

Assuming that L obs is translation invariant, the resulting expression is a function of z − z that we will denote

Let R obs = (r (obs)) be the inverse matrix of the one with coefficients (γ obs(z − z ′), j, j′ ∈ J). The log likelihood of the process will include error terms taking the form

Replacing y by its expression in (16), we have

Let us analyze the first term. We have

with the notation

Treating the other terms similarly, we can rewrite the error term in the form

and we will abbreviate this (introducing a notation for the right-hand side) as ℰ obs = (1/σ 2)||S−T ||obs 2. Thus, we can write

We have the following important proposition.

Proposition 1

Assume that γ is taken equal to the Green function of L . Then, when the grid J becomes finer and K is given by (22), one has

See the appendix for a proof of this proposition (which requires some background on the theory of Hilbert spaces with a reproducing kernel) and possible extensions. For practical purposes, we will use γ obs instead of K obs in ||·||obs, therefore assuming that the sensors are chosen according to the proposition. It is interesting to notice that the resulting norm in this case is precisely the norm that has been introduced in [16] to compare surfaces, when they are considered as elements of a reproducing kernel Hilbert space of currents. We refer to [16] for details on the mathematical construction.

In the rest of the paper, we develop a parametric procedure to estimate the template T from the observation of i.i.d. surfaces S (1), S (2),…, S ( generated as described above. This includes in particular N hidden deformation momenta α (1), α (2),…, α (, such that the complete distribution of observed and unobserved variables is

where we have written for short T = T 0,.

We now discuss the estimation of the parameter μ, and of the associated template T 0,, which is the main purpose of this paper. This will be implemented with a mode approximation of the EM algorithm, as described in the next section.

4. Algorithms for Surface Template Estimation

4.1. Mode Approximation EM

Let Θ = (α (1), α (2),…, α () be the hidden part of the process, representing the collection of initial momenta, and let S = (S (1), S (2),…, S () be the collection of observed surfaces. The complete distribution for the process, including the prior is given by (26).

The EM algorithm is an iterative method that updates a current estimation of μ using the following two E- and M-steps.

E-step: determine the conditional expectation .

M-step: maximize this expression with respect to and replace the current estimation of μ by the obtained maximizer.

The conditional expectation can be expanded as

Considering the highly nonlinear relation between α ( and the deformed surface T , an explicit computation in (27) is impossible. We will therefore rely on the classic mode approximation in the EM which replaces the conditional distribution by a Dirac measure taken at the conditional mode, yielding

where

This results in a maximum a posteriori procedure that maximizes alternatively in μ and in the α ('s. Like in [15], we will refer to it as a mode approximation to the EM (MAEM) rather than a MAP algorithm, in order to strengthen the fact that it is an approximation of the maximum a posteriori procedure relying on the likelihood of the observed data only. As illustrated in [14], the MAEM can be biased (leading to inexact estimation of the template, even with a very large number of samples), especially when the noise is important, but it is obviously more feasible than the exact EM. Notice that the MAEM method is a special form of EM algorithm, and as such optimizes a lower bound of the log likelihood of the observed data.

We summarize the two steps of the ith iteration of the MAEM in our case. Suppose μ and the α ('s are the current variables to be updated. Then the next iteration is as follows.

MAE step: with μ (and therefore T) fixed, find, for n = 1,…, N, to minimize

and replace α ( by .

M step: with α ( fixed, update μ with the minimizer of

We now discuss how each of these steps can be implemented.

4.2. MAE Step

Our goal in this section is to optimize (30). We work with fixed n and drop it from the notation to simplify the expressions. The objective function is

This problem is equivalent to the surface matching algorithm considered in [16], with a slightly different formulation since [16] optimize an energy with respect to a time-dependent momentum instead of just the initial momentum (i.e., they solved simultaneously the geodesic estimation and the matching problems). These two formulations are equivalent when using continuous time (they produce the same minima), but they yield different results when discretized. In our setting, formulating the problem as in (32) is natural, and focuses on the modeled random variable, α.

We need to compute the variation of E with respect to α. This computation will be useful for the M-step also. We first discuss the discretization of the error term, which follows [16]. Let S be a triangulated surfaces, with vertices x ( = (x 1 (,…, x () and faces F ( = (f 1 (,…, f (). Each face is represented by an ordered triple of vertices: f = (x 1 , x 2 , x 3 ), and we define the face centers and area-weighted normals by

Then a discrete approximation of ||S−S′||obs 2 is

where S is considered as fixed and U is therefore considered as a function of the vertices, x (, of S′.

With this notation, we can write

We want to compute the gradient of E and for this, apply the chain rule to the transformations α → T → U(x (), yielding

where A* is the transpose matrix of A.

The gradient of U with respect to x ( has been computed in [16], and is given as follows. We denote by F ( the set of faces (triangles) that contain a vertex x in a triangulated surface S. For f ∈ F (, we let e (f) denote the edge opposed to x in f, positively oriented in f. With S′ = T , we have

where a = 1 if g ∈ F ( and a(g) = −1 if g ∈ F (.

Now let us derive the variation of x ( with respect to the initial momentum, α. We know that x ( = x(1), where x and a evolve according to the system (7)

with x(0) = x ( and a(0) = α (and γ , γ ′ are short for γ(||x −x ||2) and γ′(||x −x ||2)). Now an infinitesimal variation α → α + δ α in the initial condition induces infinitesimal variations a + δ a and x + δ x over time, and the pair (δ x, δ a) obeys the following differential system, that can be obtained from a formal differentiation of (7):

with γ ′′ = γ′′(||x −x ||2).

One can rewrite it in the matrix form:

where with

Solving this system with initial condition δ x(0) = 0 and δ a(0) = δ α provides what we have denoted

One does not need to compute all the coefficients of the matrix d x (/d α using this equation in order to apply the transpose in (36). This is fortunate because this would constitute a computationally demanding effort given that this matrix is 3L by 3L with L large. The right hand side of (36) can be in fact computed directly using a single dynamical system, given by

where J(t) is defined in (39). If (44) is solved from time t = 1 to time t = 0 with η (1) = ∂ U and η (1) = 0, then

This is a simple consequence of the theory of linear differential systems (a proof is provided in the Appendix for completeness). Note that the matrix J(t) depends on the solution of (7) computed with initial conditions x(0) = x ( and a(0) = α. To emphasize this dependency, we will denote it J(t) = J ((t) in the following.

Given this, we see that a variation α → α + δ α induces a first-order variation δ E of the energy given by

where the T-dot product is 〈δ α , α〉 = δ α · (Γ α) and Γ is the matrix formed with 3 by 3 blocs γ I d ℝ.

We choose to operate the gradient descent with respect to this dot product and therefore choose a variation proportional to δ α = −(2α + Γ −1 η (0)). So, the algorithm to compute an optimal α is the following.

Algorithm 1 (MAE Step for Surface Template Estimation)

Compute the variation ∂ U using (37).

Solve backward in time (44) initialized with η (1) = ∂ U and η = 0.

Replace α by α − ϵ(2α + (1/σ 2)Γ −1 η (0)) (using a line-search to optimize ϵ).

This algorithm has to be applied N times (for all α , k = 1,…, N) in the MAE step.

Remark 1

The matrix Γ being typically very badly conditioned, we numerically compute Γ −1 η after adding a small positive number to the diagonal of Γ. The inversion itself is computed using conjugate gradient.

4.3. M Step

There are many similarities between the M-step and the E-step variational problems, so that we will be able to only sketch the detail of the computation here. We need to minimize

with

Let us consider the variation of each term in the sum (fixing n, that we temporarily drop from the notation). Since

we can write

The function U being defined as before, we see that the derivative of the second term is given by applying the chain rule again, this time in the form

Like in the previous section, the transpose of the differential applied to the gradient of U can be computed by solving a dynamical system backward in time. In fact, it is the same system as with the variation in α, namely,

still initialized with η (1) = ∂ U and η (1) = 0, but the relevant result now is η (0). The gradient of is then

Once this is computed, the next step is to compute (reintroducing n in the notation)

This follows a similar procedure, using (44), with T 0 instead of T and μ instead of α. This requires solving

initialized with and η (1) = 0. The variation of associated to an infinitesimal variation of μ is then

We summarize the M step in the following algorithm.

Algorithm 2 (M-Step Algorithm for Surface Template Estimation)

For n = 1,…, N:

Compute ∂ U using (37).

Solve system (44) backward in time with initial condition η ((1) = ∂ U and η ((1) = 0.

Compute using (50).

Solve system (55) backward in time with and η (1) = 0.

Replace μ by μ − ϵ(μ + Γ −1 η (0)/λ), ϵ being optimized with a line search.

4.4. Surface Template Estimation Algorithm

We finally summarize the surface template estimation algorithm:

Algorithm 3 (Surface Template Estimation)

Having the hypertemplate T 0 and observed surfaces S (1),…, S (, the goal is to estimate the template T. Let T, μ, α (1),…, α ( denote the current estimation with initial guess T = T 0, μ = 0, α ( = 0. Then, in the next iteration, update with the following steps:

With T fixed, apply Algorithm 1 to update each α (, n = 1,…, N.

With α ('s fixed, apply Algorithm 2 to obtain a new value for μ.

Solve (7) initialized with the hypertemplate T 0 and the newly obtained μ to update the estimated template, T.

5. Result and Discussion

We applied the algorithm to surface data of human brain's caudate, thalamus, and hippocampus. All data are courtesy of Center for Imaging Science at Johns Hopkins University. Each surface has around 5–10 thousand triangle cells. We randomly chose one as the hypertemplate and the others as observed surfaces. In these experiments, we set λ = 1.0 and σ 2 = 1.0.

Figures 2 and 3 are the template estimation result for caudate and thalamus, respectively.

Figure 2

Estimating the surface template from 9 caudate data. (a)–(i) observed surfaces. (j) is the hypertemplate. (k) is the result.

Figure 3

Estimating the surface template from 9 thalamus data. (a)–(i) observed surfaces. (j) is the hypertemplate. (k) is the result.

We also applied the algorithm to 101 hippocampus surface data in the BIRN Project (Biomedical Informatics Research Network). In Figure 4, Panels (a)–(h) are 8 examples of the 101 observations. Panel (i) is the hypertemplate and Panel (j) is the estimated template.

Figure 4

Estimating the surface template from 101 data. (a)–(h) are 8 examples out of 101 observed surfaces. (i) is the hypertemplate. (j) is the result.

The result is visually satisfying in the sense that the estimated template is found to agree with a qualitative representation of the population. For example, in the caudate experiment, the estimated template has an obviously narrower upper tip than the hypertemplate. This captures the population characteristic since most observed data have narrower upper tips. Notice that the obtained template does not look smoother than the rest of the population, as would typically yield template estimation methods that average over fixed parametrizations.

Figure 5 shows how the energy in (31) changes with the iteration in the caudate experiment. This confirms the effectiveness and convergence of the algorithm. One can see the energy drops quickly in the first twenty iterations, then gradually slows down. After the 35th iteration, the energy changes little and the estimated template remains stable.

Figure 5

Energy change with the iteration.

In our model, the hypertemplate can be provided by an atlas obtained from other studies, although we here simply choose one of the surfaces in the population. Actually, as Figure 6 shows, different choices for the hypertemplate yield very similar results.

Figure 6

For the same observed population, we choose different surfaces as hypertemplate. The results only have minor differences.

6. Conclusion

In this paper, we have presented a Bayesian approach for surface template estimation. We have built, for this purpose, a generative statistical model for surfaces: the construction first applies a random initial momentum to the template surface, then assumes an observation process using test functions and noise. The template is assumed to be generated as a random deformation of a hypertemplate, completing the Bayesian model. We used a mode approximation EM scheme to estimate the surface template, and introduced for this purpose a novel surface matching algorithm optimizing with respect to the initial momentum. The procedure has been tested with caudate, thalamus, and hippocampus surface data, showing its effectiveness and convergence, and also experimentally proved to be robust to variations in the choice of the hypertemplate.