Ct3d: tracking microglia motility in 3D using a novel cosegmentation approach.

MOTIVATION: Cell tracking is an important method to quantitatively analyze time-lapse microscopy data. While numerous methods and tools exist for tracking cells in 2D time-lapse images, only few and very application-specific tracking tools are available for 3D time-lapse images, which is of high relevance in immunoimaging, in particular for studying the motility of microglia in vivo.
RESULTS: We introduce a novel algorithm for tracking cells in 3D time-lapse microscopy data, based on computing cosegmentations between component trees representing individual time frames using the so-called tree-assignments. For the first time, our method allows to track microglia in three dimensional confocal time-lapse microscopy images. We also evaluate our method on synthetically generated data, demonstrating that our algorithm is robust even in the presence of different types of inhomogeneous background noise. AVAILABILITY: Our algorithm is implemented in the ct3d package, which is available under http://www.picb.ac.cn/patterns/Software/ct3d; supplementary videos are available from http://www.picb.ac.cn/patterns/Supplements/ct3d.

1 INTRODUCTION

Capturing the motility of cells using time-lapse microscopy has become an important approach to understanding processes such as the cell cycle (Harder ), neuronal division and migration (Norden ), immune response (Cahalan and Parker, 2008) or the development of cancer (Ianzini ). Based on phase-contrast, confocal or two-photon microscopy, such live cell imaging protocols are now commonly established and corresponding equipment is commonly available. This has triggered the need for computational methods to quantitatively analyze time-lapse microscopy data. In this context, identifying individual cells and tracking their identities over time is one of the basic ingredients for computational analysis. Hence, cell tracking algorithms have attracted considerable attention in recent years (Meijering ; Miura, 2005). Here, we introduce a novel algorithm for cell tracking that allows to track cells, in particular zebrafish microglia, in 3D two-photon image sequences over time.

The majority of cell tracking algorithms, as surveyed by Meijering ) or Miura (2005), deals with cell tracking in 2D over time. Methods range from linking cells identified in individual frames using different segmentation approaches to active-contour (Dufour ; Sacan ; Shen ) or level-set algorithms (Dzyubachyk ; Li ; Mukherjee ; Nath ). The challenges imposed by the nature of the images to be analyzed lie in phenomena such as cell divisions (Al-Kofahi ; Li ), cells entering or leaving the displayed area, or a large number of cells that needs to be tracked simultaneously. In addition, cell tracking is often complicated by background inhomogeneity, for instance due to uneven illumination (Leong ), and cells touching each other. While these issues have been addressed extensively for tracking cells in 2D, surprisingly few approaches have addressed cell tracking in 3D. Besides naive thresholding approaches, there are only few advanced approaches, such as the active-contour based method proposed by Dufour ). Recently, several authors (Jaensch ; Kerekes ) proposed reliable methods for tracking centrosomes in Caenorhabditise legans embryos. Yet, these approaches are tailored toward tracking small, bright and circular objects which e.g. resemble a Gaussian spot of a specific size. Such assumptions, however, are not satisfied by the complex and highly variable shapes of microglia under consideration here. Cell tracking is also relevant in the context of tracking cell populations in vitro, which has attracted considerable attention recently (House ; Ong ; Padfield ).

The lack of methods for tracking cells in 3D has been reported as a limiting factor, for instance in the context of immunoimaging (Cahalan and Parker, 2008). Despite the well-established protocol to capture microglia, innate immune cells in the central nervous system, in 3D using two-photon microscopy following the seminal works by Nimmerjahn ) and Davalos ), motility analysis has been performed by (and limited to) manual estimations derived from 2D projections (Davalos ) in the numerous studies following these protocols. In fact, tracking microglia cells is complicated by several aspects. Microglia tightly contact specific brain structures in their resting state (Wake ), often making it difficult to clearly separate them from their surrounding tissue. Furthermore, the extension and retraction of the so-called microglia processes makes it practically impossible to separate them from other cells or surrounding tissues in a 2D projection. As we demonstrate in this study, cosegmentation-based cell tracking may overcome these difficulties and allows to reliably track microglia in 3D, both in resting state and when moving in activated state, as displayed in Figure 1.

Fig. 1.

3D motion patterns of two microglia in vivo reconstructed using ct3d. The red and green areas indicate initial positions of the two microglia. While the red cell remains in resting state, the green cell is activated through an induced injury and migrates along a trajectory (orange line, trajectory obtained by ct3d; blue line, trajectory obtained from manual annotation) toward a site of injury (purple dot); see Supplementary Video 1 for a rendered animation of the same data.

From an algorithmic point of view, our method can be seen as a broad generalization of thresholding methods. Otsu's early and still commonly used approach (Otsu, 1975) picks a cut-off intensity based on the gray-value histogram of an image, considering pixel intensities below this threshold as background and pixels exceeding the threshold intensity as foreground. To deal with background inhomogeneities and objects of varying intensities, different approaches such as locally adaptive thresholding (Kim and Park, 2005) have been developed. Our approach utilizes a highly systematic way of picking local thresholds in a hierarchical representation of all possible thresholds of an image, the so-called component tree (Jones, 1999; Najman and Couprie, 2004). In order to pick local thresholds in the component tree, we compare the component trees of consecutive time frames by solving the so-called tree assignment problem, a natural generalization of bipartite matchings and the associated assignment problem. Comparing component trees by computing tree assignments yields a cosegmentation of two images; for cell tracking, cosegmentations between two time frames in a video sequence are of particular relevance.

While the term cosegmentation has been coined by Rother ), our approach significantly differs from their approach, which is based on comparing histograms. On the contrary, our approach is morphological in the sense that it attempts to identify overlapping regions in two images by finding an optimal tree assignment.

Using cosegmentation has potential further applications in location proteomics beyond the cell tracking problem investigated in this article. Tree assignments as a generalization of bipartite matchings were introduced and applied by the last author recently (Mosig ), and were recently shown to be computationally hard in general (Canzar, Elbassioni and Klau, personal communication, 2010). Applying tree assignments to component trees for obtaining cosegmentations is a novel contribution in this work. In fact, cosegmentations promise to be useful in other bioimaging (and eventually image processing) applications beyond cell tracking. One straightforward application where cosegmentation is of high relevance are protein colocalization studies. Studying colocalization has recently become of relevance through the availability of corresponding two- or multi-label fluorescence microscopy (Schubert ; Zinchuk and Zinchuk, 2008) or in situ hybridization (Boettiger and Levine, 2009; Carson ) techniques.

We implemented our algorithm in the publicly available ct3d software package, which is accompanied by the at3d graphical user interface. In terms of applying our algorithm, this article focuses on evaluating the performance of our cosegmentation-based approach for 3D cell tracking, leaving colocalization studies as a future direction. Cell tracking performance is evaluated both on two-photon live cell imaging data displaying zebrafish microglia in vivo, and on synthetically generated data that allow to determine the algorithm's accuracy based on the ground truth the synthetic data were generated from.

2 METHODS

2.1 Computational methods

Our algorithm is based on representing each image F1,…, F by its component tree (Jones, 1999). The component tree of an image I is obtained by considering the connected components of the thresholded versions Iθ of I under all possible thresholds θ. The set of all connected components under all thresholds is obviously hierarchically ordered by subset inclusion. This hierarchical order defines the component tree, which can be computed in linear time (Najman and Couprie, 2004). For examples of 1D images and their component trees refer to Figure 2.

Fig. 2.

Tree assignment of two (pruned) component trees for two 1D images I and J. Vertices not eliminated by the second pruning step are indicated by circles. All other non-branching vertices are eliminated in the pruning step. The tree assignment indicated by the dashed arrows is A = {(a, c), (b, d), (e, f)} with a weight of w+w+w.

Figure 3 illustrates the basic steps of our cell tracking algorithm. The outline of the algorithm is as follows: we start with computing and pruning component trees for each time frame. Then, tree assignments between each pair of consecutive component trees are computed. The tree assignments can be turned into segmentations of the original images. This produces two segmentations of each image, requiring computation of a consensus segmentation. The resulting unique segmentation of each image then requires a standard bipartite matching between consecutive time frames to track cell identities over time. Details of the individual steps are provided in the following paragraphs.

Fig. 3.

Overview of complete cell tracking algorithm

Computing and pruning component trees: for efficiently computing component trees, we relied on established algorithms based on a union-find data structure (Najman and Couprie, 2004). In order to reduce the size and the complexity of the component tree, we apply a pruning procedure to these trees. Pruning is a crucial ingredient of our algorithm, as running tree assignments on the complete component trees would be computationally too demanding. The goal of pruning is thus to eliminate as many vertices as possible, keeping only those that reflect the relevant structures of the underlying image. This is conceptually closely related to the ideas behind component filters (Salembier and Serra, 1995).

In a first pruning step, we eliminate all vertices that represent a connected component of size less than θmin or exceeding θmax, where θmin,θmax are parameters specified by the user. In a typical microscopy setting, loose upper and lower bounds on the size of the cells to be tracked are usually easy to estimate. In practice, these parameters can be chosen quite loosely, with the ratio θmax/θmin equal to 10 or larger.

In a second pruning step, we eliminate all vertices in T that do not occur immediately before or immediately after a branch in the original tree (excluding leaves), as indicated in Figure 2. As this might eventually delete relevant vertices, we introduce a single node cutoff parameter σ, and keep every vertex v with only one child vertex w, if the symmetric difference between the areas associated with v and w comprises more than σ many pixels. As a rule of thumb, choosing σ<θmin/2 is a good choice, which automatically limits the error rate to 50% in the worst case. For better worst−case guarantees, a smaller fraction of θmin can be chosen. Note that all results presented in this work were obtained using σ = 200.

We consider the size cutoffs θmin and θmax as parameters derived from rough estimates on the expected size of the cells to be identified. The main purpose is to eliminate vertices that result from noise in the input images, where such size cutoffs are known as grain filters (Vincent, 1993) as a specific type of connected operators. The expected size of noise components can be derived using random graph theory (Coupier ). As this is currently understood only for 2D images, we treat θmin as a user specified parameter.

Tree assignments: to obtain cosegmentations of the image pairs F and F, we compute a tree-assignment between the corresponding component trees T and T. As illustrated in Figure 2, a tree assignment A associates vertices (i.e. connected components) in T with vertices in T. As each assignment identifies a component in T and a component in T as a putative cell, no ancestor or descendant of the matched vertices may be part of a valid tree-assignment—otherwise, some area in an image would be occupied by two cells. If two assignments involve no overlapping components, we will also refer to them as compatible. Naturally, valid tree-assignments can be identified as pairwise compatible assignments. More precisely, as the quality of an assignment X can be weighted by the relative overlap ω of the associated components, we aim to find maximum-weighted sets of pairwise compatible assignments.

Algorithm cosegmentation-track

Input: Sequence of images F1,…, F; pruning parameters θmin, θmax, single−node cutoff σ.

Output: Sequence of segmented images S1′,…, S′.

Compute component tree T for all i ∈ [1: N].

Prune T to obtain T′ using θmin, θmax, σ.

For each i ∈ [1:N − 1], compute A = treeassign(T, T).

Use A and A to obtain two segmentations of image F; compute consensus segmentation S from these two.

For each i ∈ [1:N − 1], compute a maximum-weighted bipartite matching between the segments in S and S.

Assign random color to each segment in S1 to obtain S1′. In S′, assign the same color to the segment as the one matched in S.

Finding maximum weighted pairwise compatible assignments naturally translates into an integer linear program by introducing a binary indicator variable X for each possible assignment between vertices a and b. The linear program is established by introducing one constraint for each root–leaf path in each of the two trees; the sum of all variables involving any vertex along the path must be constraint to at most 1.

Weights for tree assignments: an important role is taken by the weights ω. A straightforward choice is the ‘relative overlap’ between the corresponding areas γ(a) and δ(b), i.e.

This score is also known as the Jaccard-Index or the Tanimoto Score. An important observation is that these weights are restricted to the interval [0, 1]. For the solution of the tree assignment, this means that in a sense also the number of segments identified will be maximized. For weighting schemes yielding unbounded values over the real numbers, large and very highly scored segments might ‘overshadow’ many small good assignments in the solution. For computing all weights between any pair of vertices in two given trees, we implemented a fast algorithm utilizing the fact that 0-weighted vertex pairs do not require a variable in the linear program, along with the property that ω=0 implies ω=0 for all descendants a′ of a and b′ of b. This leads to a speedup of an order of magnitude compared to a naive way of computing all pairwise weights.

Turning tree assignments into cosegmentations: assume A={(a1, b1),…, (a,b)} is a tree assignment between the component trees of two images I and J. Then, the areas γ(a1),…, γ(a) refer to pairwise disjoint segments in I, and can hence be considered as a segmentation of I. Correspondingly, the areas δ(b1),…, δ(b) induce a segmentation of J; note that the segments γ(a) and δ(b) necessarily overlap, as they require a non-zero weight to be included in an optimal tree-assignment.

Consensus segmentation and bipartite matching: when determining a segmentation of frame i, we are confronted with two competing options; one segmentation P resulting from the cosegmentation of frames i − 1 and i, the other one, Q, from the cosegmentation of frames i and i + 1. We resolve this by computing a consensus segmentation. We generally use P as the starting point of a consensus segmentation. Any segment in Q that does not overlap with any segment P is supplement to P to obtain the consensus segmentation P′. This ensures that cells entering the scene in frame i can be identified in frame i (rather than frame i + 1).

Filtering results: as for most segmentation and tracking approaches, the results obtained from the steps described above has a tendency toward overdetection, i.e. detecting segments that result from image noise rather than cells. To filter out those segments, we utilize life span filtering, i.e. we filter out all cells whose identity can be traced across less than a certain minimum number of frames. This cell filter, along with several other ways to eliminate cells with unsuitable size or volume features, follows corresponding features of the Celltrack software (Sacan ) for 2D cell tracking; for ct3d, they are implemented in the graphical user interface of the at3d tool shown in Figure 4.

Fig. 4.

Screenshot of at3d, which is designed for exploring cell tracking results and extracting motility parameters. It also supports features for correcting overdetection and oversegmentation. Cells can be selected and removed either individually or by filtering based on different criteria such as size or life span.

2.2 Comparison with other approaches

While a quantitative comparison with other methods is subject of Section 3, there are noteworthy commonalities and differences with other approaches on a qualitative level as summarized below:

Locally adaptive thresholding: deriving a segmentation from identifying vertices in a component tree can be seen as a way of locally adaptive thresholding; rather than determining local thresholds in pre-computed regions of interest as in Kim and Park (2005) or windows of fixed size, every component represented in the component tree is eligible to define a local threshold.

Absence of a global background model: in contrast to both thresholding and level-set methods, in particular active-contour approaches, our approach does not involve any assumptions regarding the distribution of the background intensities. This is particularly useful in the presence of background inhomogeneity.

Alternative matching schemes: one class of cell tracking approaches is based on computing bipartite matchings between segmentations of individual time frames. As bipartite matchings may not capture events such as cell division or cell fusion, recent works such as Padfield ) introduced alternative matching schemes that are more flexible. Truly generalizing bipartite matchings, tree assignments can be seen as such alternative matching scheme. They are particularly interesting for cell tracking, as they may capture events such as cell division, cell fusion or cells entering the scene. See Supplementary Video 4 for synthetic data displaying a simplistic simulation of a cell division tracked by ct3d.

2.3 Implementation

We implemented component trees, tree-assignments and the complete cell tracking algorithm, in C++ using lp_solve for solving both the tree assignment and the weighted bipartite matching (integer) linear programs, all of which is compiled in the ct3d command line tool. Cell tracking results can be further explored using at3d, which allows the user to select and extract specific cells identified by the cell tracking procedure, and derive their motility parameters such as velocity and deformation. The at3d tool is implemented using the qt framework for graphical user interfaces. Input and output of image series is designed to be compatible with other visualization software, most notably v3d (Peng ) for producing rendered visualizations of the output.

2.4 Experimental materials and methods

3D time-lapse, two-photon microscopy imaging of zebrafish microglia was performed as follows: Zebrafish preparation. Zebrafish Tg(ApoE:egfp), in which microglia express EGFP (Peri and Nüsslein-Volhard, 2008), were maintained in the National Zebrafish Resources of China (NZRC, Shanghai, China) with an automatic fish housing system (ESEN, Beijing, China) at 28○C. Embryos were raised at 28.5○C under a 14/10 h light–dark cycle in 10% Hanks solution, which consists of (in mM) 140 NaCl, 5.4 KCl, 0.25 Na2HPO4, 0.44 KH2PO4, 1.3 CaCl2, 1.0 MgSO4 and 4.2 NaHCO3 (pH 7.2). They were staged as previously described (Kimmel ). Zebrafish handling procedures were approved by Institute of Neuroscience, Shanghai Institutes for Biological Sciences, Chinese Academy of Sciences.

Time-lapse imaging: for in vivo imaging, zebrafish larvae at 5–7 days post-fertilization (dpf) were first anesthetized with Hanks solution containing 0.02% tricaine methanesulfonate (MS222) and embedded in 1.5% low-melting point agarose. Time-lapse images of microglia were captured, via a 40× water objective mounted on a two-photon microscope with 900 nm (Prairie) or a Nikon A1R confocal microscope with 488 nm. Z-stack images, which covered the whole area of microglia in the optic tectum, were collected every 2 min at a section thickness of 1 μm. Each frame (512 × 512 pixels, 14 to 34 Z-stack images) was averaged four times.

3 RESULTS

We evaluated our algorithm on two types of data. First, we applied it to an in vivo time-lapse sequence of 3D two-photon images of zebrafish midbrain, displaying the motility of microglia; second, we applied ct3d to synthetically generated data for quantifying the accuracy of our cell tracking results. Evalutation on in vivo data was accomplished by comparison with manually annotated trajectories of specific microglia in three datasets. Note that manual annotation is limited to trajectories, whereas boundaries of the cell volumes are almost impossible to obtain in 3D, beside systematic problems with manual annotations (Huth ). Hence, we additionally created synthetic ground truth data to further evaluate the performance of our method. We followed the procedure used in Dufour ), generating (noise perturbed) elliptical objects of average intensity I above Poisson distributed background noise of intensity I. In addition to the procedure from Dufour ), we created perturbed images with different types of background inhomogeneities, as shown in Figure 5: in a second set of data, we introduced a multiplicative vignetting effect to the data, following the vignetting model by Kang and Weiss (2000) under different focal lengths f and off-axis illumination parameters α. In a third set of data, we introduced an additive linear gradient along the x-axis of different slopes β. Each time series consists of 20 time frames, each of size 200 × 200 × 40 pixels. On these data, we ran ct3d with size cutoffs θmin=500 and θmax=5000, and a single−node cutoff of 200 pixels. In the resulting sequences, all cells whose identity could be traced through the complete sequence were kept, while all other cells were discarded using the at3d tool.

Fig. 5.

Sections of synthetically generated images—Left: 2D section (top) of an image perturbed background vignetting following the Kang–Weiss model (Kang and Weiss, 2000). The dashed box indicates the position of the 1D section displayed in the lower part. Right: same setting with the background perturbed by an additive linear gradient. In both instances, ct3d yields reliable tracking results (see Table 1).

Table 1.

Tracking results for synthetic data

To assess the quality of tracking results, we derived the number of identified cells, the voxel recall, i.e. what percentage of all ground truth object voxels was recovered in the tracking result, as well as the error rate defined as the ratio between the cardinality of the symmetric difference between tracking result and ground truth and the total number of voxels in the ground truth dataset. Left columns: each dataset displayed three cells with different intensities I above different levels of noise I, displaying different types of background inhomogeneities (see text). Middle columns: the three cells were correctly recovered under all settings by ct3d. Right columns: segmentation using the active contour approach by Chan and Vese (2001) is highly reliable in the absence of background inhomogeneity. With increasing level of inhomogeneity, voxel recall decreases, while the error rate increases. Supplementary Figure 1 displays some of the results summarized here.

3.1 Evaluation on synthetic data

Our results on the synthetically generated image sequences are summarized in Table 1 and indicate that ct3d is highly robust against different types and intensities of background inhomogeneities. The results suggest that ct3d has a tendency to identify components slightly (about 10%) larger than the actual objects, as can also be seen in the sample output in Supplementary Video 3. This is a natural consequence of pruning the component trees, where the vertex that would perfectly represent an object is unlikely to be part of the pruned tree. This effect can be reduced by smaller choice for the single-node cutoff parameter σ at the cost of higher computation time.

Running times varied between roughly 4 and 6 min, with an average of 303.42 s, for completely tracking one dataset. The majority of running time was spent on constructing the component trees and computing the overlap weights (14.23 s on average per time frame), whereas each tree assignment required less than one second on average; the pruned component trees typically comprised a few dozens of vertices. As a reference algorithm to compare against the performance of ct3d, we computed a segmentation of each time frame using the active contour approach by Chan and Vese (2001),2 which is a well-established and state-of-the-art representative of the large family of level set methods. As shown in Table 1, this method works highly accurate in the absence of background inhomogeneity while getting less reliable with increasing levels of background inhomogeneity, as can be expected due to the involvement of a global background model.

3.2 Tracking microglia in vivo

Figure 6 shows a result obtained from our tracking algorithm on a time series of microglia images measured as described above. We reduced resolution by half, so that the resulting width and height varied between 146 and 250 pixels, while the depth ranged between 14 and 66 layers for each time frame; each time series comprised 30–80 time frames. Gray scale resolution was reduced from 16 bit to 8 bit. We applied ct3d using parameters θmin=200, θmax=10 000 and a single−node cutoff of 200 pixels; the resulting pruned component trees contained 69 vertices on average, ranging between 40 and 168 vertices. Running times varied between roughly 2 and 10 min, with 471 s on average.

Fig. 6.

Top: visualization of trajectories obtained by manual annotation (blue lines) with trajectories obtained using ct3d (orange lines) for microglia in activated state, see (a) and (b), as well as resting state, see (c) and (d). Bottom: quantitative comparison of trajectories obtained by ct3d and the active contour approach from Chan and Vese (2001). In general, ct3d could identify the annotated cells in all time frames (columns # Frames). The root mean square distance to the annotated trajectory measures a fraction of the diameter of the annotated cell (columns cell Δ and RMS), while the Chan–Vese algorithm missed varying numbers of cells or failed completely (last column). For the Chan–Vese results, a parameter set was optimized for dataset (a). This parameter set (μ=1, ν=.5, λ1=.2, λ2=5) was also applied to datasets (b) to (d). While for dataset (a), the result is comparable to ct3d, the annotated cell was identified in only 29 out of 49 time frames in (b). In datasets (c) and (d), the annotated cell could not be identified at all using these parameters. See Supplementary Figure S2 for further illustrations.

Under the given experimental protocol, the phenomenon of overdetection, i.e. the recognition of segments that are not microglia, is inevitable. This is due to the limited specificity of the apoE-GFP gene, which is also expressed in cells other than microglia in the surrounding tissue, often at comparably high levels as in microglia. Yet, ct3d identifies microglia as segments that can be visually distinguished from non-microglia segments by a human observer due to their characteristic shape or motion patterns. The graphical user interface of the at3d tool (see Fig. 4) allows to manually eliminate false positive cells from tracking results by different filtering and visual selection functions similar to those provided by Celltrack for 2D time-lapse sequences. The at3d tool also allows to manually correct for the occasionally observed events of oversegmentation, i.e. one microglia being recognized as two segments.

We used at3d to eliminate non-microglia from all six datasets and correct oversegmentation in individual frames. We were able to reconstruct trajectories of all relevant microglia that can be visually identified in the original datasets; only one dataset was affected by a sudden ‘frameshift’, i.e. the sample changing its distance to the camera in the z direction, leading to interrupted trajectories at the corresponding time frame. In few other cases, the trajectory of a microglia was interrupted in individual frames, which we could correct using at3d. Examples of the final results are given in Supplementary Videos 1 and 2.

In order to quantitatively evaluate the quality of ct3d results on in vivo microglia data, we compared ct3d trajectories with manual annotations. To this end, we selected four microglia from four different datasets and annotated their trajectories manually using the 3D polyline markup feature of the v3d software. The root mean square distance between the ct3d and the manual trajectories turned out to be 2.9 voxels, 5.5 voxels, 2.6 voxels and 5.4 voxels, respectively, in the four different datasets. These deviations can be considered relatively small in relation to the cell diameters, which were measured as 22.8, 22, 19.6 and 44.3 voxels, respectively.

We also applied the active-contour approach from Chan and Vese (2001) to the datasets. Identifying the the four microglia in our annotated evaluation dataset required intensive tuning of the four major parameters. In fact, we could not identify a single set of parameters that works across all datasets. A parameter set tuned for a specific dataset worked comparably well on the respective dataset, but performed significantly worse or yielded no result on other datasets, see Figure 6.

4 DISCUSSION

We have presented a novel approach to tracking cells in 3D time- lapse microscopy image sequences, based on the concepts of component trees and cosegmentation. We demonstrate that this approach is robust against the numerous challenges imposed by images measured in an in vivo environment, and allows to identify microglia and their motion patterns in zebrafish neural tissue when combined with the at3d annotation tool. In a quantitative evaluation, we show that our approach is robust against different types of background inhomogeneities. This suggests that ct3d and at3d are potentially useful for in vivo imaging studies investigating other aspects than just microglia motility. In its current formulation, our cosegmentation approach relies on the assumption that the area occupied by an object overlaps between two consecutive time points. While this may not be satisfied in all cell tracking problems [e.g. when tracking centrosomes (Jaensch )], it is a reasonable assumption for many immunoimaging-related studies.

To the best of our knowledge, our approach is the first that can identify and track microglia in live cell imaging time series. In most cases, obtaining reliable trajectories still requires manual post-processing of the output. The most notorious difficulties certainly are the complex morphology— their deformation patterns, irregular shapes and interaction with the surrounding—as well as the unspecificity of the fluorescent markers available. In this light, our approach constitutes significant progress in the sense that it has sufficient sensitivity to separate microglia form their surrounding. Yet, a fully automated approach remains a major and certainly non-trivial challenge. A first step in this direction might be the combination with level-set based approaches as utilized for 2D cell tracking by Nath ) that might yield more accurate cell boundaries in some cases. Yet, ct3d promises to be a key tool for further studying open questions regarding microglia, such as to determine if and how glia and microglia share the task of finding and removing apoptotic neurons from the vertebrate brain (Peri and Nüsslein-Volhard, 2008).

Beside the direct relevance for in vivo time-lapse microscopy, our study indicates that our morphological approach to cosegmentation is both practical and of relevance in bioimaging. Consequentially, it appears a natural approach to apply cosegmentation to protein colocalization studies, which have attracted considerable attention in recent years following the availability of two- or multi-label fluorescence microscopy (Zinchuk and Zinchuk, 2008).

Another major experience that can be drawn from our work is the obvious potential of component trees and the closely connected theory of component filters in bioimaging. While component filters are well known to leave relevant gradients unchanged, recent work such as the results by Coupier ) allow to assign a statistical significance to components observed in an image. Such concepts might be particularly useful when combining component trees with cosegmentation for judging the relevance of colocalized segments observed when comparing two component trees.