A new method for multiple sperm cells tracking.

Motion analysis or quality assessment of human sperm cell is great important for clinical applications of male infertility. Sperm tracking is quite complex due to cell collision, occlusion and missed detection. The aim of this study is simultaneous tracking of multiple human sperm cells. In the first step in this research, the frame difference algorithm is used for background subtraction. There are some limitations to select an appropriate threshold value since the output accuracy is strongly dependent on the selected threshold value. To eliminate this dependency, we propose an improved non-linear diffusion filtering in the time domain. Non-linear diffusion filtering is a smoothing and noise removing approach that can preserve edges in images. Many sperms that move with different speeds in different directions eventually coincide. For multiple tracking over time, an optimal matching strategy is introduced that is based on the optimization of a new cost function. A Hungarian search method is utilized to obtain the best matching for all possible candidates. The results show nearly 3.24% frame based error in dataset of videos that contain more than 1 and less than 10 sperm cells. Hence the accuracy rate was 96.76%. These results indicate the validity of the proposed algorithm to perform multiple sperms tracking.

INTRODUCTION

Motion analysis or quality assessment of human sperm cells is of great importance for clinical applications of male infertility. In recent years, computer aided sperm analysis (CASA) systems automated and improved the accuracy of the analysis and evaluation of motion sperms. CASA describes different parameters of sperm cells motion. So it is clear that the first step is calculating the tracking parameters as accurately as possible.

World Health Organization has specified a classification in motility types, where the motility of sperm is divided into four different grades:

Type A: Sperm with rapid progressive motility. Those are the ones with more fertility skill due to their capability to reach the oocyte and to penetrate the membrane. These are normally the ones selected with the swim-up procedure for intracytoplasmic sperm injection

Type B: Sperm with slow progressive movement and possibility of reaching the oocyte but with less possibility of penetrating the membrane

Type C: Non-progressive motility sperms. They do not move forward despite their different movement of tails

Type D: Immotile or dead sperms.[1]

There are two major problems in the computerized tracking of sperm cells. First of all, due to the small size and deep-seated movement, detection of moving cells is difficult, secondly tracking of sperm cells is a complex problem due to the motion uncertainty and partial or full occlusion of the sperm cells. In general, location and orientation of the sperm cells simultaneously change in consecutive frames. As an active research topic in computer vision, visual tracking has been studied extensively. However, there are few researches in sperm tracking. Friedrich et al. used the resistive force theory based on tail movement of sperm cell to track them in 250 frame rate video.[2] In[3] unlike the conventional CASA systems, a regular light microscope with a digital camera directly attached to its eyepiece is used and correlation based template matching scheme is used for tracking. Their method needs a manual initialization in the first frame. As a good method in this area we can enumerate Xiuzhuang and Yao method.[4] They incorporate an oriented adaptive mean shift optimization into particle filter framework to enhance the efficiency of particle filter in sperm tracking.

The remainder of the paper is organized as follows: In the method section, we introduce structural algorithms including three parts: Background subtraction, performing nonlinear diffusion filtering and tracking sperm cells. A novel criterion for tracking sperm cells is derived from phase-contrast microscopy video and is described in multi cell tracking algorithm. This algorithm can perform tracking while there is entry, exit and partial covering or complete occlusion. Results give a thorough discussion about the simulation results and finally the last section concludes this paper by summarizing the key points and other related considerations.

METHODS

Recordings of sperm samples of 30 patients are carried out by a phase-contrast microscope at a magnification of ×120 in Isfahan Fertility and Infertility Center. The resolution and frame rate of these video sequences are 240 × 320 pixels and 25 fps respectively.

The tracking algorithm was set and run using MATLAB 9 software in a windows XP operating system on a 2.6 GHz Pentium IV personal computer.

Background Subtraction

The first step in object tracking is the detection of moving objects. Background subtraction is a widely used approach for detecting moving objects from static cameras. In this area, many different methods such as temporal difference, Gaussian mixture model,[5] Eigen background[6] have been proposed over the recent years.[7] Consecutive frames difference is the quickest method and it does not need a great amount of memory. However, it has a big defect, “depending on threshold, which we must select to segment moving objects.” To overcome this problem in two sections of “Non-linear Diffusion Filtering (NLD)” and “Background Subtraction Using NLD” we present a new application of non-linear diffusion filtering.

Nonlinear Diffusion Filtering

There has recently been some work to analyze the noise removing process in terms of the diffusion process. A nonlinear diffusion equation can be adopted to achieve the purpose of noise removal without blurring the edge and is a useful tool for a multi-scale description of images for image segmentation.[8] Non-linear diffusion filtering goes back to Prona and Malik.[9] Although their method in its original formulation is regarded to be ill-posed and is usually performed with explicit schemes.[9] It is only stable for very small time steps, which leads to poor efficiency and limits their practical use. In this paper, we use semi-implicit scheme proposed by Weickert et al.[10] which is stable for all time steps. This reliable scheme uses an additive operator splitting, which guarantees equal treatment of all coordinate axes.

Background Subtraction using NLD

In order to explain new application of NLD, consider we have a video sequence with T frames and dimension of M × N. If we apply frames difference algorithm, the difference matrix dimension will be M × N × (T-1). Intensity values of pixel at position of i, j in T frames (Ii, j) are shown in Figure 1a. By using an appropriate threshold, we are able to capture moments that a moving sperm passes from this position. However, selecting the appropriate threshold is a challenging task. We apply 1D-NLD to Ii, j in order to remove the noise. As we observe in Figure 1b, we are able to select an appropriate threshold value from a wide range of values. In another words, the sensitivity of threshold selection in background subtraction method is vanished.

Figure 1

Intensity variation of one pixel (a) before and (b) after applying NLD

To increase the computation speed, we apply 2D-NLD to one row of this 3D difference matrix instead of applying 1D-NLD to one pixel. After applying on whole of rows, we can observe that choosing a threshold on each difference frame is very simple and have wide range. Figure 2 shows the effect of applying three different threshold values before [Figure 2a] and after [Figure 2b] using NLD. Hence applying NLD reduces the sensitivity of results with different threshold values. Figure 3 demonstrates the three dimensional representation of one frame before and after applying NLD.Then after NLD implementation, we apply morphology operator on threshold frames to find the connected components and remove small regions.

Figure 2

Result of applying three different threshold (a) without NLD (b) with NLD

Figure 3

Three dimensional representation of one frame (a) before and (b) after applying NLD

To evaluate the detection accuracy, we define precision as the ratio of detected moving sperms to the total number of moving sperm cells actually in each frame. The computed precision for each individual sequence is listed in Table 1. The proposed method is 91.51% accurate in moving sperm detection which is comparable with.[11]

Table 1

Details about some tested sequences and detection validation on each sequence

Our goal was detection of motile sperms and the fundamental idea for doing this is to find the difference between consecutive frames, so with this algorithm dead sperms could not be detected.

Matching-based Multiple Sperm Cells Tracking

In the previous section, we described how to detect sperm cells in each frame from the sequence. Tracking multiple cells requires a global optimal assignment of cell correspondences across multiple frames of the whole sequence.

The problem of establishing correspondence among multiple frames can be posed within a graph theoretic framework. Let us consider the simple case of multiple cells observed in two successive frames. Let denote the center of detected cells in the pth frame and represents those points in the qth frame.[12] Our objective is to find the one-to-one matching from to . Np and Nq are the number 4 of sperm cells in pth and qth frames respectively. This context can be modeled with a complete bipartite graph G = (U, V, E), where , and E represent the set of matching hypothesis between each pair of sperms from frame p to q.

A weight function is critical for the correct matching of different cells across a multi-frame sequence. This weight function, which assigns two cells (wp,qi,j) is based on Euclidian distance of ith and jth sperms in two consecutive frames-pth and qth frame-respectively (Eq. 1). dmax is the maximum distance that a sperm can move between two frame of pth and qth . To calculate this distance we manually investigated the maximum movements of the cells in two consecutive frames. Based on this experiment, a single sperm does not move more than the length of a sperm head within one frame. Therefore, dmax is considered equal to the length of a sperm head.

Given the weights defined by a specific matching criterion, the global optimal correspondences between the two sets of cells can be obtained by finding the maximum matching of G. The maximum matching M between the cells on two frames can be found as

Where H is the union of all the possible matching of G.

There are several efficient algorithms that can be used to find the maximum matching of a bipartite graph and in this study Hungarian search method was used.[13]

This cost function can be defined based on motion constraints such as motion proximity, maximum velocity, common motion and etc., in our context due to motion uncertainty only maximum velocity or equivalently maximum displacement of sperm cells between two frames is a common motion feature.

Optimization of the Matching-based Algorithm

Sometimes there is more than one object in the maximum displacement area. This problem needs to be solved directly. As the head of sperm travels along a zigzag path, it is seen that the trajectory of sperm's head movement is triangular. Suppose that a1 and a2 are weight functions between frame t−1 and t and frame t and t−2 respectively for sperm 2.b1 and b2 are weight functions between frame t−1 and t and frame t and t−2 respectively for sperm 3. We need to choose either sperm 2 or sperm 3 as the next location of sperm 1. If a1 = b1 [Figure 4] we consider frame t−2 and modify the algorithm by defining a new function based on the linear combination of w and w. w is weight coefficient between two frames of t and t−1 related to cost function. Therefore, matching equation is reclaimed as below.

Figure 4

Weight function when two or more sperms are near the target sperm

Where

γ is the parameter that describes the amount of dependence on frame t-1 and t−2 [Figure 4]. The best result is found with γ = 0.6.

We should consider some limitations of the proposed algorithm:

Some sperms may enter to or exit from a frame. In this condition we enhance the method as follows:

A window [Figure 5] to exclude the margin of the frames is considered. The dimensions of this window are equal to the dimensions of the frame minus the length of a single sperm. A search within the excluded area is done for one sperm. In order to compensate for different number of cells in consecutive frames the following algorithm is employed:

Figure 5

The margin of a frame showing sperms going in or out of the frame

If frame t had more cells than frame t−1 and one or more sperms were found in the excluded area it shows that the sperms are entering so new labels are assigned to the extra cells and if frame t had less cells than frame t−1 and the excluded are in frame t−1 had one or more sperms, then it shows that sperms are moving out of the frame so some labels are eliminated

Sometimes one sperm is missed only in one frame and no cell is found in the marginal area, but it reappears in the next frame. In this situation, we estimate xji (the position of ith cell in frame t in the direction of x) by adding xt-1i (the average of previous displacement of that sperm in x) and y coordination separately (eq. 11)

Two or more sperms may coincide with each other. After collision one cell overlaps one another. But still we label two indexes on these sperms [Figure 6]

Figure 6

This figure depicts two sperms labelled as 1 and 2 at frame t-1 collide at frame t and overlap one another. However, we specify two labels on them

Partial occlusion, when sperms collide, correlation between them will arise. So watershed algorithm is utilized which is proper for separating correlated objects[14] and again label them and finally.

Figure 7 shows the results of applying the proposed tracking algorithm on sample video sequences with different sperms in its frames. At the first frame, there are five motive sperms. But some sperms enter or exit from consequent frames. For example one sperm enters to frame 10 and the other exits from frame 13. In addition, at three frames of 34-36 two sperms of 4 and 6 get close to each other, coincide and then deviate from their previous path. However, the algorithm still is able to track them.

Figure 7

The result of sperm tracking

RESULTS AND DISCUSSION

We compare our sperm tracking algorithm with the template matching algorithm,[3] the mean shift tracking method,[4] the standard particle filter,[15] combination of Kalman and particle filter.[16]

To evaluate the performance of the tested approaches, we use two measures. One is mean square error (MSE), which is defined as the mean of deviation of tracked path from ground true path. For each automatic cell trajectory, we computed the average distance (in pixels) in each frame between the manually marked locations and those computed by the algorithms. If the distance in one frame is smaller than a pre-selected threshold, then the tracking result in this frame is considered to be correct and the frame is counted as a correct frame for the algorithm. This criterion is used in sperm tracking[341516] used MSE to describe the accuracy of sperm tracking. Our approach performs sperm tracking accurately with 2.62% of average MSE criterion.

A different method is presented in,[12] which is called frame-based error Ef defined as Ef = 1−np /nT, where np is the number of frames that each sperm cell correctly tracked and labeled and nt is the total number of frames in which the sperm cell appears in the sequence.

At first we study the performance of our method on some dataset including a single sperm within each frame without the use of optimized algorithm. The computed precision of two error measures for each sequence is listed in Table 2. These are comparable with results in.[341516]

Table 2

Result of MSE and Ef in some sequences which contain only single sperm

Table 3 shows that the proposed method has the best performance at the 96.76% accuracy rate with the sequences where the density of the sperms was relatively low (<10). Its average frame-based error is about 3.24%. Figure 8 shows the results of simultaneous multiple sperm cells tracking on one frame.

Table 3

Tabulated percentages are frame-based error Ef of our proposed algorithm, Si is ith sequence and Ns is the number of sperm cells exist in video frames

Figure 8

Results of simultaneous multiple sperm cells tracking

Table 4 shows the summarized results of proposed algorithm compared with other studies while there are single or multiple sperms in frames. Failure tracking column shows the ratio between the number of completely mistaken path and the number of total sperms. It has to be mentioned that none of databases used in each study were similar. In addition Figure 8 depicts the results of the algorithm of simultaneous multiple sperm cells tracking.

Table 4

Comparative results between some reports about one or multi sperms tracking with each frame

In this paper, we proposed a new approach for automatic detection and simultaneous tracking of multiple sperm cells.

In the stage of detection, we use a background subtraction technique for detection of moving sperms in video. Traditional background subtraction method is susceptible to environmental changes, for example, gradual or sudden illumination changes. By applying the non-linear diffusion algorithm, we determine a more proper threshold value and can obtain an accurate motion mask which isn't contaminated by noise.

Our results confirm that in general the detection algorithm identify progressive sperms head very well. However due to its frame differencing functionality, which identifies moving objects that differs significantly from the previous frame, this proposed algorithm is unable to detect non-progressive or dead sperms. So a more precise method must be made in dead sperm detection.

In the next step, it is proved that the performance of matching-based multiple sperm cells tracking algorithm is greatly improved in comparison with template matching, particle filter and mean shift methods. Our approach performs with comparable computational efficiency.

We demonstrate that the proposed approach can successfully handle the challenges such as cell collision, occlusion and missed detection. But our algorithm has not fully succeeded in multiple sperm tracking, where the spermatozoa concentration is so high. Of course, these are also occlusions that are hard to resolve even to the human eye. We will address this problem in future research studies.

BIOGRAPHIES

Yoones Imani received the B.S. degree in electronics engineering from Islamic Azad University, Najaf Abad, Iran, in 2008, and the M.S degrees in communication engineering from Isfahan University of Technology, Isfahan, Iran, in 2011. He has been a Researcher in Medical Image and Signal Processing Research Center, Isfahan University of Medical Sciences, Isfahan, Iran since 2010. His research interests include image processing, object tracking, pattern recognition, speech processing, and noise reduction.

E-mail: y.imani@ec.iut.ac.ir

Niloufar Teyfouri received the BS.c. degree from Amirkabir University of Technology, Tehran, Iran in 2002 and MS.c from the science and research branch islamic azad university, Tehran in 2005, both in biomedical engineering. Now she is a researcher at the Department of Advanced Medical Technology in Isfahan University of Medical Sciences. Her research interests includes biomedical image processing.

E-mail: bioniloofar@yahoo.com

Mohammad Reza Ahmadzadeh has received his BSc in the field of Electronic Engineering from Ferdowsi University of Mashhad, Holly Mashhad, Iran in 1989. He also received his MSc in Electronic Engineering from University of Tarbiat Modarres, Tehran in 1993. He did his Ph.D at the Centre for Vision, Speech, and Signal Processing, University of Surrey. Since 2004 he has a position at Isfahan University of Technology (IUT). He is interested in Signal Processing, Image Processing, Expert Systems, Information Fusion and Integration of Multisource Remote Sensing by Dempster-Shafer theory of Evidence.

E-mail: ahmadzadeh@cc.iut.ac.ir

Marzieh Golabbakhsh received the BS.c. degree from the Isfahan University of Technology, Isfahan, Iran in 2002 and MS.c from the University of Manitoba, MB, Canada in 2005, both in electrical engineering. In 2006 she was a Research Assistant at the McGill University and Nuance Communications. She has been with Medical Image and signal Processing Center at Isfahan University of Medical Sciences since 2010 as a Research Engineer. Her primary research interests involve biomedical signal processing, speech processing and machine learning.

E-mail: marzgol@gmail.com