Sperm detection in video frames of semen sample using morphology and effective ellipse detection method.

CASA (Computer assisted semen analysis) systems are designed to assist Andrologist labour. Most available CASA systems are not accurate or so expensive. Therefore labours use manual methods to provide parameters. Although some companies have achieved appropriate accuracy, they have not released their methods. So proposing methods in this area might be useful for groups who intend to design new CASA system. One of the parameters which these systems compute is sperm count. In this paper we introduce our algorithm which can count sperms with an acceptable accuracy. Sperm count or concentration is one determinant parameter in male fertility. Our program preprocesses the video frame or image of semen sample under the microscope recorded by camera, then use morphology and effective ellipse detection method techniques to segment sperms and then count appropriate sperms.

1. INTRODUCTION

According to WHO[1], one method to indicate fertility of a man is giving a semen sample of male and surveying it under the microscope. Some important parameters are sperm concentration, motility, size, shape and so on.

Under the microscope lens, from a few to hundreds of sperms might be seen each time. This much amount of sperms makes the doctor's job difficult[2]. Not only counting and tracking hundreds of sperms by eye is time consuming, but also visual problems and fatigue can affect negatively on result. Approximately from two decades ago CASA (Computer assisted semen analysis) systems are designed to help doctors in measurement of many important determinant parameters in male fertility. Some of them have reached to good computation accuracy. For instance, SQA-V system also known as the Sperm Quality Analyzer is used to be a high performance analyzer. It provides most of clinic parameters in less than two minutes[3]. The ISAS also known as PROISER[4] is one of the most complete and easiest to use systems in the market. A list of publications with this CASA can be found on[5]. Some other successful companies are Androexpert[6], Biophos[7] and Hamiltone Thorne[8]. However, respect to commercial interests, no company or group has proposed its method openly. In this paper, we proposed an algorithm to provide information of sperm concentration parameter. This algorithm counts number of sperms of semen sample in the video frames or images which are captured by a microscopic video camera. In the fertility laboratory microscopes provides the facility to attach a camera on it. The camera is connected to a computer and there are some software that show what is seen under the microscope and able to capture a video of it. In this work we use captured videos of semen sample under microscope with 20X zoom. Although working with this zoom have some difficulties. E.g. according to fig 4, firstly, tails are not transparent enough to be helpful. Secondly, zoom is not small enough not to show many undesired cells. But if algorithm is strong enough to work on 20X zoom frames, it will be useful in wide range of aspects of CASA systems. We work on a single frame of the video sequence. We firstly filter the noise of the image then apply some morphological operations on the relevant image to prepare it for next step. We apply an ellipse detection algorithm which is faster and more effective than Hough transform[9] to segment sperms[12]. Sum of the segmented sperms is what we want to find out as the sperm concentration.

Fig 4

Binary image after closing

2. NOISE REMOVAL

The detection algorithm is implemented in Matlab. Firstly the image which is a frame of captured video by the camera is read, read image is in RGB format but colored image is not useful but redundant in this application, so it is converted to the grayscale. Image is adjusted to increase contrast. The read image contains a disturbing noise. Fig. 1 shows a frame of the video sequence. In that figure, a white stripped pattern is observable. It is obvious that it has disturbed image and should be removed. Actually the noise which causes this pattern is called periodic noise. In this kind of noise, one or some high amplitude frequencies cause an undesired ordered pattern in image[10]. Using band reject filters is the best way to remove this rather constant frequency noise[10].

Fig 1

Part of captured frame. stripped white lines are caused by noise

In this work, we just want to remove this noise by computer programming and there is no intend to make a real filter. Accordingly, we use ideal band reject filter to remove the noise[11].

Fig. 2 shows the image after filtering in spatial domain.

Fig 2

Image frame after filtering

Next step is detecting sperms. However first of all, captured frame from semen contains sperms and some other cells such as bacteria, leukocyte, epithelial cells, etc[1]. Second of all, image is captured with 20X optical zoom, so tails of sperms are not observable perfectly and it can’t help. So, the algorithm should detect the head of sperms and be able to distinguish them from other cells.

In image, spermatozoa's head has three specifications: size, form, intensity.

It is rather impossible to distinguish sperms from other cells by considering just one of these specifications, because you may find other cells with the same size. In some cases, other cells may be same in form, since all sperms do not have exactly the same shape. Although some sperm's heads are so intense, there are some cases which they are as intense as other cells. However using combination of these characteristics we managed to introduce an accurate sperm detection algorithm.

3. DETECTION ALGORITHM

The noise removed grayscale image is converted to binary. The threshold should be exactly a little bit less than lowest intensity of available spermatozoa's heads and is determined according to the model of the camera. Each camera has its own least intensity and its threshold is fixed and determined by trial and error. So, the camera should be known for the software or its information should be given to the software. By converting the image to binary format, many cells which are not sperm and some sperms which are poorly visible in the focus level disappear. Fig. 3 shows the binary image of Fig. 2.

Fig 3

Binary image of figure 2

Closing is a morphological operation which can be used to delete shapes[11]. This method deletes the shapes which structuring element can’t be completely part of it. Accordingly, by choosing a structuring element smaller than spermatozoa's heads, we are able to delete smaller cells without changing sperm's shapes. Shape of the structuring element is very important. If the shape is not chosen correctly, it may affects on shape of remained heads. So, circle structuring element is a good choice. Fig. 4 is the image after closing

After deleting some other cells by converting the image to binary format and closing, we use shape characteristics to detect sperms and in uncertain situations, intensity or other characteristics participates. Although heads of spermatozoa appear in variety forms, all of them are approximately ellipse, thus, it is rational to use an ellipse detection algorithm to detect sperms. As mentioned above, sperms do not have same form and are not completely ellipse, so the ellipse detection algorithm should be fault tolerant. In[12] an ellipse detection method is proposed. This method takes the advantage of major axis of an ellipse to find ellipse parameters quickly and effectively. This method does not require the evaluation of the targets or curvatures of the edge contours which are generally very sensitive to noisy working conditions. It has no complicated mathematical computation and requires low memory space, so that is not time consuming. Our algorithm is based on this method. However, the method needed to be changed to be convenient to our problem. First, it just could detect shapes which were absolutely ellipse, but in our case no object is completely ellipse. This problem is solved by making the algorithm much fault tolerant. Actually, the algorithm accepts some deviation. Second, the method in[12] accepts the first detected ellipse but in this work all possible choices for each object are surveyed and the best one by considering a tradeoff between determinant parameters will be chosen.

The algorithm firstly finds the edges of each object, then labels them, and then sees if each object is a sperm head. Edged cells are shown in fig. 5.

Fig 5

Edged sells

Procedure of accepting each object as a sperm contains three steps:

Step one: compute area of each object

Step 2: select tow pixels of object as major axis of an ellipse.

Step 3: find an appropriate minor axis to have an ellipse looks like the object. If this minor axis found save it and go to step two to find a better corresponding ellipse else just go back to step two and choose another pair.

Algorithm firstly computes the area of each object. If the area has correct size, algorithm will to step two.

In step two, two pixels of object are selected. Algorithm checks if the line between two pixels could be a candidate for major axis of an ellipse like (x1, y1) and (x2, y2) in fig. 8. If the answer is no, another pair is chosen. If the answer is yes algorithm goes to step three.

Fig 8

adhered sperms. (a) Adhered sperms in real (left) and edged (right) frames. (b) red ellipses are detected. But they are wrong. They can’t be sperm heads. (c) correct sperm detection.

In step three, for all other edge pixels of object, the algorithm checks how long the minor axis should be to have this pixel on the ellipse, according to two selected points in previous step. To make this we first compute the parameters of the ellipse. According to[12], for each pair of pixels (x1, y1) and (x2, y2), we assumed they are two vertices on the major axis of an ellipse. By choosing the third edge pixel as (x, y) in fig. 8 we can calculate four parameters for the assumed ellipse as following:

x0=(x1+x2)/2      (1)

y0=(y1+y2)/2      (2)

a=[(x1-x2)2+(y1+y2)2]½/2      (3)

α=atan[(y2-y1)/(x2-x1)]      (4)

Where (x0, y0) is the center of the assumed ellipse, a is the half-length of the major axis and α is the orientation of the ellipse.

Fig. 6 shows ellipse geometry. f1 and f2 are foci of the ellipse and (x, y) is the third point used to calculate the fifth parameter. The distance between (x, y) and (x0, y0) should be less than the distance between (x1, y1) and (x0, y0) or between (x2, y2) and (x0, y0). So the half-length of the minor axis can be estimated by the following equation:

Fig 6

Ellipse geometry

b2= (a2d2sin2β) / (a2- d2 cos2β)      (5)

which is:

cos β= (a2+d2 –f2)/(2ad)      (6)

and d in the distance between (x, y) and (x0, y0).

b is the half length of minor axis.

After finding the minor axis for three selected points, without changing the major axis we chose another edge pixel as the third point and find its appropriate minor axis. After doing this for all other pixels, we know that, which pixels has approximately same minor axis size. In other words for each minor axis length how many object's pixels are on the ellipse. Appropriate minor axis should be in proportion to major axis and ellipse which is created according to this minor length should have enough intersections to the object. But just having enough intersections is not enough. For instance, in Fig. 7 an object is shown which in not sperm.

Fig 7

An ellipse does not correspond to a sperm. (a) the object which is not a sperm. (b)edge of the object after making the image binary. Pointers show the head of major axis of an ellipse is intersected in several places by object. The ellipse is shown in red. Although enough intersections are seen, the object is not ellipse.

If line between pointed pixels is major axis, by choosing an appropriate minor axis, we are able to make an ellipse which is intersected in several places by object, for instance, the red ellipse in Fig. 7, but most of them are in one side, so another misunderstanding may take place. Suppose that object is ellipse and is detected, but if algorithm just consider number of intersections, wrong axis will be chosen. Best answer may have fewer intersections. Our algorithm assumes that ellipse has two sides which are separated by major axis. To make correct decision, we should have enough intersections in both sides. It can’t be an appropriate minor axis if intersections in one side are much more than threshold, in another side are less.

At last, in this step, between possible choices for minor axis, the one which its ellipse has more intersections to the object in both sides, is chosen. Then algorithm goes back to step two, chooses another pair, and goes on. If the new ellipse has better proportion with area of the object, the previous ellipse parameters will be replaced by the new ones. After choosing all pairs in step two and applying the loop, if no ellipse is detected, the object will be rejected else object could be a sperm. If length of the major axis of detected ellipse is in proportion to area and intensity, the object will be accepted as a sperm.

Sometimes two sperms are adhered. Thus, after edge detection, there is an object with double area. An example is shown in Fig. 8.

To detect both sperms we apply algorithm on object and expect one of them to be detected. Then, just pixels of the detected sperm is deleted and after connecting two endpoints of remained part, we apply our algorithm once again to find out if it detects the second part as a sperm. One important issue in adhered sperms is, the best axis of ellipse for the first detected sperm is not necessarily appropriate for the second sperm. For instance, fig. 8b shows the wrong choosing of the first ellipse axis. Although the chosen major axis makes an ellipse which has lots of intersections to the object, it is not a right choice, because the chosen ellipse may disturb the second one. Actually, each object is recognized as one sperm which is not correct. To gain the best result, the algorithm checks all possible combinations of first ones and second ones. Fig. 8c is the correct choice. Each object is recognized as two sperms. Although, for the first detected sperm of the object the number of intersections of determined ellipse is less than the ellipse of the corresponding object in fig. 8b, the added number of intersections of both ellipses of each object is more.

We applied our algorithm on several frames. It worked more accurate than manual method and was not more time consuming of that. Fig. 10a is a complete frame. Variety of cells and sperms are available in it so it is a sample to evaluate the algorithm. Fig. 10b is the detected ellipses for which are correspond to sperms in fig. 10a. Objects which an ellipse is detected for them are considered as sperm head.

Fig 10

60% zoom sample image of 20X microscopic zoom. (a) Is a frame sample frame of semen. (b) Ellipses shows detected sperms.

4. EXPERIMENTS

We firstly chose 50 images of different semen samples which were counted by an operator normally. Although the operator is able to count without fault, he usually counts fast and not exactly, because he can get the needed information without spending much time. So, his results are not exact usually. Then these 50 video frames were examined by our algorithm and at last these frames were counted attentively and its results were considered as gold standard. Results are shown in Table 1. To scan our algorithm results we divided the sperms into two categories, single and adhered. Single sperms are not merged to any kind of cell but in adhered cases, two sperms or one sperm and a cell are overlapped or connected. In cases of Single sperm detection the algorithm works better than adhered cases. Actually about operator results, the accuracy of counting singles and adheres is not severable, since the operator just says a number by taking a look at the frame and we have no idea which sperms are counted. According to Fig. 9, in frames which have less than 40 sperms the operator counts more exactly because the frames are not crowded and operator is able to count without spending much time and counting some sperms more than twice. But in crowded cases which usually more than 40 sperms are in the frame, the algorithm works more accurate. One reason is, in crowded frames sperms are not well distributed and sometimes the operator confuse whether he have counted relevant sperm or not and he might count it twice or does not count it at all. The second reason is, exact counting takes time and needs patience which usually the operator doesn’t have them. Finally by comparing the average of the fault according to table 1, we aim to verify the acceptable accuracy of our algorithm. The Mean ± Standard deviation of fault for manual operator analysis is 6.21%±4.34 and for algorithm is 5.5%±3.83 which are close to each other. Another important factor in table 1 is false positive and negative detections. False positive cases are the cells which are considered as sperm by mistake and false negatives are the ones which are sperm but not detected. The result of false positive and false negative detection in single sperm cases shows that the algorithm is strongly powerful not to consider other kinds of cells as sperm but in adhered cases it is not that accurate. It sometimes considers some big sells as adhered cells which contain one or two sperms. However, accuracy of the algorithm is over 94 percents and could be useful in laboratory in some simple cases in order to reduce computation time and human cost. At last it should be mentioned in laboratories which CASAs are not used, the operator counts sperms under the microscope, so the number and place of sperms change regularly and the results are much more conjectural. Hence, the result of algorithm is not undependable compared with it.

Table 1

Experimental results base on 50 frames of different semen samples.

Fig 9

this chart illustrates the percentage of fault in counting for each frame. Horizontal axis shows the number of sperms of the frame according to the gold measure. Vertical axis shows the percentage of false detection for each frame. Results according to the algorithm are shown with the blue dots and for the operator are shown in red.tow lines shows the results with linear lines.

CONCLUSION

In this paper we proposed an algorithm to detect sperms in a video frame which is given by camera from semen sample. We tried to find sperm heads in semen according to characteristics such as their ellipse shape, intensity and area. Algorithm worked for all normal and most unusual sperms. It also, even was able to detect many adhere sperms.

In future work, we plan to increase the accuracy of method. Another possible work which is base on this, is tracking of sperms. In tracking, firstly sperms should be detected and then tracked. Another area which this method could be useful is working on morphology of sperm. Computing the deviation of pixels of each sperm from detected ellipse could be helpful to indicate the morphological category of spermatozoa.