Detection and segmentation of erythrocytes in blood smear images using a line operator and watershed algorithm.

Most of the erythrocyte related diseases are detectable by hematology images analysis. At the first step of this analysis, segmentation and detection of blood cells are inevitable. In this study, a novel method using a line operator and watershed algorithm is rendered for erythrocyte detection and segmentation in blood smear images, as well as reducing over-segmentation in watershed algorithm that is useful for segmentation of different types of blood cells having partial overlap. This method uses gray scale structure of blood cell, which is obtained by exertion of Euclidian distance transform on binary images. Applying this transform, the gray intensity of cell images gradually reduces from the center of cells to their margins. For detecting this intensity variation structure, a line operator measuring gray level variations along several directional line segments is applied. Line segments have maximum and minimum gray level variations has a special pattern that is applicable for detections of the central regions of cells. Intersection of these regions with the signs which are obtained by calculating of local maxima in the watershed algorithm was applied for cells' centers detection, as well as a reduction in over-segmentation of watershed algorithm. This method creates 1300 sign in segmentation of 1274 erythrocytes available in 25 blood smear images. Accuracy and sensitivity of the proposed method are equal to 95.9% and 97.99%, respectively. The results show the proposed method's capability in detection of erythrocytes in blood smear images.

INTRODUCTION

There are three kinds of floating particles in human being's blood: Erythrocyte, leukocyte, hematoblast. Erythrocytes are similar to some extent. There are several types of leukocyte, but five of them are common and in contrast with erythrocytes they have nucleus. Most of the diseases are diagnosable by the shape and size of erythrocyte, number of them in the blood sample and the ratio of the area between those of them which include oxygen and total area of the cell.[1]

Man's blood is studied by hematologists in lab. They observe blood smears under the microscope and do works like: Erythrocyte counting, leukocyte differential counting and abnormality detection of blood cells. Manipulating by man is time consuming and high risk. Therefore, many research efforts are considered for mechanization of this process.

Blood smear preparation does not run in just one way. There are various standards to do this, and it brings about images with different color. So, applicable algorithms, which use gray scale intensity form of colorful blood smear images may be robust.[234567] In this study, we also use the gray scale intensity form for processing.

First step in blood image processing as like as other images is segmentation. Accuracy of the next steps is highly dependent to this step. Many methods are rendered for blood image segmentation and they consider leukocyte segmentation in general. Some of them are automatic thresholding of image,[8910] use of the energy operator,[11] use of morphology,[12131415] use of neural network,[161718] fuzzy methods,[31920] formable models,[21222324] use of entropy,[25] watershed transform,[72627] mask design[28] and polar transform.[29]

Most of these methods cannot do well when there is a contact or overlap between erythrocytes, and then segmentation's accuracy would not be acceptable. Among the segmentation methods, watershed algorithm[30] could segment the blood cells with appropriate accuracy. To prevent over-segmentation in this method, image markers are employed. But, image marker's preparation in images with object overlapping like blood microscopic images is not easy. Distance transform for these images has a good result. To improve accuracy, points called local minima are considered as only agents of objects. These are start points of the watershed algorithm on image markers. Segmentation's accuracy in more overlapped images is related to the number of these start points as agent of each object.

In this paper, first we design a new mask to extract central areas of erythrocytes, and then the central areas are used as the start points to improve the performance of watershed algorithm. This paper is organized as follows:

In Section 2, we introduce the proposed method in subsections of pre-processing step, line operator, first-order derivative of Gaussian (FDOG) filter and watershed algorithm. Section 3 is devoted to discussion and experimental results. Finally, conclusion is given in Section 4.

MATERIALS AND METHODS

Pre-processing Step

To implement the distance transform, watershed algorithm and line operator after that, we need a binary image form of an original image in a way that the cells and background be differentiable. Inside of some erythrocytes, there is a bright spot. These spots are shown in Figure 1a. To obtain an appropriate marker image for the watershed algorithm, these spots should be removed. Although, to solve this problem a closing operator can be used on gray level image in Figure 1b directly, but in the resulted image cell margins are disappeared. To prevent this problem, firstly a mean filter on gray level image is exerted, and then two successive erosion operators with a circular constructive element, which has the same amount of appropriate diameter as the mentioned closing operator, are exerted to create Figure 1c. The diameter of this element is chosen to be about half of diameter of large bright spots. These bright spot diameters are estimated visually about half of large cell diameters in the images of this study. Since small size of this constructive element cannot remove the bright spots of erythrocytes, and large size of it attaches separate objects of the image which remove cell margins. Circular operator's diameter for images of this study is chosen to be 5 pixels, which is almost half of large bright spot diameters. Secondly, morphological reconstruction of image in Figure 1c under the musk of gray intensity image in Figure 1b is performed. As a result, the solved image of Figure 1d in which the cell margins are retained and bright inside spots removed is given.

Figure 1

(a) The original image. (b) The gray intensity image of the original image. (c) The erosion image of Figure 1b using circular structuring element in diameter of 5 pixels. (d) The morphological reconstructed image under the mask in Figure 1b. (e) The binary image of Figure 1d produced by global thresholding. (f) The complement distance transform of Figure 1e. (g) The binary image of Figure 1f using (3). (h) The peak image of Figure 1f obtained using (5). (i) The score image of Figure 1h. (j) The filtered image of Figure 1i with first-order derivative of Gaussian filter. (k) The binary image resulted from thersholding of the difference image between Figure 1i and Figure 1j using Otsu's method. (l) Local maxima of Figure 1f. (m) The multiplied image of Figure 1l and 1k. (n) The centers of bright spots in m on the original image with blue points. (o) The Watershed transforms of Figure 1f using markers in Figure 1m. (p) The watershed lines of Figure 1o are seen black in the original image

Line Operator

Line operator is applied in various kinds of images to detect linear structures. For example, Zwiggelaar et al. used it in mammography images.[31] Ricci and Perfetti used it to extract retinal blood vessels.[32] Lu and Lim devised a new line operator for detection of optic disc and macula in retinal images.[33] The optic disk structure in the retina images is such a way that its center is brighter than surroundings. They showed after applying bilateral filter to the retinal images the brightness of the optic disk is gradually reduced from its center to surrounding. Then, they proposed a line operator which can be used to locate structures with such brightness variations.

Figure 2a shows an image of the Euclidean distance transform[34] exerted on a binary image of a circle. This function is performed on binary images and calculates the Euclidean distance between each pixel and the nearest non-zero pixel to it. As it can be seen, the brightness of the circle is gradually reduced from its center to surrounding similar to the brightness of the optic disk in the retinal images. Therefore, after exertion of the distance transform on the binary images produced from the blood smear images, which have circular cell shapes, we can use the line operator as like as Lu and Lim method.[33]

Figure 2

(a) The Euclidean distance transform of a binary image of a circle. (b) The line operator with the length 21 and 20 different directional line segments. (c) The convolution mask that is valued 1 in the quadrants I and III and valued – 1 in the quadrants II and IV

This operator for each image pixel at (x, y) determines n line segments L, i = 1,…, n with length P (number of pixels on the line) and multiple definite directions that center at (x, y). Intensity of image's pixels located on these lines is shown by matrix I(x, y). In Figure 2b the line operator with 20 line segments in various directions and length P = 21 is shown. Each line L at one special direction is divided into two line segments L and L, with the same length . Image variation along each line is estimated with Eq. 1:

In this equation, f(.) shows a mean function. f(I(x, y)) and f(I(x, y)) show the mean image intensity in lines L and L, respectively. D = (D1(x, y),…, D(x, y),…, D(x, y)) saves the variations in n-directional lines having different directions. The direction of the line segments with maximum and minimum variation has a special pattern in surroundings of cell centers that can be used to locate them.

In the quadrants I and III of assumed circular erythrocyte in Figure 2a, the variation of image along line segments 1-10 (i.e. L in Figure 2a) reach the minimum, while along lines 11-20 (i.e. L in Figure 2a) reach the maximum. But this variation in quadrants II and IV, for lines 1-10 and 11-20 instead reach the maximum and minimum, respectively. So based on the direction of line segments, an image called “orientation map” is defined as Eq. 2:

where D(x, y) is image variation vector which is defined in Eq. 1. Orientation map can be converted to binary image according to the following equation:

In this equation, n is the number of the line segment directions, and Q(x, y) is the binary orientation map.

Figure 1e is created using global thresholding by Otsu's method[35] on the image of Figure 1d, and Figure 1f is produced using the complementary of distance transform exerted on Figure 1e. Figure 1g is the binary orientation map (Q) of Figure 1f which is created using.[3] In this figure, there are small shapes like the image of Figure 2c in which quadrants II and IV are dark. Because the line numbers with the maximum variations lie between 1 and whereas for the quadrants I and III lie between and n. Therefore, these quadrants are bright.

In this paper, we use a line operator with 20 line segments because more line segments have a low impact on the orientation map. The line length P is determined by:

In this equation, R is representative of the circular area's radius in Figure 1a. The parameter K controlling the length of line segments, which is usually between and , is related to the cell's diameter in the blood smear images. Variations in images’ dimension are possible by use of R.

In images of this study, we get . The special patterns of cells in the binary image are located using the mask shown in Figure 2c.

This mask called convolution mask consists of four quadrants where its pixels in the quadrants I and III have the value 1, and in quadrants II and IV have the value −1. The orientation map is converted to a peak image (p) according to Eq. 5

where (x0, y0) is the coordinate of the studied pixel. M(x, y) and O(x, y) show the values of the convolution mask and the orientation map at (x, y) respectively. m is the radius of convolution mask set as like as the line length P. Figure 1h shows the peak image produced using Eq. 5.

In the peak image, the cell centers are brighter than their surroundings. To locate these bright center areas, the peak image should be divided into two categories of concentric circular areas, bright central areas and dark surrounding areas. To do this, first brightness difference (Diff) of concentric circular areas in Figure 1f is evaluated according to Eq. 6.

where P is the peak image, and d is the distance between each pixel and its neighborhoods. R1 and R2, R2 = 2 × R1, determine the radius of an internal and external circle, respectively. N and N are the number of pixels in the internal and external circles, respectively. In this paper, R1 is set to where P is the length of the line segment. The values of brightness difference are positive in the central area of cells because their centers are brighter than their surroundings.

After that, the central bright areas are distinguished from the dark surrounding areas with synthesis of the brightness difference image and peak image as follows:

where P (x, y) symbolize the normalized peak image and symbol * symbolize dot product. The syntax (Diff (x, y) >0) sets zero all negative pixels of the difference image. In this way, central areas of cells get a high score and will be more distinguished.

Figure 1i shows the score image S(x, y) resulted from applying Eq. 7 to the peak image of Figure 1h, but these areas are not yet separated from the surrounding areas completely. It happens especially in the cell centers which have more overlapping.

FDOG Filter

The FDOG filter is used to eliminate full connection among the cells having more overlapping.[36]

This filter attenuates the central areas with low local variance but passes the surroundings with high local variance. This filter is applied in six different directions according to Eq. 8 and the maximum value of directions is assigned to the pixel value.

In this filter t is set to a constant value 3 because most of values of Gaussian filter is in [−3S 3S]. L is neighborhood length in direction y and is selected on the base of S. Whenever S is small (large) then L is so. In this study, we set S = 2 and L = 10 according to approximate the diameter of central bright areas. The score image in Figure 1i is passed through FDOG filter and Figure 1j is resulted. Then, we calculate the difference between input and output images of this filter and transform the resulted image to a binary image by the Otsu's thresholding method.[35] By applying the erosion operator with diameter 3 to the difference thresholded image, the image of Figure 1k in which the centers of cells are seen with white color spots is made.

Watershed Transform

The watershed algorithm was first put forward by Beucher and Lantuéjoul in the segmentation of grayscale images[37] and is used beside morphological tools as a powerful tool in complicated image segmentations. It is not easy to determine the watersheds in an image and many algorithms were proposed for this purpose, but most of them were time consuming and or did not have favorite results. In 1991, Vincent and Soille[38] introduced a fast, flexible and accurate algorithm for the watersheds determination. In this algorithm, the image is considered as earth's surface, and there are holes in low altitude places. Water comes up from underground, and valleys are filled with water. A dam is built wherever water of two different low altitudes get together. The algorithm is finished after filling up all area. These dams are watershed boundaries, which are also object boundaries in the image segmentations.

Mere use of the watershed algorithm in the image segmentation causes problem. Marking is used to solve this problem. In the blood smear images, the distance transform is used for the marking.[34] Local maxima in the image of Figure 1f are extracted and dilated with a constructive circular element with diameter 3 to get a better segmentation. However, as Figure 1l shows there are some pixels among overlapped cells, which have local maxima criteria in addition to cell centers, and they are selected as markers. We multiply binary image Figure 1k in the binary image Figure 1l to eliminate these additional local maxima. So, the markers in the cell centers remain, and the markers among the overlapped cells that cause the over-segmentation are eliminated. Figure 1m shows the result.

In this study, these markers are superimposed on complement image of Figure 1f as local minima points in the watershed algorithm to start this algorithm from them. The result of this algorithm is shown in Figure 1o in a binary form. In Figure 1p, watershed lines with dark color are shown on the origin colorful image.

With obtaining the watershed local minima markers by the proposed method, they can be used to detect erythrocyte centers automatically. In Figure 1n, these centers are shown in blue.

DISCUSSION AND EXPERIMENTAL RESULTS

In this study, liner operator is used to extract erythrocyte centers as well as to reduce the over-segmentation problem in the watershed algorithm. First, distance transform exerts on binary image of blood cells, and favorite structure of the brightness variation is created to be used in the line operator. Then, by applying line operator, the image of cell transformed into special circular pattern so that, in the quadrants I and III is bright and in the quadrants II and IV is dark. This produced pattern is correlated to the designed mask. After that, we try to create the local minima as a marker for the watershed algorithm. This is done by extraction of the local maxima in the image resulted from the distance transform. However, there is a marker in boundaries between over lapped cells and is resulted due to mere use of local maxima, the additional markers is eliminated by multiplying the binary marker image in the binary image of central areas of cells, which are located by mask designing. In Figure 3, the proposed method is shown on a new image.

Figure 3

(a) The original image. (b) The gray intensity form of theoriginal image. (c) The erosion image with circular structuring element in diameter of 5 pixels. (d) The morphological reconstructed image under the mask of Figure 3b. (e) The binary image of d using global thresholding. (f) The complement distance transform of Figure 3e. (g) The binary image of Figure 3f using equation 3. (h) The peak image from f obtained using equation 5. (i) The score image of Figure 3h. (j) The filtered image of Figure 3i using first-order derivative of Gaussian filter. (k) The binary image resulted from thersholding of difference image between Figure 3i and Figure 3j using Otsu's method. (l) The local maxima of Figure 3f. (m) The multiplied image of Figure 3l and Figure 3k. (n) The centers of bright spots in Figure 3m on the original image, shown by blue points. (o) The watershed transforms of Figure 3f using markers in Figure 3m. (p) The watershed lines of Figure 3o are seen black in the original image

Additional markers which cause the over-segmentation in the images of Figures 1a and 3a eliminated by the proposed method are shown in the first and second rows of the first column in Figure 4, respectively. The second and third columns show execution of the watershed algorithm for two Figures 1a and 3a with mere use of the local maxima as the regional minima. The green arrows show some removed over-segmentations by the proposed method. The proposed algorithm is performed on 25 images acquired by light microscope using a digital camera with magnification of 100. The resolution of images is 576 × 720.

Figure 4

The first column from left: The eliminated local maxima causing the over-segmentation. The second column: Watershed algorithm with mere use of local maxima as markers (without elimination of local maxima in the first column). The third column: The watershed lines of the binary images in the second column on the original images. Some of additional segmentations eliminated by the proposed method are shown by green arrows

There are 1274 erythrocytes all of them are completely located inside the images and 23 leucocytes having a partial over lapping with erythrocytes. 1300 markers are extracted for erythrocytes, but there are 1430 markers when mere use of the local maxima is considered. The evaluation of proposed method in terms of accuracy and sensitivity are given by:

where TP is true positives (number of erythrocytes each one of them just located with one marker), and TN is true negatives (number of images in their backgrounds, where there is not any blood cell, is not located any marker). FP is false positives (number of erythrocytes located with more than one marker and background images in which located a marker), and FN is false negatives (number of erythrocytes in which there is not any marker).

Table 1 shows these results. The accuracy and sensitivity for the proposed method are equal to 95.9% and 97.99%, respectively. These terms when mere use of local maxima was considered are 84.5% and 96.4%.

Table 1

Comparison between the proposed method and mere use of local maxima

Figure 5 shows the result for each image. The red squares show the number of watershed segmentation with mere use of the regional maxima as the markers. The blue squares show the watershed segmentation with elimination of additional markers illustrated in this paper. The green squares show the number of blood cells computed manually. The markers on the boundary of the image are not considered in this plot because the watershed algorithm creates additional markers for the blood cells cut on the image boundary. The proposed method has some limits. First, the extracted pattern by the line operator deteriorates in the overlapping cells. Second, the proposed method is slow. Third, some of the additional markers are not eliminated completely and these cause some over-segmentation problems.

Figure 5

Segmentation results of the proposed method: The red squares show the number of watershed segmentations with mere use of local maxima as markers. The blue squares show the number of watershed segmentations using the proposed method. The green squares show the number of blood cells with manual counting

CONCLUSION

In this study, an algorithm was introduced for over-segmentation reduction in the watershed algorithm in the microscopic images of blood cells and also for locating of them by the line operator. By use of this algorithm, 1300 markers are created as centers for 1274 erythrocytes in 25 blood smear images. It is a promising result to erythrocyte locating or counting, and also over-segmentation reduction using watershed algorithm to segment them.

BIOGRAPHIES

Hassan Khajehpour received the BS degree from Shahid Bahonar University, Kerman, Iran, in electrical engineering, in 2006 and the MS degree from the Isfahan University of Medical Sciences, Iran, in 2013, in biomedical engineering. His research interests are in biomedical image processing, Heart functional assessment, Computer vision.

E-mail: hassan.khajehpoor@yahoo.com

Alireza Mehri Dehnavi was born in Isfahan province at 1961. He had educated in Electronic Engineering at Isfahan University of Technology at 1988. He had finished Master of Engineering in Measurement and Instrumentation at Indian Institute of Technology Roorkee (IIT Roorkee) in India at 1992. He has finished his PhD in Medical Engineering at Liverpool University in UK at 1996. He currently is an Associate Professor of Medical Engineering at Medical Physics and Engineering Department in Medical School of Isfahan university of Medical Sciences. His research interests are medical optics, devices and signal processing.

E-mail: mehri@med.mui.ac.ir

Hossein Taghizad received the BS degree from Sahand University of Technology, Iran, in 2008 and the MS degree from the Isfahan University of Medical Sciences, Iran, in 2012, both in biomedical engineering. He is currently working toward the PhD at Memphis university, USA. His most research interests are in bioinformatics and biomedical image processing.

E-mail: htghizad@memphis.edu

Esmat Khajehpour obtained his BS in Computer Engineering from the Department of Computer & Electrical Engineering of Shahid Bahonar University, Kerman, Iran, and the MS in Medical Informatics from Tehran University of Medical Sciences, in 2012. His research interests are in automatic diagnosis using Fuzzy and Neural Network.

E-mail: E_khajehpoor@yahoo.com

Mohammadreza Naeemabadi received the B.S degrees from Islamic Azad University of Mashhad (IAUM) as top student in 2010 and immediately began his M.S education in Medical University of Isfahan (MUI), both in biomedical engineering. In 2006, his most research interest are tele health system and medical image processing.

E-mail: mr.naeemabadi@googlemail.com