Improving the Otsu method for MRA image vessel extraction via resampling and ensemble learning.

Accurate extraction of vessels plays an important role in assisting diagnosis, treatment, and surgical planning. The Otsu method has been used for extracting vessels in medical images. However, blood vessels in magnetic resonance angiography (MRA) image are considered as a sparse distribution. Pixels on vessels in MRA image are considered as an imbalanced data in classification of vessels and non-vessel tissues. To extract vessels accurately, a novel method using resampling technique and ensemble learning is proposed for solving the imbalanced classification problem. Each pixel is sampled multiple times through multiple local patches within the image. Then, vessel or non-vessel tissue is determined by the ensemble voting mechanism via a p-tile algorithm. Experimental results show that the proposed method is able to outperform the traditional Otsu method by extracting vessels in MRA images more accurately.

Introduction

Magnetic resonance angiography (MRA) [1, 2] is a primary choice for vascular imaging in clinical medicine. It uses the characteristics of magnetic resonance imaging (MRI) [3, 4] of blood flow to display blood vessels and blood flow signals. As a non-invasive technique, MRA is safer than computed tomographic angiography and digital subtraction angiography. Since it has no radiation damage, contrast agent response and complications are significantly reduced. For this reason, it has been widely used in clinical applications. MRA methods can be categorised into a time of flight (TOF), phase contrast (PC), and black blood techniques. Extracting blood vessels on MRA images is able to develop brain/cardiovascular imaging and related treatment techniques such as the guidance of intracranial stenting for preventing stroke [5]. Accurate segmentation of vascular structures is of great value for assisting diagnosis, treatment, and surgical planning. For example, the extraction of the middle axis of blood vessels in MRA brain images is the basis for quantitative analysis of blood vessels, three-dimensional (3D) reconstruction, and other related issues in the clinical practise of diagnosing and treating cerebrovascular diseases.

High flow velocity vessel is often a low signal on MRI. On the other hand, when radio-frequency (RF) pulses excite the imaging plane [6-8], high blood flow velocity results in a flow effect caused by temporal misalignment of the blood. The loss of the liquid signal depends on the pulse sequence, flow rate, and layer thickness. Furthermore, static tissue with multiple excitations is saturated and has low signal. Blood flowing signals from the outside of the plane into the plane may be stronger than static tissue signals. As a classical MRA technique, TOF [9] is based on the inflow enhancement effect of blood, which refers to the observation that unsaturated proton group (blood) flows into the imaging plane to form a high signal, while the surrounding stationary tissue is stimulated by multiple RF pulses with a low signal. Owing to the short pulse interval, the stationary tissue in the scan plane is repeatedly excited by RF pulses. Longitudinal magnetic moment cannot be fully relaxed and is saturated, so the signal is very weak. For intravascular blood flow, if the blood flow velocity is fast enough, the saturated protons excited in the imaging volume flow out of the scanning plane. Fully magnetised spins exist outside the imaging volume, known as unsaturated spins, flow into the scanning plane. Since the longitudinal magnetic moment is large, a strong signal is acquired, so signals inside and outside the blood vessel are distinguished to produce a contrast. For PC technique [1], the phase difference is proportional to the blood flow velocity, so the blood flow is acquired with the high signal, which makes a good contrast between blood flow and static tissue. The faster the blood flow, the stronger the signal of MRA blood flow is. The scan time of the PC is longer than the TOF method, but it can measure blood flow velocity and mark blood flow direction. As an advantage, the PC method is sensitive to extremely slow blood flow and can distinguish between vascular occlusion and extremely slow blood flow.

Accurate segmentation of MRA is the key to the visualisation of blood vessels, diagnosis, assessment, and treatment of diseases. The Otsu method has been widely used for extracting vessels on MRA images [10-13]. In the traditional Otsu method [14, 15], the greater the variance between the background and the target, the greater the difference between the two parts is. When the foreground is wrongly divided into the background or the background is misclassified, the difference between the two parts will be smaller. Therefore, vessel segmentation that maximises the variance between classes means that the probability of wrong segmentation is the smallest. The Otsu method is able to automatically threshold grey image to a binary image, and therefore extract foreground objects in an automatic process. For an image, in which foreground and background are clearly separated with obvious grey values, the Otsu method can segment foreground objects quickly and automatically without human interventions. However, since the Otsu method is a histogram-based technique, thresholding result may not be good if valley between bimodal peaks is not sharp. Furthermore, noise in the foreground and background of the image also corrupt thresholding results of the Otsu method. In MRA images, vessels can be considered as foreground and other non-vessel tissues are categorised as background.

Furthermore, vessel pixels on MRA images are generally considered as a sparse distribution [16]. They are rare samples compared with background pixels or other non-vessel tissue pixels. In pattern classification problems, there are also imbalanced classes with a few samples [17]. Although they are important in many applications, they cannot be correctly classified using traditional classification methods [18]. When traditional machine learning methods are used to solve these imbalanced classification problems, the performance of classifiers has degraded seriously; therefore, the resulting classifiers have a large degree of bias. The most common manifestation is that the classification correctness rate of rare samples is much lower than that of large classes. To combat the imbalanced class problem for sparse data, resampling technique has been used for detecting rare classes [19]. To convert unbalanced classification to a balanced classification problem, data sample frequency in rare classes can be increased by resampling and then be fed into the classifier. The main goal of balanced classification is increasing the frequency of a few class data samples or decreasing the frequency of class with more data samples. This is achieved by getting approximately a similar number of samples of the two classes.

To extract rare samples of vessel pixels from MRA images, we focus on improving the Otsu method for vessel extraction by using resampling technique and ensemble learning. Sparse vessel pixels are resampled with more instances to the relatively balanced class with background pixels. Although the combination of resampling technique and Otsu method has been proposed for natural image segmentation [20] or quantification of vessel cross-section structures [21], to the best of our knowledge, the combination technique has not been applied on MRA image with anatomical planes (e.g. axial, sagittal, or coronal planes). In this Letter, Section 1 introduces MRA segmentation and motivation of the proposed method. Section 2 will present the novel method. Results and discussion will be given in Sections 3 and 4. Section 5 will present the conclusion.

Method

The major framework of the proposed method is demonstrated in Fig. 1. It is seen that pixels in an MRA image are resampled and then fed into Otsu method for extracting local vessel objects in the local patches. The details of the vessel extraction are presented as follows. The circle data in Fig. 1 represents foreground vessel pixels and the triangle data denotes background pixels of other tissues.

Fig. 1

Framework of the proposed method

IXI dataset contains around 600 MR images acquired from healthy subjects. It is used in the proposed method for developing an algorithm and evaluating performance. All the MRA images are in the Neuroimaging Informatics Technology Initiative format and downloaded from the website [22]. Data were acquired from three different MR scanners including Philips 3T (repetition time (TR) = 16.7, echo time (TE) = 5.8, number of phase encoding steps = 286, acquisition matrix = 288 × 286, flip angle = 16.0), Philips 1.5 T (TR = 20, TE = 6.9, number of phase encoding steps = 286, flip angle = 25), and general electric (GE) 1.5 T systems. The 60 subjects’ data were used for extracting vessels. Image size is 512 px (the x-direction) by 512 px (the y-direction) by 100 slices (the z-direction). All axial, sagittal, and coronal planes are evaluated by the proposed method and compared with the traditional Otsu method.

For 3D volume data of MRA, 2D plane images are produced at first. After a 3D volume data is loaded, maximum intensity projection (MIP) [23] is applied on the 3D volume data along with axial, sagittal, and coronal directions. MIP utilises all the pixels in the volume data, which have the highest value along the line of sight. Since the imaging data comes from the acquired 3D volume data, the direction of projection can be changed arbitrarily. MIP technique can truly reflect the difference in tissue density and clearly show contrast agent-enhanced blood vessel morphology, direction, abnormal changes, and calcification of vessels. MIP encodes the maximum signal in the volume data on the trajectory projected along the viewing direction. It projects the high-intensity signal in the 3D space into one plane to generate a continuous blood vessel image. However, it is not possible to distinguish the 3D structure of the signal distribution; therefore, overlapping structures cannot be fully displayed. After MIP on a 3D volume data, vessel distribution on axial, sagittal, and coronal plane is displayed and three images are generated for the vessel extraction procedure.

For image resampling technique, the resampling process is essentially an image restoration process. Changing an image's size (width and height) is a very common operation. Almost every image processing software has this function. The image size modification is actually an image resampling process. The enlarged image is called upsamping and the downscaled image is called downsampling. Resampling changes the amount of image data as you zoom the picture. When downsampling (reducing the number of pixels), some information will be deleted from the image. When resampling upwards (increasing the number of pixels or increasing pixel sampling), new pixels will be added. An interpolation algorithm can be specified to determine how to add or remove pixels. Resampling technique reconstructs digital image to represent a 2D continuous function of the original image at first. Resampling technique then samples new pixel pitch and pixel position. On the basis of the reconstructed continuous function, the technique uses the value of the surrounding pixels to estimate and interpolate the value of the newly sampled pixels. It is equivalent to a 2D convolution operation with the sampling function on the input image [24]. This resampling process has presented successful application outcomes on natural images. However, medical image encodes important information related to medical physics. Any added, erased, or modified information may influence radiologist's diagnostics. For this reason, this type of convolution-based image resampling strategy modifies pixel information and may be not good at MRA image vessel segmentation.

In the proposed method, we resample the MRA image with multiple patches to detect foreground and background with the Otsu method. We propose a solution to enhance the image contrast and realise the idea of learning ensemble. For W and H as the width and height of the image I, we randomly select the centre pixel (X, Y) of the ith window as follows: where rand1(1, W) represents a random number between 1 and W and rand1(1, H) denotes a random number between 1 and H. X and Y are the coordinates of the centre point of the local ith window. W and H represent the width and height of the original MR brain image. Therefore, the centre pixel of the ith window may be any pixel in the image I randomly selected. After the centre of the ith, window is determined, in order to obtain the width and height of the ith window, a similar random selection strategy is used where if the ith window width or height is outside of the original MR brain image dimension, the local ith window is truncated to fit the original MR brain image size. Note that the random function rand2 is different from the previous rand1 since random numbers they generated are different at different times. The region of the ith window outside of the original MR brain image dimension is automatically discarded. The ith window is equivalent to be a local patch of the original MR brain image.

On the basis of the centres (X, Y) of the ith window, the sub-image is extracted from the original MR brain image. An Otsu method is applied on the locally extracted patch of MRA image and then searches optimal threshold for minimising the variance within the class of vessel pixels and the class of non-vessel tissue pixels. The searching process is presented as follows: where and are probabilities of the class of vessel pixels and the class of non-vessel tissue pixels. Two classes are separated by the optimal threshold t. Therefore, each pixel on locally extracted patches can be grouped into vessel pixel class and non-vessel tissue pixel class.

The ensemble method means that the meta-classifier has better generalisation performance than a single classifier included by combining different classifiers into one meta-classifier. The commonly used integration method is the principle of majority voting. The majority voting principle refers to the prediction results of most classifiers as the final prediction category. That is, the results with more than 50% of the votes are used as landmarks. Strictly speaking, majority voting is only used in the case of two classifications. However, we can also generalise the principle of majority voting to multi-category, which is called the simple majority voting. Therefore, the key to ensemble learning is two-fold: (i) how to build classifiers with differences and (ii) how to integrate the results of these classifiers.

After Otsu method operations on a sequence of local patches, each pixel has been counted as vessel pixels or not. The global counting number can be calculated as follows: where p represents a pixel in the image and there are L patches in total. If the counted numbers of a pixel as a foreground pixel is larger, this pixel has a higher probability to be true vessel pixel in the original image. A global threshold is selected to determine the final segmentation image. If a pixel value is greater than the global threshold, the pixel is grouped as vessel pixel in the final image. Otherwise, it is considered as a non-vessel tissue pixel.

The global threshold is semi-automatically selected by using p-tile thresholding algorithm in the proposed method. The p-tile algorithm [25] is an automatic threshold selection algorithm based on grey histogram statistics. The algorithm needs to be based on certain prior conditions – the area ratio P% between the background and the target. The principle of the algorithm to select the threshold value and accumulate the grey-scale histograms in order until the accumulated value is greater than or equal to the area occupied by the foreground image (target). The grey scale at this time is the required threshold. Since each pixel will generate a global counting number after a sequence of Otsu operations on local patches, the p-tile algorithm will directly use global counting numbers for building a histogram. This is different from the traditional application of p-tile algorithm on grey-scale image segmentation, which utilises grey levels for building a grey histogram. According to the distribution of vessel pixels in MRA image, the p values are empirically set as 0.95–0.99.

Results

The proposed method is tested and evaluated on axial, sagittal, or coronal planes of MRA images of IXI dataset. The image generated by the MIP algorithm is used as the reference image since all blood vessels are presented without any post-processing steps. As shown in Fig. 2a, MIP algorithm is performed on an MRA dataset with an axial plane and a reference image is displayed. After a sequence of Otsu method operations, a global counting number map is produced based on (4) and displayed as Fig. 2b. The traditional Otsu method is applied to the image generated by the MIP algorithm. As shown in Fig. 2c, some vessels are disconnected and details of thin vessels are lost. For the proposed method, vessel details can be retained and there are no obvious disconnected segments of vessels as shown in Fig. 2d. From comparison among Figs. 2e–g, it is seen that the proposed method Fig. 2g is able to retain connected vessels as presented in the reference patch Fig. 2e. However, the traditional Otsu method Fig. 2f cannot identify vessels accurately, which are disconnected by the wrong detection.

Fig. 2

Vessel extraction of the first axial brain image

a Evaluated by the reference image

b Global counting number map is also presented for performance comparison

c Traditional Otsu method

d Proposed method

e Patch extracted from the window in (a) is presented as a reference

f, g Corresponding patches extracted from the traditional Otsu method (c) and the proposed method (d)

Vessel extraction of the second axial brain image demonstrates similar results as shown in Fig. 3. The proposed method (Fig. 3d) is able to extract accurate vessels compared with the traditional Otsu method (Fig. 3c). Similar to Figs. 2e–g, the proposed method outperforms the traditional Otsu method by retaining details of vessels in image patches as shown in Figs. 3e–g. On the basis of extracted patches of Figs. 2e–g and 3e–g, ground-truth vessel segments are extracted from Figs. 2e and 3e for quantitative evaluation of the proposed method. The numbers of pixels with overlaps between Figs. 2e and f, Figs. 2e and g, Figs. 3e and f, and Figs. 3e and g are calculated. Accuracies of vessel segmentation of the traditional Otsu method and the proposed method are 35.01 and 79.02% for the patch in Fig. 2, respectively. Furthermore, accuracies are 44.5 and 75.69% for the traditional Otsu method and the proposed method in the patch of Fig. 3, respectively.

Fig. 3

Vessel extraction of the second axial brain image

a Evaluated by the reference image

b Global counting number map is also presented for performance comparison

c Traditional Otsu method

d Proposed method

e. Patch extracted from the window in (a) is presented as a reference

f, g Corresponding patches extracted from the traditional Otsu method (c) and the proposed method (d)

For sagittal and coronal brain images, vessel extraction is also processed by the proposed method and the traditional Otsu method. As shown in Figs. 4d and 5d, the proposed method can extract vessels more accurately than those of the traditional Otsu method. The Otsu method wrongly detects and extracts vessels without complete vessel segments. In comparison to the reference images, the proposed method can extract vessels accurately.

Fig. 4

Vessel extraction of the first sagittal brain image

a Evaluated by the reference image

b Global counting number map is also presented for performance comparison

c Traditional Otsu method

d Proposed method

Fig. 5

Vessel extraction of the first coronal brain image

a Evaluated by the reference image

b Global counting number map is also presented for performance comparison

c Traditional Otsu method

d Proposed method

For demonstrating the procedure of the proposed method, Fig. 6 shows the middle results of randomly selected windows. It is seen that each window has different sizes and the traditional Otsu thresholding method is applied to each window. Each window can be considered as a different view to identify vessels and then learning ensemble is used for detecting the most accurate vessel from multiple views.

Fig. 6

Vessel extraction on local windows, whose sizes and positions are randomly selected. Different viewpoints on different windows provide a diversity of thresholding results and then extract the whole vessels from learning ensemble

a Local window

b Local window

c Local window

Discussion

Otsu method is a thresholding method for segmentation of blood vessels in a medical image. It is assumed that the target and background of a medical image occupy different grey-scale ranges. The difference in grey values between the target and the adjacent pixels within the background may be small. Pixels on both sides may have large differences in grey values. If an appropriate grey threshold T is selected and then the grey value of each pixel in the medical image is compared with the threshold T, pixels can be divided into two categories: vessel pixels and non-vessel tissue pixels. Segmentation is usually based on prior knowledge. However, the number of vessel pixels is much fewer than non-vessel tissue pixels in MRA images. Experimental results show that sparse pixels of blood vessels can be resampled to increase rare class samples for better thresholding-based vessel extraction in MRA images.

Although the resampling technique has achieved good results on some datasets, there are also some drawbacks to this method. The resampling method does not add any new data. It just repeats some samples or adds some artificially generated rare class samples. It is over learning [26] if resampling duplicates some rare class samples, causing the classifier to focus too much on these repeated samples. Resampling cannot completely solve the scarcity of rare class samples and the insufficiency of data representation, so its performance may be degraded. In addition, the proposed method requires ensemble learning on multiple segmentation results through a voting process. Therefore, more computational costs and time are needed in comparison with the traditional Otsu method. When the number of candidate segmentation results becomes large, the proposed method may be unable to meet real-time calculation. This may limit the application of the proposed method.

Since vessel extraction is an important task for analysing vessel information and diagnosing vessel-related diseases, accuracy should be improved further in the future work. Owing to different parameters of data acquisition in MRA, vessel images generally vary from different contrasts. However, contrast causes Otsu method-based thresholding results unstable. For this reason, improving the proposed method on ultra-low contrast MRA vessel images will be studied in the future.

Conclusion

In conclusion, a novel vessel extraction method of MRA images is proposed. Owing to sparse distribution of vessel pixels in MRA image, a resampling technique is used for upsampling pixels in multiple local patches of MRA image. Therefore, imbalanced classification problem is alleviated. An ensemble voting is applied on thresholding vessels on multiple local patches to extract the final vessels. Experimental results demonstrate that the proposed method outperforms the traditional Otsu method in extracting vessels of MRA images. Refined ensemble voting will be studied in the future work.