Automatic and fast segmentation of breast region-of-interest (ROI) and density in MRIs.

Accurate segmentation of the breast region of interest (BROI) and breast density (BD) is a significant challenge during the analysis of breast MR images. Most of the existing methods for breast segmentation are semi-automatic and limited in their ability to achieve accurate results. This is because of difficulties in removing landmarks from noisy magnetic resonance images (MRI) due to similar intensity levels and the close connection to BROI. This study proposes an innovative, fully automatic and fast segmentation approach to identify and remove landmarks such as the heart and pectoral muscles. The BROI segmentation is carried out with a framework consisting of three major steps. Firstly, we use adaptive wiener filtering and k-means clustering to minimize the influence of noises, preserve edges and remove unwanted artefacts. The second step systematically excludes the heart area by utilizing active contour based level sets where initial contour points are determined by the maximum entropy thresholding and convolution method. Finally, a pectoral muscle is removed by using morphological operations and local adaptive thresholding on MR images. Prior to the elimination of the pectoral muscle, the MR image is sub divided into three sections: left, right, and central based on the geometrical information. Subsequently, a BD segmentation is achieved with 4 level fuzzy c-means (FCM) thresholding on the denoised BROI segmentation. The proposed method is validated using the 1350 breast images from 15 female subjects. The pixel-based quantitative analysis showed excellent segmentation results when compared with manually drawn BROI and BD. Furthermore, the presented results in terms of evaluation matrices: Acc, Sp, AUC, MR, P, Se and DSC demonstrate the high quality of segmentations using the proposed method. The average computational time for the segmentation of BROI and BD is 1 minute and 50 seconds.

Introduction

Breast cancer is a major cause of death in women [1]. It is reported that, in a lifetime of women worldwide, one in eight will develop breast cancer [2], [3]. Also, the reported statistics reveal that over 2 million women are suffering from breast cancer in the US alone [4]. To reduce the mortality rate from breast cancer, early diagnosis and treatments are essential [5]. MRI is a well-established imaging technique to identify and mitigate breast diseases by generating a series of 3D images that a radiologist uses to manually detect the diseased part and identify problems [6], [7]. The manual process is time consuming because of the high number of images [8], [9]. Hence, automatic computer-algorithm based image analysis has become essential for performing computer-aided detection and diagnosis, which aim to provide prompt output and help radiologist to accurately locate the diseased area.

The main objective of this study is to segment the breast region of interest (BROI) and breast density (BD) from breast MRIs. First, BROI segmentation can serve as the fundamental step for avoiding irrelevant structures such as unwanted background and organs like the heart, liver, and chest, improving efficiency and accuracy during further analysis like tumor segmentation [10]. Tumor segmentation in breast MRIs is considered to be a laborious and error-prone procedure. Also, tumors normally reside inside the BROI. Therefore, prior to tumor segmentation, it is essential to identify the BROI [11]. BROI segmentation is also useful for applications such as BD measurement [12] and performance improvement of DCE-MRI in terms of pharmacokinetics-model calibration (PMC) [12], [13]. During PMC, the properties of the interior chest wall should be determined, which requires pectoral muscle segmentation [14]. Second, the ratio of BD can be considered as a strong indicator for the estimation of breast cancer risk. Also, breast tissue pattern asymmetry in left and right breast is considered to be an abnormal biological process that leads to cancer [15]. BD does not have a distinct shape and pattern and may be found anywhere within the image. Moreover, intensity inhomogeneity is a common problem within breast MRIs since the bias field adds more challenges by producing similar intensity around the BROI.

For BROI segmentation, different techniques have been reported in the literature. Ertas et al. [16], performed morphological operation and intensity thresholding for the segmentation. However, the results are better when the chest wall has high contrast. Several other methods such as intensity histogram [17], [18], wavelet analysis [32], [19] and active contour [20], [21], fuzzy c-means [22], [23], region growing [24], [25] are proposed. The performance of these methods rely on the contrast between the border regions and can fail in the cases that have similar intensity distribution. A fully automated method reported by Wang et al. [29] extracts breast area on non-fat-suppressed MRI images. The author explained that the properties of pectoral muscle and the breast-air boundaries are similar in 3D and exhibit smooth sheet like surfaces and use a Hessian-based filter to suppress the lower contrast and non-specific shapes. However, this method does not include breast density segmentation and may not produce an accurate mask. Khalvati et al. [26] reported a multi-atlas segmentation algorithm that creates a breast atlas with the help of phase congruency. This segmentation process is reliant upon the shape and intensity based registration prior to the segmentation. Gubern-Merida et al. [33] proposed a probabilistic atlas based approach for breast segmentation. However, the accuracy depends upon the size and variability of the database and requires an atlas that is representative of the population, which is computationally expensive. An edge based approach was proposed by [28] that is independent from the visible contrast between the breast ROI and chest wall. This method calculates cost function using edge information obtained from tunable Gabor filter. The precision of this method depends upon the information from the adjacent slices and accurate initial-border determination. Despite the advancement in BROI segmentation, fully automatic, accurate and fast segmentation of BROI still require much attention. This is because: 1. MR breast imaging contains breast structures in different shapes and no clear boundary of breast landmarks, which requires manual correction [34]. 2. There is a bilateral asymmetry between left and right breast regions requiring separate analysis [35]. 3. The pectoral muscle is closely attached and possesses similar intensity to the BROI which gives false positives and requires manual corrections [36]. 4. It is observed from the literature that several existing methods are supervised and require prior information before the segmentation process which results in computational complexity [37]. For BD segmentation, we note that there is a significant range of studies carried out in a semi-automated segmentation using an interactive thresholding method [43], [44] and user-assisted clustering methods [45]. These non-automated methods are subjective and create inter- and intra-reader variability [46]. It can be time-consuming, and therefore unsuitable for processing larger databases. To cope with above-mentioned difficulties, some attempts on automated methods have been studied such as adaptive thresholding [47], atlas-based method [48], Gaussian distribution curve fitting [49], hierarchical support vector machine [50], and Otsu thresholding algorithm [51]. It is observed that supervised methods provide more accurate results, but require a complex and costly labelling and analysis by expert radiologists prior to segmentation. Hence, for automatic segmentation of BD, unsupervised methods produce efficient results. Also, the most effective way to minimize computation time is to reduce the number of pixels processed. To overcome these problems, we propose a novel framework which is fully automatic, unsupervised, fast and efficient. The proposed model is divided into two steps: 1. BROI segmentation 2. BD segmentation.

During BROI segmentation, first, we aim to de-noise the MR image and precisely remove air-background using pixelwise adaptive wiener filtering (PAWF) technique [38], [39] and k-means clustering [40]. PAWF technique can preserve the edges and high frequency parts of an image unlike a normal filtering technique and k-means clustering will automatically cluster the whole image on different group, based on the correlation of pixels intensity. Second, the heart area, a brighter part of image is segmented using active contour level-set method [41]. The novelty in this method involves the calculation of initial contour by using maximum entropy thresholding and convolution technique [63], [65], which provides accurate segmentation and reduction of computation speed. Third, the segmentation of the pectoral muscle is performed. The orientation of the pectoral muscle and breast density of the left, right, and central area of the breast are different. Hence, we apply a morphological operation on a different orientation to enhance the gap between the pectoral muscle and breast density. The resultant image is binarized using an adaptive thresholding technique to exclude the pectoral muscle. Finally, we use polynomial curve fitting [42] to smoothen the acquired BROI segmentation. During BD segmentation, initially, we de-noise the result image from BROI segmentation. It is observed from the experiment that, the volume and intensity of BD in left and right breasts are different, hence, the single threshold value could not provide accurate segmentation. We divide the BROI segmentation image according to its geometrical information and calculate a different threshold value for left and right breast using four level FCM [52]. This study calculates FCM within the BROI rather than on an entire image.

Rest of the paper is organized as follows. A detailed methodology of the segmentations (BROI and BD) is reported in Section 2 which includes the explanation of eliminating unwanted landmarks. In section 3, the experimental results are analyzed and discussed. Finally, a concluding remark drawn in this study is given in Section 4.

Methods

The general workflow for BROI and BD segmentations is illustrated in Figure 1. Each step is successively explained in the following sections.

Figure 1

General workflow of segmentation procedure to extract BROI and BD.

Segmentation Methodology of BROI

Pre-processing Step

The process begins with rescaling the image to the fixed size so that each image in the different databases posses similar properties. The rescaled image dimension equals 328 on the row whereas the columns are calculated accordingly, to preserve the aspect ratio. The original breast MRIs are fundamentally corrupted by random noise from the image acquisition process that leads to uncertainties during the measurement of any quantitative biomarker [53]. Hence, pre-processing the rescaled image is an important step for removing undesirable noise such as additive white Gaussian noise (AWGN) and irrelevant details that affect the BROI segmentation. Gaussian filtering has been thoroughly studied for noise suppression and smoothing [54]. This process blurs an image with Gaussian function and involves a convolution mask where pixel values are modified according to the neighboring pixels. However, the Gaussian filter is not always suitable for denoising since it also removes high-frequency signal components leaving a blurred edge or boarder as shown in Figure 2(a) [55]. The edge preserving denoising technique should be adopted as the edges are the important features during segmentation. Hence, we apply a pixelwise adaptive wiener filtering technique that effectively removes the noise while preserving the edges [56], [57]. The denoised result image obtained from Gaussian filter has blurred edges. However, the result from pixelwise adaptive wiener filtering technique show that sharp edges are preserved as shown by the red arrowhead in Figure 2(a) and (b).

Figure 2

Illustration of filtering technique in regards to the accurate edge preservation (a) Gaussian filter (b) pixelwise adaptive wiener filter.

Let us consider the pixel position and the window mask of around its neighborhood. We conducted experiments to see the effect of different window size in the MR images. We found out that the use of larger window size clears the noise but destroys the useful edge information. On the other hand, the use of smaller window size are not capable of clearing the noise from the image. Based on our experiment, we fixed the window size () as 10x10. The value presented is suitable for the database we have used. However, it can be slightly optimized to suit the other database of MR image.

The pixelwise adaptive wiener filter is given by Eq. (1) [58]: where, and is the local mean and variance. is the average value of across noisy image i.e. . The computation of local mean and and variance is provided Eq. (2): where X and Y are the horizontal and vertical arrays of pixels in the window mask.

The color distribution of the denoised MRI image is studied using K-means clustering method as presented in Figure 3. This method follows two steps that divide a set of data into k number of clusters. Initially, k centroid is calculated and the data point is allocated to the cluster as the nearest centroid from the particular data point. The distance between the centroid and the data point is calculated with the Euclidean distance. Once the data point is clustered, a new centroid is recalculated and the procedure is repeated until convergence has been reached [59]. We clustered an image into 10 different colors which is sufficient to observe the level of detail of landmarks and their color distribution. In Figure 3(a), blue color in the color bar signifies the darkest and the red color represents the brightest intensity area. Let us consider a set of n data points as and k cluster has centroid as . The number of clusters in our case is 10. Initially, we select random centroid points and assign elements to the cluster as presented in the equation (3). Now, update the center of cluster and repeat until the centroid converges using Eq. (3). During the experiment conducted on 15 MR breast scans, we discovered that 1st and 2nd cluster always represent air background and partial lung area. These clusters do not possess useful information and can be eliminated. Moreover, clusters 3 to 10 characterize BROI, pectoral muscle, heart, some region of lung and BD. This means that most of the useful information can be represented above cluster 3 and is preserved as shown in Figure 3(b). Note that we have conducted experiments with different cluster numbers and is empirically set as 10.

Figure 3

Illustration of (a) Color distribution of MRI image using 10 clusters of the K-means clustering algorithm. (b) Resultant binarised image after removing noise and first two layers.

Heart area segmentation

In the MR images, heart area, the central part of the image appears to have the brightest intensity in the image and differs in shape and size. It is observed that some images have low contrast and close boundaries which creates complication in the segmentation process. The elimination of this area is vital for the accurate segmentation of the BROI. We obtained 10 clusters in the previous section. From our experiment, we noticed that the last four clusters are the brightest clusters in terms of color intensity and cover heart area and gradually spread towards the pectoral muscle and BROI as shown in Figure 4(a), (b). Hence, we combine these clusters and use the active contour level set method [41] to segment heart area. The active contour model uses level set method to evolve its initial contour. The detection of boundaries rely on the mumford–Shah segmentation technique [60] for the evolution process of contour. Hence, the objects with discontinuous and undefined boundaries can be detected with this model. The process begins by detecting an initial contour point for the evolution process. The initial contour point is detected using maximum entropy thresholding and convolution method [63], [65]. Initially, a preprocessed image is binarized with maximum entropy thresholding. A convolution process is carried out between the binarized image and a square window of 50 pixel × 50 pixel. Note that, we fixed our window size from several experiments. The convolution between image and the mask image is given by Eq. (4): where is the dimension of the image to be convolved and is the dimension of mask image. is the coefficient of mask image at position . The centre of the window point provides a weighted sum of each pixel in the binarized image. The pixel that gives the highest weighted sum as an initial contour point is considered. We draw a circle (initial contour, C) from the initial contour points. Let us consider two forces of the initial contour C, be and . is the force to shrink the contour and is the force to expand the contour. These two forces are balanced when they reach the desirable boundary of the interested object. The minimal partition problem used to minimize an energy is represented in Eq. (5): In this work, the initial contour is located around the mid section of the heart area as shown in Figure 4(b). Moreover, is always zero and is greater than 0, hence, we always expand initial contour (C). When the initial contour C reaches the equilibrium, and becomes zero, and segmentation is achieved. The iteration process is controlled by level set formulation as shown in Eq. (6). Where is the heaviside function and is the input image. To obtain the minimum of F, s derivatives is found and set to zeros and and and ϕ is updated in Euler–Lagrange equation as shown in Eq. (7): where is the Dirac function. The experiment shows that, the heart area normally resides within this circular radius of 80 pixels from the initial contour points. Hence, we permit the evolution process only on the circular area of 80 pixel radius from the initial contour point and will stop automatically. This process improves accuracy and saves computational time. The final segmented heart area is represented by blue as shown in Figure 4(c).

Figure 4

Elimination process of heart area. a) 10 clustered MRI image b) Four brightest cluster c) Final segmented heart area.

Pectoral muscle segmentation

The other important step is identifying the pectoral muscle. This step is vital because pectoral muscle and BROI shares a similar pixel intensity especially in the presence of dense BD [61], [62], making segmentation difficult and inaccurate. Hence, we include shape and geometrical information of the pectoral muscle and BD in MRIs obtained using the several experiments. The pectoral muscle is attached just below the BROI and above the lung and heart region ans spreads towards the bottom left and right corner as shown in Figure 5(a). Also, the BD is found to be thick in the BROI region and gradually becomes narrower and ends at the left and right corner. In the narrow section, the pectoral muscle and BD are closely connected. However, there is a small space between these two regions in MR images. We use a morphological opening operation to make this gap smooth and clear. Moreover, we use local adaptive thresholding for the binarization of the resultant image obtained from morphological opening operation. Finally, the greatest area from the connected-component labeling is selected as a pectoral muscle.

Figure 5

Identification of pectoral muscle area using a morphological operation. (a) MRI image. (b) A 2D image slice is divided into three different areas: 1) left 2) right and 3) central breast area. Each area is processed with a morphological operation in different orientation that develops a response image. These response images are merged to produce a single image.

Figure 5(b) depicts the model for producing a response image in each orientation. It is observed that the angle of inclination of left breast tissue near the pectoral muscle varies between 180 to 270 degrees and right breast tissue near the pectoral muscle varies between 270 to 360 degrees as demonstrated by the green lines in Figure 5(b) respectively. Similarly, the breast tissue inclination in the central area is 0 degree. To generate a response image using morphological opening operation in different orientation, we divide a denoised image obtained after eliminating the heart area into 1. Left breast area 2. Right breast area and 3. Central breast area using shape and geometrical information. Hence, a five response image on the left and right breast areas with 15 degree increment in angle of inclination and 1 response image on central breast area with 0 degree orientation is generated. The separate response images are generated using morphological opening operation with the structuring elements in terms of lines in different orientations and the response images are merged together. The morphological opening operation is found to be very effective in smoothing the space between the breast tissue and pectoral muscle. This operation eliminates the objects that are smaller than the line structuring element with the scale of 5 pixel in different orientations to reconstruct the remaining shape of the objects. Let be the denoised image after eliminating the heart area. Morphological opening operation performs both erosion and dilation using the same structuring elements on the image and satisfies Eq. (8): where indicates a line shaped structuring element with the scale of 5 pixel in different orientations, is the set of and ∪ denotes union of set. The response image is given by geometric interpretation where unions of all translations of structuring elements fit the entire image . The important thing to noticed here is that the brightest feature smaller than the scale of line structuring elements in their respective orientation is greatly reduced in terms of intensity. Also, it eliminates small specularities and textural fluctuations.

Figure 6 show the images, before and after the morphological opening operation in the left and right breast areas. Figure 6 I ((a) and (b)) are the original image of left and right breast areas respectively. After the morphological opening operation, the gap between the pectoral muscle and BD is clear and smooth as shown in Figure 6 II ((a) and (b)) on the left and right breast areas. After the morphological opening operation, an adaptive local thresholding is used separately in three different areas to segment the pectoral muscle. In the global thresholding approach, a single thresholding value is produced for a whole image based on the global characteristics of the image. In contrast, adaptive local thresholding calculates a local thresholding value based on the characteristics of the window around the pixel, i.e. it changes the threshold value dynamically in the image. Since the calculation of a local threshold based on the histogram is computationally expensive, we have chosen a local threshold value calculated using the statistical parameter, mean and local intensity distribution. Adaptive local thresholding typically takes a grayscale input image and produces a binary image as an output as shown in Eq. (9) which is dependent upon the window size. is the binarized image, . The threshold value is achieved using sauvola's technique. This technique uses mean, and standard deviation to calculate the threshold value of each pixel within a defined window size as shown in the Eq. (10): where R is the maximum value of standard deviation and fixed as 128 for the grayscale image. k is the bias and takes the positive value between . Since the algorithm is not very sensitive of k, we calculate the threshold value without involving k as shown in Eq. (11): The smaller window size is found to be more sensitive to noise and generates unusable white pixels around the image. Increasing the window size will produce a denoised and clear image and tends to merge the edges. Hence it is necessary to select the right window size of our requirement for the segmentation of the pectoral muscle. We have conducted an experiment to find the right window size as shown in Figure 7. The first and second row in Figure 7 shows a resulting image from adaptive thresholding using different window size on left and right breast respectively. A window size of 5, 10, 20, 30, and 50 pixels are considered, which is depicted in Figure 7 I ((a), (b), (c), (d), and (e)) on left breast and Figure 7 II ((a), (b), (c), (d), and (e)) on right breast respectively. It is observed that a window size of 20 pixels is found to be suitable to produce an accurate results.

Figure 6

Illustration of morphological opening operation to obtain a separation between pectoral muscle and breast tissues. (I) The original image of left and right breast area (II) The resultant image after using morphological opening operation on left and right breast area.

Figure 7

Response of different window size during adaptive thresholding. Resulting image obtained from adaptive thresholding using different window size (5, 10, 20, 30, and 50) respectively I) on left breast area II) on right breast area.

A resultant image from left, right and central part of the image are merged together to produce a single image. To segment the pectoral muscle, we remove the area above the central point which is highlighted in blue as shown in Figure 8(a). Note that the central point is already detected in the previous steps. Figure 8(b) shows the remaining part after the upper region is removed. Finally, a pectoral muscle is segmented by extracting the greatest area using the connected-component labeling as shown in Figure 8(c).

Figure 8

Illustration of extraction of pectoral muscle after local adaptive thresholding. (a) Resultant image of local adaptive thresholding. The blue in upper part represents the area above the central point of breast image and lower part represents the heart area (b) Resultant image after removing a area above the central point and heart area. (c) Extraction of pectoral muscle by selecting the greatest area of connected-component labeling.

Segmentation Methodology of BD

We segmented a BD with thresholding method using fuzzy c-means clustering technique. The MRI images are noisy and the literature reveals that the conventional FCM is not efficient to produce a threshold for noisy image and sometimes produces false positives in the segmented images [30]. Hence, we denoised the obtained BROI segmentation. Also, high-level FCM is required to produce efficient results which are computationally expensive. To cope with the computing problem, we used conventional FCM clustering method within the BROI area.

FCM clustering is based on the minimization of the following objective function [31] as shown in the Eq. (11). First of all, we define a number of clusters, and the random initialization of membership matrix in Eq. (13) is done. The centre of the cluster is calculated as shown in Eq. (14) using the membership matrix, . The membership matrix is updated according to the position of the cluster centre. The change in the membership matrix is calculated and compared with old membership matrix. If the objective function is minimized, the process is stopped otherwise a new center of clusters is determined and membership matrix is updated according to the new centers. The process continues until the objective function is minimized as shown in Eq. (12). where N and C are the number of data points and number of cluster centers. represents the membership function of data and cluster center. m and are the fuzziness index ≥1 and cluster center. The membership function and cluster centers are calculated as shown in Eq. (13) and Eq. (14): The membership function and cluster centers is calculated and repeated unless , where ϵ is the termination iteration between 0 and 1. Each pixel of the image is assigned to the respective cluster with the highest membership value. We conducted the experiment on FCM with several clusters. According to the experiment, FCM with 4 clusters is found to be effective for producing an accurate thresholding value. A threshold value is produced by averaging the mean of maximum and minimum value of the third cluster. Moreover, it is observed that the mean histogram intensity of left and right BROI is different. Hence, we calculated separate thresholds by using 4 level FCM of left and right BROI. The obtained segmentation result is accurate with faster computation.

Figure 9 shows the mean histogram of intensity information in the left and right BROI. For demonstration, we choose six MR images and observed that the right BROI has a higher intensity level as compared with the left BROI. Thus, we came to the conclusion that a single thresholding value produced by thresholding would not be sufficient to achieve high accuracy in the segmentation of BD. To solve this problem, we divided a BROI image into three different areas following similar steps during BROI segmentation. Since, the central area has no BD, we focus on the left and right BROI. Figure 10 demonstrates the left (blue solid arrowhead) and right (blue dotted arrowhead) BROI respectively. We conducted our experiment separately on the left and right BROI to extract BD and later resultant images are merged to produce the final result.

Figure 9

Mean value of histogram in terms of intensity of the left and right BROI.

Figure 10

A process to obtain BD from BROI from breast slices.

Ethics statement

Human studies were approved by Victoria University Committee and by the Institutional Review Board. MR imaging was conducted in accordance with guidelines defined by Affiliated Zhongshan Hospital of Dalian University to achieve safe and reliable scanning. The experiment was approved specifically by the ethics committee. Written consent was obtained from each case subject after the imaging procedures had been conveyed.

Results and Discussion

Image source and evaluation criteria

The experiment was performed on 15 female subjects (T1-weighted MR scans) with an age range between 22 and 54 years without any symptom of breast diseases. It was performed on a Philips Achieva 3.0T scanner using the turbo spin echo pulse sequence without fat suppression. Each patients MR scan covers entire breast with the total number of 90 image slices with 2 mm thickness. The other imaging parameters considered are: TR/TE = 645/9.0 ms, echo train = 5, slice gap = 0, phase encoding R-L, bandwidth per pixel = 174 Hz, field of view = 330 mm, imaging matrix = 328 × 384, and parallel imaging with SENSE factor = 2. The presented database consist of the variety of breast sizes, shapes, and breast tissues patterns. The proposed algorithm is applied to the individual slices to complete a 3D breast volume using Matlab R2013b running under Intel(R) core(TM) i5-4570s CPU@ 2.90 GHz with 8GB of RAM.

The performance of the proposed algorithm is tested with the quantitative analysis using a pixel-based classification technique where pixels are classified as BROI, BD or background. As a result, each pixel in the images are classified as classification (true positive (TP) and true negative (TN)) and misclassification (false positive (FP) and false negative (FN)). Based on these predictions, the performance of our algorithm is compared in terms of Accuracy (Acc), Precision (P), Sensitivity (Se) or Recall, Specificity (Sp), Area under ROC curve (AUC), Misclassification rate (MR), Dice similarity coefficient (DSC) and Jaccard Coefficient [66], [67], [70], [27]. These performance metrics are defined as shown in Eq. (15) to Eq. (21):TP and TN denote the pixel that is correctly identified as BROI/BD or background pixels. Similarly FP and FN represent the incorrectly identified BROI or BD, and background pixels. A and GT denotes automatically and manually obtained segmentations. Acc is the measure of the total number of correctly classified pixels (sum of true positives and true negatives) to the number of total pixels in an image [63], [64]. Precision is the proportion of correctly predicted positive observations to the total predicted positive observations [67]. Although both accuracy and precision depict the closeness of measurement to an actual value, precision reflects the reproducible measurements even if they are far from accepted value. The metrics Se and Sp are derived respectively from the proportion of positive and negative pixels in the ground truth image that are truly identified [68], [69]. A result with high sensitivity and specificity are considered as an accurate segmentation. A metric misclassification rate or error rate MR is the measure of how often the predictions are wrong. The best misclassification rate is 0.0 and the worst is 1.0. Also, receiving operating characteristics (ROC) curve [71] is used to estimate the trade-off between Se and Sp that is considered as non-parametric performance measurement. The ROC curve can be considered as a binary classifier and is plotted with the different values of independent threshold values within a certain interval. A curve representing a false positive rate (1-Sp) on the x-axis vs true positive rate (Se) on the y-axis is plotted. The ROC curve is the measure of predictive measure and is considered as an ideal curve when it is closer to the top left corner. The value of AUC greater than 90% is considered to be an excellent result. To further validate the performance of the developed algorithm, we have calculated an overlap based metrics known as Dice similarity coefficient (DSC) and Jaccard coefficient (JC) (DSC) [72]. It is the measure of overlap between two binary images to demonstrate the segmentation performance. The value of these overlap based metrics ranged from 0 (no overlap) to 1 (perfect overlap).

BROI segmentation results

Figure 11 shows the comparison of results produced by the proposed BROI segmentation method with the ground truth image which is manually segmented by an expert radiologist. The images with different size and shape are considered for the demonstration and yield the accurate segmentation results. First column ((a) (d) and (g)) shows the ground truth image whereas second column ((b), (e), and (h)) and third column ((c), (f), and (i)) shows the automatically segmented results respectively. In order to further validate the robustness of the proposed BROI segmentation, we performed the quantitative analysis using 8 metrics. Table 1 shows the performance of the proposed model in terms of accuracy, specificity, area under ROC curve, misclassification rate, precision, sensitivity and Dice similarity coefficient. Note that the values presented is the table are the average value of each slice in the MR image. Experiment shows that the obtained results are accurate while compared with the manual segmentation. In terms of accuracy, specificity and AUC, all the results are above 95%, proving the effectiveness of the proposed algorithm. It is observed that, the algorithm demonstrates a very good result with a minimum of 88% and maximum of 96% precision rate. The sensitivity of the proposed algorithm is high with an average value of 95.73%. The overlap ratio demonstrated by the Dice similarity and Jaccard coefficient is high with an average of 96.35% and 92.86% respectively.

Figure 11

Results of BROI segmentation on the MRI images with different levels of BD and different breast shapes. The images in the first column are the manually segmented ground truth images. Similarly, second and third columns are the automatically segmented results with the proposed method and its mask on the original image to visually inspect the accuracy.

Table 1

This table shows the resultant performance of BROI segmentation using the proposed method in 15 different cases in terms of accuracy (Acc), specificity (Sp), area under the curve (AUC), misclassification rate (MR), precision (P), sensitivity (Se) or recall, and Dice similarity coefficient (DSC).

DB	Acc	Sp	AUC	MR	P	Se	DSC	JC
1	0.9776	0.9830	0.97	0.0224	0.9425	0.9473	0.9555	0.9147
2	0.9648	0.9740	0.96	0.0398	0.8921	0.9413	0.9475	0.9002
3	0.9803	0.9783	0.99	0.0197	0.8800	0.9937	0.9564	0.9164
4	0.9871	0.9898	0.98	0.0129	0.9588	0.9591	0.9888	0.9778
5	0.9578	0.9554	0.95	0.0389	0.8960	0.9602	0.9599	0.9024
6	0.9819	0.9848	0.97	0.0181	0.9402	0.9414	0.9424	0.8910
7	0.9814	0.9845	0.97	0.0174	0.8873	0.9501	0.9647	0.9318
8	0.9897	0.9912	0.96	0.0122	0.9347	0.9123	0.9674	0.9368
9	0.9699	0.9671	0.97	0.0301	0.8945	0.9733	0.9542	0.9124
10	0.9829	0.9824	0.99	0.0171	0.9489	0.9868	0.9874	0.9751
11	0.9682	0.9644	0.99	0.0318	0.9421	0.9928	0.9867	0.9737
12	0.9834	0.9866	0.98	0.0166	0.9246	0.9420	0.9632	0.9290
13	0.9709	0.9728	0.98	0.0291	0.9272	0.9468	0.9568	0.9171
14	0.9795	0.9800	0.98	0.0205	0.9021	0.9679	0.9548	0.9135
15	0.9853	0.9888	0.98	0.0147	0.9301	0.9447	0.9673	0.9366
Avg	0.9773	0.9789	0.97	0.0228	0.9201	0.9573	0.9635	0.9286

BD segmentation results

Figure 12 shows the comparison of results produced by the proposed BD segmentation method and ground truth image which is manually segmented by an expert radiologist. The images with different level of breast tissue are considered for the demonstration and yield the accurate segmentation results. First column ((a) (d) and (g)) shows the ground truth image whereas second column ((b), (e), and (h)) and third column ((c), (f), and (i)) shows the automatically segmented results respectively. In order to further validate the robustness of the proposed BD segmentation, we performed the quantitative analysis using 8 metrics. Table 2 shows the performance of the proposed model in terms of accuracy, specificity, area under ROC curve, misclassification rate, precision, sensitivity and Dice similarity coefficient. Note that the values presented is the table are the average value of each slice in the MR image. Experiment shows that the obtained results are accurate and highly comparable with results obtained from the manual segmentation. In terms of 3 metrics (accuracy, specificity and AUC), the results are above 95%, proving the effectiveness of the proposed algorithm. It is observed that, the algorithm demonstrates an outstanding result with an average of 95.05% precision rate. The overlap ratio demonstrated by the Dice similarity and Jaccard coefficient is high with an average of 91.60% and 84.53% respectively. In terms of sensitivity, resultant values are slightly low but satisfactory values compared with other parameters. This is because some part of the BD has very low intensity and could be missed during the segmentations.

Figure 12

Results of BD segmentation on the MRI images with different levels of BD and different breast shapes. The images in the first column are the manually segmented ground truth images. Similarly, second and third columns are the automatically segmented results with the proposed method and its mask on the original image to visually inspect the accuracy.

Table 2

This table shows the resultant performance of BD segmentation using the proposed method in 15 different cases in terms of accuracy (Acc), specificity (Sp), area under the curve (AUC) and misclassification (MR), precision (P), sensitivity (Se) or recall, and Dice similarity coefficient (DSC).

DB	Acc	Sp	AUC	MR	P	Se	DSC	JC
1	0.9879	0.9898	0.96	0.0133	0.9514	0.8140	0.9046	0.8260
2	0.9799	0.9854	0.96	0.0321	0.9701	0.8190	0.9321	0.8728
3	0.9874	0.9872	0.97	0.0126	0.9549	0.8199	0.8946	0.8092
4	0.9912	0.9897	0.96	0.0088	0.9301	0.8145	0.8984	0.8155
5	0.9478	0.9701	0.93	0.0522	0.9647	0.8075	0.9141	0.8417
6	0.9885	0.9819	0.97	0.0115	0.9611	0.8110	0.9231	0.8571
7	0.9749	0.9814	0.97	0.0354	0.9302	0.8297	0.9145	0.8424
8	0.9845	0.9989	0.94	0.0155	0.9482	0.8176	0.9108	0.8362
9	0.9855	0.9676	0.96	0.0212	0.9444	0.8412	0.9402	0.8871
10	0.9788	0.9701	0.97	0.0277	0.9589	0.8250	0.9001	0.8183
11	0.9612	0.9627	0.98	0.0388	0.9628	0.8555	0.9374	0.8821
12	0.9888	0.9797	0.97	0.0112	0.9494	0.8002	0.9045	0.8256
13	0.9659	0.9645	0.97	0.0341	0.9579	0.8109	0.9141	0.8417
14	0.9873	0.9823	0.96	0.0127	0.9312	0.8125	0.9214	0.8542
15	0.9898	0.9781	0.95	0.0102	0.9415	0.8212	0.9306	0.8702
Avg	0.9800	0.9793	0.96	0.0225	0.9505	0.8199	0.9160	0.8453

Discussion

Obtaining automatic, fast and accurate segmentation of the BROI and BD from MR images is a significant and challenging problem. The breast images can be found in different shapes, sizes and density patterns. Moreover, the pectoral muscles are closely connected and shares similar intensity distribution with BROI. Hence, the initial identification of landmarks such as lung, heart and pectoral muscle is a vital step to facilitate the efficient BROI and BD segmentation process.

This study performs a stepwise analysis on landmarks such as lung, heart, and pectoral muscle and gradually eliminates them to achieve the final segmentation results. Prior to the landmark identification, we utilized the pre-processing step which improves the segmentation process. During the heart segmentation, an active contour level set method is used on the last 4 clusters from k-means clustering obtained during the pre-processing step. These 4 clusters are the brightest and the experiment shows that the heart region lies within these clusters. Active contour uses the level set method to create a force to either shrink or expand the contour from the initial contour point. The initial contour point is always selected approximately around the centre of the heart region using the maximum entropy thresholding and convolution method. We fix the initial contour (circle) at the radius (5 pixel) from the initial contour point. This is because we wanted to expand the contour from the central area of heart and limit it beyond the circular radius of 80 pixels. We observed that circular radius of 80 pixel is sufficient to identify the heart area from the rescaled image. Furthermore, the segmentation is obtained within a few iterations which results in faster computation time.

During pectoral muscle segmentation, the image is divided into three sections. The angular orientation of the pectoral muscle and breast density boundary is different in three different sections. So, each segment needs to be analysed separately. Also, the analysis of smaller segments reduces the processing time. Finally, the resultant image from the three segments are merged to generate the final segmented image. In the next step, we preserved the breast area above the central point to have fewer components for the extraction of the pectoral muscle as shown in Figure 8 (a), (b).

The image obtained from BROI segmentation is further processed for BD segmentation. In the conventional method, the 4 level FCM thresholding technique is used to develop a single threshold valve for the entire image which includes both breasts. However, the analysis of the mean histogram based on the intensity of left and right breasts showed that the left and right breast have dissimilar mean intensity. Hence, the BROI is divided into three sections based on their geometrical information and the 4 level FCM thresholding is applied separately to develop two threshold values for left and right breasts. It is observed that accuracy of separate thresholding in left and right BROI is better than the single thresholding technique as shown in Table 3. Furthermore, since varied thresholding uses smaller area, the process became faster.

Table 3

Comparison of accuracy using three and four clustered FCM on BROI after segmentation.

Database	ACC with 4 level FCM (single threshold)	ACC with 4 level FCM (double threshold)
1	0.9601	0.9776
2	0.9421	0.9648
3	0.9656	0.9803
4	0.9611	0.9871
5	0.9594	0.9578
6	0.9658	0.9819
7	0.9470	0.9814
8	0.9523	0.9897
9	0.9349	0.9699
10	0.9231	0.9829
11	0.9451	0.9682
12	0.9532	0.9834
13	0.9475	0.9709
14	0.9546	0.9795
15	0.9529	0.9853
Avg	0.9509	0.9773

The proposed method was tested with 15 different MR images developed form the same imaging technique with different scenarios i.e. variety of breast sizes, shapes and BD patterns. The result demonstrates that, our method accurately segments BROI and BD in the different scenarios which can also be observed with the segmentation result as shown in Figure 11 and Figure 12. To evaluate the performance of the proposed method, eight evaluation metrics are calculated. The obtained results shows that the proposed method is efficient for the segmentation of BROI and BD from MR images.

We choose nine recent methods in the literature such as [73], [27], [75], [33], [51], [26], [28], [76], and [74] to analyze the BROI segmentation and three methods such as [75], [33] and [77] for the BD segmentation, the results of which are compared with the proposed method. The quantitative comparison is done with the available five metrics (Acc, Sp, Se, DSC and JC) as shown in Table 4. In terms of Acc and DSC, our method outperforms all the recent results for both BROI and BD segmentation. JC obtained from the proposed method is marginally better or comparable with the existing literature. The results associated with Sp and Se are highly comparable with most of the results.

Table 4

Quantitative comparison of performance of BROI and BD segmentation using the proposed method with the recently developed other approaches.

	Acc	Sp	Se	DSC	JC
BROI
Gallego et al. 2012 [73]	–	–	0.8900	0.8800	0.7900
Wu et al. 2013 [27]	–	–	–	0.9500	–
Ivanovska et al. 2014 [75]	–	0.9900	0.9800	0.9600	-
Gubern et al. 2015 [33]	–	–	–	0.9400	–
Jose et al. 2015 [51]	–	–	–	0.9220	–
Khalvati et al. 2015 [26]	–	–	–	0.9400	–
Milenkovic et al. 2015 [28]	–	–	–	0.961	–
Doran et al. 2017 [76]	–	–	–	0.924	0.8590
Aida et al. 2017 [74]	0.9733	0.9810	0.9491	0.9630	0.9290
PROPOSED METHOD	0.9773	0.9789	0.9573	0.9635	0.9286
BD
Ivanovska et al. 2014 [75]	–	0.9900	0.8100	0.8300	–
Gubern et al. 2015 [33]	–	–	–	0.80	–
Thakran et al. 2018 [77]	–	–	0.8900	0.9000	0.8400
PROPOSED METHOD	0.9800	0.9793	0.8199	0.9160	0.8453

For automatic BROI and BD segmentation, we have compared our DSC value with different methods in the literature. In the model based method followed by Gallego et al. [73], a mean Se, DSC and JC obtained was 89%, 88% and 79%. The processing time taken to run was less than a minute to segment BROI per volume with the size of . The edge based approach followed by Wu et al. [27] demonstrates an average DSC rate of 95%. The processing time taken for BD segmentation was 4.5 minutes with MR image of per volume. Gubern et al. [33] and Kalvati et al. [26] use the atlas based method and showed that the mean DSC obtained was 94%. Gubern-Merida et al. [33] reported a computational time of 8 min for BROI and BD segmentation for the image of per volume. Similarly, Kalvati et al. [26] showed that the processing time for his method was 2 min for BROI segmentation for the MR image of per volume. A automatic BROI segmentation in the axial breast MR images proposed by Milenkovic et al. [28] obtained a overall DSC value of 96.1% and the computational time was 4.1 minutes on the MR image of per volume for BROI segmentation.

The computation time in the proposed algorithm mainly depends upon the resolution of the MR images and the clusters for the Kmeans and Fuzzy cmean clustering technique used for the experiments. The number of clusters should be determined so that the BROI and BD regions can be preserved with faster execution of the algorithm. We execute our algorithm for the several times to optimize the solution and the experiments takes an average of 1 minute and 50 seconds for BROI and BD segmentation with the resolution of per volume. The execution time of our algorithm is significantly less than the other recent approaches tested on the similar hardware. The performance can be further improved with the implementation of GPU.

Breast MR databases are not available online. Results presented in the state of art are calculated from their own private databases. Hence, it is not always suitable to compare the results developed from different databases. Furthermore, during the segmentation process, the various methods might have several assumptions and considerations that make direct comparison problematic. For instance, Wang et al. [29] results are dependent upon the presence of fat in the anterior side of chest wall and Wu et al. [27] does not consider challenging cases. Also, the processing speed depends upon the resolution of MR image of different databases.

Conclusion

In this paper, we proposed an automatic method for the accurate segmentation of the BROI and BD. BROI segmentation is achieved by combining pixelwise adaptive filtering, k-means clustering and morphological operations with the application of local adaptive thresholding. BD segmentation is obtained by a combined method using fuzzy c means thresholding and mean value histogram. These frameworks have been tested on 15 different cases that comprised of different shapes and density patterns. Furthermore, quantitative analysis was carried with different evaluation metrics (Acc, Sp, AUC, MR, P, Se, DSC and JC) to demonstrate the segmentation quality when compared with manually segmented results by an expert. Most particularly, it is observed that the proposed algorithm is highly effective on breast MRIs with dense BD that has an similar intensity level to the area near the pectoral muscle. The presented model can act as a preliminary step that further assists in the diagnosis of breast cancer.

Declarations

Author contribution statement

Dinesh Pandey: Conceived and designed the experiments; Performed the experiments; Analyzed and interpreted the data; Contributed reagents, materials, analysis tools or data; Wrote the paper.

Xiaoxia Yin, Hua Wang: Performed the experiments; Analyzed and interpreted the data; Wrote the paper.

Min-Ying Su, Jeon-Hor Chen, Jianlin Wu: Conceived and designed the experiments; Contributed reagents, materials, analysis tools or data.

Yanchun Zhang: Performed the experiments; Wrote the paper.

Funding statement

This research did not receive any specific grant from funding agencies in the public, commercial, or not-for-profit sectors.

Competing interest statement

The authors declare no conflict of interest.

Additional information

No additional information is available for this paper.