Mitosis detection in breast cancer histological images An ICPR 2012 contest.

INTRODUCTION: In the framework of the Cognitive Microscope (MICO) project, we have set up a contest about mitosis detection in images of H and E stained slides of breast cancer for the conference ICPR 2012. Mitotic count is an important parameter for the prognosis of breast cancer. However, mitosis detection in digital histopathology is a challenging problem that needs a deeper study. Indeed, mitosis detection is difficult because mitosis are small objects with a large variety of shapes, and they can thus be easily confused with some other objects or artefacts present in the image. We added a further dimension to the contest by using two different slide scanners having different resolutions and producing red-green-blue (RGB) images, and a multi-spectral microscope producing images in 10 different spectral bands and 17 layers Z-stack. 17 teams participated in the study and the best team achieved a recall rate of 0.7 and precision of 0.89. CONTEXT: Several studies on automatic tools to process digitized slides have been reported focusing mainly on nuclei or tubule detection. Mitosis detection is a challenging problem that has not yet been addressed well in the literature. AIMS: Mitotic count is an important parameter in breast cancer grading as it gives an evaluation of the aggressiveness of the tumor. However, consistency, reproducibility and agreement on mitotic count for the same slide can vary largely among pathologists. An automatic tool for this task may help for reaching a better consistency, and at the same time reducing the burden of this demanding task for the pathologists. SUBJECTS AND METHODS: Professor Frιdιrique Capron team of the pathology department at Pitiι-Salpκtriθre Hospital in Paris, France, has selected a set of five slides of breast cancer. The slides are stained with H and E. They have been scanned by three different equipments: Aperio ScanScope XT slide scanner, Hamamatsu NanoZoomer 2.0-HT slide scanner and 10 bands multispectral microscope. The data set is made up of 50 high power fields (HPF) coming from 5 different slides scanned at ×40 magnification. There are 10 HPFs/slide. The pathologist has annotated all the mitotic cells manually. A HPF has a size of 512 μm × 512 μm (that is an area of 0.262 mm (2) , which is a surface equivalent to that of a microscope field diameter of 0.58 mm. These 50 HPFs contain a total of 326 mitotic cells on images of both scanners, and 322 mitotic cells on the multispectral microscope.
RESULTS: Up to 129 teams have registered to the contest. However, only 17 teams submitted their detection of mitotic cells. The performance of the best team is very promising, with F-measure as high as 0.78. However, the database we provided is by far too small for a good assessment of reliability and robustness of the proposed algorithms.
CONCLUSIONS: Mitotic count is an important criterion in the grading of many types of cancers, however, very little research has been made on automatic mitotic cell detection, mainly because of a lack of available data. A main objective of this contest was to propose a database of mitotic cells on digitized breast cancer histopathology slides to initiate works on automated mitotic cell detection. In the future, we would like to extend this database to have much more images from different patients and also for different types of cancers. In addition, mitotic cells should be annotated by several pathologists to reflect the partial agreement among them.

INTRODUCTION

Nottingham grading system[1] is an international grading system for breast cancer recommended by the World Health Organization. It is derived from the assessment of three morphological features on slides stained with H and E: Tubule formation, nuclear pleomorphism, and mitotic count.

Mitotic count is an important parameter in breast cancer grading as it gives an evaluation of the aggressiveness of the tumor. However, consistency, reproducibility and agreement on mitotic count for the same slide can vary largely among pathologists.[23] An automatic tool for this task may help for reaching a better consistency, and at the same time reducing the burden of this demanding task for pathologists.

Detection of mitotic cells is a very challenging task because they are small objects with a large variety of shape configurations and a low frequency of appearance. Some examples of ground truth mitotic cells are shown in Figure 1. The objective of the contest is to encourage works on the detection of mitosis on H and E stained histological images of the breast cancers.

Figure 1

Example of ground truth mitotic cells for scanners

Several studies on automatic tools to process digitized slides have been reported[4] focusing mainly on nuclei or tubule detection. Mitosis detection is a challenging problem that has not yet been addressed well in the literature. Only few works concern detection of mitosis. Beliën et al.,[5] counted mitoses on Feulgen stained breast cancer sections. Recently Liu et al.,[6] and Huh et al.,[7] proposed mitosis detection in time-lapse phase contrast microscopy image sequences of stem cell populations and Schlachter et al.[8] performed detection of mitoses in fluorescence staining of colorectal cancer. Roullier et al.,[9] propose detection of mitotic cells on breast cancer slides with an immunohistochemical staining that highlights specifically mitosis.

The only work concerning mitosis detection on H and E stained slides is by Malon et al.,[10] who propose the use of convolutional neural networks (CNN). Sertel et al.,[11] presented a method for the detection of mitosis and karyorrhexis cells (dying cells) without distinction, but for breast cancer grading, only mitotic cells must be counted.

SUBJECTS AND METHODS

Dataset

Professor Frédérique Capron’s team of the pathology department at Pitié-Salpêtrière Hospital in Paris, France, has provided a set of five slides of breast cancer. The slides are stained with H and E. They have been scanned by three different equipments:

Aperio ScanScope XT slide scanner (scanner A);

Hamamatsu NanoZoomer 2.0-HT slide scanner (scanner H);

And 10 bands multispectral microscope (microscope M). The spectral bands are all in the visible spectrum. In addition, for each spectral band, the digitization has been performed at 17 different focus planes (17 layers Z-stack), each consecutive planes being separated from each other by 500 nm.

Ground Truth

The data set is made up of 50 high power fields (HPF) coming from 5 different slides scanned at ×40 magnification. There are 10 HPFs per slide. The pathologist has annotated all the mitotic cells manually. She made the annotations in each selected HPF on the images generated by the scanner A, the scanner H and the multispectral microscope M.

A HPF has a size of 512 μm × 512 μm (that is an area of 0.262 mm2), which is a surface equivalent to that of a microscope field diameter of 0.58 mm. These 50 HPFs contain a total of 326 mitotic cells on images of both scanners, and 322 mitotic cells on the microscope M.

Table 1 gives the number of mitotic cells in the training data set and in the evaluation data set. There are more mitotic cells on the scanner images as compared to the microscope M images. This discrepancy has its origin in the smaller size of multispectral images as compared to the scanner images. Four multispectral images are needed to cover almost the entire surface of a single scanner HPF. However, small gaps remain between the four multispectral images and the same area of a scanner HPF [Figure 2]. As a result, few mitotic cells visible on the border of scanner HPFs are missing on the multispectral images.

Table 1

Number of HPFs and mitotic cells in training and evaluation data sets

Figure 2

Location of quarters a, b, c and d of multispectral microscope in scanner image

Resolution of Scanners and Microscope

Scanner A has a resolution of 0.2456 μm/pixel. Scanner H has a slightly better resolution of 0.2273 μm (horizontal) and 0.22753 μm (vertical) per pixel. Note that a pixel of scanner H is not exactly a square. At last, multispectral microscope M has the best resolution of 0.185 μm per pixel. Table 2 shows the resolutions of the different scanners and the microscope. For example, a mitosis having an area of 50 μm2 will cover about 830 pixels of the image produced by scanner A, about 965 pixels of the image produced by scanner H, and about 1460 pixels of the image produced by multispectral microscope M.

Table 2

Resolution of the scanners A and H and the multispectral microscope M

For each slide, there is one RGB image produced by scanner A, one RGB image produced by scanner H, and 170 grey scale images for the multispectral microscope M (10 spectral bands and 17 layers Z-stack for each spectral band).

Multispectral Microscope M

The camera attached on top of the microscope generates images of 1360 × 1360 pixels. However, to cover an area of 512 μm × 512 μm, 2767 × 2767 pixels are needed. Therefore, we will use four images to cover the same area as the two scanners. However, these four images do not cover completely the 512 μm × 512 μm area, 47 pixels are missing in width and in height to cover fully the area.

Each image, covering a quarter of a scanner image, is labeled a, b, c or d depending on its position in the scanner image. Figure 2 shows the location of each quarter a, b, c, d. As the quarters do not cover completely the 512 μm × 512 μm area, compared to the scanner images, there is a small gap on the borders, and also a small gap between quarters a, b, c and d.

Figure 3 shows the spectral coverage of each of the 10 spectral bands of the microscope M. All the bands are in the visible spectrum.

Figure 3

Spectral bands of the multispectral microscope and examples for each band

Evaluation Metrics

The main goal of the contest is to be able to give the mitotic count on each slide. A segmented mitosis would be counted as correctly detected if its centroid is localized within a range of 8 μm of the centroid of ground truth mitosis. The evaluation metrics are defined as follows:

TP = number of true positives, that is the number of candidate mitotic cells that are ground truth mitotic cells.

FP = number of false positives, that is the number of candidate mitotic cells that are not ground truth mitotic cells.

FN = number of false negatives, that is the number of ground truth mitotic cells that have not been detected.

Recall (sensitivity) =

Precision (positive predicitive value) =

RESULTS

The ground truth and images of training data set have been provided at the beginning of the contest on November 2011. At the end of the contest, in August 2012, contestants received images of the evaluation data set, but not the corresponding ground truth. All the rankings are made according to F-measure.

Up to 129 teams have registered to the contest. They downloaded and worked on the training data set to prepare and tune their algorithms for detection of mitotic cells. At the end of the contest, they received the evaluation data set. However, only 17 teams submitted their detection of mitotic cells. Team names are listed in Table 3. Detection results and rankings are given in Table 4 for scanner A, Table 5 for scanner H and Table 6 for microscope M.

Table 3

List of contestants

Table 4

Detection results and rankings for scanner Aperio (rankings are according to F-measure)

Table 5

Detection results and rankings for scanner Hamamatsu (rankings are according to F-measure)

Table 6

Detection results and rankings for multispectral microscope (rankings are according to F-measure)

Overall, detection of mitotic cells is better on scanner A than on scanner H. Detection results on multispectral microscope are very poor as compared to scanners A and H. This is shown by the results of NEC, Shiraz University of Technology (SUTECH) and Image and Pervasive Access Lab (IPAL) teams who had better detection on scanner A respectively with 59, 72 and 74 true positives, whereas these figures are respectively 44, 61 and 71 for scanner H. However, NEC and SUTECH had more false positives on scanner A (respectively 20 and 31) than on scanner H (respectively 14 and 13). Although, IPAL had much more false positives on scanner H (56) than on scanner A (32). A few examples of false positives and false negatives are presented in Figures 4 and 5.

Figure 4

Some examples of false positives. The false mitotic cell objects are located in the center of each image

Figure 5

Some examples of false negatives. The not detected mitotic cell objects are located in the center or each image

DISCUSSION

The general processing method developed by most teams for detection of mitotic cells is globally made up of four steps.

Detection of candidate blobs or seed points using thresholding and mathematical morphology.

Blob segmentation with level-set or active contours.

Computation of features on segmented blobs (radiometry, morphology, texture).

Classification of candidate blobs as mitosis or non-mitosis object.

For Isik University team, the classifier used was adaboost classifier while for IPAL team, it was a decision tree. IPAL team also used a selection of color channels of different color models (RGB, hue-saturation-value (HSV), Lab, Luv) and computed the features on the selected channels.

NEC team is the only one team who applied their method on all the provided images (both scanners and the multispectral microscope). They used a CNN as classifier. Their method is efficient as they ranked high for both scanners, and first for the multispectral microscope.

Warwick team introduced a tumor segmentation to discard non-tumor areas from the images as these areas are full of lymphoid, inflammatory or apoptotic cells, which are not relevant for cancer grading. Hence mitosis detection is performed only on tumor areas. They made statistical modeling of mitotic cells from their grey level intensities. To match the distribution of grey level intensities of each class (mitotic cell/background), they used a Gamma distribution for mitotic cells and a Gaussian distribution for background.

Istituto Dalle Molle di Studi sull’Intelligenza Artificiale (IDSIA) team approach relies on a single step processing: The use of a CNN to compute a map of probabilities of mitosis over the whole image. Their CNN has been trained with the ground truth mitosis provided in the training data set. Their approach proved to be very efficient as they clearly had the best F-measure on scanner images, and a very low number of false positives as compared to their immediate competitors.

An improved version of this successful challenge will involve a much larger number of mitosis, images from more slides and multiple pathologists’ collaborative/cooperative annotations. Besides, some slides will be dedicated to test only without any HPF of these slides included in the training data set.

CONCLUSION

Mitotic count is an important criterion in the grading of many types of cancers; however, very little research has been made on automatic mitotic cell detection, mainly because of a lack of available data. A main objective of this contest was to propose a database of mitotic cells on digitized breast cancer histopathology slides to initiate works on automated mitotic cell detection.

Up to 129 teams have registered to the contest and downloaded the training data set. In the end, 17 of them submitted their detection results on the evaluation data set. The performance of the best team is very promising, with F-measure as high as 0.78. However, the database we provided is by far too small for a good assessment of reliability and robustness of the proposed algorithms.

In the future, we would like to extend this database to have much more images from different patients and also for different types of cancers. In addition, mitotic cells should be annotated by several pathologists to reflect the partial agreement among them.