A fast and effective detection framework for whole-slide histopathology image analysis.

Pathologists generally pan, focus, zoom and scan tissue biopsies either under microscopes or on digital images for diagnosis. With the rapid development of whole-slide digital scanners for histopathology, computer-assisted digital pathology image analysis has attracted increasing clinical attention. Thus, the working style of pathologists is also beginning to change. Computer-assisted image analysis systems have been developed to help pathologists perform basic examinations. This paper presents a novel lightweight detection framework for automatic tumor detection in whole-slide histopathology images. We develop the Double Magnification Combination (DMC) classifier, which is a modified DenseNet-40 to make patch-level predictions with only 0.3 million parameters. To improve the detection performance of multiple instances, we propose an improved adaptive sampling method with superpixel segmentation and introduce a new heuristic factor, local sampling density, as the convergence condition of iterations. In postprocessing, we use a CNN model with 4 convolutional layers to regulate the patch-level predictions based on the predictions of adjacent sampling points and use linear interpolation to generate a tumor probability heatmap. The entire framework was trained and validated using the dataset from the Camelyon16 Grand Challenge and Hubei Cancer Hospital. In our experiments, the average AUC was 0.95 in the test set for pixel-level detection.

Introduction

In the past 100 years, pathologists have used microscopy to observe glass slides for clinical and pharmaceutical research, and more importantly, for providing definitive disease diagnoses to guide patient treatment and management decisions [1]. With the rapid development of whole-slide digital scanners for histopathology, computer-assisted digital pathology image analysis has increasingly attracted clinical attention [2]. In this rapidly growing field of digital pathology, computer-assisted image analysis systems have been confirmed to help pathologists diagnose tumors and cancer subtypes. In clinical practice, accurately distinguishing regions (normal and tumor) in digital pathology images is an important task that helps pathologists perform basic examinations and complement their opinion [3]. Thus, the workload of pathologists would be greatly reduced without any loss in sensitivity at the patient level. Pathologists can focus on making more complex and detailed diagnoses to ultimately provide more accurate results [4, 5].

Whole-slide digital scanners have become more prevalent in clinical hospitals and make it easier to digitize, store, share, visualize and analyze histopathology slides. Moreover, as one of the newest forms of “big data”, whole-slide images (WSIs) in histopathology are constantly being produced every day. Typically, each WSI could have a full spatial resolution of 80K × 80K pixels and is approximately 2 GB in compressed storage size at 40× magnification. This high volume of data requires the development of a fast and effective processing pipeline for analyzing digital image data.

In recent years, there has been increasing interest in developing computer-assisted image analysis methods in pathology. A variety of competitions have emerged to promote intelligent algorithm research on digital tumor histopathology. The early competition task was to perform cell segmentation and image-related feature extraction. The tasks are the classification and grading of more complex whole-slide pathological images.

The classification and grading of pathological images is the last step in the automatic analysis of pathological sections, and it is also a crucial step. In recent years, with the powerful tool of deep learning, researchers have applied CNNs in various cancer detection tasks and achieved good results. The champion team of Camelyon16, Wang [6], obtained an area under the receiver operating characteristic curve (AUC) of 0.925 for WSI classification using GoogLeNet and random forest classifier with feature engineering. Cruz-Roa [7] proposed HASHI based on a patch-based classifier with a 2-layer CNN, probability gradient from a heatmap, and Quasi-Monte Carlo sampling for WSI. The adaptive sampling algorithm used in this paper is derived from this method.

Han [8] proposed a multiclassification task to identify subordinate classes of breast cancer that uses a combination model of CNNs to analyze breast cancer histopathological images from the BreaKHis dataset. Valkonen [9] extracted a large number of quantitative descriptors of image texture, spatial structure, and distribution of nuclei and applied a random forest model to output confidence values indicating the likelihood of cancer cells. Xu [10] used a pre-trained AlexNet to extract the features of input patches and trained a linear SVM for segmentation in the MICCAI brain tumor challenge. Wan [11] constructed combinations of feature sets, including pixel-, object-, and semantic-level features derived from CNN, and utilized multiple SVM classifiers to determine breast cancer grades. Bayramoglu [12] proposed a multitask CNN to predict both malignancy and image magnification levels simultaneously to improve performance on the BreaKHis dataset. Alsubaie [13] proposed a deep CNN under multi-resolution to perform lung adenocarcinoma pattern classification. Sirinukunwattana [14] presented a segmentation performance comparison of 10 different network architectures for histology image classification problems.

In the BACH challenge of ICIAR 2018, one of the tasks consisted of performing pixel-wise labeling of clinical hematoxylin-eosin-stained histopathological WSIs in four classes. Many new methods for the automatic classification of breast cancer biopsies were proposed, and CNN dominated the challenge [15]. In [16], a fully convolutional network based on DenseNet [17] was proposed for performing pixel-wise labeling of WSIs. In [18], a two-stage patch-based approach was proposed, which consisted of an autoencoder to extract image features and an image-wise CNN to perform the classification of the whole image. In [19], an ensemble of four modified Inception-V3 models was proposed for increasing the generalization capability of different networks trained on random subsets of training data. For WSI, a sliding window was used to uniformly extract patches, and a refined heatmap using ResNet-34 was used to reduce potential misclassifications. [20] used an ImageNet pre-trained on DenseNet-161 for the segmentation of WSIs. [21] used an encoder-decoder network. The encoder is composed of five convolutional processing blocks that integrate dense skip connections, group and dilated convolutions, and a self-attention mechanism following SENet [22], and the decoder follows the U-Net [23] structure with skip connections between the down-sample and up-sample.

Li [24] proposed a neural conditional random field (NCRF) deep learning framework to detect cancer metastasis in WSIs. NCRF considers 9 spatially adjacent patches through a fully connected CRF, which is incorporated on top of a CNN feature extractor based on ResNet. Tokunaga [25] aggregated three expert CNNs based on U-Net by using three different magnification images and used a modified Xception [26] model to adaptively change the weight of each expert network depending on the input image. Li [27] developed a graph convolutional neural network to learn global topological representations of WSI for providing more accurate survival risk predictions. Wang [28] proposed a recalibrated multi-instance network for adaptively aggregating the patch information to image-level prediction of whole slide gastric image, which improved image-level classification accuracy by assigning different weights to each instance. Sun [29] applied U-Net to extract pixel-level features and adopt multiple classic fine-tuned CNN to obtain patch-level features, then jointed them by a hierarchical conditional random field method to localize abnormal (cancer) regions in gastric histopathology images.

In recent years, deep learning in solving image classification tasks, such as classification on ImageNet, has been greatly successful. Deep convolutional neural network (DCNN) models have been reported to surpass human performance. These models are typically used to process relatively small-sized natural images (200 × 200 pixels), but WSI is over hundreds of times the size of a natural image. Therefore, most pathology image analysis methods take a patch-based classification approach that first segments a large image into small patches and then classifies each patch. This piecemeal approach has limited their analysis to small regions of interest (ROIs) within the larger WSI. Thus, the overall size of the neural network can be allocated in the GPU memory. The issue associated with this approach is the need to use a sampling mechanism to traverse the entire pathology image. Dense uniform or regular sampling is one of the practical options, but the efficiency is not high. Even if there is no overlap between sample patches, a full detection process for the WSI is required to extract tens or hundreds of thousands of patches. In contrast, the adaptive sampling method is a more effective strategy for dealing with WSIs because it adaptively chooses regions with high uncertainty of a tissue patch being cancerous or not. For regions wherein the predictor has a greater uncertainty about cancer and normal tissue classification, more patch samples will be classified to improve the confidence of the adaptive sampling method for those regions of ambiguity [7].

To establish a complete WSI processing pipeline, there are still some issues to discuss after the patch-based classification and adaptive sampling mechanism are selected, such as, how to develop an efficient and accurate classifier, what is the more appropriate convergence condition of the iterative sampling process, how to use images under a wider range of magnifications?

Here, we present a deep learning-based approach for the identification of tumor metastasis on WSIs from the Camelyon16 dataset [30]. In summary, the main contributions of our study are as follows:

Based on High-throughput Adaptive Sampling for Whole-slide Histopathology Image analysis (HASHI) [7], we propose an improved adaptive sampling method with superpixel segmentation and introduce a new heuristic factor, with local sampling density as the convergence condition of iterations to improve the detection effect of multiple instances.

We develop the Double Magnification Combination (DMC) classifier, which is a modified DenseNet-40 to make patch-level predictions to discriminate tumor patches from normal patches. The lightweight network use 20× and 40× magnification images with only 0.3 million parameters, and uses the large-margin Gaussian Mixture (L-GM) loss function [31] to improve the generalization performance.

In postprocessing, we train a CNN model with 4 convolutional layers to regulate the patch-level predictions based on the predictions of adjacent sampling points.

The source code for our approach has been made publicly available at https://gitee.com/w3STeam/Pathological-images and https://github.com/JustinRuan/Pathological-images.

Materials and methods

Our tumor metastasis detection framework consists of a patch-based classifier, an improved adaptive sampling method, and a postprocessing filter. The complete pipeline is divided into two stages, namely, the sampling stage and the postprocessing stage, as shown in Fig 1.

Fig 1

An overview of our proposed workflow.

Patch extraction and preprocessing

Our model was trained with the Camelyon16 dataset, which consists of 400 WSIs total, split into 270 WSIs for training and 130 WSIs for testing. Here, the extraction of patches was divided into two cases: extraction for generating a training set and extraction in adaptive sampling for detection.

To focus our training data set on regions of the slide most likely to contain tumor metastasis, we first identified tissue within the WSI and excluded background white space. There are many methods based on threshold segmentation, such as [6, 9, 11]. We adopt a fixed-level threshold segmentation method in the HSV color space to exclude the obvious background region. The final mask images were generated by combining the masks from the S and V channels. The constraint of effective coverage is that the threshold of the V channel is between 0.2 and 0.8, and the threshold of the S channel is greater than 0.1.

According to the detection results and pathologist’s annotation, we extracted four types of patches: normal, tumor, edge inner, and edge outer. The labeling of a patch is determined by the proportion of the tumor area within the patch. When the proportion of tumor area is less than 50%, this patch is normal (Label 0); otherwise, it is tumor (Label 1). For the labeling under the combination of double magnifications, we adopted the "or" logic here. Here We used morphological methods to extract edge regions and increased the number of training samples at the edge of annotations to improve the performance of the classifier. Because the patches at the edge of annotations are usually transitional regions from tumor tissue to normal tissue, most of the “hard examples” are concentrated in these positions.

At a sampling point, we simultaneously extracted two patches under 20× and 40× magnifications, and the size of the patches was 256×256, as shown in Fig 2. Preprocessing and normalization were not applied to these saved patches to preserve the inherent fluctuation characteristics caused by staining. These are also what the classifier needs to fit. We obtained a total of 1,694,228 patches, with 1,336,704 labeled as normal, 230,966 labeled as tumor, 57,384 labeled as edge inner, and 69,174 labeled as edge outer. It is worth noting that we balanced the number of positive and negative samples in a WSI. The sampling interval in the normal regions is larger than that in the tumor region. In this way, the number of negative samples in a WSI does not exceed 5~6 times the number of positive samples. Then, we constructed several balanced training sample sets (1:1) through random sampling for improving the performance of the patch-based classifier.

Fig 2

Extract two magnification patches at a sampling point.

For sampling and predicting, we first used the threshold segmentation method mentioned above to calculate the effective region of the slide. Then, we directly extracted the 20× and 40× patches at pseudo-random sampling points and input them into the patch-based classifier. As in training, these extracted patches do not require any preprocessing or normalization.

Architecture of the patch-based classifier

To explore the appropriate classifier structure, We chose nine classic pre-trained ImageNet networks to test the patch-based classifier under three magnifications. We replaced the original top layer with a new one to connect each feature extraction part, which consists of a Global Average Pool (GAP) and two fully-connected (FC) layers. We used the prepared patches on three different magnifications to fine-tuning the top layer of each transfer model and tested the accuracy of these models. All testing results of transfer-learning are shown in S1 Table in S1 Appendix. According to the results of transfer-learning, the DenseNet family has the best feature extraction performance for pathological image blocks. The patches under 20× have the best distinguishable characteristics that can be extracted by CNN, as a result of the balance of the texture details and texture range in view. Although the 10× patches have a larger field of view, they are down-sampled to the same size resulting in the loss of texture and degradation of classification performance. Compared to 20×, the classifiers under 10× are a little worse. Under 40×, the field of view in a patch becomes very small in a patch. When the patches are extracted from the transitional zone from tumor to normal near the edge of annotations, these patches under 40× are no significant and typical texture features, and even look the same as the patches in normal regions. So, it is difficult to train a better classifier under this magnification individually. On the other hand, the patch-based classifier calculates a tumor feature based on an entire 256 × 256 image, and the calculated patch-level prediction is stored in a tumor feature map based on the central coordinate of this patch. Thus, with the same image size, the prediction under higher magnification can more accurately represent the tumor feature (probability) at the sampling point (the center of a patch). From the perspective of spatial location, we argue that the prediction of a patch with the same size under 40× is more accurately express the tumor feature at the center of the patch, and facilitate the generation of more detailed segmentation boundaries. Moreover, in the pixel segmentation experiment, the accuracy under the 20× and 40× magnifications alone is better than that under the 10× alone.

Pathologists usually check images by changing their magnification and scope in the WSI. Ways to use images under a wider range of magnifications are worth studying. Inspired by this, we investigated the patch-based classifier with multiple magnifications. We finally chose the combination of patches under 20× and 40× as inputs at the same sampling coordinates.

Our patch-based classifier was derived from DenseNet40 (= 3x6x2+2x1+1+1 = 40). The network consists of three dense blocks defined in DenseNet. Each block consists of 6 dense layers that each contains two convolution layers. Between two adjacent dense blocks, there is a translation layer that consists of one convolution layer. And only two transport layers are used here. The network also contains a convolutional layer at the input and a fully connected layer at the top. We called it the Double Magnification Combination (DMC) patched-based classifier. The network contains two inputs and three outputs, as shown in Fig 3. Our modified network has only 0.3 million parameters. The growth rate (‘k’ in [17]) is set to16 to reduce the parameters of the model by using very narrow layers, at the same time, keeping up with the performance of our patch-based classifier.

Fig 3

DMC patch-based classifier architecture.

To improve accuracy and generalization, the same network was used to process two types of patches under 20× and 40×. The two patches at the corresponding sampling points are put into the same batch in a fixed order. Each three-channel image generated separately a 384-dimensional feature by DenseNet. Because each sampling point includes two input images (patches), we stitch the two-output feature of the same sample point into a 696-dimensional vector and consider it as the fusion feature of the sampling point.

After the output features are continuously stitched and split, the main classifier uses double-magnifications patches at the same sampling point for prediction, while the outputs of the auxiliary classifier under single magnification are only used as additional outputs for training. We used the regularization term to drive the output features of the auxiliary classifier under 20× and 40× to obey the same Gaussian mixture distribution. The final outputs here are 2-dimensional features, not probabilities. The “SoftMax” layer is not included in the network because our adaptive sampling algorithm mainly uses feature space.

Training

We used the training data set with two magnifications to train our patch-based classifier. To encourage the robustness and generalization of our network, we used two loss functions with three prediction outputs (, , and ). Among them, corresponds to the predictions of patches only under 20×, corresponds to the predictions under 40×, and refers to the prediction of the fusion feature. Here, y refers to the ground truth under double magnification, y20 and y40 and so forth.

The first loss function uses only the cross-entropy loss between and y. Here, is the cross-entropy loss function.

Through loss1, the accuracy of the main classifier is improved, and the fusion features under double magnification can be better discovered and utilized. After loss1 is backpropagated, the temporarily generated graph used to compute the gradient of the network needs to be preserved. Next, we perform the backpropagation of the second loss in (2), which consists of the average of cross-entropy loss under single magnification and a regularization term based on the large-margin Gaussian Mixture (L-GM) loss [31]. The regularization term simultaneously drives the deep model to generate the same Gaussian mixture-distributed features under two different magnifications. Because of its use, the generalization capability of the trained model is improved.

Here, the coefficient of the first term is 0.5 indicates that the two cross-entropy losses at both magnifications have the same weight, that is, the classification error is reflected in their average under both magnifications. L-GM loss includes a nonnegative hyper-parameter α for controlling the expected margin between two classes in the training set. And its default value is 1.0 in [31]. We followed this setting. And w is the weight of the regularization term , which is 0.001 by default. Without data amplification, we trained the whole network for 40 epochs with a learning rate of 1 × 10−3 and 40 epochs with a learning rate of 1 × 10−4. The training was performed using PyTorch.

Improved adaptive sampling method

HASHI [7] provided a feasible solution for slide-level scanning and prediction on WSIs. After training a patch-based CNN classifier, HASHI extracts patches from the WSI using Quasi-Monte Carlo sampling and predicts the tumor probabilities of these patches. These predictions are used to build an interpolated probability map, which is used to identify suspicious regions for further sampling. The newly sampled patches are used to produce an improved probability map estimation. The iterative process does not end until the limit of the maximum iterations is reached, and the final probability map is produced.

Our inspiration comes mainly from HASHI, and the main objective of our improved method is to try to optimize the following aspects.

Change in the algorithmic structure. At the initial sampling, a regular sampling process based on superpixel segmentation is added. After an adaptive sampling of the full slide, an iterative process based on the partial superpixels is added.

Change in the selection conditions of the sampling points. The original algorithm sorts by the gradient of the probability map, then select the coordinates within the larger half for sampling. Our algorithm uses a cluster-based heuristic factor to select sampling points based on feature gradients.

Change in the convergence condition. Compared to the maximum number of iterations in the predecessor, we introduced a new statistical factor, local sampling density, to judge whether the iterations should be terminated.

Regular sampling during initialization

For the detection of whole-slide images, the general standard multiple instance assumption needs to be considered. In HASHI, sampling points are more likely to be enriched near a larger area of the tumor region. A larger area has a longer edge, which corresponds to a tumor probability gradient change. The adaptive sampling algorithm preferentially detects these locations where the tumor probability gradient changes are large. In contrast, small tumor areas do not have significant tumor gradient changes, which may result in under-sampling in certain suspicious regions. Also, when the areas of the tumor regions in the WSI are small, the iterative sampling process may untimely terminate due to the limit of the number of iterations. To avoid this, we have to increase the number of maximum iterations or the number of samples per iteration, which means that it is necessary to guarantee a minimum number of samples.

We extracted the thumbnail I of a WSI X under 1.25× magnification (Level 5) and separated it using the SLIC [32] algorithm (compactness = 20). The boundaries B of the segmented superpixel regions S were extracted. Then, we performed regular sampling at uniform spatial intervals on the boundaries of S. Here, the number of superpixels S is proportional to the area of the WSI. The area of each superpixel S is approximately 1000 pixels under 1.25×, which is equivalent to the area of four 256×256 patches under 20×. In this way, a set of center coordinates C of patches is obtained and used to generate the first gradient map in the feature space. Unlike the original algorithm, our gradient maps are based on features rather than probabilities. Because the feature space has a larger dynamic range than the probability space, more edge details are obtained.

Adaptive sampling within full scope

The first stage strategy extracts the random coordinates C of N sampling points using Quasi-Monte Carlo sampling and merges them with the previous regular coordinates C. Here, we chose Halton sequences [33-35] to generate the coordinates of the sampling points. The patches were extracted in pairs under double magnification and put into our two-input classifier. The predications of these patches produced an initial coarse estimation of a linear interpolated feature map M. Then, we generated a gradient map M of the estimated feature map using the Sobel algorithm. Next, Mini Batch K-Means clustering [36] was applied on M to partition the feature gradients into two clusters. At least one of the cluster centers μ0 of gradients is close to zero, which corresponds to flat regions in the gradient map (typical tumor or normal regions in the WSI). If another cluster center μ1 is also close to zero, no significant edges are found in the current M. The edges of M correspond to the regions with large gradient change, that is, the uncertain or suspected tumor regions. The cluster center μ1 corresponding to the possible gradient edge should have a larger value.

Here, we introduced a new heuristic factor f to determine whether the edges of M are found, as shown in (3). The value range of f is from 0 to 0.3.

If f is greater than the threshold T (0.03), our adaptive sampling algorithm only focuses on the position where the feature gradient is greater than f. Otherwise, the sampling algorithm continues to pseudorandomly search sampling coordinates in the full image.

In summary, the generation algorithm of the sampling points is divided into three cases. The first case is to randomly generate sampling points using a Halton sequence in the full scope of a WSI. The second case is that none of the uncertain or suspected tumor regions have been found in M. The generation algorithm pseudorandomly searches sampling coordinates and preferentially selects sampling coordinates with higher gradients in its 16×16 neighborhood under 1.25×. This is equivalent to reducing the size of the gradient map M to the original one-sixteenth size through maximum pooling. The generation algorithm searches for sampling coordinates based on this reduced gradient map to enhance its overall discovery capabilities. The third case is that the generation algorithm focuses on searching for suspicious regions when f is greater than T. At this time, only the coordinates whose corresponding gradient is greater than f will be selected. If the algorithm cannot find enough sample points that satisfy the constraint at once, then it will look again in the neighborhood of the sample points just selected.

Through iterative adaptive sampling, M is continuously refined until the convergence condition of the first sampling stage is reached. Here, we introduced a new statistical factor, local sampling density ρ, which is the number of previous sampling points in the neighborhood of a new sampling point. There are two forms of neighborhoods here. One of them, ρ, is to define the range of the neighborhood by distance. The other, ρ, is defined by belonging to the same superpixel, which is only used in the second stage.

Here, c is the coordinate of the sampling point i in the current iteration. is the coordinate of j in previous iterations. The symbol ‖∙‖ is a distance function, such as Chebyshev Distance. ε indicates the size of the neighborhood of sampling points. s is a superpixel in the segmentation results S.

When the average of ρ of the current iteration is greater than the threshold T, the sampling process will enter the second stage. An adaptive sampling process similar to the first stage is performed in a part of the superpixels. We usually set T to 1 or 2, since setting it to a larger value has little effect on the results but takes more time. See the S1 Appendix for this algorithm pseudocode.

Adaptive sampling within enabled superpixels

Once the average of ρ reaches the threshold T, the adaptive sampling algorithm will further explore regions where the sampling density is low but the tumor probability is high. Therefore, we excluded part of a WSI based on the superpixels obtained earlier. We counted the number of sample points contained in each superpixel, that is, the local sampling density ρ. Based on the interpolation feature map and the gradient map, the two maximums and in the ith superpixel were also calculated. Here, we used three thresholds T, and to determine which regions need further inspection.

Because the output of our binary patch-based classifier is the tumor feature of a patch, if we use the Sigmoid function to regress a feature into a tumor probability, the feature value -1 corresponds to the tumor probability of 27%. Here the threshold represents the lower limit of the tumor feature in a superpixel, generally set to -1. When there is a feature larger than in a superpixel, it means that there is a point with a tumor probability greater than 27% inside. In the next iteration, such superpixels will be further explored. is 0.1; this gradient threshold constrains the regions in S from being too flat. Because such flat regions are generally far from the boundaries of the tumor, too much sampling does not contribute much. ρ(s) indicates whether the superpixel s is fully sampled.

When S is updated, the adaptive sampling process is executed again until the average of ρ reaches T. By iteratively updating S and sampling, S finally becomes an empty set, and the entire algorithm will end. It should be noted that if the two thresholds and are sufficiently small, such as -3 and 0, the entire sampling process will degenerate into uniform sampling.

Fig 4A shows the 112 (purple) sampling points generated in the first round of the adaptive sampling, and 12 of which were generated by regular sampling during initialization. In Fig 4B–4D, the next three rounds of sampling are shown here, and the 4th round reached the termination condition T. The sampling points were densely generated where the predicted tumor probability exceeds 0.3 and they had a larger gradient in the estimated feature space. Fig 4E shows the contour lines of the tumor probability map with different colors. The red line indicated the ground truth.

Fig 4

The process of the adaptive sampling.

Please refer to the S1 Appendix for the detailed process of the sampling algorithm.

Algorithm 1: Adaptive gradient-based sampling

Input:

: CNN-trained model

X: WSI

T: maximum iterations

N: number of sample points extracted

A: area of each superpixel

d: spaced intervals of sampling

T, T, and : thresholds

M, f, ← ϕ

S, C ← regular sampling based on superpixels (X,A,d)

S = S

For i = 1 to in T do:

C ← sampling point generation (N, M, f, T, S,…)

C =

Predictions ← patch classification (, C)

M ← feature map interpolation (, C)

M ← feature gradient (M)

μ0, μ1 ← clustering (M)

f = min(0.5*(μ0+μ1),0.3)

← average local sampling density within the neighborhood ()

←

If :

ρ ← local sampling density within superpixels ()

S ← update enabled regions ()

If S is ϕ:

Return

Here, C refers to the set of center coordinates of patches, which are obtained by regular sampling based on superpixel segmentation. C refers to the set of center coordinates of patches, which are obtained by random sampling (Quasi-Monte Carlo) process. We only returned the coordinates c and predictions f (1-dim feature) of each sample point in all iterations. Next, the postprocessing generates a heat map of tumor probability.

In our experiments, a slide was required to extract nearly 59000 patches of size 256×256 on average using uncovered regular sampling under 20×. Our sampling method with parameters T = 1 and N = 2000 only needed to extract an average of 7400 patches, which is only 1/8 of the workload of the uncovered regular sampling.

Postprocessing

In postprocessing, each obtained prediction is adjusted based on the predictions of its neighboring sampling points. A CNN model with 4 convolutional layers was trained to regulate the patch-level predictions under 1.25×, as shown in Table 1. We can think of this as adaptive filtering of patch-level features, so we also called it a slide filter. The input of the slide filter is a 64×64 single-channel matrix centered at each sample point, which includes the feature of its adjacent sampling points. If the sample point is in a tumor region, it is a tumor/positive sample and labeled as 1; otherwise, it is a normal/negative sample and labeled as 0. Corresponding to 20×, the size of the input matrix is 1024×1024 pixels, and the area is equivalent to the 16 non-overlapping patches used in the patch-based classifier.

Table 1

The slide filter (CNN model) in postprocessing.

Layer (type)	Output Shape	Param
Conv2d+ReLU	[32, 64, 64]	320
Conv2d+ReLU+MaxPool2d	[32, 32, 32]	9,248
Conv2d+ReLU+MaxPool2d	[48, 16, 16]	13,872
Conv2d+ReLU+MaxPool2d	[64, 8, 8]	27,712
AvgPool2d	[64, 1, 1]	0
Linear	[2]	130
Total params		51,282

According to the pathologist’s annotations and the obtained predictions, we generated a training set of 188360 balanced samples (the ratio of normal to tumor samples is 1:1). The loss function used in training consists of two parts: cross-entropy loss and L-GM loss. The network was trained on patches of shape = 64×64 pixels, with batches of size = 200, and weight of L-GM loss = 0.001.

The corrected patch-level predictions are generated by a weighted average with the new patch-level predictions and the original prediction. Then, according to the corrected predictions and sampling coordinates, a tumor probability heat map is generated by the Sigmoid function and linear interpolation. Here, we did not use fully-conv (FC) net to directly generate a heatmap under 1.25×, because this required higher hardware requirements (GPU memory capacity).

We used the tumor probability heatmap to compute the evaluation for each WSI. In Fig 5, the contour lines of the probability maps are shown in (A). (C) is a partial enlargement of the lower right corner of (A). Here, the contours with different colors correspond to different probabilities. (B) and (D) show the predictions corresponding to the left side using our adaptive sampling method. The yellow regions in (B) and (D) indicate the ground truth. We gave more examples in the S1 Appendix.

Fig 5

Tumor probability heatmap and predictions of sampling points.

Results & discussion

This paper used the H&E-stained WSIs of the Camelyon16 challenge, which is aimed at detecting metastasis on the WSIs of lymph node sections [6], and 40 H&E-stained WSIs provided by Hubei Cancer Hospital (HCH). We used these datasets to detect tumor regions. The tumor regions in the HCH dataset are generally large and typical, as shown in Fig 6. In the two subplots, the blue regions are the marked tumor regions. In Fig 6B, a green region is the excluded region. In terms of the number of tumor regions per slide, the test set samples in Camelyon16 contain an average of 33 compared to an average of 11 for the HCH samples.

Fig 6

Examples of WSI in the two datasets.

Evaluating the patch-based classifier

In this section, we mainly evaluate the performance of the patch-based classifier. To compare the classification performance of different networks, we used the prepared dataset to evaluate their patch-level F1 scores, as shown in Table 2. Here, “40×” refers to the DenseNet-40 with a single input under 40×. “DMC” refers to our modified DenseNet-40 with two magnification inputs, “DMC 40×” refers to the auxiliary classifier only for 40×, “DMC 40×+20×” refers to the main classifier using fusion features under two magnifications. “L-GM” refers to the L-GM loss used in training. From the patch-level results of the classifier, the performance of the auxiliary classifier under 40× of DMC is similar to the single-input classifier under 40×, and that of the auxiliary classifier under 20× is better than the corresponding single-input classifier. When the fused features under both magnifications are used at the same time, the performance was improved by nearly 2~3%. Because a pair of patches overlap at the center point and the field of view is different, so the spatial attention mechanism was introduced. For the performance of patch-level detection, our experience is the use of L-GM loss in training has no significant effect on single-input or dual-input classifiers.

Table 2

The classifier detection performance.

Methodology		Patch Level						Pixel Level
		Train			Test			Train		Test
		F1(Normal)	F1(Tumor)	F1(Avg)	F1(Normal)	F1(Tumor)	F1(Avg)	F1	AUC	F1	AUC
40×		0.9546	0.9548	0.9547	0.9711	0.8827	0.9269	0.5170	0.9616	0.4595	0.9030
20×		0.9567	0.9570	0.9568	0.9701	0.8805	0.9253	0.5738	0.9286	0.5036	0.8571
DMC	40×	0.9512	0.9514	0.9513	0.9711	0.8844	0.9277	-	-	-	-
	20×	0.9712	0.9713	0.9712	0.9802	0.9242	0.9522	-	-	-	-
	40×+20×	0.9737	0.9740	0.9738	0.9810	0.9275	0.9542	0.6007	0.8454	0.5518	0.8630
DMC+L-GM	40×	0.9517	0.9524	0.9520	0.9714	0.8872	0.9293	-	-	-	-
	20×	0.9702	0.9702	0.9702	0.9811	0.9277	0.9544	-	-	-	-
	40×+20×	0.9723	0.9725	0.9724	0.9815	0.9296	0.9556	0.7111	0.9681	0.6121	0.9279

Next, we evaluated the performance of pixel-level detection, and this evaluation more tested the generalization capability of patch-based classifiers. Using the pathologist’s annotation as the ground truth, ROC analysis at the pixel level heat map was performed, and the measures used for comparing the algorithms were F1 score and area under the ROC curve (AUC). In Table 2, the pixel level detection results all used our proposed adaptive sampling algorithm, and the parameter configuration for our model involved the threshold T of the local sampling density (T = 1) with 2000 samples per iteration (N = 2000) and the area of each superpixel A = 1000 pixels with spatial intervals of regular sampling d = 60 pixels under 1.25× magnification. Besides, the pixels with tumor probability greater than 0.5 were considered positive in the heat map.

Regarding the F1 scores in Table 2, there is not much difference in accuracy between single-input or dual-input patch-based classifiers, but there is a significant difference in the results of pixel-level segmentation. In the pixel-level segmentation task, the F1 scores of each patch-based classifier are much lower than the scores of the classifier during training and testing of patches. Note that, the loss of the pixel-level segmentation task is not used to optimize the performance of patch-based classifiers; it represents the generalization performance of the classifier. This is because, although we extracted millions of patches from WSI for training these patch-based classifiers, the input images during our adaptive random sampling are almost impossible to be the same as those in the training set. In other words, the patches extracted during the adaptive sampling are more varied. Moreover, the prediction error at any sampling point has an impact on the accuracy of the segmentation boundary near it. The superposition effect brought by the sampling mechanism makes it possible to obtain correct results only when the robustness of the patch-based classifier is sufficient. Regarding the F1 score of pixel level, the performance of our adaptive sampling algorithm on DMC is nearly 20% higher than that of the classifiers with single magnification. This shows the advantages of the dual input structure.

From the detection results of the pixel level, L-GM loss is necessary for DCM. The use of L-GM loss increases the margin between the centers of the two classes, During the adaptive sampling process, more the features of sampling patches fall into this gap, resulting in the deterioration of the detection results. The contours of the probability of 0.5 at the heat map generated by DMC are closer to the ground truth. Regarding ROC AUC of pixel level, the heat map generated by DMC with L-GM loss is also the best. On the WSIs of the training set, DMC performs similarly to the single input classifier under 40×. But the score of DMC is 2.5% higher than the classifier under 40× on the WSIs of the test set. We think that the higher the AUC, the better the selection and prediction of sampling points.

Evaluating adaptive sampling algorithms

In this section, we mainly evaluate the performance of the sampling algorithms by comparing the probability heatmaps using the same DMC. As before, the measures used for comparing the algorithms were the F1 score and AUC at the pixel level.

Table 3 shows the pixel detection performance comparison between HASHI and our sampling method on tumor samples. In the parameters of HASHI, the number of samples per iteration was fixedly set to 400, and the maximum iterations T was set to 20, 30, and 40, respectively. In our sampling method, we evaluated both non-post-processing and post-processing (The slide filter was marked as ‘SF’ in Table 3). We report the average F1 score and AUC for these approaches with the Camelyon16 and HCH datasets. Here the experiment WSIs were divided into three groups: Camelyon16 Train, Camelyon16 Test, and HCH Test. The patches for training the DMC classifier were extracted from WSIs of training data of Camelyon16. In other words, a part of the sampled patches may exist in the training set. So, the classifier had higher classification performance for such WSIs. The other two datasets were never seen by the patch-based classifier.

Table 3

The pixel-level detection performance on different sampling algorithms with DMC classifier.

Methodology	Camelyon16 Train		Camelyon16 Test		HCH Test		Number of sampling points
	F1	AUC	F1	AUC	F1	AUC
Our method	0.7111±0.1839	0.9681±0.0782	0.6121±0.2631	0.9279±0.1028	0.6999±0.2041	0.9342±0.0485	7402±2028
Our method2*	0.7113±0.1813	0.9782±0.0524	0.6173±0.2720	0.9527±0.0819	0.7439±0.1929	0.9577±0.0342	7402±2028
HASHI T = 20	0.5879±0.2633	0.9393±0.0978	0.5695±0.3090	0.8782±0.1815	0.6810±0.2269	0.9451±0.0424	8000
HASHI T = 30	0.6129±0.2682	0.9660±0.0491	0.5425±0.3317	0.8451±0.2259	0.6844±0.2257	0.9415±0.0447	12000
HASHI T = 40	0.6424±0.2379	0.9690±0.0425	0.5574±0.3122	0.8787±0.1864	0.6832±0.2311	0.9409±0.0454	16000

*Our method2 is the method using Slide Filter in postprocessing.

Compared to the F1 score of the patch level, the score of the pixel level did have a significant decline. On the other hand, AUC at pixel level was still relatively high, and that of our method exceeded 0.95 on all datasets. Because the DMC classifier was trained on labeled patches and had not been trained using pixel-level labeled data on WSIs. Therefore, the probability heatmap is better in overall probability prediction, but the contours of the probability of 0.5 at the heat map were still a little bit different from the ground truth.

For HASHI, when the accuracy of the classifier is sufficient on Group Camelyon16 Train, both F1 and AUC will increase as the number of sampling points increases. On the other verification groups, F1 and AUC did not improve even if the number of sampling points was doubled. Because for the verification groups, the F1 score of the tumor patches was 0.9296, which was 0.04 lower than that of the training set in Table 2.

Compared with HASHI, our proposed adaptive sampling method has better results. The F1 and AUC of our method with post-processing are the highest of all tests. Our F1 score is at least 5.8% higher than its predecessors, and AUC is at least 3.2% higher than that. Regarding the slide filter, the post-processing has a 1.6% improvement on F1 and 1.9% on AUC. It is worth noting that our F1 and AUC without slide filter were not significantly different from HASHI on Group HCH. This was not the case with Group Camelyon16 Test. This is because the area of the tumor regions in each slide of HCH is on average 8 to 9 times larger than Camelyon16, but the number of regions is generally relatively small. In other words, the detection target is relatively significant. Therefore, HASHI is more suitable for the detection of such WSIs, and our proposed method can detect more and smaller tumor regions.

Two evaluation in Camelyon16

In this section, we briefly present two evaluation results in Camelyon16: Slide-based Evaluation and Lesion-based Evaluation.

Slide-based evaluation

This evaluation task is to distinguish between slides containing metastasis and normal slides and rank them by the area under ROC curve (AUC) [6]. For the slide-based classification task, the postprocessing method takes a prediction result for each WSI as input and produces a single probability of tumor for the entire WSI as output. Here, we extracted 5 statistical features from the positive part of the predictions , whose tumor probability is greater than 0.5. These features included the number of sample points that meet the probability requirements, the maximum tumor probability among them, and a normalized histogram with three bins based on these tumor probabilities. We computed these features over the predictions across all cases, and we trained and compared 4 classifiers to discriminate whether a WSI includes tumor regions. The merits of the algorithms will be assessed for discriminating between slides containing metastasis and normal slides. Receiver operating characteristic (ROC) analysis at the slide level will be performed, and the measure used for comparing the algorithms will be the area under the ROC curve (AUC) [37]. On the independent test cases, the Lagrangian-based S3VM [38, 39] model achieved an AUC of 0.9920, as shown in Fig 7. Our score is very close to the score (0.9935) of the top-ranked team on the leaderboard of the Camelyon16 ISBI challenge [37].

Fig 7

Receiver Operating Characteristic (ROC) curve of slide-based classification.

Lesion-based evaluation

The second evaluation task is to test the detection/ localization performance, which is summarized using free-response operating characteristic (FROC) curves [37]. For lesion-based detection, a pair probability and corresponding coordinate of each predicted cancer lesion within the WSI need to be given with few false positives. Our approach is similar to [40], which used a non-maxima suppression method. In contrast, we used the Isolation Forest algorithm [41] and the K-means (K = 2) clustering to find automatic segmentation thresholds for a tumor probability heatmap.

The FROC curve is defined as the plot of sensitivity versus the average number of false positives per image [37]. As shown in Fig 8, our method achieved a score of 0.7694 at 1 FP per WSI on the training cases and a score of 0.7373 at 1 FP on the test cases. Table 4 shows the comparison with other methods using Camelyon16. Our score has reached the level of human performance. However, there is still a large gap between the current best score (0.8533), which was achieved by Fast ScanNet [42]. Fast ScanNet used a fully convolutional network without an up-sampling path to generate a probability heatmap with a much smaller size than the input image, then performed a dense scan on ROIs and stitched the predictions into a complete heatmap. The label of the patch-based training sample of Fast ScanNet was pixel-level and our classifier used patch level, it is not difficult to understand that the former performed well in FROC of the pixel-level detection. From our heat map, our proposed method usually combines multiple smaller tumor regions into one larger region for reporting. In the FROC measurement, this caused many small tumor regions to be detected but not reported. Another phenomenon is that the F1 score is not high (<0.75) and the AUC is high (>0.95) in Table 3.

Fig 8

FROC curve of the lesion-based detection.

Table 4

Detection performance comparison with Camelyon16.

Team	AUC	FROC
Human performance	0.9660	0.7325
HMS and MIT	0.9935	0.8074
Our method	0.9920	0.7373
Fast ScanNet-16	0.9875	0.8533
HMS, Gordon Center, MGH	0.9763	0.7600
CUHK	0.9415	0.7030
EXB Research	0.9156	0.5111
DeepCare, Inc.	0.8833	0.2430
Middle East Tech. Uni.	0.8632	0.3822
NLP LOGIX Co.	0.8298	0.3859
Smart Imaging Tech. Co.	0.8207	0.3385
Univ. of Toronto	0.8149	0.3822
Radboud Uni.	0.7786	0.5748

On the other hand, Fast ScanNet still needs to completely scan the entire area to be detected, but our adaptive sampling algorithm does not need to do this. At the same time, Fast ScanNet used an FC net to generate heat maps, and a large number of large-size (2866×2866) feature maps were produced during the convolution process. Therefore, it put forward higher requirements for the memory capacity of GPU. And our method only needs to use 256×256 input images, as long as your computer can run PyTorch, you can complete the WSI detection task. When computing resources are limited, our proposed algorithm is a feasible and effective method.

Next, we discuss this issue in detail in the next section.

Model runtime efficiency

Due to the use of a lightweight network, the computational complexity is relatively low, and approximately 286.2 pairs of double-magnification patches can be predicted per second. The performance test was performed on a PC with a 3.2 GHz Intel i7-8700 CPU with 16 GB of memory and an NVIDIA GeForce GTX 1080 8 GB.

The core of our proposed model is the DMC classifier, which is called thousands of times. However, it only contains 306,498 parameters. Compared with the patch-based classic network, the parameter size of VGG16 is 460 times that of our model, the size of GoogLeNet is 80 times, the size of ResNet-50 is 85 times, and the size of DenseNet-121 is 27 times. Therefore, the DMC classifier only takes one second to predict nearly 300 pairs of 256×256 patches from the saved small JPG files.

A heat map of a WSI under 1.25× contains an average of 15.1 million pixels. When a full dense scan of a WSI is performed at equal intervals without coverage under 20×, it is necessary to extract and predict approximately 59,000 patches of size 256×256. As shown in Table 3, when the number of samples per iteration (400) was fixed in HASHI, the number of extracted patches of per WSI is directly proportional to the maximum iterations T. When T was equal to 20, HASHI here needed to predict 8000 samples, which accounted for 13.6% of the number of the full dense scan. Our algorithm uses a cluster-based heuristic factor to select sampling points, instead of selecting the larger half ones of the gradient in HASHI. For scanning a slide, the number of patches extracted by our sampling algorithm is different for each slide. Typically, about 7400 samples are extracted for each slide by our method, which accounted for 9.1 ~ 15.9% of the number of the full dense scan. The computational complexity using the local sampling density (T = 1) is equivalent to HASHI using the parameter T = 20. In Table 3, the detection results of our method are better. On the WSIs of the training set, the F1 score and AUC are 12.3% and 3.9% higher than HASHI. On the WSIs of the two test sets, the F1 score and AUC have improved by 5.5% and 4.3% on average compared to HASHI.

In time consumption, our method usually takes 2 to 3 minutes to complete a WSI with i7 CPU and single GTX 1080 8 GB. Our proposed method reduces the detection area by at least 85% in the adaptive sampling manner and saves the computing load of each sampling point with the lightweight network. Through the divide-and-conquer approach, the need for the memory capacity of GPU is drastically reduced, and at the same time, detection effects can meet the needs of preliminary screening in clinical diagnosis.

Conclusion

We proposed a novel lightweight detection framework for automatic tumor detection in whole-slide histopathology images. Compared to classic CNN models, our DMC model with dual inputs and three outputs is easier to train, with higher computational efficiency with only 0.3 million parameters. Our improved adaptive sampling method uses a new heuristic factor as the convergence condition of iterations for improving the detection performance of multiple instances, which is only 1/8 of the workload of the uncovered regular sampling. In post-processing, the patch-level predictions are regulated based on the predictions of adjacent sampling points to improve the pixel level and lesion level accuracy. Our experiments revealed that our method also has reached the state of the art on the pixel level and lesion level detection of gigapixel pathology slides with limited computing resources. In clinical practice, the ability to use more computer resources for detecting whole-slide images will greatly promote the practical application of automatic diagnostic technology.

With the continuous popularization of breast cancer screening, more and more early-stage breast cancers containing carcinoma in situ have been discovered. On whole-slide images, how to accurately identify the presence and proportion of carcinoma in situ and invasive cancer is extremely important for selecting the appropriate treatment and the best benefit for the patient. In future work, we aim to study region detection of carcinoma in situ and invasive cancer. We could explore a new clustering model for encoding histology WSI to analyze the texture features on tissue structure in a larger field of view.

(DOCX)

Click here for additional data file.

4 Mar 2021

PONE-D-21-02087

A fast and effective detection framework for Whole-Slide Histopathology Image analysis

PLOS ONE

Dear Dr. Yue,

Thank you for submitting your manuscript to PLOS ONE. After careful consideration, we feel that it has merit but does not fully meet PLOS ONE’s publication criteria as it currently stands. Therefore, we invite you to submit a revised version of the manuscript that addresses the points raised during the review process.

The manuscript had been reviewed by 2 reviewers. Reviewer 1 was of the view that manuscript partly describes a technically sound piece of scientific research and recommended major revision. Reviewer 2 was of the opinion that your manuscript describe technically piece of scientific research, however, he had made certain observations and also recommended major revision.

After thorough consideration of comments of Reviewer 1 and Reviewer 2, my decision is "major revision". Please incorporate comments raised by both reviewers.

: I have noted that one of the reviewers has asked for more context in the literature review, and suggested specific papers to be cited. While you may take on-board their suggested papers if you feel that they are relevant for your manuscript, or just take on-board the general suggestion for providing some more context in the literature review, there is no requirement from the journal to cite these papers

Please submit your revised manuscript by Apr 18 2021 11:59PM. If you will need more time than this to complete your revisions, please reply to this message or contact the journal office at plosone@plos.org. When you're ready to submit your revision, log on to https://www.editorialmanager.com/pone/ and select the 'Submissions Needing Revision' folder to locate your manuscript file.

Please include the following items when submitting your revised manuscript:

A rebuttal letter that responds to each point raised by the academic editor and reviewer(s). You should upload this letter as a separate file labeled 'Response to Reviewers'.

A marked-up copy of your manuscript that highlights changes made to the original version. You should upload this as a separate file labeled 'Revised Manuscript with Track Changes'.

An unmarked version of your revised paper without tracked changes. You should upload this as a separate file labeled 'Manuscript'.

If you would like to make changes to your financial disclosure, please include your updated statement in your cover letter. Guidelines for resubmitting your figure files are available below the reviewer comments at the end of this letter.

If applicable, we recommend that you deposit your laboratory protocols in protocols.io to enhance the reproducibility of your results. Protocols.io assigns your protocol its own identifier (DOI) so that it can be cited independently in the future. For instructions see: http://journals.plos.org/plosone/s/submission-guidelines#loc-laboratory-protocols

We look forward to receiving your revised manuscript.

Kind regards,

Gulistan Raja

Academic Editor

PLOS ONE

Journal Requirements:

When submitting your revision, we need you to address these additional requirements.

1. Please ensure that your manuscript meets PLOS ONE's style requirements, including those for file naming. The PLOS ONE style templates can be found at

https://journals.plos.org/plosone/s/file?id=wjVg/PLOSOne_formatting_sample_main_body.pdf and

https://journals.plos.org/plosone/s/file?id=ba62/PLOSOne_formatting_sample_title_authors_affiliations.pdf

2. Please include captions for your Supporting Information files at the end of your manuscript, and update any in-text citations to match accordingly. Please see our Supporting Information guidelines for more information: http://journals.plos.org/plosone/s/supporting-information.

[Note: HTML markup is below. Please do not edit.]

Reviewers' comments:

Reviewer's Responses to Questions

Comments to the Author

1. Is the manuscript technically sound, and do the data support the conclusions?

The manuscript must describe a technically sound piece of scientific research with data that supports the conclusions. Experiments must have been conducted rigorously, with appropriate controls, replication, and sample sizes. The conclusions must be drawn appropriately based on the data presented.

Reviewer #1: Partly

Reviewer #2: Yes

**********

2. Has the statistical analysis been performed appropriately and rigorously?

Reviewer #1: Yes

Reviewer #2: Yes

**********

3. Have the authors made all data underlying the findings in their manuscript fully available?

The PLOS Data policy requires authors to make all data underlying the findings described in their manuscript fully available without restriction, with rare exception (please refer to the Data Availability Statement in the manuscript PDF file). The data should be provided as part of the manuscript or its supporting information, or deposited to a public repository. For example, in addition to summary statistics, the data points behind means, medians and variance measures should be available. If there are restrictions on publicly sharing data—e.g. participant privacy or use of data from a third party—those must be specified.

Reviewer #1: Yes

Reviewer #2: Yes

**********

4. Is the manuscript presented in an intelligible fashion and written in standard English?

PLOS ONE does not copyedit accepted manuscripts, so the language in submitted articles must be clear, correct, and unambiguous. Any typographical or grammatical errors should be corrected at revision, so please note any specific errors here.

Reviewer #1: No

Reviewer #2: No

**********

5. Review Comments to the Author

Please use the space provided to explain your answers to the questions above. You may also include additional comments for the author, including concerns about dual publication, research ethics, or publication ethics. (Please upload your review as an attachment if it exceeds 20,000 characters)

Reviewer #1: In this paper, a “A fast and effective detection framework for Whole-Slide Histopathology Image analysis” is introduced, where a Double Magnification Combination (DMC) classifier, which is a modified DenseNet-40 to make patch-level predictions with only 0.3 million parameters. The detailed comments are given below from nine respects:

1. Language: The language is OK for me to understand the authors idea. However, there are still many grammar problems, such as Line 29, 30, 31, 35, 41-42, 46, 81-82, 97, 102, 105-107. Except these problems, there are still many others to improve. In total, the language quality of this paper is very poor and need to improve quite a lot for a final publication.

2. Application value This work focus on the histopathology image analysis, which is meaningful to support the cancer clinick work in practical. This is good.

3. Scientific value and novelty: The scientific value of this work is somehow limitied by the novelty of this work. In my opinion, although it said that this work developed some new method, it is a combination of some exsiting famous methods in the histopathology image analysis domain. Please explain more detailes: (1) in Line 134, what is the “simple threshold-based segmentation method”? (2) in Line 155-156, “For the choice of patch-based classifier structure, we chose nine classic pre-trained ImageNet networks to test the patch-based 156 classifier under three magnifications”, what is the motivation to use these methods? (3) in Line 162-163, “Our modified network has only 0.3 million parameters with 3 dense blocks defined in DenseNet, and 163 each block consists of 12 convolution layers. The growth rate (‘k’ in [17]) is 16”, what did authors use Densenet40? Due to the normally used structure Densenet is Densenet41, lease give the structure of Densenet40. Why did the authors set growth rate k=16?

4.Experimental data: based on my research experiences in the histopathology image analysis domain, the quality and data size in this paper is OK.

5. Experimental result: in this paper, a large number of contrast experiments were carried out to show the effectiveness of the new method, and good experimentals results were obtained.

6. Figure quality: Figure 1-6 have low quality to improve.

7. Table quality: the layouts of some tables are uncomfortable to read. Please update.

8.Euqation quality: I can understand most of the equations in this paper, and did not find obvious problems. However, in Line 181-199, these two loss functions are well-known, so the authors do not need to introduce them so detailed.

9. Reference quality: most references are ok, but some are too old and some new papers should be read and added.

[1] Gastric histopathology image segmentation using a hierarchical conditional random field （2020）

[2] RMDL: Recalibrated multi-instance deep learning for whole slide gastric image classification （2019）

[3] Superpixel-Based Conditional Random Fields (SuperCRF): Incorporating Global and Local Context for Enhanced Deep Learning in Melanoma Histopathology （2019）

[4] A review for cervical histopathology image analysis using machine vision approaches (2020)

[5] A Cervical Histopathology Image Clustering Approach Using Graph Based Features (2020)

[6] A Comprehensive Review of Markov Random Field and Conditional Random Field Approaches in Pathology Image Analysis (2021)

[7] Computerized Spermatogenesis Staging (CSS) of Mouse Testis Sections via Quantitative Histomorphological Analysis (2020)

Reviewer #2: Dear authors,

the research work is very interesting. The appendix is very useful. Globally, we miss the big picture in favor of a lot of technical details that should be better organized for a general audience. The methodology is basic but you bring good performances for time computing and recognition scores as well (on par with human discrepancy)

My detailed remarks /questions follow with citations of your text :

- 80 000 x 80 000 at 40x : 2G.

I am surprised with the size at 40x I would say 20 G ?

“Because the patches at the edge of annotations are usually transitional regions from tumor tissue to normal tissue, most of the “hard examples” are concentrated in these positions. The same is true for the labeling of patch pairs under double magnifications”

Are you sure ? For me, the border of the tumoral areas are quite visually marked even I agree there is a transition that makes it more interesting to be precise from a computational point of view.

Here, the sampling interval in the normal regions is larger than that in the tumor region.

Here stands for where ? What do you mean ?

because the prediction of a patch with the same size under 40× is more accurate for the center of the patch. ??

Rephrase it or explain it more clearly

The structure of the patch-based classifier was derived from DenseNet-40, so we called it the Double Magnification Combination (DMC) patched-based classifier.

So ?? I do not see the logical implication ?

The growth rate (‘k’ in [17]) is 16.

is set to 16 and explain what it is

Here, y refers to the ground truth under double magnification, y 20 and y 40 and so forth.

The prediction is the same in classif, isn’t it ?

Here, α is 1.0, and w is the weight of the L-GM loss, which is 0.001 by

explain alpha et w why 0,5 and w if w is set to a fixed value. It complexity the method for no added value. Or you make w varying ?

The predications ?

:-) predictions of course

Ca and Cr a and r stands for what

Globally as there is a lot of parameters make it clear the choice of subscript

Here, T f represents the lower limit of the tumor feature with a superpixel, generally set to -1, which indicates that a superpixel within a tumor probability of more than 27% should be considered

Make it clearer please??

Fig 4 is not very clear ?

the 12 points for instance

Here, we did not use fully-conv (FC) net to directly generate a heatmap under 1.25×, because this requ ired higher hardware requirements (GPU memory capacity).

But you use a GPU in the results ? (see “The performance test was performed on a PC with a 3.2 GHz Intel i7-8700 CPU with 16 GB of memory and an NVIDIA GeForce GTX 1080 8 GB”.)

When the features under both magnifications are used at the same time in DMC, the performance was improved by nearly 2~3%. Because a pair of patches overlap at the center point and the field of view is different, so the spatial attention mechanism was introduced. For the performance of patch-level detection, our experience is the use of L-GM loss in training has no significant effect on single-input or dual-input classifiers.

Results interesting : x20 seems to be enough

Is it very useful to add ‘0x then. ?

Although our training and testing sets contain millions of patches extracted from WSIs, the patches extracted during the adaptive sampling process were still more complex.

I do not understand the logical articulation.

Table 3 interesting

HCH very different from Camelyon : can you elaborate on it a bit more as you started to do in the following lines.

This is because the area of the tumor regions in each slide of HCH is on average 8 to 9 times larger than Camelyon16, but the number of regions is generally relatively small. In other words, the detection target is relatively significant. Therefore, HASHI is more suitable for the detection of such WSIs, and our proposed method can detect more and smaller tumor regions.

For instance, show HCH Images and the difference with Camelyons

Therefore, it put forward higher requirements for the memory capacity of GPU. And our method only needs to use 256×256 input image, as long as your computer can run PyTorch, you can complete the WSI detection task. When computing resources are limited, our proposed algorithm is a feasible and effective method.

The core of our proposed model is DMC classifier, which is called thousands of times.

Therefore, the DMC classifier only takes one second to predict nearly 300 pairs of 256×256 patches from the saved small JPG files.

This part is useful for the reproducibility. Many thanks.

Globally improve the quality of rendering of the Figures (resolution I guess ?)

**********

6. PLOS authors have the option to publish the peer review history of their article (what does this mean?). If published, this will include your full peer review and any attached files.

If you choose “no”, your identity will remain anonymous but your review may still be made public.

Do you want your identity to be public for this peer review? For information about this choice, including consent withdrawal, please see our Privacy Policy.

Reviewer #1: No

Reviewer #2: No

[NOTE: If reviewer comments were submitted as an attachment file, they will be attached to this email and accessible via the submission site. Please log into your account, locate the manuscript record, and check for the action link "View Attachments". If this link does not appear, there are no attachment files.]

While revising your submission, please upload your figure files to the Preflight Analysis and Conversion Engine (PACE) digital diagnostic tool, https://pacev2.apexcovantage.com/. PACE helps ensure that figures meet PLOS requirements. To use PACE, you must first register as a user. Registration is free. Then, login and navigate to the UPLOAD tab, where you will find detailed instructions on how to use the tool. If you encounter any issues or have any questions when using PACE, please email PLOS at figures@plos.org. Please note that Supporting Information files do not need this step.

10 Apr 2021

Reviewer#1, Concern # 1: in Line 134, what is the “simple threshold-based segmentation method”?

Author response: For excluding the obvious background region, we apply three fixed-level thresholds in HSV to the thumbnail image of WSIs by experience. In the S channel, its value is required to be greater than 0.1. The value of the V channel is required to be between 0.2 and 0.8. These have been introduced in Line 135-136 of the original paper.

Reviewer#1, Concern # 2: in Line 155-156, “For the choice of patch-based classifier structure, we chose nine classic pre-trained ImageNet networks to test the patch-based classifier under three magnifications”, what is the motivation to use these methods?

Author response: We tried to find a suitable classifier structure using transfer learning. The features of patches were extracted by nine pre-trained models. And we replaced the original top layer with a new one to connect each feature extraction part, which consists of a Global Average Pool (GAP) and two fully-connected layers. We used the prepared patches on three different magnifications to fine-tuning the top layer of each transfer model and tested the accuracy of these 18 models. All testing results of transfer-learning are shown in Supplementary Table 1. According to the results of transfer-learning, the DenseNet family has the best performance of feature extraction for pathological image blocks. Therefore, we construct our network based on DenseNet.

Supplementary Table 1 Accuracy of patch-based classification on different magnification using different Transfer models

Model name Parameters

(M) Image size Binary Accuracy

10× 20× 40×

Inception_v3 23.9 224x224x3 0.9231 0.9573 0.8835

DenseNet121 8.1 299x299x3 0.9329 0.9640 0.8946

DenseNet169 14.3 299x299x3 0.9273 0.9656 0.8978

DenseNet201 20.2 299x299x3 0.9329 0.9651 0.8946

ResNet50 25.6 299x299x3 0.8604 0.9300 0.8146

Inception_ResNet_v2 55.9 224x224x3 0.9218 0.9599 0.8904

VGG16 138.4 299x299x3 0.9186 0.9537 0.8757

MobileNet_v2 3.5 299x299x3 0.9063 0.9506 0.8752

NASNet(mobile) 5.3 299x299x3 0.9100 0.9595 0.8849

We revised the manuscript accordingly, and the above table has been added to the supplementary materials.

Reviewer#1, Concern # 3: in Line 162-163, “Our modified network has only 0.3 million parameters with 3 dense blocks defined in DenseNet, and 163 each block consists of 12 convolution layers. The growth rate (‘k’ in [17]) is 16”, what did authors use Densenet40 ? Due to the normally used structure DenseNet is Densenet41, lease give the structure of Densenet40. Why did the authors set growth rate k=16?

Author response: Our network consists of three dense blocks defined in DenseNet. Each block consists of 6 dense layers that each contains two convolution layers. Between two adjacent dense blocks, there is a translation layer that consists of one convolution layer. And only two transport layers are used here. The network also contains a convolutional layer at the input and a fully connected layer at the top. Therefore, we refer to this network architecture as DenseNet40 (=3x6x2+2x1+1+1=40).

Here we set the growth rate k to 16 to reduce the parameters of the model by using very narrow layers, at the same time, keeping up with the performance of patch classification.

In the corresponding section of the revised manuscript, we provide a detailed description.

Reviewer#1, Concern # 4: in Line 181-199 , these two loss functions are well-known, so the authors do not need to introduce them so detailed.

Author response: We very much appreciate the comments. We have trimmed this section.

Reviewer#1, Concern # 5: most references are ok, but some are too old and some new papers should be read and added.

[1] Gastric histopathology image segmentation using a hierarchical conditional random field （2020）

[2] RMDL: Recalibrated multi-instance deep learning for whole slide gastric image classification （2019）

[3] Superpixel-Based Conditional Random Fields (SuperCRF): Incorporating Global and Local Context for Enhanced Deep Learning in Melanoma Histopathology （2019）

[4] A review for cervical histopathology image analysis using machine vision approaches (2020)

[5] A Cervical Histopathology Image Clustering Approach Using Graph Based Features (2020)

[6] A Comprehensive Review of Markov Random Field and Conditional Random Field Approaches in Pathology Image Analysis (2021)

[7] Computerized Spermatogenesis Staging (CSS) of Mouse Testis Sections via Quantitative Histomorphological Analysis (2020)

Author response: Thank you for the suggestion about these references. We updated the introduction section by explaining the recent studies related to WSI analysis, as suggested by the reviewer. Since the manuscript focuses on the classification and localization of tumor regions at the pixel level on a WSI, we do not cite all the above literature. We hope that the reviewer will be satisfied.

Reviewer#1, Concern # 6: Figure 1-6 have low quality to improve. And the layouts of some tables are uncomfortable to read. Please update.

Author response: Thank you very much for the feedback about the quality of the figures and tables. We will try our best to improve.

Reviewer#1, Concern # 7: There are still many grammar problems, such as Line 29, 30, 31, 35, 41-42, 46, 81-82, 97, 102, 105-107. Except these problems, there are still many others to improve.

Author response: We very much appreciate the comments. We tried to revise the paper. We hope the English grammar would be better now. We also hope that we answer the reviewer questions properly.

Reviewer#2, Concern # 1: 80 000 x 80 000 at 40x : 2G. I am surprised with the size at 40x I would say 20 G ?

Author response: Whole-slide images are stored in a multi-resolution pyramid structure. Image files contain multiple down-sampled versions of the original image. Each image in the pyramid is stored as a series of tiles, to facilitate rapid retrieval of subregions of the image. In other words, the WSIs have been highly compressed. For example, the compressed storage space of the No. 76 tumor sample is 2.24GB with 114688 x 100352 pixels at 40x, but the uncompressed size is 32.2GB.

Reviewer#2, Concern # 3: “Because the patches at the edge of annotations are usually transitional regions from tumor tissue to normal tissue, most of the “hard examples” are concentrated in these positions. The same is true for the labeling of patch pairs under double magnifications”

Are you sure ? For me, the border of the tumoral areas are quite visually marked even I agree there is a transition that makes it more interesting to be precise from a computational point of view.

Author response: We apologize for the inaccurate description and confusion here. In the latest version, the labeling method has been adjusted. The labeling of a patch is determined by the proportion of the tumor area within the patch. When the proportion of tumor area is less than 50%, this patch is normal (Label 0); otherwise, it is tumor (Label 1). For the labeling under the combination of double magnifications, we adopted the "or" logic here.

Compared with the method of judging whether the center point of a patch is in the ground truth annotation, these updated labels make the patch-based classifier more robust. The accuracy of the predicted segmentation boundary is attempted to be ensured by our adaptive algorithm, which can perform further sampling in suspicious regions or boundaries.

The experimental results in the manuscript are obtained using the above-mentioned lastest annotation method. In the revised version, we made corresponding corrections.

Reviewer#2, Concern # 4: Here, the sampling interval in the normal regions is larger than that in the tumor region.

Here stands for where ? What do you mean ?

Author response: Sorry for not expressing clearly here. We balanced the number of positive and negative samples in a WSI by dynamically changing the sampling interval of different regions. If the area of normal regions in a WSI is much larger than the tumor area, we will increase the sampling interval in the normal regions. In this way, the number of negative samples in a WSI does not exceed 5~6 times the number of positive samples. Then, we constructed balanced training sample sets (1:1) through random sampling for improving the performance of the patch-based classifier. We have corrected this part in the manuscript.

Reviewer#2, Concern # 5: because the prediction of a patch with the same size under 40× is more accurate for the center of the patch. ??

Rephrase it or explain it more clearly

Author response:

We use patches of the same image size under different magnifications, and the centers of the two patches under different magnifications are coincident. According to the experimental results of transfer learning in Supplementary Table 1, we observed the following phenomena.

The patches under 20× have the best distinguishable characteristics which can be extracted by CNN, as a result of the balance of texture details and texture range in view. Although the 10× patches have a larger field of view, they are downsampled to the same size resulting in the loss of texture and degradation of classification performance. Compared to 20×, the classifiers under 10× are a little worse. Under 40×, the field of view in a patch becomes very small in a patch. When the patches are extracted from the transitional zone from tumor to normal near the edge of annotations, these patches under 40× are no significant and typical texture features, and even look the same as the patches in normal regions. So, it is difficult to train a better classifier under this magnification individually. On the other hand, the patch-based classifier calculates a tumor feature based on the entire 256 × 256 image, and the calculated patch-level prediction is stored in a tumor feature map based on the central coordinate of the patch.

Thus, with the same image size, the prediction under higher magnification can more accurately represent the tumor feature (probability) at the sampling point (the center of a patch). From the perspective of spatial location, we argue that the prediction of a patch with the same size under 40× is more accurately express the tumor feature at the center of the patch, and facilitate the generation of more detailed segmentation boundaries. Moreover, in the pixel segmentation experiment, the accuracy under the 20× and 40× magnifications alone is better than that under the 10× alone. In the end, our patch-based classifier chose the combined input under 20× and 40×.

Reviewer#2, Concern # 6: The structure of the patch-based classifier was derived from DenseNet-40, so we called it the Double Magnification Combination (DMC) patched-based classifier.

So ?? I do not see the logical implication ?

Author response: We apologize for the illogical expression here; this “so” is redundant.

Reviewer#2, Concern # 7: The growth rate (‘k’ in [17]) is 16. is set to 16 and explain what it is

Author response: The growth rate is a hyper-parameter of the DenseNet model. In a dense block, from input to output, the number of feature-maps of the convolutional layers continues to increase, and the increment is the growth rate mentioned here.

We set the growth rate to 16 to reduce the parameters of the model by using very narrow layers, at the same time, keeping up with the performance of our patch classification.

Reviewer#2, Concern # 8: Here, y refers to the ground truth under double magnification, y 20 and y 40 and so forth. The prediction is the same in classif, isn’t it ?

Author response: As mentioned earlier, the labeling of a patch is determined by the proportion of the tumor area within the patch. When the proportion of tumor area is less than 50%, this patch is normal (Label 0); otherwise, it is tumor (Label 1). For the labeling under the double magnification combination, we adopted the "or" logic here.

Thus, inside the normal and tumor regions, the labels y20 and y40 of a pair of patches overlap at the center point are the same. The y20 and y40 of a pair of patches extracted from the edges of manual annotations may be different. And the label y takes their logical union.

Reviewer#2, Concern # 9: Here, α is 1.0, and w is the weight of the L-GM loss, which is 0.001 by explain alpha et w why 0,5 and w if w is set to a fixed value. It complexity the method for no added value. Or you make w varying ?

Author response: α is a hyper-parameter of L-GM loss. In the reference, its default value is 1. Here, we followed this setting.

w is the weight of the regularization term of the loss function 〖loss﷩2〗, and It is an empirical value.

The first factor of 0.5 in 〖loss〗_2 indicates that the two cross-entropy losses at both magnifications have the same weight, that is, the classification error is reflected in their average under both magnifications.

Reviewer#2, Concern # 10: Ca and Cr a and r stands for what

Globally as there is a lot of parameters make it clear the choice of subscript

Author response: We very much appreciate the comments. In the revised manuscript, we tried our best to clarify the meaning of these symbols.

In the pseudo-code of Algorithm 1, C_R refers to the set of center coordinates of patches, which are obtained by regular sampling based on superpixel segmentation. The subscript "R" here is the initials of “Regular”. 〖C﷩A〗 refers to the set of center coordinates of patches, which are obtained by random sampling (Quasi-Monte Carlo) process. The subscript "A" here is the second letter of “random”, since "R" has already been used before.

Reviewer#2, Concern # 11: Here, T f represents the lower limit of the tumor feature with a superpixel, generally set to -1, which indicates that a superpixel within a tumor probability of more than 27% should be considered

Make it clearer please??

Author response: Because the output of our binary patch-based classifier is the tumor feature of a patch, if we use the Sigmoid function to regress a feature into a tumor probability, the feature value -1 corresponds to the tumor probability of 27%. Here the threshold T_f^sp represents the lower limit of the tumor feature in a superpixel, generally set to -1. When there is a feature larger than T_f^sp in a superpixel, it means that there is a point with tumor probability greater than 27% inside. In the next iteration, such superpixels will be further explored.

We have revised this in the manuscript.

Reviewer#2, Concern # 12: Fig 4 is not very clear ?

the 12 points for instance

Author response: We very much appreciate the comments. We will try our best to improve.

Reviewer#2, Concern # 13: Here, we did not use fully-conv (FC) net to directly generate a heatmap under 1.25×, because this required higher hardware requirements (GPU memory capacity).

But you use a GPU in the results ? (see “The performance test was performed on a PC with a 3.2 GHz Intel i7-8700 CPU with 16 GB of memory and an NVIDIA GeForce GTX 1080 8 GB”.)

Author response: Due to limited resources, we only put the training and prediction of the DMC patch-based classifier to run on GPU. While in our adaptive sampling, the generation of sampling coordinates, the operations of feature maps M_feat and gradient maps M_grad are all done on the CPU.

Reviewer#2, Concern # 14: When the features under both magnifications are used at the same time in DMC, the performance was improved by nearly 2~3%. Because a pair of patches overlap at the center point and the field of view is different, so the spatial attention mechanism was introduced. For the performance of patch-level detection, our experience is the use of L-GM loss in training has no significant effect on single-input or dual-input classifiers.

Results interesting : x20 seems to be enough

Is it very useful to add ‘0x then. ?

Reviewer#2, Concern # 15: Although our training and testing sets contain millions of patches extracted from WSIs, the patches extracted during the adaptive sampling process were still more complex.

I do not understand the logical articulation.

Author response: These two issues are related and we give clarification here together.

Pathologists usually check images by changing their magnification and scope in the WSI. Ways to use images under a wider range of magnifications are worth studying. Inspired by this, we investigated the patch-based classifier with multiple magnifications. Regarding the F1 scores in Table 2, there is not much difference in accuracy between single-input or dual-input patch-based classifiers, but there is a significant difference in the results of pixel-level segmentation. In the pixel-level segmentation task, the F1 scores of each patch-based classifier are much lower than the scores of the classifier during training and testing. The loss of the pixel-level segmentation task is not used to optimize the performance of patch-based classifiers; it represents the generalization performance of the classifier. This is because, although we extracted millions of patches from WSI for training these patch-based classifiers, the input images during our adaptive random sampling are almost impossible to be the same as those in the training set. In other words, the patches extracted during the adaptive sampling are more varied. Moreover, the prediction error at any sampling point has an impact on the accuracy of the segmentation boundary near it. The superposition effect brought by the sampling mechanism makes it possible to obtain correct results only when the robustness of the patch-based classifier is sufficient. Regarding the F1 score of pixel level, the performance of our adaptive sampling algorithm on DMC is nearly 20% higher than that of the classifiers with single magnification. This shows the advantages of the dual input structure.

Reviewer#2, Concern # 16: Table 3 interesting

HCH very different from Camelyon : can you elaborate on it a bit more as you started to do in the following lines.

This is because the area of the tumor regions in each slide of HCH is on average 8 to 9 times larger than Camelyon16, but the number of regions is generally relatively small. In other words, the detection target is relatively significant. Therefore, HASHI is more suitable for the detection of such WSIs, and our proposed method can detect more and smaller tumor regions.

For instance, show HCH Images and the difference with Camelyons

Author response: We very much appreciate the comments. In the revised manuscript, we have made additions in Fig. 6.

Submitted filename: PONE-D-21-02087 Response to Reviewers.docx

Click here for additional data file.

28 Apr 2021

A fast and effective detection framework for Whole-Slide Histopathology Image analysis

PONE-D-21-02087R1

Dear Dr. Yue,

We’re pleased to inform you that your manuscript has been judged scientifically suitable for publication and will be formally accepted for publication once it meets all outstanding technical requirements.

Within one week, you’ll receive an e-mail detailing the required amendments. When these have been addressed, you’ll receive a formal acceptance letter and your manuscript will be scheduled for publication.

An invoice for payment will follow shortly after the formal acceptance. To ensure an efficient process, please log into Editorial Manager at http://www.editorialmanager.com/pone/, click the 'Update My Information' link at the top of the page, and double check that your user information is up-to-date. If you have any billing related questions, please contact our Author Billing department directly at authorbilling@plos.org.

If your institution or institutions have a press office, please notify them about your upcoming paper to help maximize its impact. If they’ll be preparing press materials, please inform our press team as soon as possible -- no later than 48 hours after receiving the formal acceptance. Your manuscript will remain under strict press embargo until 2 pm Eastern Time on the date of publication. For more information, please contact onepress@plos.org.

Kind regards,

Gulistan Raja

Academic Editor

PLOS ONE

Additional Editor Comments (optional):

Reviewers' comments:

Reviewer's Responses to Questions

Comments to the Author

1. If the authors have adequately addressed your comments raised in a previous round of review and you feel that this manuscript is now acceptable for publication, you may indicate that here to bypass the “Comments to the Author” section, enter your conflict of interest statement in the “Confidential to Editor” section, and submit your "Accept" recommendation.

Reviewer #1: All comments have been addressed

Reviewer #2: All comments have been addressed

**********

2. Is the manuscript technically sound, and do the data support the conclusions?

The manuscript must describe a technically sound piece of scientific research with data that supports the conclusions. Experiments must have been conducted rigorously, with appropriate controls, replication, and sample sizes. The conclusions must be drawn appropriately based on the data presented.

Reviewer #1: Yes

Reviewer #2: Yes

**********

3. Has the statistical analysis been performed appropriately and rigorously?

Reviewer #1: Yes

Reviewer #2: Yes

**********

4. Have the authors made all data underlying the findings in their manuscript fully available?

The PLOS Data policy requires authors to make all data underlying the findings described in their manuscript fully available without restriction, with rare exception (please refer to the Data Availability Statement in the manuscript PDF file). The data should be provided as part of the manuscript or its supporting information, or deposited to a public repository. For example, in addition to summary statistics, the data points behind means, medians and variance measures should be available. If there are restrictions on publicly sharing data—e.g. participant privacy or use of data from a third party—those must be specified.

Reviewer #1: Yes

Reviewer #2: Yes

**********

5. Is the manuscript presented in an intelligible fashion and written in standard English?

PLOS ONE does not copyedit accepted manuscripts, so the language in submitted articles must be clear, correct, and unambiguous. Any typographical or grammatical errors should be corrected at revision, so please note any specific errors here.

Reviewer #1: Yes

Reviewer #2: Yes

**********

6. Review Comments to the Author

Please use the space provided to explain your answers to the questions above. You may also include additional comments for the author, including concerns about dual publication, research ethics, or publication ethics. (Please upload your review as an attachment if it exceeds 20,000 characters)

Reviewer #1: This paper was revised well.

However, the language quality is still not enough.

Please ask some native English language speakers to help to improve.

Reviewer #2: I understand then all your choices concerning the strategy.

So you work on compressed image. I think it is important to express it And which compression method or format if not tiff.

I would put Re and Ra then for the subscript in Algo 1 (for Regular and Random)

I would make a precision about the use of GPU only for training.

My comment about the x10, x20, x40 concerning the field of view and the image size : you should have worked at the real size in micrometers. Anyway the results is interesting as the CNN are usually built for fixed size images (in pixels).

The multiresolution issue is still hardly tackled with these techniques, true.

**********

7. PLOS authors have the option to publish the peer review history of their article (what does this mean?). If published, this will include your full peer review and any attached files.

If you choose “no”, your identity will remain anonymous but your review may still be made public.

Do you want your identity to be public for this peer review? For information about this choice, including consent withdrawal, please see our Privacy Policy.

Reviewer #1: No

Reviewer #2: Yes: Nicolas Loménie

3 May 2021

PONE-D-21-02087R1

A fast and effective detection framework for Whole-Slide Histopathology Image analysis

Dear Dr. Yue:

I'm pleased to inform you that your manuscript has been deemed suitable for publication in PLOS ONE. Congratulations! Your manuscript is now with our production department.

If your institution or institutions have a press office, please let them know about your upcoming paper now to help maximize its impact. If they'll be preparing press materials, please inform our press team within the next 48 hours. Your manuscript will remain under strict press embargo until 2 pm Eastern Time on the date of publication. For more information please contact onepress@plos.org.

If we can help with anything else, please email us at plosone@plos.org.

Thank you for submitting your work to PLOS ONE and supporting open access.

Kind regards,

PLOS ONE Editorial Office Staff

on behalf of

Dr. Gulistan Raja

Academic Editor

PLOS ONE