Quantification of histopathological findings using a novel image analysis platform.

Digital pathology, including image analysis and automatic diagnosis of pathological tissue, has been developed remarkably. HALO is an image analysis platform specialized for the study of pathological tissues, which enables tissue segmentation by using artificial intelligence. In this study, we used HALO to quantify various histopathological changes and findings that were difficult to analyze using conventional image processing software. Using the tissue classifier module, the morphological features of degeneration/necrosis of the hepatocytes and muscle fibers, bile duct in the liver, basophilic tubules and hyaline casts in the kidney, cortex in the thymus, and red pulp, white pulp, and marginal zone in the spleen were learned and separated, and areas of interest were quantified. Furthermore, using the cytonuclear module and vacuole module in combination with the tissue classifier module, the number of erythroblasts in the red pulp of the spleen and each area of acinar cells in the parotid gland were quantified. The results of quantitative analysis were correlated with the histopathological grades evaluated by pathologists. By using artificial intelligence and other functions of HALO, we recognized morphological features, analyzed histopathological changes, and quantified the histopathological grades of various findings. The analysis of histopathological changes using HALO is expected to support pathology evaluations. ©2019 The Japanese Society of Toxicologic Pathology.

Introduction

Pathological evaluation is typically conducted by observing tissue specimens. The evaluation results are very important from the viewpoint of examining histopathological changes directly. However, pathological evaluation often generates bias among facilities or observers. One of the means for reducing the bias is to quantify the histopathological changes. Appropriate quantification of histopathological changes consistent with the observation results enables pathological evaluation with objectivity and high reproducibility. Furthermore, it is possible to quantify the degree of subtle histopathological changes that cannot be distinguished by observers. An objective and highly accurate quantitative analysis can lead to an improvement in reliability and persuasiveness. If quantitative analysis is automated, the efficiency of the pathological evaluation is expected to be improved.

Recently, with the development of information technology, digital pathology has also developed remarkably. An increasing number of researchers have been attempting to analyze and diagnose pathological tissues using machine learning or artificial intelligence (AI)[1], [2], [3], [4], [5], [6]. Particularly, in the clinical field, automatic recognition of morphologically complex cancer tissues has become possible with high accuracy[7], [8], [9]. On the other hand, in the nonclinical field, there are many cases of simply quantifying specific stained regions, but there are not so many cases of analyzing and quantifying complex tissue morphology.

We have attempted to analyze and quantify various histopathological findings using conventional image processing software in order to turn the quantitative results into indicators of drug efficacy or toxicity[6], [10]. Several findings were quantified by devising innovative means of morphological processing and functions of image processing software. These quantitative findings were correlated with evaluations by pathologists. However, quantitative analysis requires high expertise in the setting of conditions and construction of algorithms for image processing software. Therefore, it is not easy for everyone to analyze a tissue image, and this practice tends to be of low versatility.

Recent reports have used the image analysis platform HALO, which can easily separate tissue classes and quantify various morphological features in pathological tissue[11], [12], [13]. This makes many morphometric measurements of pathological tissue possible. Using HALO, we attempted to quantify various findings, including hepatocellular degeneration/necrosis and atrophy of the marginal zone in the spleen, that were hard or impossible to analyze with conventional image processing software[6]. In this report, we introduce quantifiable examples of several histopathological findings and mention future prospects.

Methods, Results, and Discussion

In this study, we used tissue specimens that were fixed in formalin, embedded in paraffin, and stained with hematoxylin and eosin (HE) or immunostained. Whole digital slide images were obtained using virtual microscopy (Aperio AT2, Leica Biosystems, Wetzlar, Germany). Using HALO (v2.2.1870.17, Indica Labs, Albuquerque, NM, USA), histopathological features of target findings were analyzed and quantified. Table 1 shows a summary of target findings, measurement parameters, and the applied HALO modules. The tissue classifier module has a random forests algorithm, which is one of the machine learning methods for constructing a multitude of decision trees during training[14]. In each experiment, the number of specimens per grade was one (n=1), and tissue classes were separated based on learning from several representative images. The detailed methods of each analysis are described in the following section.

Table 1.

Summary of Quantified Findings, Measurement Parameters, and the Applied HALO Module

Hepatocellular degeneration/necrosis in the liver

Hepatocellular degeneration/necrosis in a specimen stained with HE is relatively easy to recognize because morphological changes such as eosinophilic cytoplasm and atrophied nuclei are comprehensively observed. However, when analyzing detailed morphological features, the eosinophilic color tone present within hepatocytes undergoing degeneration/necrosis is also present in normal hepatocytes. This makes it hard to distinguish them clearly using conventional image analysis[6]. In this study, the morphological features of hepatocytes undergoing degeneration/necrosis, red blood cells, and the other components (mainly normal hepatocytes) were learned and separated by the tissue classifier module (random forest algorithm) of HALO using specimens graded as − (no change), + (slight), or ++ (moderate) for the degree of hepatocellular degeneration/necrosis. The area of interest was then quantified. Fig. 1b, d, f, and h show the images separated using the tissue classifier module. Degeneration/necrosis area is expressed as a ratio to the whole liver area analyzed. The results of the quantitative analysis were correlated with the degree of degeneration/necrosis graded by pathologists (Fig. 1i).

Fig. 1.

Quantification of hepatocellular degeneration/necrosis area. (a, c, e, g) Original images. (b, d, f, h) Segmented images of degeneration/necrosis (green), red blood cells (red), and the other regions (pink) through the use of the tissue classifier module. (g, h) Enlarged images of blue frames in (e) and (f), respectively. Grade of degeneration/necrosis evaluated by pathologists: (a, b) no change, −; (c, d) slight, +; (e–h) moderate, ++. Bar = 1 mm. (i) Quantitative results for degeneration/necrosis area (% of the whole area of the liver section).

Bile duct proliferation in the liver

The bile duct is a stimulus-sensitive tissue component known to proliferate in many hepatobiliary disorders. The morphological features of the bile duct, infiltrating cells with form similar to the bile duct, red blood cells, and other components (mainly normal hepatocytes) were learned and separated by the tissue classifier module using HE-stained specimens graded as −, +, or ++ for the degree of bile duct proliferation. The area of interest was then quantified. Figure 2b, d, and f show the images separated using the tissue classifier module. Bile duct area is expressed as a ratio to the whole liver area analyzed. The results of the quantitative analysis showed that the area (number) of the bile duct increased in correlation with the degree of bile duct proliferation graded by pathologists (Fig. 2g). Although this did not occur in the present study, it should be noted that if an extremely large bile duct is present within the analyzed region, it is necessary to exclude it by limiting the size of bile duct because the area of such a large bile duct may be equal to that of the proliferated bile duct.

Fig. 2.

Quantification of the bile duct area in the liver. (a, c, e) Original images. (b, d, f) Segmented images of bile duct (green), infiltrating cells (light blue), red blood cells (orange), and other regions (pink) using the tissue classifier module. Grade of proliferation of bile duct evaluated by pathologists: (a, b) no change, −; (c, d) slight, +; (e, f) moderate, ++. Bar = 100 μm. (g) Quantitative results for bile duct area (% of the whole area of the liver section).

Basophilic tubules and hyaline casts in the kidney

Various pathological changes in the kidney stained with HE can be observed due to renal dysfunction. Basophilization of the renal tubule is a regenerative pathological change that occurs in the process of repairing renal tubule injury. Hyaline casts are often caused by impairment of the glomerular filtration barrier. The morphological features of basophilic tubules, hyaline casts, red blood cells, and other components (normal tubules, glomeruli, and the renal pelvis) were learned and separated by the tissue classifier module using HE-stained specimens in which the degree of basophilic tubules was graded as − or + and that of hyaline casts was graded as − or +. Each area of interest was then quantified. Each area is expressed as a ratio to the whole kidney area analyzed. Figure 3b, d, and f show the separated images. The results of the quantitative analysis showed that basophilic tubules and hyaline casts increased in correlation with the degree graded by pathologists (Fig. 3g).

Fig. 3.

Quantification of the areas of basophilic tubules and hyaline casts in the kidney. (a, c, e) Original images. (b, d, f) Segmented images of basophilic tubules (blue), hyaline casts (yellow), red blood cells (red), and other regions (pink) using the tissue classifier module. (e, f) Enlarged images of green frames in (c) and (d), respectively. Grade of basophilic tubules and hyaline casts evaluated by pathologists: (a, b) no change, −; (c–f) slight, +. Bar = 1 mm (a–d) and 100 μm (e, f). (g) Quantitative results for segmented areas (% of the whole area of the kidney section).

It was not possible to segment tissue classes included in the other components using the tissue classifier module. Automated detection of glomeruli using the histogram of oriented gradients (HOG) has been reported[4], [5]. By using an analysis that makes use of a deep learning algorithm, it may be possible to analyze more detailed tissue forms including glomeruli.

Cortical atrophy in the thymus

Histologically, the structure of the thymus can be divided into the cortex and medulla, and these zones are often changed morphologically due to stress or immune stimulation. The morphological features of the cortex, medulla, capsule, and red blood cells in the thymus stained with HE were learned and separated by the tissue classifier module using specimens graded as −, +, or ++ for the degree of cortical atrophy. The area of interest was then quantified. Figure 4b, d, and f show the separated images. The results of the quantitative analysis were correlated with the degree of cortical atrophy graded by pathologists (Fig. 4g), which suggested that cortical atrophy contributed greatly to atrophy of the thymus.

Fig. 4.

Quantification of the cortex area in the thymus. (a, c, e) Original images. (b, d, f) Segmented images of the cortex (blue), medulla (green), red blood cells (red), and capsule (orange) using the tissue classifier module. Grade of cortical atrophy evaluated by pathologists: (a, b) no change, −; (c, d) slight, +; (e, f) moderate, ++. Bar = 1 mm. (g) Quantitative results for segmented areas.

Degeneration/necrosis in the skeletal muscle fibers

Skeletal muscle fibers undergoing degeneration/necrosis, in which the cell walls were impaired and blood containing IgG flowed into the cells, were immunostained with anti-IgG antibody, as described in a previous report[15]. However, IgG is also observed in other regions including connective tissue and interstitial tissue, making it hard to distinguish only muscle fibers undergoing degeneration/necrosis by conventional image analysis. In this study, the morphological features of anti-IgG antibody-immunostained muscle fibers, an anti-IgG antibody-immunostained “other region”, and an unstained region were learned and separated by the tissue classifier module using immunostained specimens graded as − or + for the degree of staining intensity of the muscle fibers. Each area of interest was then quantified. Figure 5b and d show the separated images. Each area is expressed as a ratio to the whole muscle area analyzed. The results of the quantitative analysis were correlated with the degree of degeneration/necrosis graded by pathologists (Fig. 5e).

Fig. 5.

Quantification of the IgG-stained muscle fibers area. (a, c) Original images. (b, d) Segmented images of the IgG-stained muscle fiber region (red), IgG-stained other region (yellow), and unstained region (light blue) using the tissue classifier module. Grade of the IgG-stained muscle fiber (degeneration/necrosis of muscle fibers) area evaluated by pathologists: (a, b) no change, −; (c, d) slight, +. Bar = 200 μm. (e) Quantitative results for segmented areas (% of the whole area of the skeletal muscle section).

Atrophy of the white pulp and marginal zone, and extramedullary hematopoiesis in the spleen

The region of red pulp, white pulp, and marginal zone in the spleen often change as a result of the immune stimulation. The degree of extramedullary hematopoiesis in the spleen is affected by the condition of hematopoiesis in the bone marrow. In this study, atrophy of the white pulp, atrophy of the marginal zone, and extramedullary hematopoiesis were analyzed in the spleen stained with HE. First, the morphological features of the red pulp, white pulp, and marginal zone were learned and separated by the tissue classifier module. Each area of interest was then quantified. Figure 6b, e, h, k, n, and q show the separated images. The results of the quantitative analysis were correlated with the degree of atrophy of the marginal zone graded by pathologists as −, +, or, ++ (Fig. 6s). Next, the cytonuclear module was used to recognize, within the red pulp region separated in the first step, the nuclei of erythroblasts and red blood cells based upon their color tone and size, and each cytoplasmic area was set so as not to overlap neighboring cells. Cell numbers in each area of interest were then quantified. Figure 6c, f, i, l, o, and r show the analyzed images. The results of the quantitative analysis were correlated with the degree of decreased extramedullary hematopoiesis (erythroblasts) graded by pathologists (Fig. 6t).

Fig. 6.

Quantification of the red pulp, white pulp, and marginal zone areas in the spleen and number of erythroblasts in the red pulp. (a, d, g, j, m, p) Original images. (b, e, h, k, n, q) Segmented images of the red pulp (pink), white pulp (blue), and marginal zone (light blue) using the tissue classifier module. (c, f, j, l, o, r) Analyzed images of erythroblasts (blue) and red blood cells (red) using the cytonuclear module. (d–f, j–l, p–r) Enlarged images of green frames in (a–c), (g–i), and (m–o), respectively. Grade of atrophy of the marginal zone evaluated by pathologists: (a, b, d, e) no change, −; (g, h, j, k) slight, +; (m, n, p, q) moderate, ++. Grade of decreased extramedullary hematopoiesis evaluated by pathologists: (a, c, d, f, g, i, j, l) no change, −; (m, o, p, r) slight, +. Bar = 1 mm (a–c, g–i, m–o) and 100 μm (d–f, j–l, p–r). (s) Quantitative results for segmented areas. (t) Quantitative results for erythroblast numbers.

Atrophy of acinar cells in the parotid gland

In this study, the size of the acinar cells was analyzed in the parotid gland stained with HE. First, the morphological features of acinar cells and other regions (duct, interstitial connective tissue, and capsule) were learned and separated by the tissue classifier module. Figure 7b and e show the separated images. Next, the area of each acinar cell within the acinar cell region (12.5 mm2 area as wide as possible in the specimen) separated in the first step was quantified using the vacuole module. Although acinar cells are not vacuoles, their morphological features, including their nuclei and distinct contours, are similar to vacuoles. Therefore, each acinar cell could be analyzed by the vacuole module in this study. Figure 7c and f show the analyzed images. The areas of the acinar cells were quantified and sorted in increments of 10 μm2, and the total numbers of acinar cells per size were calculated (Fig. 7g). There was an increased number of smaller acinar cells in the specimen for which the degree of atrophy of the acinar cells was evaluated as +.

Fig. 7.

Quantification of the number of acinar cells in the parotid gland. (a, d) Original images. (b, e) Segmented images of acinar cells (yellow) and other regions (pink) using the tissue classifier module. (c, f) Analyzed images of acinar cell size using the vacuole module. Nuclei are blue. Grade of atrophy of acinar cells evaluated by pathologists: (a, c) no change, − (orange); (d, f) slight, + (red). Bar = 100 μm. (g) Quantitative results for acinar cell size. Total area of acinar cells per size. The numbers of acinar cells within 10-μm2 size fractions ranging from 0 to 50 μm2 and the number of those >50 μm2 in size were calculated.

Conclusion

In this report, we presented several examples of analyses in which various histopathological changes and findings were quantified using HALO. By appropriately training the tissue classifier module to learn the morphological features of each tissue class, the tissues were separated into different classes, and the areas of interest were quantified. Additionally, after separation by the tissue classifier module, each cell was analyzed in detail by another module, such as the cytonuclear or vacuole modules. Although not described in detail in this report, it was possible to analyze and quantify various findings in the liver, adipocyte, sublingual gland, and adrenal grand (Table 1) by using modules that could analyze their morphological features. Furthermore, by using various modules in combination, it will be possible to automatically analyze other findings, not just the examples shown in this report. In fact, many cases of analyses using various modules with HALO have been reported[16], [17], [18], [19], [20].

It was hard for the tissue classifier module with the random forest algorithm to recognize more detailed tissue forms such as glomeruli. However, by using deep learning with a conventional neural network, it may be possible in the future to recognize the morphological features of glomerular or other detailed tissue structures of various findings, including those of inflammatory cells, single cell death, or mitosis.

Morphometry and diagnosis are not as popular yet in nonclinical fields compared with clinical cancer diagnosis. However, as shown in this report, various findings in nonclinical fields may be analyzed by using HALO. Each HALO module is specialized for analyzing target morphological features of pathological tissue. The setting of parameters and operation of HALO are simple, making it easy to handle for researchers who lack image analysis experience.

This report only shows typical results of quantitative analysis using several examples. Whether or not a specimen can be analyzed and HALO can be put into practical use for image analysis using the same setting parameters depends on the uniformity of the specimen’s color tone, and the method may need to be validated before each study. However, HALO is not a complicated analysis methods and is user-friendly. This makes it a highly versatile tool in a variety of pathological examinations. In the future, we expect that image analysis of pathological tissue will further develop and that the quantitative data of findings will support pathologic evaluations and image diagnoses and will be used as indicators of drug efficacy or drug toxicity.

Disclosure of Potential Conflicts of Interest

All authors are employees of Mitsubishi Tanabe Pharma Corporation. The authors declare that they have no conflicts of interest.