A user-friendly tool for cloud-based whole slide image segmentation with examples from renal histopathology.

Background: Image-based machine learning tools hold great promise for clinical applications in pathology research. However, the ideal end-users of these computational tools (e.g., pathologists and biological scientists) often lack the programming experience required for the setup and use of these tools which often rely on the use of command line interfaces.
Methods: We have developed Histo-Cloud, a tool for segmentation of whole slide images (WSIs) that has an easy-to-use graphical user interface. This tool runs a state-of-the-art convolutional neural network (CNN) for segmentation of WSIs in the cloud and allows the extraction of features from segmented regions for further analysis.
Results: By segmenting glomeruli, interstitial fibrosis and tubular atrophy, and vascular structures from renal and non-renal WSIs, we demonstrate the scalability, best practices for transfer learning, and effects of dataset variability. Finally, we demonstrate an application for animal model research, analyzing glomerular features in three murine models. Conclusions: Histo-Cloud is open source, accessible over the internet, and adaptable for segmentation of any histological structure regardless of stain.

Introduction

Recent advances in machine learning techniques have led to previously unachievable performance for image analysis tasks. In particular, convolutional neural networks (CNNs)[1], a form of deep learning, have great potential for impactful applications in the computational analysis of image structures. Successful adoption of these tools to biomedical image data promises a paradigm shift in both biological science and healthcare[2].

In the field of pathology, the practice of digitizing histological slides has become common practice[3], facilitating the application of CNNs for analysis. Digitally scanned histology slides, known as whole slide images (WSIs), are often gigapixels in size. Parsing WSIs into biologically relevant sub-compartments (commonly known as segmentation) is often an important first step for tissue analysis and pathological examination[4]. Due to the size of WSIs and the diversity of structures that can be present, downstream machine learning tasks (such as slide classification) can also benefit from segmentation, which can help limit the regions of interest considered[5].

CNNs have been successfully utilized by many research groups for the segmentation of WSIs[4-9]. However, thus far tools to segment WSIs have been complex to deploy and use, requiring knowledge of the command line interface and computational expertize[10-12]. The ideal user for these tools is the pathologist or biological scientist, whose clinical workflow or research questions could benefit from fast and accurate segmentation of relevant structures[2].

To address this gap, we have developed Histo-Cloud, a powerful tool for the segmentation of WSIs and deployed it as a suite of easy-to-use plugins using the Digital Slide Archive (DSA)[13], an open-source cloud-based WSI repository with a built-in slide viewer. Histo-Cloud was designed with flexibility in mind and is agnostic to tissue type or structure. Segmentation of new structures of interest is possible by retraining the CNN used for segmentation, which can be conveniently performed within the cloud interface.

Methods

Human data collection followed a protocol approved by the Institutional Review Board at University at Buffalo (STUDY00002731, STUDY00003929, STUDY00004044, STUDY00004235, STUDY00005089, and STUDY00005541) prior to commencement. Computational image analysis is done in this study using retrospective data qualified for a waiver of the consent process.

WSIs for GlomTrainSet, GlomTestSet 1, and GlomTestSet 4

These datasets were used for the segmentation of glomeruli. This dataset consists of both human and murine renal tissue WSIs from various institutes as well as publicly available repositories, using diverse stains and different scanners. The institutions included the University of California at Davis (UC Davis), Johns Hopkins University (JHU), Kidney Translational Research Center (KTRC) at Washington University School of Medicine at St. Louis (WUSTL), Seoul National University Hospital Human Biobank (SNUHHB), Vanderbilt University Medical Center (VUMC), University at Buffalo (UB), University Hospital Cologne (UHC), and the publicly available Genotype-Tissue Expression (GTEx) portal, a repository that hosts human autopsy WSIs.

The GlomTrainSet consisted of 743 WSIs, 428 from humans and 315 from murine tissues, containing a total of 61,734 manually verified glomerular annotations. GlomTestSet 1 consisted of 100 holdout slides from the same data sources as GlomTrainSet. This included 3816 glomeruli, 37.8 GB of compressed image data, and a combined total of more than 0.24 trillion image pixels. GlomTestSet 4 contained an additional 1528 WSIs from the same sources that were used to study the scalability and prediction time of the method.

The human renal tissues manifest disease pathology spanning various stages of diabetic nephropathy; various classes of lupus nephritis; renal transplant protocol biopsies, including time-zero, protocol, and indication biopsy cases; human autopsy renal tissues publicly available via GTEx with diversity in age, sex, and race; and renal biopsies with pathologies that include membranous nephropathy, thrombotic microangiopathy, pauci-immune glomerulonephritis, focal segmental glomerulosclerosis (FSGS), mesangiopathic glomerulonephritis, arteriolosclerosis, hypertension, IgA nephropathy, chronic tubulointerstitial nephritis, acute tubular necrosis, Fabry disease, amyloid nephropathy, membranoproliferative glomerulonephritis, light chain cast nephropathy, minimal change disease, post-infectious glomerulonephritis, idiopathic nodular glomerulosclerosis, and anti-glomerular basement membrane disease. The human data were collected in accordance with protocols approved by Institutional Review Board at the UC Davis, JHU, KTRC, WUSTL, SNUHHB, VUMC, and UB. The SNUHHB data were shared under IRB number H-1812-159-998.

Murine renal tissues included in GlomTrainSet and GlomTestSet 1 came from three different models. For the first model wild-type, FVB/N mice were subjected to a combination of four interventions that induce a post-adaptive form of FSGS. The interventional process includes 0.9% saline drinking water, angiotensin II infused via an osmotic pump, uni-nephrectomy, and deoxycorticosterone delivered by implantation of a subcutaneous pellet, summarized as the SAND model[14,15]. The second model was a streptozotocin (STZ) diabetes murine model that manifests nephropathy; a detailed description of this model is discussed in our prior work[16]. The third model was a nephrin knockdown (nephrin KD) murine model, was implemented using a published protocol[17], and shows mesangial hypercellularity and sclerosis, glomerular basement membrane thickening, and podocyte loss.

The tissues were sectioned at 2–5 µm thickness for staining and imaging. The data consist of tissues stained with diverse histological stains, including hematoxylin & eosin (H&E), periodic acid-Schiff (PAS) with hematoxylin (PAS-H) counterstain, Silver, Trichrome, Verhoeff’s Van Gieson, Jones, and Congo red. The slides were scanned using different brightfield microscopy WSI scanners, including Aperio VERSA digital whole slide scanner (Leica Biosystems, Buffalo Grove, IL), Nanozoomer (Hamamatsu, Shizuoka, Japan), and MoticEasyScan Pro (Motic, San Antonio, TX), at 40X resolution. The pixel resolution of the images used was 0.13 to 0.25 µm.

WSIs for VessTrainSet, VessTestSet, and GlomTestSet 2

This human dataset was used to test the adaptability of the model for vessels. In total there were 939 annotated arteries, 6023 arterioles, and 4507 glomeruli. VessTrainSet contained 226 renal tissue WSIs. VessTestSet contained an additional 58 holdout slides. Multiple stains per case were used. This dataset was manually annotated for relevant structures to establish a ground-truth.

The renal tissue WSIs came from UHC via co-author J.U.B. Diagnoses included thrombotic microangiopathy, hypertension-associated nephropathy, and vasculitis. Tissues were sectioned at 2–3 µm thickness. Diverse histologic stains were used, including H&E, PAS-H, Masson trichome, and Jones methenamine silver, for staining the tissue to depict different pathobiological features. A brightfield microscopy scanner Nanozoomer (Hamamatsu, Shizuoka, Japan) was used for WSI scanning at 40X resolution. The pixel resolution of the images used was 0.25 µm. Note that the VessTestSet dataset was used to construct the GlomTestSet 2 dataset to conduct the study discussed in Glomeruli segmentation—scalability.

WSIs for IFTASet 1, IFTASet 2, IFTASet 3, IFTATestSet 2, and GlomTestSet 3

These datasets were used for the segmentation of IFTA. The human renal tissues for this part of the study came from four institutions: the University of California, Davis; the University of California, Los Angeles (UCLA); University of Coimbra (Portugal); and University Hospital Cologne (UHC).

Tissues were obtained from renal allograft nephropathy with no prior history of rejection. For this study, periodic acid-Schiff (PAS)-stained renal tissue WSIs of renal allograft nephropathy were used for training (IFTASet 1, n = 20; IFTASet 2, n = 48; and IFTASet 3, n = 22). One slide was selected per case for each institution. The WSIs per set were uniformly chosen from four IFTA classes defined based on semiquantitative scores (ci/ct scores: 0, 1, 2, and 3); ci/ct scoring is a method defined in Banff 2018 criteria[18] for assessing IFTA in transplant biopsies. A minimum of five slides per class were used for each set. The cases were reviewed to ensure the following selection criteria were met: (1) the amount of early or evolving IFTA with variable intermixed edema was minimized, (2) no active inflammation, (3) no prior history of rejection, and (4) cases were selected to represent the full range of IFTA severity. All types of IFTA, including classic, endocrinization, and thyroidization patterns, were included in the analysis, without distinguishing between the types. IFTATestSet 2 was provided by UHC, and contained 17 WSIs. This dataset followed similar case selection criteria as above with two slides from class 0 and five slides each from the remaining three classes.

The human data were collected in accordance with protocols approved by Institutional Review Boards at the UC Davis, UCLA, University of Coimbra, and the University at Buffalo. Deidentified images from UHC throughout this paper were used for retrospective research, and such is permitted under German law to conduct without IRB approval. The tissues were sectioned at 2–3 µm thickness and stained using PAS-H. Imaging was done using different brightfield microscopy WSI scanners, including Aperio CS virtual slide imaging system, Aperio AT2 (Leica Biosystems, Buffalo Grove, IL), and Nanozoomer (Hamamatsu, Shizuoka, Japan) at 40X resolution. Pixel resolution of the images used was 0.25 µm. Note that the IFTATestSet 2 dataset was used to construct the GlomTestSet 3 dataset to conduct the study discussed in Glomeruli segmentation—scalability.

KPMP WSI dataset

This dataset was used to test the adaptability of the model for IFTA. This part of the study used 26 renal tissue biopsy whole slide images (WSIs) of 26 chronic kidney disease (CKD) subjects from the Kidney Precision Medicine Project. The selection of these slides followed the same criteria described in the section above: WSIs for IFTASet 1, IFTASet 2, IFTASet 3, IFTATestSet 2, and GlomTestSet 3. The recruitment sites were Brigham & Women’s Hospital, Cleveland Clinic, Joslin Diabetes Center/ Beth Israel Deaconess Medical Center, and the University of Texas at Southwestern. The inclusion criteria for CKD subjects for biopsy include subjects diagnosed with diabetic kidney disease (type 1 or 2) and hypertensive kidney disease. For the former, the subjects are included based on eGFR in the range of 30–59 mL/min/1.73 m2 or eGFR ≥ 60 with urinary protein to creatinine ratio (uPCR) >150 mg/g or urinary albumin to creatinine ratio (uACR) >30 mg/g. For the latter, the subjects are included based on eGFR in the range of 30–59 mL/min/1.73 m2 or eGFR ≥ 60 with uPCR in the range of 150–2000 mg/g or uACR in the range of 30–2000 mg/g. The study is overseen by three independent bodies, including a data safety monitoring board, a central institutional review board (WUSTL), and an NIH-NIDDK convened the external expert panel. More details about the rationale and design of KPMP cases are available in a recent publication[19]. The tissues were sectioned at 2–3 µm thickness, and the PAS-H stained tissues were used for the study presented in this work. Imaging was done using an Aperio GT450 brightfield microscopy WSI scanner (Leica Biosystems, Buffalo Grove, IL) at 40X resolution. The pixel resolution of the images used was 0.25 µm.

WSIs for murine kidney tissue for the study discussed in murine model analysis—utility

For this part of study three murine model renal tissue WSIs were employed. These models include an aging model, and two type 2 diabetic nephropathy (T2DN) models (KKAy and Db/Db). We used eight mice (four young and four old) WSIs for the aging model, 20 mice (ten KKAy or disease and ten C57/BL6 or control) WSIs for the KKAy model, and 14 mice (7 Db/Db or disease and 7 Db/m or wild-type control) WSIs for the Db/Db model.

The aging studies were performed in 4-month-old and 21-month-old C57/BL6 male mice obtained from the NIA aging rodent colony[20]. For the KKAy model (see published description[21]), male mice that develop spontaneous diabetes of polygenic origin were used. For the Db/Db model, male mice with a BKS background featuring a leptin receptor mutation were used. These mice depict spontaneous/congenital diabetes due to leptin signaling abnormalities[22]. Animal studies were performed in accordance with protocols approved by the Institutional Animal Care and Use Committee at the Georgetown University, National Institutes of Health, JHU, and UB, are consistent with federal guidelines and regulations, and are in accordance with recommendations of the American Veterinary Medical Association guidelines on euthanasia. Tissues were sectioned at 2–3 µm thickness, and the PAS-H was used for staining. The slides were scanned using different brightfield microscopy WSI scanners, including Nanozoomer (Hamamatsu, Shizuoka, Japan) and MoticEasyScan Pro (Motic, San Antonio, TX), at 40X resolution. The pixel resolution of the images used was 0.25 µm.

Software

With the goal of developing a tool with class-leading WSI segmentation accuracy as well as easy accessibility to computational non-experts, we have integrated the popular semantic segmentation network Deeplab V3+[23] with the DSA[13], an open-source cloud-based histology management program. Specifically, we have created a suite of easy-to-use plugins using HistomicsUI, an application programming interface of the DSA for running Python codes. These plugins efficiently run the DeepLab network for native segmentation of WSIs, making testing new slides accessible through the HistomicsUI graphical user interface (the slide-viewing component of the DSA). Using the HistomicsUI interface, users can interactively view the computational annotations, and further refine such annotations for training new models. The modified HistomicsTK-Deeplab codebase is available via GitHub and also as a pre-built Docker image for easy installation. This software is deployed in the cloud and is accessible via the web, making it easily accessible to the community as a plug-and-play tool (Fig. 1). The open-source plugins are available to the digital pathology community for use and further development.

Fig. 1

The user interface of the segmentation tool (available via the web).

a The left  column shows the controls for the segmentation plugin: is required arguments and contains optional parameters. WSI stands for whole slide image and IO stands for Input/Output. The right column shows the WSI viewer controls and annotations created by the plugin. The green annotations are computationally predicted and are easily editable by the user. Slides are analyzed by clicking the button in the top left corner. b The options from the plugin. Under the section, a user can specify a directory full of annotated WSIs to use for network training with the option, and where to save the trained model with the

option. The option gives users the ability to choose which annotation layers should be used for training and multi-class segmentation models can be trained. To speed up the training process, a previously trained segmentation model can be used for transfer learning by specifying the . Hyperparameters for training the network is automatically set to defaults that work well but can be modified using the options in the section. c shows the plugin which can be used to extract image and morphology features from annotated objects. These features are written to the slide metadata and can be plotted from within the online interface via the tab (on the right). d shows the welcome screen of the online interface athena.ccr.buffalo.edu.

Functionality

We have developed several plugin tools with various functions. (1) The plugin (Fig. 1a) segments WSIs using a previously trained model. (2) The plugin can be used to train new models from a folder of annotated WSIs (Fig. 1b). Histo-Cloud generates predictions as a series of image contours or sparse heatmaps which are written to JavaScript Object Notation (JSON) format for display in HistomicsUI as annotation layers. The code is modular, with the ability to handle multi-class segmentation, and includes the option to tweak the network hyperparameters for advanced users. We include the ability to ignore image regions (Supp. Fig. 5), this is useful to exclude ambiguous image regions from the training set, and may also be of interest for users who wish to only annotate part of a large WSI. During training and testing, a progress bar is shown so the user can gauge the time to completion (Supp. Fig. 5). (3) Functionality was included for conversion between JSON annotations and the XML format ( and plugins). The XML format is used to display contours in Aperio ImageScope (Leica, Buffalo Grove, IL) which is a popular WSI viewer. (4) The plugin (see Fig. 1c) was built for extraction of image and contour-based features from annotated regions in the slides. The features are written into the slide metadata (on DSA) in JSON format. For further data exploration, features saved into the slide metadata can be plotted pairwise using a scatterplot tool available in HistomicsUI (Fig. 1c) for a single slide or across a folder of WSIs. Features can also be saved in spreadsheet format for local download and further analysis.

Computational model

We used the official implementation of the Deeplab V3 + segmentation network[23], modified to work natively on WSIs. This implementation was accomplished by adapting the way the network ingests data and extracting patches from WSIs as needed during training using the large_image Python library[24]. A similar method (HistoFetch) is described more extensively in a recently published preprint[25], which shows on-the-fly patch extraction speeds and overall training time for unsupervised tasks. The HistoFetch method was adapted in this work to perform a supervised segmentation task by creating additional patch selection criteria intended to proactively balance uneven class distributions during patch extraction. Note that during development the code was migrated to use large_image[24] for reading WSI data rather than the openslide[26] library, as the former supports a larger number of slide formats. To convert the ground-truth annotations to masks for semantic segmentation, the HistomicsUI JSON annotations are converted into the Aperio ImageScope XML format, and the XML_to_mask conversion code from the original H-AI-L study[7] was reused for generating ground-truth masks. This code follows the way openslide and large_image read WSI patches via specifying the location and scale of the patches. The min and max indices of each contour annotation are written into the metadata of the XML, allowing for faster reference of which contours are in the image region requested.

A flowchart providing an overview of this training input pipeline is presented in Supp. Fig. 1. A similar pipeline is used during prediction (segmentation of slides), but patches are extracted deterministically from an overlapping grid pattern (excluding non-tissue regions) to ensure full tissue segmentation. The training and testing perform fast color thresholding of the tissue region which is saved as a portable network graphics (PNG) mask for reference (to avoid repeated operations). This process ensures the network does not train on non-tissue regions, and thus speeds the prediction process. During the development, we found that occasionally providing the network with background (non-tissue) patches helped generalize the batch normalization parameters during training. We, therefore, implemented a parameter that defines the probability of selection of patches that may include the background region. Default of 0.1 was found to work well in generalizing the batch normalization layers.

Iterative learning and annotation ingestion

In a previous study, we showed that the human-in-the-loop annotation strategy significantly reduces the annotation burden when developing a tissue segmentation model[7]. This strategy uses a model trained on a limited dataset to run inference on new slides, which are corrected by an annotator. We find that the correction of computational annotations is faster than fully annotating newly added data, reducing the amount of effort required to build a robust training set. Additionally, this strategy allows the annotator to constantly interact with the system, monitoring its performance, and selecting slides where the model struggles the most for incorporation into the training set.

Human-in-the-loop annotation is possible using Histo-cloud through alternating use of the training and testing plugins. Practically, we expect that most users will start an annotation project from scratch and have made using pretrained ImageNet weights the default behavior of the training plugin. However, if a user would like to import data annotated in another system or format, we have included the plugin (which is described in the Functionality section above). This plugin is capable of ingestion of data annotations in the Aperio XML format and could be used to incorporate additional externally annotated data.

If an advanced user wishes to convert previously annotated data into the XML format for ingestion into the system, we direct them to the mask_to_xml script: https://github.com/SarderLab/Histo-cloud/blob/main/histomicstk/deeplab/utils/mask_to_xml.py This script was developed for the conversion of rasterized annotations into the XML format and is used internally by Histo-Cloud for a display of network predictions in HistomicsUI. For advanced users who wish to upload and manage XML annotations from the command line interface, we have also included scripts which satisfy these requirements in the source code: https://github.com/SarderLab/Histo-cloud/tree/main/batch_upload_xmls_to_girder_client.

Training and testing

Training of models was done on a server equipped with two Intel Xeon Silver 4114 (10 core) processors, with 64 GB RAM and dual Nvidia Quadro RTX 5000 graphical processing units (GPU) with 16 GB of video random access memory (VRAM). These resources allowed training with a batch size of 12 using image patches of size 512 × 512 pixels. A batch size of 12 is the minimum recommended for training the batch normalization parameters in the DeepLab implementation document. The Athena server (open for public use) has only one GPU with 8 GB of VRAM. We have therefore disabled training of the batch normalization parameters by default in the training plugin (which can be enabled in the advanced parameter section) and have set a default batch size of 2. All trained networks used a base learning rate of 1e−3 with polynomial decay using the momentum optimizer (momentum value = 0.9).

All models use the Xception 65 network backbone[23], with DeepLab parameters atrous_rates = 6, 12, and 18, output_stride = 16, and decoder_output_stride = 4 for both training and prediction. The glomerulus model was trained for 400,000 steps and was initialized using the ImageNet model. The vessel segmentation models were trained for 100,000 steps, and the IFTA segmentation models were trained for 50,000 steps using the ImageNet model as a starting point for transfer learning. Details on the trained models are outlined in Table 1.

Table 1

Data used and models trained.

Tasks	Structures segmented	Models trained	Initialization for transfer learning	Training WSIs	Holdout test WSIs	Independent test WSIs	Training steps
Data and models
Glomeruli segmentation	Glomeruli	Glomerulus model	ImageNet	743	100	58 (GlomTestSet 2)	400,000
						17 (GlomTestSet 3)
Vessel segmentation	Glomeruli, Arterioles, Arteries		Random model	226	58	Qualitative assessment of publicly available GTEx tissue WSIs from multiple organs	100,000
			GTEx model
			Glomerulus model
			ImageNet
IFTA segmentation	IFTA, Glomeruli	Institution 1	ImageNet	12	29	17 (IFTATestSet 2)	50,000
		Institution 2	ImageNet	24
		Institution 3	ImageNet	12
		Combined 1/3rd	ImageNet	16
		Combined full	ImageNet	48
						26 (KPMPTestSet)
Murine model feature analysis	Glomeruli	Used Glomerulus model for segmentation				4 old, 4 young	0
						10 KKAy T2DN, 10 C57 control
						7 Db/Db T2DN, 7 Db/M control

Different segmentation tasks, corresponding trained models, segmented structures, an initial model used for transfer learning, whole slide images (WSIs) used for training, hold-out testing, independent testing, and training steps. We note that GlomTestSet 2 is the same as the vessel segmentation holdout dataset (58 WSIs). GlomTestSet 3 is also the same as IFTATestSet 2 (17 WSIs).

As part of the input pipeline, WSI patches can be extracted efficiently at downsampled resolutions. The patch downsample rate is user-specified, and multiple downsample rates can be specified during training, which are randomly cycled for patch extraction. For training, downsample rates of 1, 2, 3, and 4 with respect to the native slide resolution were used, a randomly selected downsample rate from the list was used for each extracted training patch. For prediction, a downsample rate of 2 was used for all experiments, we found this choice was a good compromise between prediction speed and accuracy. We believe that the multi-resolution training strategy helped the network to generalize. We found the glomerulus model works equally well in both 40X and 20X WSIs (both using a prediction downsample of 2). Further, the vessel segmentation model was trained using 40X WSIs, and successfully applied to the 20X GTEx WSIs for testing.

Using a large patch size for prediction increased segmentation performance, giving the network a larger field of view and reducing-edge artifacts. For practical purposes, we settled on a default patch size of 2000 × 2000 pixels. For prediction, it was found that using a stride of 1000 pixels gave sufficient overlap between extracted patches. During prediction, the indices of the extracted patches are tracked, and the resulting bitmap prediction is used to populate a full WSI mask using the similar method as discussed in the original H-AI-L study[7]. To reduce the number of artifacts at the edge of the predicted patches, a parameter to remove the border of the predictions was included. Practically this parameter was set to remove 100 pixels from the border of each prediction.

To improve speed and to keep the memory requirements of code implementation low, network predictions are not up-sampled. Instead, the coordinates of the extracted contours or heatmap indices are up-sampled prior to JSON creation. Using DeepLab parameters, namely, output_stride = 16 and decoder_output_stride = 4, result in a prediction bitmap that is 25% of the size of the input resolution. With a default downsample of 2 used for prediction, the resultant WSI mask is one-eighth of the size of the pixel resolution of the original WSI. We found that 32 GB of RAM is enough to successfully segment even very large slides.

When experimenting with the network logits for the generation of the ROC plots (Fig. 4a, b), we converted the code to stitch the patch predictions together by averaging the logits of overlapping patches.

Fig. 4

Interstitial fibrosis and tubular atrophy (IFTA) segmentation results—multi-institute study.

a Receiver operating characteristic (ROC) plots showing the segmentation performance of five trained IFTA models on 29 holdout whole slide images (WSIs), IFTATestSet 1. Models—Institution 1, Institution 2, and Institution 3 were trained using datasets from three different institutions (with 12, 24, and 12 WSIs respectively). The Combined full model was trained by pooling these three datasets (48 WSIs). The Combined 1/3rd model used 1/3rd of the pooled training set, randomly selected (16 WSIs). This last model yielded better IFTA segmentation performance than the first three models, highlighting the importance of dataset diversity. The combined full model offered slightly better performance than the Combined 1/3rd model. b shows the performance of the five models on the independent test dataset IFTATestSet 2 with 17 WSIs. This dataset originated from an independent institution than those used in [a] and was annotated by an independent annotator. We observed the same performance trend as in [a]. c shows the pairwise Intraclass correlation coefficients (ICC) (p value < 0.05) for percent IFTA scored visually by three additional annotators and estimated based on computational segmentation using the Combined full model (computer) for the 26 WSIs in KPMPTestSet. The kidney precision medicine project (KPMP) cohort acted as another independent test set which was never seen by our trained model. d shows computational IFTA predictions using the Combined full model on the holdout WSIs IFTATestSet 1. The left shows the traditional contour predictions, the right shows the corresponding heatmap predictions developed specifically for structures with poorly defined boundaries.

Statistical analysis

Intraclass correlation coefficient measure (ICC)[27,28] was used for the study shown in Fig. 4c, and corresponding r with null hypothesis r = 0 vs alternative r > 0 was used to measure significance. The ICC values were calculated using two-way random effects, absolute agreement, and single rater/measurement.