Parathyroid Frozen Section Interpretation via Desktop Telepathology Systems: A Validation Study.

BACKGROUND: Telepathology can potentially be utilized as an alternative to having on-site pathology services for rural and regional hospitals. The goal of the study was to validate two small-footprint desktop telepathology systems for remote parathyroid frozen sections. SUBJECTS AND METHODS: Three pathologists retrospectively diagnosed 76 parathyroidectomy frozen sections of 52 patients from three pathology services in Australia using the "live-view mode" of MikroScan D2 and Aperio LV1 and in-house direct microscopy. The final paraffin section diagnosis served as the "gold standard" for accuracy evaluation. Concordance rates of the telepathology systems with direct microscopy, inter-pathologist and intra-pathologist agreement, and the time taken to report each slide were analyzed.
RESULTS: Both telepathology systems showed high diagnostic accuracy (>99%) and high concordance (>99%) with direct microscopy. High inter-pathologist agreement for telepathology systems was demonstrated by overall kappa values of 0.92 for Aperio LV1 and 0.85 for MikroScan D2. High kappa values (from 0.85 to 1) for intra-pathologist agreement within the three systems were also observed. The time taken per slide by Aperio LV1 and MicroScan D2 within three pathologists was about 3.0 times (P < 0.001, 95% confidence interval [CI]: 2.8-3.2) and 7.7 times (P < 0.001, 95% CI: 7.1-8.3) as long as direct microscopy, respectively, while MikroScan D2 took about 2.6 times as long as Aperio LV1 (P < 0.001, 95% CI: 2.4-2.7). All pathologists evaluated Aperio LV1 as being more user-friendly.
CONCLUSIONS: Telepathology diagnosis of parathyroidectomy frozen sections through small-footprint desktop systems is accurate, reliable, and comparable with in-house direct microscopy. Telepathology systems take longer than direct microscopy; however, the time taken is within clinically acceptable limits. Aperio LV1 takes shorter time than MikroScan D2 and is more user-friendly.

INTRODUCTION

Access to the medical services continues to be an issue for many rural and regional areas in countries such as Australia. In the Australian context, when frozen sections are provided to the hospitals without an in-house laboratory, a pathologist has to travel to the operating theatre, setup the equipment, and wait for and then process the specimen. The time taken by the pathologist in diagnosing the prepared slide(s) is a fraction of the total time expended in any particular case. The utilization of new and current technology to bridge this gap in a cost-effective and safe manner is of great interest to public and private health services. Telepathology has the ability to minimize the nonprofessional “down-time” and provide intraoperative consultations to the hospitals that do not have an in-house pathologist and may prove to be cost-effective.

In the 1980's, telepathology was first used to deliver pathology services remotely,[1] and since then there have been multiple technological advances.[2] Till date, a number of modalities, such as sending of selected static images, videotelemicroscopy, robotic microscopy, whole-slide imaging (WSI), and hybrid devices with both WSI and robotic microscopy capacities, have been used in telepathology for consultation, diagnosis, education, and quality assurance.[3] Recently developed are hybrid desktop systems which are compact, cost-effective, and provide remotely controlled live imaging.

Validation studies using various telepathology systems for the primary diagnosis of paraffin and frozen sections of various tissue types have been reported, and we were only able to identify two reports which included parathyroid cases (≤5 cases) in among frozen sections of other organs.[45] A very recent large sample-sized study of WSI of paraffin-embedded rather than frozen section specimens included more parathyroid cases (10 cases) and demonstrated its noninferiority to conventional microscopy for primary diagnosis.[6] However, no study has specifically addressed the use of telepathology utilizing desktop-sized hybrid devices for parathyroid frozen sections.

International guidelines recommend validating a telepathology system for its intended use before formally using it to deliver a service.[78] The purpose of this study was to validate two commercially available desktop telepathology systems for remote parathyroidectomy frozen section interpretation.

SUBJECTS AND METHODS

Ethics and case series

Ethics committee approval was obtained from the Western Sydney Local Health District Human Research Ethics Committee (HREC Ref: (4275) AU RED LNR/15/WMEAD/153).

Three pathology services in Sydney, Austpath Laboratories (Austpath), Sydney South West Pathology Service (SSWPS), and Institute for Clinical Pathology and Medical Research (ICPMR), each submitted 20 consecutive parathyroidectomy cases. The participating pathologists were not involved in the selection of the cases. At Austpath, the frozen section specimens were accessioned from June 1, 2007 to April 30, 2015, at SSWPS from February 1, 2015 to March 31, 2016, and at ICPMR from April 1, 2014 to April 30, 2015. No additional inclusion or exclusion criteria were applied. This resulted in a total of 60 parathyroidectomy cases and 86 frozen sections (25, 37, and 24 from each service, respectively). All cases were de-identified and assigned research accession numbers. The final paraffin section diagnosis for each frozen section specimen was recorded from the archived report. According to the paraffin section diagnoses, the final sample consists of 36 parathyroid adenomas, 26 parathyroid hyperplasias, 18 normal parathyroids, two lymph nodes, two normal thyroid tissues, one fat tissue, and one with unclear recording of paraffin section diagnosis.

Telepathology systems

Two compact, desktop sized, low-volume white light whole-slide scanners with both WSI and live-view modes were installed in Austpath as the central facility. MikroScan D2 (11 × 13 × 8 inches in size with two-slide loading capacity, MikroScan Technologies, Vista, California, USA) was initially installed in Austpath in 2013 for evaluation in a van-based mobile frozen section service. Aperio LV1 (17.9 × 15.9 × 20.5 inches in size with four-slide loading capacity, Leica Biosystems, Vista, California, USA) was loaned for 10 days in Austpath to perform this study in 2016. Each scanner included a motorized microscope with a set of objective lenses. Although WSI had been successfully demonstrated for frozen section interpretation by others,[49101112131415] we adopted their live-view mode instead of WSI during the study. WSI requires slide scanning and transfer of large files, which were more time-consuming in these two low-volume telepathology systems in addition to the challenge of internet congestion in the area where the study was conducted (4G connection was used). The live-view mode enabled remote users to navigate the slide and control the focus. This can be more useful when evaluating frozen sections which, not uncommonly, have tissue folds[3] and are usually thicker than paraffin sections.

Interpretation of parathyroidectomy frozen sections through telepathology and direct microscopy

Three pathologists (Pathologist 1, Pathologist 2, and Pathologist 3 from Austpath, ICPMR and SSWPS, respectively) interpreted the parathyroidectomy frozen sections in the same order of slides using Aperio LV1, MikroScan D2, and in-house direct microscopy (direct microscopy) in sequence without knowledge of the previous frozen section and paraffin section diagnosis. No counterbalancing of the study design was applied. There were more than 30 years of board-certified anatomical pathology experience for Pathologists 1 and 3 and 6 years for Pathologist 2. Pathologist 1 specialized in head-and-neck pathology. The specialty areas of Pathologist 2 were skin, breast, soft tissue, and endocrine pathology. Pathologist 3 had special interest in endocrine, breast, gynecology, and gastrointestinal pathology. All the three pathologists had no previous experience in digital pathology reporting except for reading WSI cases for a quality assurance program in which virtual microscope images have been used as survey material since 2006. A short machine training session of each telepathology system had been given to each pathologist before the formal conduction of the study. As per the guideline,[7] a washout period of ≥2 weeks between each examination was observed.

During each examination, one trained histotechnologist assisted with setting up the local scanner and computer, connecting to the remote computer through a third-party web conferencing application (WebEx Productivity Tools, [Cisco System Inc., USA] for MikroScan D2 and TeamViewer 11 [Team Viewer GmbH, Göppingen, Germany] for Aperio LV1), and loading the slides (one slide at a time). The stopwatch was started at the moment when the slide was loaded. The pathologist then remotely controlled the robotic microscope and viewed images on his personal computer (Pathologist 1: Same building different room; Pathologist 2: 1 kilometer away; and Pathologist 3: 20 kilometers away). Once the diagnosis was made, the pathologist typed the diagnosis into the web conferencing module. The histotechnologist then stopped the stopwatch and recorded the diagnosis and the time taken to reach the diagnosis.

For direct microscopy, each pathologist interpreted the slides through a conventional microscope in his own office. The on-site histotechnologist started the stopwatch as soon as the pathologist received the slide and stopped when the pathologist communicated the diagnosis.

Data analysis

The final paraffin section diagnosis for each slide was standardized as either “parathyroid” (including parathyroid adenoma, hyperplasia, and normal parathyroid) or “not parathyroid” (including lymph node, thyroid, and fat tissue) and served as the reference (“gold standard”) to determine the diagnostic accuracy. The reviewed frozen section diagnoses were similarly classified into the above two categories with the addition of “defer to paraffin section.”

Microsoft Excel 2010 (Microsoft Corporation) and IBM SPSS Statistics version 24.0 (IBM Corporation, Armonk, New York, USA) were used to analyze the data. The validity of all three systems and the concordance rates of telepathology systems with direct microscopy were calculated. The inter-pathologist and intra-pathologist agreement was evaluated by overall kappa values (Fleiss’ kappa[16] for three and Cohen's kappa[17] for two raters). Logarithmic transformation (ln) of time taken before analysis was applied to stabilize the variance of data. Repeated measures analysis of variance (ANOVA) of transformed data was conducted with “system” treated as a three-level factor within the three-level “pathologist” factor. Where a significant within “slides” effect was observed, the Bonferroni-corrected pairwise comparisons were performed. P < 0.05 was considered as statistically significant.

RESULTS

Data summary

Eighty-six frozen sections were accessioned and of these ten were excluded for technical reasons as follows: one for incorrect accession (not frozen section), four for failure of loading onto these telepathology systems because of broken and repaired slides, one for unclear record of final diagnosis, and four for significantly faded stains. Seventy-six frozen sections (72 “parathyroid” and 4 “not parathyroid” according to the final paraffin diagnoses) from 52 cases were analyzed.

Validity of telepathology systems

As shown in Tables 1 and 2, all three systems showed very high diagnostic accuracy in identifying parathyroid from frozen section specimens (100.0% for direct microscopy, 99.6% for Aperio LV1, and 99.1% for MikroScan D2). One normal parathyroid specimen (FST33B) was deferred by one pathologist in Aperio LV1, and there were no false diagnoses and the 100.0% of positive predictive value and negative predictive value (NPV) confirmed the efficiency of this system. Two normal parathyroid specimens (FST33B and FST19D) were falsely diagnosed as either “no parathyroid tissue” or “thymus” by one pathologist through MikroScan D2, which led to a false-negative rate (FNR) of 0.9% and NPV of 85.7% for this system. Retrospective analysis revealed that FST33B was mainly fat, and the pathologist had commented that it was a “terrible section.” FST19D was falsely diagnosed as “not parathyroid, thymus” by one pathologist through MikroScan D2 but correctly diagnosed as “parathyroid” by the same pathologist through the other two systems.

Table 1

Reviewed frozen section diagnosis through three systems compared with final paraffin sections

Table 2

Validity of direct microscopy, Aperio LV1, and MikroScan D2 (%, n=228)

Reliability of telepathology systems

When compared with direct microscopy, an overall concordance rate of 99.6% (1/228 discrepancy due to a deferral through telepathology) for Aperio LV1 and 99.1% for MikroScan D2 (2/228 discrepancies due to false-negative diagnosis) was observed. As shown in Table 3, the direct microscopy showed a perfect inter-pathologist agreement with an overall kappa value of 1. Aperio LV1 and MikroScan D2 also presented high inter-pathologist concordance with an overall kappa value of 0.92 and 0.85 (both >0.75), respectively.

Table 3

Inter-pathologist and intra-pathologist agreement of two telepathology systems and direct microscopy (Fleiss’ kappa)

The overall kappa values of intra-pathologist agreement [Table 3] also demonstrated a high replicability of frozen section diagnosis of parathyroidectomy specimens for each pathologist through three different systems (0.85–1, all >0.75).

Turnaround times

As shown in Figure 1 and Table 4, the median of time taken to reach the diagnosis of each parathyroid frozen section slide by pathologist 1, 2, and 3 was 45 s, 20 s, and 20.5 s through direct microscopy; 105 s, 60 s, and 78.5 s through Aperio LV1; and 185 s, 200 s, and 223 s through MikroScan D2, respectively. The statistical analysis of the log-transformed data, repeated measures ANOVA with Greenhouse-Geisser correction followed by Bonferroni-corrected pairwise comparisons, revealed that Aperio LV1 and MikroScan D2 took about 3.0 times (P < 0.001, 95% confidence interval [CI]: 2.8–3.2) and 7.7 times (P < 0.001, 95% CI: 7.1–8.3) as long as direct microscopy within three pathologists, respectively. MikroScan D2 took about 2.6 times as long as Aperio LV1 (P < 0.001, 95% CI: 2.4–2.7).

Figure 1

Boxplot of time taken per slide of parathyroid frozen sections reviewed by three pathologists through three systems (n = 76). As the data showed large variance in some groups, a logarithmic transformation of the data was applied before the repeated measures analysis of variance and post hoc tests. (The dark line in the middle of the boxes is the median of the values of time taken. The small circles (○) represent the outliers with values more than 1.5 × interquartile range. The asterisks (*) are extreme outliers with values more than 3 × interquartile range. †Another pathologist (Pathologist 4) instead of Pathologist 2 interpreted all the 76 slides through Aperio LV1.) s, seconds; PG, Pathologist; IQR: interquartile range

Table 4

Summary of time taken per slide by three pathologists through three systems and repeated measures analysis of variance of log-transformed data with post hoc tests

DISCUSSION

In this study, we specifically validated two small-footprint, low-volume, hybrid telepathology systems for parathyroidectomy frozen section interpretation. We demonstrated a very high diagnostic accuracy of both systems (99. 6% for Aperio LV1 and 99.1% for MikroScan D2 compared with the paraffin section diagnosis). A meta-analysis of frozen sections in 2000 revealed a slightly lower overall diagnostic accuracy of 0.91 for telepathology compared to 0.98 for conventional direct microscopy.[18] At that time, images were generated and transmitted through online video conference mode or as selected static images. The application of dynamic (including digital camera and robotic microscopy) or WSI telepathology in frozen section diagnosis of a variety of tissue types has demonstrated an improved diagnostic accuracy (from 88.2% to 100% throughout 16 years, with an average of more than 95%).[591011121920212223242526]

Parathyroidectomy frozen sections can be less challenging with a narrower range of diagnostic possibilities than those in neuropathology and general surgery. In line with our usual diagnostic practice, the frozen section diagnosis in this study was categorized as “parathyroid,” “not parathyroid,” or “defer to paraffin section.” Two normal parathyroid specimens were incorrectly diagnosed as “not parathyroid” through MikroScan D2, which led to an FNR of 0.9%. One of these could be attributed to the sample and section quality, as it was recorded that the specimen was fatty and the section was obtained with difficulty. The other was a diagnostic error of the pathologist rather than an inferior performance of the telepathology system.

When compared to the direct microscopy, an overall concordance rate of 99.6% for Aperio LV1 and 99.1% for MikroScan D2 was recorded. The overall kappa values of intra-pathologist concordance within these two telepathology systems and direct microscopy reached 0.85–1 for each pathologist. The inter-pathologist agreement for these two telepathology systems similarly reached a high level (overall kappa value of 0.92 for Aperio LV1 and 0.85 for MikroScan D2). This means that both telepathology systems were comparable to the direct microscopy and were sufficiently reliable to be used by different pathologists or the same pathologist at different times. Similar high levels of concordance between dynamic/WSI telepathology and in-house conventional microscopy in frozen section diagnosis have also been reported.[1245121524]

In this study, the diagnosis by both telepathology systems took longer than direct microscopy. Aperio LV1 took 3.0 times, and MikroScan D2 took 7.7 times as long as direct microscopy (both P < 0.001). Preliminary scanning, transmission of the remote operation commands and response of the robotic microscope to the commands, digitizing and transmission of the images, and prolongation of transmission due to internet congestion contributed to the longer time taken by the telepathology systems.

Published turnaround time for the frozen section by telepathology varies[9101415192226] and depends mainly on the specimens interpreted, devices used, and pathologists’ experience in frozen section diagnosis and using digital images.

The time taken by telepathology diagnosis of parathyroid frozen sections in this study (median value around 4 min per slide) was shorter than those reported in neuropathology (around 10 min per slide[1026]) and similar to those recorded in studies of mixed tissue types (around 3 min per slide[5]).

Evans et al.[10] recorded a significantly lower time taken by their newly implemented WSI scanner than the previous robotic microscopy system during their routine telepathology frozen section practice. In this study, we reviewed the same cases through two similar telepathology systems and still observed a significantly shorter time taken for Aperio LV1 than for MikroScan D2 (P < 0.001). A more user-friendly operating experience with Aperio LV1 was also informally recorded (no questionnaires were performed). MikroScan D2 and Aperio LV1 were released in 2010 and 2015, respectively.[2728] The 5-year gap in an actively evolving field could be one of the main reasons for the differences.

Besides the issue of the longer time taken, a broken, oversized, or suboptimal slide can be a challenge to the telepathology systems. However, in the clinical situation, this will be rectified by cutting and staining another section.

Our study has demonstrated the accuracy and reliability of telepathology in the frozen section diagnosis of parathyroidectomy specimens. For the surgeons using this service, a 6-min prolongation was acceptable and did not adversely impair best patient care. Given this short delay, regional and country hospitals would be able to consider this as a financially viable alternative to having an on-site frozen section service. However, as the order effects of the repeated measures design of the study was not controlled by counterbalancing, unplanned change of the participating pathologist occurred for part of the study, and relatively few “not parathyroid” cases were included as subjects in the sample, systematic and sampling biases might influence the outcomes of this study. Additional studies with more powerful design and greater case numbers and evaluation of costs will be needed to confirm and reinforce our findings before implementing telepathology for parathyroid frozen sections more widely in clinical practice. Training on any specific telepathology system is still necessary, and each laboratory will need to formulate its own protocols.

CONCLUSIONS

Telepathology diagnosis of parathyroidectomy frozen sections through small-footprint desktop systems is accurate, reliable, and comparable with in-house direct microscopy. Telepathology systems take longer than direct microscopy; however, the time taken is within clinically acceptable limits. Aperio LV1 takes shorter time than MikroScan D2 and is more user-friendly.

Financial support and sponsorship

Nil.

Conflicts of interest

There are no conflicts of interest.