Imaging file management to support international telepathology.

BACKGROUND: Telepathology practice across international borders has become increasingly popular. Our telepathology consultation service with a laboratory in China was hampered by latency issues when viewing whole slide images.
OBJECTIVE: The aim was to explore data transfer solutions to improve the viewing experience of digital consult cases.
METHODS: Whole slide image files residing on a server in China were transferred to our data center in the USA using an open source product (Fast Data Transfer). A faster more automated commercial high speed file transfer software solution (Aspera) was also tested.
RESULTS: Transferring files with the open source product provided transfer speeds of 2-3 Mbps, but suffered from intermittent dropped connections. Employing commercial software permitted more reliable transmission of digital files with 75-100 Mbps transfer speeds.
CONCLUSIONS: Successful global telepathology requires dedicated image management. Transfer of files to local servers by employing high speed data transfer tools helped overcome network latency issues, improved the overall turn-around time of digital consultations, and enhanced the viewing experience for end-user digital consultants.

INTRODUCTION

Telepathology practice across international borders has become increasingly popular. Global digital pathology networks between pathology laboratories, academic centers and commercial entities have capitalized on whole slide imaging (WSI) technology, the Internet and cloud services. At the University of Pittsburgh Medical Center (UPMC) we launched a telepathology consultation service with KingMed Diagnostics Laboratory in Guangzhou, China. A customized web-based digital pathology consultation portal was developed for this purpose.[12] Glass slides were scanned using a Nanozoomer scanner (NanoZoomer 2.0-HT, Hamamatsu). WSI submitted for consultation resided on the client's server (Hammamatsu NDP.serve) in China. Pathology consultants from Pittsburgh in the USA used the web portal to securely access these images on the client's server. Workflow (e.g., case triage and transcription) and reporting was incorporated into this web-based application.

With the aforementioned information technology client-specific infrastructure, we were able to initially avoid lengthy transfers of large WSI files over the Internet. However, after 2 years of practice network latency issues increased that negatively impacted viewing of WSIs and hence interpretation of digital consultation cases. Reasons for the delay in processing network data were often hard to determine and, therefore, resolve. Ping tools identified decreased download and upload speeds for connections to servers in China compared with those located locally.[3] Although the current telepathology workload is manageable, the delays experienced by pathologists (lengthy time waiting for images to open, pixelated images when panning and zooming) were frustrating.

Therefore, the aim of our study was to explore various image file transfer solutions to improve the viewing experience of digital consult cases.

METHODS

To solve ongoing latency issues, two attempts were made to transfer WSI files from China to an image server at UPMC in our data center. The first solution employed an open source product (Fast Data Transfer [FDT], by CERN. FDT uses standard Transmission Control Protocol (TCP) and employs a zip utility. A command line utility was established to facilitate a batch transfer process of image files [Figure 1]. The second solution employed commercial file transfer software (Aspera). Aspera uses User Datagram Protocol (UDP), does not involve data compression, and permitted immediate file transfers that did not need to be initiated by a user [Figure 2]. The Aspera software was configured using a “hot folder” for automated file transfer. With this set up, every time a file was dropped into the folder on the server in China it was immediately transferred over to the UPMC server. Since Aspera can be configured to take up as much of the sender's bandwidth network pipe as needed, our team set it up to use 80 Mbps of the client's 100 Mbps Internet connection to transfer files off business hours. Once images were successfully transferred to the USA, when consultants logged into the web-based portal to access their digital pathology cases they were offered the option to view digital slides launched either from the server in China or locally in Pittsburgh.

Figure 1

Process steps involved using an open source tool to transfer images

Figure 2

Process steps involved using Aspera to transfer images

RESULTS

Table 1 compares file sizes, transfer times, and image quality parameters for the two transfer methods used. Transferring files with the open source product FDT provided fast transfer speeds of 2–3 Mbps, but suffered from intermittent dropped connections. Although these interruptions varied throughout the day, they were most marked at 6 am in Pittsburgh (Eastern Time Zone). This required constant monitoring to ensure complete transfers. The image transfers with FDT required 5–15 min of extra processing time to zip files and involved a batch process. It was also noted that the open source zip utility did not compress files much more than they were already being compressed by the Nanozoomer scanner (NanoZoomer 2.0-HT, Hamamatsu). Employing the commercial file transfer software, Aspera permitted much faster (75–100 Mbps) transfer speeds. Furthermore, no monitoring was required. Use of hot folders did not add any extra processing time to the transfers and avoided files having to sit in a queue. In addition, the file sizes being transferred with FDT were larger because the open source software zipped up multiple images (anywhere from 10 to 30 files) into one file, whereas Aspera sent over one image file at a time. FDT was used to transfer 8 cases (101 slides) including hematopathology and breast/gynecologic consults. Aspera was used to transfer 5 hematopathology cases (62 slides). All of these cases comprised large specimens that contained anywhere from 8-15 slides, except for 1 biopsy that included 2 slides.

Table 1

Data comparison between open source and commercial image transfer tools. The open source software file sizes represent multiple whole slide images batched together, whereas only one file at a time was transmitted with the commercial software

CONCLUSIONS

Successful global telepathology requires dedicated image management, which includes fast and reliable movement of digital data. Traditional transmission of data over the Internet can be slow due to congestion, degradation, erratic transfer speed, and bottlenecks (e.g., packet loss). Attempts in Radiology to improve data transmission time between medical imaging systems have been based mainly on image compression.[456] According to the American Telemedicine Association (ATA) clinical guidelines for telepathology, compression technology may be applied with telepathology so long as it does not compromise the image for clinical use.[7] The guidelines state that reversible (lossless) compression may always be used as there is no impact on the image. However, irreversible (lossy) compression may be used to reduce transmission time or storage space only if the resulting quality is sufficient to reliably perform the clinical task. Fortunately, prior studies have shown that image compression does not appear to noticeably affect image quality and diagnostic accuracy, at least for store-and-forward telepathology.[8] WSIs seem to also be compressible to high levels before impacting interpretation performance.[9]

Use of compression methods to improve data transmission time may not always be effective in high-speed networks. Newer transmission methods and bulk data transport tools, such as Aspera, which use alternative technology (e.g. UDP-based application-level protocols vs. traditional TCP-based transport) offer innovative solutions. For example, some investigators were able to improve Radiology image transmission time by using parallelism (increased number of parallel network streams) without compression.[10] It is important to be aware, however, that overwhelming a network with too many streams may cause congestion and therefore actually have a negative effect on throughput achieved. Despite the fact that transferring files from China to our local server delayed the availability of images for consultants to view by up to 24 h, this measure improved the overall turn-around time of digital consultations because pathologists found it easier to work with locally stored WSIs. Although we resolved the latency issue impacting our consulting pathologists, the exact reason causing this latency has not been clarified. Network latency is not limited to file transfers between China and the USA, but can be an issue anywhere in the world, even within the USA.

Although biological materials (slides and blocks with human tissue) cannot leave China to be sent to our institution, we are not aware of any regulations restricting the movement of images out of China. Nevertheless, cooperative approval was obtained from the Guangdong province government department. It is unclear how our telepathology experience was impacted by the Golden Shield Project (i.e. Great Firewall of China), which is China's censorship of the Internet. Using a combination of firewalls and proxy servers at Internet gateways China are able to analyze and manipulate Internet traffic. Nonetheless, transfer of digital files helped us overcome network latency issues, which in turn enhanced the viewing experience for end-user digital consultants.