Robotic Telecytology for Remote Cytologic Evaluation without an On-site Cytotechnologist or Cytopathologist: An Active Quality Assessment and Experience of Over 400 Cases.

BACKGROUND: The first satellite center to offer interventional radiology procedures at Memorial Sloan Kettering Cancer Center opened in October 2014. Two of the procedures offered, fine needle aspirations and core biopsies, required a rapid on-site cytologic evaluation of smears and biopsy touch imprints for cellular content and adequacy. The volume and frequency of such evaluations did not justify hiring on-site cytotechnologists, and therefore, a dynamic robotic telecytology (TC) solution was created. In this article, we provide data on our experience with this active implementation. Sakura VisionTek was selected as our robotic TC solution.
METHODS: A retrospective analysis of all TC evaluations from this satellite site was performed. Information was collected on demographics, lesion location, imaging modality; a comparison of TC-assisted adequacy with final adequacy was also conducted.
RESULTS: An analysis of 439 cases was performed over a period of 23 months with perfect correlation in 92.7% (407/439) of the cases. An adequacy upgrade (inadequate specimen becomes adequate) in 6.6% (29/439) of the cases. An adequacy downgrade (adequate specimen becomes inadequate), is near zero at 0.7% (3/439) of the cases.
CONCLUSIONS: Dynamic robotic TC is effective for immediate evaluations performed without on-site cytotechnology staff. The overall intent of this article is to present data and concordance rates as outcome metrics. Thus far, such outcome metrics have exceeded our expectations. Our TC implementation shows high, perfect concordance. Adequacy upgrades are minor but more relevant and impressive is a near zero adequacy downgrade. Our full implementation has been so successful that plans are in place for configurations at future satellite sites.

INTRODUCTION

In the past, clinical services at our institution were centralized geographically. Memorial Sloan-Kettering Cancer Center (MSK) has since expanded to include multiple satellite centers across the New York metropolitan area that provide a consistent standard of care in locations that are more accessible for our patients. One remote center in West Harrison, NY opened in October of 2014, located approximately 30 miles north of the main campus on the upper east side of Manhattan and serving residents of the upper New York City area and upstate New York. This remote center was promoted, offering minimally invasive interventional radiology procedures for diagnostic purposes; a trend becoming more prevalent in recent years. Such procedures utilize cytologic evaluation such as fine needle aspiration (FNA) and core biopsies (CBs) for cancer diagnoses.

FNA and CBs mandate rapid on-site adequacy evaluation (ROSE) of smears and biopsy touch imprints for cellular content and adequacy. ROSE guides the person performing the procedure in deciding whether to stop or to obtain additional tissue. For instance, with lymphomas, immediate cytologic adequacy assessments are critical in determining sufficient tissue for flow cytometry and molecular studies in addition to obtaining CBs.

An anticipated low ROSE volume and frequency for this remote center could not justify hiring cytotechnologists on-site. The implementation of telecytology (TC) enabled us to perform ROSE without the physical presence of a cytotechnologist at this facility. In this research article, we provide outcomes metrics that demonstrate evidence of efficacy for our robotic TC solution and architecture. A corresponding technical note describes in detail the architectural framework along with the cultural and organizational shifts that occurred as a result of the implementation.

METHODS

The radiologist performs the smear and stains them. They are trained and tested for their proficiency before they can use the robotic microscope for adequacy assessment. Cytotechnologist evaluation of the specimen is performed remotely utilizing robotic microscopy and desktop sharing applications over the institutional intranet. Sakura VisionTek (SkVT) was selected as our dynamic robotic TC solution. Image viewing and control are remote via the intranet with the use of HD monitors on the main campus. The HD monitors are 24 inches with a minimum resolution of 1080 pixels linked to a CPU with 8 GB of RAM. Domain network access and speed are optimal at 1 GB/s at both satellite location and the main campus. Desktop sharing to operate the SkVT is performed behind the institutional firewall through the remote desktop application. MSK's WebEx application enables the interventional radiology team to share the remote viewing session with cytotechnologists. The cytotechnologist is in close communication with the radiologist by phone, so s (he) can obtain all relevant directly from the radiologist as well as communicate the adequacy assessment results. This is further detailed in the corresponding technical note.

Before implementation, validation was performed for the SkVT system with twenty different cases reviewed by two cytotechnologists and a cytopathologist, and each reviewer had a discrepancy rate of <10%. After implementation, we performed retrospectively an analysis of cellular content and adequacy for FNAs and touch preparations of CBs for all consecutive remote TC ROSE procedures obtained in a period of 23 months using SkVT dynamic robotic telepathology. Patient age and sex data were collected along with lesion location and acquisition modality (i.e., ultrasound [US], computed tomography, and/or fluoroscopy).

Cellular content and adequacy were determined based on correlation with clinical-radiological findings available at the time of the procedure. A determination of adequacy at the time of procedure means that sufficient tissue is available for a diagnostic evaluation performed later to render the final cytologic interpretation. This diagnostic evaluation includes sufficient tissue availability to perform the necessary immunohistochemistry, molecular, and flow cytometric studies. A determination of adequacy at the time of procedure translates into a prompt halt to further procedural attempts in obtaining tissue. The adequacy was recorded in paper form, and the information was transferred to the final cytology report. No deferrals were issued and the specimens were either considered adequate or inadequate. Only adequacy assessments and not preliminary diagnoses are officially provided on-site.

The preliminary ROSE adequacy assessment obtained at the time of the procedure was then compared to the final cytopathologist-rendered adequacy assessment rendered when all the material obtained during the procedure was available for review. Concordance was defined as correlation between the preliminary adequacy assessment and the final cytopathologist-rendered adequacy assessment (either adequate or inadequate). A perfect concordance would be when both the preliminary adequacy assessment and the final cytopathologist-rendered adequacy assessment are the same (either both adequate or both inadequate). Because adequacy determination at the time of procedure did not include officially preliminary diagnoses, concordance meant that the appropriate level of ancillary testing could be performed to enable a more definitive final cytologic interpretation.

An adequacy upgrade occurred when the preliminary adequacy assessment was considered inadequate, but the final cytopathologist-rendered adequacy assessment was determined to be adequate. An adequacy downgrade occurs when preliminary adequacy assessment was deemed adequate but the final cytopathologist-rendered adequacy assessment was determined to be inadequate.

RESULTS

Figures 1 and 2 are screenshots of the SkVT user interface. Table 1 presents data on demographics and lesion location. The mean age was 61, and there was a slight female predominance. Table 2 shows data on imaging modality. The majority of procedures requiring rapid on-site evaluation used US guidance. Table 3 compares TC preliminary adequacy and final cytopathologist-rendered adequacy in a two-by-two matrix.

Figure 1

Opening screen of the Sakura VisionTek showing four slide thumbnails which can be opened for live viewing

Figure 2

Live view of a cytologic preparation for cellular content and adequacy

Table 1

Demographics and lesion location

Table 2

Acquisition modality

Table 3

Comparison of robotic microscope assisted adequacy with final adequacy assessment

Our study shows that TC preliminary adequacy assessment is highly concordant with the final cytopathologist-rendered adequacy assessment. Over a 23 months period, 439 consecutive cases were performed with perfect correlation in 92.7% (407/439) of the cases. An adequacy upgrade (inadequate specimen becomes adequate) occurred in 6.6% (29/439) of the cases. This is not necessarily unexpected because only part of material is available during the initial review. An adequacy downgrade (adequate specimen becomes inadequate) is near zero at 0.7% (3/439) of the cases. The latter is the more relevant metric.

DISCUSSION

ROSE for minimally invasive procedures has improved patient care by decreasing the need for repeat procedures and more effectively triaging material for ancillary studies.[123] Only a small fraction of minimally invasive procedures require ROSE. The number of ROSE requests, however, increased due to our institution's rapid growth within the New York metropolitan area.

The allocation of a dedicated cytotechnologist for on-site coverage was not cost-effective due to the relatively low anticipated volume of ROSE requests at the emerging facility. When the first satellite center to offer interventional radiology procedures opened in October 2014, the cytology service encountered the challenge of providing ad hoc adequacy assessments without on-site cytotechnologists. Finding a feasible TC solution became an imperative.

The vast majority of TC options utilize static or live streaming.[45] Finding the regions of interest on the cytologic preparation is required for both modalities and proves difficult without having on-site cytotechnologists. Requiring the radiology team to find regions of interest was simply unreasonable.

Digital slide scanning technology for TC was proposed as a viable option.[6] We investigated this solution for our large cytologic adequacy assessment operation. After evaluating several vendors, challenges associated with digital slide scanning became apparent. The time required to scan the slides with Z-stacking was particularly problematic. In addition, the robotic functionalities were unable to provide the quick depth of field focusing needed for cytologic evaluation. Such barriers precluded an efficient ROSE workflow at our satellite facility, and we, therefore, implemented dynamic robotic TC so that the centralized cytotechnologist team could support the site remotely. With this dynamic robotic TC solution, we, fortunately, have not experienced any instances in which the system was down, although we had some instances in which the image could not be assessed due to temporary networking issues. The network issues resulted in a delay in adequacy assessment but did not prevent assessment. As a contingency protocol, if adequacy cannot be given, the radiologist is advised to three passes without adequacy assessment if clinically feasible.

The American Telemedicine Association has advocated performance validation for telepathology assessments for clinical diagnostic purposes with system deployments.[7] There are no recommendations as to the number of cases, design, and benchmark metrics, however. Due to the lack of guidance on outcomes data, criteria used for optimal benchmark measurement and validation of user competency and TC validation has been inconsistent.[8]

Of the numerous telepathology studies available, considerable variability exists in case number, design, and captured benchmark and validation metrics. Remote dynamic robotic functionality in pathology has been utilized more successfully in frozen section diagnosis and noncytology teleconsultation.[9101112131415161718] Experience with remote dynamic robotic functionality in cytology is less common[19202122] with static and live streaming modalities being utilized more regularly.

Ours is a description of our experience from a quality assurance perspective and actually one of the largest active evaluations of dynamic robotic TC (n = 439) relative to other successful series that utilize this functionality of the SkVT.[2324] Several recent reported dynamic robotic TC series are nonactive,[192023] and therefore, unable to accurately measure the performance of TC in a live environment with time constraints and other pressures. Our study being an active implementation provides the evidence for extensibility for other institutions of ROSE with robotic TC in an actual time-sensitive intraoperative environment.

The effectiveness of our active TC implementation was validated through the analysis of concordance of cellular content and adequacy between interpretations rendered from TC images versus slides reviewed during final cytologic interpretation. Cellular content and adequacy is the quality outcome measure we evaluate for ROSE at our institution, which is driven by cytotechnologists rather than cytopathologists. ROSE for adequacy has historically been driven by cytotechnologists and not cytopathologists at our institution. Cytotechnologists are only permitted to assess and render determinations for cellular content and adequacy of cytologic preparations and therefore rendering an intraoperative cytologic diagnosis is not part of our standard ROSE workflow.

Cytologic diagnosis is arguably a different mental task from evaluating cellular content and adequacy. Although cytotechnologists only perform adequacy assessments without providing final diagnoses, some knowledge of cytologic diagnosis is required to perform assessments of cellular content and adequacy. For instance, an adequacy rendering in a tumor specimen cannot be performed without diagnostic recognition for the presence of tumor cells in the preparation.

Our series appears to be the first to evaluate cellular content and adequacy as a quality outcome measure.[192023] Other studies assess TC effectiveness via diagnostic agreement rates with cytologic diagnosis retroactively. Our study is thus applicable to similar institutions with cytotechnologist driven ROSE services where preliminary cytologic diagnoses are not rendered and where more definitive assessment for cytologic diagnoses are performed after appropriate processing of cytologic specimens.

In our series, perfect concordance or accuracy of the preliminary adequacy assessment versus the final cytopathologist-rendered adequacy assessment for cellular content and adequacy was at 92.7% (407/439). This was comparable to our prior conventional on-site evaluation rates as well as previous reports, which have demonstrated an 80%–95% concordance rate for TC and 66.7%–97% for conventional on-site methods.[202526272829] Several factors contributed to this high perfect concordance rate including our cytotechnologist diagnostic skills, usability ease and functionality of the robotic microscope, and our cytotechnologist workflow familiarity with providing independent adequacy assessment before the introduction of this TC implementation in our institution.

Our adequacy upgrade rate in which lesions initially designated as inadequate became adequate on the evaluation of the entire specimen was low at 6.6% (29/439). Experiencing adequacy upgrades is not unexpected because all diagnostic material may not be available at the time of preliminary adequacy assessment. Furthermore, most of the upgraded cases were soft tissue lesions or lesions associated with marked fibrosis. This factor contributes to adequacy upgrades because smears and touch preparations of such lesions yield few cells on initial adequacy assessment. Diagnostic cells only become appreciable when the entire specimen is evaluated at final cytopathologist-rendered adequacy assessment.

Most impressive in our study is the adequacy downgrade rate in which lesions initially considered adequate were later deemed inadequate. This discordance happened in only 0.7% (3/439), an extraordinarily low number when considering our large case volume. In our opinion, an adequacy downgrade is the most critical metric because a preliminary adequacy assessment that is incorrectly designated as adequate will often prompt the clinician to end the procedure. A required repeat procedure not only creates anxiety for patients, but it also exposes them to morbidity and complication risks associated with additional procedures.

The leading cause of downgraded cases was misinterpretation of benign cells as malignant cells. This occurred in three cases; two of which were from liver and one from kidney [Table 3]. In the liver cases, there was an over-interpretation of reactive hepatocytes for hepatocellular carcinoma. Hepatocellular carcinoma is one of the entities notoriously difficult on adequacy based on smears. Particularly with well-differentiated hepatocellular carcinoma, the diagnosis is nearly always based on histologic sections and rapid on-site evaluation based on smears.

Similarly, with the kidney case, reactive tubular cells were misinterpreted for renal cell carcinoma. Such over interpretations are not necessarily a product of robotic TC and arguably still occur with conventional on-site cytologic evaluation. In such difficult cases like those involved in adequacy downgrades, the issue is communicated to the radiologist. As with our perfect concordance rates, we acknowledge that the skill level of our cytotechnologists contributes to this low adequacy downgrade rate and that similar adequacy downgrades may not be replicable in different institutions. In terms of the training and experience of our cytotechnologists, a large majority of our cytotechnologists are graduates of our cytotechnology school and have been exposed to a large volume of adequacy assessment procedures. Our cytotechnologists are continuously monitored for their discrepancy rates, and discrepancies are reviewed and discussed in our monthly quality assurance conferences.

CONCLUSIONS

Dynamic robotic TC has been effective at our institution for on-site evaluations when cytopathology staff is not physically present. Our dynamic robotic TC implementation and experience of over 400 cases showed high perfect concordance. Cases with adequacy upgrades were minor but not unexpected since only part of material is available during the initial review. Most relevant was a near zero adequacy downgrade of cases. We acknowledge that the skill level of our cytotechnologists contributes to these findings and that similar adequacy downgrades may not be replicated in every institution. We feel that because our series documents actively cases in real-world settings, it has applicability and extensibility for institutions with large-scale ROSE operations driven by cytotechnologists.

Financial support and sponsorship

MSK Cancer Center Support Grant/Core Grant (P30 CA008748).

Conflicts of interest

There are no conflicts of interest.