A study assessing the feasibility and diagnostic accuracy of real-time teledermatopathology.

Dermatopathology represents the gold standard for the diagnosis of skin diseases and neoplasms that cannot be diagnosed on clinical grounds alone. The aim of this study was to test the feasibility and to assess the accuracy of an Internet-based real-time (live) teledermatopathology consultation. Twenty teaching cases and 10 randomly selected routine cases were presented to four expert dermatopathologists, first by real-time teledermatopathology and, subsequently, in a blinded fashion, using light microscopy. Throughout the study the overall diagnostic accuracy did not differ for the two methods. However, the mean level of confidence and the mean observation times differed significantly between real-time teledermatopathology and light microscopy (92.6±0.24% versus 99.5±0.02%, and 96.31±11.55 sec versus 25.47±3.85 sec, respectively). Assessment of routine cases did not produce significant diagnostic differences between the two methods. These results prove that real-time teledermatopathology offers an affordable and technically simple technology that lends itself to training as well as to diagnosis of lesions from routine practice by experts situated at remote sites.

Introduction

Dermatopathology is generally regarded as the gold standard for identification with specificity of those skin diseases that cannot be diagnosed on clinical grounds alone. In many instances those same skin diseases are diagnostically vexing when studied by conventional microscopy, and identification of them often requires expert consultation not readily available. This problem is most prominent in the diagnosis of melanocytic skin lesions. Dermatopathologic diagnosis of melanoma represents a special circumstance because it is not based on a single criterion, but on a constellation of criteria that are not applied consistently by individual dermatopathologists [1-3]. That being so, a true gold standard is lacking and the availability of experts gains increasing importance to guarantee a specific diagnosis and the best available treatment according to the diagnosis for the patient.

The introduction of teledermatopathology into the routine practice of dermatopathology offers several opportunities for monitoring and improving the quality of diagnosis of “difficult cases” not only in the realm of melanocytic neoplasms, but in every area of pathology in general [4,5]. The broad application of teledermatology in dermatopathology has been hampered by several obstacles, among them technical obstacles, that have prevented the use of it routinely [6]. Methods of teledermatopathology include electronic transmission of still images [7], distant control of a motorized microscope, real-time transmission of digital images from the microscope (videoconferencing), and the assessment of digital images with a high resolution in combination with software that enables the user to load only the desired part of the huge image files (“virtual microscopy”) [8]. Although “virtual microscopy“ represents the latest development in telepathology and is an attractive option for the diagnosis of selected cases, it is time consuming and technically challenging [9]. New Internet-based software, like Skype™, now permits establishment of real-time teledermatopathology, a dynamic method that offers an affordable and technically simple alternative to that what was employed before (Ref: http://www.HL7.com.au/Skype-Video-Conferencing.htm).

In this study we tested the feasibility and accuracy of such a method. This objective was obtained in two phases, the first one designed to prove the principle of real-time teledermatopathology and the second one to set up a field experiment to verify its feasibility for routine histopathologic evaluation of skin lesions.

Materials and methods

Study cases

The Ethics Committee of the Medical University of Vienna approved this study. For the initial phase of the study, 20 specimens of skin lesions with clear-cut diagnoses were selected and designated as “teaching cases” after they had been reviewed and had been diagnosed without any ambiguity by at least two dermatopathologists at the Department of Dermatology, Medical University of Vienna, Vienna, Austria. These “teaching cases” included benign and malignant skin lesions and biopsies of inflammatory skin diseases (Table 1). For the second phase of the study, a total number of 10 cases was randomly selected during routine consultation over a period of two weeks and designated as “routine cases.” For each set of cases, i.e., teaching cases and routine cases, a Skype™ conference was initiated with the remote specialist at the Ackerman Academy of Dermatopathology, New York, NY, USA, and cases were presented in real-time to each of the four participating dermatopathologists who made the diagnosis of the cases “on the screen“ over the Internet. Upon request, information regarding medical history, clinical setting and clinical diagnosis was provided. The diagnosis, differential diagnoses, level of confidence in the diagnosis, and the time needed to make a diagnosis, were recorded. After two weeks, the actual slides were reviewed by the same dermatopathologists in blinded fashion using conventional light microscopy. The same parameters that had been recorded during the teledermatopathology session were noted. The quality of diagnosis rendered remotely compared to the diagnosis “under the microscope“ was subjected to statistical analysis.

TABLE 1

Histopathologic diagnoses of slides used for teledermatopathology and microscopy including 20 “teaching cases” (left column) and 10 “routine cases” (right column)

	Part a “Teaching Cases”		Part b “Routine Cases”
1	Granuloma annulare	1	Desmoplastic melanoma
2	Reed’s nevus	2	Basal cell carcinoma
3	Invasive melanoma	3	Pityriasis lichenoides chronica*
4	Scabies	4	Irritated seborrheic keratosis
5	Squamous cell carcinoma	5	Pityriasis rosea*
6	Isthmus catagen cyst	6	Basal cell carcinoma
7	Basal cell carcinoma (BCC)	7	Eczema/dermatitis
8	Syringoma	8	Lichen sclerosus et atrophicans
9	Xanthogranuloma	9	Neurofibroma
10	Melanoma in situ	10	Nodular melanoma*
11	Blue nevus
12	Chronic pigmented purpura
13	Hidrocystoma
14	Psoriasis
15	Dermatofibroma
16	Herpes simplex
17	Leukocytoclastic vasculitis
18	Lichen planus
19	Congenital melanocytic nevus
20	Keratoacanthoma

Diagnosis of these cases included one ore more differential diagnosis as discussed in the text.

Technical equipment

Skype™ is an Internet-based communication software that offers free Internet calls using a headset or free video calls over a web camera. Teledermatopathology sessions were initiated by sending a request for teleconsultation to the remote study site. During the session, a corporate analysis of sections by video conferencing took place in real-time. The system consisted of a remotely controlled microscope (Olympus microscope BX41-TF5; Olympus, Tokyo, Japan) attached to a digital video camera equipped with live video stream transmitted at 800 x 600 pixels at 30 frames per second. An Internet-connected personal computer with a Window XP operating system (Microsoft, Redmond, VA) was used for the study. LAN connected the two study sites at a maximum transmission rate via Internet of 54 Mbit/sec. This system allowed remote operation of all the movable parts of the light microscope; the video signal was shared between the client at the center where the session was initiated and the expert center and was displayed on a viewing screen. In addition, an audio connection was established via Skype™.

Outcome, quality measurements, and statistical analysis

The observer’s diagnosis “on the screen” was contrasted with the diagnosis “under the microscope” (looking at the same section of tissue in a blinded fashion). Each time the observer was asked to provide the following information: a specific diagnosis, the level of confidence in that diagnosis, and one or more differential diagnoses. A comparison between the proportion of correct specific diagnoses by each mode of examination, the plausibility of the specific and differential diagnoses and the level of confidence in the specific diagnosis was performed. With regard to the specific diagnosis, the agreement between both modalities was calculated. In addition, the time needed for each case by teleconsultation was recorded and compared with the time needed for the diagnosis of the sections “under the microscope.” Each participant was asked to assess the quality of transmission and of the slides, respectively. Finally, the request of the observer for additional clinical information was recorded.

Results

Characterization of study cases for study phases 1 and 2

According to the study protocol, cases differed between the two parts of the study in regard to diagnostic difficulty, types of diseases and unambiguity of diagnosis. Since all specimens came from the same institution, the quality of the slides was comparable throughout the study. Accordingly, study participants rated the technical quality of the slides as “good” or “excellent” (data not shown). Moreover, all specimens were prepared from punch biopsies, biopsies or excisions. While the “teaching cases” included stereotypic presentations of common skin diseases (Table 1), “routine cases” that were randomly collected during a two-week period also included cases without a clear-cut diagnosis (Table 1). Specifically, case 3 was diagnosed as pityriasis lichenoides chronica, but a drug eruption or skin lesion of lupus erythematosus could not be ruled out by the dermatopathologist on site. Similarly, case 5 came with the provisional diagnosis “pityriasis rosea, rule out eczema or psoriasis.” Finally, for case 10 no distinction between a primary melanoma and a melanoma metastasis was provided during initial diagnosis.

All study participants were experienced dermatopathologists. Two were board-certified dermatologists and two board-certified pathologists. Furthermore, there was some heterogeneity in regard to duration of professional experience and places of dermatopathology training. However all participants shared at least a one-year period of dermatopathology training at the same institution.

Assessment of “teaching cases” proves the principle that real-time teledermatopathology compares to conventional light microscopy in regard to diagnostic accuracy

The first phase of the study was set up as a proof of principle to test the method of real-time teledermatopathology. Remote “teleconsultation” diagnoses of 20 teaching cases (Table 1) were compared with diagnoses when assessing the same sections directly “under the microscope.” Overall diagnostic accuracy, level of confidence and time needed to come to a diagnosis were evaluated. Figure 1a illustrates that the overall diagnostic accuracy did not differ for the two methods. However, the overall mean level of confidence differed significantly between teleconsultation and direct slide assessment by light microscopy (92.6±0.24% versus 99.5±0.02%, p=0.008). As can be seen from Figure 1b, teledermatopathology was also associated with a higher degree of interobserver variability in terms of confidence in the diagnosis. In line with this, observers relied more on additional clinical information (i.e., biopsy site, number of lesions, age of patient) when making the “on screen” diagnosis (data not shown). Finally, as shown in Figure 1c, the time that was needed to come to a diagnosis was significantly longer for teleconsultation sessions (96.31±11.55 sec versus 25.47±3.85 sec, p<0.001).

Figure 1

Proof of principle of live teledermatopathology evaluating diagnostic accuracy, level of confidence and time to diagnosis during assessment of “teaching cases.” a. Bars represent the proportion of correct (corr) diagnoses during “online” (open bars) and “slide” (closed bars) assessment under the microscope by individual observers (A, B, C, D). b. Bars represent the percentages of cases that yielded a confidence level of 100% by individual observers (A, B, C, D) during online” (open bars) and “slide” (closed bars) evaluation. c. Bars represent time for diagnosis (sec) needed by individual observers (A, B, C, D) during teleconsultation (left panel) and direct slide assessment (right panel). P<0.001 between the two groups.

Real-time teledermatopathology and direct slide assessment by light microscope of routine cases yield a high degree of interobserver agreement

Randomly selected routine specimens were used to compare the accuracy of teleconsultation with direct slide assessment by light microscopy. This field experiment was designed to mirror the actual situation of routine dermatopathology consultation. Of the 10 randomly selected cases, three specimens had been signed out originally without a final diagnosis (Table 1). According to the “real-life” setting, rather than giving the number of correct diagnoses, interobserver agreement between teleconsultation and direct slide evaluation and the time needed to come to a diagnosis were assessed. Figure 2a shows that the proportion of cases in which all observers agreed did not differ significantly between the two methods: Observers agreed in six out of 10 cases with the “on screen” diagnosis compared to seven out of 10 cases of “under the microscope” diagnosis. Comparable to the results obtained during phase 1, Figure 2b shows that the duration to come to a diagnosis was significantly longer for the method of real-time teledermatopathology compared to direct slide evaluation by light microscopy (112.05±19.95 sec versus 35.43±7.47 sec, p<0.001). Moreover, the time for diagnosis differed significantly between study observers (p=0.027, Figure 2b). Further analysis revealed that the time difference between teleconsultation and direct slide assessment is observer dependent and statistically significant (p= 0.001). This data show that teleconsultation compares to direct slide assessment under the microscope in terms of diagnostic accuracy.

Figure 2

Comparison of level of agreement and time needed for diagnosis in the evaluation of “routine cases.” a. Bars represent percentage of cases with perfect agreement among observers during teleconsultation (left) and direct slide assessment (right). b. Bars represent time needed for diagnosis (sec) by individual observers (A, B, C, D) during teleconsultation (left panel) and direct slide assessment (right panel).

Discussion

Real-time teledermatopathology represents a novel technique allowing for remote interactive teaching, expert consultation of difficult cases and second opinion consultation in sites where availability of pathologists is limited [10]. The live teledermatopathology sessions include the interactive control of a microscope by a remote presenter and a viewing pathologist. Slides can be viewed in their entirety at different magnification levels, and the images displayed on the video screen represent the actual slide. Furthermore, the viewing pathologist can direct the remote operator to areas of specific interest within the sample. Prerequisites for the successful implementation of such a method into daily teaching and dermatopathology consultation practice are high diagnostic accuracy and precision, cost and time effectiveness, and practicability.

The results of this preliminary study prove that real-time (live) teledermatopathology offers an affordable and simple technology that lends itself to training as well as to diagnosis of difficult lesions by experts situated at remote sites.

During the first study phase, the feasibility of teledermatopathology as a diagnostic tool was assessed. Using teaching cases with common, clear-cut diagnoses we could demonstrate that the diagnostic accuracy was comparable between the two methods. This is the first and major requirement when searching for a method that could expand or, in some instances, replace direct slide assessment by light microscopy. Furthermore, we found that although the confidence in the final diagnosis revealed some interobserver differences for the method of teleconsultation, the overall confidence levels did not differ for the two methods.

Another critical factor when it comes to practicability of a method is how time consuming it is likely to be. In this study we observed that in both settings, namely, evaluation of “teaching cases” and assessment of “routine cases” study participants needed significantly longer to reach a diagnosis when using teledermatopathology. Still, on average it took less than two minutes to make a diagnosis “on screen.” The impressively short time for the “under the microscope” diagnosis that ranged between 25 and 35 seconds underlines the study participants’ high level of expertise. However, due to the relatively short interval between the teledermatopathology consultation and the direct microscopic evaluation, we cannot rule out that participants remembered some cases during the second evaluation and that this bias contributed to the unusually short time needed for “under the microscope” diagnosis.

The fourth parameter that critically influences the practicability of a method of teledermatopathology is its cost effectiveness. We started out with the aim of finding a setup that includes a technically superior method that was at the same time affordable even for small-sized dermatopathology practices and hospital departments. The method of live teledermatopathology that we chose actually fulfills these requirements. The total cost was estimated at approximately €20,000, which included a camera-equipped light microscope, a personal computer (PC) and the necessary image software. The free Internet communication software that was used during the study allowed top quality sound and image communication during teleconsultation. Of course, a fast Internet connection is a prerequisite for this setup. The main advantage of this method over the more sophisticated method of “virtual microscopy” that uses digital image files that have to be generated and then uploaded on the computer or on a server [8] are the comparative low costs and time requirements associated with real-time teledermatopathology. A limitation is the dependency on the operator at the presenting site. On the other hand, this presumed limitation is advantageous if the presenting physician wants to consult with the remote expert. “Virtual microscopy” allows assessing the uploaded specimen directly without being dependent on direct interaction with the remote site presenting the slide. Additionally, this method allows storage of image files for future evaluation. However, in regard to its practical use for remote expert consultation, one has to bear in mind that this method is cost intensive especially for the site that seeks expert consultation. This might be one of the limiting factors for its integration into daily practice, especially in rural areas.

The validity of real-time telepathology has been tested in a recent study in China [11]. In that study four pathologists evaluated 600 specimens from 16 organ systems first by telepathology and subsequently by light microscopy. Comparable to our results, the investigators found a high level of diagnostic agreement between both methods. In line with our results, slide review by teleconsultation took three to four times longer compared to direct assessment under the microscope.

In the case of virtual microscopy several studies have shown its value in regard to feasibility and diagnostic accuracy [12,13]. Our study is the first to evaluate the use of real-time teledermatopathology for both teaching purposes and remote expert diagnosis.

From our data we conclude that live teledermatopathology is a suitable method for histopathologic diagnosis of skin diseases. The differences seen in the time needed to make a diagnosis “on screen” was likely due to the variable degrees of experience in this method among participants. Future studies will need to evaluate the effect of adequate training in live teledermatopathology and to assess its true potential in everyday dermatopathology practice.