Whole slide imaging compared with light microscopy for primary diagnosis in surgical neuropathology: a validation study.

BACKGROUND: Digital pathology practice is rapidly gaining popularity among practicing anatomic pathologists. Acceptance is higher among the newer generation of pathologists who are willing to adapt to this new diagnostic method due to the advantages offered by whole slide imaging (WSI) compared to traditional light microscopy (TLM). We performed this validation study because we plan to implement the WSI system for diagnostic services.
OBJECTIVES: Determine the feasibility of using digital pathology for diagnostic services by assessing the equivalency of WSI and TLM.
DESIGN: A laboratory-based cross-sectional study.
SETTING: Central laboratory at a tertiary health care center.
MATERIALS AND METHODS: Four practicing surgical pathologists participated in this study. Each pathologist blindly reviewed 60 surgical neuropathology cases with a minimum 8-week washout-period between the two diagnostic modalities (WSI vs. TLM). Intraobserver concordance rates between WSI and TLM diagnoses as compared to the original diagnosis were calculated. MAIN OUTCOME MEASURES: Overall intraobserver concordance rates between each diagnostic method (WSI and TLM) and original diagnosis. SAMPLE SIZE: 60 in-house surgical neuropathology cases.
RESULTS: The overall intraobserver concordance rate between TLM and original diagnosis was 86.3% (range 76.7%-91.7%) versus 80.8% for WSI (range 68.3%-88.3%). These findings are suggestive of the superiority of TLM, but the Fleiss' Kappa statistic indicated that the two methods are equivalent, despite the low level of the K value.
CONCLUSION: WSI is not inferior to the light microscopy and is feasible for primary diagnosis in surgical neuropathology. However, to ensure the best results, only formally trained neuropathologists should handle the digital neuropathology service. LIMITATIONS: Only one diagnostic slide per case rather than the whole set of slides, sample size was relatively small, and there was an insufficient number of participating neuropathologists. CONFLICT OF INTEREST: None.

INTRODUCTION

Digitizing glass slides has revolutionized the practice of anatomical pathology, improving the diagnostic accuracy, turn around time, and outreach service coverage. Using slide scanners, glass slides are converted into digital slides rapidly accessible to the pathologist. This technology potentially enables neuropathologists to remotely provide an opinion during crucial intraoperative consultations.[1] Digital pathology has the potential to deliver efficient and accurate clinical diagnosis to accommodate the rising needs for neuropathological services in under-privileged and remote areas. As a new practice, digital pathology may be an ideal solution to provide neuropathology consultation to laboratories without such expertise due to limited resources. Digitally scanned whole slide images may enable the neuropathologist to provide high-quality and fast primary or secondary diagnoses and improve patient care. Saudi Arabia has only ten formally trained and certified neuropathologists. Digital pathology, despite its challenges, may overcome this shortage by supporting the need for the deployment of a nationwide digital neuropathology services.

Neuropathology frozen sections are stressful to general pathologists. Therefore, a digital pathology solution may be helpful. Fallon et al have shown a high correlation between frozen section slides and WSI with 52 ovarian frozen section diagnoses.[2] This accuracy rate maybe similar to neuropathology frozen sections. Several studies concluded that WSI as a diagnostic modality is not inferior to TLM.[2,5-21] Validation of the digital pathology system is required to ensure laboratory compliance with accrediting organizations such as the Joint Commission and College of American Pathologists (CAP) and to assure the accuracy of diagnosis. Validation studies have shown a high concordance rate between WSI and TLM in different sub-specialties of anatomical pathology.[6,8-10,12,13,15,18] Digital pathology, for intraoperative consultations, primary/secondary diagnosis, multi-disciplinary meetings, and educational purposes, is already being used in some parts of the world (e.g. United States of America, Canada and parts of Europe).[22-25] Furthermore, the recent approval of WSI for primary diagnosis by the US Food and Drug Administration (FDA) should encourage the adoption of digital pathology for clinical purposes globally. The FDA considers WSI systems to consist of two subsystems: (1) the image acquisition component (ie, the scanner) and (2) the workstation (ie, the image viewing software, computer, and display). The FDA also considers this entire digital pathology system to be a closed unit precluding the substitution of unapproved components.[26]

The current CAP guidelines recommend that all pathology departments adopting WSI for diagnosis should conduct their own validation. This is done by evaluation of at least 60 specimens to assess intraobserver variations in diagnoses between digital slides and glass slides, with a washout period of at least 2 weeks.[27] In this study, we investigate the feasibility of WSI for diagnostic surgical neuropathology services at our center in Saudi Arabia.

MATERIALS AND METHODS

This study was conducted in the Department of Pathology and Laboratory Medicine at King Abdulaziz Medical City in Riyadh, Saudi Arabia. The goal was primarily to validate the technology using the guidelines published by the College of American Pathologists and Laboratory Quality Center.[1] The study protocol was approved by the Institutional Review Board at King Abdullah International Medical Research Center (KAIMRC).

Sixty recent surgical neuropathology cases were retrieved from the archives of our anatomic pathology division. The accuracy of the original diagnosis is ensured as the malignant cases were co-signed by two board certified neuropathologists per departmental policy while the benign lesions were considered straightforward and also diagnosed by a neuropathologist. Instead of scanning all slides for each case, the first author reviewed all slides and selected one diagnostic high quality slide from each case to simplify the methodology in order to guarantee participation from the limited number of study candidates. Additionally the one diagnostic slide approach has been used in several studies.[2-4] The sixty selected cases covered a broad range of surgical neuropathological entities encountered at our institution. The original surgical pathology numbers were replaced by a study identifier (case number). The 60 diagnostic hematoxylin and eosin-stained slides were scanned using the Aperio scanner (ScanScope AT Turbo) at 40×.

Four practicing surgical pathologists were carefully selected to participate in this study. One is board-certified in neuropathology while the others either received considerable exposure to neuropathology services during their residency training (6 months or more) or had sufficient expertise being senior pathologists (ranging between 25-40 years of general practice including neuropathology cases).

Two pathologists at a time were given either a set of digital or glass slides independently as well as the clinical information available at the time of the original diagnosis. After a washout period of at least 8 weeks, the sets of slides were switched between the two pathologists. The same process was followed with the two remaining pathologists. For each pathologist, the glass slides were reviewed using their own light microscope and the digital slides were reviewed using a high-resolution large monitor.

The diagnostic accuracy was based on whether or not a study diagnosis was clinically and significantly different from the original diagnosis. Diagnostic discrepancies were classified as major when there was a definite influence on clinical management/outcome and minor when there was little or no clinical significance. In keeping with the widely accepted definitions, a major discordance was defined as a difference in diagnosis that would be associated with a change in patient management. On the other hand, a minor discordance was defined as a difference in diagnosis that was not associated with a change in patient management. Statistical analysis was performed using SAS version 9.4 through the Department of Biostatistics and Bioinformatics at KAIMRC.

RESULTS

The overall intraobserver concordance rate between the original and TLM diagnoses was 86.3% (range 76.7%–91.7%) () while the rate was slightly lower (80.8%, range 68.3%–88.3%) using the WSI diagnostic method (). This showed a 6% difference in favor of the TLM method. Using the statistical module (GEE) to predict which of the two methods was superior, the resultant odds ratio of 1.5 indicated a 50% greater chance in favor of the TLM method. The overall intra-observer concordance rate between TLM and WSI was 82.1% ().

Table 1.

Intraobserver concordance rates between TLM and original diagnoses by pathologist.

	Pathologist 1	Pathologist 2	Pathologist 3	Pathologist 4	Total (%)
Concordant	53 (88.3)	46 (76.7)	55 (91.7)	53 (88.3)	86.3
Minor discordance	3 (5)	4 (6.7)	4 (6.7)	5 (8.3)	6.7
Major discordance	4 (6.7)	10 (16.7)	1 (1.7)	2 (3.3)	7.1

Data are number (%).

Table 2.

Intraobserver concordance rates between WSI and original diagnoses by pathologist.

	Pathologist 1	Pathologist 2	Pathologist 3	Pathologist 4	Total (%)
Concordant	49 (81.7)	41 (68.3)	51 (85)	53 (88.3)	80.8
Minor discordance	2 (3.3)	8 (13.3)	7 (11.7)	7 (11.7)	10
Major discordance	9 (15)	11 (18.3)	2 (3.3)	0	9.2

Data are number (%).

Table 3.

Intraobserver concordance rates between TLM and WSI diagnoses by pathologist.

	Pathologist 1	Pathologist 2	Pathologist 3	Pathologist 4	Total (%)
Concordant	49 (81.7)	43 (71.7)	52 (86.7)	53 (88.3)	82.1
Minor discordance	2 (3.3)	7 (11.7)	5 (8.3)	5 (8.3)	7.9
Major discordance	9 (15)	10 (16.7)	3 (5)	2 (3.3)	10

Data are number (%).

There appeared to be a minor superiority for TLM over WSI so we generated the Fleiss kappa statistic for multi-rater agreement using bootstrapping with five subject replacement for each method 1000 times. In each of the samples, a Fleiss kappa statistic was generated for each method and an average Kappa (K) was calculated. The result showed an almost equal level of intraobserver agreement between the TLM with a K value (mean and standard deviation) of 0.14 (0.07) and the WSI with a K value of 0.20 (0.06). There was a nonsignificant difference in the standard error (P=.007) between the two Kappa means which indicates equivalency between the two methods even though the K value was low. The original diagnosis for each of the 60 cases is shown in

Table 4.

Original diagnoses for the 60 cases.

Case #	Original diagnosis
1	Vasculitis with fibrinoid necrosis
2	Meningioma (psammomatous)
3	Metastatic renal cell carcinoma
4	Medulloblastoma (desmoplastic/nodular)
5	Lymphoma (PTLD c/w DLBCL)
6	Meningioma (brain invasive)
7	Ependymoma
8	Pituitary adenoma
9	Colloid cyst
10	Anaplastic ependymoma
11	Choroid plexus papilloma
12	Chordoma
13	Medulloblastoma (large cell/anaplastic)
14	Craniopharyngioma
15	Meningioma (microcystic)
16	Medulloblastoma (classic)
17	Metastatic colonic adenocarcinoma
18	Hemangioblastoma
19	Myxopapillary ependymoma
20	Diffuse gliomas (oligodendroglioma)
21	Schwannoma
22	Caseating granuloma (tuberculoma)
23	Cavernoma
24	Pilocytic astrocytoma
25	Diffuse gliomas (astrocytoma)
26	Glioblastoma
27	Meningioma
28	Brain abscess
29	Germinoma
30	Anaplastic meningioma
31	Lymphoma (DLBCL)
32	Pilocytic astrocytoma
33	Craniopharyngioma
34	Pineoblastoma
35	Langerhans cell histiocytosis
36	Hemangiopericytoma
37	Dysembryoplastic neuroepithelial tumor
38	Pituitary adenoma
39	Glioblastoma (giant cell)
40	Central neurocytoma
41	Anaplastic ependymoma
42	Spinal cord schistosomiasis
43	Diffuse glioma (oligodendroglioma)
44	Central neurocytoma (atypical)
45	Meningioma (microcystic)
46	Meningioma (chordoid)
47	Diffuse glioma (oligodendroglioma)
48	Metastatic melanoma
49	Metastatic choriocarcinoma
50	Meningioma (chordoid)
51	Glioblastoma (gliosarcoma)
52	Meningioma (secretory)
53	Medulloblastoma (large cell/anaplastic)
54	Embryonal tumor with multilayered rosettes (ependymoblastoma)
55	Craniopharyngioma
56	Occipital encephalocele
57	Glioblastoma
58	Hemangioblastoma
59	Meningocele
60	Diffuse glioma (oligodendroglioma)

PTLD=Posttransplant lymphoproliferative disorder; DLBCL=Diffuse large B cell lymphoma.

DISCUSSION

Following the current CAP with comprehensive statistics, we confirmed the validity of using digital pathology for the practice of surgical neuropathology. This finding supports the results of a relatively recent systematic review and the multicenter blinded randomized study by Mukhopadhyay et al.[20] Additionally, intraobserver variability in concordance rates ranged from 68.3% to 88.3% for WSI and 76.7% to 91.7% for TLM with the highest rates for WSI achieved by pathologist 4 (neuropathologist) who also scored high for TLM. In our study, the total major discordance rate for WSI was 2.1% higher than TLM which is higher than the 0.4% reported by Mukhopadhyay et al.[20] Interestingly, there was zero percent major discordance rates for WSI by pathologist 4, a board-certified neuropathologist. None of the participating pathologists received prior training to use the WSI for primary diagnosis.

The literature related to digital pathology in the context of surgical neuropathology is scarce. Pekmezci et al identified two potential pitfalls (identification of mitoses and recognition of nuclear details). These pitfalls could limit the utility of WSI for accurate diagnosis and grading of tumors in surgical neuropathology.[2] However, in our study, those pitfalls reported by Pekmezci et al were not substantiated. After careful review of digitized slides we found that all discordant cases were attributed to misinterpretation as the image quality was good enough to appreciate the cytological details and to recognize mitotic figures ().

Figure 1.

(whole slide images of four discrepant cases): A. Atypical central neurocytoma showing one readily recognized mitotic figure (green arrow) B. Diffuse astrocytoma misdiagnosed as gliosis. C. Microcystic/angiomatous meningioma misdiagnosed as clear cell meningioma. D. Dysembryoplastic neuropithelial tumor (DNET) with several floating neurons misdiagnosed as oligodendroglioma.

Our study evaluated the accuracy of both diagnostic modalities, but not the efficiency in terms of interpretative time for digital images versus glass slides. Mills et al investigated the efficiency of digital pathology compared to glass slide interpretation. Reading pathologists recorded assessment times for each modality. Initially, the digital read was slower than the microscopy but at the end of their study, the digital read was comparable to the optical times.[28] The strengths of our study include the blinded design, the inclusion of an extended wash-out period and the participation of four pathologists with variable expertise, one of whom is a neuropathologist. In terms of limitations, we acknowledge the small sample size, the provision of only one diagnostic slides instead of the whole set, and the lack of a sufficient number of participating neuropathologists. The lack of access to the patient's records or ancillary testing results is compensated by the provision of the diagnostic H&E slides with the accompanying brief clinical histories.

WSI as a diagnostic modality is not inferior to TLM and gradual transitioning into digital pathology is possible with close monitoring and sufficient training. The pre-analytical phase should be well controlled with quality H&E slides. However, to ensure the best results, only formally trained neuropathologists should handle the digital neuropathology service.