Multicenter study on the diagnostic performance of multiframe volumetric laser endomicroscopy targets for Barrett's esophagus neoplasia with histopathology correlation.

Volumetric laser endomicroscopy (VLE) has been shown to improve detection of early neoplasia in Barrett's esophagus (BE). However, diagnostic performance using histopathology-correlated VLE regions of interest (ROIs) has not been adequately studied. We evaluated the diagnostic accuracy of VLE assessors for identification of early BE neoplasia in histopathology-correlated VLE ROIs. In total, 191 ROIs (120 nondysplastic and 71 neoplastic) from 50 BE patients were evaluated in a random order using a web-based module. All ROIs contained histopathology correlations enabled by VLE laser marking. Assessors were blinded to endoscopic BE images and histology. ROIs were first scored as nondysplastic or neoplastic. Level of confidence was assigned to the predicted diagnosis. Outcome measures were: (i) diagnostic performance of VLE assessors for identification of BE neoplasia in all VLE ROIs, defined as accuracy, sensitivity, and specificity; (ii) diagnostic performance of VLE assessors for only high level of confidence predictions; and (iii) interobserver agreement. Accuracy, sensitivity, and specificity for BE neoplasia identification were 79% (confidence interval [CI], 75-83), 75% (CI, 71-79), and 81% (CI, 76-86), respectively. When neoplasia was identified with a high level of confidence, accuracy, sensitivity, and specificity were 88%, 83%, and 90%, respectively. The overall strength of interobserver agreement was fair (k = 0.29). VLE assessors can identify BE neoplasia with reasonable diagnostic accuracy in histopathology-correlated VLE ROIs, and accuracy is enhanced when BE neoplasia is identified with high level of confidence. Future work should focus on renewed VLE image reviewing criteria and real-time automatic assessment of VLE scans.

INTRODUCTION

Patients with a Barrett’s esophagus (BE) are at increased risk for developing esophageal adenocarcinoma (EAC) and therefore undergo regular endoscopic surveillance. Early neoplastic lesions in BE, however, are difficult to detect endoscopically, as their appearance is often subtle., The current BE surveillance protocol is therefore suboptimal, as early neoplasia can be missed and random biopsies are associated with sampling error, which can adversely impact patient outcomes and health care costs.,

Volumetric laser endomicroscopy (VLE) is a novel balloon-based imaging technique that may improve the detection of early neoplasia in BE and/or reduce the need for random biopsies. High-resolution cross-sectional images are created based on differences in optical scattering of tissue structures. In 90 seconds, a 6-cm circumferential scan is made visualizing the esophageal mucosa and subsurface layers with near-microscopic resolution. VLE areas with an abnormal appearance can be marked by a VLE-guided laser marking tool allowing for optimal biopsy correlation.

In the past, VLE has shown promising accuracy for differentiating early BE neoplasia (high-grade dysplasia [HGD] and EAC) from nondysplastic Barrett’s esophagus (NDBE). However, in these studies only single VLE images (still images) of regions of interest (ROIs) were assessed, VLE images were carefully preselected, and no direct histopathology correlation was available because of a lack of VLE laser marking at that time. To accurately evaluate the diagnostic performance of VLE for BE neoplasia detection, multiframe segments (vs. single still images) of histopathology-correlated ROIs should be assessed. The aim of this study was, therefore, to assess the diagnostic performance of VLE users for identification of early BE neoplasia in histopathology-correlated VLE ROIs.

METHODS

Setting and design

Histopathology-correlated VLE ROIs were prospectively obtained at the Amsterdam University Medical Centers, location Academic Medical Center Amsterdam, Catharina Hospital Eindhoven, and St. Antonius Hospital Nieuwegein—all tertiary referral centers for Barrett’s neoplasia in the Netherlands (Fig. 1). Official approval was obtained by the institutional review board, and patients were included under protocol NTR 6728, registered at http://www.trialregister.nl. Informed consent was obtained from all patients.

Fig. 1

(A) Two volumetric laser endomicroscopy (VLE) regions of interest with nondysplastic Barrett’s esophagus. The left image clearly displays a normal esophageal layering of the mucosa. The right image contains the VLE feature lack of layering; however, there is no visible increased signal surface intensity or the presence of multiple irregular glands. (B) Two VLE regions of interest with Barrett’s neoplasia. Both images contain the abnormal VLE features multiple irregular glands and lack of layering. In the VLE images, the laser marks are visible as small areas of high surface signal intensity indicated by the yellow delineations.

Patients

All patients received surveillance of known BE or were referred for endoscopic treatment of early BE neoplasia. Inclusion criteria included patients with a histopathological diagnosis of nondysplastic BE or early BE neoplasia (i.e. HGD and/or EAC). Patients with endoscopic suspicion of advanced neoplastic lesions (Paris type 0–I or 0–III lesions), significant stenosis of the esophagus, reflux esophagitis (higher than grade B), and esophageal tears, ulcers, or varices were excluded from the study.

Endoscopic and VLE procedure

All endoscopic procedures were performed by three endoscopists (JB, WC, and BW) with extensive experience in the use of advanced imaging techniques and endoscopic treatment of early BE neoplasia. After thorough endoscopic inspection of the Barrett’s segment, length of the BE segment was recorded according to the Prague C&M classification, and in case of a lesion, overview and detailed images of the lesion were obtained.

Subsequently, the VLE procedure (Nvision VLE Imaging System, NinePoint Medical) was performed, creating a 90-second full scan of a 6-cm BE segment. The BE segment was systematically laser marked every 2 cm at 3, 6, 9, and 12 o’clock, similar to the Seattle protocol for random biopsies. These superficial marks (i.e. laser marks) were targeted at 3, 6, 9, and 12 o’clock and were not specifically targeted at suspicious VLE areas, to ensure objective collection of VLE ROIs. The location of every laser-marked ROI was systematically recorded on the reports at the time of the procedure by consensus of two VLE research fellows (MS and JG) and one of the expert endoscopists. Following completion of the VLE procedure, targeted biopsies were obtained in the middle of all areas that were laser marked, and careful attention was attributed to collect every biopsy specimen into a different pathology jar. See Figure 2 for a schematic overview of the laser marking process. All histology slides were evaluated by an expert BE pathologist (SM) and summarized in Table 1.

Fig. 2

These images show the volumetric laser endomicroscopy (VLE) laser marking process, which provided the histopathology correlation. During the endoscopy, laser marks may be appreciated as white superficial cautery marks, and these were targeted in the Barrett’s esophagus. In between the laser mark set, a biopsy was obtained to provide VLE–histology correlation for the region of interest. In the VLE image, the laser marks (yellow box) are visible as small areas of high surface signal intensity.

Table 1

Histology findings in the study population

Histology findings	Number of biopsies/VLE ROIs
Gastric cardia	30
Nondysplastic Barrett esophagus	235
Indefinite for dysplasia	11
Low-grade dysplasia	9
High-grade dysplasia	35
Adenocarcinoma	36

ROI, regions of interest; VLE, volumetric laser endomicroscopy.

Selection of VLE ROI

All histologically confirmed nondysplastic and dysplastic ROIs were extracted from the VLE full scan. VLE ROIs were excluded based on the following criteria: the corresponding biopsy-contained gastric mucosa, low-grade dysplasia (LGD), or indefinite for dysplasia (IND), the ROI did not include VLE laser marks to ensure careful histopathology correlation, and presence of inadequate ROI quality defined as >25% of decentering and/or low signal-to-noise ratio.

To optimize VLE interpretation in the web-based module, ROIs were extracted from full scans and subsequently transformed to high-quality cross-sectional view videos. Each video consisted of 25 sequential frames above and 25 frames below the area of the laser mark, corresponding to 2.5 × 2.5 mm, reflecting a biopsy specimen.

VLE assessors

Ten recognized VLE assessors, all experienced in VLE interpretation, participated in this study. All VLE assessors had participated in multiple VLE interobserver studies, prospective VLE trials, and had >3 years of experience with the technique. Assessors were blinded to histology and clinical endoscopic information, and the distribution of nondysplastic and neoplastic videos was not specified. An instructional VLE training session was provided before starting the assessment phase. The assessment phase lasted 8 weeks, in which a minimum of 5 and a maximum of 35 VLE ROI videos were allowed to be rated each session. Assessors were not provided with feedback on their diagnostic performance after completing each case. To increase the total amount of videos that were scored and to limit the assessment time, every VLE user randomly assessed 100/191 videos, creating 1000 assessments (100 videos × 10 assessors). Using the web-based module, we guaranteed every video was evaluated by at least four assessors. To balance out the nondysplastic versus neoplastic cases, and to allow for a feasible amount of VLE ROIs to be evaluated by 10 assessors, automatic random exclusion of NDBE ROIs was performed.

Online video assessment

The web-based module allowed for online evaluation of the VLE ROI video (Fig. 3). During the assessment phase, assessors were first asked to rate the video as ‘NDBE’ or ‘neoplastic’. Corresponding level of confidence was scored as ‘high’, ‘moderate’, or ‘low’. Figure 3 provides a screenshot of the assessment phase. Assessment time was defined as the time in seconds to complete one video assessment and answer the corresponding questions. Assessors used previously defined features associated with BE neoplasia: higher surface signal intensity compared with subsurface intensity, absence of mucosal layering, and presence of irregular epithelial glands. The laser marks were carefully outlined in the video module and the area in between the laser marks was evaluated by the assessors. As the ROIs might contain different types of epithelium (squamous, BE, and gastric cardia), assessors were asked to evaluate only BE.

Fig. 3

(A) Web-based module showing a nondysplastic Barrett’s volumetric laser endomicroscopy (VLE) region of interest. Corresponding questions, including levels of confidence, were scored for each region of interest. In the VLE image, the laser marks (yellow box) are visible as small areas of high surface signal intensity. (B) Web-based module showing a neoplastic Barrett’s VLE region of interest. Corresponding questions, including levels of confidence, were scored for each region of interest. In the VLE image, the laser marks (yellow box) are visible as small areas of high surface signal intensity.

Primary and secondary outcome measurements

The primary outcome measure was the diagnostic performance of VLE assessors, defined as accuracy, sensitivity, and specificity for the correct differentiation between ‘nondysplastic BE’ and ‘neoplastic BE’ (HGD/EAC). Secondary outcome measures were: association between level of confidence or VLE experience and diagnostic performance, interobserver agreement, and assessment time.

Statistical analysis

Data are presented as the mean (standard deviation [SD]) or as the median (range) for continuous variables depending on the normality of data distribution. Calculations were done with percentages for categorical variables. Accuracy, sensitivity, and specificity of correctly identifying NDBE or BE neoplasia was calculated per VLE ROI. Interobserver agreement was assessed using multirater kappa statistics. Mixed-effect logistic regression was performed to evaluate the effect of endoscopists’ VLE expertise and level of confidence on the correct ROI diagnosis. Random effect was set for each endoscopist to capture the correlation among the ROIs that have been assessed by the same endoscopists. As this was the first study to evaluate the ability of VLE assessors for identification of BE neoplasia using VLE ROIs, no formal sample size calculation was conducted. P ≤ 0.05 was considered statistically significant. All statistical analyses were performed using SPSS version 25.0 (Armonk, NY).

RESULTS

Patients and VLE scans

ROIs were derived from 50 patients (41 men and 9 women), with a mean age of 69 years (SD ± 11 years). The median circumferential extent of BE was 2 cm (interquartile range [IQR] 1–5), and the median maximum extent was 5 cm (IQR 4–7). Three patients were excluded due to technical failures during the VLE procedure, including balloon leakage after the balloon’s black registration line was hit by the VLE laser marking system, and no evident visual endoscopic appearance of both laser marks enabling adequate VLE to histology correlation. There were no complications related to the endoscopic procedure and/or targeted biopsies. In total, 365 ROIs were obtained for eligibility of which 59 were excluded because of inadequate quality (n = 9), gastric mucosa (n = 30), LGD (n = 9), and IND (n = 11), as shown in Figure 4. After automatic random exclusion of 115 NDBE VLE ROIs, 191 ROIs (120 NDBE and 71 HGD/EAC) were assessed by 10 VLE assessors. This equaled 9741 evaluated VLE frames (191 targets × 51 frames).

Fig. 4

Flow diagram outlining the patient inclusion scheme. EAC, esophageal adenocarcinoma; HGD, high-grade dysplasia; IND, Indefinite for dysplasia; LGD, low-grade dysplasia; NDBE, nondysplastic Barrett esophagus; ROC, region of interest; VLE, volumetric laser endomicroscopy.

Primary outcome measurements

Diagnostic performance of VLE experts

Accuracy, sensitivity, and specificity for BE neoplasia identification was 79% (confidence interval [CI], 75–83), 75% (CI, 71–79), and 81% (CI, 76–86), respectively. Table 2 provides a summary of the diagnostic performance of individual VLE assessors, and Figure 5 outlines the clustered diagnostic performance of the assessors.

Table 2

Diagnostic performance per assessor for the identification of Barrett’s neoplasia using volumetric laser endomicroscopy

Assessor	Accuracy (%)	Sensitivity (%)	Specificity (%)
1	66	70	64
2	83	82	84
3	77	81	76
4	83	76	85
5	76	70	81
6	85	82	86
7	74	69	77
8	82	77	84
9	78	69	84
10	83	72	88
Total	79 (75–83)	75 (71–79)	81 (76–86)

Bold values are provided as the mean and/or median (confidence interval).

Fig. 5

Diagnostic performance per assessor for the identification of Barrett’s neoplasia using volumetric laser endomicroscopy (VLE) regions of interest.

Secondary outcome measurements

Correlation between level of confidence and diagnostic performance

In total, 10% of the ROIs were scored with a low level of confidence, 28% with moderate level of confidence, and 63% with a high level of confidence. When neoplasia was identified with a high level of confidence, accuracy, sensitivity, and specificity were 88%, 83%, and 90%, respectively. When neoplasia was identified with a low level of confidence, accuracy, sensitivity, and specificity were 43%, 48%, and 40%, respectively. High level of confidence was associated with a higher rate of correct diagnosis compared to moderate and low level of confidence (P < 0.001), as shown in Figure 6A.

Fig. 6

(A) Association between level of confidence and diagnostic accuracy. The graph shows a significant increase in diagnostic accuracy when volumetric laser endomicroscopy (VLE) regions of interest are scored by assessors with a high level of confidence, P < 0.001. (B) Diagnostic accuracy for the identification of Barrett’s neoplasia evaluated when the VLE procedures previously performed by assessors were grouped in three categories. There was no significant association between more extensive VLE experience and increased diagnostic accuracy.

Interobserver agreement

The interobserver agreement between assessors for the diagnosis of BE neoplasia, defined by the median kappa (IQR), was 0.29 (0.18–0.37).

Correlation between VLE experience and diagnostic accuracy

We grouped the number of VLE procedures previously performed by assessors into the following categories: 50–99, 100–300, and >300. There was no significant association between more extensive VLE experience and diagnostic accuracy when differentiating between nondysplastic and neoplastic ROIs, as shown in Figure 6B.

Assessment time

The median time for assessors to analyze one video was 42 seconds (range 8–299). We grouped the assessment time into the following categories: <20 seconds, 20–39 seconds, 40–59, and ≥60 seconds. In total, 114/1000 assessments were scored in <20 seconds with a corresponding accuracy of 90%, sensitivity of 92%, and specificity of 86%. Assessment time >60 seconds (294/1000 assessments) was associated with suboptimal diagnostic accuracy of 71%, sensitivity of 64%, and specificity of 75%. A short assessment time of <20 seconds was significantly associated with a higher rate of correct nondysplastic and neoplastic diagnosis compared with longer assessment time, P = 0.004.

Of the high level of confidence predictions, 81% was scored in <20 seconds. A significant Spearman’s rank correlation was found between assessment time and level of confidence, P < 0.001.

DISCUSSION

Previous studies suggest that VLE assessment by experts can predict the presence of neoplasia with high accuracy., However, these studies have utilized VLE still images without optimal histological correlation. The recent introduction of VLE-guided laser marking allows for optimal correlation between VLE ROIs and histopathology. We evaluated the diagnostic performance of 10 recognized VLE assessors to differentiate between nondysplastic and neoplastic histopathology-correlated ROIs using short multiframe VLE video sequences instead of still images.

We found a promising performance of VLE assessors for identification of early BE neoplasia: When neoplasia was identified with a high level of confidence, accuracy, sensitivity, and specificity were 88%, 83%, and 90%, respectively. In our study, assessors rated the videos as NDBE or neoplastic based on their experience with the previously established VLE scoring criteria. Assessors were not obligated to use one particular scoring system, and possibly VLE users have developed their own intuitive scoring system over hundreds of cases with review of pathology as feedback. Moreover, interpretation of these VLE features is highly subjective. These rationales may explain the relatively low interobserver agreement found in our study. Moreover, the highest overall sensitivity in our study, which was reached by two assessors, was 82%. Therefore, our study suggests that these VLE features are not sensitive enough to detect all neoplastic cases, and future refinement is thus needed for clinical practice. Additionally, a post hoc evaluation of the neoplastic ROIs with an incorrect diagnosis showed that most cases contained absence of VLE surface signal intensity or only a subtle increase and partial lack of layering. The subtle differences in VLE features visible in these gray-scale VLE images, however, might be too complex for the human brain to interpret. Recently, the VLE system has been upgraded by an artificial intelligent tool for more uniform and enhanced interpretation of VLE features showing promising results., Future studies should focus on the development and standardization of refined VLE scoring criteria and should determine the incremental yield of these artificial intelligent systems for VLE interpretation.

We observed a significantly higher rate of correct diagnosis when assessors evaluated an ROI in <20 seconds. A logical explanation may be that experienced VLE users can readily and swiftly identify neoplasia at first sight. However, this does not mean that physicians should rush their assessment without carefully scrutinizing the VLE imagery. This theory is supported by an overall high rate of ‘moderate’ to ‘high’ level of confidence assessments (91%) and corresponding increased diagnostic performance. Additionally, of the high level of confidence predictions, 81% were scored in <20 seconds. In a post hoc evaluation of the VLE ROIs, a combination of the VLE features of multiple atypical glands in combination with lack of layering or high surface signal intensity were primarily causing these high confidence predictions. In particular, multiple atypical glands are more readily visible in contrast to subjective changes in surface signal intensity. In contrast, low level of confidence and assessment time >60 seconds were associated with suboptimal diagnostic accuracy. These ROIs showed no increase in VLE surface signal intensity or only a slight increase in surface signal and partial lack of layering. Therefore, while the interpretation of VLE ROIs by assessors is often straightforward, in a substantial number of cases, interpretation is more difficult and results in careful scrutinizing of features, leading to low level of confidence and long assessment time.

Our study has several strengths, including the rigorous study design with VLE laser marking allowing for the assessment of histopathology-correlated VLE ROIs and the prospective multicenter data acquisition protocol ensuring generalizability of VLE data. A high number of experienced VLE assessors enhance the external validity of these results. The assessment of multiple VLE frames within one ROI in our study provided a more reliable estimate of the true diagnostic performance in clinical practice compared to a preselection of high-quality still images, which does not reflect the interpretation of VLE during endoscopy. All VLE assessors received an instructional VLE training session prior to starting the online assessment. Additionally, in our study we included subtle neoplastic irregularities or even ‘endoscopically invisible neoplasia’, whereas other studies have included lesions that were more apparent endoscopically., Finally, the random laser marking protocol used in our study decreases the chance of selection bias, as VLE laser marking was not driven by the human perception of visually dysplastic or nondysplastic areas.

We acknowledge the limitations of this study. First, VLE was performed in three tertiary referral centers possibly resulting in capturing mainly high-quality multiframe image stacks. Second, ROIs were transformed to cross-sectional videos and interpreted using a web-based module. This does not directly mirror real-time clinical use of the VLE console, as the VLE full scan of the corresponding BE segment cannot be assessed, and no clinical endoscopic data were provided. Fourth, we did not perform a patient-based analysis (i.e. one ROI from one patient), resulting in the possibility of statistical dependencies in ROIs obtained from the same patient. However, ROIs were randomized between assessors, limiting this effect. Finally, we analyzed NDBE and neoplastic VLE images only, reflecting the more obvious pathological cases. The rationale for this approach can be explained, because the histopathological diagnosis of LGD by pathologists shows a considerable interobserver variation.

In summary, we have shown that experienced VLE assessors can identify BE neoplasia with promising diagnostic accuracy in histopathology-correlated VLE ROIs, especially when BE neoplasia was identified with a high level of confidence or short assessment time. Future work should focus on development and standardization of refined VLE scoring criteria and comparison of VLE assessors and computer-aided detection algorithms for the identification of BE neoplasia.

Funding

The collaboration project is financed by the Ministry of Economic Affairs of the Netherlands by means of the PPP Allowance made available by the Top Sector Life Sciences & Health to Amsterdam UMC to stimulate public–private partnerships.

Conflicts of interest

M.R. Struyvenberg: None.

A.J.de Groof: None.

A. Kahn: None.

B.L.A.M. Weusten: Pentax Medical (research support, speaking fees).

D.E. Fleischer: None.

E.K. Ganguly: Boston Scientific (consultant).

V.J.A. Konda: Pentax (research grant).

C. J. Lightdale: None.

D.K. Pleskow: NinePoint Medical (consultant), Boston Scientific/Olympus/Fujifilm/ Medtronic/CSA (consultant).

A. Sethi: Boston Scientific/Olympus/Fujifilm (consultant).

M.S. Smith: NinePoint Medical (consultant).

A.J. Trindade: NinePoint Medical (research support), Olympus/Pentax (consultant).

M.B. Wallace: NinePoint Medical (research support), Fujifilm/Boston Scientific/Olympus/ Medtronic (research grants).

H.C. Wolfsen: None.

G.J. Tearney: NinePoint Medical (consultant and royalties), Boston Scientific (research support), iLumen (research support).

S.L. Meijer: None.

C.L. Leggett: NinePoint Medical (indirect research support).

J.J.G.H.M. Bergman: NinePoint Medical (research support), Fujifilm (speaking fees).

W.L. Curvers: None.