Development and Validation of Confocal Endomicroscopy Diagnostic Criteria for Low-Grade Dysplasia in Barrett's Esophagus.

OBJECTIVES: Low-grade dysplasia (LGD) in Barrett's esophagus (BE) is generally inconspicuous on conventional and magnified endoscopy. Probe-based confocal laser endomicroscopy (pCLE) provides insight into gastro-intestinal mucosa at cellular resolution. We aimed to identify endomicroscopic features and develop pCLE diagnostic criteria for BE-related LGD.
METHODS: This was a retrospective study on pCLE videos generated in 2 prospective studies. In phase I, 2 investigators assessed 30 videos to identify LGD endomicroscopic features, which were then validated in an independent video set (n = 25). Criteria with average accuracy >80% and interobserver agreement κ > 0.4 were taken forward. In phase II, 6 endoscopists evaluated the criteria in an independent video set (n = 57). The area under receiver operating characteristic curve was constructed to find the best cutoff. Sensitivity, specificity, interobserver, and intraobserver agreements were calculated.
RESULTS: In phase I, 6 out of 8 criteria achieved the agreement and accuracy thresholds (i) dark nonround glands, (ii) irregular gland shape, (iii) lack of goblet cells, (iv) sharp cutoff of darkness, (v) variable cell size, and (vi) cellular stratification. The best cutoff for LGD diagnosis was 3 out of 6 positive criteria. In phase II, the diagnostic criteria had a sensitivity and specificity for LGD of 81.9% and 74.6%, respectively, with an area under receiver operating characteristic of 0.888. The interobserver agreement was substantial (κ = 0.654), and the mean intraobserver agreement was moderate (κ = 0.590).
CONCLUSIONS: We have generated and validated pCLE criteria for LGD in BE. Using these criteria, pCLE diagnosis of LGD is reproducible and has a substantial interobserver agreement.

INTRODUCTION

Barrett's esophagus (BE) is a premalignant condition to esophageal adenocarcinoma, with an estimated annual cancer risk of approximately 0.3% (1). Patients with BE are generally recommended endoscopic surveillance for early cancer detection (2,3). The cancer risk is significantly higher in dysplastic BE, with low-grade (LGD) and high-grade dysplasia (HGD) carrying a risk of progression of 0.5%–13% per year (4,5) and up to 2%–10% per year, respectively (6). A timely diagnosis of dysplasia is therefore essential for cancer prevention. Gastroenterology Societies recommend endoscopic ablation for both HGD and LGD (3,7), as there is randomized controlled evidence that this significantly reduces the rate of cancer progression (8).

The diagnosis of BE-related dysplasia, particularly LGD, is difficult as this is generally inconspicuous on the conventional and magnified endoscopic imaging. For this reason, it is recommended that multiple random biopsies be taken according to the Seattle protocol to allow sufficient tissue sampling for the histopathological diagnosis (2,3). The caveats of the random biopsy protocol are long procedure time, poor tolerance by the patient, limited adherence by the endoscopists, and high cost for the health care system (9). Thus, several endoscopic techniques have been investigated to improve targeting biopsies (10). However, to date none of the enhanced imaging techniques has replaced random biopsies in routine practice (2). In particular, there is no evidence that LGD can be diagnosed by enhanced magnified and nonmagnified endoscopic modalities.

Probe-based confocal laser endomicroscopy (pCLE) has the advantage to provide microscopic views at cellular resolution of the gastro-intestinal mucosa to allow real-time histological diagnosis (10). Although pCLE provides point imaging and is subject to the same limitation of sampling error as biopsy forceps, it provides the possibility to swipe across the mucosal plane to interrogate a larger mucosal surface than a standard biopsy. pCLE has been validated for a diagnosis of HGD in BE, with 6 pCLE criteria being described: saw-toothed epithelial surface, enlarged cells, pleomorphic cells, not equidistant glands, glands unequal in size and shape, and lack of goblet cells (11). These yielded a diagnostic accuracy of 81.5%, with a substantial interobserver agreement (κ = 0.61). In a cross-sectional study on 101 patients, the combination of pCLE and high-resolution endoscopy had a sensitivity and a specificity for HGD/intramucosal cancer (IMC) of 93.5 and 67.1%, respectively (12). We previously used these diagnostic criteria in a small cross-sectional cohort study (n = 56), where pCLE had a sensitivity and a specificity for any grade of dysplasia of 96% and 74%, respectively (13). However, in the per-location analysis the sensitivity was lower at 83%, suggesting that the diagnostic criteria need refinement to allow a more robust diagnosis of LGD. Thus, the aim of this study were (i) to identify endomicroscopic features of BE-related LGD and (ii) to develop and validate pCLE diagnostic criteria for LGD in BE.

METHODS

Study design

Good quality video sequences were retrieved from 2 video libraries of pCLE procedures performed previously within 2 ethically approved studies evaluating patients with BE with and without dysplasia. The Cambridge library derived from a prospective single center study, which recruited 55 nonconsecutive patients with BE between 2012 and 2014 (13). pCLE was performed on the endoscopic areas targeted by autofluorescence imaging marked by argon plasma coagulation, with targeted biopsies taken within the same area to ensure correspondence between pCLE location and histologic assessment. This library consisted of over 300 video sequences from 194 endoscopic locations, of which 24 had LGD histology, 21 had HGD, and 7 IMC. The rate of LGD visible on white light endoscopy in this library was 14%. The Modena library derived from a single-center study recruiting 100 consecutive patients with BE between 2011 and 2012, of which 50 received pCLE diagnosis (14). The correspondence between pCLE and targeted biopsy location was ensured by immediate tissue sampling sequential to pCLE analysis. In the pCLE group, 14 patients received an LGD diagnosis. None of the cases of LGD included in this library corresponded to the lesions visible on WLE.

Similarly to the work conducted for the development of HGD criteria (11), this study consisted of 2 phases. The first phase aimed to the generation of the diagnostic criteria for LGD and their initial external validation, while the second phase aimed to the definitive external validation of the criteria by a larger panel of investigators with different level of experience in pCLE.

The reference standard was considered the histological diagnosis on the targeted biopsies, which was scored prior to the result of the index test (pCLE). Dysplasia was graded according to the Vienna classification (15) and confirmed by at least 2 independent pathologists with extensive experience in the diagnosis of BE (M.O.D. and L.R.B.). In case of discordance between the 2 pathologists, the cases were excluded from the study.

All video sequences were selected and collated by an investigator, who completed the online pCLE training module, but had no previous experience in pCLE.

Generation and preliminary validation of the diagnostic criteria (phase I)

Thirty good quality videos (non-dysplastic BE (NDBE), n = 10; LGD, n = 10; and HGD, n = 10) from the Cambridge library were assessed in an unblinded manner by an endoscopist with previous experience in pCLE (M.d.P.) and a pathologist with extensive experience in BE diagnosis (M.O.D.) to identify recurrent pCLE features of LGD (Figure 1). We included HGD videos in this phase so that criteria, which are specific for LGD, could be identified. These 2 investigators independently assessed each of these criteria in a different set of good quality videos (NDBE, n = 15; LGD, n = 10), blinded to the histologic diagnosis. The criteria were scored in a binary fashion to assess for each of their sensitivity, specificity, and interobserver agreements. Only the criteria with accuracy (average of sensitivity and specificity) >80% and κ-value > 0.4 were taken forward. The candidate criteria were further evaluated on the same video set by a third investigator with previous experience in pCLE diagnosis (H.B.) to ensure reproducibility of the interpretation, before taking the criteria to the second phase. A sensitivity and specificity analysis for various cutoffs on the number of criteria for a positive call of LGD was performed to determine a useful range of cutoffs.

Figure 1.

Schematic representation of study flowchart. In phase I (left), 2 investigators (one pathologist and one probe-based confocal laser endomicroscopy (pCLE)-experienced endoscopist) identified recurrent endomicroscopic features in low-grade dysplasia by blinded assessment of pCLE video sequences. Three investigators (one pathologist and 2 pCLE-experienced endoscopists) performed the initial validation of the criteria for the selection of the final diagnostic panel. In phase II (right), 6 endoscopists with different level of experience in pCLE performed the external validation on an independent video set (n = 57). The diagnostic accuracy and the interobserver agreement were calculated.

Validation of the diagnostic criteria (phase II)

For the purpose of the external validation, an independent set of 57 good and intermediate quality videos (NDBE, n = 23; LGD, n = 24; HGD, n = 10) were selected from the Modena and Cambridge libraries (Figure 1). The intermediate quality videos were allowed in this phase as long as they contained interpretable frames for optical diagnosis. Cambridge library videos used in phase II were different from those used in phase I. The videos were coded and collated in random order into a power point presentation. Six endoscopists, of which 3 had previous pCLE experience (at least 10 pCLE procedures for BE diagnosis) and 3 had no previous pCLE expertise, evaluated the diagnostic criteria obtained in phase I. Prior to the video examination, a training session was held with a detailed explanation of the pCLE features relative to the diagnostic criteria and joint assessment of 20 videos from phase I. In each video sequence, the 6 investigators, blinded to the histologic diagnosis, assessed each diagnostic criterion in a binary fashion (positive/negative) and made an overall diagnosis (dysplasia/no dysplasia), indicating the level of confidence (high/low). During the assessment, the investigators had the possibility to pause and rewind as it is done during live pCLE examination with the trackball. Although it was sufficient for a single frame to make a call of positive criterion, it was the discretion of the investigator to evaluate the significance of this and also based on the quality of the video and the presence of artifacts. We did not expect each case of LGD to have all 6 criteria positive as the videos were randomly selected. The assessors were not asked to make a differential diagnosis between LGD and HGD, as this was beyond the scope of this study. After the first 20 videos, a 30-minute interactive discussion was held to address the discrepancies on the diagnoses and compare them with the gold standard histopathology. Following this, the investigators assessed the remaining 37 videos.

Sample size and power analysis

There has been no previous study assessing the diagnostic accuracy of pCLE for LGD. Previously, a panel of pCLE criteria had a sensitivity of 75.6% for HGD. Our preliminary study suggested that the same diagnostic panel had a sensitivity in the per-location analysis of 83% (13). Assuming 80% as the true sensitivity for dysplasia of a panel of pCLE criteria, at least 18 dysplastic cases are required to formally show a sensitivity of at least 55% at a significance level of 0.05 and with a power of 0.7. To keep the data set balanced, we aimed for a similar number of controls.

Statistical analysis

Sensitivity, specificity, and accuracy were calculated as averages over all investigators. The lower bounds of the upper 95% credible intervals are provided for sensitivity and specificity. The credible interval for k out of n calls is obtained as the upper 95% quantile of the Beta (k+1, n-k+1) distribution (which is the posterior with a uniform Beta prior), where k is the average number of correct calls of all investigators and n is the number of samples with LGD for sensitivity and the number control samples for specificity. Intraobserver agreement was calculated using Cohen's κ statistic. Interobserver agreement among investigators was calculated using Fleiss' κ statistic. For the analyses assessing an agreement for minimum number of positive criteria for LGD diagnosis (Tables 2 and 4), a cutoff of n positive criteria was applied, regardless of what criteria were called positive. The differences between experienced and nonexperienced investigators in κ statistic were statistically assessed assuming approximate normality of the estimator for each group using its standard error. The differences between accuracies for these 2 groups were assessed with a t test on the logit-transformed values. Area under receiver operating characteristic (AROC) denotes the area under the convex hull of the receiver operating curve of sensitivity plotted over specificity for all cutoffs on the number of investigators making a call and is an additional measure alongside the average accuracy for the joint performance of all investigators. All analyses were performed in the R statistical environment.

Table 2.

Performance of different cutoffs on 6 pCLE diagnostic criteria in the blinded validation (phase I) by 3 endoscopists (in brackets the lower bound of the upper 95% credible interval)

Table 4.

Intraobserver agreement for the overall diagnosis and the individual diagnostic criteria

RESULTS

Phase I

Unblinded assessment of dysplastic and nondysplastic cases (NDBE, n = 10; LGD, n = 10; and HGD, n = 10) by one pathologist and one endoscopist led to the identification of 8 recurrent confocal features in LGD cases. These were (I) dark nonround glands, (II) irregular gland shape, (III) lack of goblet cells, (IV) disrupted honeycomb pattern, (V) variable degree of darkness with sharp cutoff, (VI) variable size of cells, (VII) pencil-shaped nuclei, and (VIII) cellular stratification. These 8 criteria were assessed in an independent set of 25 videos by the same 2 investigators blinded to the histologic diagnosis. The overall sensitivity, specificity, accuracy, and interobserver agreements and AROC for each criterion are shown in Table 1. Two of these criteria (disrupted honeycomb pattern and pencil-shaped nuclei) were discarded as they did not meet the minimum thresholds of 80% accuracy and κ > 0.4. The remaining 6 represented the final panel (Figure 2; see Figure 1, Supplementary Digital Content 2, http://links.lww.com/CTG/A24). An internal validation of these 6 criteria was performed by a second endoscopist experienced in pCLE. The performance of the criteria was similar, with only 2 criteria showing a slightly lower overall diagnostic accuracy (73.3% for variable degree of darkness with sharp cutoff and 74.6% for variable size of cells). Analyzing the results for all 3 investigators, we assessed the optimal cutoff for the minimum number of positive criteria to make a call of LGD as shown in Table 2. Apart from applying the pCLE criteria, the investigators also provided an “overall diagnosis” of LGD or no dysplasia, which is also shown in Table 2. The highest average accuracy (86.1%) and interobserver agreement (0.82) was found for a cutoff of 5 positive criteria, the highest AROC (0.923) for a cutoff of 4. However, even a cutoff of 3 criteria showed a high sensitivity and an AROC of 0.917, which was close to the optimum. In view of the need for a high sensitivity of the procedure, we opted for a final cutoff of 3 out of 6 criteria to be validated in phase II of this study.

Table 1.

Performance of the initial 8 diagnostic criteria in the blinded external validation (phase I) by 2 investigators (in brackets the lower bound of the upper 95% credible interval)

Figure 2.

Final probe-based confocal laser endomicroscopy (pCLE) diagnostic panel, with pCLE view (top) and histological finding from the same endoscopic area (bottom). (a) dark nonround glands, (b) irregular gland shape, (c) lack of goblet cells, (d) variable degree of darkness with sharp cutoff (yellow arrowhead indicate transition points), (e) variable cell size (yellow arrowheads indicate the site of cellular pleomorphism), and (f) cellular stratification (yellow arrows indicate the 2 levels where cells are located). Although precise correspondence between glandular elements showed in the confocal and pathology image is difficult to be ascertained, similar features were observed on pCLE and histology.

Phase II

The 6 criteria identified in phase I were evaluated by 6 endoscopists (3 experienced and 3 non-experienced in pCLE), blinded to histologic diagnosis and any other clinical information, on a set of 57 videos (NDBE, n = 23; LGD, n = 24; and HGD, n = 10). The performance of each of the 6 criteria, the “overall diagnosis,” and the diagnosis obtained applying a cutoff of ≥ 3 positive criteria are shown in Table 3. The accuracy of each criterion ranged from 69.2% to 77.6%, with an overall agreement ranging from poor to good (κ value between 0.273 and 0.667). The “overall diagnosis” and the cutoff of ≥3 positive criteria had accuracy of 79.1% and 78.3%, and AROC values of 0.921 and 0.888, respectively. Notably, the sensitivity slightly increased with the cut-off ≥3 positive criteria, while the overall accuracy was the same compared with “overall diagnosis”; that is, a cutoff procedure has the desirable effect of increasing sensitivity. It also increases the interobserver agreement, which is highest (0.654) with the cutoff of 3 positive criteria. There was a marginal improvement in the agreement levels after the interactive discussion, which was not statistically significant (κ value increased from 0.540 to 6.444, P = 0.11). The intraobserver agreement for the overall diagnosis showed a large degree of variation from fair to almost perfect across different investigators (Table 4). When we looked at the individual criteria, similarly to what observed for inter-rater analysis, the intraobserver agreement was better for the first 3 criteria and varied on average from fair to moderate (Table 4). In order to exclude that the new LGD criteria could miss a diagnosis of HGD, which could be detrimental for the utility of pCLE in the assessment of the overall dysplasia, we also included 10 HGD cases in the phase II validation. Reassuringly, the sensitivity for any grade of dysplasia slightly increased to 84.3% for the “overall diagnosis” and 84.8% for the cutoff of ≥3 positive criteria (see Table 1, Supplementary Digital Content 1, http://links.lww.com/CTG/A23). The sensitivity for HGD only was 95.0% for the “overall diagnosis” and 91.7% for the cutoff of ≥3 positive criteria (see Table 2, Supplementary Digital Content 1, http://links.lww.com/CTG/A23). These data indicate that the new LGD criteria can also accurately diagnose HGD.

Table 3.

Performance of the selected 6 pCLE criteria for a diagnosis of LGD in the blinded validation (phase II, 23 NDBE and 24 LGD cases) by 6 endoscopists (the lower bound of the upper 95% credible interval is shown in brackets)

Finally, we were interested in the effect of previous pCLE experience in performance (Table 5). The endoscopists experienced in pCLE performed slightly better than the nonexperienced ones on all statistics, even though this did not reach statistical significance (P = 0.1 for overall diagnosis and P = 0.08 for cutoff ≥3 positive criteria). Interestingly, the cutoff compared to the overall diagnosis increased sensitivity for experienced investigators (from 83.3% to 88.9%), but not for nonexperts (from 76.4% to 75%).

Table 5.

Performance of the panel of pCLE criteria for an LGD diagnosis using either a cutoff of 3 out 6 positive criteria (cutoff 3/6) or the overall diagnosis

DISCUSSION

In this study, we applied a robust methodology to develop and validate novel pCLE diagnostic criteria for LGD in BE.

The American Society of Gastrointestinal Endoscopy program for the Preservation and Incorporation of Valuable Endoscopic Innovations (PIVI) has set the minimum thresholds of sensitivity, specificity, and negative predictive value (NPV) of novel imaging modalities for a diagnosis of HGD/IMC at 90%, 80%, and 98%, respectively. A recent meta-analysis showed that pCLE has sensitivity, NPV, and specificity for a diagnosis of HGD and IMC of 90.3%, 95.1%, and 77.3%, respectively (16). Hence, the minimum NPV and specificity PIVI thresholds of 98% and 80% were not met (17). Virtually, all the cases of IMC and approximately 2/3 of cases of HGD are associated to endoscopically visible lesions (18,19); hence, the role of pCLE to inform clinical management is limited in these cases. In contrast, LGD in BE is generally inconspicuous on white light endoscopy and even enhanced imaging techniques and represents only about 10% of visible neoplasia (20). Hence, applying the PIVI threshold to endoscopic diagnosis of LGD would be far too stringent. Nevertheless, LGD correlates with a significantly increased risk of cancer progression (5,21,22), indicating that an endoscopic diagnosis of LGD is essential and that availability of imaging techniques to aid this diagnosis is warranted. To date, there are no studies that have addressed the accuracy of enhanced imaging modalities specifically for a diagnosis of LGD.

LGD is associated with cellular and architectural features that are often difficult to interpret even by expert pathologists. As a result, interobserver agreement among pathologists is poor or moderate at best (22). In our study, the interobserver agreement for the 6 diagnostic criteria ranged in phase I from moderate to substantial and in phase II from poor to substantial. However, it was very encouraging that when the criteria were used as a panel, the agreement among the endoscopists was substantial (κ = 0.654).

We found some overlapping pCLE features between HGD and LGD, such as the lack of goblet cells, the irregular gland structures, and the cellular pleomorphisms. This was not surprising given that the pathological cutoffs between LGD and HGD occur along a continuum with clear distinctions at either end of the spectrum but with somewhat overlapping features at the upper end of LGD, which is one of the reasons for the well-documented interobserver and intraobserver variability in the grading of dysplasia in BE. However, compared with HGD/IMC, the confocal features of LGD are subtler and the level of architectural and cellular distortion far less pronounced, which explains the lack of saw-toothed epithelial surface and non-equidistant glands in LGD cases. We believe that the dark glands in dysplastic cases reflect the high cellularity in dysplasia, which does not allow penetration of fluorescein in the intercellular spaces, and the variable degree of darkness reflects a sudden transition in the level of cellularity and stratification. Overall, it was encouraging that the pCLE criteria for LGD performed equally well to the validated pCLE criteria for HGD/IMC. In particular, compared to the HGD criteria, the LGD has a slightly better sensitivity (81.9% vs 75%), but lower specificity (74.6% vs 85.2%). The sensitivity increased when we included the cases of HGD, suggesting that the revised pCLE criteria can be used for a diagnosis of any grade of dysplasia, which is very useful in clinical practice.

This study has several strengths. We applied a strict methodology for the identification and the validation of the diagnostic criteria, with validation of these by larger panel of endoscopists. The good diagnostic accuracy achieved in phase II indicates that the diagnostic panel can be successfully applied even by nonexperienced operators. In addition, we found a clear correspondence between the recurrent confocal features in cases with LGD and the histologic appearances in the same cases. This indicates that it is very unlikely that the confocal criteria represent imaging artifacts. Finally, we used a robust pathological diagnosis of dysplasia with review of the cases by at least 2 pathologists and exclusion of cases where agreement was not achieved.

However, this study has also some limitations. The images were retrieved retrospectively, and 2 of the investigators were involved in the diagnostic procedures. Nevertheless, the original video libraries were historical and the retrieval of the videos was performed by investigators who did not take part in diagnostic procedures and validation. This ensured that the observers were blinded to the histologic diagnosis. Another limitation is the lack of endoscopic images corresponding to the confocal videos. This is a useful adjunct in clinical practice; however, it can also influence the pretest diagnosis. Furthermore, although all the efforts were made in the original trials to ensure correspondence between location of the video analysis and forceps biopsy, we cannot exclude that in some cases histology may not accurately represent the cellular structures shown in the videos. This is an inherent limitation of all endoscopy studies assessing magnified imaging modalities. Finally, we did not evaluate the differential diagnosis between LGD and HGD by pCLE. The primary aim of the study was to develop diagnostic criteria for LGD. Given the well-described difficulties in the differential diagnosis between LGD and HGD on conventional pathology and the fact that the clinical management of LGD and HGD is now very similar (3,7), we think that it is not possible nor clinically useful to distinguish different degrees of dysplasia by pCLE.

In summary, we have developed novel pCLE criteria for a diagnosis of LGD in BE and we propose that these should be used in future prospective studies aiming to endoscopic diagnosis of LGD.

CONFLICTS OF INTEREST

Guarantor of the article: Massimiliano di Pietro, MD.

Specific author contributions: study concept and design: M.d.P., H.B., M.O.D., and L.W. Acquisition of data: M.d.P., H.B., M.O.D., J.O.-F.-S., M.I., I.M., L.R.B., and K.R. Analysis and interpretation of data: M.d.P., M.O.D., and L.W. Statistical analysis: M.d.P., R.P., and L.W. Drafting of the article: M.d.P. and L.W. Technical and material support: P.S. and H.A. Final approval of the manuscript: All authors.

Financial support: The Addenbrookes Charitable Trust and the Kathy Shaw Memorial (Oesophageal Cancer) Fund funded the endoscopy equipment at Cambridge University Hospital. This study received infrastructure support from the Experimental Cancer Medicine Center and from the Cambridge Cancer Centre.

Potential competing interests: None to report.

WHAT IS KNOWN

✓ LGD in BE carries a significant cancer risk.

✓ Currently, there are no imaging modalities to aid this diagnosis.

✓ pCLE allows in vivo view of the gastrointestinal mucosa at cellular resolution.

WHAT IS NEW HERE

✓ We identified 6 pCLE features that are recurrently associated with LGD in BE, 3 of which have never been described before.

✓ As a panel, these diagnostic criteria led to the diagnosis of LGD in BE with 81.9% sensitivity and 74.6% specificity.

✓ The interobserver agreement for the overall pCLE diagnosis of LGD was substantial, indicating steep learning curve.

TRANSLATIONAL IMPACT

✓ The new pCLE criteria for diagnosis of LGD in BE can be used in future prospective studies.