Do endosonographers agree on the presence of bile duct sludge and the subsequent need for intervention?

Background and study aims  Endoscopic ultrasonography (EUS) is a tool widely used to diagnose bile duct lithiasis. In approximately one out of five patients with positive findings at EUS, sludge is detected in the bile duct instead of stones. The objective of this study was to establish the agreement among endosonographers regarding: 1. presence of common bile duct (CBD) stones, microlithiasis and sludge; and 2. the need for subsequent treatment. Patients and methods  30 EUS videos of patients with an intermediate probability of CBD stones were evaluated by 41 endosonographers. Experience in EUS and endoscopic retrograde cholangiopancreatography, and the endosonographers' type of practices were recorded. Fleiss' kappa statistics were used to quantify the agreement. Associations between levels of experience and both EUS ratings and treatment decisions were investigated using mixed effects models. Results  A total of 1230 ratings and treatment decisions were evaluated. The overall agreement on EUS findings was fair (Fleiss' κ 0.32). The agreement on presence of stones was moderate (κ 0.46). For microlithiasis it was fair (κ 0.25) and for sludge it was slight (κ 0.16). In cases with CBD stones there was an almost perfect agreement for the decision to subsequently perform an ERC + ES. In case of presumed microlithiasis or sludge an ERC was opted for in 78 % and 51 % of cases, respectively. Differences in experience and types of practice appear unrelated to the agreement on both EUS findings and the decision for subsequent treatment. Conclusions  There is only slight agreement among endosonographers regarding the presence of bile duct sludge. Regarding the need for subsequent treatment of bile duct sludge there is no consensus. The Author(s). This is an open access article published by Thieme under the terms of the Creative Commons Attribution-NonDerivative-NonCommercial License, permitting copying and reproduction so long as the original work is given appropriate credit. Contents may not be used for commercial purposes, or adapted, remixed, transformed or built upon. (https://creativecommons.org/licenses/by-nc-nd/4.0/).

Introduction

In patients with suspected bile duct stones, endoscopic ultrasonography (EUS) is an excellent tool to prevent unnecessary endoscopic retrograde cholangiography (ERC) with endoscopic sphincterotomy (ES) and its complications. EUS in patients with suspected bile duct stones rules out the presence of bile duct stones in up to 65 % of patients 1 2 . In patients without cholangitis, the 2019 ESGE guideline on endoscopic management of common bile duct (CBD) stones recommends to proceed to ERC + ES only when CBD stones are proven on imaging modalities that have a high specificity 3 .

In approximately one out of five patients with positive findings at EUS performed for this indication, bile duct sludge is detected instead of stones 4 5 . Sludge is an ultrasound diagnosis usually described as layered, mobile, low-amplitude echoes without shadowing occurring in either the gallbladder and/or bile ducts 6 7 . Studies comparing EUS and microscopic examination of bile concluded that sludge represents bile precipitate out of solution 8 9 . Usually this precipitate consists of cholesterol monohydrate crystals, calcium bilirubinate granules and other calcium salts. Sludge is considered an early and reversible state of bile stone disease only to be treated when causing biliary symptoms 10 11 . Gallbladder sludge in symptomatic patients is considered an indication for cholecystectomy.

Sludge and small stones or microlithiasis (defined as stones < 3 mm) in the CBD, are known to have an increased tendency to pass spontaneously into the bowel in comparison to stones > 3 mm 5 12 13 14 15 16 17 . They have also been associated with an increased incidence of biliary pancreatitis 18 19 . However, because the likelihood of developing biliary complications in patients with bile duct microlithiasis or sludge is currently unknown, it is unclear if detection of sludge or microlithiasis in the bile duct at EUS should prompt for an ERC + ES, or whether a watchful waiting strategy can be adopted.

EUS is a known operator-dependent technique 20 . Moreover, agreement regarding the presence of sludge or microlithiasis on EUS and the potential indication for treatment have not been investigated.

The purpose of the current study was to determine the interobserver agreement among endosonographers regarding the presence or absence of CBD stones, microlithiasis and sludge, and their advice for subsequent treatment. We also evaluated the role of the endosonographers’ experience in EUS and/or endoscopic retrograde cholangiopancreatography (ERCP), and the endosonographers’ type of practice regarding EUS findings and treatment decisions.

Patients and methods

Study aims

The objective of this study was to establish the agreement among endosonographers regarding: 1. presence of CBD stones, microlithiasis and sludge, and 2. the need for subsequent treatment.

Study design

The study protocol was approved by the local medical ethics committee (METC Leiden, Den Haag, Delft, Z19.050). Videos of linear EUS procedures of patients (> 18 years old) with intermediate probability of bile duct lithiasis, according to the 2010 ASGE criteria 21 , were recorded for the purpose of this study.

Video fragments

Videos were digitally recorded by 5 different endosonographers from different hospitals using linear EUS-endoscopes (GF-UCT 180 or GF-UCT 260, Olympus, Leiderdorp, the Netherlands) and different processors (Aloka F75, Aloka α10 or Aloka α7, Hitachi Medical Systems B.V., Reeuwijk, the Netherlands). All recording endosonographers had more than 10 years of clinical experience performing and teaching EUS. Aiming for a significant proportion of invited endosonographers to comply with our protocol and complete the evaluation we chose to limit the number of video fragments to 30 and the length of the fragments to 30 seconds. Fifteen EUS video fragments were filmed from the duodenal bulb, and 15 from the descending duodenum. Videos were edited by one endosonographer using Movavi video editor plus, version 15.4.0 (Movavi Software Limited, Limassol, Cyprus). Video fragments were presented in the best possible video-format (mp4-files, framesize 1920x1080, speed 30 frames/sec) incorporated in a web-based survey using Castor v2020.1.16 (Castor EDC, Amsterdam, the Netherlands). None of the endosonographers involved in recording and editing the videos was involved in rating the video fragments for the purpose of this study.

Endosonographers

Endosonographers in the Netherlands are all formally trained in endosonography for at least a year at a referral center. A total of 113 Endosonographers was invited to participate in this study by e-mail. Of the invited endosonographers 58 did not respond to the invitation, 3 responded not to be willing to participate, and 11 did not finish the survey before the deadline. For each of the 41 participants experience in both EUS and ERCP including the number of years performing these interventions, and number of procedures performed annually, were recorded. Types of practice (tertiary care/community hospital) were also registered.

Evaluation of video fragments

Each video fragment was rated by each observer independently for the presence or absence of stones, microlithiasis or sludge in the CBD ( Fig. 1 ). Sludge was defined as echoic, cloud shaped and mobile bile duct content, without acoustic shadowing. Microlithiasis was defined as stones < 3 mm in size with or without acoustic shadowing ( Table 1 ) 22 23 24 . The raters were asked whether or not they would proceed to ERC + ES in each case.

Fig. 1 

Images of evaluated EUS videos. Microlithiasis, sludge or artifact?

EUS definitions of various presentations of bile duct lithiasis.

	Definition
Sludge	Layered, cloud shaped, mobile echoic bile duct content, without acoustic shadowing
Microlithiasis	Hyperechoic circumscript bile duct content, < 3 mm with or without acoustic shadowing
Stone(s)	Hyperechoic circumscript bile duct content, ≥ 3 mm with or without acoustic shadowing

References: 21 22 23

EUS, endoscopic ultrasonography.

The primary endpoint was the interobserver agreement among endosonographers on the presence or absence of CBD stones, microlithiasis or sludge.

Secondary endpoints were the interobserver agreement on the need for ERC + ES (yes/no) and differences in interobserver agreement between experts and non-experts.

Statistics

The analyses were performed using R version 4.0.2 (2020–06–22) (R Core Team 2020). Assuming that 41 endosonographers would evaluate 30 videos, and that the prevalence of various EUS diagnoses would be “clean” (50 %), “sludge” (15 %), “microlithiasis” (5 %), or “one or more stones” (30 %), we performed a power calculation using simulation. A total of 500 datasets was generated with varying values of agreement from which Fleiss’ κ and corresponding 95 % confidence intervals were calculated. The resulting estimates are shown in Fig. 1a and Table 1a ( Appendix ). To approximate the values of the lower and upper bounds of the 95 % confidence interval for any given κ-value, linear regression models were fitted in which the non-linear association with the corresponding κ-value was modeled using natural cubic splines with 3 degrees of freedom. The fitted values show that for slight agreement (κ 0.1), the 95 % confidence interval can be expected to range from approximately 0.05 to 0.15, and that for moderate agreement (κ 0.4) the corresponding 95 % confidence interval is expected to range from approximately 0.27 to 0.53.

Characteristics of the participating raters were summarized using medians and ranges or counts and proportions, as appropriate.

Multirater Fleiss’ κ-values were calculated to quantify the agreement between raters. Κ-statistics were interpreted based on the convention by Landis and Koch, and compared with a two-sided paired t-test with level α = 0.05, using the “linearization method” proposed by Gwet et al. 25 26 . The linearity correction is necessary since most agreement coefficients are not linear statistics which is a requirement for the standard t-test.

To estimate the expected proportion of videos for which an endoscopist would advise ERC + ES, while taking into account the correlation between the evaluations of multiple videos by the same endoscopist, we fitted a logistic mixed model for the advice of ERC + ES depending on the diagnosis and included endoscopist specific (random) effects for the intercept and diagnosis 27 .

Associations between levels of experience and either the diagnosis of “sludge” or the advice for subsequent ERC + ES were investigated using logistic mixed effects models, that use the experience level as fixed effect covariate and take into account the correlation between repeated measurements within endoscopists (i. e., the same endoscopist scoring multiple videos) by including an endoscopist specific (random) intercept. The results are marginalized using the technique described by Hedeker et al., as implemented in the R package GLMMadaptive and can be interpreted on the population level 28 29 .

Results

Endosonographers and their experience

The web-based survey and evaluation of the 30 EUS video fragments was completed by 41 endosonographers. Eight endosonographers (19 %) worked in academic practice and the remaining 33 (81 %) in community hospitals ( Table 2 ). Median post-training EUS experience was 6 years (range 1–25 years) while performing a median of 60 (20–300) procedures annually. The majority of endosonographers (27 of 41, 66 %) also performed ERCP procedures, with a median procedural experience of 10 years (range 1–30 years) while performing a median of 73 (20–300) ERCP procedures annually ( Table 2 ).

Endosonographers and their experience.

Type of practice (n = 41)	n (%)
Academic hospital	8 (19)
Community hospital	33 (81)
EUS experience (n = 41)	Median (range)
Years	6 (1–25)
Procedures/year	60 (20–300)
ERCP experience (n = 27)	Median (range)
Years	10 (1–30)
Procedures/year	73 (30–400)

EUS, endoscopic ultrasonography; ERCP, endoscopic retrograde cholangiopancreatography.

Interobserver agreement on EUS diagnosis and treatment decisions

A total of 1230 ratings (41 × 30 video fragments) and 1230 treatment decisions were evaluated. The overall interobserver agreement on EUS findings was fair (Fleiss’ κ 0.32). The agreement on presence or absence of stones was moderate (both κ 0.46). For microlithiasis the agreement was fair (κ 0.25) and for sludge there was slight agreement (κ 0.16) ( Table 3 ).

Ratings, interobserver agreement, and advice about treatment .

EUS diagnosis	n	Fleiss’ κ (95 %CI)	Advice about ERCP
			Yes (%)	No
Clean CBD	646	0.46 (0.32–0.60)	1 (0)	645
Sludge	235	0.16 (0.07–0.25)	120 (51)	115
Microlithiasis	194	0.25 (0.07–0.43)	152 (78)	42
One or more stone(s)	155	0.46 (0.13–0.78)	154 (99)	1
Overall	1230	0.35 (0.21–0.48)	427 (35)	803

EUS, endoscopic ultrasonography; CBD, common bile duct.

In 803 out of 1230 cases (65 %) the evaluation of an EUS video fragment led to the advice not to proceed to ERC + ES. The overall interobserver agreement to proceed to ERC + ES was moderate (κ 0.41, 95 % CI [0.25–0.55]). In cases with CBD stones or with a “clean” CBD there was almost perfect agreement for the decision to subsequently perform an ERC + ES or not. In patients with presumed microlithiasis or sludge it was chosen to perform ERC + ES in 78 % and 51 % of cases, respectively ( Table 3 ).

In cases where microlithiasis was diagnosed the advice to proceed to ERC + ES ranged from 20 % to 100 % among endosonographers. When sludge was diagnosed the advice to proceed to ERC + ES ranged from 0 % to 100 %. Seven of 41 raters (17 %) always advised against ERC + ES, and 13 raters (32 %) always advised to proceed to ERC + ES in these cases ( Fig. 2 and Fig. 3 ).

Fig. 2

 Proportion of videos for which ERCP was advised per endoscopists by EUS diagnosis (sludge vs. other).

Fig. 3

 Estimated probability of advice to proceed with ERCP per endosonographer per diagnosis.

Experience and interobserver agreement

Agreement for EUS diagnosis among academic hospital endosonographers was fair (κ 0.32, 95 %CI [0.16–0.48]) as was agreement among community hospitals endosonographers (κ 0.32, 95 %CI [0.20–0.43]). For the advice to proceed to ERC + ES there was moderate agreement amongst both academic and community hospital endosonographers, κ 0.41, 95 %CI [0.23–0.59] vs κ 0.40, 95 %CI [0.25–0.55] respectively.

There was no evidence for an association between differences in EUS experience (either in years or in number of procedures performed annually) and the proportion of sludge diagnoses ( Table 4 ). There was no significant association between the advice to proceed to ERC + ES and whether or not the endosonographer performs ERCPs him- or herself, ERCP experience in years, or number of ERCPs performed annually.

Effect of experience on sludge diagnosis and the decision to proceed to ERCP.

EUS experience and sludge diagnosis (n = 41)	OR	P value	95 % CI
Years of experience in EUS	0.99	0.66	0.96–1.03
EUS procedures/year	1.01	0.38	0.98–1.04
Advise to proceed to ERCP (n = 41)	OR	P value	95 % CI
ERCP performers vs non-performers	1.26	0.23	0.87–1.8
ERCP experience and advice to proceed to ERCP (n = 27)	OR	P value	95 % CI
Years of experience in ERCP	0.99	0.96	0.96–1.04
ERCP procedures/year	0.99	0.74	0.96–1.03

ERCP, endoscopic retrograde cholangiopancreatography; EUS, endoscopic ultrasonography.

Endoscope position and interobserver agreement

The agreement on EUS diagnosis of video fragments recorded from both the duodenal bulb and the descending duodenum was fair (κ 0.36, 95 % CI [0.17–0.56] and κ 0.28, 95 % CI [0.14–0.43] respectively). There was no significant difference in agreement regarding videos from different scope positions ( P  = 0.50).

Discussion

Endosonography plays a pivotal role in selecting patients with suspected bile duct lithiasis for ERC + ES. However, interobserver agreement among endosonographers regarding presence of CBD stones, which was the primary endpoint of this study, was only moderate. Moreover, there was only slight interobserver agreement among endosonographers regarding presence of microlithiasis and sludge, the second primary endpoint.

In cases with either stones or a clean bile duct there was an almost perfect agreement to either proceed to or refrain from ERC + ES. In cases with microlithiasis or sludge in the bile duct there was no consensus about the need for subsequent ERC + ES. There was no evidence for a relation between differences in experience in both EUS and/or ERC + ES, types of practice, and agreement on both EUS findings and the decision for subsequent treatment.

This was the first study to evaluate interobserver agreement of endosonographers rating EUS video fragments of patients with suspected bile duct lithiasis. Although interobserver agreement on gallbladder sludge is probably better than the only slight agreement among endosonographers on the presence of sludge in de CBD, the findings of this study shed a different light on previous publications regarding the finding of sludge (in gallbladder and/or CBD) at EUS.

Sludge accounts for 20 % to 25 % of the abnormalities detected in the CBD at EUS in patients with suspected bile duct lithiasis 4 5 . This estimate is based on two retrospective studies only and may be an overestimation of the true prevalence. Biliary sludge is detected in up to 75 % of patients with idiopathic pancreatitis at EUS in patients with their gallbladder in situ. In idiopathic pancreatitis in postcholecystectomy patients CBD sludge is detected in 10 % to 15 % of cases 30 . In these patients initially suspected to have suffered from idiopathic pancreatitis the diagnosis of sludge in the CBD is relevant since it is considered an indication for cholecystectomy and ERC + ES respectively 31 32 . In the current study comprising of patients with an intermediate probability of gallstones but not having suffered from pancreatitis, ERC + ES was considered not indicated by the endosonographer in 65 % of the cases. This proportion is similar to the findings of two meta-analyses studying the role of EUS in patients with suspected bile duct lithiasis in clinical practice 1 2 .

Strengths of the current study are the substantial number of endosonographers involved when compared to previously published EUS interobserver studies on chronic pancreatitis, findings in familial pancreatic cancer kindred’s and pancreatic cysts 33 34 35 36 . The 41 contributing endosonographers from 27 different hospitals represent approximately 25 % of the total number of endosonographers in the Netherlands. The results of our study therefore signify a rational representation of how findings of biliary sludge are diagnosed and interpreted throughout the Netherlands.

Limitations of our study are the fact that 30-second video fragments were used. Having a second look when in doubt means playing the video again, which is very different from repositioning the endoscope to confirm or reject a potential diagnosis from another angle.

The only moderate agreement on the presence or absence of bile duct stones is likely to be a reflection of the known operator dependency of EUS 37 . Not only have learning curves of advanced endoscopy trainees proven to be highly variable in prospective studies 38 39 , good or even excellent interobserver agreements are rarely found in studies on interobserver variability in EUS 33 34 35 36 .

Improving interobserver agreement of EUS in patients with suspected bile duct lithiasis starts with basic knowledge regarding the limitations of EUS imaging and potential pitfalls of image interpretation and “tricks” to overcome this. All this should be taught as part of a formal training program learning EUS. Evaluation of the CBD with a linear EUS scope requires careful tracing of the CBD from the liver hilum all the way down to the ampulla by means of different scope positions. It may be difficult to distinguish sludge from EUS imaging artifacts caused by, for example the cystic duct junction or the crossing of vessels, especially in a non-dilated duct and/or a non-stable position of the endoscope. Side-lobe artifacts, off-axis secondary projections of the ultrasound beam, can be misinterpreted as sludge 40 . Repositioning the transducer can cause these artifacts to disappear. Adherence to the simple rule that “when a EUS finding cannot be reproduced it does not exist”, prevents a lot of over interpretation and false positive findings.

Knowledge regarding the natural behavior of bile duct stones is limited. The Swedish Gallriks study describes an increased likelihood of biliary complications (biliary pancreatitis, cholangitis or bile duct obstruction) of up to 25 % during a follow-up of 4 years after cholecystectomy when leaving bile duct stones in situ, in comparison to a likelihood of 13 % of complications after clearance of the bile duct (Odds Ratio [OR] 0.44, 95 %CI 0.35–0.55) 41 . This justifies performing ERC + ES in all patients with bile duct stones, regardless of the risk of complications 3 . However, the advantage of an intervention aiming to remove bile duct stones is less clear in stones < 4 mm, which may be related to the increased possibility of spontaneous passage into the bowel. Although bile duct sludge is increasingly detected with the growing use of EUS in recent years, its natural behavior is currently unknown. To answer the question whether the EUS findings CBD sludge and microlithiasis in patients with an intermediate probability of bile duct stones justify a subsequent ERC + ES, or that a watchful waiting strategy can be adopted, further studies are needed.

Conclusions

In summary, our study demonstrates that there is moderate interobserver agreement regarding presence or absence of bile duct stones, and only limited agreement regarding the presence of CBD sludge and microlithiasis among endosonographers. In cases with either sludge or microlithiasis in the bile duct at EUS, there is a lack of consensus whether a subsequent ERC + ES should be performed. Agreement regarding EUS findings or treatment decisions appear unrelated to experience of the rating endosonographer. To guide clinicians and prevent potentially unnecessary ERC + ES, further research into the reliability of an EUS diagnosis of biliary sludge and the need for intervention is indicated.