Trainee caseload correlates with ERCP success rates but not with procedure-related complications: results from a prospective study (the QUASIE cohort).

BACKGROUND AND STUDY AIM: Endoscopy society guidelines recommend a minimum of 200 cases for endoscopic retrograde cholangiopancreatography (ERCP) trainees in order to ensure competency and quality standards. However, there are few data regarding procedure-related complication rates and added risk for patients during this learning process. We aimed to evaluate the correlation between trainee caseload and procedure- and patient-related outcomes in an ERCP training program, and to assess the risk factors for ERCP failure and complications. PATIENTS AND METHODS: We conducted a prospective study of all procedures performed in the ERCP training program at Colentina Clinical Hospital, Bucharest, Romania. Relevant data for each procedure (diagnosis, cannulation method, outcome, and complications during the following 30 days) as well as operator experience were documented. Univariable and multivariable analysis of the risk factors for ERCP failure and complications was done by analyzing the procedures completed by expert and trainee endoscopists during the study period.
RESULTS: The analysis included 534 ERCPs performed by 1 expert and 3 supervised trainees during a 12-month period. Technical success rates were comparable in the trainee and expert groups, and no statistically significant difference was found between the two groups with regard to procedure-related complications and mortality. The more experienced trainees had a better chance of successfully completing a procedure (odds ratio of 1.1 for each additional 10 ERCPs performed), but post-ERCP complications were unrelated to individual trainee caseloads on multivariable analysis.
CONCLUSION: The ERCP technical success rate increases with trainee experience, reflecting the learning curve of individual operators. However, the complication rates are similar across different levels of operator experience, indicating that ERCPs performed by supervised trainees imply no additional risk for patients.

Introduction

Endoscopic retrograde cholangiography (ERCP) has evolved from a diagnostic to mainly a therapeutic procedure during the past decades, allowing the minimally invasive treatment of both benign and malignant diseases of the bile ducts and pancreas. Although ERCP is generally considered a safe procedure, complications including post-ERCP pancreatitis, bleeding, and perforation can occur in up to 10 % of cases, with an associated mortality rate of about 1 % 1 2.

Data from large nationwide registries have shown that more experienced endoscopists with high case volumes have higher success rates and fewer procedure-related complications than do less experienced endoscopists with low case volumes 3 4. These results are supported by earlier studies from teaching hospitals showing that most trainees achieve satisfactory levels of competency after performing more than 150 to 200 procedures 5 6.

Based on these observations, competency in ERCP is now granted based on specific requirements, such as completing a minimum of 200 procedures and achieving an overall biliary cannulation rate of at least 85 % 7 8. However, currently there are limited data available on the additional risk for complications when procedures are performed during training programs. We aimed to explore the relationship between the learning curve of endoscopy fellows and procedure-related complications in a training program setting.

Patients and methods

Patients

We conducted a prospective study of all patients undergoing ERCP in our unit during a 12-month period from January 2014 to January 2015. Each patient’s age, sex, and indication for ERCP were documented. Information about the type and level of difficulty of each procedure according to the proposed American Society for Gastrointestinal Endoscopy (ASGE) scale (Table 1 9), the patient’s papillary anatomy (native papilla/previous sphincterotomy), and the cannulation method (guidewire, contrast injection, precut sphincterotomy) was provided by the attending endoscopist through a standard report form. Patients were followed for up to 30 days after the procedure via their attending physician or primary caregiver, and procedure-related complications (bleeding, post-ERCP pancreatitis, perforation, cholangitis, and death) were documented. The study design was approved by the local ethics committee, and all patients signed a standard informed consent form before undergoing the endoscopic procedure.

Scale of difficulty for biliopancreatic procedures.

Level of difficulty *
I	Deep CBD / PD cannulation, stent replacement
II	Distal CBD stricture, PD stenting, stone extraction < 10 mm
III	Stone extraction > 10 mm, PD stone extraction < 5 mm, papilla minor cannulation in patient with pancreas divisum, hilar stricture
IV	Whipple / Roux-en-Y anatomy, migrated PD stent, intraductal imaging

CBD, common bile duct; PD, pancreatic duct.

Addition of 1 point for previously failed procedure, procedure conducted outside office hours, and procedure conducted in patients younger than 5 years of age.

Training program

The standard procedure in our unit is first to attempt cannulation of a native papilla by the guidewire technique and in case of failure subsequently to use other techniques (contrast injection, precut sphincterotomy) to obtain selective cannulation of the desired duct. In this study, there was no time limit for attempted cannulation; however, if the procedure was performed by a trainee, a maximum of 10 minutes was allowed for cannulation before the expert operator took over the procedure. If at the end of the 10-minute interval the cannulation was unsuccessful, the expert could decide either to take over and continue the procedure or to repeat the procedure after an interval of 24 to 48 hours. This decision was based in each case on the expert’s clinical judgment. Only operators who performed at least 20 procedures during the study period according to this protocol were included in the final analysis. A 100-mg diclofenac suppository was administered intrarectally to all patients after their procedures in order to minimize the risk for post-ERCP pancreatitis.

Outcome measures

The outcome measures for our study were successful cannulation of the desired duct, rates of technical success (overall success of the procedure), and procedure-related complication rates. Technical success was defined as completion of the intended procedure (e. g., stent insertion, stone extraction). Procedure-related complications (post-ERCP pancreatitis, cholangitis, bleeding, and 30-day mortality) were documented and graded as mild, moderate, or severe according to the criteria proposed by Cotton et al. 10.

Learning curve and procedure outcome

The caseload of each trainee was divided into blocks of 10 successive procedures in order to evaluate the role of the learning curve in relation to the overall success rate of the procedures and the occurrence of complications. The number of cases completed before the study period was available and taken into account for each trainee. A case in which an expert took over from a trainee during any part of the procedure was excluded from the final analysis but was included in the trainee’s total caseload. We compared the adverse event rates and technical success rates by blocks of procedures for the entire subgroup of cases conducted by trainees in order to assess the relationship between trainee experience and procedure-related outcomes.

Sample size estimation

Based on an estimated 90 % technical success rate in the expert group of cases, we calculated that a total of 438 procedures (219 in each group) would be required to detect a 10 % difference between the expert and trainee groups at an alpha of 0.05 and a beta of 0.200. This sample size would also allow the detection of an increase in the overall adverse event rate from an estimated 10 % in the expert group to 20 % in the trainee group at a power of 80 % and a one-sided P value of less than 0.05.

Data analysis

Data were recorded and analyzed with SPSS for Windows, Version 16.0 (SPSS Inc., Chicago, Illinois, USA). Results were reported as mean and standard deviation for variables with a normal distribution, and as median, minimum, and maximum for variables with a non-normal distribution. Univariable analysis was conducted with the chi-squared test for nominal variables, with nonparametric tests (Mann – Whitney U test, Kruskall – Wallis test) for variables with a non-normal distribution, and with Student’s t test for variables with a normal distribution.

Multivariable analysis by logistic regression was used to analyze risk factors for technical success as well as for procedure-related complications. For the regression model, the traditional risk factors for procedure-related complications were selected as covariates before initiation of the study: sex, age, bilirubin levels, technical difficulty of the procedure (ASGE scale), method of cannulation, and operator experience (expert vs. trainee). Two-sided hypothesis testing was used, with a P value of less than 0.05 considered statistically significant.

Results

From January 31, 2014, to January 31, 2015, a total of 581 consecutive procedures were performed in the endoscopy department at Colentina Clinical Hospital. A total of 534 ERCPs conducted in 476 patients (range 1 – 4 procedures per patient) by 4 endoscopists (3 trainees starting with an average experience of 60 ERCPs and 1 expert operator with experience of more than 3000 ERCPs) were included in the final analysis (Fig. 1). The characteristics of the patients included in the study are detailed in Table 2. The trainees performed 283 procedures (53 %) and an expert endoscopist performed 251 procedures (47 %). The level of difficulty of the caseloads according to the ASGE scale is detailed in Table 3.

Fig. 1

 Flowchart of the procedures performed during the study period. ERCP, endoscopic retrograde cholangiopancreatography.

Characteristics of patients included in a study comparing outcomes of ERCP performed by trainess with outcomes of ERCP performed by an expert.

Characteristic	Total	Trainees	Expert	P value
Female sex, n (%)	271 (50.7)	133 (47)	138 (55)	0.06
Age, mean (SD), y	64 (15.3)	65.5 (14.7)	62.2 (16.1)	0.017*
Native papilla, n (%)	413 (77.3)	210 (74.2)	203 (80.8)
Indication for ERCP, n (%)
CBD stones	325 (60.8)	164 (57.9)	161 (64.1)
Malignant CBD stricture	121 (22.6)	70 (24.7)	51 (20.3)	0.34
Other	88 (16.5)	49 (17.3)	39 (15.5)
Increased bilirubin (> 1 mg/dL)	370 (69.2)	207 (73.1)	163 (64.9)	0.07

SD, standard deviation; ERCP, endoscopic retrograde cholangiopancreatography; CBD, common bile duct.

Statistically significant with Student’s t test.

Comparison of trainee and expert caseloads in regard to difficulty.

	Trainees	Expert	P value
ASGE level of difficulty
Level I	20	27
Level II	221	195	0.081
Level III	41	28
Level IV	1	1
Patient with native papilla	210 /283	203 /251	0.072

ASGE, American Society for Gastrointestinal Endoscopy.

Not significant with Mann – Whitney U test.

Not significant with chi-squared test.

Technical success was achieved in 77.7 % of the trainee procedures and in 76.8 % of the expert procedures (P = 0.75, chi-squared test) (Table 4). After adjustment for re-interventions, the technical success rate per patient increased to 88.1 % in the trainee group and to 90.1 % in the expert group (P = 0.54, chi-squared test).

Comparison of outcome measures in the entire study group: procedures completed by trainees vs. those completed by the expert.

Outcome measure	Trainees, n (%) [95 %CI]	Expert, n (%) [95 %CI]	RR [95 %CI]	P value
Technical success	220 (77.7) [72.8 – 82.8]	193 (76.8) [71.2 – 82.0]	1.01 [0.9 – 1.1]	0.75
Adverse events	25 (8.8) [5.8 – 12.8]	17 (6.7) [4.0 – 10.6]	1.3 [0.7 – 2.4]	0.42
PEP	16 (5.6) [3.3 – 9.0]	15 (5.9) [3.4 – 9.7]	0.95 [0.5 – 1.9]	0.88
Cholangitis	7 (2.4) [1.0 – 5.0]	3 (1.1) [0.2 – 3.4]	2.1 [0.5 – 8.0]	0.34
Mortality	4 (1.4) [0.4 – 3.6]	1 (0.4) [0.1 – 2.2]	3.6 [0.4 – 31.6]	0.37
Bleeding	4 (1.4) [0.4 – 3.6]	9 (3.5) [1.4 – 6.2]	0.4 [0.1 – 1.4]	0.24

CI, confidence interval; RR, relative risk; PEP, post – endoscopic retrograde cholangiopancreatography (ERCP) pancreatitis.

Procedure-related complications occurred in 25 procedures in the trainee group (8.8 %) and in 17 procedures in the expert group (6.7 %), with no significant difference between the two groups (P = 0.42, chi-squared test). All of the complications were classified as mild, except for severe pancreatitis in 3 patients (2 in the trainee group and 1 in the expert group). During the 30 days of follow-up, 5 patients died; 3 deaths were related to post-ERCP pancreatitis (respiratory failure in elderly and frail patients) and 2 deaths were caused by underlying disease unrelated to the endoscopic procedure (stage IV pancreatic adenocarcinoma in both cases).

Patients with a native papilla

There were 413 patients with a native papilla, 210 in the trainee group and 203 in the expert group. There was an overall successful cannulation rate of 85.2 %, with no significant difference between the two study groups (86.6 % in the trainee group and 83 % in the expert group, P = 0.43) (Table 5). After adjustment for re-interventions, the overall successful cannulation rate per patient was 92.3 %, with a 92.8 % successful cannulation rate in the expert group and a 91.8 % success rate in the trainee group (P = 0.8, chi-squared test).

Comparison of outcome measures in the patients with a native papilla: procedures completed by trainees vs. those completed by the expert.

Outcome measure	Trainees, n (%) [95 %CI]	Expert, n (%) [95 %CI]	RR [95 %CI]	P value
Successful cannulation	182 (86.6)	170 (83)		0.43
Technical success	162 (77.1) [71.6 – 83.3]	150 (73.8) [70.3 – 82.4]	1.0 [0.9 – 1.1]	0.81
Adverse event	20 (9.5) [6.0 – 14.5]	13 (6.4) [3.5 – 10.8]	1.5 [0.8 – 2.9]	0.27
PEP	14 (6.6) [3.7 – 11.0]	13 (6.4) [3.5 – 10.7]	1.0 [0.5 – 2.2]	1
Cholangitis	4 (1.9) [0.5 – 4.9]	1 (0.05) [0.0 – 1.8]	8.7 [0.5 – 160.6]	0.37
Bleeding	4 (1.9) [0.5 – 4.8]	9 (4.4) [2.1 – 8.3]	0.43 [0.13 – 1.37]	0.16
Mortality	3 (1.4) [0.3 – 4.1]	0 (0) [0.0 – 1.8]	6.8 [0.3 – 130.2]	0.62

CI, confidence interval; RR, relative risk; PEP, post – endoscopic retrograde cholangiopancreatography (ERCP) pancreatitis.

In regard to cannulation technique, most procedures were performed with the guidewire technique (83 %), followed by precut sphincterotomy (11 %) and contrast-guided cannulation (6 %); significantly more precut procedures were performed in the expert group (32 vs. 7, P < 0.001, chi-squared test).

No significant differences were found between the expert and trainee groups regarding procedural technical success and rate and type of procedure-related complications across the entire study group (Table 4) and in the patients with a native papilla (Table 5).

Risk factors for procedural failure and complications

On multivariable analysis, female sex was associated with an increased risk for any procedure-related adverse event (odds ratio [OR] 2.2, 95 %CI 1.1 – 4.5). In the patients with a native papilla, increases in the bilirubin level were associated with a higher risk for post-ERCP cholangitis and a higher risk for technical failure, with ORs of 1.14 (95 %CI 1.03 – 1.28) and 1.04 (95 %CI 1.01 – 1.06), respectively, for each increase in the serum bilirubin level of 1 mg/dL (Table 6). No risk factors for post-ERCP pancreatitis were identified on multivariable analysis, although a trend toward more post-ERCP pancreatitis in female patients was noted (P = 0.1).

Risk factors for various procedure-related outcomes obtained with multivariate analysis (logistic regression).

Factor	Outcome	OR	95 %CI
Female sex	Any adverse event	2.2	1.1 – 4.5
Bilirubin level	Cholangitis	1.14	1.03 – 1.28*
Bilirubin level	Technical failure	1.04	1.01 – 1.06*

OR, odds ratio; CI, confidence interval.

Relative risk for every increase in serum bilirubin of 1 mg/dL.

Trainee caseload and procedure outcome

On multivariable analysis, after adjustment for age and sex of the patients and grade of difficulty of the procedures, the procedures performed by trainees with a higher caseload were more likely to succeed than those performed by less experienced trainees (OR 1.1, 95 %CI 1.01 – 1.19 for each additional 10 procedures performed). However, no statistically significant difference in the adverse event rates according to trainee experience was found on multivariable analysis (OR 1.04, 95 %CI 0.93 – 1.17 for each additional 10 procedures performed) (Fig. 2, Fig. 3, Table 7).

Fig. 2

 Distribution of procedures and procedure-related complications according to the number of procedures previously performed by the trainee endoscopist. Trainee caseload is divided in blocks of 10 procedures.

Fig. 3

 Distribution of successful procedures according to the number of procedures previously performed by the trainee endoscopist.

Trainee caseload cross-tabulation.

Count		Trainee caseload															Total
		4	5	6	7	8	9	10	11	12	13	14	15	16	17	18
Adverse event	No	10	8	9	29	27	28	25	20	17	19	17	18	11	9	9	256
	Yes	0	1	1	1	2	2	4	3	3	1	2	2	1	2	0	25
Total		10	9	10	30	29	30	29	23	20	20	19	20	12	11	9	281
Outcome	Failed	5	2	1	9	6	8	7	5	4	3	8	2	1	0	1	62
	Successful	5	7	9	21	23	22	23	18	15	17	12	18	11	11	8	220
Total		10	9	10	30	29	30	30	23	19	20	20	20	12	11	9	282

Discussion

The main finding of our study is that although technical success increases with increased trainee caseload, there seems to be no correlation between trainee caseload and procedure-related complications. Also, our data suggest that within a training program, quality standards (i. e., successful cannulation, technical success) can be met with no additional risk for patients undergoing procedures. These are salient findings because patient safety during training programs in endoscopy is of paramount importance. Currently, however, few data are available concerning procedure outcomes and complication rates in this setting 11.

Learning curve studies in ERCP trainees show that approximately 200 procedures are required before a trainee can consistently achieve technical success 12 13. Furthermore, a higher volume of procedures per endoscopist 4 14 and per endoscopy service 15 seems to correlate with increased procedural success rates and fewer complications. We aimed to evaluate whether there might be a similar correlation between the learning curve for ERCP and procedure-related complications.

All the trainees in our study had previous experience performing ERCP, with an average caseload of 60 procedures before the start of the study period. The technical success rates and complication rates of the individual trainees did not differ significantly, but the technical success rates significantly improved as the volume of procedures performed increased.

On multivariable analysis, a higher trainee caseload correlated with improved technical success rates, a finding consistent with data from previous studies 4 5 6 7. However, the risk for procedure-related complications was no greater when procedures were performed by a trainee than when they were performed by an expert, regardless of the trainee caseload or procedure difficulty. Also, the overall cannulation and technical success rates were similar in the two study groups and in accordance with proposed quality standards for ERCP 16. This finding is extremely important because it confirms that within the confines of a well-structured training program, patients undergoing ERCPs performed by trainee endoscopists are not subjected to any additional risk.

An interesting finding is that the rates of successful cannulation were similar in the trainee group (86.6 %) and the expert group (83 %), with a per-patient successful cannulation rate of 92.3 %. Given the relatively limited number of procedures analyzed and the small number of endoscopists in our study, these results require further validation in randomized controlled trials. However, it seems likely that supervised trainees can achieve the quality standards currently proposed (i. e., a cannulation rate of approximately 85 %).

An important limitation of our study is the lack of randomization of procedures between the trainees and the expert. Although our comparison shows that the two groups were very similar in terms of patient age, diagnoses, and procedure complexity, there was still a high risk for selection bias before the procedures. This theoretical selection bias and other potential confounding factors could also account for the relatively low cannulation rate and technical success rate of the expert endoscopist compared with previously reported data 1 2 3 4. We used multivariable analysis to account for traditional risk factors for complications (e. g., age, bilirubin levels, failed cannulation, malignant strictures) in an attempt to minimize the risk for such bias.

Another potential limitation is the exclusion of cases in which more than one operator was involved in completing the procedure (crossover cases) from the final analysis. In the setting of a training program, it is inevitable that some procedures will have to be completed by experts, but evaluating and attributing the additional risk for the patient is very difficult in such cases because the role of the expert can vary considerably between interventions. In our cohort, only 47 procedures (8.1 %) were excluded from the final analysis, and we did not detect an increased complication rate in these cases, with an overall adverse event rate of 8.5 % and no fatal cases reported in this particular subgroup.

With regard to post-ERCP complications, our data suggest that the risk for complications is no greater in procedures carried out by a trainee than in those carried out by an experienced endoscopist. However, given the number of procedures included in the analysis, we acknowledge that our study may not have been adequately powered to detect potentially significant differences in regard to adverse events with a low prevalence. This issue needs to be explored further, preferably in a multicenter trial setting.

Despite these limitations, we believe that our study provides important evidence toward a better understanding of ERCP and ways to improve teaching. The fact that the procedures carried out by supervised trainees carried no additional risk for the patients while technical success rates remained satisfactory is an important finding that helps shed light on an area where it is sorely needed – procedure outcome and patient safety in endoscopy teaching programs.