Development and validation of a risk prediction model and scoring system for post-endoscopic retrograde cholangiopancreatography pancreatitis.

BACKGROUND: A few models have been proposed for the prediction of the risk of post-endoscopic retrograde cholangiopancreatography pancreatitis (PEP), but many include factors that are not assessed routinely. Herein, we intend to develop and validate a predictive model for the occurrence of PEP.
METHODS: Data of patients who underwent endoscopic retrograde cholangiopancreatography (ERCP) from January 01, 2016 to May 16, 2019 (training set and internal test set) and from May 17, 2019 to December 25, 2019 (external test set) were retrospectively collected. The performance of the model was validated in the two validation cohorts.
RESULTS: A total of 342 patients were included for the external test set, and 47 (13.7%) developed PEP. The variables included in the scoring system were gastrectomy history, high direct bilirubin (DBIL), high albumin (ALB), villous type of papillary orifice, nodular type of papillary orifice, pancreatic guidewire passages, precut sphincterotomy, and high operator experience. A total score >5 indicated high risk. In the external test set, the area under the curve (AUC) was 0.718, the sensitivity was 0.723, and the specificity was 0.676. In the external test set, the probability of PEP was 6.1%, 17.0%, and 37.5% in patients with low (<0), moderate (0-5), and high (>5) risk scores, respectively.
CONCLUSIONS: This study established a scoring system for predicting the risk of PEP using routinely measured clinical variables. Its application in routine work warrants further investigation. 2020 Annals of Translational Medicine. All rights reserved.

Introduction

Endoscopic retrograde cholangiopancreatography (ERCP) is a procedure extensively used in the diagnosis and treatment of pancreatobiliary diseases, although it does entail some risk (1,2). Infection (0.6–5%), hemorrhage (0.3–2%), and perforation (0.1–1.1%) are occasional complications of this treatment, but the most common complication is acute pancreatitis, which occurs in 1.3–11% of cases. Post-ERCP pancreatitis (PEP) is not only the most common complication but also the most feared because of the risk of morbidity and mortality (0.6–1.1%) (3-5). There is still uncertainty about which patients will develop PEP, and a number of chemical, hydrostatic, enzymatic, mechanical, and thermal factors have been speculated to be involved in PEP. Studies found that any manipulation causing edema of papilla and injury of pancreatic duct may result in PEP. Mechanical injury caused by difficult cannulation and thermal injury caused by using electrocautery current during sphincterotomy lead to edema of pancreatic orifice, obstructing outflow of pancreatic juice, thus may induce PEP. Contrast injection into pancreatic duct and mechanical manipulation in the pancreatic duct may cause activation of protease and eventually result in PEP (3,6-11). Patients at high-risk of PEP should be monitored more closely in order to receive timely preventive treatment (12). Because of the high occurrence and high morbidity of PEP (3), there is an urgent need for reliable models that can predict PEP.

Most of the related studies have examined the influence of different risk factors on the occurrence of PEP, and some have established a prediction model for PEP by combining the previous studies of relevant risk factors (13-15). Many of the models have low credibility due to using too few cases or having a lack of validation, while others are too complex to be applied in clinical practice. Thus, a large number of cases were included in this study, and an external test was conducted. A scoring standard for the prediction of PEP was established to facilitate its use in clinical practice (12).

Therefore, the aim of the present study was to develop and validate a predictive model and scoring system for the occurrence of PEP. We present the following article in accordance with the TRIPOD reporting checklist (available at http://dx.doi.org/10.21037/atm-20-5769).

Methods

Study design and patients

For the training and internal test set, patients who underwent ERCP from January 1, 2016, to May 16, 2019, were retrospectively enrolled at Nanjing Drum Tower Hospital Clinical College of Nanjing Medical University. For the external test set, patients were retrospectively enrolled from May 17, 2019, to December 25, 2019, at Nanjing Drum Tower Hospital Clinical College of Nanjing Medical University. The study was conducted in accordance with the Declaration of Helsinki (as revised in 2013). The Medical Ethics Committee of Nanjing Drum Tower Hospital Clinical College of Nanjing Medical University approved this retrospective data-only study (No. 2019-135-01). Written informed consent was obtained from all patients.

All patients who underwent ERCP from January 1, 2016, to May 16, 2019, were included. The exclusion criteria were the following: (I) non-native papilla (including post-sphincterotomy, post-papillectomy, post-papillary balloon dilatation, and post-choledochojejunostomy); (II) failed operation; (III) age <18 years; (IV) body weight <40 kg; (V) unable to provide informed consent; (VI) missing any of the indicators (). The exclusion criteria for the external test set (from May 17, 2019 to December 25, 2019) were the same as those of the training and internal test sets.

Table S1

Missing data

Variables	Missing (n)	Variables	Missing (n)	Variables	Missing (n)
CRP	76	INR	212	PT	212
TT	212	APTT	212	GGT	44
ALT	44	LDH	35	AST	35
UA	51	TC	77	TBA	44
TP	43	TG	77	ALB	43
DBIL	44	ALP	44	CREA	51
GLU	51	Na	46	K	46
N_rate	1	WBC	1	RBC	6
PLT	1	Hb	1	Whether there was postoperative pancreatitis	52
Papillary orifice type	4	Ca	46

Data collection

For the training and internal test sets, patient-related data and procedure-related data were extracted from the medical charts, based on a review of the risk factors of PEP (3).

Patient-related data included sex, age, surgical history, gastrectomy history, cholecystectomy history, drinking history, smoking history, hypertension, diabetes, coronary heart disease, chronic pancreatitis history, acute pancreatitis history, C-reactive protein (CRP), international normalized ratio (INR), prothrombin time (PT), activated partial thromboplastin time (APTT), γ-glutamyl transferase (GGT), alanine aminotransferase (ALT), lactate dehydrogenase (LDH), aspartate aminotransferase (AST), uric acid (UA), total cholesterol (TC), total bile acids (TBA), total proteins (TP), triglycerides (TG), albumin (ALB), direct bilirubin (DBIL), alkaline phosphatase (ALP), creatinine (CREA), glucose (GLU), calcium (Ca), sodium (Na), potassium (K), pancreatic diseases, sphincter of Oddi dysfunction (SOD), hilar bile duct stricture, distal biliary stricture, common bile duct stone, and adenoma of the duodenal papilla.

Procedure-related data included precut sphincterotomy, endoscopic sphincterotomy, pancreatic guidewire passages, biliary stent placement, papillary balloon dilatation, nasobiliary drainage, difficult biliary cannulation (duration of cannulation attempts >10 minutes, and/or >5 attempts), contrast injection to the pancreatic duct, epinephrine injection around the papilla, dilated extrahepatic bile duct, type of papillary orifice, pancreatic duct stenting, and operator experience (high grade: >200 ERCP procedures total and/or >50/year).

For the external test set, preoperative data of blood routine, coagulation, biochemistry, and other related laboratory examinations, along with imaging examinations, the process of the ERCP operation, and operator experience were also extracted from the charts. The patient routinely received indomethacin suppository as per the judgment of the operator. If the patient had new or aggravated epigastric pain within 72 hours, abdominal imaging was routinely performed according to the general situation of the patient, and, if necessary, enzyme inhibition treatment was given.

Definitions

The diagnosis of postoperative pancreatitis was based on the Atlanta criteria (16). At least two of the following elements had to be present: (I) upper abdominal pain suggestive of pancreatitis, usually epigastric pain radiating to the back; (II) serum amylase at least three times the upper limit of normal levels; and (III) computed tomography (CT), magnetic resonance imaging (MRI), or abdominal ultrasound scans suggestive of pancreatitis. In this study, the patients who experienced new-onset or aggravation of epigastric pain within 3 days after ERCP procedure met the diagnostic criteria for PEP. In this study, the serum amylase of each patient was tested at 3, 12, and 24 hours after operation. If the amylase levels were 3 times higher than the upper limit of the normal value (the upper limit of the normal value of serum amylase in this hospital was 110 U/L), the patient was potentially diagnosed with PEP.

Statistical analysis

Because the data of training set and internal test set were extremely unbalanced (PEP:No PEP =1:12.8), we used the upsampling method to randomly divide patients who underwent ERCP from January 1, 2016, to May 16, 2019 into a training set and an internal test set in a ratio of 7:3. Stratified sampling was used to ensure the data distribution of the training set and the internal test set were consistent, and then upsampling and 10-fold cross-validation were performed in the training set; that is, the training set was divided into 10 mutually exclusive subsets of similar size, and each subset was kept as consistent as possible in the data distribution, which was obtained by stratified sampling from the training set. Then, each time the positive and negative samples in the union of 9 subsets were upsampled at a ratio of 1:4 as the training set, the remaining subset was used as the internal test set to select the best model. The continuous data were dichotomized according to the cutoff value of each indicator based on the optimal sensitivity and specificity (17). Categorical data were expressed as numbers, rates, and percentages, and were compared with the Chi-square test or Fisher’s exact test. In the training set, the relevant factors determined by univariate analysis (P<0.15) were included in the multivariate logistic regression analysis (backward stepwise regression selection). A receiver operating characteristics (ROC) curve was used to examine the sensitivity and specificity of the model, and the area under the ROC curve (AUC) was used for discrimination. A score was attributed to each variable based on the β value of the variables. The total score was then associated with the occurrence of PEP. The resulting model was tested in the training set and the internal test set. Finally, the model was validated using the external test set. All data were analyzed using R 3.5.2. P values <0.05 were considered statistically significant.

Results

Characteristics of the patients

Data of 3,987 patients who underwent ERCP from January 2016 to May 2019 were retrospectively collected. After excluding the patients with non-native papilla, failed ERCP, and unavailable operation records, a total of 2,547 patients were placed into the next phase for the extraction of the indicators. Among these 2,547 patients, 504 patients had missing indicators (), and so 2,043 patients were ultimately included in the final analysis. From the 2,043 included patients, 30% were randomly selected to be in the internal test set, and the remaining 70% were placed in the training set (). The frequency of PEP occurrence among the 2,043 included patients was 148 (7.2%). presents the characteristics of the patients. A total of 506 patients who were scheduled to undergo ERCP were retrospectively enrolled from May 17, 2019 to December 25, 2019. Among these, 164 patients were excluded for having non-native papilla. In the end, 342 patients were included for model validation, and 47 (13.7%) had PEP.

Figure 1

Patient flow diagram showing training set, internal test set, and external test set of patients who underwent ERCP. ERCP, endoscopic retrograde cholangiopancreatography.

Table 1

Baseline characteristics in the training and test set

Clinicopathological factors (N, %)	Training set			Internal test set			External test set
	No PEP (N=1,327)	PEP (N=104)		No PEP (N=568)	PEP (N=44)		No PEP (N=295)	PEP (N=47)
Sex
Female	589 (44.4)	56 (53.8)		275 (48.4)	25 (56.8)		132 (44.7)	22 (46.8)
Male	738 (55.6)	48(46.2)		293 (51.6)	19 (43.2)		163 (55.3)	25 (53.2)
Age
≤60	524 (39.5)	51 (49.0)		231 (40.7)	16 (36.4)		122 (41.4)	24 (51.1)
>60	803 (60.5)	53 (51.0)		337 (59.3)	28 (63.6)		173 (58.6)	23 (48.9)
Gastrectomy history
No	1,290 (97.2)	99 (95.2)		556 (97.9)	39 (88.6)		283 (95.9)	46 (97.9)
Yes	37 (2.8)	5 (4.8)		12 (2.1)	5 (11.4)		12 (4.1)	1 (2.1)
Drinking history
No	1,202 (90.6)	94 (90.4)		520 (91.5)	40 (90.9)		261 (88.5)	40 (85.1)
Yes	125 (9.4)	10 (9.6)		48 (8.5)	4 (9.1)		34 (11.5)	7 (14.9)
Smoking history
No	1,154 (87.0)	91 (87.5)		493 (86.8)	40 (90.9)		237 (80.3)	37 (78.7)
Yes	173 (13.0)	13 (12.5)		75 (13.2)	4 (9.1)		58 (19.7)	10 (21.3)
Hypertension
No	875 (65.9)	75 (72.1)		392(69.0)	31 (70.5)		178 (60.3)	38 (80.9)
Yes	452 (34.1)	29 (27.9)		176 (31.0)	13 (29.5)		117 (39.7)	9 (19.1)
Diabetes
No	1,130 (85.2)	88 (84.6)		492 (86.6)	39 (88.6)		239 (81.0)	43 (91.5)
Yes	197 (14.8)	16 (15.4)		76 (13.4)	5 (11.4)		56 (19.0)	4 (8.5)
Coronary heart disease
No	1,274 (96.0)	102 (98.1)		545 (96.0)	43 (97.7)		276 (93.6)	46 (97.9)
Yes	53 (4.0)	2 (1.9)		23 (4.1)	1 (2.3)		19 (6.4)	1 (2.1)
Chronic pancreatitis
No	1,310 (98.7)	104 (100.0)		560 (98.6)	44 (100.0)		291 (98.6)	44 (93.6)
Yes	17 (1.3)	0 (0.0)		8 (1.4)	0 (0.0)		4 (1.4)	3 (6.4)
Acute pancreatitis history
No	1,317 (99.2)	104 (100.0)		562 (98.9)	43 (97.7)		287 (97.3)	45 (95.7)
Yes	10 (0.8)	0 (0.0)		6 (1.1)	1 (2.3)		8 (2.7)	2 (4.3)
DBIL
≤7.4	477 (35.9)	52 (50.0)		206 (36.3)	20 (45.5)		129 (43.7)	26 (55.3)
>7.4	850 (64.1)	52 (50.0)		362 (63.7)	24 (54.5)		166 (56.3)	21 (44.7)
N_rate
≤65	662 (49.9)	66 (63.5)		293 (51.6)	26 (59.1)		147 (49.8)	31 (66.0)
>65	665 (50.1)	38 (36.5)		275 (48.4)	18 (40.9)		148 (50.2)	16 (34.0)
ALP
≤152.6	622 (46.9)	65 (62.5)		261 (46.0)	21 (47.7)		136 (46.1)	25 (53.2)
>152.6	705 (53.1)	39 (37.5)		307 (54.0)	23 (52.3)		159 (53.9)	22 (46.8)
WBC
≤5.5	606 (45.7)	64 (61.5)		262 (46.1)	20 (45.5)		146 (49.5)	27 (57.4)
>5.5	721 (54.3)	40 (38.5)		306 (53.9)	24 (54.5)		149 (50.5)	20 (42.6)
TBIL
≤21.8	619 (46.6)	61 (58.7)		272 (47.9)	24 (54.5)		152 (51.5)	29 (61.7)
>21.8	708 (53.4)	43 (41.3)		296 (52.1)	20 (45.5)		143 (48.5)	18 (38.3)
ALB
≤37.6	700 (52.8)	41 (39.4)		301 (53.0)	17 (38.6)		154 (52.2)	16 (34.0)
>37.6	627 (47.2)	63 (60.6)		267 (47.0)	27 (61.4)		141 (47.8)	31 (66.0)
Pancreatic disease
No	1,272 (95.9)	97 (93.3)		539 (94.9)	41 (93.2)		284 (96.3)	40 (85.1)
Yes	55 (4.1)	7 (6.7)		29 (5.1)	3 (6.8)		11 (3.7)	7 (14.9)
Common bile duct stone
No	314 (23.7)	35 (33.7)		152 (26.8)	19 (43.2)		92 (31.2)	19 (40.4)
Yes	1,013 (76.3)	69 (66.3)		416 (73.2)	25 (56.8)		203 (68.8)	28 (59.6)
Hilar bile duct stricture
No	1,213 (91.4)	91 (87.5)		501 (88.2)	37 (84.1)		263 (89.2)	43 (91.5)
Yes	114 (8.6)	15 (12.5)		67 (11.8)	7 (15.9)		32 (10.8)	4 (8.5)
Distal biliary stricture
No	1,198 (90.3)	89 (85.6)		519 (91.4)	38 (86.4)		259 (87.8)	42 (89.4)
Yes	129 (9.7)	15 (14.4)		49 (8.6)	6 (13.6)		36 (12.2)	5 (10.6)
SOD
No	1,323 (99.7)	104 (100.0)		567 (99.8)	43 (97.7)		293 (99.3)	47 (100.0)
Yes	4 (0.3)	0 (0.0)		1 (0.18)	1(2.3)		2 (0.7)	0 (0.0)
Type of papillary orifice
Others	154 (11.6)	7 (6.7)		64 (11.3)	1 (2.3)		171 (58.0)	21 (44.7)
Villous type	1,056 (79.6)	82 (78.8)		450 (79.2)	36 (81.8)		124 (42.0)	26 (55.3)
Granular type	117 (8.8)	15 (14.4)		54 (9.5)	7 (15.9)
Precut sphincterotomy
No	1,230 (92.7)	87 (83.7)		522 (91.9)	40 (90.9)		275 (93.2)	40 (85.1)
Yes	97 (7.3)	17 (16.3)		46 (8.1)	4 (9.1)		20 (6.8)	7 (14.9)
Endoscopic sphincterotomy
No	344 (25.9)	27 (26.0)		147 (25.9)	12 (27.3)		66 (22.4)	8 (17.0)
Yes	983 (74.1)	77 (74.0)		421 (74.1)	32 (72.7)		229 (77.6)	39 (83.0)
Pancreatic guidewire passages
No	1,006 (75.8)	56 (53.8)		418 (73.6)	21 (47.7)		178 (60.3)	15 (31.9)
Yes	321 (24.2)	48 (46.2)		150 (26.4)	23 (52.3)		117 (39.7)	32 (68.1)
Biliary stent placement
No	978 (73.7)	67 (64.4)		413 (72.7)	22 (50)		250 (84.7)	42 (89.4)
Yes	349 (26.3)	37 (35.6)		155 (27.3)	22 (50)		45 (15.3)	5 (10.6)
Papillary balloon dilation
No	933 (70.3)	77 (74.0)		427 (75.2)	30 (68.2)		193 (65.4)	32 (68.1)
Yes	394 (29.7)	26 (26.0)		141 (24.8)	14 (31.8)		102 (34.6)	15 (31.9)
Nasobiliary drainage
No	306 (23.1)	31 (29.8)		124 (21.8)	14 (31.8)		68 (23.1)	18 (38.3)
Yes	1,021 (76.9)	73 (70.2)		444 (78.2)	30 (68.2)		227 (76.9)	29 (61.7)
Difficult biliary cannulation
No	1,241 (93.5)	95 (91.3)		521 (91.7)	38 (86.4)		261 (88.5)	35 (74.5)
Yes	86 (6.5)	9 (8.7)		47 (8.27)	6 (13.6)		34 (11.5)	12 (25.5)
Contrast injection to the pancreatic duct
No	1,185 (89.3)	88 (84.6)		518 (91.2)	39 (88.6)		254 (86.1)	32 (68.1)
Yes	142 (10.7)	16 (15.4)		50 (8.8)	5 (11.4)		41 (13.9)	15 (31.9)
Cholangiectasis
No	634 (47.8)	60 (57.7)		279 (49.1)	28 (63.6)		62 (21.0)	13 (27.7)
Yes	693 (52.2)	44 (42.3)		289 (50.9)	16 (36.4)		233 (79.0)	34 (72.3)
Pancreatic duct stenting
No	1,047 (78.9)	63 (60.6)		434 (76.4)	24 (54.5)		212 (71.9)	22 (46.8)
Yes	280 (21.1)	41 (39.4)		134 (23.6)	20 (45.5)		83 (28.1)	25 (53.2)
Operator experience
Low	182 (13.7)	18 (17.3)		72 (12.7)	4 (9.1)		136 (46.1)	25 (53.2)
High	1,145 (86.3)	86 (82.7)		496 (87.3)	40 (90.9)		159 (53.9)	22 (46.8)

PEP, post-ERCP pancreatitis; DBIL, direct bilirubin; ALP, alkaline phosphatase; WBC, white blood cells; TBIL, total bilirubin; ALB, albumin; SOD, sphincter of Oddi dysfunction.

Univariate and multivariate analysis of the factors associated with PEP

Univariate analysis was performed using the training set to determine the factors associated with PEP (), and the identified factors were entered into the multivariate model (). Gastrectomy history (OR =3.136, 95% CI: 1.843–5.335, P<0.001), DBIL (OR =0.557, 95% CI: 0.430–0.722, P<0.001), ALB (OR =1.358, 95% CI: 1.048–1.759, P=0.021), common bile duct stone (OR =0.738, 95% CI: 0.562–0.967, P=0.027), villous type of papillary orifice (OR =2.119, 95% CI: 1.514–3.196, P<0.001), nodular type of papillary orifice (OR =4.477, 95% CI: 1.370–2.784, P<0.001), precut sphincterotomy (OR =1.953, 95% CI: 1.370–2.784, P<0.001), and high operator experience (OR =0.691, 95% CI: 0.507–0.941, P=0.019) were independently associated with the risk of PEP (). Among the above-mentioned variables, high DBIL, common bile duct stone, and high operator experience were protective factors, while the other variables were risk factors. A predictive model was then constructed as follows: Logit (P = PEP) =−2.27 + 1.14 × gastrectomy history – 0.58 × DBIL + 0.31 × ALB + 0.67 × precut sphincterotomy + 0.81 × pancreatic guidewire passages −0.37 × high operator experience −0.30 × common bile duct stone + 1.50 × nodular type of papillary orifice (or 0.79 × villous type of papillary orifice). Scores were attributed to each variable ().

Table S2

Univariate regression analysis

Factor (N, %)	PEP (N=104)	No PEP (N=1,327)	P value
Sex (female)	56 (53.8)	589 (44.4)	0.077
Age (<60)	51 (49.0)	524 (39.5)	0.070
Gastrectomy history	5 (4.81)	37 (2.79)	0.138
Drinking history	10 (9.62)	125 (9.42)	1.000
Smoking history	13 (12.5)	173 (13.0)	0.966
Hypertension	29 (27.9)	452 (34.1)	0.239
Diabetes	16 (15.4)	197 (14.8)	0.995
Coronary heart diseases	2 (1.92)	53 (3.99)	0.428
Chronic pancreatitis	0 (0)	17 (1.28)	0.489
Acute pancreatitis history	0 (0)	10 (0.754)	0.781
DBIL (<7.4)	52 (50.0)	477 (35.9)	0.005
N_rate (<65)	66 (63.5)	662 (49.9)	0.010
ALP (<152.6)	65 (62.5)	622 (46.9)	0.002
WBC (<5.5)	64 (61.5)	606 (45.7)	0.002
TBIL (<21.8)	61 (58.7)	619 (46.6)	0.023
ALB (<37.6)	41 (39.4)	700 (52.8)	0.011
Pancreatic diseases	7 (6.73)	55 (4.14)	0.319
Common bile duct stone	69 (66.3)	1,013 (76.3)	0.030
Hilar bile duct stricture	13 (12.5)	114 (8.59)	0.242
Distal biliary stricture	15 (14.4)	129 (9.72)	0.172
SOD	0 (0)	4 (0.301)	1
Papillary orifice (villous type)	82 (78.8)	1,056 (79.6)	0.069
Precut sphincterotomy	17 (16.3)	97 (7.31)	0.002
Endoscopic sphincterotomy	77 (74.0)	983 (74.1)	1
Pancreatic guidewire passages	48 (46.2)	321 (24.2)	0.007
Biliary stent placement	37 (35.6)	349 (26.3)	0.052
Papillary balloon dilatation	27 (26.0)	394 (29.7)	0.489
Nasobiliary drainage	73 (70.2)	1,021 (76.9)	0.149
Difficult biliary cannulation	9 (8.65)	86 (6.48)	0.514
Contrast injection to the pancreatic duct	16 (15.4)	142 (10.7)	0.192
Cholangiectasis	44 (42.3)	693 (52.2)	0.065
Pancreatic duct stenting	41 (39.4)	280 (21.1)	<0.001
Operator experience (high)	86 (82.7)	1,145 (86.3)	0.118

PEP, post-ERCP pancreatitis; DBIL, direct bilirubin; ALP, alkaline phosphatase; WBC, white blood cells; TBIL, total bilirubin; ALB, albumin; SOD, sphincter of Oddi dysfunction.

Table 2

Multivariate regression model

Variables	OR	95% CI	β	P	Cutoff value	Scoring
Patient-related
Gastrectomy history	3.136	1.843–5.335	1.14	<0.001		5.5
DBIL	0.557	0.430–0.722	−0.58	<0.001	>7.4	−3
ALB	1.358	1.048–1.759	0.31	0.021	>37.6	1.5
Common bile duct stone	0.738	0.562–0.967	−0.30	0.027		−1.5
Procedure-related
Papillary orifice (villous type)	2.119	1.514–3.196	0.79	<0.001		4
Papillary orifice (Nodular type)	4.477	2.755–7.275	1.50	<0.001		7.5
Pancreatic guidewire passages	2.245	1.743–2.891	0.81	<0.001		4
Precut sphincterotomy	1.953	1.370–2.784	0.67	<0.001		3
Operator-related
High operator experience	0.691	0.507–0.941	−0.37	0.019		−2

OR, odds ratio; CI, confidence interval; DBIL, direct bilirubin; ALB, albumin.

Figure 2

Forrest plot for the factors involved in the risk of post-ERCP pancreatitis. ERCP, endoscopic retrograde cholangiopancreatography.

Table 3

Scoring system table

Risk score quantiles	N	PEP, n (%)	OR (95% CI)	P
Training set
Range (−7.5, 12)
Score <0	897	38 (4.20)	Ref	–
0≤ Score ≤5	426	48 (11.3)	2.87 (1.85, 4.48)	<0.001
Score >5	108	18 (16.7)	4.52 (2.43, 8.15)	<0.001
Internal test set
Range (−7.5, 9)
Score <0	377	11 (2.90)	Ref	–
0≤ Score ≤5	190	23 (12.10)	4.58 (2.23,9.97)	<0.001
Score >5	45	10 (22.2)	9.51 (3.72,24.14)	<0.001
External test set
Range (−7.5,9.5)
Score <0	147	9 (6.10)	Ref	–
0≤ Score ≤5	171	29 (17.0)	3.13 (1.48,7.24)	<0.001
Score >5	24	9 (37.5)	9.20 (3.15,27.29)	<0.001

PEP, post-ERCP pancreatitis; OR, odds ratio; CI, confidence interval.

ROC analysis

The model in the training set had an AUC of 0.793, a sensitivity of 0.727, and a specificity of 0.797; the model in the internal test set had an AUC of 0.725, a sensitivity of 0.705, and a specificity of 0.700; the model in the external test set had an AUC of 0.718, a sensitivity of 0.723, and a specificity of 0.676 ().

Figure 3

ROC curve for risk of post-ERCP pancreatitis in the training set, internal test set, and external test set. ROC, receiver operating characteristics; ERCP, endoscopic retrograde cholangiopancreatography.

Predictive ability of the model

The variables included in the scoring system were gastrectomy history (score =5.5), high DBIL (DBIL >7.4 µmol/L; score =−3), high ALB (ALB >37.6 g/L; score =1.5), common bile duct stone (score =−1.5), villous type of papillary orifice (score =4), nodular type of papillary orifice (score =7.5), pancreatic guidewire passages (score =4), precut sphincterotomy (score =3), and high operator experience (score =−2) (). Scores of 0 and 5 were determined as the optimal cutoff points for low risk (score ≤0) and high risk (score >5), with moderate risk located in between (score between 1 and 5). In the training set, compared with the low-risk group, the occurrence of PEP was increased in the moderate-risk (OR =2.87, 95% CI: 1.85­–4.48, P<0.0001) and high-risk (OR =4.52, 95% CI: 2.43–8.15, P<0.0001) groups. In the internal test set, the moderate-risk (OR =4.58, 95% CI: 2.23–9.97, P<0.0001) and high-risk (OR =9.51, 95% CI: 3.72–24.14, P<0.0001) groups had an increased occurrence of PEP compared with the low-risk group. In the external test set, compared with the low-risk group, the occurrence of PEP was also increased in the moderate-risk (OR =3.13, 95% CI: 1.48–7.24, P<0.0001) and high-risk (OR =9.20, 95% CI: 3.15–27.29, P<0.0001) groups ().

Probability of PEP

presents the probability of PEP. The probability of PEP correlated with the degree of risk in the groups. In the training set, among patients at low, moderate, and high risk of PEP, the actual occurrence of PEP was 4.2%, 11.3%, and 16.7%, respectively. In the internal test set, among patients at low, moderate, and high risk of PEP, the actual occurrence of PEP was 2.9%, 12.1%, and 22.2%, respectively; in the external test set, those frequencies were 6.1%, 17.0%, and 37.5%, respectively.

Figure 4

Histogram for the risk of PEP. According to the scoring system, a histogram of the proportions of the low-, medium-, and high-risk populations were drawn. PEP, post-ERCP pancreatitis; ERCP, endoscopic retrograde cholangiopancreatography.

Discussion

A few models have been proposed for the prediction of the risk of PEP (13-15). Many of the models have low credibility due to using a low number of cases or from a lack of validation, while others are too complex for clinical practice. Therefore, this study aimed to develop and validate a predictive model and scoring system for the occurrence of PEP using comprehensive patient, operation, and operator variables. This study successfully established a predictive model for PEP using nine variables that yielded an AUC of 0.718, a sensitivity of 0.723, and a specificity of 0.676 in the external test set of patients.

The pathogenesis of PEP is complex and is probably the result of the combination of chemical, hydrostatic, enzymatic, mechanical, and thermal insults that occur during ERCP (3). Previous studies identified a number of factors that could be associated with PEP. Two reviews showed that patient-related factors (young age, female sex, possible sphincter of Oddi dysfunction, history of recurrent pancreatitis, history of PEP, and normal TBIL and DBIL), procedure-related factors (ampullectomy, multiple pancreatic duct injections, difficult cannulation, pancreatic sphincterotomy, precut sphincterotomy, balloon dilation, and minor papilla sphincterotomy), and operator-related factors (inadequate training, low experience, and trainees involved in the procedure) were associated with an increased risk of PEP (3,11). Iorgulescu et al. (10) showed that difficult sphincterotomy, precut sphincterotomy, failure of deobstruction, pancreatic sphincterotomy, multiple pancreatic duct injections, sphincter of Oddi dysfunction, and the absence of chronic pancreatitis changes were risk factors for PEP. On the basis of the previous studies, this study also analyzed multiple factors according to patient-related, operation-related, and operator-related categories to determine the factors associated with PEP and construct a prediction model.

The present study identified gastrectomy history, high ALB, villous type of papillary orifice, nodular type of papillary orifice, pancreatic guidewire passages, and precut sphincterotomy as being risk factors for PEP, and high DBIL, common bile duct stone, and high operator experience as being protective factors. Because of the surgically altered anatomy, ERCP is more difficult in patients with previous gastrectomy compared with patients with the native anatomy. The papillary area in the second part of the duodenum can only be reached through the afferent loop. As a result, the papilla of Vater appears upside-down compared with its orientation during standard ERCP, making ECRP more difficult (18). High serum ALB levels might be an indicator of dehydration, and aggressive hydration has been reported as a means to prevent PEP. Previous studies suggest that aggressive hydration with lactated Ringer’s solution may alleviate activation of zymogen and inflammation response, and improve perfusion of pancreas parenchyma (19). Our study suggests that high serum ALB levels are a risk factor for PEP, but there is currently no relevant literature suggesting a relationship between ALB and PEP. Thus, further multicenter studies are needed for confirmation, and basic research is required to clarify its underlying mechanism. The type of papillary orifice of Vater will influence the difficulty of cannulation (20,21). A precut sphincterotomy is usually applied after difficult cannulation, which will increase the success rate of ERCP but will increase the risk of PEP (22). Previous studies have found that the guidewire entering the pancreatic duct may be a risk factor for PEP (23,24), and our research also confirms this. A pancreatic duct guidewire entering too deep may cause a certain degree of damage to the pancreatic duct, thereby promoting the occurrence of PEP (25-27). On the other hand, high DBIL, common bile duct stone, and high operator experience were identified as protective factors for PEP. The experience of the operator is a well-recognized factor in avoiding iatrogenic injury during ERCP and the occurrence of PEP (3,8) Furthermore, one study revealed that patients with obstructive indications of ERCP, such as common bile duct stone and high DBIL levels, tolerated ERCP better than those without such indications (28).

Previous researchers have tried to propose models based on a number of factors for the prediction of the risk of PEP (8,13,14). The major limitations of these models include a low credibility due to too small sample size, a lack of validation, and impracticable complexity. DiMagno et al. (13) constructed two models (pre-ERCP and post-ERCP) based on protective (current smoking, chronic liver disease-biliary, and chronic liver disease-transplant/hepatectomy complications) and predictive (younger age, possible sphincter of Oddi dysfunction, pancreatic sphincterotomy, and moderate-difficult cannulation) factors, and achieved an AUC of 0.73–0.74. Meanwhile, Coté et al. (14) showed that severe comorbidities, high pain, longer procedure, sphincter of Oddi performance, and greater use of opiates and anxiolytics were independently associated with PEP and could be used to construct a model with an AUC of 0.78–0.83. Cheng et al. (8) constructed a multivariate risk model based on minor papilla sphincterotomy, possible sphincter of Oddi dysfunction, history of PEP, younger age, multiple contrast injections, and trainee involvement. However, these models are more or less flawed. For example, opioid and anxiolytic use and minor papilla cannulation occur in only a small proportion of the ERCP population, and thus have limited clinical significance. In the present study, gastrectomy history, high DBIL, high ALB, common bile duct stones, villous type of papillary orifice, nodular type of papillary orifice, pancreatic guidewire passages, precut sphincterotomy, and high operator experience were independently associated with PEP and included in the model, which achieved AUCs of 0.718–0.793, a sensitivity of 0.705­–0.727, and a specificity of 0.676–0.797. Using clinical variables that are routinely assessed/observed in the clinical setting, the model could identify three risk levels with good discrimination among the three groups. Some factors identified in the present study and used to construct the model were also included in the previous models (8,13,14), but not all. Nevertheless, as described above, all included factors had a biological basis as a possible explanation for their association with the occurrence of PEP. Differences among models can be due to a number of factors, including the study population and hospital setting. Additional studies are required to validate the model in a variety of populations and to improve upon its performance.

Some limitations to this study should also be addressed. All patients were from a single-center study, and the sample was relatively small, which might have introduced some bias based on local practice. A multicenter study is required to validate the model and refine its prediction ability. Moreover, a substantial number of patients had to be excluded because of missing data, but only patients with complete datasets were included. Additional studies with larger sample sizes or prospective studies are necessary to reduce these biases.

Conclusions

This study successfully established and validated a predictive model and scoring system for PEP based on variables that are routinely assessed or observed during the management of patients undergoing ERCP. The model stratified risk into three levels and achieved AUCs of 0.718–0.793, a sensitivity of 0.705–0.727, and a specificity of 0.676–0.797. This model may become useful model for the risk prediction of PEP, but further validation in routine practice is necessary.

The article’s supplementary files as