A novel machine learning model and a public online prediction platform for prediction of post-ERCP-cholecystitis (PEC).

Background: Endoscopic retrograde cholangiopancreatography (ERCP) is an established treatment for common bile duct (CBD) stones. Post- ERCP cholecystitis (PEC) is a known complication of such procedure and there are no effective models and clinical applicable tools for PEC prediction.
Methods: A random forest (RF) machine learning model was developed to predict PEC. Eligible patients at The First Hospital of Lanzhou University in China with common bile duct (CBD) stones and gallbladders in-situ were enrolled from 2010 to 2019. Logistic regression analysis was used to compare the predictive discrimination and accuracy values based on receiver operation characteristics (ROC) curve and decision and clinical impact curve. The RF model was further validated by another 117 patients. This study was registered with ClinicalTrials.gov, NCT04234126. Findings: A total of 1117 patients were enrolled (90 PEC, 8.06%) to build the predictive model for PEC. The RF method identified white blood cell (WBC) count, endoscopic papillary balloon dilatation (EPBD), increase in WBC, residual CBD stones after ERCP, serum amylase levels, and mechanical lithotripsy as the top six predictive factors and has a sensitivity of 0.822, specificity of 0.853 and accuracy of 0.855, with the area under curve (AUC) value of 0.890. A separate logistic regression prediction model was built with sensitivity, specificity, and AUC of 0.811, 0.791, and 0.864, respectively. An additional 117 patients (11 PEC, 9.40%) were used to validate the RF model, with an AUC of 0.889 compared to an AUC of 0.884 with the logistic regression model. Interpretation: The results suggest that the proposed RF model based on the top six PEC risk factors could be a promising tool to predict the occurrence of PEC.

Evidence before this study

Post-ERCP cholecystitis (PEC) is a severe complication of routine Endoscopic retrograde cholangiopancreatography (ERCP) therapy and its potential risk factors remain unclear. We searched PubMed and the Cochrane Library for peer reviewed articles published in any language up to Oct 1, 2021, using the search terms "Post-ERCP-cholecystitis," "PEC" or "ERCP complication". We could not find any effective models and clinical application tools for PEC prediction.

Added value of this study

To the best of our knowledge, this is the first applicable PEC model based on random forest (RF) compared with a traditional logistic regression model using patients' baseline clinical features, preoperative laboratory tests, imaging results, ERCP procedures, and postoperative laboratory parameters to predict PEC. For the application, we established an online user-friendly platform which could help endoscopists and patients identify risk probability to prevent PEC.

Implications of all evidence available

PEC is a serious but unpredictable complication in routine ERCP therapy, which is a hindrance for ERCP. Our RF model firstly built a PEC prediction calculator by machine learning and an online platform with good predictive ability.

Alt-text: Unlabelled box

Introduction

Endoscopic retrograde cholangiopancreatography (ERCP) is an established treatment for common bile duct (CBD) stones. For patients with gallbladders in-situ, post-ERCP cholecystitis (PEC) is a serious complication. The diagnosis and treatment of PEC are often delayed because of its nonspecific symptoms which can be easily mistaken for acute cholangitis or acute pancreatitis. PEC is a complicated adverse event and can progress quickly, and its management is more complex than the routine therapy for conventional cholecystitis. Because PEC is difficult to predict, most patients require emergency cholecystectomy or percutaneous transhepatic gallbladder drainage (PTGBD). There are currently no good methods to identify risk factors for PEC. Therefore, effective predictive models are needed to predict and manage this serious complication. The use of machine learning methods has improved artificial intelligence and has been used in clinical predictions., We proposed a random forest (RF) model by utilizing patients' baseline clinical parameters, preoperative laboratory tests, imaging results, ERCP procedures, and postoperative laboratory data to predict PEC.

Methods

This retrospective study was conducted in the surgical endoscopy center of The First Hospital of Lanzhou University in China. All patients signed informed consent before ERCP, the study was in accordance with the Declaration of Helsinki, and was registered with Clinicaltrial.gov (NCT04234126). The study was approved by the Ethics Committee of The First Hospital of Lanzhou University (LDYYLL2021-257). The study adheres to STROBE guidelines.

Patients enrollment

Between January 2010 and December 2019, patients (age between 18 and 80 years) with CBD stones and gallbladder in-situ who underwent therapeutic ERCP were enrolled. Exclusion criteria included: current acute cholecystitis, cholecystectomy immediately after ERCP, cases with missing data, or a previous history of ERCP.

A total of 2282 eligible patients who underwent ERCP were screened. Among these, 642 patients who underwent immediate cholecystectomy after ERCP, 324 patients with a previous history of therapeutic ERCP, 154 patients with intercurrent acute cholecystitis, and 35 patients with insufficient data were excluded. And 10 patients were omited in data analysis. Finally, 1117 eligible patients were enrolled in the study to build model. Subsequently, a total of 117 eligible patients (11 PEC, 9.40%) from January 2020 to May 2021 were recruited to validate the predictive value of the model.

Flowchart of model patients enrollment. According to the enrollment and exclusion criteria. All 2282 eligible patients from January 2010 to December 2019 were enrolled firstly, exclusion including 642 patients who underwent immediate cholecystectomy after ERCP, 324 patients with history of ERCP, 154 patients with intercurrent acute cholecystitis and 35 patients with insufficient data and10 patients were omited in data analysis. Finally, 1117 eligible patients were enrolled to build RF model. For validation, 117 eligible patients from January 2020 to May 2021 were recruited to validate the predictive value.

Diagnostic criteria for PEC

PEC is an uncommon complication and often mistaken for acute cholangitis because of similar symptoms. PEC typically presents as acute cholecystitis within one or two weeks of an ERCP procedure. However, there are cases of acute exacerbation of delayed cholecystitis 3 months after discharge from the hospital. According to the 2018 Tokyo Guidelines for acute cholecystitis, PEC may present with the following: (A) local signs of inflammation, including (1) right upper abdominal mass/pain/tenderness and (2) positive Murphy's sign; (B) systemic signs of inflammation, including (1) fever, (2) elevated C reactive protein (CRP) levels, and (3) elevated white blood cell (WBC) counts; and (C) imaging features of acute cholecystitis on abdominal ultrasonography, computerized tomography (CT) or magnetic resonance imaging (MRI) scans. A definitive diagnosis of acute cholecystitis will include at least one item in A + at least one item in B + C. (Severity grading of PEC in Supplementary Table 1)

Data collection

More than 54 parameters were included and evaluated in the model: 1. Patients' basic characteristics, including age, sex, and history of hypertension, diabetes, and pancreatitis 2. Preoperative and postoperative laboratory tests, including WBC and neutrophil ratio (N%), and levels of serum amylase, total cholesterol, triglyceride, total bilirubin, direct and indirect bilirubin, serum alkaline phosphatase (ALP), alanine aminotransferase (ALT), aspartate aminotransferase (AST), and gamma-glutamyl transpeptidase (γ-GT); 3. Imaging parameters include abdominal ultrasonography and MRI scans, with the diagnoses of CBD stones, gallbladder stones, gallbladder stone size, gallbladder thickness and opacification, and bile duct diameters before ERCP; 4. Documentation of ERCP procedures, including endoscopic sphincterotomy (EST), length of the EST, basket stone extraction or balloon stone extraction, mechanical lithotripsy, visualization of the cystic duct during ERCP, endoscopic papillary balloon dilatation (EPBD), endoscopic retrograde biliary drainage (ERBD) with stent insertion and the length and size of the stent, and endoscopic nasociliary drainage (ENBD). All of the data were collected and analyzed.,

Statistical analysis

Machine learning method

In this study, the modeling process was implemented through the sci-kit-learn library (version 0.19.2) in Python (version 3.7.1). Tenfold cross-validation and external test set validation were both employed to validate the reliability of the model. The training set had 930 cases (75 PEC, 8.06%) and the cross-validation set had 187 cases (15 PEC, 8.02%). The model building and optimization work were divided into three stepwise phases. In the initial phase, the classification models with all features were built. The evaluation criteria of the 10-fold cross-validation were used as the score to optimize the hyperparameters, which were the number of trees (n_estimators), the number of features to consider when looking for the best split (max_features), the maximum depth of the trees (max_depth), and the weights of the classes (class_weight). According to the optimized model, the importance ranking of all features was calculated. During the second phase, according to the ranking of the features, the components used for modeling were added one by one, and the evaluation criteria of the 10-fold cross-validation of these models were calculated. To reduce the risk of overfitting the model to improve generalizability, it was expected that an acceptable model with fewer features would be established.

In the RF model, all samples were randomly divided into a training set and a validation set at a ratio of 5:1. To evaluate the reliability of the models, five commonly used evaluation criteria were used to compare the two methods in the receiver operating characteristic (ROC) curve, including the sensitivity, specificity, accuracy, and the area under the curve (AUC), the prediction accuracy was tested by decision curve and clinical impact curve.

Logistic regression model method

All patients data were analyzed with SPSS v.22.0 (IBM, Armonk, New York, USA). Patients were divided into two groups based on the occurrence of PEC. Data with a normal distribution of the variables are expressed as the mean ± standard deviation. Frequencies (percentages) were used to describe the classification of variables. Univariate and multivariate logistic regressions classified the risk factors for PEC. The regressions also used either the chi-square test or the Student's t-test. All p-values below 0.05 were considered statistically significant. Adjusted outcome ratios (ORs) and corresponding 95% confidence intervals (95% CIs) were calculated. The multivariate results were used to develop a logistic prediction model. The prediction value was calculated by the AUC of ROC curve with Hosmer-Lemeshow test, decision curve and clinical impact curve.,

External validation analysis

A comparison was used to validate the accuracy of the RF prediction value. We used the six top-ranking risk factors, to calculate the sensitivity, specificity, accuracy, and AUC to validate the RF model. In addition, prediction accuracy was tested by decision curve and clinical impact curve to validate the model. Finally, an internet prediction platform will be established to apply the model for public.

Role of the funding source

The funders had no involvement in study design, data collection, data analysis, interpretation of findings, the writing of this paper, or the decision to submit the paper for publication. There was no commercial support. The corresponding author (SYL, JQY, and WBM) had full access to all the data in the study and had final responsibility for the decision to submit for publication.

Results

Patient characteristics

A total of 1117 ERCP patients with gallbladder in-situ between January 2010 and December 2019 were enrolled. All patients’ demographics were analyzed before modeling. No significant differences were found in age, sex, hypertension, multiple CBD stones, the diameter of the bile duct (mm), bile duct dilation, duodenal diverticulum, or the size of CBD stones before ERCP. However, patients with acute pancreatitis, diabetes, gallbladder stones, multiple gallbladder stones, and gallbladder opacification before ERCP had a higher rate of PEC (Table 1).

Table 1

All patients’ baseline clinical features analysis.

	PEC(N = 90)	Non-PEC(N = 1027)	χ2/Z	P
Age (year)	59.72 ± 17.52	58.66 ± 17.19	1.004	0.575
Sex (male)	42(46.7%)	549(53.5%)	0.001	0.534
Hypertension	20(22.3%)	201(19.6%)	0.366	0.314
Diabetes	13(14.5%)	69(6.8%)	7.261	0.011
Acute pancreatitis	22(24.5%)	109(10.7%)	15.291	0.001
Gallbladder status
Gallbladder stones	86(95.6%)	814(79.3%)	14.025	0.001
Gallbladder opacification	78(86.7%)	445(44.5%)	62.415	0.000
Multiple stones	70(60%)	653(63.58%)	7.303	0.007
Gallbladder thickness ≥4 mm	71(78.89%)	728(70.89%)	2.602	0.107
The size of gallbladder stones(mm)	8.694 ± 0.906	9.201 ± 0.589	4.630	0.603
CBD stones
Multiple CBD stones	64(71.12%)	802(78.09%)	2.314	0.128
Diameter of bile duct(mm)	9.704 ± 3.670	10.252 ± 3.500	1.002	0.162
Bile duct expansion	60(66.67%)	582(56.67%)	3.383	0.066
Duodenal diverticulum	25(27.78%)	230(23.40%)	1.361	0.234
The size of CBD stones(mm)	7.78 ± 2.176	7.49 ± 2.354	3.681	0.258

NOTE. CBD, common bile duct.

Machine learning model by RF

All parameters were analyzed as shown in RF flowchart (Figure 2), and the model was finally built. Six elements were selected according to their relevance ranking, including WBC count, EPBD, the increase in WBC, residual CBD stones after ERCP, serum amylase levels, and mechanical lithotripsy. (Supplementary Table 2) For current data, the calibration recognition in the ROC curve (Figure 3) with sensitivity, specificity, accuracy, and AUC of the interactive test data set results were 0.822, 0.853, 0.855, and 0.890, respectively, and the test set results were 0.734, 0.838, 0.829, and 0.817, the decision curve (Figure 4) and clinical impact curve (Figure 5) showed the accuracy. Compared with other endoscopy models, this machine learning model is more practical in clinical therapy (Supplementary Table 3). In the final phase, the selected significant features were built, and the four hyperparameters were optimized again. For the final optimized model, the four hyperparameters were 700, 3, 43, and 1:5.55. Therefore, the recognition ability is relatively stable compared to other current analyses, and the model has a good generalize ability.16, 17, 18, 19, 20

Figure 2

Flowchart of RF model.

The calculating flowchart of machine learning, performance of different parameters with different features by decision trees in the training set. With the increase of the features in the random forest by building random bagging decision trees, the prediction parameters of the model also get changed. When the top-ranking 6 risk factors are included (the dotted line), it proves that the specificity and sensitivity of the model is the best result.

Figure 3

ROC curves for RF and logistic model.

a (blue): ROC curve for logistic regression model calibration test, risk factors including WBC, serum amylase levels, gallbladder stones, gallbladder opacification, ERBD, mechanical lithotripsy, EPBD and residual CBD stones after ERCP, the sensitivity, specificity and AUC were 0.811 and 0.791 and 0.864.

b (green): ROC curve for RF model calibration test by predictive value in RF model by decision trees, factors including WBC count, EPBD, the increase in WBC, residual CBD stones after ERCP, serum amylase levels, and mechanical lithotripsy.The sensitivity, specificity, accuracy, and AUC of the interactive test data set results were 0.822, 0.853, 0.855, and 0.890.

Figure 4

The decision curve analysis of the RF model.

In the figure, the red curve represents the predicted performance of the RF model respectively. In addition, there are two lines, which represent two extreme cases. The gray vertical line represents the hypothesis that all patients have PEC; the black horizontal line represents the hypothesis that no PEC occurs.The curve showed that when the PEC probability was between 0.1 and 0.9 in the training set. PEC could be discriminated when using this RF predictive model to make clinical decisions. Within reasonable threshold probabilities, the predictive model by whole 6 top ranking features achieves a higher benefit.

Figure 5

The clinical impact curve of the RF model.

The clinical impact curve analysis showed the clinical predictive efficacy, the red curve (Number high risk) represents the number of people classified as positive (high risk) by the RF model at each threshold probability; the blue curve (Number high risk with outcome) is the number of true positives at each threshold probability. When the threshold probability is greater than 75% of the predicted score probability value, the RF model determines that the prediction accuracy in the training set is highly matched with the actual PEC population, which confirms that the RF model has a very high clinical efficiency.

Prediction risk factors and model for PEC by logistic analysis

To evaluate the predictive performance of the RF model, univariate and multivariate logistic regression model was built in all cases. A total of 43 variables, including ERCP-related procedures and laboratory data collected during ERCP were assessed. Logistic regression analysis found that WBC (OR=1.153; 95% CI: 1.036-1.282; p = 0.009), serum amylase levels (OR=1.001; 95% CI: 1.000–1.001; p = 0.009), gallbladder stones (OR=10.191; 95%CI: 2.275-45.649; p = 0.002), gallbladder opacification (OR=9.688; 95%CI: 2.833–33.125; p = 0.000), ERBD (OR=2.055; 95% CI: 1.146–3.685; p = 0.016), mechanical lithotripsy (OR=2.294; 95% CI: 1.293–4.072; p = 0.005), EPBD (OR=3.634; 95% CI: 2.186–6.041; p = 0.000), and residual CBD stones after ERCP (OR=2.491; 95% CI: 1.480–4.192; p = 0.001) were risk factors for PEC (Table 2).

Table 2

Univariate and multivariate logistic regressions risk factors for PEC.

n/N	Univariate analysis		Multivariate analysis
	OR (95% CI)	p value	OR (95% CI)	p value
WBC	1.155(1.104–1.207)	0.000	1.153(1.036–1.282)	0.009
Neutrophil ratio	1.030(1.014–1.047)	0.001	0.981(0.961–1.002)	0.081
AST levels	1.001(1.001–1.002)	0.001	1.000(0.998–1.003)	0.731
Serum amylase levels	1.001(1.001–1.002)	0.000	1.001(1.000–1.001)	0.009
Direct bilirubin serum levels	1.005(1.002–1.007)	0.001	1.010(0.996–1.024)	0.168
Total bilirubin serum levels	1.003(1.001–1.004)	0.003	0.996(0.986–1.005)	0.382
Blood sugar levels	1.102(1.027–1.184)	0.007	1.013(0.914–1.122)	0.808
Gallbladder stones	11.582(2.829–47.421)	0.001	10.191(2.275–45.649)	0.002
Gallbladder opacification	5.839(3.141–10.854)	0.001	9.688(2.833–33.125)	0.000
Gallbladder thickness	2.347(1.654–3.330)	0.001	0.549(0.285–1.058)	0.073
ERBD	2.847(1.786–4.539)	0.000	2.055(1.146–3.685)	0.016
Mechanical lithotripsy	3.172(1.957–5.142)	0.001	2.294(1.293–4.072)	0.005
EPBD	4.245(2.709–6.867)	0.000	3.634(2.186–6.041)	0.000
Residual CBD stones after ERCP	3.352(2.163–5.194)	0.000	2.491(1.480–4.192)	0.001
The increase in WBC	1.132(1.082–1.183)	0.001	0.999(0.910–1.096)	0.984
The change in AST levels	1.001(1.001–1.002)	0.001	1.000(0.996–1.003)	0.839
The change in ALT levels	1.002(1.001–1.003)	0.002	1.000(0.998–1.002)	0.793

NOTE. WBC, White blood count, AST, aspartate aminotransferase, ALT,alanine aminotransferase,CBD, common bile duct ERCP,endoscopic retrograde cholangiopancreatography, EPBD, endoscopic papillary balloon dilatation, ERBD, endoscopic retrograde biliary drainage.

After determining the independent risk factors, a predictive model was made. In the ROC curve, the AUC was 0.864 (95% CI 0.826–0.903) and the sensitivity and specificity were 0.811 and 0.791, respectively (Figure 3). The calibration by the Hosmer-Lemeshow test showed χ2 was 11.031, and the P-value was 0.200.

Validation of the RF model in predicting PEC

A total of 117 patients (11 PEC, 9.40%) from January 2020 to May 2021 according to the same criteria were enrolled to further validate the built model. The RF model was validated based on the six top-ranking risk factors and got a comparable identification performance by the ROC curve(Supplementary Fig. 1) the sensitivity, specificity, accuracy, and AUC as 0.909, 0.757, 0.769, and 0.889, respectively. The decision curve (Supplementary Fig. 2) and clinical impact curve (Supplementary Fig. 3) validated the model with good accuracy for prediction. The external validation model showed that the RF model is in line with the clinical setting.

Web server of the model

In order to facilitate the application of the model, we established an online user-friendly platform called China Prediction Platform of Digital Disease (CPPDD), including the application web server of RF model with 6 features at http://101.35.163.113/PEC/ for the PEC prediction. Users could predict PEC by submitting the order by 6 features into the corresponding text boxes on the web page (Figure 6), the units of the value of WBC and amylase should be consistent with the interface requirements. After calculating the outputs of the sample, the resulting page will display whether the sample is distinguished with PEC or not.

Figure 6

The public internet calculator for PEC discrimination by 6 features.

The application web server of RF model with 6 features available at http://101.35.163.113/PEC/ for the PEC prediction. Users could predict PEC by submitting 6 features into the text boxes or submitting multiple samples by uploading documents.There is a sample file for reference, but the units of the value of WBC and amylase should be consistent with the interface requirements. After calculating the outputs by the RF model, the result probability will show whether the sample is distinguished with PEC or not.

Discussion

PEC is a severe but often missed complication than PEP (post ERCP pancreatitis) after ERCP therapy.1, 2, 3, 4 Freemen et al. firstly showed that the incidence of cholecystitis was 0.5% (11/2347) in 16 days after ERCP in Europe. In 2018, Cao et al. reported that the incidence of acute PEC within 2 weeks was 1.35% (36/2672). In 2020, Ting et al. reported that the overall PEC incidence within 2 weeks was 0.96% (13/1345) in Taiwan. In our study, the incidence of PEC (8.06%,90/1117) was much higher than the previous three reported studies. This is because we enrolled CBD stones patients with gallbladders in-situ instead of all ERCP patients, which is the target population of PEC. Furthermore, this study also extended observation time up to 3 months after the ERCP procedure.

In some cases, PEC can be severe and potentially fatal. The risks factors and mechanism of PEC are complex, and their relationship with co-existing cholangitis and gallbladder status is unclear. Therefore, PEC is difficult to predict even for experienced endoscopists. A few independent risk factors for PEC, such as acute pancreatitis, chronic cholecystitis, the placement of biliary stents, intraoperative cholangiography, increased preoperative WBC, and the anatomical structure of the biliary tract., However, there are no systematic methods to integrate these risk factors to accurately predict PEC. Only a logistic ROC curve by Cao et al. had an AUC of 0.85 (95% CI: 0.80–0.91), with sensitivity and specificity of 0.823 and 0.733. In the present study, we utilize 54 clinical parameters to create the RF model for analysis, thus, proving our model a much better prediction for PEC.

With the development of machine learning, the RF model has provided a better means to create applicable medical prediction models. Previous studies by Wu et al., showed that RF models could improve the diagnosis of gastric cancer and pulmonary tuberculosis. In this study, the RF model identified WBC count, EPBD, the increase of WBC, residual CBD stones after ERCP, serum amylase levels, and mechanical lithotripsy as the six top-ranking risks factors of PEC. The model has a sensitivity, specificity, accuracy, and AUC of 0.822, 0.853, 0.855, and 0.890, respectively, suggesting that the RF method has a higher predictive ability. In addition, all data were also evaluated using logistic regression analysis, the ROC curve had a sensitivity, specificity, and AUC of 0.811 0.791, and 0.864. An additional cohort of patients’ data was used to validate the accuracy of the RF and logistic regression models. The RF model had a better ROC curve and clinical decision curve. Moreover, we compared the calibration with other ERCP models by the AUC index. The result demonstrated that the AUC of 0.890 of the RF model was higher than other models, indicating that the RF model is more generalizable and can be applied in the clinical setting to predict PEC.

Cao et al. reported that WBC count and acute preoperative pancreatitis as risk factors for PEC, and serum amylase levels are a sign of acute preoperative pancreatitis. Ting et al. reported cyst duct stones are also a potential risk factor. Our result is consistent with the report of previous studies, suggesting that WBC count, serum amylase levels, and residual CBD stones are common risk factors for PEC.,

Regarding other risk factors, if the WBC increased after ERCP in patients, even within normal range, biliary inflammation likely occurred. The incidence of PEC will be much higher in these patients. EPBD is used to dilate the biliary sphincter to facilitate stone extraction. Compared with EST, EPBD after EST may increase ascending biliary reflux, resulting in PEC. Mechanical lithotripsy for CBD stone fragmentation, resulting in residual biliary stone and sludge increasing the risk of bile stasis and infections.

The strength of this study is that different models were compared to predict PEC. According to the ROC curve of the multivariate logistic model, the AUC was 0.864 (95% CI, 0.826–0.903), and the sensitivity and specificity were 0.811 and 0.791. The RF model has the sensitivity, specificity, and AUC of 0.822.0.853 and 0.890, which demonstrated that RF has the excellent predictive ability. External validation performance and comparison with other ERCP models also proved that the RF model has better predictive values for PEC. For further application, we established an online user-friendly prediction platform for worldwide endoscopists and patients.

Our study had some limitations. In the selection of patients, only CBD stone patients with cholangitis were included in the study instead of all ERCP patients. In addition, this was a single-center retrospective study with an inherent bias in data collection. For future studies on PEC, multicenter validation and large-scale prospective studies should be conducted.

In conclusion, the RF model which identified WBC count, EPBD, increase in WBC, residual CBD stones after ERCP, serum amylase levels, and mechanical lithotripsy as potential risk factors that can accurately predict PEC. Our study suggests that the proposed RF model based on the top six PEC risk factors could be a promising tool to predict the occurrence of PEC. Further studies should corroborate further our fundings.

Funding

This work was supported by National Natural Science Foundation of China (8187103130; 32160255); Gansu Competitive Foundation Projects for Technology Development and Innovation (1602FKDA001); Gansu Province Science and Technology Planning Project (20YF8WA085); Science and Technology Planning Project of Chengguan District in Lanzhou (2020JSCX0043). The Foundation of The First Hospital of Lanzhou University (ldyyyn-2018-16).

Contributors

The list of full authors in this study is as follows: Xu Zhang, Ping Yue, Jinduo Zhang, Man Yang, Jinhua Chen, Bowen Zhang, Wei Luo, Mingyuan Wang, Zijian Da, Yanyan Lin, Wence Zhou, Lei Zhang, Kexiang Zhu, Yu Ren, Liping Yang, Shuyan Li, Jinqiu Yuan, Wenbo Meng, Joseph W. Leung, Xun Li. XZ, PY, JDZ, and MY make the same contribution to this work. Correspondence Auther: SYL, JQY, and WB M make the same contribution to this work.XZ, PY, JDZ, MY, JHC, SYL, JQY, JWL, and WBM: protocol development, drafting of this manuscript, critical revision of the manuscript for significant intellectual content. XZ, PY, JDZ, MY, WL, MYW, ZJD, YYL, YR, and LPY: conducted clinical trials, patient enrollment, and acquired data. JDZ, KXZ, LZ, PY, WCZ, XL, JWL, and WBM: providing personnel, environmental support, and tools and instruments that are vital for the project. XZ, BWZ, JQY, and WBM: taking responsibility for statistical analysis, logical interpretation, and presentation of the results. XZ, PY, ZJD, MYW, and SYL: taking responsible for pathology and figures.XZ, PY, JDZ, MY, JHC, SYL, JQY, JWL, and WBM: Review the article before submission not only for spelling and grammar but also for its intellectual content. PY, SYL, KXZ, LZ, WBM, JWL, WCZ, and XL: constructing an idea or hypothesis for the manuscript, providing critical revision.XZ, PY, JDZ, MY, and SYL accessed and were responsible for the raw data and the models in the study, and XZ, SYL, JQY, and WBM decided to submit the manuscript for publication.

Data sharing statement

Except for the patients' privacy, the statistical analysis plan, RF model, online prediction platform, RF model files, and de-identified results of these analyses are available for scientific researchers upon reasonable request through the first or corresponding author.

Declaration of interests

None.